How to create a Proxy for an OCI container bridged to a LAN address?

If I create a docker OCI container and I bridge it to my LAN whereby it gets a DHCP address from my router, I have no way to remap a port like I can in docker run or docker compose.

The proxy command:

incus config device add container1 hostport80 proxy connect="tcp:127.0.0.1:3000" listen="tcp:0.0.0.0:80"

works just fine if the container is running on the incus NAT network and remapping a port to the incus host as above.

However, if I did something like:

incus network create LAN --type=macvlan parent=bridge0
incus launch docker:appx:latest container1 -c environment.TZ="America/Chicago" -c boot.autostart=true --network LAN

The above container gets an address on the LAN, but the proxy command cannot remap port 3000 to port 80. Is there a way to make this happen with OCI containers?

I have used caddy as a reverse proxy to solve this problem.

Caddy is on the LAN and listens on your port and then routes to anything else it can connect to.

I can’t think of a way for Incus to do it directly.

Thanks, but I am talking about an app on a LAN and not one that is being offered out to the WAN. This is a local address only and no domain.

From a high level view this is by design and where Incus is different from Docker.

Docker is installed on the system and used a local virtual network which can be only accessed if you perform a port mapping between Docker and your host. The equivalent for Incus would be to use the incus bridge and add a proxy like described above.

Configure your instance device as ipvlan, macvlan or routed will give them direct access to your local network. As such there is no need for a proxy to connect to them as you access them directly. In other words the device is now public access able without any restrictions.

You can of course write your own nft / iptables rules to redirect the port but this is outside of Incus as far as I know. Think best practice it to just use the provided Incus bridge and define either proxies or use a reverse proxy. It is much more secure than assigning to the local network directly.
In some cases you might be able to change the port of the application by using an environment variable like you do with your timezone for example but that depends of course on the OCI image provider.

There many more options how you can configure things in Incus compared to Docker or any kind of other virtualisation. You just need to get your head around it…

1 Like

I understand your use case and caddy can do that. If I can find some time on the weekend I can provide more details.

Here is a quick example. If you launch caddy on your LAN bridge this should work. It works for me.

Of course nm-bridge a bridge I created on my host that bridges to the local LAN.

incus launch docker:caddy caddy --network nm-bridge
incus file push caddy-Caddyfile caddy/etc/caddy/Caddyfile
incus restart caddy

Where caddy-Caddyfile looks like this:

# The Caddyfile is an easy way to configure your Caddy web server.
#
# Unless the file starts with a global options block, the first
# uncommented line is always the address of your site.
#
# To use your own domain name (with automatic HTTPS), first make
# sure your domain's A/AAAA DNS records are properly pointed to
# this machine's public IP, then replace ":80" below with your
# domain name.

:80 {
	# Set this path to your site's directory.
	#root * /usr/share/caddy

	# Enable the static file server.
	#file_server

	# Another common task is to set up a reverse proxy:
	reverse_proxy caddy-2:3000

	# Or serve a PHP site through php-fpm:
	# php_fastcgi localhost:9000
}

# Refer to the Caddy docs for more information:
# https://caddyserver.com/docs/caddyfile

If you want your port 3000 app to be on the Incus managed bridge then you need to add some routing to your host.

Useful information. Let me be more concise:

  1. My bridge “nm-bridge” is on the main LAN.
  2. Containers that are created using the bridge get a DHCP address on the LAN.
  3. Suppose for example, container “Test” gets the address 192.168.1.20.
  4. The incus server would be a completely different address…eg. 192.168.1.50.
  5. Assume that the docker OCI container app runs at port 3000.
  6. Assume that the goal is to offer the app at 192.168.1.20:80.

Unless I am missing something, it does not look like your example is capable of doing that. Also, this ends up creating a caddy container for every OCI container app I have unless I misunderstand.

Yeah, you are right. Unless you setup your DNS to point different domains or subdomains to the same IP you will end up with more than one caddy container.

I run my own DNS so I can use a single caddy instance. I try not to think about IP addresses.

If you create your own containers you can run caddy and your app in the same container. Then they would have the same IP. But if you were doing that then why not just use port 80 in the first place.

These app containers are a pain. It reminds me why I was so excited to start using Incus and could stop messing around with Docker. The use case that Incus’s OCI support covers is great. Anything more then I would use Podman. It sounds like it would be a better fit.