Incus-Compose: How to expose a container to the LAN and/or get it's own IP? (Hosting pihole/DNS server)

Incus
, ,
1

This thread is basically the equivalent to my prior thread (so here we are again), which tackles the same.

Stack/Background

Using IncusOS (so no shell/file access on the server!) with (obviously) Incus. I thought let’s try out the cool new Incus-Compose.

Off-topic: Finding out how to connect incus-compose to a remote server

And after a little fiddling around finding out you need to set INCUS_REMOTE (or using --remote) as an env variable to the IncusOS server remote name, it also works.
Note this information is hidden in the --help page, I was not able to find it in the docs:

--remote string       remote to connect to [$INCUS_REMOTE]

Although it elaborates on this and seems to suggest using a remote instance is anyway a good idea, it does not explain this crucial fact for getting started in that doc.
So maybe that should be added?

Use case

I want to host pihole in the network, which has a good Docker(-Compose) documentation, so that sounds good.

The problem is: For pihole, obviously, port 53 needs to be allocated. Because AFAIK DNS servers always need to be served from port 53.
You cannot configure clients to “just” use a different port.

Problem

So when you use port 53 in your compose file, you get this error:

19:59 ERR Starting resources project=your-dir error="in an operation: starting an instance: instance(pihole): in an operation: operation c2f49b82-300e-43fb-aa05-70c131a58ae2: Error occurred when starting proxy device: Error: listen udp 0.0.0.0:53: bind: address already in use"

In that long form it’s a little obscure, but if you read the end it is obvious port 53 is already used.

So the Pihole docs kinda have docs about this problem and I assume it’s the same, it’s may be caused systemd-resolved on the host.

Now again, whatever may cause this (though I would be interested in what makes this happen, after all, I hope no public-facing DNS server runs on IncusOS by default?), I cannot just disable it on the host (nor do want it) as they recommend.

Also, of course, we are not even at the part where the network config may need adjustments, because the DNS server to be used by other containers/services is then hosted on the same host (physically).

What I am looking for

Thus I need to either:

  • expose the container as it’s own “host” to the network as the docs explain. However, I am not sure how to apply that to Incus-Compose. (see below)
  • or do some clever routing or so (if possible?), so that the DNS port, is forwarded correctly to pihole? Or disable the “thing” (service?) that binds this on IncusOS, apparently.

Note:

  • in any case, I would have nothing against having the container have it’s own IP in my LAN to make it easier addressable and prevent such port conflicts in general, but AFAIK, this always makes the instance (container) totally exposed to the network, does not it? Aka, it always exposes all ports then and I cannot configure what ports should be forwarded from the outer network (LAN).
  • IMHO, all other ports for HTTPS, HTTP etc. do not matter and could be changed.

What I did

It’s basically took the “official” docker-compose and ended up making only small changes:

  • dropping all suggested cap_add as I just want to use it as a DNS server, not DHCP.
  • Instead of a volume with a fixed host file path (which caused a funny failed to add a bind-mount for service pihole: not on the same host as I obviously – note I am using a “remote” IncusOS server

My naive try was to combine some custom network options in incus-compose with the “usual” config for an external IP in docker-compose:

# More info at https://github.com/pi-hole/docker-pi-hole/ and https://docs.pi-hole.net/
services:
  pihole:
    container_name: pihole
    image: docker.io/pihole/pihole:latest
    networks:
      lan:
        ipv4_address: 192.168.178.53
    ports:
      # DNS Ports
      - "153:53/tcp"
      - "53:53/udp"
      # Default HTTP Port
      - "80:80/tcp"
      # Default HTTPs Port. FTL will generate a self-signed certificate
      - "443:443/tcp"
      # Uncomment the line below if you are using Pi-hole as your DHCP server
      #- "67:67/udp"
      # Uncomment the line below if you are using Pi-hole as your NTP server
      #- "123:123/udp"
    environment:
      # Set the appropriate timezone for your location (https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), e.g:
      TZ: 'Europe/Berlin'
      # Set a password to access the web interface. Not setting one will result in a random password being assigned
      #FTLCONF_webserver_api_password: 'correct horse battery staple'
      # If using Docker's default `bridge` network setting the dns listening mode should be set to 'ALL'
      FTLCONF_dns_listeningMode: 'ALL'
    # Volumes store your data between container upgrades
    volumes:
      # For persisting Pi-hole's databases and common configuration file
      - 'etc-pihole:/etc/pihole'
      # Uncomment the below if you have custom dnsmasq config files that you want to persist. Not needed for most starting fresh with Pi-hole v6. If you're upgrading from v5 you and have used this directory before, you should keep it enabled for the first v6 container start to allow for a complete migration. It can be removed afterwards. Needs environment variable FTLCONF_misc_etc_dnsmasq_d: 'true'
      #- './etc-dnsmasq.d:/etc/dnsmasq.d'
    # cap_add:
      # See https://docs.pi-hole.net/docker/#note-on-capabilities
      # Required if you are using Pi-hole as your DHCP server, else not needed
#      - NET_ADMIN
      # Required if you are using Pi-hole as your NTP client to be able to set the host's system time
#      - SYS_TIME
      # Optional, if Pi-hole should get some more processing time
#      - SYS_NICE
    restart: unless-stopped
    x-incus:
      limits.memory: 512MiB

networks:
  lan:
    x-incus:
      ipv4.address: 192.168.178.1/25
    external: true

volumes:
  etc-pihole:

This ends up with some weird error, I do:

21:48 ERR Ensuring resources project=compose-pihole error="in an operation: instance(pihole): operation 013e0f76-7040-4885-914e-6b60959080cd: Failed creating instance record: Failed initializing instance: Invalid devices: Device validation failed for \"eth0\": Cannot use manually specified ipv4.address when using unmanaged parent bridge"

And when use a pure docker config for the network:

networks:
  lan:
    driver: bridge
    ipam:
      config:
        - subnet: 192.168.178.0/25
          gateway: 192.168.178.1

I just get this error:

21:44 ERR Getting project resources in reCreate project=compose-pihole error="service \"pihole\": cannot assign a static IP on network \"lan\" with no address - the gateway isn't known until the network is created; set an explicit CIDR on the network instead"

Also, I do not see that this leads me to the aim of exposing the

That said, I guess, I could of course follow the “usual” guide on how to configure an instance to be exposed after creation and just do not configure it in the compose file. But this kinda misses the point, does not it?
Also, it would, I guess, mean that if I re-create (update or so) the container, it will discard that adjustments and I would need to have to redo them.

2

About INCUS_REMOTE:

You might use Windows · incus-compose docs but I’ll make sure to document it and --remote better.

Using this compose.incus.yaml:

services:
  pihole:
    ports: !reset []
    networks:
      lan:
        ipv4_address: 192.168.178.53/24

networks:
  lan:
    name: lan
    external: true

Allongside the upstream pihole config should allow you to access pihole on 192.168.178.53 with all the ports. The trick here is ports: !reset [] this instructs ic to not expose any ports on the host.

It might work without the name: lan, please report back if so.

I’ll make sure that this works as well in the future.

Thanks for using/trying incus-compose!

3

docs extended and ipam.config will work with future releases.

4

Okay I would have never expected it to be documented there. After all, as a Linux user, I strictly skipped every doc about Windows. :upside_down_face:
But thanks yes!

Using:

    ports: !reset []
    networks:
      lan:
        ipv4_address: 192.168.178.53/24
# ...

networks:
  lan:
    name: lan
    external: true

it, however, still shows:

18:57 ERR Getting project resources in reCreate project=compose-pihole error="service \"pihole\": cannot assign a static IP on network \"lan\" with no address - the gateway isn't known until the network is created; set an explicit CIDR on the network instead"

“name” or not, does not matter.

5

I’m missed that you need to set the gateway:

services:
  pihole:
    ports: !reset []
    networks:
      lan:
        ipv4_address: 192.168.178.53/24
        x-incus:
          ipv4.gateway: 192.168.178.1

networks:
  lan:
    name: lan
    external: true
6

The ports: !reset [] thing is only needed when overloading “compose.yaml” with “compose.incus.yaml”, if you edit the upstreams docker-compose.yaml you don’t need that.

I recommend you to not touch the file and put incus-compose specific settings in that separate file that makes comparing it later easier.