Wait for net ready in incus oci containers

, , ,
1

Many of my containers assume that the network is ready when the application runs.

I’m running for example caddy:

services:
  caddy:
    image: docker.io/library/caddy:2.11.4-alpine
    volumes:
      - ./caddy:/etc/caddy:ro
      - ./sites:/var/www:ro
      - data:/data/caddy
    networks:
      default:
        x-incus:
          ipv4.address: 10.179.215.4/24
          ipv4.gateway: 10.179.215.1

networks:
  default:
    external: true
    x-incus-compose:
      network: incusbr0

volumes:
  data:

Then caddy logs on first attempt:

caddy-1 | 2026/07/05 19:55:32.015       ERROR   tls.obtain      will retry      {"error": "[docs.incus-compose.org] Obtain: [docs.incus-compose.org] creating new order: provisioning client: performing request: Get \"https://acme-v02.api.letsencrypt.org/directory\": dial tcp: lookup acme-v02.api.letsencrypt.org on [::1]:53: read udp [::1]:53161->[::1]:53: read: connection refused (ca=https://acme-v02.api.letsencrypt.org/directory)", "attempt": 1, "retrying_in": 60, "elapsed": 0.504413698, "max_duration": 2592000}

I can work around this by sleeping before running caddy:

services:
   caddy:
      command: /bin/sh -c "/bin/sleep 2; caddy run --config /etc/caddy/Caddyfile --adapter caddyfile"

But I realy don’t like having to change the command.

Also thought about writing a wrapper that executes the real command in the container after the net is ready, but that is another moving part I which I don’t want to maintain.

Any other solution or workaround?

2

File an Incus bug. There’s a decent chance we can make forknet wait up to 5s for initial connectivity or something (when DHCP is expected).

1 Like
3

Abusing this thread for another incus-compose related question…

In incus-compose/client/resource_instance.go at 8442bd326bf68cf79f0d197fe45e4ce4453c4cb1 · lxc/incus-compose · GitHub

I’m using (*client.ProtocolIncus).CreateInstanceFile to create directories and files.

An instance that gets created with files (for example secrets) often fails during testing, I had ic-healthd to use that and had a lot of troubles.

Troubles been:

  • Socket gone
  • PID was -1 ?
  • Start said “ok” but instance was still Stopped.

If you like I create you a minimal compose file and a test that you can fire x times with go test -count=X ?

I can’t think of another way to test / reproduce that and this has been bugging me a lot.

Do you have anything in mind that could cause these troubles?

4

Try using the modern SFTP API instead of the legacy REST file API.
Not to say we shouldn’t track down issue in the REST one, but it would be good to know if the modern one (that we use for incus file) is also affected.

1 Like
5

SFTP seems to work nice, thank you!

I was used to have errors when supplying the token to ic-healthd with a secret, no more!

6

Want me to come up with a test case that fails on networking with 7.2?

7

Not super useful given that we basically only run the testsuite in Github Actions and we don’t have network in this environment.

8

You don’t know how much troubles this saved me from, thank you a lot!