Incus-compose - the missing docker-compose for incus

1.0.0-beta11

Moved to 1.0.0 from 0.0.1 as 1.0.0 will be the final version.

Features

  • Healthy Dependency support

    • up now wires service_healthy dependencies into instance start ordering/waiting.
    • New up --dependency-timeout flag controls how long to wait for healthy dependencies.
  • Healthd expanded and rewritten

    • ic-healthd now also runs when services use restart policies or are depended on via service_healthy, not only explicit healthcheck.
    • Supports healthcheck timing fields:
      • start_period
      • start_interval
      • interval
      • timeout
      • retries
    • Health state is standardized via user.healthcheck.status.
    • Better checker behavior, logging, reload handling, and start-period handling.
  • Compose shm_size support

    • shm_size now maps to a /dev/shm tmpfs device.
  • Compose container_name support

    • If container_name is set, it is used as the Incus instance name.
    • For scaled services, names become container_name-1, container_name-2, etc.
  • Improved CLI progress handling

    • Healthd commands now participate in progress reporting.
    • Progress output moved to stdout and is less likely to corrupt logs.
  • Better DNS/IP handling

    • DNS watcher was rewritten for concurrent per-action updates.
    • Instance IP handling was refactored to support multiple interfaces/IP sets.
  • Examples and dev workflow

    • Added examples/many-dependencies.
    • Added examples/wikijs.
    • Moved test/fixtures/immich to examples/immich.
    • Added timestamped test logs under test/logs/.
    • Added just test-slow.

Fixes

  • exec now targets the requested service

    • Fixes a bug where exec could choose the wrong instance from the stack.
  • list / ps output is sorted

    • Makes output deterministic. Fixes issue #46.
  • Progress no longer overwrites logs

    • Fixes issue #43.
    • Also fixes several writer/stdout/stderr inconsistencies.
  • Healthd stability fixes

    • Graceful healthd deletion/reload improvements.
    • Checker optimization and bug fixes.
    • Normal checker now starts correctly after the start-period checker succeeds.
  • Instance lifecycle fixes

    • Fixes around already-running instances.
    • Fixes recreate/resource lifecycle behavior.
    • Timeout handling fixed: Incus timeout 0 means “do not wait”, so internal default now maps to -1 where needed.
  • Storage volume fixes

    • Better shifted-volume validation.
    • Validation delayed from Ensure() to Start() where runtime UID/GID data is available.
  • Image handling fixes

    • Image properties are copied correctly instead of sharing/mutating state.
  • Network deletion fixes

    • Several fixes around deleting project networks, external networks, and managed/dangling networks.
  • Error handling cleanup

    • Uses errors.Is(err, ErrNotFound) in more places.
    • Better debug logging and error wrapping.

Backward-incompatible / behavior changes

  • Direct Incus URL env vars were removed from documented/runtime connection flow

    • Removed from docs:
      • INCUS_COMPOSE_URL
      • INCUS_COMPOSE_CERT
      • INCUS_COMPOSE_KEY
    • Use Incus CLI remotes instead:
      • --remote
      • INCUS_REMOTE
      • Incus default remote
  • Timeout flags changed from integer seconds to Go duration values internally

    • Affected commands include up, down, start, stop, restart, healthd paths.
    • Safer CLI usage is now explicit durations like --timeout 10s.
  • Healthd sidecar name changed

    • Current docs/code use {project}-ic-healthd.
    • Scripts expecting plain ic-healthd need updating.

Overall: 1.0.0-beta11 is mostly a health/dependency/lifecycle hardening release.

1 Like

1.0.0-beta12

What’s new since beta11:

  • --image-cache global flag (env INCUS_COMPOSE_IMAGE_CACHE) — point the image cache at a different Incus project; default is default
  • --rmi / --images on down — remove project images on teardown, matching docker compose behaviour
  • --with-deps scoping fix — up/down follow depends_on automatically; all other commands require explicit --with-deps
  • Healthd is now skipped when no services require it
  • Extra storage volume config support via x-incus-compose
  • Build image name corrected to localhost/<service>

1.0.0-beta13

incus-compose is now part of the lxc organization on Github - thank you @stgraber.

It contains an important fix to not assume the availability of incusbr0.

The main branch of the project is now in a feature freeze, please test it through and through.
If there have been enough testers it will be released around the beginning of July 2026.

I hope for additional committers and maybe co-maintainers.

1 Like

@neitsab @pyrodogg @kgoetz @Sagi @edorgeville @bburky

Please test the 1.0.0-beta13 as said in the post before it’s in a feature freeze and the 1.0.0 should come soon.

Also let me know if you want to be credited as testers.

A background post about hooks in this:

I’d like to discuss this change from beta9.

Compose and local host file access are a convenient tools for quickly iterating on config for services. It doesn’t involve having to manually push/pull files to the container (nor manually setupvolume file mount). Just edit files at their original host location and restart the container for the app to pick up the latest config.

# App is Glance https://github.com/glanceapp/glance#installation
# both ./assets/ and ./config/ are provided as bound directories
# multiple config files exist in the config directory
$ vim config/home.yml #one of multiple files in config/ (or save from other terminal)
$ incus-compose restart

This changed in beta9 where directories are only mapped one-way, one-time on image creation. While this seeding does help to get the host files into a Volume, it breaks any notion of ongoing connection to the original files/dirs. You need to completely take down the volume(s) for them to be reseeded.

Taking down the whole project has the effect of deleting all volumes which may not be desired.

I tried configuring ./config/glance.yml:/app/config/glance.yml as a a single file but the app refuses to start. I’m not sure if the file is provided in time.

So it seems that bi-directional directory bind mounts aren’t supported, this is just an on-ramp to forced Volume use.

I could also see this being an issue if an expected directory to be bound was rather large, like a media library, and the expected behavior is that no bytes are duplicated, the container simply has access to the host directory. I glossed over this when I updated immich and now see that the information it was last seeded with is decoupled from it’s current state.

Hey Skyler,

thanks for following the topic and all the testing you did, it’s very helpfull!

Now that you name it, you’r right, this hidden feature will be a problem in many use cases.

This should copy the file into the container, the problem is that the copy is sadly delayed as in it will be copied after the start of the app, the only way I found to have this working with the right timing is to create an image and start the image…

Yes.

Not sure I understand that one, do you mean sometimes you have seen bind-mounts sometimes volumes? That’s ugly. :confused:


We could opt-in that feature instead by using the long syntax of volumes and adding x-incus-compose.copy = "true" what do you think?

For the immich example I mean that following the immich install instructions simply, it leads one to using bind-mounts out of the box. The image library exists outside the container. Incus-compose takes that and seeds a volume. Thereafter, immich-server-1 is running on the volume.

The user might expect things are still being saved to the host directory, which is not happening. This could confuse back-up efforts if it’s not noticed what should be backed up. In the case of a brand new install it’s fairly trivial to see that nothing has been modified on the host. It’s quite different in-situ when the library has prior content.

1 Like

I wanna be compatible with docker-compose wherever possible, We have to make copy opt-in.

To have bind-mounts working with “https” remotes we need a hacky workaround…

1.) Add a field useUnix to Client.InstanceConfig.
2.) Make all uses of Client.incus use Client.Connection(unix bool) (*incusClient.ProtocolIncus, error) for instances only.
3.) When a unix client is requested return the cached, if no cached see if the https endpoint matches one of the connection info addresses, i not error.

The hacky workaround above allows us both https:// for healthd and unix:// for instances with bind mounts.

I’m no fan of it at all, it adds a lot complexity, still I think its required.


service.{name}.volumes.{name}.volume.nocopy.

It seems like docker/docker-compose copies data from the container to the volume when creating a new volume, that’s complete new to me.

That whould have been a blocker in a release, thank you for reporting!


I don’t know why I didn’t recognize that earlier: incus is smart with bind mounts:

local incus:

INCUS_REMOTE=local-https incus init docker.io:library/busybox:glibc tmp
# shifted directory works
INCUS_REMOTE=local-https incus config device add tmp bind0 disk source=$PWD path=data/ shift=true
# Single file readonly bind-mount works
INCUS_REMOTE=local-https incus config device add tmp bind1 disk source=/etc/timezone path=etc/timezone readonly=true shift=true
Device bind0 added to tmp
INCUS_REMOTE=local-https incus config set tmp oci.entrypoint="/bin/sleep 365d"
INCUS_REMOTE=local-https incus start tmp
INCUS_REMOTE=local-https incus exec tmp /bin/sh
~ # ls -l /data/
total 288
-rw-r--r--    1 1000     1000          8778 Dec 20 23:14 CODE_OF_CONDUCT.md
-rw-r--r--    1 1000     1000          7323 Jun 17 12:52 CONTRIBUTING.md
-rw-r--r--    1 1000     1000         11358 Dec 20 23:15 LICENSE
-rw-r--r--    1 1000     1000          3363 Jun 19 13:50 README.md
drwxr-xr-x    1 1000     1000            46 Jun 17 22:51 bin
drwxr-xr-x    1 1000     1000          1034 Jun 20 20:07 client
drwxr-xr-x    1 1000     1000            46 Apr 24 12:53 cmd
-rw-r--r--    1 1000     1000        215135 Jun 18 09:19 coverage.out
drwxr-xr-x    1 1000     1000          1306 Jun 17 23:19 dist
drwxr-xr-x    1 1000     1000           276 Jun 17 12:52 docs
drwxr-xr-x    1 1000     1000           102 Jun 17 12:52 examples
-rw-r--r--    1 1000     1000          3350 Jun 17 23:12 go.mod
-rw-r--r--    1 1000     1000         22395 Jun 17 12:51 go.sum
-rw-r--r--    1 1000     1000          8128 Jun 18 08:10 justfile
drwxr-xr-x    1 1000     1000           144 Jun 17 12:52 project
drwxr-xr-x    1 1000     1000            70 Jun 17 12:52 scripts
-rw-r--r--    1 1000     1000           141 Dec 20 23:15 staticcheck.conf
drwxr-xr-x    1 1000     1000            70 Jun 17 12:51 test
drwxr-xr-x    1 1000     1000            36 Jun 17 06:43 work
~ # ls -l /etc/timezone
-rw-r--r--    1 root     root            14 Jan  5 03:23 /etc/timezone
~ #

remote incus:

INCUS_REMOTE=pve01 incus init docker.io:library/alpine:latest tmp
INCUS_REMOTE=pve01 incus config device add tmp bind0 disk source=$PWD path=data/ shift=true
Error: Failed add validation for device "bind0": Missing source path "<my-home>/projects/incus-compose" for disk "bind0"

Will make the feature you know since beta9 service.{name}.volumes.{name}.x-incus-compose.seed="true" all optional and propagate the error above, easy fix :slight_smile:

What do you think is seeding something people will ever need?

1 Like

Thanks!

I’m not sure personally how much the seeding will be used. If it were inflating things from an image maybe useful, but the host file system is already there already.

1.0.0-beta14

Highlights

Bind-mounts are no longer seeded by default — bind-mount volumes now default to non-seeded (simple disk device pass-through). To get the previous behavior of copying files into the instance, set x-incus-compose.seed: true on the volume.

Features

  • Network project/profile support — new x-incus-compose.network.project and x-incus-compose.network.profile options to control which Incus project and profile healthd uses.

Fixes

  • Client connection stability — fixed several data races in the client (reused ProtocolIncus, noColor context var, random string generation)
  • Network profile fallback — use devices.eth0.parent when devices.eth0.network isn’t available in a profile
  • FindHealthd improved — more robust healthd discovery
  • Error handling — switched to errors.As() for proper error unwrapping

Internal / Testing

  • Removed all remaining testify/suite usage
  • Refactored serviceToInstance() into smaller helpers
  • Split project/project.go into smaller files
  • Removed dangling test fixtures
  • Snapshot tests updated for new bind-mount behavior
2 Likes

1.0.0-beta15

New features

  • Self-update commandincus-compose self-update checks GitHub releases and updates the binary in place. Only available for release builds and when the binary directory is writable.
  • Environment variables for all CLI flags – Every global flag can now be set via INCUS_COMPOSE_* environment variables (e.g. INCUS_COMPOSE_FILE, INCUS_COMPOSE_PROJECT_NAME, INCUS_COMPOSE_DEBUG). See the environment variables docs for the full list.
  • Configurable worker count--workers / INCUS_COMPOSE_WORKERS controls the number of concurrent workers (default: 10).

Bug fixes

  • Fixed .incus.yaml overlay loadingdocker-compose.incus.yaml and compose.incus.yaml overlay files were not being loaded correctly (#6).
  • Progress rendering improvements – Enhanced progress display and error rendering (#7).
  • Healthd retry calculation – Fixed retries = start_period / start_interval in healthd start logic.
  • Client cleanups – Small fixes in the client package.

Documentation

  • Improved healthd documentation.

I have been following this project with great interest. Hoping to move some docker-compose based containers to incus. Ran into some bumps that I am trying to figure out.
The system has a pre-existing unmanaged host bridge (br0) which is part of the incus default profile. This bridge has an IP address assigned to it.

If a compose file does NOT have a restart policy defined, incus-compose up starts the container successfully.
If a restart policy is defined [always|unless-stopped], ‘incus-compose up’ fails with ERR Starting healthd error="result is nil while parsing ip \"\""

incus-compose up --no-healthd starts the container successfully.
incus-compose up --healthd-network br0 fails with ERR Starting healthd error="result is nil while parsing ip \"\""

I created an incus managed bridge incus network create incusbr0 --type bridge
'incus-compose up --healthd-network incusbr0 starts successfully along with the sidecar ic-healthd container.

Does the healthd sidecar require an incus managed bridge?

Secondly, I wanted to define this in the compose file instead of a command line option. Added:

x-incus-compose:
  healthd-network: incusbr0

Is this correct? It seems to be ignored, still requiring --healthd-network incusbr0 to launch.

This is great to hear and helps me a lot.

This is broken, will fix it soon.

No. healthd needs a bridge to bind its endpoint to and get an IP from, as well as an IP to connect to incus — but support for other bridges than incusbr0 is just broken at the moment, will fix.

This is also broken, will fix alongside the above.

Thank you again,
René

1 Like

Wrote an issue for that: healthd sidecar networking and finding the incus ip is broken · Issue #18 · lxc/incus-compose · GitHub

Is there planned support for bind mounted network shares?

Similar to what currently works with incus native and OCI containers? incus config device add ... disk source=<bind mounted network share> path=...

Thank you.

Bind mounts are already supported, is there anything special you need for network shares?

I read Bind mounts (local connections only) in compose-compatibility.md and took it to mean that only host local bind mounts are supported.

But I gave it a try anyway and tried to bring up a container.

docker-compose.yml

 services:
  kiwix:
    image: ghcr.io/kiwix/kiwix-tools:latest
    container_name: kiwix
    ports:
      - 8980:8980
    volumes:
      - ./data:/data
      - /media/zim:/media/zim
    command: "kiwix-serve --library --port 8980 -L 6 --monitorLibrary /data/library.xml"
    restart: unless-stopped

where /media/zim is a CIFS/samba share

Got this error while bringing up the container:

$ incus-compose up --healthd-network incusbr0
15:14 ERR Starting resources project=kiwix error="in an operation: instance(kiwix): Failed to setup device mount \"bind-media-zim\": idmapping abilities are required but aren't supported on system"
ensure   image              ghcr.io/lxc/incus-compose/ic-healthd:~ [done]
ensure   storage-volume     ic-healthd                             [done]
ensure   instance           kiwix-ic-healthd                       [done]
ensure   image              ghcr.io/kiwix/kiwix-tools:latest       [done]
ensure   network            default                                [done]
ensure   instance           kiwix                                  [done]
start    instance           kiwix                                  [error: in an operation: Failed to setup device mount "bind-media-zim": idmapping abilities are required but aren't supported on system]

Also, the failure seems to leave the healthd sidecar orphaned.

$ incus list --project kiwix
+------------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+
|       NAME       |  STATE  |         IPV4          |                     IPV6                      |      TYPE       | SNAPSHOTS |
+------------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+
| kiwix            | STOPPED |                       |                                               | CONTAINER (APP) | 1         |
+------------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+
| kiwix-ic-healthd | RUNNING | 10.189.245.126 (eth0) | fd42:7c7c:5f7:a084:1266:6aff:feb0:dd02 (eth0) | CONTAINER (APP) | 1         |
+------------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+

$ incus-compose down
ensure   instance           kiwix                                  [done]
ensure   network            default                                [done]
delete   instance           kiwix                                  [done]
delete   network            default                                [done]

$ incus list --project kiwix
+------------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+
|       NAME       |  STATE  |         IPV4          |                     IPV6                      |      TYPE       | SNAPSHOTS |
+------------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+
| kiwix-ic-healthd | RUNNING | 10.189.245.126 (eth0) | fd42:7c7c:5f7:a084:1266:6aff:feb0:dd02 (eth0) | CONTAINER (APP) | 1         |
+------------------+---------+-----------------------+-----------------------------------------------+-----------------+-----------+


1.0.0-beta.16

To update to 1.0.0-beta.16 please use the script as I had to change to the format to beta.XX for a correct semver version.

The script:

curl -sSfL https://raw.githubusercontent.com/lxc/incus-compose/main/install.sh | sh -s -- -b ~/.local/bin

Breaking changes

  • The defunct x-incus-compose.network-profile is gone, see “healthd connection config” below.
  • compose now defaults to incus listen on all interfaces, if you don’t do so use INCUS_COMPOSE_HEALTHD_INCUS see the healthd doc.

New features

Health checking

  • list now has a separate HEALTH column instead of appending the health
    result to STATUS. Columns: KIND, NAME, INCUSNAME, IMAGE, STATUS, HEALTH,
    ADDRESSES.
  • Every instance reports a health value. Services without a healthcheck show
    “Unknown” rather than a blank.
  • up/start wait for the healthcheck to report healthy after starting an
    instance that defines a healthcheck (polled every 500ms, bounded by
    --timeout). This makes depends_on: service_healthy reliable.
  • Default --timeout raised from 10s to 1 minute.
  • ic-healthd now reports its own status (healthy on start, unhealthy on
    shutdown) and locates the daemon instance via a user.healthcheck.daemon
    marker instead of assuming a fixed location.

healthd connection config

issue 18

  • New --healthd-incus flag / INCUS_COMPOSE_HEALTHD_INCUS env var sets the
    API URL the sidecar connects to. Empty = auto-detect the IP from the bridge
    it is attached to.

  • New top-level extension x-incus-compose.healthd:

    x-incus-compose:
      healthd:
        incus: https://:8443
        network: default:default
    

    network is <project>:<network> or a plain bridge name; incus is the API
    URL. Both default to the project’s own network and the connection’s port.

  • This replaces the removed x-incus-compose.network-profile extension
    (breaking change).

up scale-down

  • up reconciles the service count in both directions, matching
    docker compose up. Instances above the desired count (deploy.replicas or
    --scale) are torn down, highest index first. A manual --scale N applies
    only to that invocation; the next plain up restores replicas. Fixes #12.
1 Like