Early development version of the resurrected `incus-compose` online

incus-compose

“Bring Docker Compose workflows to Incus!” incus-compose implements the Compose specification for the Incus ecosystem, allowing you to define and run multi-container applications using the same docker-compose.yml files you already know.

Why incus-compose?

Incus provides powerful system containers and virtual machines with superior security and isolation, but lacks the declarative multi-container orchestration that Docker Compose offers. This tool bridges that gap:

  • Use existing docker-compose.yml files with Incus containers
  • Leverage Incus’s native OCI registry support for image pulling
  • Run Docker/OCI images directly from registries
  • Manage complex multi-container applications with familiar commands
  • Benefit from Incus’s resource efficiency and security model

Quick Links

Full Documentation | Contributing

Status

Beta - testing the beta1 release of incus-compose.

What works:

  • up, down, list (and ps), start, stop, restart, exec, config, logs commands
  • Compose project parsing via compose-go
  • OCI image pulling from docker.io, ghcr.io, and other registries
  • Bridge networks with automatic name sanitization
  • Storage volumes with UID/GID shifting for proper permissions
  • Bind mounts (local connections only)
  • Port forwarding via proxy devices
  • Incus project isolation

What’s coming:

  • VM instance support alongside containers
  • Container image building via Podman/Docker
  • Advanced compose features (healthchecks, resource limits, etc.)

Architecture

incus-compose uses a resource-first design, see Architecture Documentation for details.

Quick Start

Prerequisites

# Add OCI image remotes to Incus
incus remote add --protocol oci docker.io https://docker.io
incus remote add --protocol oci ghcr.io https://ghcr.io
incus remote add --protocol oci registry.gitlab.com https://registry.gitlab.com

Installation

Binary:
https://gitlab.com/r3j0/incus-compose/-/releases

Source:

# Build from source
git clone https://gitlab.com/r3j0/incus-compose
cd incus-compose
just build
# Or install directly
go install gitlab.com/r3j0/incus-compose/cmd/incus-compose@latest

Usage

# Create a compose.yaml
cat > compose.yaml <<EOF
services:
  web:
    image: docker.io/nginx:alpine
    ports:
      - "8080:80"
    volumes:
      - web-data:/usr/share/nginx/html
volumes:
  web-data:
EOF

# Start services
incus-compose up

# View logs
incus-compose logs -f

# List running services
incus-compose list

# Stop and remove
incus-compose down

See Getting Started for detailed examples.

Credits

This project builds on work by @bketelsen.
Some components are adapted from docker compose.
This project uses AI tools as development aids (drafting, iteration, reviews, tests, and documentation).
Architecture, constraints, and final code decisions are owned by the human committers.

11 Likes

This is awesome news. I stopped mostly because I knew how much work it would be to get real health-check based orchestration going (don’t start wordpress until mysql is alive and serving on port 3306) — but this was before I started using any AI assistance. I hope you’re able to carry the torch and make something great!

4 Likes

Thanks Brian, also wanna public say sorry for not providing future work on my ugly patch, private stuff came in between. Sorry.

Open source shouldn’t mean guilt
no obligations, no worries!

1 Like

Thank you for doing this work, I am really happy to see this continue.

I did a quick clone, followed by a go build -o incus-compose cmd/incus-compose/main.go and then found out API access through the unix socket is not currently supported. To be continued after I sort http API access w/ client certificates :slight_smile:

Lovely! Did a big refactor of the whole app, it contains a compatibliity fix with the cli. Will push soon.

Btw, welcome!

Pushed my rewrite/refactor. Test as you wish, should work by default using the “local” remote of your incus cli.

wow, that’s a lot of change in a single commit.

Ye, it’s “my” own project and in early development :slight_smile:

Hope it’s the last big refactor.

1 Like

Having an issue with Networks on new projects, fixing it.

EDIT: Done.

OK, some more testing. To take incus-compose for a spin, I’m trying the docker-compose files released by the Immich project:

wget -O docker-compose.yml https://github.com/immich-app/immich/releases/latest/download/docker-compose.yml
wget -O .env https://github.com/immich-app/immich/releases/latest/download/example.env

Then incus-compose -p immich-test up results in:

11:04 ERR Error(s) during up url=https://localhost:8443 error=“failed to create image "ghcr.io/immich-app/postgres:18-vectorchord0.5.3": Failed remote image download: Failed getting remote image info: Image not found”
11:04 ERR Command returned error=“failed to create image "ghcr.io/immich-app/postgres:18-vectorchord0.5.3": Failed remote image download: Failed getting remote image info: Image not found”

Checking incus monitor output for more details gives:

location: none
metadata:
context:
name: mmich-app/postgres:18-vectorchord0.5.3
stderr: ‘Failed to run: skopeo --insecure-policy inspect docker://ghcr.io/mmich-app/postgres:18-vectorchord0.5.3:
exit status 1 (time=“2025-12-29T11:04:03+01:00” level=fatal msg=“Error parsing
image name "docker://ghcr.io/mmich-app/postgres:18-vectorchord0.5.3": Requesting
bearer token: received unexpected HTTP status: 403 Forbidden”)’
stdout: “”
level: debug
message: Error getting image alias
timestamp: “2025-12-29T11:04:03.960371876+01:00”
type: logging

Curiously skopeo is passed docker://ghcr.io/mmich-app/ (notice the missing i), which is a path that does not exist.

I have yet to find where the first character of the path gets removed, it’s rather weird. Not sure if this a local issue, a bug in incus-compose, a bug in incus, or something with skopeo.

Version information:

skopeo version 1.18.0
incus 6.20
incuscompose main@82c3ce34193bf06a43ba5cacbd5ad3f49476c7f7

This appears to be the offending line:

	r.Operation.ImageNoRemote = strings.TrimLeft(ref.String(), r.Operation.Remote+"/")

That function strips all leftmost occurences of whatever is in r.Operation.Remote+"/", which I’m guessing in this case includes an i.

I think this is what you had intended?

diff --git a/client/resources.go b/client/resources.go
index c8669ba..53150f6 100644
--- a/client/resources.go
+++ b/client/resources.go
@@ -358,7 +358,7 @@ func (r *Image) sanitize() error {
 
        r.Operation.Cache = r.project.imageCache
 
-       r.Operation.ImageNoRemote = strings.TrimLeft(ref.String(), r.Operation.Remote+"/")
+       r.Operation.ImageNoRemote, _ = strings.CutPrefix(ref.String(), r.Operation.Remote+"/")
 
        r.incusName = r.name

With that change, and some edits to the docker-compose.yml to turn binds into volumes (still using the API) , incus-compose succesfully brought up Immich’s docker-compose config.

To get things to work in my local setup, I patched in a --pool-name cli parameter to change from the default pool (see below). Then, exploring Immich some more, I discovered that I would actually like to have the ability to map docker-compose volumes to incus storage pools. I could see a --volume volname:poolname or something, or alternatively some config file that defines such mappings.

@jochumdev do you have any such needs, or appetite for such a change?

diff --git a/cmd/incus-compose/main.go b/cmd/incus-compose/main.go
index 85464e9..71cd71e 100644
--- a/cmd/incus-compose/main.go
+++ b/cmd/incus-compose/main.go
@@ -887,6 +887,12 @@ func main() {
                                Aliases: []string{"p"},
                                Usage:   `Project name`,
                        },
+                       // TODO: check for valid pool name; idk. what incus permits?
+                       &cli.StringFlag{
+                               Name:    "pool-name",
+                               Usage:   `Pool name`,
+                               Value:   "default",
+                       },
                        &cli.StringSliceFlag{
                                Name:    "file",
                                Aliases: []string{"f"},
@@ -937,6 +943,7 @@ func main() {
                                opts := []client.Option{
                                        client.URL(url),
                                        client.InsecureSkipVerify(),
+                                       client.DefaultStoragePool(cmd.String("pool-name")),
                                }
 
                                // Add TLS client certificate if provided
@@ -979,6 +986,7 @@ func main() {
 
                        opts := []client.Option{
                                client.ProvideConnection(instanceServer, imageCache),
+                               client.DefaultStoragePool(cmd.String("pool-name")),
                        }
 
                        slog.Debug("Using connection", "remote", remote)
1 Like

Yes :slight_smile: I’ll change that. How would you like to get mentioned in the README?

EDIT: Thinking about “Sagi did some testing and bugfixing”.

Yes, I do.

@bketelsen was exploring x-incus- attributes in compose files. Maybe we can do the same?

But there is another refactoring coming up. To archive “worker-pools” I need to make each Resource in client return an Operation like that one.

type Operation struct {
	Context context.Context
	handler func(error) error

	// Set in `Operation.Handle` or directly with `NewDoneOperation`
	done bool

	// Set in `Operation.Handle` or directly with `NewErrorOperation`
	Error error
}

func (o *Operation) Handle() error {
	if o.done {
		return o.Error
	}

	select {
	case <-o.Context.Done():
		return o.Context.Err()
	default:
		err := o.handler(o.Error)
		o.Error = err
		o.done = true
		return err
	}
}

This refactor is nearly done. After I pushed that I’m happy to accept PRs.

About my use of Gitlab, are you happy to use Gitlab?

I forgot to say: Thank you, very happy that you do tests. This helps a lot.

1 Like

I appreciate you taking the time to credit people, though I don’t feel I have done much :slight_smile: I’ll let you decide.

My use case would be to use (supported) docker-compose.yml from various projects, but run them on Incus. For upgrades, it would be simpler if no modifications would need to be made to such files. So personally, I would have a slight preference for some kind of Incus-specific config file on the side, a bit like a .env file.

I don’t think I understand what you are after, but I’ll probably find out in the future :smiley:.

Sure, I think I can find my way and send a patch or two if I run into something.

Ye, sure. Will do so after some more tests of you. Pick a name and link :slight_smile:

Same use case we have, let’s test lots of those.

In short, I wanna have parallelism :slight_smile:

1 Like

I’ve just noticed that I have zombie cgroups floating around that correspond to instances that incus-compose created. They persist when the instances are shut down and when incus is restarted. I am both unsure how this came about, or how to resolve this situation (edit: I can rmdir the cgroups manually).

# incus list --project immich-test
+-------------------------+---------+------+------+-----------------+-----------+
|          NAME           |  STATE  | IPV4 | IPV6 |      TYPE       | SNAPSHOTS |
+-------------------------+---------+------+------+-----------------+-----------+
| database                | STOPPED |      |      | CONTAINER (APP) | 0         |
+-------------------------+---------+------+------+-----------------+-----------+
| immich-machine-learning | STOPPED |      |      | CONTAINER (APP) | 0         |
+-------------------------+---------+------+------+-----------------+-----------+
| immich-server           | STOPPED |      |      | CONTAINER (APP) | 0         |
+-------------------------+---------+------+------+-----------------+-----------+
| redis                   | STOPPED |      |      | CONTAINER (APP) | 0         |
+-------------------------+---------+------+------+-----------------+-----------+
# find /sys/fs/cgroup/ -type d -name 'lxc.monitor*immich*'
/sys/fs/cgroup/lxc.monitor.immich-test_immich-machine-learning-1
/sys/fs/cgroup/lxc.monitor.immich-test_redis
/sys/fs/cgroup/lxc.monitor.immich-test_immich-server-4
/sys/fs/cgroup/lxc.monitor.immich-test_database-2
/sys/fs/cgroup/lxc.monitor.immich-test_database
/sys/fs/cgroup/lxc.monitor.immich-test_immich-server-2
/sys/fs/cgroup/lxc.monitor.immich-test_redis-2
/sys/fs/cgroup/lxc.monitor.immich-test_immich-server-5
/sys/fs/cgroup/lxc.monitor.immich-test_immich-machine-learning
/sys/fs/cgroup/lxc.monitor.immich-test_immich-server-3
/sys/fs/cgroup/lxc.monitor.immich-test_database-1
/sys/fs/cgroup/lxc.monitor.immich-test_redis-3
/sys/fs/cgroup/lxc.monitor.immich-test_immich-server-1
/sys/fs/cgroup/lxc.monitor.immich-test_immich-server
/sys/fs/cgroup/lxc.monitor.immich-test_redis-1

Any idea if this is related to behaviour of incus-compose itself?

Another observation that may more useful for direct improvement of incus-compose itself. Again going back to the immich docker-compose files; it seems the following bit of config for the immich-server component:

    ports:
      - '2283:2283'

gets translated in the following incus device config for the corresponding immich-server instance:

devices:
  proxy-0.0.0.0-0:
    connect: tcp:127.0.0.1:0
    listen: tcp:0.0.0.0:0
    type: proxy
  proxy-0.0.0.0-2283:
    connect: tcp:127.0.0.1:2283
    listen: tcp:0.0.0.0:2283
    type: proxy

The second one makes sense to me; the first does not. I have not yet looked at the code.

1 Like

I’m not, have to test/ play around with that.

If you like manualy create oci containers and check the behaviour, you can extract some commands here on howto do so https://github.com/nuttyb-community/server/blob/main/justfile.

It would be nice to know which features trigger that also the code itself does not enable any “debugging” options or the like.

I’ll make sure this bug is gone together with the the other 2 bugs you reported and fixed.

That will be part of the major architectural update I’ll push soon to incus-compose.

It will feature:

  • An extensible client library that wraps the incus go client library
  • Detailed unit tests
  • Parallel image downloads
  • A better ps with more incus related infos
  • down --project aliased to down --volumes = everything except images gone.
  • “hooks” for everything
  • Somehow good docs
  • The base layer for progress reports (they will be easy to implement after)

I write it with the pilosopy “Keep it simple stupid” and “boring”.