Proof Of Concept: incus-compose

Incus has no concept of health checks like docker does, which complicates things like zero-downtime deployments.

1 Like

@stgraber Would adding a health check concept for Incus be something that you could see as in scope for Incus in the future, or do you envision this being handled outside of Incus? I personally like the concept of having it in Incus myself, but I’m still getting my feet wet here.

We do have a mechanism that instances can use to report back their readiness, but that’s not likely to get much support from regular OCI images.

We offer a guest facing API through /dev/incus/sock which allows an instance to flip its status between RUNNING and READY. The intent is for READY to then be the state you’re looking for in the API when getting the list of targets for a reverse proxy or the like.

There also are some vague plans to allow delaying things like advertising of BGP routes until an instance has reached READY state, basically avoiding any traffic from being pushed to it until the service is ready.

1 Like

Interesting, I wasn’t aware of this api. So if we wanted feature parity with docker’s health checks we could create a health proxy service that performs the docker style checks (is mysql listening on port 3306, etc) and then update the guest status with the incus api.

There are a lot of variations on the health checks Docker allows, but it doesn’t seem to be complicated work, just time consuming.

Hello bketelson,

Is there a how-to for the “installation” and maybe a mapping what is working with incus compose and what is not working?

I compiled incus-compose on another machine and scp’ed it to my incus server. But that seems not to work, as i get always a warning, that something is not implement without any further information.

My plan was, to write write special dockerfiles that hopefully get used by incus-compose (e.g. s6-init) when i define that in the composefile and to config the containers with my predefined files, profiles and environment vars… for easily deploying my used images.

But at the moment im stuck with that warning.

Looking forward to your feedback.

BR Jan

just used incus-compose up… it’s working now :man_facepalming:t2:

1 Like

Hi bketelsen,

is there a way to do something like this:


#     - frigate_storage:/media/frigate
     - type: disk
       source: default_frigate_storage
       target: /media/frigate
       x-incus-shift: true
networks:
  eno1:
    external: true


volumes:
  frigate_storage:
    driver: incus
    driver_opts:
      pool: storage_ext

My goal is to add an existing storage volume to the container.

Looking forward to your feedback.

Best regards

Jan

I don’t think that would work today, can you file an issue to track the request?

1 Like

@stgraber Would it seem possible to consider offering (partial) compatibility with the Docker Engine API, in so the regular docker compose CLI can be used just by providing a custom docker context?

Not likely to be something we’d care enough about to do natively, but someone can probably write an API bridge.

Just FYI, that repo has been archived and there is no pointer to a new repo or effort. I guess the POC didn’t work out?

2 Likes

I was following this whole discussion on incus compose and thought it was kind of weird from the start. To understand my reasoning we have to understand why docker compose is popular and what type of problem it solves.

I think docker compose became popular mostly because docker did not keep track of how it was created, and a lack of documentation would lead to set-ups which were not reproducible. To solve this problem, a declarative, readable file serves as a documentation and as a deployment method. Great stuff, but it’s solving a problem which existed before.

Incus has two main ways of making declarative reproducible deployments. The first one is by using profiles (with cloud-init) and the second one is by using ansible+terraform. If these solutions already exist and already have people using them, what problem is making an incus compose command solving? Nothing really!

People are interested in them because they’re coming from other environments and wish to keep using the same tools. But in this case, it’s much more trouble to build a new tool than to learn something which already exists. Try using what already exists and help improve it. It’ll be much easier.

With that being said, I think incus still has some issues in the app deployment using profiles since some of the input configuration is not set through profiles (last time I tried, they were only applied if provided on input). If this is fixed and people learn how to use this method, an incus compose is not really necessary.

Think there is one point we need to acknowledge or better shouldn’t ignore.

How to convert docker settings into Incus ones. All the world has examples how to get their application deployed using docker (compose) or kubernetes but non has an Incus example.

Thought that this project was kind of an attempt to close the bridge but not really an success as it looks.

Right, someone needs to take the lead and start building or adding this to Incus. Ansible+Terraform are a good alternative and I wonder how they deal with all the docker (compose) and kubernetes stuff. Haven’t looked but if there is any kind of collection it might be worth to picky back on this?

Properly many of us struggle to find a solution to improve our deployments…

1 Like

At ssh2incus.com we are working on a more advanced “swiss knife” tool that will allow, among other cool features, using Dockerfile and docker compose equivalents for Incus.
Beta version will be available in early 2026.

While working on our tool for Incus we were not aware of this incus-compose PoC project but it’s good to know that there’s some interest in the community :slight_smile:

1 Like

This sounds really cool! I looked through your projects and I must say you have developed what seems like a really interesting software suite. Can you share some more details on how this “swiss-army tool” will work? Is it going to be usable as a drop-in replacement for Docker or Podman?

@joeweaver, we are not reinventing the wheel with Incus to replace docker or podman or anything else.

our goal is to allow using Incus for rapid instance deployment with orchestration similar to docker compose, image building and management similar to docker build and docker image, as well as other solutions such as scheduled instance backups and fast restoration using remote storage over s3 and sftp and much more.

Sounds like an awesome substitute if you want to spin up containers quickly without having to bother with creating an underlying VM running docker or container runtime. Way easier and more convenient to have it native in incus for many use cases. Thanks.

Perhaps there is another way for Incus to become attractive to the vast numbers of developers worldwide, and importantly provide them with a seamless onramp to go from development to production — by providing Devcontainer support across Incus LXC and OCI.

I’m interested in Containerlab inside a VM or LXC.

If Incus supported Devcontainer it is likely that support in VS Code and other Editors and IDEs would appear.

Incus needs to be on the DevOps desktop and supporting Devcontainer would go a long way towards putting it there especially in a seamless remote development setup.

Being able to spin up Dev environments in an Incus LXC then populate inside with Docker all via Devcontainer would be awesome.