Early development version of the resurrected `incus-compose` online

Incus
,
33

Just tried my first incus-compose with Immich on a remote IncusOS server. Here’s the compose.yaml I used:

# docs: https://docs.immich.app/install/docker-compose
name: immich

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    # extends:
    #   file: hwaccel.transcoding.yml
    #   service: cpu # set to one of [nvenc, quicksync, rkmpp, vaapi, vaapi-wsl] for accelerated transcoding
    volumes:
      # Do not edit the next line. If you want to change the media storage location on your system, edit the value of UPLOAD_LOCATION in the .env file
      - ${UPLOAD_LOCATION}:/data
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    ports:
      - '2283:2283'
    depends_on:
      - redis
      - database
    restart: unless-stopped
    healthcheck:
      disable: false

  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, rocm, openvino, rknn] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}
    # extends: # uncomment this section for hardware acceleration - see https://docs.immich.app/features/ml-hardware-acceleration
    #   file: hwaccel.ml.yml
    #   service: cpu # set to one of [armnn, cuda, rocm, openvino, openvino-wsl, rknn] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - immich-model-cache:/cache
    env_file:
      - .env
    restart: unless-stopped
    healthcheck:
      disable: false

  redis:
    container_name: immich_redis
    image: docker.io/valkey/valkey:9@sha256:3b55fbaa0cd93cf0d9d961f405e4dfcc70efe325e2d84da207a0a8e6d8fde4f9
    healthcheck:
      test: redis-cli ping || exit 1
    restart: unless-stopped

  database:
    container_name: immich_postgres
    image: ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
      POSTGRES_INITDB_ARGS: '--data-checksums'
      # Uncomment the DB_STORAGE_TYPE: 'HDD' var if your database isn't stored on SSDs
      # DB_STORAGE_TYPE: 'HDD'
    volumes:
      # Do not edit the next line. If you want to change the database storage location on your system, edit the value of DB_DATA_LOCATION in the .env file
      - ${DB_DATA_LOCATION}:/var/lib/postgresql/data
    shm_size: 128mb
    restart: unless-stopped
    healthcheck:
      disable: false

volumes:
  library:
  immich-model-cache:
  postgres:

I only changed restart: always to unless-stopped and the bind mounted volumes to regular ones declared at the bottom.

Accompanying .env file:

# You can find documentation for all the supported env variables at https://docs.immich.app/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=library

# The location where your database files are stored. Network shares are not supported for the database
DB_DATA_LOCATION=postgres

# To set a timezone, uncomment the next line and change Etc/UTC to a TZ identifier from this list: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones#List
TZ=Europe/Paris

# The Immich version to use. You can pin this to a specific version like "v2.1.0"
IMMICH_VERSION=v2

# Connection secret for postgres. You should change it to a random password
# Please use only the characters `A-Za-z0-9`, without special characters or spaces
DB_PASSWORD=supersecretpassword

# The values below this line do not need to be changed
###################################################################################
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

Doing incus-compose up trips up on the remote names:

error="image source error getting image server for docker.io: The remote \"docker.io\" doesn't exist"
error="image source error getting image server for ghcr.io: The remote \"ghcr.io\" doesn't exist"

Indeed, I’ve added those as docker and ghcr in my remotes :sweat_smile:

After renaming those I got:

15:53 ERR Getting healthd image url=https://XXX.XXX.XXX.XXX:8443 project=immich image=registry.gitlab.com:r3j0/incus-compose/ic-healthd:latest error="invalid format: image(registry.gitlab.com:r3j0/incus-compose/ic-healthd:latest): invalid reference format"

I then added GitLab as a remote:

incus remote add registry.gitlab.com https://registry.gitlab.com --protocol=oci

And then it hangs on:

15:56 DBG Done url=https://XXX.XXX.XXX.XXX:8443 name=registry.gitlab.com:r3j0/incus-compose/ic-healthd:latest kind=image action=ensure

Full debug output:

15:56 DBG Using connection remote=my-remote
15:56 DBG Got project url=https://re.da.ct.ed:8443 name=default incus_name=default
15:56 DBG Got project url=https://re.da.ct.ed:8443 name=immich incus_name=immich
15:56 DBG Found healthchecks url=https://re.da.ct.ed:8443 project=immich incus_project=immich
15:56 DBG Getting image url=https://re.da.ct.ed:8443 project=immich incus_project=immich image=ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23 service=database
15:56 DBG Getting image url=https://re.da.ct.ed:8443 project=immich incus_project=immich image=ghcr.io/immich-app/immich-server:v2 service=immich-server
15:56 DBG Getting image url=https://re.da.ct.ed:8443 project=immich incus_project=immich image=ghcr.io/immich-app/immich-machine-learning:v2 service=immich-machine-learning
15:56 DBG Getting image url=https://re.da.ct.ed:8443 project=immich incus_project=immich image=docker.io/valkey/valkey:9@sha256:3b55fbaa0cd93cf0d9d961f405e4dfcc70efe325e2d84da207a0a8e6d8fde4f9 service=redis
15:56 DBG Prepared healthd sidecar image url=https://re.da.ct.ed:8443 project=immich incus_project=immich name=ic-healthd
15:56 DBG network:default
15:56 DBG network:default
15:56 DBG network:default
15:56 DBG Ensure url=https://re.da.ct.ed:8443 project=immich incus_project=immich resources="[network(ic-h3vxrnqwhm) instance(database-1) instance(immich-machine-learning-1) instance(redis-1) instance(immich-server-1) image(ghcr.io/immich-app/postgres@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23) image(ghcr.io/immich-app/immich-server:v2) image(ghcr.io/immich-app/immich-machine-learning:v2) image(docker.io/valkey/valkey@sha256:3b55fbaa0cd93cf0d9d961f405e4dfcc70efe325e2d84da207a0a8e6d8fde4f9) image(registry.gitlab.com:r3j0/incus-compose/ic-healthd:latest) healthd(ic-healthd)]"
15:56 DBG Done url=https://re.da.ct.ed:8443 name=docker.io/valkey/valkey:9@sha256:3b55fbaa0cd93cf0d9d961f405e4dfcc70efe325e2d84da207a0a8e6d8fde4f9 kind=image action=ensure
15:56 DBG Done url=https://re.da.ct.ed:8443 name=ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23 kind=image action=ensure
15:56 DBG Done url=https://re.da.ct.ed:8443 name=registry.gitlab.com:r3j0/incus-compose/ic-healthd:latest kind=image action=ensure

I tried adding --no-helathd but it remains stuck:

incus-compose up --remote re.da.ct.ed --no-healthd --debug
16:00 DBG Using connection remote=re.da.ct.ed
16:00 DBG Got project url=https://re.da.ct.ed:8443 name=default incus_name=default
16:00 DBG Got project url=https://re.da.ct.ed:8443 name=immich incus_name=immich
16:00 DBG Getting image url=https://re.da.ct.ed:8443 project=immich incus_project=immich image=docker.io/valkey/valkey:9@sha256:3b55fbaa0cd93cf0d9d961f405e4dfcc70efe325e2d84da207a0a8e6d8fde4f9 service=redis
16:00 DBG Getting image url=https://re.da.ct.ed:8443 project=immich incus_project=immich image=ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23 service=database
16:00 DBG Getting image url=https://re.da.ct.ed:8443 project=immich incus_project=immich image=ghcr.io/immich-app/immich-server:v2 service=immich-server
16:00 DBG Getting image url=https://re.da.ct.ed:8443 project=immich incus_project=immich image=ghcr.io/immich-app/immich-machine-learning:v2 service=immich-machine-learning
16:00 DBG network:default
16:00 DBG network:default
16:00 DBG network:default
16:00 DBG Ensure url=https://re.da.ct.ed:8443 project=immich incus_project=immich resources="[network(ic-h3vxrnqwhm) instance(redis-1) instance(database-1) instance(immich-machine-learning-1) instance(immich-server-1) image(docker.io/valkey/valkey@sha256:3b55fbaa0cd93cf0d9d961f405e4dfcc70efe325e2d84da207a0a8e6d8fde4f9) image(ghcr.io/immich-app/postgres@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23) image(ghcr.io/immich-app/immich-server:v2) image(ghcr.io/immich-app/immich-machine-learning:v2)]"
16:00 DBG Done url=https://re.da.ct.ed:8443 name=docker.io/valkey/valkey:9@sha256:3b55fbaa0cd93cf0d9d961f405e4dfcc70efe325e2d84da207a0a8e6d8fde4f9 kind=image action=ensure
16:00 DBG Done url=https://re.da.ct.ed:8443 name=ghcr.io/immich-app/postgres:14-vectorchord0.4.3-pgvectors0.2.0@sha256:bcf63357191b76a916ae5eb93464d65c07511da41e3bf7a8416db519b40b1c23 kind=image action=ensure
16:08 DBG Result with error url=https://re.da.ct.ed:8443 name=ghcr.io/immich-app/immich-server:v2 kind=image action=ensure error="in an operation: Failed remote image download: Alias already exists: ghcr.io/immich-app/immich-server:v2"

So yeah, I’m currently stumped :sweat_smile: I suspect that the difference in default storage pool name between regular incus (default) and IncusOS (where the default storage is called local) may be the cause.

On another note, I agree with what @edorgeville said:

add a –remote flag (possible improvement: use same remote as currently selected by incus CLI),

This is a big one for me as well: I incus remote switch to my server so as not to have to add the remote name all the time, and I was expecting incus-compose to use the currently selected one. That would fit with how incus behaves, instead of hardcoding local unless overriden with --remote of $INCUS_REMOTE.

Meanwhile: thank you very much for bringing us this far, looking forward to using it more!