Migrating from Docker to Incus with CephFS: Storage Integration Challenge

Incus
1

I’m currently in the process of transitioning my infrastructure from Docker to Incus containers. Along the way, I’ve encountered a challenge related to my CephFS storage setup that I haven’t been able to resolve, despite numerous attempts.

In my current Docker-based setup, I use the CephFS volume plugin to provide each container with its own persistent volume on a Ceph cluster. Here’s a simplified version of the volume configuration:

driver: cephfs:latest
driver_opts:
  client_name: dockeruser
  keyring: /etc/ceph/dockeruser.keyring
  mount_opts: mds_namespace=dockervolumes
  remote_path: /container1
  servers: ip1, ip2, ip3

My goal is to achieve the same with Incus: individual, persistent root storage for each container, backed by CephFS.

Has anyone here successfully set up something similar with Incus? I’d greatly appreciate any advice, suggestions, or best practices you can share.

2

For anyone coming across this post in the future:
The scenario described here unfortunately isn’t possible with CephFS. For more details, see this discussion: HowTo: CephFS with LXD Clustering

For my particular use case, I’ve decided to go with Ceph RBD instead.