Hi,
I was taken by surprise lately by the “phase out access to our image server for LXD users” subject.
Had the intention to migrate from LXD to Incus, but was waiting for Incus releases to stable out a bit.
An other hurdle to take while migrating, is that we use LXD on AmazonLinux2023 using snapd.
I see Incus has chosen not to release snaps, so that option is not there.
Any chance there will be packages for AmazonLinux2023?
Kind regards
It looks like AmazonLinux2023 is a Fedora-style Linux distribution, Relationship to Fedora - Amazon Linux 2023
See at Packages for ganto/lxc4 the available packages.
How does one add packages to AmazonLinux2023?
@Conan_Kudo has some in progress work to get Incus natively into Fedora, which should then make it easy to trickle into CentOS Stream and EPEL. But that’s work in progress and there hasn’t been any new builds in the past 3 weeks. Bugzilla for this is https://bugzilla.redhat.com/show_bug.cgi?id=2260492
@ganto has been maintaining a great LXC, LXD and now Incus COPR repository for Fedora users for a number of years now, I don’t know how modern the current Amazon Linux is and whether any of those builds may work on it, but it may be a start too: ganto/lxc4 Copr
And lastly, I just discovered another COPR repository, this one directly targeting EPEL9 and with recent builds: zarhi/incus_R9 Copr
That last one is probably the most likely to work today on any RHEL9 derivative, but I’ve only discovered it a few minutes ago and haven’t had any interaction with its maintainer, so someone would need to test it and see if it actually works properly 
It’s also possible to run incus in a privileged docker/podman container, if you like to keep it running containerized.
This is my implementation, but it’s also not hard to make your own.
@cmspam I like that idea very much! Thank you for sharing. Maybe even good for Incus to include that as release target.
Have you found any limitations yet?
For instance, I use lvm. So I added that support to the image now.
But I remember with LXD that you had to set;
snap set lxd lvm.external=true
to give the host a up2date view on the LVM status.
Anything similar we could do in this Docker setup?
Well, we’re using a privileged container, so I think anything you pass through to the container /dev wise, be it hard disk or whatever else, it can be accessed freely by the container, I would think.
Did you try LVM and it didn’t work? I haven’t tried with LVM, only with btrfs and zfs.
Limitation-wise, there were some issues with btrfs if using an alpine-based build. But with the debian-based build, and everything necessary passed through as a volume to the container, everything I have tried has worked.
Did you try LVM and it didn’t work? I haven’t tried with LVM, only with btrfs and zfs.
docker exec incus apt-get install -y lvm2
I did now, but I get an error. Both running the incus command from the instance of within docker.
incus storage create test lvm source=/dev/nvme2n1
Error: Custom loop file locations are not supported
This seems because the disk was attached after the Incus docker container was started.
So I need a way to have updated /dev content in the docker container.
After restarting the container I get
incus storage create test lvm source=/dev/nvme2n1
Error: Error creating LVM thin pool named "IncusThinPool": Failed to run: lvcreate --yes --wipesignatures y --thinpool test/IncusThinPool --extents 100%FREE: exit status 5 (/dev/test2/lvol0: not found: device not cleared
Aborting. Failed to wipe start of new LV.)
Probably something similar about not having an up to date /dev in the container.
Can you tell me what command have you run the container with? My thoughts are, if you pass through all the related devices, it should work.
You could try passing through all of /dev perhaps.
At least, let’s say, you can try running it with the flag below?
--device /dev/nvme2n1
Or you can try, maybe
--volume /dev:/dev:rw
I’ve not tried lvm, so unfortunately I don’t have a great answer until I try it.
I needed
--volume /dev:/dev
but also inside the docker container
apt-get install udev
/usr/lib/systemd/systemd-udevd &
After that I can create lvm based storage and incus containers on it
Great. I can add the dependencies to the Dockerfile
An update, I’ve added lvm, udev, and so on to the Dockerfile, so you should be able to use cmspam/incus-docker:latest now without modification. Just pass through what needed to be passed through. Please try if you can.
I updated it and marked it ready for the reviewer to start last week.
Nice! Thanks for your work on this!