Incus on OpenSuse(Tumbleweed, MicroOS)

1

I am a big fan on incus, also a fan of Suse’s MicroOS and Tumbleweed.

Although I hear much of how next version of Fedora, Debian, and Ubuntu will get incus, I hear nothing of Suse. LXD is included in Suse, so it surprises me that there was so little done about incus as of yet.

I checked the open build system, and didn’t find any working, recent Incus build, so I branched one I found, and updated it with the latest 0.5.1, added the web UI, added SELinux configuration, and added some overlooked dependencies needed for VMs to work.

This is my branch on the open build system.

I also made web UI available here.

The problem is, this is actually the first project I have attempted of this nature, and I think it is hacky, not tidy, and not done properly. I think too much is hard coded, too much is haphazard, and often I feel I don’t know what I’m doing etc. Although I made it to a stage where everything works, I’m not happy with the way I get there.

I am hoping someone could take a look at it, maybe help make it cleaner or more ‘as it should be’ in the .spec file and so on, or even branch it or take the next steps, with the ultimate goal to get it included in Suse in the very near future. At least, we can have a ‘good’ repository to use like they have on debian, fedora, etc.

But, if someone is a Suse user who really wants incus, feel free to use the RPMs built from here. I will try to at least keep it up to date.

1 Like
2

Yeah, we’ve been nagging @cyphar quite repeatedly about it but he’s been a bit busy lately :wink:

3

I’m surprised already to receive a response, and from the god himself :smile:
I rely on you for my zfs module, my kernel, and my incus setup on .deb distributions, so thank you very much for all your work.

I see, he is the one who does the suse-related matters. In this case, maybe just a matter of time. I look forward to seeing it hopefully be included soon.

4

Yeah, Aleksa is the original creator of Incus and the long time packager of LXD in OpenSUSE.

He did mention some issues with newer liblxc related to the version of meson in SUSE, this is what caused LXD to be stuck on an older version and caused some difficulty in getting Incus packaged too. Hopefully that gets resolved soon though!

5

I’m experimenting with MicroOS at the moment (seems great for a low maintenance container host) - many thanks for packaging incus.

I’ve been building a custom iso with kiwi-ng inside a privileged tumbleweed podman container. Mounting a zram device to the build directory makes builds complete in around 4 mins (& cleaning the build directory very quick) - see my notes for kpartx & how to create a build container at that link.

  • In my image I use systemd daemons & busybox applets as much as possible (so breakage should be zero even with a rolling release)

  • To use encrypted passwords in image configuration:

SALT=$(LC_ALL=C tr -dc '[:alnum:]' < /dev/urandom | head -c50)
openssl passwd -1 -salt $SALT YOUR_PASSWORD

In your build environment setup the kiwi-ng repo & install the depends:

zypper in git kiwi-systemdeps-iso-media kiwi-systemdeps-bootloaders kiwi-systemdeps-disk-images
  • Building the oem image type generates a qemu image & an iso image:
sudo kiwi-ng --profile Standard system build \
--description $DEVOPS/kiwi/build-tests/x86/tumbleweed/test-image-MicroOS \
--target-dir $DEVOPS
  • if testing images with qemu / Virtual Machine Manager - remove the iso image in the vm configuration / apply & add it back again / apply - after each build - libvirt seems to keep the old iso in memory (but possibly this is due to my use of zram)
1 Like
6

This is killing me, literally :wink: any news on this ? Packages on the official repos for SLE Micro / Leap Micro ?

7

@cyphar any update?

1 Like
8

The current distribution method of the project appears to be a bottleneck. While Debian/Ubuntu repositories provided by @stgraber cater to production and enterprise use cases effectively, there’s a noticeable gap for the EL family and OpenSUSE/SUSE.

Distributions like Archlinux, Gentoo, NixOS, Void Linux, and Fedora have community-maintained packages, which are commendable efforts targeting more specialized or workstation-oriented use cases. However, these do not fully address the broader needs.

An officially supported package in EPEL could potentially encompass all use cases for the EL family. The SRPM or spec file from this effort could then be adapted for SUSE/OpenSUSE or vice-versa. Alternatively, distributing the packages under the Linux Containers project could ensure uniform building and testing processes.

This bottleneck may be impeding the wider adoption of Incus. Collaborative efforts from EL packagers to create a high-quality, packaged product could significantly reduce individual efforts.

9

Hello!
@cyphar there are some news on the opensuse package of Incus?

10

I recently decided to try out OpenSuse and Incus at the same time. I make extensive use of LXD and wanted to compare them. But, if no Incus on OpenSuse, may have to give up on it. No word on it?

11

I’m working on a package. However, there are still some issues I’m debugging and so it’s not ready to be used yet. Hopefully I can get the issues fixed up and submitted to Tumbleweed soon.

(The main thing blocking it was that Leap didn’t have a new enough version of Meson to build LXC >= 4.x, which meant that we couldn’t even release newer versions of LXD let alone Incus. Yes, we could’ve just shipped for Tumbleweed but I use Leap on my server so I wouldn’t have been able to test it on a proper system without Leap support. We finally got a newer version of Meson this year but I was busy with other things and so didn’t get around to packaging it until now.)

It does suck that openSUSE was the first distro to have proper native packaging for LXD, but we’re the last to have Incus packages… :smiling_face_with_tear:

2 Likes
12

I just happened to stumble across this today, glad I put off this task for a bit. Will do some testing on a local MicroOS server.

13

Just did a quick test as for tumbleweed amd64 as your build seemed successfull, so added the repo and installed incus. All seems to work fine? Should I expect any issues or is it party time? :smiley:

14

There is an issue with VMs I’m trying to debug. LXD VMs work fine, but Incus ones don’t (even though the packaging is almost identical). The VMs start but they can’t get an IPv4 address and we can’t connect to the vm-agent (presumably because of the IPv4 issue). Nothing in the log either.

Containers work fine though – I’ve already migrated my home server to Incus with the packages I’ve prepared and haven’t had any issues in the past few weeks.

15

On microos all features were working but some unlisted qemu dependencies were needed for vms.

16

Sounds like a networking/firewalling problem. Are you saying you have LXD and incus installed on the same host side by side? Do you have both lxdbr0 and incusbr0 bridges? Do your firewall rules allow traffic on incusbr0?

17

No, this is on an Incus-only host and container networking for the bridge interface works fine. The only issue is with VMs. I’m not even sure the IPv4 thing is the root cause of the issue, I had to debug some other packaging issues so it’s possible there’s just something else I’m missing. The main problem is that incus console v1 just hangs once it connects, so I can’t even try to debug the networking oddity.

18

My above comment is about a system with the packaging fixes to add the missing QEMU dependencies we talked about on OBS.

Are you sure launching a VM works completely? Can you connect to the console? On Tumbleweed containers can get addresses but VMs get stuck in this state:

+------+---------+---------------------+-----------------------------------------------+-----------------+-----------+
| NAME |  STATE  |        IPV4         |                     IPV6                      |      TYPE       | SNAPSHOTS |
+------+---------+---------------------+-----------------------------------------------+-----------------+-----------+
| c1   | RUNNING | 10.254.12.65 (eth0) | fd42:abc3:6168:c75f:216:3eff:fe9e:bcd4 (eth0) | CONTAINER       | 0         |
+------+---------+---------------------+-----------------------------------------------+-----------------+-----------+
| v1   | RUNNING |                     | fd42:abc3:6168:c75f:216:3eff:fe41:daa2 (eth0) | VIRTUAL-MACHINE | 0         |
+------+---------+---------------------+-----------------------------------------------+-----------------+-----------+

On my machines, incus console v1 just freezes. So the VM is in a “running” state but it’s clearly not usable.

19

At the time I tested it, which is when I made the comment on OBS, yes, VMs worked. At least an alpine vm I tested with all default settings (images:alpine/3.20 --vm)

I could connect to the VM with ‘incus exec alpinevm ash’ and it worked fine.

But now that you mention this problem, I decided to try it again with a blank install of microos in a VM. And it’s still working fine.

localhost:~ # incus ls
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+
|   NAME   |  STATE  |        IPV4        |                     IPV6                     |      TYPE       | SNAPSHOTS |
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+
| alpinevm | RUNNING | 10.169.56.8 (eth0) | fd42:8b5:9e3e:168e:216:3eff:fe3f:e45f (eth0) | VIRTUAL-MACHINE | 0         |
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+
localhost:~ # incus exec alpinevm ash
~ # uname -r
6.6.60-0-virt
~ # exit
localhost:~ # uname -r
6.11.7-1-default

So I decided to try a debian VM to see if the OS was the issue.

+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+
|   NAME   |  STATE  |        IPV4        |                     IPV6                     |      TYPE       | SNAPSHOTS |
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+
| alpinevm | RUNNING | 10.169.56.8 (eth0) | fd42:8b5:9e3e:168e:216:3eff:fe3f:e45f (eth0) | VIRTUAL-MACHINE | 0         |
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+
| debvm    | RUNNING |                    | fd42:8b5:9e3e:168e:216:3eff:fedd:7d1a (eth0) | VIRTUAL-MACHINE | 0         |
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+

And I get the problem you mentioned.

localhost:~ # incus ls
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+
|   NAME   |  STATE  |        IPV4        |                     IPV6                     |      TYPE       | SNAPSHOTS |
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+
| alpinevm | RUNNING | 10.169.56.8 (eth0) | fd42:8b5:9e3e:168e:216:3eff:fe3f:e45f (eth0) | VIRTUAL-MACHINE | 0         |
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+
| debvm    | RUNNING |                    | fd42:8b5:9e3e:168e:216:3eff:fedd:7d1a (eth0) | VIRTUAL-MACHINE | 0         |
+----------+---------+--------------------+----------------------------------------------+-----------------+-----------+

So I noticed, with alpine, I need to disable secureboot.
Let’s try with debian.

localhost:~ # incus config set debvm security.secureboot=false
localhost:~ # incus start debvm

And…

localhost:~ # incus ls
+----------+---------+-----------------------+----------------------------------------------+-----------------+-----------+
|   NAME   |  STATE  |         IPV4          |                     IPV6                     |      TYPE       | SNAPSHOTS |
+----------+---------+-----------------------+----------------------------------------------+-----------------+-----------+
| alpinevm | RUNNING | 10.169.56.8 (eth0)    | fd42:8b5:9e3e:168e:216:3eff:fe3f:e45f (eth0) | VIRTUAL-MACHINE | 0         |
+----------+---------+-----------------------+----------------------------------------------+-----------------+-----------+
| debvm    | RUNNING | 10.169.56.76 (enp5s0) |                                              | VIRTUAL-MACHINE | 0         |
+----------+---------+-----------------------+----------------------------------------------+-----------------+-----------+
localhost:~ # incus exec debvm bash
root@distrobuilder-05821261-ac12-4c05-9f0e-8c6884d3c8a2:~# uname -r
6.1.0-27-amd64
root@distrobuilder-05821261-ac12-4c05-9f0e-8c6884d3c8a2:~# exit
exit
localhost:~ #

So something is a problem with secureboot. The issue we have has some correlation.

20

Then perhaps the issue you have is with UDP checksums on DHCP requests.