We are a single board manufacturer, and we are looking to create an appliance image with Incus-hosted OpenWrt for our products. This allows us to provide our own base system (currently demoing NixOS but can use vendor SDK image as well) to handle hardware support, saving us waiting for upstream to support our devices first. The base system can also be used to host heavier services that are not well suited for OpenWrt, thus provides better hardware utilization. I have been using OpenWrt over Incus & NixOS at home for a while, so I feel confident in this solution.
I have quickly created a basic image based on the existing NixOS configurations with Incus installed. However, I’m not sure how to configure Incus from chroot, as I cannot start the daemon in chroot environment.
I am not quite sure I got it right. Incus has configuration files to run through systemd while others have created non-systemd configurations as well. Is the issue relating to the service management of Incus in that chroot?
No the issue is about managing Incus in that chroot, but the normal method requires the daemon to be running, and the default daemon management systemd won’t launch the unit in a chroot. We’d like to solve this in a non hacky way (forcing systemd unit to launch is considered hacky) thus the question.
Though now you said it, maybe we can just manually launch incusd in the background and then do the configuration?
We are trying to create an image that has incus preinstalled with a container configured. This seems to only create an image that to be used by incus?
This is incorrect, NixOS and Nix have no common lineage with Debian. The only other distribution/package manger that can make that claim is Guix as, iirc, it was forked from Nix.
EDIT: the phrasing was ambiguous, I meant to say that the only other distro that Nix and NixOS have in common in terms of lineage is Guix
I’m not sure how relevant this is to Radxa’s desired use case. The closest I can think of that aligns with what is desired is to invoke incusd manually.
@RadxaYuntian have you looked at the systemd unit files and tried to mirror what’s done there? That could help you out a bit
You could, but at the same time I’m wondering why you need the chroot in the first place.
Couldn’t you (or Radxa rather) create a base using NixOS or Debian with Incus preinstalled and limit services and users to LXC instances? This was pretty heavily implied in your original post.
It should still constrain your users to supported distributions while allowing for environments that’s one for one with upstream distributions as supplied by the Linux Container image server.
It would also bring you somewhat close to how other atomic distributions (Silverblue and the rest of the Fedora Atomic family) intends users to handle mutable state with Podman.
EDIT: Please pardon the earlier delete, I thought I wasn’t replying to your post :P.
The plan is to host an OpenWrt instances with the provided NixOS image. To the end user it is as if they are using just another OpenWrt image, without the need for us to actually port all kinds of vendors kernel to OpenWrt’s build infrastructure (I’m currently working on Allwinner and their SDK is driving me insane to package it for Debian).
To this end, I need to interact with Incus somehow during the image building stage. With NixOS I have nixos-enter which I think is based on chroot. With Debian you can use systemd-nspawn. But the underlying blocking issue is the same which is how to interact with Incus in either of those environment.
One convoluted method I could think of would be to create a “scratch” instance with distrobuilder, make the necessary modifications to a copy of the OpenWRT definition, build it, and store it on the local Incus image store; and that this would all happen during installation/startup/boot.
We are OK with the provided Incus instance images. I’m using it at home. We don’t need to manually build it.
One backup plan is to launch the built image in VM to configure the Incus. In VM the systemd service for incusd can properly start. But this feels too heavy.
Basically just install the instance and configure the network to that instance. The host OS’s network will be configured with VLAN enabled bridges, so the external devices / OpenWrt / host OS can communicate with each other. This means OpenWrt itself must be configured as well, but this is easier when we can directly modify the rootfs.
Just doing a follow up, I did some experimenting and see no reason why it wouldn’t be possible to perform (on Debian 12 using the Zabbly repo, command may differ on different distributions): /opt/incus/lib/systemd/incusd --group incus-admin as a background task and then initialize Incus and create some instances; after doing that in a chroot, just reboot with Incus started by systemd and it should be good to go.
I did the test in a LXC Debian VM and performed the above instructions and still retained the containers I created when starting the daemon manually when booting Incus with systemd
,
