My lxd setup was running very well for the last year but last week stopped working after apt upgrade.
When I try to start a container, it says “Required tool ‘zpool’ is missing”, but manually running zpool command is absolutely OK.
Ubuntu Kernel: 6.5.0-14-generic #14~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC
zfs kernel module version: 2.2.0-0ubuntu1~23.10
LXD version: 5.0.2
zpool version: zfs-2.1.5-1ubuntu6~22.04.2
Does anyone have idea what goes wrong here?
I’m sorry but given the recent actions from Canonical regarding LXD:
We really can’t be providing support to LXD users on this forum anymore.
You may want to consider switching to Incus instead, or if you’d like to stay on LXD, you should reach out on the Canonical forum instead.
Sorry about that!
On this one, my guess would be that the LXD 5.0.x snap is lacking the ZFS 2.2 tools, so your kernel version of ZFS is more recent than what the snap can handle.
@tomp should be able to confirm/fix.
Yes indeed that is the issue, we’ve landed a fix in 5.0/edge channel here as it was affecting Jammy HWE kernel users that recently got upgraded to 6.5.
We will include this in the LXD 5.0.3 release.
You can confirm the fix by doing sudo snap refresh --channel=5.0/edge
Oh, this is the first time I heard about Incus. I will definitely give it a try!
Confirmed. After running
sudo snap refresh --channel=5.0/edge lxd
It is working now! Thanks a lot!
Do you know when LXD 5.0.3 will be released?
Worked for me too! Thanks for creating the thread!
We are planning a pre-release cherry-pick to fix just this issue soon. Will advise when it occurs here.
There is a fixed revision in 5.0/candidate channel now. Will be pushing to 5.0/stable soon.
Thanks a lot for letting us know. Looking forward to the stable version!
We are planning to push this to 5.0/stable on Monday.
LXD 5.0.2 with support for ZFS 2.2 is now being rolled out progressively to the 5.0/stable snap channel.
I was just bitten by the opposite problem:
root@db:/home/brian# ls /snap/lxd/27049
bin commands criu etc lib lxc lxcfs meta share snap usr wrappers zfs-0.8 zfs-2.0 zfs-2.1 zfs-2.2
root@db:/home/brian# ls /snap/lxd/28155
bin commands criu etc lib lxc lxcfs meta share snap wrappers zfs-2.1 zfs-2.2
All hell has broken lose. Roll back the snap then?
snap refresh lxd --channel=5.20/stable
No: the muppets have also ensured that a rollback is impossible.
# systemctl stop snap.lxd.daemon.unix.socket
# lxd --debug --group lxd
...
time="2024-04-13T08:38:42Z" level=info msg="Daemon stopped"
Error: Failed to initialize global database: failed to ensure schema: schema version '73' is more recent than expected '69'
It seems going back to 5.21 plus HWE kernel upgrade is the only option - followed by migration to incus.
And I’m never touching a system with snapd again, except to remove it.
EDIT: It looks like they realised their mistake and released 5.21.1 with zfs-0.8 support: LXD 5.21.1 LTS has been released - News - Ubuntu Community Hub
(and turned 5.21 into an LTS release)