LXD is no longer starting: converting NULL to string is unsupported

I’ve been using LXD as the base for my server for years, but I’m afraid it has broken on me. The daemon isn’t running:

$ journalctl -u snap.lxd.daemon.service
<snip>
Oct 21 21:54:37 parallax.kyrofa.com systemd[1]: Started Service for snap application lxd.daemon.
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: => Preparing the system (21723)
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Loading snap configuration
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Setting up mntns symlink (mnt:[4026532245])
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Setting up kmod wrapper
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Preparing /boot
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Preparing a clean copy of /run
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Preparing /run/bin
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Preparing a clean copy of /etc
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Preparing a clean copy of /usr/share/misc
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Setting up ceph configuration
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Setting up LVM configuration
Oct 21 21:54:37 parallax.kyrofa.com lxd.daemon[5022]: ==> Rotating logs
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5022]: ==> Setting up ZFS (0.8)
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5022]: ==> Escaping the systemd cgroups
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5022]: ====> Detected cgroup V1
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5022]: ==> Escaping the systemd process resource limits
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5022]: ==> Disabling shiftfs on this kernel (auto)
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: Closed liblxcfs.so
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: Running destructor lxcfs_exit
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: Running constructor lxcfs_init to reload liblxcfs
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: mount namespace: 5
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: hierarchies:
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   0: fd:   6:
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   1: fd:   7: name=systemd
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   2: fd:   8: net_cls,net_prio
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   3: fd:   9: cpu,cpuacct
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   4: fd:  10: devices
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   5: fd:  11: rdma
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   6: fd:  12: freezer
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   7: fd:  13: perf_event
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   8: fd:  14: pids
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:   9: fd:  15: cpuset
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:  10: fd:  16: hugetlb
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:  11: fd:  17: memory
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]:  12: fd:  19: blkio
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: Kernel supports pidfds
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: Kernel does not support swap accounting
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: api_extensions:
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - cgroups
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - sys_cpu_online
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - proc_cpuinfo
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - proc_diskstats
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - proc_loadavg
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - proc_meminfo
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - proc_stat
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - proc_swaps
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - proc_uptime
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - shared_pidns
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - cpuview_daemon
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - loadavg_daemon
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: - pidfds
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[4186]: Reloaded LXCFS
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5022]: => Re-using existing LXCFS
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5022]: => Starting LXD
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5161]: t=2021-10-21T21:54:38-0700 lvl=warn msg=" - Couldn't find the CGroup blkio.weight, disk priority will be ignored"
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5161]: t=2021-10-21T21:54:38-0700 lvl=warn msg=" - Couldn't find the CGroup memory swap accounting, swap limits will be ignored"
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5161]: t=2021-10-21T21:54:38-0700 lvl=eror msg="Failed to start the daemon" err="Failed to load network \"gitlab-ci\" in project \"default\": sql: Scan error on column index 2, name \"description\": converting NULL to string is unsupported"
Oct 21 21:54:38 parallax.kyrofa.com lxd.daemon[5161]: Error: Failed to load network "gitlab-ci" in project "default": sql: Scan error on column index 2, name "description": converting NULL to string is unsupported
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5022]: => LXD failed to start
Oct 21 21:54:39 parallax.kyrofa.com systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 21 21:54:39 parallax.kyrofa.com systemd[1]: snap.lxd.daemon.service: Failed with result 'exit-code'.
Oct 21 21:54:39 parallax.kyrofa.com systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 6.
Oct 21 21:54:39 parallax.kyrofa.com systemd[1]: Stopped Service for snap application lxd.daemon.
Oct 21 21:54:39 parallax.kyrofa.com systemd[1]: Started Service for snap application lxd.daemon.
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: => Preparing the system (21723)
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Loading snap configuration
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Setting up mntns symlink (mnt:[4026532245])
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Setting up kmod wrapper
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Preparing /boot
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Preparing a clean copy of /run
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Preparing /run/bin
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Preparing a clean copy of /etc
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Preparing a clean copy of /usr/share/misc
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Setting up ceph configuration
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Setting up LVM configuration
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Rotating logs
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Setting up ZFS (0.8)
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Escaping the systemd cgroups
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ====> Detected cgroup V1
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Escaping the systemd process resource limits
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: ==> Disabling shiftfs on this kernel (auto)
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: Closed liblxcfs.so
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: Running destructor lxcfs_exit
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: Running constructor lxcfs_init to reload liblxcfs
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: mount namespace: 5
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: hierarchies:
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   0: fd:   6:
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   1: fd:   7: name=systemd
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   2: fd:   8: net_cls,net_prio
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   3: fd:   9: cpu,cpuacct
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   4: fd:  10: devices
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   5: fd:  11: rdma
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   6: fd:  12: freezer
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   7: fd:  13: perf_event
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   8: fd:  14: pids
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:   9: fd:  15: cpuset
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:  10: fd:  16: hugetlb
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:  11: fd:  17: memory
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]:  12: fd:  19: blkio
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: Kernel supports pidfds
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: Kernel does not support swap accounting
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: api_extensions:
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - cgroups
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - sys_cpu_online
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - proc_cpuinfo
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - proc_diskstats
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - proc_loadavg
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - proc_meminfo
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - proc_stat
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - proc_swaps
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - proc_uptime
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - shared_pidns
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - cpuview_daemon
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - loadavg_daemon
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: - pidfds
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[4186]: Reloaded LXCFS
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: => Re-using existing LXCFS
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5209]: => Starting LXD
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5348]: t=2021-10-21T21:54:39-0700 lvl=warn msg=" - Couldn't find the CGroup blkio.weight, disk priority will be ignored"
Oct 21 21:54:39 parallax.kyrofa.com lxd.daemon[5348]: t=2021-10-21T21:54:39-0700 lvl=warn msg=" - Couldn't find the CGroup memory swap accounting, swap limits will be ignored"
Oct 21 21:54:40 parallax.kyrofa.com lxd.daemon[5348]: t=2021-10-21T21:54:40-0700 lvl=eror msg="Failed to start the daemon" err="Failed to load network \"gitlab-ci\" in project \"default\": sql: Scan error on column index 2, name \"description\": converting NULL to string is unsupported"
Oct 21 21:54:40 parallax.kyrofa.com lxd.daemon[5348]: Error: Failed to load network "gitlab-ci" in project "default": sql: Scan error on column index 2, name "description": converting NULL to string is unsupported
Oct 21 21:54:40 parallax.kyrofa.com lxd.daemon[5209]: => LXD failed to start
Oct 21 21:54:40 parallax.kyrofa.com systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
Oct 21 21:54:40 parallax.kyrofa.com systemd[1]: snap.lxd.daemon.service: Failed with result 'exit-code'.
Oct 21 21:54:41 parallax.kyrofa.com systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 7.
Oct 21 21:54:41 parallax.kyrofa.com systemd[1]: Stopped Service for snap application lxd.daemon.
Oct 21 21:54:41 parallax.kyrofa.com systemd[1]: snap.lxd.daemon.service: Start request repeated too quickly.
Oct 21 21:54:41 parallax.kyrofa.com systemd[1]: snap.lxd.daemon.service: Failed with result 'exit-code'.
Oct 21 21:54:41 parallax.kyrofa.com systemd[1]: Failed to start Service for snap application lxd.daemon.
$ snap list lxd
Name  Version  Rev    Tracking       Publisher   Notes
lxd   4.19     21723  latest/stable  canonical✓  -

I’m afraid that error isn’t quite enough for me to know where to look. I didn’t change anything recently. Any advice would be appreciated! Would be handy to have my server back.

This has already been fixed but there is also a workaround here:

1 Like

Thank you @tomp, I’m back up and running now.

2 Likes