Lxd init --preseed < lxd.seed fails in untended curtin installation

On Ubuntu 20.04 using MAAS + curtin + ansible to deploy servers, the following command fails only when in this setup.

# ensure lxd snap package is fixed to 4.17/stable
snap refresh --channel=4.17 lxd
cat << EOF > /etc/lxd.seed
config: {}
cluster: null
networks:
- config:
    ipv4.address: auto
    ipv6.address: none
  description: ""
  managed: false
  name: lxdbr0
  type: ""
storage_pools:
- config:
    size: 300GB
  description: ""
  name: default
  driver: zfs
profiles:
- config: {}
  description: ""
  devices:
    eth0:
      name: eth0
      nictype: bridged
      parent: lxdbr0
      type: nic
    root:
      path: /
      pool: default
      type: disk
  name: default
EOF

# following command executed via ansible-pull
/snap/bin/lxd init --preseed < /etc/lxd.seed

results in:

internal error, please report: running \"lxd\" failed: cannot find installed snap \"lxd\" at revision 21029: missing file /snap/lxd/21029/meta/snap.yaml

full Ansible shell output:

/snap/bin/lxd init --preseed < /etc/lxd.seed && touch /etc/lxd.init", "delta": "0:00:00.033602", "end": "2021-08-16 13:21:35.655923", "msg": "non-zero return code", "rc": 46, "start": "2021-08-16 13:21:35.622321", "stderr": "internal error, please report: running \"lxd\" failed: cannot find installed snap \"lxd\" at revision 21029: missing file /snap/lxd/21029/meta/snap.yaml", "stderr_lines": ["internal error, please report: running \"lxd\" failed: cannot find installed snap \"lxd\" at revision 21029: missing file /snap/lxd/21029/meta/snap.yaml"

when login for debugging the server, the snap refresh did work fine as I get the proper version:

lxd --version
4.17

Is there something which prevents from lxd init --preseed from a chroot during MAAS curtin installation? shall one use lxd init --auto instead? are there some conflicts with cloud-init and installing LXD as late curtin command?

That error is coming from snapd and suggests that the snap isn’t properly functioning, indeed most likely caused because of you interacting with it from within a chroot.

snapd relies on systemd to handle its mounts, so in this case, I’d guess /snap/lxd/… isn’t mounted, causing the error.

I’m not familiar with this use of cloud-init and curtin, but maybe there’s a way for it to install and configure LXD on first boot instead of through a chroot during the installation?

As an alternative you could probably script your way around that lack of systemd by locating the right snap squashfs in /var/lib/snapd and manually mounting that over /snap/lxd/REVISION, then unmounting it after doing the configuration, but that feels very hackish.

Thanks for the hints, postponing the lxd init --preseed with the following cloud-config in /etc/cloud/cloud.cfg.d/99_lxd-init.cfg did the trick.

#cloud-config
runcmd:
 - '/snap/bin/lxd init --preseed < /etc/lxd.seed'

I also checked cloud-init LXD module, yet it does not support storage pool size so rely on this runcmd: for the time being.