I have a fairly new installation of Ubuntu 20.04 desktop, and been trying to get a basic LXD + Juju setup going with a network bridge that can allow me to ping containers… but noticed I am getting frozen on this step
should I snap uninstall and reinstall? I tried that once, but maybe I need to do it again ? thanks
Can you do:
sudo snap set lxd daemon.debug=true
sudo systemctl reload snap.lxd.daemon
Then try again and show output from:
journalctl -u snap.lxd.daemon -n 300
sure thing
(base) emcp@2600k:~$ sudo snap set lxd daemon.debug=true
[sudo] password for emcp:
(base) emcp@2600k:~$ sudo systemctl reload snap.lxd.daemon
(base) emcp@2600k:~$ lxc launch ubuntu:20.04
Creating the instance
Error: Failed instance creation: Create instance from image: Failed to run: zfs clone default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56@readonly default/containers/viable-aphid: cannot open 'default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56@readonly': dataset does not exist
journalctl -u snap.lxd.daemon -n 300 yields the following
maj 19 11:01:53 2600k systemd[1]: Reloading Service for snap application lxd.daemon.
maj 19 11:01:53 2600k systemd[1]: Reloaded Service for snap application lxd.daemon.
maj 19 11:01:54 2600k lxd.daemon[1225]: => LXD is reloading
maj 19 11:01:54 2600k systemd[1]: snap.lxd.daemon.service: Main process exited, code=exited, status=1/FAILURE
maj 19 11:01:54 2600k systemd[1]: snap.lxd.daemon.service: Failed with result 'exit-code'.
maj 19 11:01:54 2600k systemd[1]: snap.lxd.daemon.service: Scheduled restart job, restart counter is at 1.
maj 19 11:01:54 2600k systemd[1]: Stopped Service for snap application lxd.daemon.
maj 19 11:01:54 2600k systemd[1]: Started Service for snap application lxd.daemon.
maj 19 11:01:54 2600k lxd.daemon[19722]: => Preparing the system (15067)
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Loading snap configuration
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Setting up mntns symlink (mnt:[4026532559])
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Setting up kmod wrapper
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Preparing /boot
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Preparing a clean copy of /run
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Preparing a clean copy of /etc
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Preparing a clean copy of /usr/share/misc
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Setting up ceph configuration
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Setting up LVM configuration
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Rotating logs
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Setting up ZFS (0.8)
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Escaping the systemd cgroups
maj 19 11:01:54 2600k lxd.daemon[19722]: ====> Detected cgroup V1
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Escaping the systemd process resource limits
maj 19 11:01:54 2600k lxd.daemon[19722]: ==> Disabling shiftfs on this kernel (auto)
maj 19 11:01:54 2600k lxd.daemon[19722]: => Re-using existing LXCFS
maj 19 11:01:54 2600k lxd.daemon[19722]: => Starting LXD
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="LXD 4.1 is starting in normal mode" path=/var/snap/lxd/common/lxd
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Kernel uid/gid map:"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - u 0 0 4294967295"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - g 0 0 4294967295"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Configured LXD uid/gid map:"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - u 0 1000000 1000000000"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - g 0 1000000 1000000000"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Kernel features:"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - netnsid-based network retrieval: yes"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - uevent injection: yes"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - seccomp listener: yes"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - seccomp listener continue syscalls: yes"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - unprivileged file capabilities: yes"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - cgroup layout: hybrid"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=warn msg=" - Couldn't find the CGroup blkio.weight, I/O weight limits will be ignored"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=warn msg=" - Couldn't find the CGroup memory swap accounting, swap limits will be ignored"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - shiftfs support: disabled"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Initializing local database"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Initializing database gateway"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Start database node" address= id=1 role=voter
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Starting /dev/lxd handler:"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - binding devlxd socket" socket=/var/snap/lxd/common/lxd/devlxd/sock
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="REST API daemon:"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - binding Unix socket" inherited=true socket=/var/snap/lxd/common/lxd/unix.socket
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg=" - binding TCP socket" socket=[::]:8443
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Initializing global database"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Dqlite: connected address=1 attempt=0"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Firewall xtables detected iptables is in use"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Firewall loaded driver \"xtables\""
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Initializing storage pools"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Initializing and checking storage pool \"default\""
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Mount started" driver=zfs pool=default
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Mount finished" driver=zfs pool=default
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Initializing daemon storage mounts"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Initializing networks"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="New task Operation: 52984424-6c56-47eb-bdbf-2a72849fa3ab"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Pruning leftover image files"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Started task operation: 52984424-6c56-47eb-bdbf-2a72849fa3ab"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Done pruning leftover image files"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Loading daemon configuration"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Success for task operation: 52984424-6c56-47eb-bdbf-2a72849fa3ab"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Initialized inotify with file descriptor 3"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Started seccomp handler" path=/var/snap/lxd/common/lxd/seccomp.socket
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0 user=
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="New task Operation: 29fb9360-9ae4-4a51-b703-deb85aff524d"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Pruning expired images"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Started task operation: 29fb9360-9ae4-4a51-b703-deb85aff524d"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Done pruning expired images"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Success for task operation: 29fb9360-9ae4-4a51-b703-deb85aff524d"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg=Handling ip=@ method=GET url=/internal/ready user=
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="New task Operation: a5a72f28-9bac-4b76-bb91-715dfd6d2fc6"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Pruning expired instance backups"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Started task operation: a5a72f28-9bac-4b76-bb91-715dfd6d2fc6"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Done pruning expired instance backups"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Success for task operation: a5a72f28-9bac-4b76-bb91-715dfd6d2fc6"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="New task Operation: b4c1bad9-64fb-415e-bdc5-2eaa9dd79234"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="New task Operation: 41fe0edd-8edd-4cef-bb87-4137a759d061"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Expiring log files"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Started task operation: 41fe0edd-8edd-4cef-bb87-4137a759d061"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Updating instance types"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Started task operation: b4c1bad9-64fb-415e-bdc5-2eaa9dd79234"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Done expiring log files"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Container idmap changed, remapping"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="New task Operation: d6b379f6-0c00-4bf1-9be3-82ecb32b207e"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Updating images"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Started task operation: d6b379f6-0c00-4bf1-9be3-82ecb32b207e"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Success for task operation: 41fe0edd-8edd-4cef-bb87-4137a759d061"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Done updating images"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=info msg="Done updating instance types"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="MountInstance started" driver=zfs instance=juju-34eb35-0 pool=default project=default
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Processing image" alias=20.04 fp=89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56 protocol=simplestreams server=https://cloud-images.ubuntu.com/releases
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="Connecting to a remote simplestreams server"
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=dbug msg="MountInstance finished" driver=zfs instance=juju-34eb35-0 pool=default project=default
maj 19 11:01:54 2600k lxd.daemon[19825]: t=2020-05-19T11:01:54+0200 lvl=eror msg="Failed to start instance 'juju-34eb35-0': Common start logic: Storage start: Failed to run: zfs mount default/containers/juju-34eb35-0: cannot open 'default/containers/juju-34eb35->
maj 19 11:01:55 2600k lxd.daemon[19825]: t=2020-05-19T11:01:55+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0 user=
maj 19 11:01:55 2600k lxd.daemon[19825]: t=2020-05-19T11:01:55+0200 lvl=dbug msg=Handling ip=@ method=GET url=/internal/ready user=
maj 19 11:01:55 2600k lxd.daemon[19722]: => LXD is ready
maj 19 11:01:57 2600k lxd.daemon[19825]: t=2020-05-19T11:01:57+0200 lvl=dbug msg="Image already exists in the db" image=89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56
maj 19 11:01:57 2600k lxd.daemon[19825]: t=2020-05-19T11:01:57+0200 lvl=dbug msg="Already up to date" fp=89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56
maj 19 11:01:57 2600k lxd.daemon[19825]: t=2020-05-19T11:01:57+0200 lvl=dbug msg="Success for task operation: d6b379f6-0c00-4bf1-9be3-82ecb32b207e"
maj 19 11:01:57 2600k lxd.daemon[19825]: t=2020-05-19T11:01:57+0200 lvl=dbug msg="Success for task operation: b4c1bad9-64fb-415e-bdc5-2eaa9dd79234"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0 user=
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0/events user=
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="New event listener: 2107df73-38d2-4cf5-9288-794043e188d0"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg=Handling ip=@ method=POST url=/1.0/instances user=
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="\n\t{\n\t\t\"architecture\": \"\",\n\t\t\"config\": {},\n\t\t\"devices\": {},\n\t\t\"ephemeral\": false,\n\t\t\"profiles\": null,\n\t\t\"stateful\": false,\n\t\t\"description\": \">
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Responding to instance create"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Connecting to a remote simplestreams server"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="No name provided, creating viable-aphid"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="New task Operation: e025d35d-2cf4-4411-897b-d294cdd4954d"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Started task operation: e025d35d-2cf4-4411-897b-d294cdd4954d"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Connecting to a remote simplestreams server"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="\n\t{\n\t\t\"type\": \"async\",\n\t\t\"status\": \"Operation created\",\n\t\t\"status_code\": 100,\n\t\t\"operation\": \"/1.0/operations/e025d35d-2cf4-4411-897b-d294cdd4954d\",\n\t>
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0/operations/e025d35d-2cf4-4411-897b-d294cdd4954d user=
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Image already exists in the db" image=89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=info msg="Creating container" ephemeral=false name=viable-aphid project=default
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=info msg="Created container" ephemeral=false name=viable-aphid project=default
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="CreateInstanceFromImage started" driver=zfs instance=viable-aphid pool=default project=default
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="EnsureImage started" driver=zfs fingerprint=89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56 pool=default
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Database error: &errors.errorString{s:\"No such object\"}"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="EnsureImage finished" driver=zfs fingerprint=89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56 pool=default
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="DeleteInstance started" driver=zfs instance=viable-aphid pool=default project=default
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Deleting instance volume" driver=zfs instance=viable-aphid pool=default project=default volName=viable-aphid
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="DeleteInstance finished" driver=zfs instance=viable-aphid pool=default project=default
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="CreateInstanceFromImage finished" driver=zfs instance=viable-aphid pool=default project=default
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=info msg="Deleting container" created=2020-05-19T11:02:16+0200 ephemeral=false name=viable-aphid project=default used=1970-01-01T01:00:00+0100
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Database error: &errors.errorString{s:\"No such object\"}"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=info msg="Deleted container" created=2020-05-19T11:02:16+0200 ephemeral=false name=viable-aphid project=default used=1970-01-01T01:00:00+0100
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Failure for task operation: e025d35d-2cf4-4411-897b-d294cdd4954d: Create instance from image: Failed to run: zfs clone default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df>
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Event listener finished: 2107df73-38d2-4cf5-9288-794043e188d0"
maj 19 11:02:16 2600k lxd.daemon[19825]: t=2020-05-19T11:02:16+0200 lvl=dbug msg="Disconnected event listener: 2107df73-38d2-4cf5-9288-794043e188d0"
seems I need to wipe a cache or reinstall?
If you do lxc image ls and check that image 89cbdbacd37e484c16816ae1ad550930c70320ef6428df is ok by you to be deleted then you can try doing lxc image delete 89cbdbacd37e484c16816ae1ad550930c70320ef6428df and see if that helps to allow you to regenerate the cached image volume on zfs.
thanks @tomp
I’m on a new installation so I will just erase everything, including that juju image… since it’s not working.
after erasing the one mentioned, still getting an error
(base) emcp@2600k:~$ lxc image ls
+-------------------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCHITECTURE | TYPE | SIZE | UPLOAD DATE |
+-------------------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
| juju/bionic/amd64 | 93b9eeb85479 | no | ubuntu 18.04 LTS amd64 (release) (20200506) | x86_64 | CONTAINER | 179.10MB | May 17, 2020 at 6:18pm (UTC) |
+-------------------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
| | 89cbdbacd37e | no | ubuntu 20.04 LTS amd64 (release) (20200513) | x86_64 | CONTAINER | 345.92MB | May 19, 2020 at 5:15am (UTC) |
+-------------------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
(base) emcp@2600k:~$ lxc image delete 89cbdbacd37e
(base) emcp@2600k:~$ lxc image ls
+-------------------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCHITECTURE | TYPE | SIZE | UPLOAD DATE |
+-------------------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
| juju/bionic/amd64 | 93b9eeb85479 | no | ubuntu 18.04 LTS amd64 (release) (20200506) | x86_64 | CONTAINER | 179.10MB | May 17, 2020 at 6:18pm (UTC) |
+-------------------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
(base) emcp@2600k:~$ lxc launch ubuntu:20.04
Creating the instance
Error: Failed instance creation: Create instance from image: Failed to run: zfs clone default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56@readonly default/containers/evolved-jennet: cannot open 'default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56@readonly': dataset does not exist
seems 18.04 image also is affected
(base) emcp@2600k:~$ lxc launch ubuntu:20.04
Creating the instance
Error: Failed instance creation: Create instance from image: Failed to run: zfs clone default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56@readonly default/containers/welcomed-sole: cannot open 'default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56@readonly': dataset does not exist
(base) emcp@2600k:~$ lxc list
+----------------+---------+------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+----------------+---------+------+------+-----------+-----------+
| informed-akita | STOPPED | | | CONTAINER | 0 |
+----------------+---------+------+------+-----------+-----------+
| juju-34eb35-0 | STOPPED | | | CONTAINER | 0 |
+----------------+---------+------+------+-----------+-----------+
(base) emcp@2600k:~$ lxc image ls
+-------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCHITECTURE | TYPE | SIZE | UPLOAD DATE |
+-------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
| | 89cbdbacd37e | no | ubuntu 20.04 LTS amd64 (release) (20200513) | x86_64 | CONTAINER | 345.92MB | May 19, 2020 at 9:36am (UTC) |
+-------+--------------+--------+---------------------------------------------+--------------+-----------+----------+------------------------------+
(base) emcp@2600k:~$ lxc launch ubuntu:18.04
Creating the instance
Error: Failed instance creation: Create instance from image: Failed to run: zfs clone default/images/93b9eeb85479af2029203b4a56a2f1fdca6a0e1bf23cdc26b567790bf0f3f3bd@readonly default/containers/precious-sawfly: cannot open 'default/images/93b9eeb85479af2029203b4a56a2f1fdca6a0e1bf23cdc26b567790bf0f3f3bd@readonly': dataset does not exist
(base) emcp@2600k:~$
@tomp I also had this, I had to manually delete the entry from the database with lxd sql
thank you @turtle0x1 can you be a little more specific what you did… I am very hesitant to just start playing around with the SQL DB… did you go drop a table or remove a particular row ?
thank you
It may not be the same issue on closer inspection,
I had a very similar issue where the database kept hanging on to an image reference no matter how many times I did lxc delete fingerprint and I couldn’t launch any containers with the image because I got the same error dataset does not exist eventually I removed the row from the database using lxc sql which allowed the image to re-download and it worked fine
my system was non production though - id wait
my system is not production per se either… I may elect to just uninstall and startup over then…
I’ve ended up reinstalling the entire LTS… will cross my toes I can get it going now. thank you those who responded
I reinstalled the entire OS, performed a snap install… setup defaults on everything in init except ipv6 (so I can use it with juju) and still it is freezing on this step
(base) emcp@2600k:~$ lxc launch ubuntu:20.04
Creating the instance
Retrieving image: Unpack: 100% (2.90GB/s)
I am beginning to wonder if I should just back down to 18.04 LTS to get a working system for now
I tried sshing again into another session to perform lxc list and found it frozen as well
EDIT:
After waiting and waiting… my original command of launching turned out to be frozen due to a timed out connection over SSH…
(base) emcp@2600k:~$ lxc launch ubuntu:20.04
Creating the instance
Retrieving image: Unpack: 100% (2.90GB/s)
client_loop: send disconnect: Broken pipe
Logged back in and took a peak with your requested commands
(base) emcp@2600k:~$ sudo lxc list
To start your first instance, try: lxc launch ubuntu:18.04
+------------+---------+------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------------+---------+------+------+-----------+-----------+
| fleet-hare | STOPPED | | | CONTAINER | 0 |
+------------+---------+------+------+-----------+-----------+
(base) emcp@2600k:~$ lxc start fleet-hare
Error: Common start logic: Storage start: Failed to run: zfs mount default/containers/fleet-hare: cannot open 'default/containers/fleet-hare': dataset does not exist
Try `lxc info --show-log fleet-hare` for more info
(base) emcp@2600k:~$ sudo snap set lxd daemon.debug=true
(base) emcp@2600k:~$ sudo systemctl reload snap.lxd.daemon
(base) emcp@2600k:~$ sudo lxc list
+------------+---------+------+------+-----------+-----------+
| NAME | STATE | IPV4 | IPV6 | TYPE | SNAPSHOTS |
+------------+---------+------+------+-----------+-----------+
| fleet-hare | STOPPED | | | CONTAINER | 0 |
+------------+---------+------+------+-----------+-----------+
(base) emcp@2600k:~$ lxc start fleet-hare
Error: Common start logic: Storage start: Failed to run: zfs mount default/containers/fleet-hare: cannot open 'default/containers/fleet-hare': dataset does not exist
Try `lxc info --show-log fleet-hare` for more info
(base) emcp@2600k:~$ lxc info --show-log fleet-hare
Name: fleet-hare
Location: none
Remote: unix://
Architecture: x86_64
Created: 2020/05/20 17:50 UTC
Status: Stopped
Type: container
Profiles: default
Log:
lxc 20200520180735.214 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
lxc 20200520180735.238 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
lxc 20200520180743.959 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
lxc 20200520180743.959 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
lxc 20200520180743.959 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
lxc 20200520180746.812 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
lxc 20200520180746.821 INFO confile - confile.c:set_config_idmaps:2012 - Read uid map: type u nsid 0 hostid 1000000 range 1000000000
lxc 20200520180746.821 INFO confile - confile.c:set_config_idmaps:2012 - Read uid map: type g nsid 0 hostid 1000000 range 1000000000
lxc 20200520180746.821 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
lxc 20200520180757.811 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
lxc 20200520180757.814 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
lxc 20200520180757.814 TRACE commands - commands.c:lxc_cmd:286 - Connection refused - Command "get_state" failed to connect command socket
(base) emcp@2600k:~$
Edit 2: On the plus side, I got an 18.04 image to start!
(base) emcp@2600k:~$ lxc launch ubuntu:18.04
Creating the instance
Instance name is: neat-killdeer
Starting neat-killdeer
(base) emcp@2600k:~$
It would be good to see where its failing by gathering output of journalctl -u snap.lxd.daemon -n 300 when it fails or is slow.
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=dbug msg="Updated metadata for task Operation: 14483e82-71ba-4210-9ae4-d2ecdffc1a06"
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=dbug msg="Updated metadata for task Operation: 14483e82-71ba-4210-9ae4-d2ecdffc1a06"
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=dbug msg="Updated metadata for task Operation: 14483e82-71ba-4210-9ae4-d2ecdffc1a06"
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=dbug msg="Updated metadata for task Operation: 14483e82-71ba-4210-9ae4-d2ecdffc1a06"
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=info msg="Image downloaded" alias=18.04 image=5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce operation=14483e82-71ba-4210-9ae4-d2ecdffc1a06 server=https://cloud-images.ubuntu.com/releases trigger=/1.0/operations/14483e82-71ba-4210-9ae4-d2ecdffc1a06
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=info msg="Creating container" ephemeral=false name=neat-killdeer project=default
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=info msg="Created container" ephemeral=false name=neat-killdeer project=default
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=dbug msg="CreateInstanceFromImage started" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=dbug msg="EnsureImage started" driver=zfs fingerprint=5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce pool=default
maj 20 20:15:20 2600k lxd.daemon[2879]: t=2020-05-20T20:15:20+0200 lvl=dbug msg="Database error: &errors.errorString{s:\"No such object\"}"
maj 20 20:15:21 2600k lxd.daemon[2879]: t=2020-05-20T20:15:21+0200 lvl=dbug msg="Mounted ZFS dataset" dev=default/images/5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce driver=zfs path=/var/snap/lxd/common/lxd/storage-pools/default/images/5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce pool=default
maj 20 20:15:21 2600k lxd.daemon[2879]: t=2020-05-20T20:15:21+0200 lvl=dbug msg="Updated metadata for task Operation: 14483e82-71ba-4210-9ae4-d2ecdffc1a06"
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Unmounted ZFS dataset" dev=default/images/5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce driver=zfs path=/var/snap/lxd/common/lxd/storage-pools/default/images/5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce pool=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Database error: &errors.errorString{s:\"No such object\"}"
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="EnsureImage finished" driver=zfs fingerprint=5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce pool=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Mounted ZFS dataset" dev=default/containers/neat-killdeer driver=zfs path=/var/snap/lxd/common/lxd/storage-pools/default/containers/neat-killdeer pool=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Unmounted ZFS dataset" dev=default/containers/neat-killdeer driver=zfs path=/var/snap/lxd/common/lxd/storage-pools/default/containers/neat-killdeer pool=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="CreateInstanceFromImage finished" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="UpdateInstanceBackupFile started" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Mounted ZFS dataset" dev=default/containers/neat-killdeer driver=zfs path=/var/snap/lxd/common/lxd/storage-pools/default/containers/neat-killdeer pool=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Unmounted ZFS dataset" dev=default/containers/neat-killdeer driver=zfs path=/var/snap/lxd/common/lxd/storage-pools/default/containers/neat-killdeer pool=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="UpdateInstanceBackupFile finished" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Success for task operation: 14483e82-71ba-4210-9ae4-d2ecdffc1a06"
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0/instances/neat-killdeer user=
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg=Handling ip=@ method=PUT url=/1.0/instances/neat-killdeer/state user=
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="\n\t{\n\t\t\"action\": \"start\",\n\t\t\"timeout\": -1,\n\t\t\"force\": false,\n\t\t\"stateful\": false\n\t}"
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="New task Operation: 8cc3b0a7-39ef-4a7c-8d3a-581de95291dc"
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Started task operation: 8cc3b0a7-39ef-4a7c-8d3a-581de95291dc"
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="\n\t{\n\t\t\"type\": \"async\",\n\t\t\"status\": \"Operation created\",\n\t\t\"status_code\": 100,\n\t\t\"operation\": \"/1.0/operations/8cc3b0a7-39ef-4a7c-8d3a-581de95291dc\",\n\t\t\"error_code\": 0,\n\t\t\"error\": \"\",\n\t\t\"metadata\": {\n\t\t\t\"id\": \"8cc3b0a7-39ef-4a7c->
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Container idmap changed, remapping"
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Updated metadata for task Operation: 8cc3b0a7-39ef-4a7c-8d3a-581de95291dc"
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0/operations/8cc3b0a7-39ef-4a7c-8d3a-581de95291dc user=
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="MountInstance started" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="Mounted ZFS dataset" dev=default/containers/neat-killdeer driver=zfs path=/var/snap/lxd/common/lxd/storage-pools/default/containers/neat-killdeer pool=default
maj 20 20:15:25 2600k lxd.daemon[2879]: t=2020-05-20T20:15:25+0200 lvl=dbug msg="MountInstance finished" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:27 2600k lxd.daemon[2879]: t=2020-05-20T20:15:27+0200 lvl=dbug msg="Updated metadata for task Operation: 8cc3b0a7-39ef-4a7c-8d3a-581de95291dc"
maj 20 20:15:27 2600k lxd.daemon[2879]: t=2020-05-20T20:15:27+0200 lvl=dbug msg="Scheduler: network: vethc843d54b has been added: updating network priorities"
maj 20 20:15:27 2600k lxd.daemon[2879]: t=2020-05-20T20:15:27+0200 lvl=dbug msg="Scheduler: network: vethce7bccfe has been added: updating network priorities"
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg="MountInstance started" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg="MountInstance finished" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg="UpdateInstanceBackupFile started" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg="UpdateInstanceBackupFile finished" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg="MountInstance started" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg="MountInstance finished" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=info msg="Starting container" action=start created=2020-05-20T20:15:20+0200 ephemeral=false name=neat-killdeer project=default stateful=false used=1970-01-01T01:00:00+0100
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg=Handling ip=@ method=GET url=/internal/containers/2/onstart user=
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg="MountInstance started" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg="MountInstance finished" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:15:28 2600k lxd.daemon[2879]: t=2020-05-20T20:15:28+0200 lvl=dbug msg="Scheduler: container neat-killdeer started: re-balancing"
maj 20 20:15:28 2600k lxd.daemon[2019]: Closed liblxcfs.so
maj 20 20:15:28 2600k lxd.daemon[2019]: Running destructor lxcfs_exit
maj 20 20:15:28 2600k lxd.daemon[2019]: Running constructor lxcfs_init to reload liblxcfs
maj 20 20:15:28 2600k lxd.daemon[2019]: mount namespace: 5
maj 20 20:15:28 2600k lxd.daemon[2019]: hierarchies:
maj 20 20:15:28 2600k lxd.daemon[2019]: 0: fd: 6:
maj 20 20:15:28 2600k lxd.daemon[2019]: 1: fd: 7: name=systemd
maj 20 20:15:28 2600k lxd.daemon[2019]: 2: fd: 8: cpu,cpuacct
maj 20 20:15:28 2600k lxd.daemon[2019]: 3: fd: 9: freezer
maj 20 20:15:28 2600k lxd.daemon[2019]: 4: fd: 10: net_cls,net_prio
maj 20 20:15:28 2600k lxd.daemon[2019]: 5: fd: 11: pids
maj 20 20:15:28 2600k lxd.daemon[2019]: 6: fd: 12: memory
maj 20 20:15:28 2600k lxd.daemon[2019]: 7: fd: 13: perf_event
maj 20 20:15:28 2600k lxd.daemon[2019]: 8: fd: 14: hugetlb
maj 20 20:15:28 2600k lxd.daemon[2019]: 9: fd: 15: blkio
maj 20 20:15:28 2600k lxd.daemon[2019]: 10: fd: 16: cpuset
maj 20 20:15:28 2600k lxd.daemon[2019]: 11: fd: 17: devices
maj 20 20:15:28 2600k lxd.daemon[2019]: 12: fd: 19: rdma
maj 20 20:15:28 2600k lxd.daemon[2019]: Kernel supports pidfds
maj 20 20:15:28 2600k lxd.daemon[2019]: api_extensions:
maj 20 20:15:28 2600k lxd.daemon[2019]: - cgroups
maj 20 20:15:28 2600k lxd.daemon[2019]: - sys_cpu_online
maj 20 20:15:28 2600k lxd.daemon[2019]: - proc_cpuinfo
maj 20 20:15:28 2600k lxd.daemon[2019]: - proc_diskstats
maj 20 20:15:28 2600k lxd.daemon[2019]: - proc_loadavg
maj 20 20:15:28 2600k lxd.daemon[2019]: - proc_meminfo
maj 20 20:15:28 2600k lxd.daemon[2019]: - proc_stat
maj 20 20:15:28 2600k lxd.daemon[2019]: - proc_swaps
EDIT: since that last log was maybe cluttered, I repeated the start command and scrolled to latest timestamp
maj 20 20:55:54 2600k lxd.daemon[2879]: t=2020-05-20T20:55:54+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0 user=
maj 20 20:55:54 2600k lxd.daemon[2879]: t=2020-05-20T20:55:54+0200 lvl=dbug msg=Handling ip=@ method=GET url="/1.0/instances?recursion=2" user=
maj 20 20:55:54 2600k lxd.daemon[2879]: t=2020-05-20T20:55:54+0200 lvl=dbug msg="GetInstanceUsage started" driver=zfs instance=fleet-hare pool=default project=default
maj 20 20:55:54 2600k lxd.daemon[2879]: t=2020-05-20T20:55:54+0200 lvl=dbug msg="GetInstanceUsage finished" driver=zfs instance=fleet-hare pool=default project=default
maj 20 20:55:54 2600k lxd.daemon[2879]: t=2020-05-20T20:55:54+0200 lvl=eror msg="Error getting disk usage" err="Failed to run: zfs get -H -p -o value used default/containers/fleet-hare: cannot open 'default/containers/fleet-hare': dataset does not exist" instance=fleet-hare project=default
maj 20 20:55:54 2600k lxd.daemon[2879]: t=2020-05-20T20:55:54+0200 lvl=dbug msg="GetInstanceUsage started" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:55:54 2600k lxd.daemon[2879]: t=2020-05-20T20:55:54+0200 lvl=dbug msg="GetInstanceUsage started" driver=zfs instance=flask-app pool=default project=default
maj 20 20:55:54 2600k lxd.daemon[2879]: t=2020-05-20T20:55:54+0200 lvl=dbug msg="GetInstanceUsage finished" driver=zfs instance=neat-killdeer pool=default project=default
maj 20 20:55:54 2600k lxd.daemon[2879]: t=2020-05-20T20:55:54+0200 lvl=dbug msg="GetInstanceUsage finished" driver=zfs instance=flask-app pool=default project=default
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0 user=
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0/instances/fleet-hare user=
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0/events user=
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="New event listener: 83b0c4a1-e172-4f16-a01b-320f6b0f64ca"
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg=Handling ip=@ method=PUT url=/1.0/instances/fleet-hare/state user=
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="\n\t{\n\t\t\"action\": \"start\",\n\t\t\"timeout\": 0,\n\t\t\"force\": false,\n\t\t\"stateful\": false\n\t}"
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="New task Operation: 3e1d6177-3a75-4565-bb64-066480baf212"
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="Started task operation: 3e1d6177-3a75-4565-bb64-066480baf212"
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="\n\t{\n\t\t\"type\": \"async\",\n\t\t\"status\": \"Operation created\",\n\t\t\"status_code\": 100,\n\t\t\"operation\": \"/1.0/operations/3e1d6177-3a75-4565-bb64-066480baf212\",\n\t\t\"error_code\": 0,\n\t\t\"error\": \"\",\n\t\t\"metadata\": {\n\t\t\t\"id\": \"3e1d6177-3a75-4565->
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="Container idmap changed, remapping"
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="Updated metadata for task Operation: 3e1d6177-3a75-4565-bb64-066480baf212"
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0/operations/3e1d6177-3a75-4565-bb64-066480baf212 user=
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="MountInstance started" driver=zfs instance=fleet-hare pool=default project=default
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="MountInstance finished" driver=zfs instance=fleet-hare pool=default project=default
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="Failure for task operation: 3e1d6177-3a75-4565-bb64-066480baf212: Common start logic: Storage start: Failed to run: zfs mount default/containers/fleet-hare: cannot open 'default/containers/fleet-hare': dataset does not exist"
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="Event listener finished: 83b0c4a1-e172-4f16-a01b-320f6b0f64ca"
maj 20 20:56:00 2600k lxd.daemon[2879]: t=2020-05-20T20:56:00+0200 lvl=dbug msg="Disconnected event listener: 83b0c4a1-e172-4f16-a01b-320f6b0f64ca"
lines 221-301/301 (END)
Please show output of zfs list
Something strange is going on with your zfs pool
(base) emcp@2600k:~$ zfs list
Command 'zfs' not found, but can be installed with:
sudo apt install zfsutils-linux # version 0.8.3-1ubuntu12, or
sudo apt install zfs-fuse # version 0.7.0-20
guessing first choice is fine here?
Edit: choose first option and ran
(base) emcp@2600k:~$ sudo zfs list
NAME USED AVAIL REFER MOUNTPOINT
default 598M 39,6G 24K none
default/containers 255M 39,6G 24K none
default/containers/flask-app 248M 39,6G 566M /var/snap/lxd/common/lxd/storage-pools/default/containers/flask-app
default/containers/neat-killdeer 7,03M 39,6G 343M /var/snap/lxd/common/lxd/storage-pools/default/containers/neat-killdeer
default/custom 24K 39,6G 24K none
default/deleted 120K 39,6G 24K none
default/deleted/containers 24K 39,6G 24K none
default/deleted/custom 24K 39,6G 24K none
default/deleted/images 24K 39,6G 24K none
default/deleted/virtual-machines 24K 39,6G 24K none
default/images 343M 39,6G 24K none
default/images/5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce 343M 39,6G 343M /var/snap/lxd/common/lxd/storage-pools/default/images/5539cdd71a7b2f56a75d9bb8f45c70b6c1618217ae4df5977215c5636bf8adce
default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56 24K 39,6G 24K /var/snap/lxd/common/lxd/storage-pools/default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56
default/virtual-machines 24K 39,6G 24K none
(base) emcp@2600k:~$
Yep so the default/containers/fleet-hare has been removed somehow.
Can you try creating another container and give me the full logs for lxd from that point please.
(base) emcp@2600k:~$ lxc launch ubuntu:20.04
Creating the instance
Error: Failed instance creation: Create instance from image: Failed to run: zfs clone default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56@readonly default/containers/promoted-iguana: cannot open 'default/images/89cbdbacd37e484c16816ae1ad550930c70320ef6428df4eb723e2aae4c78b56@readonly': dataset does not exist
(base) emcp@2600k:~$
maj 21 16:31:25 2600k lxd.daemon[2879]: t=2020-05-21T16:31:25+0200 lvl=dbug msg=Handling ip=10.43.98.9:57286 method=GET url="/1.0/containers?recursion=1" user=>
maj 21 16:31:25 2600k lxd.daemon[2879]: t=2020-05-21T16:31:25+0200 lvl=dbug msg="Found cert" name=78521d2c79f49f3a038549aec3bf8ce633eb86a1bcd28e02d7f358bda1b6d>
maj 21 16:31:25 2600k lxd.daemon[2879]: t=2020-05-21T16:31:25+0200 lvl=dbug msg=Handling ip=10.43.98.9:57288 method=GET url=/1.0/containers/juju-77b949-0/state>
maj 21 16:31:25 2600k lxd.daemon[2879]: t=2020-05-21T16:31:25+0200 lvl=dbug msg="GetInstanceUsage started" driver=zfs instance=juju-77b949-0 pool=default proje>
maj 21 16:31:25 2600k lxd.daemon[2879]: t=2020-05-21T16:31:25+0200 lvl=dbug msg="GetInstanceUsage finished" driver=zfs instance=juju-77b949-0 pool=default proj>
maj 21 16:37:20 2600k lxd.daemon[2879]: t=2020-05-21T16:37:20+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0 user=
maj 21 16:37:20 2600k lxd.daemon[2879]: t=2020-05-21T16:37:20+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0/events user=
maj 21 16:37:20 2600k lxd.daemon[2879]: t=2020-05-21T16:37:20+0200 lvl=dbug msg="New event listener: 2ccb8fc1-83ae-4b46-84fa-60789098a1b9"
maj 21 16:37:20 2600k lxd.daemon[2879]: t=2020-05-21T16:37:20+0200 lvl=dbug msg=Handling ip=@ method=POST url=/1.0/instances user=
maj 21 16:37:20 2600k lxd.daemon[2879]: t=2020-05-21T16:37:20+0200 lvl=dbug msg="\n\t{\n\t\t\"architecture\": \"\",\n\t\t\"config\": {},\n\t\t\"devices\": {},\>
maj 21 16:37:20 2600k lxd.daemon[2879]: t=2020-05-21T16:37:20+0200 lvl=dbug msg="Responding to instance create"
maj 21 16:37:20 2600k lxd.daemon[2879]: t=2020-05-21T16:37:20+0200 lvl=dbug msg="Connecting to a remote simplestreams server"
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="No name provided, creating correct-marten"
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="New task Operation: 9f4374cc-c846-406e-ae53-7578c9fb5deb"
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="Started task operation: 9f4374cc-c846-406e-ae53-7578c9fb5deb"
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="Connecting to a remote simplestreams server"
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="\n\t{\n\t\t\"type\": \"async\",\n\t\t\"status\": \"Operation created\",\n\t\t\>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0/operations/9f4374cc-c846-406e-ae53-7578c9fb5d>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="Image already exists in the db" image=89cbdbacd37e484c16816ae1ad550930c70320ef>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=info msg="Creating container" ephemeral=false name=correct-marten project=default
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=info msg="Created container" ephemeral=false name=correct-marten project=default
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="CreateInstanceFromImage started" driver=zfs instance=correct-marten pool=defau>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="EnsureImage started" driver=zfs fingerprint=89cbdbacd37e484c16816ae1ad550930c7>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="Database error: &errors.errorString{s:\"No such object\"}"
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="EnsureImage finished" driver=zfs fingerprint=89cbdbacd37e484c16816ae1ad550930c>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="DeleteInstance started" driver=zfs instance=correct-marten pool=default projec>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="Deleting instance volume" driver=zfs instance=correct-marten pool=default proj>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="DeleteInstance finished" driver=zfs instance=correct-marten pool=default proje>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="CreateInstanceFromImage finished" driver=zfs instance=correct-marten pool=defa>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=info msg="Deleting container" created=2020-05-21T16:37:22+0200 ephemeral=false name=corr>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="Database error: &errors.errorString{s:\"No such object\"}"
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=info msg="Deleted container" created=2020-05-21T16:37:22+0200 ephemeral=false name=corre>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="Failure for task operation: 9f4374cc-c846-406e-ae53-7578c9fb5deb: Create insta>
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="Event listener finished: 2ccb8fc1-83ae-4b46-84fa-60789098a1b9"
maj 21 16:37:22 2600k lxd.daemon[2879]: t=2020-05-21T16:37:22+0200 lvl=dbug msg="Disconnected event listener: 2ccb8fc1-83ae-4b46-84fa-60789098a1b9"
maj 21 16:37:34 2600k lxd.daemon[2879]: t=2020-05-21T16:37:34+0200 lvl=dbug msg=Handling ip=@ method=GET url=/1.0 user=
maj 21 16:37:34 2600k lxd.daemon[2879]: t=2020-05-21T16:37:34+0200 lvl=dbug msg=Handling ip=@ method=GET url="/1.0/instances?recursion=2" user=
maj 21 16:37:34 2600k lxd.daemon[2879]: t=2020-05-21T16:37:34+0200 lvl=dbug msg="GetInstanceUsage started" driver=zfs instance=fleet-hare pool=default project=>
maj 21 16:37:34 2600k lxd.daemon[2879]: t=2020-05-21T16:37:34+0200 lvl=dbug msg="GetInstanceUsage finished" driver=zfs instance=fleet-hare pool=default project>
maj 21 16:37:34 2600k lxd.daemon[2879]: t=2020-05-21T16:37:34+0200 lvl=eror msg="Error getting disk usage" err="Failed to run: zfs get -H -p -o value used defa>
maj 21 16:37:34 2600k lxd.daemon[2879]: t=2020-05-21T16:37:34+0200 lvl=dbug msg="GetInstanceUsage started" driver=zfs instance=juju-77b949-0 pool=default proje>
maj 21 16:37:34 2600k lxd.daemon[2879]: t=2020-05-21T16:37:34+0200 lvl=dbug msg="GetInstanceUsage finished" driver=zfs instance=juju-77b949-0 pool=default proj>
lines 255-301/301 (END)
@stgraber any ideas with this, there is some very strange zfs behaviour going on.
I have been reinstalling Ubuntu 20.04 a lot, trying to get a good partition setup going. I succeeded in doing that last night, went to install LXD and Juju… and now the problem popped up with ubuntu 18.04 images too!
I cannot install juju now, because it relies on Ubuntu 18.04.
It seems to freeze mid-installation of the image, forcing me to hard reboot
how can I flush or re-init LXD properly ? Do i erase the zfs pool and then re-init ?
EDIT: I got 18.04 working by the following commands
sudo snap remove lxd
sudo zpool remove default
This deletes the zfs pool and upon reinstallatino and reinit, I successfully got everything going
EDIT2:
And now it seems juju bootstrapping freezes upon setup
but on retry it succeeds
