ZFS snapshot restore issue

HI @stgraber hope you are doing good ,

i am using zfs file system and i have restored to latest snapshot successfully but now when i am going to take snapshot again its giving me following error

Error: Failed to run: zfs mount zfs/containers/kilback: mount: /var/snap/lxd/common/lxd/storage-pools/zfs/containers/kilback: zfs/containers/kilback already mounted or mount point busy.
cannot mount ‘zfs/containers/kilback’: No such device or address

its very annoying :confused: please help me ,

Hmm, No such device or address isn’t something I’ve seen from ZFS before…
Can you show dmesg and zfs list -t all?

[1819283.149991] [] ? page_fault+0x28/0x30
[1819283.155609] Task in /lxc.payload.adi killed as a result of limit of /lxc.payload.adi
[1819283.166431] memory: usage 500000kB, limit 500000kB, failcnt 2358300
[1819283.173003] memory+swap: usage 0kB, limit 9007199254740988kB, failcnt 0
[1819283.181276] kmem: usage 9496kB, limit 9007199254740988kB, failcnt 0
[1819283.189211] Memory cgroup stats for /lxc.payload.adi: cache:34824KB rss:455680KB rss_huge:6144KB mapped_file:24168KB dirty:0KB writeback:0KB inactive_anon:34236KB active_anon:455844KB inactive_file:212KB active_file:212KB unevictable:0KB
[1819283.221890] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
[1819283.230823] [26274] 1000000 26274 34721 366 34 3 0 0 systemd
[1819283.241320] [26358] 1000000 26358 18692 6069 38 3 0 0 systemd-journal
[1819283.252694] [26422] 1000105 26422 11279 115 26 3 0 0 dbus-daemon
[1819283.263669] [26477] 1000000 26477 7416 71 19 3 0 0 cron
[1819283.274028] [26478] 1000000 26478 9495 118 22 3 0 0 systemd-logind
[1819283.285287] [26556] 1000000 26556 5121 264 12 3 0 0 dhclient
[1819283.295980] [26628] 1000000 26628 92464 4034 141 4 0 0 php-fpm7.0
[1819283.306858] [26631] 1000107 26631 74187 1637 99 4 0 0 freshclam
[1819283.317652] [26641] 1000000 26641 3575 32 12 3 0 0 agetty
[1819283.328200] [26658] 1000000 26658 28659 141 24 3 0 0 monit
[1819283.338648] [26783] 1000033 26783 92464 1805 122 4 0 0 php-fpm7.0
[1819283.349550] [26785] 1000033 26785 92464 1805 122 4 0 0 php-fpm7.0
[1819283.359055] [26811] 1000000 26811 81716 1323 79 4 0 0 nginx
[1819283.369497] [26812] 1000033 26812 81716 1396 79 4 0 0 nginx
[1819283.379951] [26813] 1000033 26813 81716 1374 79 4 0 0 nginx
[1819283.390399] [27065] 1000108 27065 163462 15614 107 3 0 0 mysqld
[1819283.401013] [31499] 1000106 31499 14071 193 31 3 0 0 exim4
[1819283.411492] [ 7143] 1000107 7143 172671 92075 260 4 0 0 freshclam
[1819283.422211] Memory cgroup out of memory: Kill process 7143 (freshclam) score 738 or sacrifice child
[1819283.432919] Killed process 7143 (freshclam) total-vm:690684kB, anon-rss:368168kB, file-rss:132kB, shmem-rss:0kB
[1819283.457568] oom_reaper: reaped process 7143 (freshclam), now anon-rss:0kB, file-rss:0kB, shmem-rss:0kB

this is the output of dmesg

zfs list -t all
NAME USED AVAIL REFER MOUNTPOINT
default 3.39G 248G 19K none
default@zfs-auto-snap_hourly-2020-08-20-1017 0 - 19K -
default@zfs-auto-snap_hourly-2020-08-20-1117 0 - 19K -
default@zfs-auto-snap_hourly-2020-08-20-1217 0 - 19K -
default@zfs-auto-snap_frequent-2020-08-20-1300 0 - 19K -
default@zfs-auto-snap_frequent-2020-08-20-1315 0 - 19K -
default@zfs-auto-snap_frequent-2020-08-20-1330 0 - 19K -
default@zfs-auto-snap_frequent-2020-08-20-1345 0 - 19K -
default/containers 1.26G 248G 19K none
default/containers@zfs-auto-snap_frequent-2020-08-20-0930 0 - 19K -
default/containers/adi 145M 248G 712M /var/snap/lxd/common/lxd/storage-pools/default/containers/adi
default/containers/adi@zfs-auto-snap_frequent-2020-08-20-0930 366K - 712M -
default/containers/elentra 433M 248G 998M /var/snap/lxd/common/lxd/storage-pools/default/containers/elentra
default/containers/elentra@snapshot-snap1 20.5K - 998M -
default/containers/elentra@snapshot-latest 20.5K - 998M -
default/containers/elentra@snapshot-prelatest 67.5K - 998M -
default/containers/elentra@zfs-auto-snap_frequent-2020-08-20-0930 499K - 998M -
default/containers/integ901 146M 248G 712M /var/snap/lxd/common/lxd/storage-pools/default/containers/integ901
default/containers/integ901@zfs-auto-snap_frequent-2020-08-20-0930 450K - 711M -
default/containers/integ901@snapshot-snap1 17K - 711M -
default/containers/integ901@snapshot-snap2 19.5K - 711M -
default/containers/new 2.07M 248G 133M /var/snap/lxd/common/lxd/storage-pools/default/containers/new
default/containers/new@snapshot-snap1 12K - 133M -
default/containers/new@snapshot-snap2 12.5K - 133M -
default/containers/proxy 122M 248G 405M /var/snap/lxd/common/lxd/storage-pools/default/containers/proxy
default/containers/proxy@zfs-auto-snap_frequent-2020-08-20-0930 1.11M - 405M -
default/containers/wonderwoman1 120M 248G 749M /var/snap/lxd/common/lxd/storage-pools/default/containers/wonderwoman1
default/containers/wonderwoman1@zfs-auto-snap_frequent-2020-08-20-0930 373K - 749M -
default/containers/yt77 323M 248G 730M /var/snap/lxd/common/lxd/storage-pools/default/containers/yt77
default/containers/yt77@zfs-auto-snap_frequent-2020-08-20-0930 364K - 711M -
default/custom 19K 248G 19K none
default/custom@zfs-auto-snap_hourly-2020-08-20-1017 0 - 19K -
default/custom@zfs-auto-snap_hourly-2020-08-20-1117 0 - 19K -
default/custom@zfs-auto-snap_hourly-2020-08-20-1217 0 - 19K -
default/custom@zfs-auto-snap_frequent-2020-08-20-1300 0 - 19K -
default/custom@zfs-auto-snap_frequent-2020-08-20-1315 0 - 19K -
default/custom@zfs-auto-snap_frequent-2020-08-20-1330 0 - 19K -
default/custom@zfs-auto-snap_frequent-2020-08-20-1345 0 - 19K -
default/deleted 1.66G 248G 19K none
default/deleted@zfs-auto-snap_hourly-2020-08-20-1017 0 - 19K -
default/deleted@zfs-auto-snap_hourly-2020-08-20-1117 0 - 19K -
default/deleted@zfs-auto-snap_hourly-2020-08-20-1217 0 - 19K -
default/deleted@zfs-auto-snap_frequent-2020-08-20-1300 0 - 19K -
default/deleted@zfs-auto-snap_frequent-2020-08-20-1315 0 - 19K -
default/deleted@zfs-auto-snap_frequent-2020-08-20-1330 0 - 19K -
default/deleted@zfs-auto-snap_frequent-2020-08-20-1345 0 - 19K -
default/deleted/containers 19K 248G 19K none
default/deleted/containers@zfs-auto-snap_hourly-2020-08-20-1017 0 - 19K -
default/deleted/containers@zfs-auto-snap_hourly-2020-08-20-1117 0 - 19K -
default/deleted/containers@zfs-auto-snap_hourly-2020-08-20-1217 0 - 19K -
default/deleted/containers@zfs-auto-snap_frequent-2020-08-20-1300 0 - 19K -
default/deleted/containers@zfs-auto-snap_frequent-2020-08-20-1315 0 - 19K -
default/deleted/containers@zfs-auto-snap_frequent-2020-08-20-1330 0 - 19K -
default/deleted/containers@zfs-auto-snap_frequent-2020-08-20-1345 0 - 19K -
default/deleted/custom 19K 248G 19K none
default/deleted/custom@zfs-auto-snap_hourly-2020-08-20-1017 0 - 19K -
default/deleted/custom@zfs-auto-snap_hourly-2020-08-20-1117 0 - 19K -
default/deleted/custom@zfs-auto-snap_hourly-2020-08-20-1217 0 - 19K -
default/deleted/custom@zfs-auto-snap_frequent-2020-08-20-1300 0 - 19K -
default/deleted/custom@zfs-auto-snap_frequent-2020-08-20-1315 0 - 19K -
default/deleted/custom@zfs-auto-snap_frequent-2020-08-20-1330 0 - 19K -
default/deleted/custom@zfs-auto-snap_frequent-2020-08-20-1345 0 - 19K -
default/deleted/images 1.66G 248G 19K none
default/deleted/images@zfs-auto-snap_hourly-2020-08-20-1017 0 - 19K -
default/deleted/images@zfs-auto-snap_hourly-2020-08-20-1117 0 - 19K -
default/deleted/images@zfs-auto-snap_hourly-2020-08-20-1217 0 - 19K -
default/deleted/images@zfs-auto-snap_frequent-2020-08-20-1300 0 - 19K -
default/deleted/images@zfs-auto-snap_frequent-2020-08-20-1315 0 - 19K -
default/deleted/images@zfs-auto-snap_frequent-2020-08-20-1330 0 - 19K -
default/deleted/images@zfs-auto-snap_frequent-2020-08-20-1345 0 - 19K -

Hmm, that’s rather confusing, LXD references zfs/containers/kilback which would refer to a zfs zpool.

Your zfs list output shows no dataset for kilback nor a pool named zfs.

Can you show lxc storage list and lxc storage show NAME for each of them?