Protocol 58, assuming that uses normal protocol numbering would be IPv6 ICMP.
So something in the container may now be holding such a socket open, causing CRIU to fail to serialize the container?
Thank you. After I disabled ipv6 inside the container, I saw another error message.
$ lxc snapshot oddish --stateful
error: snapshot dump failed
(00.209072) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(00.209270) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(00.209305) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(00.209327) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(00.209361) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(04.061447) Warn (criu/arch/x86/crtools.c:133): Will restore 14149 with interrupted system call
(05.844452) Warn (criu/arch/x86/crtools.c:133): Will restore 14179 with interrupted system call
(06.240965) Warn (criu/arch/x86/crtools.c:133): Will restore 14183 with interrupted system call
(06.612205) Warn (criu/arch/x86/crtools.c:133): Will restore 14192 with interrupted system call
(07.098707) Warn (criu/arch/x86/crtools.c:133): Will restore 14193 with interrupted system call
(07.476052) Warn (criu/arch/x86/crtools.c:133): Will restore 14194 with interrupted system call
(07.924401) Warn (criu/arch/x86/crtools.c:133): Will restore 14195 with interrupted system call
(09.850728) Warn (criu/arch/x86/crtools.c:133): Will restore 7625 with interrupted system call
(10.198143) Warn (criu/arch/x86/crtools.c:133): Will restore 7648 with interrupted system call
(10.578350) Warn (criu/arch/x86/crtools.c:133): Will restore 7653 with interrupted system call
(10.817549) Warn (criu/arch/x86/crtools.c:133): Will restore 7659 with interrupted system call
(11.091100) Warn (criu/arch/x86/crtools.c:133): Will restore 7660 with interrupted system call
(11.417236) Warn (criu/arch/x86/crtools.c:133): Will restore 7661 with interrupted system call
(11.767017) Warn (criu/arch/x86/crtools.c:133): Will restore 7662 with interrupted system call
(12.089211) Error (criu/mount.c:1044): mnt: Can’t create a temporary directory: Value too large for defined data type
(12.115595) Error (criu/cr-dump.c:1628): Dumping FAILED.
$ sudo tail -n 100 /var/log/lxd/oddish/snapshot_dump_2018-02-08T13:12:18-08:00.log
(12.039172) 0x7f346f29e000-0x7f346f29f000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0xa000 reg fp shmid: 0xb7
(12.039184) 0x7f346f29f000-0x7f346f2a0000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0xb000 reg fp shmid: 0xb7
(12.039196) 0x7f346f2a0000-0x7f346f2b5000 (84K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0 reg fp shmid: 0xb8
(12.039208) 0x7f346f2b5000-0x7f346f4b4000 (2044K) prot 0 flags 0x2 fdflags 0 st 0x41 off 0x15000 reg fp shmid: 0xb8
(12.039220) 0x7f346f4b4000-0x7f346f4b5000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x14000 reg fp shmid: 0xb8
(12.039231) 0x7f346f4b5000-0x7f346f4b6000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x15000 reg fp shmid: 0xb8
(12.039243) 0x7f346f4b6000-0x7f346f4b8000 (8K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap shmid: 0
(12.039255) 0x7f346f4b8000-0x7f346f4bf000 (28K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0 reg fp shmid: 0xb9
(12.039267) 0x7f346f4bf000-0x7f346f6be000 (2044K) prot 0 flags 0x2 fdflags 0 st 0x41 off 0x7000 reg fp shmid: 0xb9
(12.039279) 0x7f346f6be000-0x7f346f6bf000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x6000 reg fp shmid: 0xb9
(12.039290) 0x7f346f6bf000-0x7f346f6c0000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x7000 reg fp shmid: 0xb9
(12.039302) 0x7f346f6c0000-0x7f346f85d000 (1652K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0 reg fp shmid: 0x1d
(12.039314) 0x7f346f85d000-0x7f346fa5c000 (2044K) prot 0 flags 0x2 fdflags 0 st 0x41 off 0x19d000 reg fp shmid: 0x1d
(12.039326) 0x7f346fa5c000-0x7f346fa60000 (16K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x19c000 reg fp shmid: 0x1d
(12.039338) 0x7f346fa60000-0x7f346fa62000 (8K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x1a0000 reg fp shmid: 0x1d
(12.039350) 0x7f346fa62000-0x7f346fa66000 (16K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap shmid: 0
(12.039362) 0x7f346fa66000-0x7f346fa89000 (140K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0 reg fp shmid: 0x1f
(12.039374) 0x7f346fc7d000-0x7f346fc7f000 (8K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap shmid: 0
(12.039385) 0x7f346fc89000-0x7f346fc8a000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x23000 reg fp shmid: 0x1f
(12.039397) 0x7f346fc8a000-0x7f346fc8b000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x24000 reg fp shmid: 0x1f
(12.039409) 0x7f346fc8b000-0x7f346fc8c000 (4K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap shmid: 0
(12.039421) 0x7ffe7cb42000-0x7ffe7cb64000 (136K) prot 0x3 flags 0x122 fdflags 0 st 0x201 off 0 reg ap shmid: 0
(12.039434) 0x7ffe7cbcb000-0x7ffe7cbce000 (12K) prot 0x1 flags 0x22 fdflags 0 st 0x1201 off 0 reg vvar ap shmid: 0
(12.039446) 0x7ffe7cbce000-0x7ffe7cbd0000 (8K) prot 0x5 flags 0x22 fdflags 0 st 0x209 off 0 reg vdso ap shmid: 0
(12.039458) 0xffffffffff600000-0xffffffffff601000 (4K) prot 0x5 flags 0x22 fdflags 0 st 0x204 off 0 vsys ap shmid: 0
(12.039471) Obtaining task auvx …
(12.039622) Dumping task cwd id 0x4f root id 0x4f
(12.039813) mnt: Dumping mountpoints
(12.039827) mnt: 779: 4e:/ @ ./tmp
(12.039866) mnt: Path /tmp' resolved to./tmp’ mountpoint
(12.057608) mnt: 778: 4c:/ @ ./run
(12.057644) mnt: Path /run' resolved to./run’ mountpoint
tar: ./salt/master/workers.ipc: socket ignored
tar: ./salt/master/publish_pull.ipc: socket ignored
tar: ./salt/master/master_event_pull.ipc: socket ignored
tar: ./salt/master/master_event_pub.ipc: socket ignored
tar: ./fcgiwrap.sock: socket ignored
tar: ./dbus/system_bus_socket: socket ignored
tar: ./php-fpm/php-fpm.sock: socket ignored
tar: ./udev/control: socket ignored
tar: ./lvm/lvmetad.socket: socket ignored
tar: ./systemd/coredump: socket ignored
tar: ./systemd/journal/dev-log: socket ignored
tar: ./systemd/journal/socket: socket ignored
tar: ./systemd/journal/stdout: socket ignored
tar: ./systemd/private: socket ignored
tar: ./systemd/cgroups-agent: socket ignored
tar: ./systemd/notify: socket ignored
tar: ./systemd/inaccessible/sock: socket ignored
(12.088976) mnt: 777: 20:/ @ ./sys/fs/cgroup/pids
(12.089012) mnt: 776: 21:/ @ ./sys/fs/cgroup/cpuset
(12.089025) mnt: 775: 1f:/ @ ./sys/fs/cgroup/devices
(12.089037) mnt: 771: 1c:/ @ ./sys/fs/cgroup/hugetlb
(12.089048) mnt: 762: 22:/ @ ./sys/fs/cgroup/freezer
(12.089059) mnt: 756: 1e:/ @ ./sys/fs/cgroup/perf_event
(12.089070) mnt: 755: 1d:/ @ ./sys/fs/cgroup/blkio
(12.089081) mnt: 754: 23:/ @ ./sys/fs/cgroup/memory
(12.089092) mnt: 753: 1b:/ @ ./sys/fs/cgroup/cpu,cpuacct
(12.089103) mnt: 752: 1a:/ @ ./sys/fs/cgroup/net_cls,net_prio
(12.089114) mnt: 751: 18:/ @ ./sys/fs/cgroup/systemd
(12.089126) mnt: 750: 4d:/ @ ./sys/fs/cgroup
(12.089136) mnt: Something is mounted on top of ./sys/fs/cgroup
(12.089211) Error (criu/mount.c:1044): mnt: Can’t create a temporary directory: Value too large for defined data type
(12.089700) Unlock network
(12.089759) Running network-unlock scripts
(12.114929) Unfreezing tasks into 1
(12.114969) Unseizing 12156 into 1
(12.114989) Unseizing 12350 into 1
(12.115004) Unseizing 12365 into 1
(12.115018) Unseizing 12375 into 1
(12.115036) Unseizing 12398 into 1
(12.115050) Unseizing 12400 into 1
(12.115064) Unseizing 14107 into 1
(12.115077) Unseizing 14128 into 1
(12.115092) Unseizing 14129 into 1
(12.115105) Unseizing 14108 into 1
(12.115119) Unseizing 14148 into 1
(12.115132) Unseizing 14149 into 1
(12.115156) Unseizing 14151 into 1
(12.115169) Unseizing 14177 into 1
(12.115182) Unseizing 14178 into 1
(12.115195) Unseizing 14179 into 1
(12.115233) Unseizing 14183 into 1
(12.115254) Unseizing 14192 into 1
(12.115274) Unseizing 14193 into 1
(12.115294) Unseizing 14194 into 1
(12.115314) Unseizing 14195 into 1
(12.115334) Unseizing 14111 into 1
(12.115349) Unseizing 14120 into 1
(12.115370) Unseizing 14131 into 1
(12.115384) Unseizing 14140 into 1
(12.115398) Unseizing 14141 into 1
(12.115411) Unseizing 7625 into 1
(12.115425) Unseizing 7648 into 1
(12.115439) Unseizing 7653 into 1
(12.115452) Unseizing 7659 into 1
(12.115466) Unseizing 7660 into 1
(12.115479) Unseizing 7661 into 1
(12.115492) Unseizing 7662 into 1
(12.115595) Error (criu/cr-dump.c:1628): Dumping FAILED.
I also tried stateful stop for other containers. I saw different erorrs.
$ lxc snapshot test --stateful
error: snapshot dump failed
(00.190155) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(00.190370) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(00.190403) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(00.190433) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(00.190469) Warn (criu/autofs.c:77): Failed to find pipe_ino option (old kernel?)
(01.227696) Error (criu/parasite-syscall.c:532): Unable to connect a transport socket: Permission denied
(01.227733) Error (criu/parasite-syscall.c:134): Can’t block signals for 25752: No such process
(01.227748) Error (criu/cr-dump.c:1244): Can’t infect (pid: 25752) with parasite
(01.263289) Error (criu/ptrace.c:54): Unable to detach from 25752: No such process
(01.263351) Error (criu/cr-dump.c:1628): Dumping FAILED.
sudo tail -n 50 /var/log/lxd/test/snapshot_dump_2018-02-08T13:34:22-08:00.log
(01.226839) 0x7fc83c756000-0x7fc83c7b8000 (392K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0 reg fp shmid: 0x58
(01.226853) 0x7fc83c7b8000-0x7fc83c9b7000 (2044K) prot 0 flags 0x2 fdflags 0 st 0x41 off 0x62000 reg fp shmid: 0x58
(01.226868) 0x7fc83c9b7000-0x7fc83c9b8000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x61000 reg fp shmid: 0x58
(01.226883) 0x7fc83c9b8000-0x7fc83c9b9000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x62000 reg fp shmid: 0x58
(01.226898) 0x7fc83c9b9000-0x7fc83cb8d000 (1872K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0 reg fp shmid: 0x59
(01.226913) 0x7fc83cb8d000-0x7fc83cd8c000 (2044K) prot 0 flags 0x2 fdflags 0 st 0x41 off 0x1d4000 reg fp shmid: 0x59
(01.226928) 0x7fc83cd8c000-0x7fc83cd8e000 (8K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x1d3000 reg fp shmid: 0x59
(01.226943) 0x7fc83cd8e000-0x7fc83cd92000 (16K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x1d5000 reg fp shmid: 0x59
(01.226957) 0x7fc83cd92000-0x7fc83cd93000 (4K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap shmid: 0
(01.226972) 0x7fc83cd93000-0x7fc83cdb9000 (152K) prot 0x5 flags 0x2 fdflags 0 st 0x41 off 0 reg fp shmid: 0x11
(01.226986) 0x7fc83cf22000-0x7fc83cfb2000 (576K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap shmid: 0
(01.227001) 0x7fc83cfb8000-0x7fc83cfb9000 (4K) prot 0x1 flags 0x2 fdflags 0 st 0x41 off 0x25000 reg fp shmid: 0x11
(01.227015) 0x7fc83cfb9000-0x7fc83cfba000 (4K) prot 0x3 flags 0x2 fdflags 0 st 0x41 off 0x26000 reg fp shmid: 0x11
(01.227030) 0x7fc83cfba000-0x7fc83cfbb000 (4K) prot 0x3 flags 0x22 fdflags 0 st 0x201 off 0 reg ap shmid: 0
(01.227046) 0x7ffc8b235000-0x7ffc8b257000 (136K) prot 0x3 flags 0x122 fdflags 0 st 0x201 off 0 reg ap shmid: 0
(01.227062) 0x7ffc8b354000-0x7ffc8b357000 (12K) prot 0x1 flags 0x22 fdflags 0 st 0x1201 off 0 reg vvar ap shmid: 0
(01.227077) 0x7ffc8b357000-0x7ffc8b359000 (8K) prot 0x5 flags 0x22 fdflags 0 st 0x209 off 0 reg vdso ap shmid: 0
(01.227092) 0xffffffffff600000-0xffffffffff601000 (4K) prot 0x5 flags 0x22 fdflags 0 st 0x204 off 0 vsys ap shmid: 0
(01.227107) ----------------------------------------
(01.227121)
(01.227135) Collecting fds (pid: 25752)
(01.227150) ----------------------------------------
(01.227194) Found 7 file descriptors
(01.227208) ----------------------------------------
(01.227232) Dump private signals of 25752
(01.227251) Dump shared signals of 25752
(01.227274) Parasite syscall_ip at 0x56309c046000
(01.227427) Set up parasite blob using memfd
(01.227446) Putting parasite blob into 0x7f38bd37b000->0x7fc83cf09000
(01.227484) Dumping GP/FPU registers for 25752
(01.227506) xsave runtime structure
(01.227520) -----------------------
(01.227534) cwd:0 swd:0 twd:0 fop:0 mxcsr:1f80 mxcsr_mask:ffff
(01.227549) magic1:46505853 extended_size:344 xstate_bv:7 xstate_size:340
(01.227564) xstate_bv: 7
(01.227578) -----------------------
(01.227591) Putting tsock into pid 25752
(01.227696) Error (criu/parasite-syscall.c:532): Unable to connect a transport socket: Permission denied
(01.227733) Error (criu/parasite-syscall.c:134): Can’t block signals for 25752: No such process
(01.227748) Error (criu/cr-dump.c:1244): Can’t infect (pid: 25752) with parasite
(01.228126) Unlock network
(01.228140) Running network-unlock scripts
(01.263141) Unfreezing tasks into 1
(01.263199) Unseizing 25256 into 1
(01.263224) Unseizing 25450 into 1
(01.263246) Unseizing 25479 into 1
(01.263267) Unseizing 25752 into 1
(01.263289) Error (criu/ptrace.c:54): Unable to detach from 25752: No such process
(01.263305) Unseizing 25808 into 1
(01.263351) Error (criu/cr-dump.c:1628): Dumping FAILED.
Is there anything wrong with my local environment?