Inside each container, /dev is a symlink to /dev/.lxc/ on the host. My CentOS 7 host’s /dev took a ‘rm -rf’ yesterday, wiping out all the containers’ /dev’s too. Now ssh, lxc-attach, and lxc-console don’t work. How would I recreate the LXC objects in /dev so I can gracefully shutdown each container?
Hmm, LXC using /dev/.lxc on the host is a pretty old practice so I’m not even sure what would normally be in there.
One issue with that recursive rm is that the directory being bind-mounted into the containers will no longer be visible, so re-creating /dev/.lxc and putting files in there won’t actually solve the problem.
You may be able to go to /proc/<pid of init in container>/root/dev on the host and then manually mknod the few device nodes that are critical for your containers to run. That should get things working until they get restarted.
Thank you. /dev/.lxc itself and the individual folders below it were not deleted as there were some items in use so rm couldn’t delete them.
The newest version of lxc in epel is 1.0.11. Is there a better repo I should be using?
I figured out the required missing devices by creating a new container and comparing it to an old one. The only thing I could not figure out how to recreate was:
srw-rw-rw-. 1 root root 0 Feb 6 09:42 log
I fixed the problem by creating fix.sh in /dev/.lxc with the below contents and ran it for each container shown in lxc-ls -1 --active.
#!/bin/bash if [ ! -d "$1" ]; then echo Directory does not exist exit 1 fi cd $1 mknod -m 0666 null c 1 3 mknod -m 0666 zero c 1 5 mknod -m 0666 full c 1 7 mknod -m 0666 random c 1 8 mknod -m 0666 urandom c 1 9 mknod -m 0666 tty c 5 0 ln -s lxc/console console ln -s /proc/kcore core ln -s /proc/self/fd fd ln -s /run/systemd/initctl/fifo initctl ln -s /proc/self/fd/2 stderr ln -s /proc/self/fd/0 stdin ln -s /proc/self/fd/1 stdout ln -s lxc/tty1 tty1 ln -s lxc/tty2 tty2 ln -s lxc/tty3 tty3 ln -s lxc/tty4 tty4 cd ..
I believe /dev/log is created by rsyslog or similar running inside your container, so it should be fine to ignore.
I didn’t remember 1.0.x still having that logic 1.0.x is still supported, though only for security and only for another year. Hopefully this will be updated to 2.0.x (or maybe 3.0.x) before 1.0.x reaches end of life.