Moved container from 4.5 to 4.22: Can't bind to any port

Hello,

I had a container running on a machine and exported / imported it to another machine.

Everything appeared to be running fine however processes inside the container weren’t able to bind to any port. Even a simple nc -l 5511 wouldn’t work. netstat wouldn’t show anything listening on port 5511 and netcat wouldn’t return any error. I experienced this with other services also no errors but nothing from the container itself or the host was able to connect to them.

Old system:
Debian GNU/Linux 10 (buster) (kernel 4.19.0-18-amd64)
systemd 241 (241)
lxd: 4.5

New system:
Debian GNU/Linux 11 (bullseye) (kernel 5.10.0-11-amd64)
systemd 247 (247.3-6)
lxd: 4.22

Container config:

  • lxc launch images:debian/buster debian;
  • Unprivileged container;
  • Network unmanaged by LXD/LXD, simple bridge on the host with iptables rules;
  • Systemd network in both the host and container;
  • Container systemd 241 (241);
  • No firewall was running inside the container.

The classic network manager and dhcpd replaced by systemd networkd:

systemctl disable networking.service
systemctl disable dhcpcd.service
systemctl mask networking.service
systemctl mask dhcpcd.service

vim /etc/systemd/network/10-eth0.network
	[Match]
	Name=eth0

	[Link]
	RequiredForOnline=yes

	[Network]
	Address=10.12.125.6
	Gateway=10.12.125.1
	DNS=10.12.125.1

systemctl enable systemd-networkd.service
systemctl start systemd-networkd.service
systemctl status systemd-networkd.service

rm -rf /etc/network/interfaces
apt install deborphan
apt --autoremove purge network-manager rdnssd openresolv ifupdown dhcpcd5 isc-dhcp-client isc-dhcp-common
apt --autoremove purge $(deborphan)
apt --autoremove purge $(deborphan)

The network was working properly I was able to ping hosts, install packages with APT. But services in the container weren’t able to bind to ports.

How can I debug this further?

Thank you.

Additional note: new containers made in this machine work just fine. Seems to be an issue only with the imported container.

This sounds like it may be a cgroup legacy problem, try setting this in your kernel command line and then reboot:

systemd.unified_cgroup_hierarchy=false 
1 Like

Thanks for the tip. Is there way to upgrade containers so they can use the new cgroup hierarchy?

It’s outside of LXD’s control, it has to do with the version of systemd or similar init system included by the Linux distribution you’re running in the container.