Inside the container? That fails as well…
root@mygui:/# dmesg
dmesg: read kernel buffer failed: Operation not permitted
On host, I think these are the only things logged when running the container:
...
[ 76.167264] NMI watchdog: Enabled. Permanently consumes one hw-PMU counter.
[ 776.042327] lxcbr0: port 1(veth1000_XvyQ) entered blocking state
[ 776.042329] lxcbr0: port 1(veth1000_XvyQ) entered disabled state
[ 776.042411] device veth1000_XvyQ entered promiscuous mode
[ 776.042557] eth0: renamed from veth1000_XvyQp
[ 776.069817] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[ 776.069845] IPv6: ADDRCONF(NETDEV_CHANGE): veth1000_XvyQ: link becomes ready
[ 776.069961] lxcbr0: port 1(veth1000_XvyQ) entered blocking state
[ 776.069962] lxcbr0: port 1(veth1000_XvyQ) entered forwarding state
[ 776.070008] IPv6: ADDRCONF(NETDEV_CHANGE): lxcbr0: link becomes ready
[ 776.073732] Not activating Mandatory Access Control as /sbin/tomoyo-init does not exist.
@stgraber Random thought, but is it possible that changing the umask in /etc/login.defs could be an issue? I’ve set it to:
UMASK 027
@stgraber I did some more searching for specific errors I was seeing and then tried some AI-generated solutions. Adding this to the config:
lxc.apparmor.profile = unconfined
lxc.cap.drop =
lxc.mount.entry = sys dev/.lxc/sys sysfs create=dir,optional 0 0
This seems to fix the broken systemd services in the container. But, I still get no network connection…
In journalctl there’s still a lot of these type of errors:
Jan 12 16:44:46 mygui udevadm[114]: vcs: Failed to write 'add' to '/sys/devices/virtual/vc/vcs/uevent': Permission denied
I tried things like lxc.mount.auto = sys:rw to make it writable, but still those same errors.
I can see that networkd is doing something though:
Jan 12 17:00:52 mygui systemd-networkd[120]: lo: Link UP
Jan 12 17:00:52 mygui systemd-networkd[120]: lo: Gained carrier
Jan 12 17:00:52 mygui systemd-networkd[120]: Enumeration completed
Jan 12 17:00:52 mygui systemd[1]: Started systemd-networkd.service - Network Configuration.
░░ Subject: A start job for unit systemd-networkd.service has finished successfully
░░ Defined-By: systemd
░░ Support: https://www.debian.org/support
░░
░░ A start job for unit systemd-networkd.service has finished successfully.
░░
░░ The job identifier is 86.
Jan 12 17:00:52 mygui systemd-resolved[126]: Positive Trust Anchors:
Jan 12 17:00:52 mygui systemd-resolved[126]: . IN DS 20326 8 2 e06d44b80b8f1d39a95c0b0d7c65d08458e880409bbc683457104237c7f8ec8d
Jan 12 17:00:52 mygui systemd-resolved[126]: Negative trust anchors: home.arpa 10.in-addr.arpa 16.172.in-addr.arpa 17.172.in-addr.arpa 18.172.in-addr.arpa 19.172.in-addr.arpa 20.172.in-addr.arpa 21.172.in-addr.arpa 22.172.in-addr.arpa 23.172.in-addr.arpa 24.172.in-addr.arpa 25.172.in-addr.arpa 26.172.in-addr.arpa 27.172.in-addr.arpa 28.172.in-addr.arpa 29.172.in-addr.arpa 30.172.in-addr.arpa 31.172.in-addr.arpa 170.0.0.192.in-addr.arpa 171.0.0.192.in-addr.arpa 168.192.in-addr.arpa d.f.ip6.arpa ipv4only.arpa corp home internal intranet lan local private test
Jan 12 17:00:52 mygui systemd-resolved[126]: Using system hostname 'mygui'.
Jan 12 17:00:52 mygui systemd[1]: Started systemd-resolved.service - Network Name Resolution.
Ugh, I gave up and tried to sudo and use a privileged container. But, it has the exact same issue. I can’t get a network connection no matter what I do.
I got this from dhclient journal though:
Jan 14 14:47:41 mygui systemd-networkd[106]: eth0: Gained IPv6LL
Jan 14 14:48:05 mygui dhclient[125]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5
Jan 14 14:48:10 mygui dhclient[125]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 12
Jan 14 14:48:22 mygui dhclient[125]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
Jan 14 14:48:30 mygui dhclient[125]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 14
Jan 14 14:48:44 mygui dhclient[125]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 14
Jan 14 14:48:58 mygui dhclient[125]: DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 8
Jan 14 14:49:06 mygui dhclient[125]: No DHCPOFFERS received.
Jan 14 14:49:06 mygui dhclient[125]: No working leases in persistent database - sleeping.
Okay, so networkd isn’t the problem, your host network is.
Make sure that dnsmasq is running and check for any firewall rules on the host.
> ps -A | grep dnsmasq
5806 ? 00:00:00 dnsmasq
Not sure exactly what to check for firewall (I didn’t change anything), but here’s iptables:
# iptables -L
Chain INPUT (policy ACCEPT)
target prot opt source destination
ACCEPT tcp -- anywhere anywhere tcp dpt:domain
ACCEPT udp -- anywhere anywhere udp dpt:domain
ACCEPT tcp -- anywhere anywhere tcp dpt:67
ACCEPT udp -- anywhere anywhere udp dpt:bootps
Chain FORWARD (policy ACCEPT)
target prot opt source destination
ACCEPT all -- anywhere anywhere
ACCEPT all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
I’ve tried setting the IP address manually with ip address add 192.168.1.245/24 dev eth0 (I’m just making blind guesses). But, still no network connection, although the IP address does appear under lxc-ls --fancy.
I also tried adding to config:
lxc.net.0.ipv4.address = 192.168.1.16/24
lxc.net.0.ipv4.gateway = auto
This resulted in it attempting to make a network request (instead of just saying network unreachable), but getting 100% packet loss.
I really need something working this week.
Ah, I got it. ip addr show lxcbr0 showed inet 10.0.3.1/24. So, I set the config value to lxc.net.0.ipv4.address = 10.0.3.2/24. Now the network actually works. Still need to fix DNS resolution though.
I’ve added DNS=8.8.8.8 into /etc/systemd/resolved.conf, restarted systemd-resolved and verified that the nameserver in /etc/resolv.conf is now set to 8.8.8.8. But, still get:
# ping google.com
ping: google.com: Temporary failure in name resolution
How can I get the DNS working?
@stgraber Any ideas? I’ve spent the whole day trying to fix this and not made any progress at all.
Here is a bunch of commands I’ve run that might provide some useful information:
Host:
> sudo ss -ulpn | grep 53
UNCONN 0 0 10.0.3.1:53 0.0.0.0:* users:(("dnsmasq",pid=5297,fd=6))
UNCONN 0 0 0.0.0.0:5353 0.0.0.0:* users:(("avahi-daemon",pid=3594,fd=12))
UNCONN 0 0 [::]:5353 [::]:* users:(("avahi-daemon",pid=3594,fd=13))
> sudo tcpdump -nn -i lxcbr0 port 53
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on lxcbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
18:39:23.332255 IP 10.0.3.2.40121 > 8.8.8.8.53: 39796+ [1au] A? google.com. (51)
18:39:28.334722 IP 10.0.3.2.40121 > 8.8.8.8.53: 39796+ [1au] A? google.com. (51)
18:39:33.338661 IP 10.0.3.2.40121 > 8.8.8.8.53: 39796+ [1au] A? google.com. (51)
^C
3 packets captured
3 packets received by filter
0 packets dropped by kernel
> sudo tcpdump -nn -i lxcbr0 port 53 -vv
tcpdump: listening on lxcbr0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
18:54:10.563984 IP (tos 0x0, ttl 64, id 17773, offset 0, flags [none], proto UDP (17), length 56)
10.0.3.2.55012 > 8.8.8.8.53: [bad udp cksum 0x1d47 -> 0x30db!] 50918+ A? google.com. (28)
18:54:15.566724 IP (tos 0x0, ttl 64, id 18761, offset 0, flags [none], proto UDP (17), length 56)
10.0.3.2.55012 > 8.8.8.8.53: [bad udp cksum 0x1d47 -> 0x30db!] 50918+ A? google.com. (28)
^C
2 packets captured
2 packets received by filter
0 packets dropped by kernel
Container:
root@mygui:~# dig @8.8.8.8 google.com
; <<>> DiG 9.16.50-Debian <<>> @8.8.8.8 google.com
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached
root@mygui:~# nslookup google.com
Server: 127.0.0.53
Address: 127.0.0.53#53
** server can't find google.com: SERVFAIL
root@mygui:~# resolvectl status
Global
Protocols: +LLMNR +mDNS -DNSOverTLS DNSSEC=no/unsupported
resolv.conf mode: stub
Current DNS Server: 1.1.1.1
DNS Servers 8.8.8.8 1.1.1.1
DNS Domain ~.
Link 2 (eth0)
Current Scopes: DNS LLMNR/IPv4 LLMNR/IPv6
Protocols: +DefaultRoute +LLMNR -mDNS -DNSOverTLS DNSSEC=no/unsupported
Current DNS Server: 8.8.8.8
DNS Servers: 8.8.8.8
root@mygui:~# resolvectl query google.com
google.com: resolve call failed: Lookup failed due to system error: No route to host
root@mygui:~# host -t A google.com 8.8.8.8
;; connection timed out; no servers could be reached
root@mygui:~# nmcli device show | cat
GENERAL.DEVICE: eth0
GENERAL.TYPE: ethernet
GENERAL.HWADDR: 76:D6:C4:85:20:EC
GENERAL.MTU: 1500
GENERAL.STATE: 100 (connected (externally))
GENERAL.CONNECTION: eth0
GENERAL.CON-PATH: /org/freedesktop/NetworkManager/ActiveConnection/1
WIRED-PROPERTIES.CARRIER: on
IP4.ADDRESS[1]: 10.0.3.2/24
IP4.GATEWAY: 10.0.3.1
IP4.ROUTE[1]: dst = 0.0.0.0/0, nh = 10.0.3.1, mt = 0
IP4.ROUTE[2]: dst = 10.0.3.0/24, nh = 0.0.0.0, mt = 0
IP6.ADDRESS[1]: fe80::74d6:c4ff:fe85:20ec/64
IP6.GATEWAY: --
IP6.ROUTE[1]: dst = fe80::/64, nh = ::, mt = 256
GENERAL.DEVICE: lo
GENERAL.TYPE: loopback
GENERAL.HWADDR: 00:00:00:00:00:00
GENERAL.MTU: 65536
GENERAL.STATE: 10 (unmanaged)
GENERAL.CONNECTION: --
GENERAL.CON-PATH: --
IP4.ADDRESS[1]: 127.0.0.1/8
IP4.GATEWAY: --
IP6.ADDRESS[1]: ::1/128
IP6.GATEWAY: --
root@mygui:~# nmcli connection
NAME UUID TYPE DEVICE
eth0 eeac1bb5-a134-4e39-927b-5b0f94fd139b
ethernet eth0
Wired connection 1 6cb715e3-5ee7-36ac-ad90-aa185ce238b4 ethernet --