Network Issue when calling lxc commands

Scott · June 2, 2017, 2:12am

Running LXD on ubuntu 16.04
I have setup my bridge and zfs
Assigned zfs pool and br0 during lxd init
Created first container lxc init ubuntu: container_name
In /ect/default/lxd-bridge -> confirmed that lxd-bridge is br0
Added ipv4 info for container networking
Added lxd-dnsmasq.conf for static IP assingment for container
Here is my issue:
If I start the container -> host is unreachable over network (ssh/ping)
I log in locally, shut down container and restart networking -> host is back online.
If I run lxc list -> I lose connectivity to host. (with container stopped) cannot ping/ssh
Have to restart networking

If I run ifconfig everything looks correct. br0 is online w/ correct LAN settings.

Any help would be appreciated.

rkelleyrtp · June 2, 2017, 2:19am

So, you are giving your container a static IPv4 address? What is the IP of the container? What is the IP of the Ubuntu host?

Scott · June 2, 2017, 12:18pm

The IPs are not the same 2.10 and 2.15 same network

so when the problem occurs br0 loses its ipv4 network information and is replaced w/ ipv6 info
restarting network fixes the problem. So when I start a container or even ask lxc for info it buts eno1 and br0 into the disabled state

what in lxd could be doing this? I removed the LXD_IPV4_ADDR in below (had the host there 2.10 but issue still occurs)

/etc/default/lxd-bridge
USE_LXD_BRIDGE=“true”
LXD_BRIDGE=“br0”
UPDATE_PROFILE=“true”
LXD_CONFILE="/etc/default/lxd_dnsmasq.conf"
LXD_DOMAIN=“lxd”
LXD_IPV4_ADDR=""
LXD_IPV4_NETMASK=“255.255.255.0”
LXD_IPV4_NETWORK=“192.168.5.1/24”
LXD_IPV4_DHCP_RANGE=“192.168.5.215,192.168.5.225”
LXD_IPV4_DHCP_MAX=“10”
LXD_IPV4_NAT=“true”
LXD_IPV6_ADDR=""
LXD_IPV6_MASK=""
LXD_IPV6_NETWORK=""

The primary network interface

#auto eno1
#iface eno1 inet static
auto br0
iface br0 inet static
address 192.168.5.210
netmask 255.255.255.0
network 192.168.5.0
broadcast 192.168.5.255
gateway 192.168.5.100
# dns-* options are implemented by the resolvconf package, if installed
dns-nameservers 192.168.5.100
#bridge options
bridge_ports eno1

lxd_dnsmasq.conf

dhcp-host=container_name,192.168.5.215

/var/log/lxd/container_name
lxc.cap.drop = sys_time sys_module sys_rawio
lxc.mount.auto = proc:rw sys:rw
lxc.autodev = 1
lxc.pts = 1024
lxc.mount.entry = /dev/fuse dev/fuse none bind,create=file,optional
lxc.mount.entry = /dev/net/tun dev/net/tun none bind,create=file,optional
lxc.mount.entry = /proc/sys/fs/binfmt_misc proc/sys/fs/binfmt_misc none rbind,create=dir,optional
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none rbind,create=dir,optional
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none rbind,create=dir,optional
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none rbind,create=dir,optional
lxc.mount.entry = /sys/kernel/security sys/kernel/security none rbind,create=dir,optional
lxc.mount.entry = /dev/mqueue dev/mqueue none rbind,create=dir,optional
lxc.include = /usr/share/lxc/config/common.conf.d/
lxc.logfile = /var/log/lxd/container_name/lxc.log
lxc.loglevel = warn
lxc.arch = linux64
lxc.hook.pre-start = /usr/bin/lxd callhook /var/lib/lxd 1 start
lxc.hook.post-stop = /usr/bin/lxd callhook /var/lib/lxd 1 stop
lxc.tty = 0
lxc.utsname = container_name
lxc.mount.entry = /var/lib/lxd/devlxd dev/lxd none bind,create=dir 0 0
lxc.aa_profile = lxd-container_name_</var/lib/lxd>//&:lxd-container_name_:
lxc.seccomp = /var/lib/lxd/security/seccomp/container_name
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.hwaddr = 00:16:3e:7e:35:e3
lxc.network.name = eth0
lxc.rootfs.backend = dir
lxc.rootfs = /var/lib/lxd/containers/container_name/rootfs
lxc.mount.entry = /var/lib/lxd/shmounts/container_name dev/.lxd-mounts none bind,create=dir 0 0

sudo lxc profile show default
config:
user.network_mode: link-local
description: Default LXD profile
devices:
eth0:
name: eth0
nictype: bridged
parent: br0
type: nic
name: default

rkelleyrtp · June 2, 2017, 2:07pm

Do you have the same experience (network connectivity loss) if you use a DHCP address?

Scott · June 2, 2017, 2:25pm

yes. Removed dnsmasq.conf file from lxd-bridge
restarted host
after reboot lxc container is stopped.
I have connectivity to host
I start container
1 - it does not get any IP address ipv4 / 6
2 - host ifconfig gets changes br0 does not contain static IP information just ipv6
host is offline
I stop container and restart networking.
br0 gets ipv4 info back and host back online

Scott · June 2, 2017, 2:33pm

Syslog ->

Jun 2 10:20:21 ubuntu_host systemd[1]: Started User Manager for UID 1000.
Jun 2 10:20:51 ubuntu_host systemd[1]: Starting LXD - network bridge…
Jun 2 10:20:51 ubuntu_host kernel: [ 268.779197] ip_tables: © 2000-2006 Netfilter Core Team
Jun 2 10:20:51 ubuntu_host kernel: [ 268.787159] br0: port 1(eno1) entered disabled state
Jun 2 10:20:51 ubuntu_host kernel: [ 268.807963] nf_conntrack version 0.5.0 (65536 buckets, 262144 max)
Jun 2 10:20:52 ubuntu_host kernel: [ 268.824131] ip6_tables: © 2000-2006 Netfilter Core Team
Jun 2 10:20:52 ubuntu_host kernel: [ 268.834412] br0: port 1(eno1) entered forwarding state
Jun 2 10:20:52 ubuntu_host kernel: [ 268.834427] br0: port 1(eno1) entered forwarding state
Jun 2 10:20:52 ubuntu_host systemd[1]: Started LXD - network bridge.
Jun 2 10:20:52 ubuntu_host systemd[1]: Starting LXD - main daemon…
Jun 2 10:20:52 ubuntu_host kernel: [ 268.888393] audit_printk_skb: 9 callbacks suppressed
Jun 2 10:20:52 ubuntu_host kernel: [ 268.888396] audit: type=1400 audit(1496413252.071:15): apparmor=“STATUS” operation=“profile_replace” profile=“unconfined” name="/usr/bin/lxc-start" pid=3552 comm=“apparmor_parser”
Jun 2 10:20:52 ubuntu_host kernel: [ 268.895232] audit: type=1400 audit(1496413252.079:16): apparmor=“STATUS” operation=“profile_replace” profile=“unconfined” name=“lxc-container-default” pid=3556 comm=“apparmor_parser”
Jun 2 10:20:52 ubuntu_host kernel: [ 268.895243] audit: type=1400 audit(1496413252.079:17): apparmor=“STATUS” operation=“profile_replace” profile=“unconfined” name=“lxc-container-default-cgns” pid=3556 comm=“apparmor_parser”
Jun 2 10:20:52 ubuntu_host kernel: [ 268.895251] audit: type=1400 audit(1496413252.079:18): apparmor=“STATUS” operation=“profile_replace” profile=“unconfined” name=“lxc-container-default-with-mounting” pid=3556 comm=“apparmor_parser”
Jun 2 10:20:52 ubuntu_host kernel: [ 268.895259] audit: type=1400 audit(1496413252.079:19): apparmor=“STATUS” operation=“profile_replace” profile=“unconfined” name=“lxc-container-default-with-nesting” pid=3556 comm=“apparmor_parser”
Jun 2 10:20:52 ubuntu_host lxd[3559]: lvl=warn msg=“CGroup memory swap accounting is disabled, swap limits will be ignored.” t=2017-06-02T10:20:52-0400
Jun 2 10:20:52 ubuntu_host lxd[3559]: lvl=warn msg=“Unable to refresh cache, using stale entry” server=https://cloud-images.ubuntu.com/releases t=2017-06-02T10:20:52-0400
Jun 2 10:20:52 ubuntu_host systemd[1]: Started LXD - main daemon.
Jun 2 10:21:02 ubuntu_host kernel: [ 279.290506] audit: type=1400 audit(1496413262.474:20): apparmor=“STATUS” operation=“profile_load” profile=“unconfined” name=“lxd-container_name_</var/lib/lxd>” pid=3672 comm=“apparmor_parser”
Jun 2 10:21:02 ubuntu_host systemd-udevd[3678]: Could not generate persistent MAC address for veth872I25: No such file or directory
Jun 2 10:21:02 ubuntu_host kernel: [ 279.310190] device vethDDSRS6 entered promiscuous mode
Jun 2 10:21:02 ubuntu_host kernel: [ 279.310277] IPv6: ADDRCONF(NETDEV_UP): vethDDSRS6: link is not ready
Jun 2 10:21:02 ubuntu_host kernel: [ 279.384701] eth0: renamed from veth872I25
Jun 2 10:21:02 ubuntu_host kernel: [ 279.408645] IPv6: ADDRCONF(NETDEV_CHANGE): vethDDSRS6: link becomes ready
Jun 2 10:21:02 ubuntu_host kernel: [ 279.408692] br0: port 2(vethDDSRS6) entered forwarding state
Jun 2 10:21:02 ubuntu_host kernel: [ 279.408706] br0: port 2(vethDDSRS6) entered forwarding state
Jun 2 10:21:02 ubuntu_host systemd[1]: proc-sys-fs-binfmt_misc.automount: Got automount request for /proc/sys/fs/binfmt_misc, triggered by 3685 (lxd)
Jun 2 10:21:02 ubuntu_host systemd[1]: Mounting Arbitrary Executable File Formats File System…
Jun 2 10:21:02 ubuntu_host systemd[1]: Mounted Arbitrary Executable File Formats File System.
Jun 2 10:21:03 ubuntu_host kernel: [ 279.944924] audit: type=1400 audit(1496413263.130:21): apparmor=“STATUS” operation=“profile_load” label=“lxd-container_name_</var/lib/lxd>//&:lxd-container_name_://unconfined” name="/usr/bin/lxc-start" pid=3960 comm=“apparmor_parser”
Jun 2 10:21:03 ubuntu_host kernel: [ 279.945265] audit: type=1400 audit(1496413263.130:22): apparmor=“STATUS” operation=“profile_load” label=“lxd-container_name_</var/lib/lxd>//&:lxd-container_name_://unconfined” name="/usr/lib/snapd/snap-confine" pid=3962 comm=“apparmor_parser”
Jun 2 10:21:03 ubuntu_host kernel: [ 279.945279] audit: type=1400 audit(1496413263.130:23): apparmor=“STATUS” operation=“profile_load” label=“lxd-container_name_</var/lib/lxd>//&:lxd-container_name_://unconfined” name="/usr/lib/snapd/snap-confine//mount-namespace-capture-helper" pid=3962 comm=“apparmor_parser”
Jun 2 10:21:03 ubuntu_host kernel: [ 279.947502] audit: type=1400 audit(1496413263.130:24): apparmor=“STATUS” operation=“profile_load” label=“lxd-container_name_</var/lib/lxd>//&:lxd-container_name_://unconfined” name="/usr/lib/lxd/lxd-bridge-proxy" pid=3961 comm=“apparmor_parser”
Jun 2 10:21:03 ubuntu_host kernel: [ 279.948325] audit: type=1400 audit(1496413263.134:25): apparmor=“STATUS” operation=“profile_load” label=“lxd-container_name_</var/lib/lxd>//&:lxd-container_name_://unconfined” name="/usr/sbin/tcpdump" pid=3964 comm=“apparmor_parser”
Jun 2 10:21:03 ubuntu_host kernel: [ 279.949398] audit: type=1400 audit(1496413263.134:26): apparmor=“STATUS” operation=“profile_load” label=“lxd-container_name_</var/lib/lxd>//&:lxd-container_name_://unconfined” name=“lxc-container-default” pid=3958 comm=“apparmor_parser”
Jun 2 10:21:03 ubuntu_host kernel: [ 279.949410] audit: type=1400 audit(1496413263.134:27): apparmor=“STATUS” operation=“profile_load” label=“lxd-container_name_</var/lib/lxd>//&:lxd-container_name_://unconfined” name=“lxc-container-default-cgns” pid=3958 comm=“apparmor_parser”
Jun 2 10:21:03 ubuntu_host kernel: [ 279.949418] audit: type=1400 audit(1496413263.134:28): apparmor=“STATUS” operation=“profile_load” label=“lxd-container_name_</var/lib/lxd>//&:lxd-container_name_://unconfined” name=“lxc-container-default-with-mounting” pid=3958 comm=“apparmor_parser”
Jun 2 10:21:03 ubuntu_host kernel: [ 279.949427] audit: type=1400 audit(1496413263.134:29): apparmor=“STATUS” operation=“profile_load” label=“lxd-container_name_</var/lib/lxd>//&:lxd-container_name_://unconfined” name=“lxc-container-default-with-nesting” pid=3958 comm=“apparmor_parser”
Jun 2 10:21:07 ubuntu_host kernel: [ 283.880240] br0: port 1(eno1) entered forwarding state
Jun 2 10:21:17 ubuntu_host kernel: [ 294.440910] br0: port 2(vethDDSRS6) entered forwarding state
Jun 2 10:21:52 ubuntu_host kernel: [ 328.975694] br0: port 2(vethDDSRS6) entered disabled state
Jun 2 10:21:52 ubuntu_host kernel: [ 328.975924] device vethDDSRS6 left promiscuous mode
Jun 2 10:21:52 ubuntu_host kernel: [ 328.975927] br0: port 2(vethDDSRS6) entered disabled state
Jun 2 10:21:52 ubuntu_host kernel: [ 329.746137] audit_printk_skb: 18 callbacks suppressed
Jun 2 10:21:52 ubuntu_host kernel: [ 329.746141] audit: type=1400 audit(1496413312.930:36): apparmor=“STATUS” operation=“profile_remove” profile=“unconfined” name=“lxd-container_name_</var/lib/lxd>” pid=4228 comm=“apparmor_parser”
Jun 2 10:22:02 ubuntu_host systemd[1]: Stopping Raise network interfaces…
Jun 2 10:22:03 ubuntu_host ifdown[4302]: RTNETLINK answers: No such process
Jun 2 10:22:03 ubuntu_host ifdown[4302]: RTNETLINK answers: Cannot assign requested address
Jun 2 10:22:03 ubuntu_host kernel: [ 339.874468] br0: port 1(eno1) entered disabled state
Jun 2 10:22:03 ubuntu_host kernel: [ 339.883073] ixgbe 0000:01:00.0: removed PHC on eno1
Jun 2 10:22:03 ubuntu_host kernel: [ 340.206325] device eno1 left promiscuous mode
Jun 2 10:22:03 ubuntu_host kernel: [ 340.206353] br0: port 1(eno1) entered disabled state
Jun 2 10:22:03 ubuntu_host systemd[1]: Stopping ifup for br0…
Jun 2 10:22:03 ubuntu_host ifdown[4362]: /sbin/ifdown: waiting for lock on /run/network/ifstate.br0
Jun 2 10:22:03 ubuntu_host ifdown[4362]: /sbin/ifdown: interface br0 not configured
Jun 2 10:22:03 ubuntu_host systemd[1]: Stopped ifup for br0.
Jun 2 10:22:03 ubuntu_host systemd[1]: Stopped Raise network interfaces.
Jun 2 10:22:03 ubuntu_host systemd[1]: Starting Raise network interfaces…
Jun 2 10:22:03 ubuntu_host systemd-udevd[4394]: Could not generate persistent MAC address for br0: No such file or directory
Jun 2 10:22:03 ubuntu_host kernel: [ 340.313322] device eno1 entered promiscuous mode
Jun 2 10:22:03 ubuntu_host systemd[1]: Started ifup for br0.
Jun 2 10:22:03 ubuntu_host systemd[1]: Found device /sys/subsystem/net/devices/br0.
Jun 2 10:22:03 ubuntu_host sh[4433]: ifup: waiting for lock on /run/network/ifstate.br0
Jun 2 10:22:03 ubuntu_host kernel: [ 340.420710] pps pps0: new PPS source ptp0
Jun 2 10:22:03 ubuntu_host kernel: [ 340.420715] ixgbe 0000:01:00.0: registered PHC device on eno1
Jun 2 10:22:03 ubuntu_host kernel: [ 340.543533] br0: port 1(eno1) entered forwarding state
Jun 2 10:22:03 ubuntu_host kernel: [ 340.543542] br0: port 1(eno1) entered forwarding state
Jun 2 10:22:03 ubuntu_host ifup[4382]: Waiting for br0 to get ready (MAXWAIT is 32 seconds).
Jun 2 10:22:04 ubuntu_host systemd[1]: Reloading OpenBSD Secure Shell server.
Jun 2 10:22:04 ubuntu_host systemd[1]: Reloaded OpenBSD Secure Shell server.
Jun 2 10:22:04 ubuntu_host kernel: [ 340.873528] br0: port 1(eno1) entered disabled state
Jun 2 10:22:04 ubuntu_host sh[4433]: ifup: interface br0 already configured
Jun 2 10:22:04 ubuntu_host systemd[1]: Started Raise network interfaces.
Jun 2 10:22:07 ubuntu_host kernel: [ 343.878492] ixgbe 0000:01:00.0 eno1: NIC Link is Up 1 Gbps, Flow Control: None
Jun 2 10:22:07 ubuntu_host kernel: [ 343.878743] br0: port 1(eno1) entered forwarding state
Jun 2 10:22:07 ubuntu_host kernel: [ 343.878752] br0: port 1(eno1) entered forwarding state

stgraber · June 2, 2017, 2:37pm

The problem is that you’re telling both your Linux distribution and lxd-bridge to configure the exact same interface (br0).
When lxd-bridge starts (whenever you first talk to LXD), it will therefore try to configure the br0 interface which already exists, causing your loss of connectivity.

This kind of setup is very much not recommended and I have no idea whether it can even work, but you at least have one obvious configuration problem in your /etc/default/lxd-bridge.
“LXD_IPV4_ADDR” should be “192.168.5.210” and “LXD_IPV4_NETWORK” should be “192.168.5.210/24”, that should at least prevent your bridge’s IP from changing.

lxd-bridge is meant to manage a standalone bridge, not interfere with an existing system one. You’d probably be much better off just running a standalone dnsmasq on your existing br0 bridge and disabling lxd-bridge.

Scott · June 2, 2017, 2:48pm

Thanks for the info
So what is recommended?
Create a new lxdbr0 and make parent br0?
Or disable lxd-bridge via USE_LXD_BRIDGE=“False” and configure dnsmasq on br0 or does it not matter?

stgraber · June 2, 2017, 3:14pm

I’d strongly recommend the latter.

I don’t believe you can actually bridge bridges, you’d need to use macvlan in between and you may end up with rather weird issues if you do that, so disabling lxd-bridge, telling LXD to use your existing bridge and having dnsmasq manually configured on that bridge is preferred in your case.

Scott · June 2, 2017, 3:54pm

Still not getting IPv4 to container
LXD_CONFILE="/etc/dnsmaq.conf"
where I have dhcp-host=id:mycontainer,192.168.5.215
What am I missing? -> and thanks for the help
Name: my_container
Remote: unix:/var/lib/lxd/unix.socket
Architecture: x86_64
Created: 2017/06/01 18:52 UTC
Status: Running
Type: persistent
Profiles: default
Pid: 3664
Ips:
eth0: inet6 fe80::216:3eff:fe7e:35e3 vethIPHDNO
lo: inet 127.0.0.1
lo: inet6 ::1
Resources:
Processes: 26
Disk usage:
root: 12.98MB
Memory usage:
Memory (current): 80.51MB
Memory (peak): 117.10MB
Network usage:
eth0:
Bytes received: 56.32kB
Bytes sent: 8.31kB
Packets received: 393
Packets sent: 31
lo:
Bytes received: 9.46kB
Bytes sent: 9.46kB
Packets received: 116
Packets sent: 116

stgraber · June 2, 2017, 3:57pm

Can you show:

/etc/network/interfaces
/etc/dnsmasq.conf
ps aux | grep dnsmasq
ifconfig -a
lxc profile show default
lxc config show --expanded my_container

That should give me a better idea of the state of things.

Scott · June 2, 2017, 4:16pm

auto eno1 & iface inet eno1 static are commented out but gets lost on the formatting below

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

### The primary network interface
auto eno1
iface eno1 inet static
auto br0
iface br0 inet static
	address 192.168.5.210
	netmask 255.255.255.0
	network 192.168.5.0
	broadcast 192.168.5.255
	gateway 192.168.5.100
	# dns-* options are implemented by the resolvconf package, if installed
	dns-nameservers 192.168.5.100
	#bridge options
	bridge_ports eno1

In /etc/dnsmasq.conf I have to lines enabled:

server=/localnet/192.168.5.210
dhcp-host=id:my_container,192.168.5.215

ps aux

dnsmasq   3250  0.0  0.0  52868  2944 ?        S    12:00   0:00 /usr/sbin/dnsmasq -x /var/run/dnsmasq/dnsmasq.pid -u dnsmasq -r /var/run/dnsmasq/resolv.conf -7 /etc/dnsmasq.d,.dpkg-dist,.dpkg-old,.dpkg-new --local-service --trust-anchor=.,19036,8,2,49AAC11D7B6F6446702E54A1607371607A1A41855200FD2CE1CDDE32F24E8FB5
sp        4383  0.0  0.0  14224  1080 pts/0    S+   12:13   0:00 grep --color=auto dnsmasq

ifconfig -a

br0       Link encap:Ethernet  HWaddr 00:25:90:f9:50:74  
          inet addr:192.168.5.210  Bcast:192.168.5.255  Mask:255.255.255.0
          inet6 addr: fe80::225:90ff:fef9:5074/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2348 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1109 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:928225 (928.2 KB)  TX bytes:194350 (194.3 KB)

eno1      Link encap:Ethernet  HWaddr 00:25:90:f9:50:74  
          inet6 addr: fe80::225:90ff:fef9:5074/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2392 errors:0 dropped:27 overruns:0 frame:0
          TX packets:1141 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:964536 (964.5 KB)  TX bytes:196702 (196.7 KB)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:172 errors:0 dropped:0 overruns:0 frame:0
          TX packets:172 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1 
          RX bytes:12824 (12.8 KB)  TX bytes:12824 (12.8 KB)

rename3   Link encap:Ethernet  HWaddr 00:25:90:f9:50:75  
          BROADCAST MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

vethNB8V71 Link encap:Ethernet  HWaddr fe:94:eb:91:95:de  
          inet6 addr: fe80::fc94:ebff:fe91:95de/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:8 errors:0 dropped:0 overruns:0 frame:0
          TX packets:156 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:648 (648.0 B)  TX bytes:21710 (21.7 KB)

sudo lxc profile show default

config:
  user.network_mode: link-local
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: br0
    type: nic
name: default

lxc config show --expanded my_container

architecture: x86_64
config:
  user.network_mode: link-local
  volatile.base_image: 8fa08537ae51c880966626561987153e72d073cbe19dfe5abc062713d929254d
  volatile.eth0.hwaddr: 00:16:3e:45:80:86
  volatile.idmap.base: "0"
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536},{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: br0
    type: nic
  root:
    path: /
    type: disk
ephemeral: false
profiles:
- default

stgraber · June 2, 2017, 4:54pm

Reformatted your post a bit to make it more readable.

So things look good except for your dnsmasq configuration. Since you’re now running your own standalone dnsmasq, you’ll have to pass it a bunch more configuration so that it actually acts as a DHCP server.

Based on what LXD usually does, you’ll want to add something like this to your config:

strict-order
bind-interfaces
except-interface=lo
interface=br0
listen-address=192.168.5.210
dhcp-no-override
dhcp-authoritative

That won’t define a DHCP range, but hopefully that won’t prevent dnsmasq from serving your static lease.

Scott · June 2, 2017, 5:26pm

I made these changes. I can now start container w/o afffecting br0 but still does not get ipv4 config
root@hyper:/var/log/lxd/my_container# cat lxc.log
lxc 20170602170726.973 WARN lxc_start - start.c:signal_handler:322 - Invalid pid for SIGCHLD. Received pid 3380, expected pid 3390.
root@hyper:/var/log/lxd/my_container# cat lxc.conf
lxc.cap.drop = sys_time sys_module sys_rawio
lxc.mount.auto = proc:rw sys:rw
lxc.autodev = 1
lxc.pts = 1024
lxc.mount.entry = /dev/fuse dev/fuse none bind,create=file,optional
lxc.mount.entry = /dev/net/tun dev/net/tun none bind,create=file,optional
lxc.mount.entry = /proc/sys/fs/binfmt_misc proc/sys/fs/binfmt_misc none rbind,create=dir,optional
lxc.mount.entry = /sys/fs/fuse/connections sys/fs/fuse/connections none rbind,create=dir,optional
lxc.mount.entry = /sys/fs/pstore sys/fs/pstore none rbind,create=dir,optional
lxc.mount.entry = /sys/kernel/debug sys/kernel/debug none rbind,create=dir,optional
lxc.mount.entry = /sys/kernel/security sys/kernel/security none rbind,create=dir,optional
lxc.mount.entry = /dev/mqueue dev/mqueue none rbind,create=dir,optional
lxc.include = /usr/share/lxc/config/common.conf.d/
lxc.logfile = /var/log/lxd/my_container/lxc.log
lxc.loglevel = warn
lxc.arch = linux64
lxc.hook.pre-start = /usr/bin/lxd callhook /var/lib/lxd 2 start
lxc.hook.post-stop = /usr/bin/lxd callhook /var/lib/lxd 2 stop
lxc.tty = 0
lxc.utsname = my_container
lxc.mount.entry = /var/lib/lxd/devlxd dev/lxd none bind,create=dir 0 0
lxc.aa_profile = lxd-my_container_</var/lib/lxd>//&:lxd-my_container_:
lxc.seccomp = /var/lib/lxd/security/seccomp/my_container
lxc.id_map = u 0 100000 65536
lxc.id_map = g 0 100000 65536
lxc.network.type = veth
lxc.network.flags = up
lxc.network.link = br0
lxc.network.hwaddr = 00:16:3e:45:80:86
lxc.network.name = eth0
lxc.rootfs.backend = dir
lxc.rootfs = /var/lib/lxd/containers/my_container/rootfs
lxc.mount.entry = /var/lib/lxd/shmounts/my_container dev/.lxd-mounts none bind,create=dir 0 0

Jun 2 12:00:26 host systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server.
Jun 2 13:04:02 host systemd[1]: Stopping dnsmasq - A lightweight DHCP and caching DNS server…
Jun 2 13:07:24 host kernel: [ 4.601095] Key type dns_resolver registered
Jun 2 13:07:24 host systemd[1]: Starting dnsmasq - A lightweight DHCP and caching DNS server…
Jun 2 13:07:24 host dnsmasq[3097]: dnsmasq: syntax check OK.
Jun 2 13:07:25 host dnsmasq[3171]: started, version 2.75 cachesize 150
Jun 2 13:07:25 host dnsmasq[3171]: compile time options: IPv6 GNU-getopt DBus i18n IDN DHCP DHCPv6 no-Lua TFTP conntrack ipset auth DNSSEC loop-detect inotify
Jun 2 13:07:25 host dnsmasq[3171]: ignoring nameserver 192.168.5.210 - local interface
Jun 2 13:07:25 host dnsmasq[3171]: no servers found in /var/run/dnsmasq/resolv.conf, will retry
Jun 2 13:07:25 host dnsmasq[3171]: read /etc/hosts - 5 addresses
Jun 2 13:07:26 host dnsmasq[3171]: reading /var/run/dnsmasq/resolv.conf
Jun 2 13:07:26 host dnsmasq[3171]: using nameserver 192.168.5.100#53
Jun 2 13:07:26 host systemd[1]: Started dnsmasq - A lightweight DHCP and caching DNS server.

Scott · June 3, 2017, 1:04pm

I manually configured interface directly on the container. And everything is working.

stgraber · June 3, 2017, 3:54pm

Looks like you’d need to define a dhcp-range for dnsmasq to start answering DHCP queries then… When dnsmasq does DHCP you should see a bunch more stuff in your syslog.