LXD macvlan wont work

LXD
1

This will be long …
I have a dedicated server with 1 public IP, also I bought 2 more public IP with virtual MAC address (it can be generated at providers webpage).
First time when I made a setup of 2 containers with those later bought IP-s worked fine about 1 month, then came a restart and the containers never started again. Some device was missing.
I purged, reinstalled LXD, also tried with snap more than 20 times … could not made to work again.
Now from container I can’t ping anything, and it can’t be reached from public side, although it worked (one month with this config, until reboot).
What am I doing wrong ???

This is what I have now:

lxc profile list

+---------+---------+
|  NAME   | USED BY |
+---------+---------+
| default | 0       |
+---------+---------+
| macvlan | 1       |
+---------+---------+

lxc list

+-------+---------+---------------------+------+------------+-----------+
| NAME  |  STATE  |        IPV4         | IPV6 |    TYPE    | SNAPSHOTS |
+-------+---------+---------------------+------+------------+-----------+
| w3cos | RUNNING | 51.77.83.244 (eth0) |      | PERSISTENT | 0         |
+-------+---------+---------------------+------+------------+-----------+

lxc profile show macvlan

config: {}
description: Default LXD profile
devices:
  eth0:
    name: eth0
    nictype: macvlan
    parent: eno1
    type: nic
  root:
    path: /
    pool: newspool
    type: disk
name: macvlan
used_by:
- /1.0/containers/w3cos

lxc config show --expanded w3cos

architecture: x86_64
config:
  image.architecture: amd64
  image.description: ubuntu 18.04 LTS amd64 (release) (20200519.1)
  image.label: release
  image.os: ubuntu
  image.release: bionic
  image.serial: "20200519.1"
  image.version: "18.04"
  volatile.base_image: 70d3dcaabcffb1aa1644d0ce866efcb141742179e94ad72aefb8d3502338a71f
  volatile.eth0.hwaddr: 02:00:00:81:0e:ec
  volatile.idmap.base: "0"
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536}, 
{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":100000,"Nsid":0,"Maprange":65536}, 
{"Isuid":false,"Isgid":true,"Hostid":100000,"Nsid":0,"Maprange":65536}]'
  volatile.last_state.power: RUNNING
devices:
  eth0:
    name: eth0
    nictype: macvlan
    parent: eno1
    type: nic
  root:
    path: /
    pool: newspool
    type: disk
ephemeral: false
profiles:
- macvlan
stateful: false
description: ""

In container:
cat /etc/netplan/50-cloud-init.yaml

network:
	version: 2
	ethernets:
		eth0:
			dhcp4: no
			dhcp6: no
			addresses:
				- 51.77.83.244/32
			#gateway4: 54.36.111.254
			nameservers:
			   addresses:
				- 213.186.33.99
				- 8.8.8.8
			routes:
			   - to: 0.0.0.0/0
				 via: 51.77.83.244
				 on-link: true

In host:
ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
	link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
	inet 127.0.0.1/8 scope host lo
	   valid_lft forever preferred_lft forever
	inet6 ::1/128 scope host 
	   valid_lft forever preferred_lft forever
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
	link/ether a4:bf:01:2e:d7:46 brd ff:ff:ff:ff:ff:ff
	inet 54.36.111.158/24 brd 54.36.111.255 scope global eno1
	   valid_lft forever preferred_lft forever
	inet6 2001:41d0:700:119e::/64 scope global 
	   valid_lft forever preferred_lft forever
	inet6 fe80::a6bf:1ff:fe2e:d746/64 scope link 
	   valid_lft forever preferred_lft forever
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
	link/ether a4:bf:01:2e:d7:47 brd ff:ff:ff:ff:ff:ff
2

Does your provider require that those additional IPs always map to a specific MAC address?

3

I think yes, and as I said it worked fine 1 month until reboot.
BTW my setup is partially made as it is shown here:

Here is the providers user panel, where I can edit my setup:

ovh
ovh1470×499 21.4 KB

4

I will buy a gun and a bullet if it is a firewall problem, according to this document:
https://lxd.readthedocs.io/en/latest/networks/
I would appreciate any help or link about how to set up the firewall on host machine with lxd macvlan container in my case … meaning with OVH’s virtual mac and failoverIP.

5

Why do you have your default gateway commented out in netplan?

#gateway4: 54.36.111.254

And apparently you have a custom default route defined using “0.0.0.0/0” but the target is 51.77.83.244 rather than 51.77.83.254 in your commented out default gateway line.

Also, if your ISP requires your IP aliases (macvlans) to use a specific MAC address then you will need to be specifying them by setting the hwaddr setting on the container’s NIC using:

lxc config device set <container> eth0 hwaddr=<mac address>
6

About “#gateway4”:
I made a snapshot and backup from all of the config files before server restart. With that setup worked, there was no need for gateway, also the internet was reachable through 51.77.83.254.
About “hwaddr”:
I set up volatile.eth0.hwaddr: 02:00:00:81:0e:ec

BTW:
I deleted everithing again and made a new setup. Finally it’s working as I expected.
Soon will publish the how to.

1 Like
7

You should avoid manually modifying the volatile keys, these are used by LXD internally, and there meaning/usage can change without notification.

Instead please use the documented keys here:

https://linuxcontainers.org/lxd/docs/master/instances#nictype-bridged

In this case hwaddr is the one you want to set a static MAC address.

1 Like
8

So … as I could not recover nor recreate the containers as they worked at first time, had to find another solution.

  • Virtual MAC’s:
    Did not worked added with:
    lxc config device set mycontainer eth0 hwaddr a:new:mac:address
    neither:
    lxc config set mycontainer volatile.eth0.hwaddr
    (as worked first time).
    So I deleted virtual MAC addresses at OVH’s (provider) user panel.

The solution is picked up from several posts:

  • created virtual bridge for inner communication betwen host and containers
  • also added (bought) public IP addresses to containers

Here are the referral links:

9

Thank you for pointing this out, because there are many posts and Howtos out there, which use the wrong (?) syntax for setting MAC address ( lxc config set myct volatile.eth0.hwaddr xx:yy:zz:00:11 )

Perhaps setting MAC address with volatile.eth0.hwaddr was the reason for the very strange problem I encountered a few days ago: I couldn’t get networking to work on a newly created ct02 with macvlan, replacing ct01 which I had just deleted. A couple of days later, I deleted ct02 and a new created ct03, and this time running the very same lxc commands magically worked! (wrt networking)

1 Like