My issue is that I’m only able to reach a container with two network devices from Internet using port forwarding on the router if I use a specific order of the defined network devices. The macvlan device has to come first.
I would like to set up a number of web services using LXD. My idea is to use one container as an HTTP reverse proxy (defining several vhosts in apache and handle all SSL) and then install various services such as NextCloud, MythWeb and ZoneMinder in separate containers. Communication between the containers could use the default lxdbr0 and the reverse proxy should communicate with the outside world using macvlan.
After reading a few topics here, and especially the following two great replies by @stgraber, I thought I understood a bit more about how a macvlan setup works:
I use the following profile to create the basics of the reverse proxy container:
config: environment.http_proxy: "" user.network-config: | version: 1 config: - type: physical name: eth0 subnets: - type: dhcp ipv4: true control: auto - type: physical name: eth1 subnets: - type: dhcp ipv4: true control: auto user.network_mode: "" user.user-data: | #cloud-config package_upgreade: true packages: - apache2 timezone: Europe/Stockholm description: Default LXD profile devices: eth0: name: eth0 nictype: bridged parent: lxdbr0 type: nic eth1: name: eth1 nictype: macvlan parent: eno1 type: nic root: path: / pool: default type: disk name: proxy
This works fine on my local network. The container obtains an IP address via DHCP on the LAN (eth1) and is also connected to the lxdbr0 network (eth0 - and assigned an IP address there as well). When accessing it from another computer on the LAN it shows the default Ubuntu/Apache web page.
In order to reach the container from the Internet, I added port forwarding in my router, an Ubiquity EdgeRouter X, forwarding port 80 on the “Internet interface” to port 80 on the (eth1) IP address of the container. Still I’m unable to access it from the Internet.
Testing / Trouble Shooting
To make sure port forwarding is at all working as intended, I installed Apache on a local machine on the same LAN and reconfigured port forwarding to point to that IP address instead. Now I could see the default web page when accessing my domain from the Internet. I then created another container with a single network device configured as the macvlan device above. It got its own IP address from the DHCP server on the LAN and after reconfiguring port forwarding again I could reach this container from the Internet.
I then created a copy of the proxy profile above where I changed the order of the network devices so that the macvlan device is listed first as eth0 and lxdbr0 is second as eth1:
... devices: eth0: name: eth0 nictype: macvlan parent: eno1 type: nic eth1: name: eth1 nictype: bridged parent: lxdbr0 type: nic ...
To my surprise I could now reach this container from the Internet (after reconfiguring port forwarding again of course).
To me this behavior seems strange. Am I missing something or have I stumbled upon something strange. Could this come from limitations/bugs in the (single) NIC on my host (eno1 in the config above) or its kernel modules (I understood that “mileage may vary” based on one of the post linked above)?
- My host runs Ubuntu 16.04 as do my containers tested here.
- I don’t use VLANs on my network but I do know my router is VLAN “aware”. I think the switch on my LAN is also capable of VLAN tagging, but this functionality is not enabled.
- I run LXD 3.0.1 (Xenial backports I think)