Hi: Docker has an easy way to share network namespace using network_mode (docker compose). The result is both containers will share the same exact network route (ip r) and assume will also share the same firewall rules.
Is there a way to do the same in Lxc?
If not, what alternatives would you suggest?
LXD doesn’t provide that feature as it would almost never work since we run full Linux distributions in our containers and they usually reset all networking on boot which would interfere with your other container.
If you’re using low-level LXC though (lxc-* tools), then yes, we do allow it.
https://linuxcontainers.org/lxc/manpages//man1/lxc-start.1.html
Look for the share-net option.
Thank you so much for responding and I will look into the lower level Lxc. I read an article that suggested you should use either Lxc or Lxd user interface and not both.
Do you agree?
Maybe I should be a little more specific on the use case before deciding on a path. I want to use a local dns server on home network to resolve dns and pass to haproxy to route to a lxc container on lan including a VPN container. Options:
1- Have a stand alone Vpn container and share with other containers. I thought sharing the network namespace would be a good way to do this, but seems I would need the lower level lxc from your earlier post.
2- Install openvpn directly in the container for any apps needing vpn access. I tried this but when vpn active could not reach apps from Haproxy which I am sure is a network issue, but could not resolve. Any ideas? While not as flexible as option 1., this would probably be fine for my needs at this point and I would have confidence apps correctly going though Vpn and firewall rules working correctly.
3- Using lxd to create a vpn network using the ip of the vpn container. I saw some posts suggesting this is a viable solution, but could not get this to work. Would this solution function the sames as sharing network namespace?
I could go back to a Docker solution but like that lxc runs directly off the linux kernel and do not have to rely on unknown docker images from the docker hub.
Thanks again for any help.
I think we’d need more info to help, as there are many ways to configure a solution.
You mentioned a home network, with a local dns server and haproxy, but then you started talking about VPN, but its not clear where that comes into it? Also, where is haproxy in relation to this, and is the IP being resolved for it an external IP or internal IP?
Perhaps on a higher level, it would be useful to understand what you’re trying to achieve (e.g. host multiple websites behind a NAT router?), as well as the specifics?
I have a standard 192.x home network and wanted to use Lxc to host some services with the ability to reach these services from other devices on the home network (tablets, and other pcs) without having to remember the ips and port numbers. Also, I did not want to have all 192.x ips for every lxc service since I had problems obtaining 192.x ip for lxc containers with virtual box as host for ubuntu 20.04.
First- Using the routed option I created a dns masq container with a 192.x address to be reachable on lan with dns entries for each service (service1.home) with the ip address being a separate 192.x Haproxy container also using the routed option.
Second- Using Haproxy as a reverse proxy it directs traffic to the appropriate lxc container and port on the lxdbr0 with the 10.x ips.
All this seemed to work fine when I used ip address of the dns masq server on my tablet, I was able to reach the service using http://service1.home
The problem I ran into is how can I route traffic reliably through a VPN for some services without lossing connectivity to those services on my Lan. Not sure how to setup networking to allow this. Do I need some routes or iptable rules with Vpn to allow traffic from lan? While I have learned a lot, this networking stuff is not exactly simple.
I am certainly open to a simpler solution if my approach is all wrong.
Thanks for any suggestions.
So I think the easiest approach would be to use an unmanaged bridge inside the virtualbox that is running your LXD service which connects to the VMs external interface that is in turn connected to your actual home network.
The issue you’ll face is that virtualbox by default prevents multiple MACs on a VMs NIC, so you need to ensure that the VM has that option enabled. See https://www.nakivo.com/blog/virtualbox-network-setting-guide/ and promiscuous mode on the bridge adapter.
Once you’ve done that you can then setup an unmanaged bridge called br0 connected to your VMs network interface in the Ubuntu VM, using, for example, netplan:
https://netplan.io/examples/#configuring-network-bridges
Then you can create your instances with NICs that connect to that bridge (either by adding NIC devices to the instance or setting up a profile with it):
lxc init images:ubuntu/focal c1
lxc config device add c1 eth0 parent=br0
lxc start c1
This will then connect your instance to your internal network, and it will perform DHCP just like any other device on your network. You can optionally go into the instance and modify its config to assign static IPs if you like, or modify your network’s DHCP server to assign a static IP to the instance’s MAC address.
Now you need to resolve (excuse the pun) the DNS issue. For this you can use any existing DNS server you use to add entries for your LXD instances, or you can setup a separate instance that runs a DNS server as you suggested earlier.
As for the VPN, I’m still not sure how that somes into it, do you mean a network level VPN on your router/gateway, or a service running on your PC?
Tomp- I want to thank you for taking the time for writing a very detailed and extensive response. I agee this may of gotten a little of the track when I started using routed instead of the normal dhcp process with the br0.
Before I ask any other questions, I want review everything and see if I can get the br0 process working.
I have to say as new user of both virtual box and lxc all the the networking complexities were less than apparent.
I enjoy the learning process although frustrating at times and definitely appreciate your guidance.
I finally got the Br0 to work and dhcp assign the 192.x ip from router. When I went back and checked my virtual box settings promiscuous mode was correct with “allow all” . Only other setting choice was adapter type which was set to Intel Pro/1000 which I change to PCnet Fast III.
Instead of configuring the Br0 using nmcli, I used netplan. The result looked the same with ifconfig showing that br0 now had the correct ip.
I restarted the container and whoola I had the 192.x assigned to the new test container I created. I believe changing the network adapter resolved the issue.
I am going to redeploy the containers with the Br0 and see where I am at with the VPN.