I spent several head scratching months working out how best to meet my goals. Forewarning, I can be dense, if I break out the big crayons when explaining something it’s because I confidently don’t understand it. If I gloss over something, it’s because I don’t even understand how I don’t understand it, lol. Please note that I am self-taught in several IT areas, check out my profile for details if that matters to your responses.
Goals
I want to be able to self host several vlan capable subnets from a single IP; then cluster multiple like devices from one or more IPs … and I eventually landed on LXD due to it’s capabilities as they related to my goals.
Spoiler: thus far everything works, with one peculiarity, hence this post.
Capabilities
- LXD can host vlan capable subnets from a single IP via OVN;
- LXD can cluster which is made more attractive with MicroCloud; but waiting on OVN integration.
- LXD project’s can have their own OVN network; this is a very attractive feature.
- LXD purports to work with JuJu; I imagine the developers wouldn’t lie about this, right…
@stgraber and team, you all are making a great product.
Controls
- Host: Ubuntu 22.04, LXD 5.9
- Containers: Ubuntu 22.04
- Network: IPV4 only, bridge, uplink pointing back to the bridge (@tomp advised in some random post), ovn subnets. vlan capability is untested.
- Applications: neovim, nginx, ufw
Implementation
I have OVN setup with the uplink pointing to the lxdbr0
bridge. I have placed a reverse proxy container on the default (lxdbr0) bridge network. For the reverse proxy container, I used the instructions that @simos indicated here excepting that I am self hosting and I am only using Nginx; none of that Apache2 slowness, . I have my reverse proxy container attached to two different networks lxdbr0 (bridge) and ovntest (ovn) via separate nics. The ovntest network is associated with project-test. The test website is up and currently running on the internet with SSL as provided by @simos instructions; however due to the nature of OVN, I encountered two different issues:
- From within the containers, Nginx consistently failed on
nginx -t
where-ever I had anything set to<containername>.lxd
as provided in the @simos instructions. The work around discovered that where-ever the instructions provided the use ofcontainername.lxd
, I replaced that entry with the containers IPV4 address. Which for peace of mind meant manually setting the container’s IP from the host via:
lxc config set <containername> --project=<projectname> ipv4.address=<ip of container>
- Once all the IP addresses aligned; I was able to reach my test website container as anticipated, but I then encountered my second OVN related issue. While the web-server and reverse-proxy worked functioned for their intended purpose; I lost all other internet connectivity. Which meant no updates were possible and it that posed security risks, and I wasn’t able to install certbot as the @simos instructions provided. I was able to alleviate this issue by running the following command on the host:
lxc network set uplink dns.nameservers <ip of lxdbr0>
After which, all connectivity returned to the subnets and their associated containers; and everything else proceeded as intended. Life is good.
TL;DR
The one issue that persists is that in order to get the reverse proxy container to connect to all the subnets, I have to run dhclient
from within the reverse proxy container after each machine restart. Of course, I can automate a work around from within reverse proxy container, but I didn’t know if there was a configuration setting that I missed. Please advise.
Thanks for your responses,
shimmy
Merry Christmas and Happy Holidays.