Running OpenWrt (as a firewall/router) on a LXD host

LXD
, , ,
#1

Hi everyone,

So I’m getting started with LXD (super cool BTW, thank you all!) and to get my feet wet I decided to setup an all-in-one home-media-server/firewall doodah on a small mini-pc I had lying around, for the host OS I’m running Arch Linux (BTW :wink: ) on the latest zen kernel and for networking I’m using systemd-networkd and systemd-resolved.
Everything is working fine except for 2 things:
1- cannot pass a physical wifi device from host to container (ethernet works just fine though)
2- cannot reach the internet from the host even though host and openwrt can communicate (tested using iperf)

So here’s exactly what I’ve done so far:
:ballot_box_with_check: 1. setup a bridge device on the host to act as an internal switch

#/etc/systemd/network/br0.netdev

[NetDev]
Name=br0
Kind=bridge

#/etc/systemd/network/br0.network

[Match]
Name=br0

[Link]
MTUBytes=9000

[Network]
DHCP=yes

:ballot_box_with_check: 2. enable LXD socket: systemctl enable --now lxd.socket
:ballot_box_with_check: 2.1 Initialize LXD using: lxd init (without the default bridge).
:ballot_box_with_check: 3. create an OpenWrt instance: lxc launch images:openwrt/21.02 openwrt
:ballot_box_with_check: 3.1 add a wan interface: lxc config device add openwrt eth0 nic nictype=physical parent=eno1 name=eth0
:ballot_box_with_check: 3.2 add a lan interface: lxc config device add openwrt eth1 nic nictype=bridged parent=br0 name=eth1
:negative_squared_cross_mark: 3.3 add a wireless lan interface: lxc config device add openwrt wlan nic nictype=physical parent=wlan0 name=wlan

Error: Failed to start device “wlan”: Failed to attach interface: wlan0 to wlan: attaching specified netdev to the container failed

:ballot_box_with_check: 3.4 enable autostart: lxc config set openwrt boot.autostart true
:ballot_box_with_check: 3.5 set boot priority: lxc config set openwrt boot.autostart.priority 100
:ballot_box_with_check: 4. restart openwrt: lxc restart openwrt
:ballot_box_with_check: 5. connect to the OpenWrt web interface and configure wan and lan on eth0 and eth1 respectively (by default eth0 gets assigned as wan).
:ballot_box_with_check: 5.1 make OpenWrt is able to reach the internet. (e.g. ping google.com)
:ballot_box_with_check: 5.2 refresh package list and upgrade packages: opkg update && opkg upgrade $(opkg list-upgradable | cut -d " " -f 1)
:ballot_box_with_check: 5.3 install iperf opkg install iperf
:ballot_box_with_check: 5.4 run iperf in server mode iperf -s &
:ballot_box_with_check: 5.5 find out what ip address was assigned to host on lan: cat /tmp/dhcp.leases
:ballot_box_with_check: 5.6 ssh into host: ssh <user>@<assigned ip>
:ballot_box_with_check: 5.7 test connection between host and openwrt: iperf -c <openwrt-ip> -P $(nproc)
:negative_squared_cross_mark: 5.8 check if host can reach the internet ping google.com or ping 8.8.8.8

1 Like
#2

There are some restrictions regarding passing in wifi devices, as I believe the parent device has to also be passed in.

Are you using the snap lxd package?

What is the role of the wifi interface, is it running hostapd as an access point or just as a client?

#3

Quick update, so I managed to get the WAN connection working by switching over to openwrt/snapshot instead of openwrt/21.02.

How can I do that ? I thought I was passing in the whole device.

pacman -S lxd (Official arch repos).

On the host it’s unmanaged, the plan was to hand it over to OpenWrt to run it in AP mode as WLAN.

#4

Do you have the iw tool installed on the host?

#5

affirmative

#6

I’m not sure I’m afraid. There have certainly been issues around moving wireless NICs into containers in the past.

You didn’t answer my other question though?

#7

Is it a hardware limitation perhaps ? is there a way to pass the PCI/USB device itself to the container ?

I’m sorry which one ?

#8

Ah i missed it, this one.
I would suggest you run hostapd on the server, and connect it to a manual bridge which you can then connect the container to. This should be OK as normally you can’t bridge wifi interfaces, but in this case as its the access point it should be OK.

#9

Good shout, I will give it a shot and see how it goes.
Quick question though, is there any way to pass a USB device to a container ?

#10

You can use Instance configuration - LXD documentation or sometimes what you actually want/need to pass is the unix device(s) provided by the usb device, for that you can use Instance configuration - LXD documentation and/or Instance configuration - LXD documentation

#11

Were you ever able to get this setup as it’s exactly what I’m working on learning how to do, any insights on the process would be greatly appreciated

#12

Hi @wikiwheeler,
To be honest It’s been a while and I haven’t unpacked my little home server since moving houses few months back but I did get everything working back then except for WiFi (which should be doable as well albeit a little hacky), best thing to do would be to hook up a couple of separate access points to the box and mesh them using 802.11r fast roaming.
Basically you wanna create a container as a firewall (whatever image you’d like to use, be it openwrt or anything else) and pass your physical interface to the container (it’s best to have at least 3 physical ports though you can do with just 2 in which you won’t have out-of-band management).
here’s what you need to do in its simplest form:

  1. Create a bridge (a network switch in software) on the host (I do this with systemd-networkd), bare in mind the bridge can be made on top of a bond (link aggregation for extra bandwidth or simple failover)
/etc/systemd/network/br0.netdev

[NetDev]
Name=br0
Kind=bridge

/etc/systemd/network/br0.network

[Match]
Name=br0

[Network]
DHCP=ipv4

systemctl restart systemd-networkd
  1. Make a new profile and remove the default interface
lxc profile cp default firewall
lxc profile device remove firewall eth0
  1. initialise a new (openwrt) container and add the physical interface and bridge to it and then start it. (bare in mind the physical interface will no longer be available on the host)
lxc init images:openwrt/22.03 -p firewall openwrt
lxc config device add openwrt eth0 nic nictype=physical parent=eno1 name=eth0
#note that here LXD is taking care of creating a virtual interface that acts as a link between your container and the software switch aka bridge that we previously created
lxc config device add openwrt eth1 nic nictype=bridged parent=br0 name=eth1
lxc start openwrt
  1. Create a LAN interface in openwrt so that you can connect to it
lxc exec openwrt -- busybox sh

/etc/config/network

...

config interface 'lan'
        option proto 'static'
        option ipaddr '192.168.10.1'
        option netmask '255.255.255.0'
        option device 'br-lan'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth1'

service network restart
  1. connect to the web interface and make sure everything works
#13

I’m fairly ignorant of the latest as I’m gave up 99% of TV, Movies, etc over 20 years ago. Why were you wanting to hookup wifi? My best guess is wireless smart devices directly connected to running apps in your LXC containers?

#14

Well the aim was to have a single physical box running everything (home media etc…) including a firewall (router) so inevitably I would need wireless access too for my wireless devices, ps. you can get these relatively cheap but really decent all-in-one mini pc/firewalls off aliexpress in particular this one even has 10Gig networking with built in wifi so that would make it a one-stop, all-in-one destination for a small homelab but like I said It’s best to use a couple of external access points running openwrt and mesh them using 802.11r through the openwrt firewall (container) running off your main box.

#15

I replied to a guy on a reddit selfhost forum and got this set up openwrt-x86/README.md at main · PrivateRouter-LLC/openwrt-x86 · GitHub on an old Asus A55A. I want to learn LXD-LXC so I figure if I can recreate this using LXC containers it would stretch my current knowledge and capabilities. I’ve been a GUI Linux user since 2003 but I’m not in the IT field just interested in learning certain use cases like installing a custom Android rom or trying new distros but I want to learn the command line and setup something like this. I appreciate you sharing what you learned on connecting your setup.