Error attempting to attach LXD VM NIC to host's physical network interface

LXD
1

Edit: After reading the docs again, it seems like physical LXD networks are exclusively (?) intended to be used with OVN? I’m a little confused, forgive me.

Summary

I’m receiving an error when I attempt to attach a “physical” network interface to an LXD virtual machine.

I thought that maybe, instead of having to create a virtual bridged adapter on the host, bound to the eth0 interface, I could instead use the physical LXD network type to attach VMs directly to the host’s network interface. I’m not sure if this is an intended use case for the LXD physical network type or not. Based on my interpretation of the documentation, however, it does seem to be an intended use case.

Reproduction Steps

On the LXD host, I created a “physical” network, according to the LXD documentation, associated with the parent interface eth0.

~# lxc network create physicalnetwork --type=physical parent=eth0

Then, I created an LXD profile that uses the LXD physical network for the VM network interface device.

~# lxc profile create physnet
~# lxc profile edit physnet # use the YAML below

devices:
  eth0:
    name: eth0
    network: physicalnetwork
    type: nic

Then I launched a VM, using this additional profile.

~# lxc launch --vm --profile default --profile physnet images:ubuntu/23.04 u01

After that, I received the following error:

Creating u01
Starting u01
Error: Failed to start device "eth0": Failed to get PCI device info for "eth0": Device uevent file could not be parsed

I can’t find any references to this error on Google, so it seems like an entirely new issue.

Attempted to use Secondary Host Network Interface

Since I failed to bind an LXD VM to the host network interface eth0, I thought that maybe I could succeed by adding a second network interface to the host.

  • I shutdown the LXD host VM in Hyper-V.
  • In the Hyper-V VM settings, I added a second network interface to the host, attached to the “external” virtual switch
  • I started the LXD host back up
  • I recreated the physical LXD network and profile, pointing to eth1 instead of eth0
  • I attempted to create a VM with the new profile
  • I received the same error as before

Environment

  • The LXD host is a virtual machine running on Windows 11 Hyper-V.
  • I have an “external” Hyper-V virtual switch that allows me to attach the LXD host directly to my home network.
  • The LXD host VM only has the one network interface.
2

So to clarify.

  • Yes, really the physical network type is only suitable for use with ovn networks (at this time). @ru-fu and I did discuss the pros and cons of adding support for using physical networks to the physical NIC type. The pros are consistency, and the cons are that, even if it did work, it would only work for a single running instance at a time (because the parent NIC is transferred/taken over by the instance).
  • To share a physical NIC with multiple instances you can use macvlan NIC type (Linux Containers - LXD - Has been moved to Canonical) (if you don’t mind not being able to communicate with the host) or setting up a manual bridge with the physical interface connected and then using bridged NIC type (Linux Containers - LXD - Has been moved to Canonical) with the parent set to the bridge interface.

But when we were discussing this with @ru-fu we did add some notes to the physical NIC documentation (Linux Containers - LXD - Has been moved to Canonical):

A physical NIC provides straight physical device pass-through from the host. The targeted device will vanish from the host and appear in the instance (which means that you can have only one physical NIC for each targeted device).

The specific error you got suggests that the physical NIC is missing a udev slot name.
Perhaps its something to do with the type of NIC that hyperv is supplying in the guest.

1 Like
3

I ended up doing this last night. It works great on my bare metal LXD host. Unfortunately, as you pointed out, the LXD VM can’t access the physical host. Not even ICMP traffic makes it to the host IP. That’s a pretty big bummer, as it partially defeats the purpose of connecting the VM to my physical (home) network.

Problems With Macvlan on Hyper-V

However, when creating VMs on my LXD host, running as a Hyper-V VM, it completely fails. This is an easy problem to reproduce. Simply deploy an LXD host onto Hyper-V, and attempt to create an LXD VM using a macvlan interface. It simply doesn’t work. dhclient won’t even get a response from the DHCP server, and I’m not sure if the DHCP server is even receiving the DISCOVER requests from the LXD host.

:thinking: Why am I deploying LXD as a Hyper-V guest? Well, I use Windows 11 Professional on my desktop workstation. Deploying an LXD host as a Hyper-V VM makes it:

  • Easy for me to play with locally
  • Spin up / down LXD environments on-demand
  • Run many side-by-side LXD instances (including nested VMs), due to abundant CPU/RAM/NVMe disk resources

For the time being, it seems as if Hyper-V is simply incompatible with macvlan. I submitted feedback on the Feedback Hub app in the Windows Store. Here’s the link to my feedback (opens Feedback Hub app), along with a screenshot:

image
image1259×838 100 KB

Various Sub-Types of Macvlan

I learned that there are actually 5 different sub-types of macvlan, from the RedHat blog.

It seems like the macvlan bridge sub-type might allow LXD VMs to communicate with the host.

Question: What sub-type of macvlan is currently used by LXD?
Question: Would supporting macvtap still skip the need for the bridge-parent host interface, and allow LXD VMs to communicate with the host?

Hyper-V Issue With LXD Physical NIC

Unfortunately Hyper-V doesn’t have any detailed configuration for network interfaces, that would allow specifying a different virtual hardware type. For now, it seems like using Hyper-V with an LXD physical NIC isn’t going to be supported. I wanted to make sure it was documented here though.

4

Lxd vms use macvtap in bridge mode.

Most likely issue is that the hyperv vm nic itself isnt operating in gratuitous mode (i.e it has max filtering enabled) so doesnt support having multiple mac addresses concurrently.

This would also affect bridging.

1 Like
5

Did you mean macvlan in bridge mode?

6

Yes. Macvtap is what’s used for macvlan for vms.