Physical ethernet nic for guest

LXD
1

Hello:

LXD 3.0.3 on Ubuntu 18.04.

I don’t have ‘infiniband’ on my host machine, but it seems maybe I can use a spare ethernet port on my host directly in the guest???

I was able to successfully (maybe) add the ethernet port to a container; it disappeared from the host. The container was running at the time, so I restarted it. But then got the error:

Error: Common start logic: Missing parent 'eno2' for nic 'eth1'

eno2 is the nic name on the host. The command I used to add it to the container was:

lxc config device add webserver eth1 nic nictype=physical parent=eno2

If that is wrong, what should I have used?

Also - can I set nic passthrough on creation so the container ONLY has the physical nic?

Finally, can I get my eno2 back on the host without rebooting now?

Thank you!

2

Did it maybe re-appear as eth1 on the host?

If so, that’s something we’ll be adding more logic to prevent, @tomp is looking at some of those issues now.

3

Try adding the passthrough device with an explicit interface name, e.g.

lxc config device add webserver eth1 nic nictype=physical parent=eno2 name=eth1

I’ve seen that sometimes the auto naming of the device inside the container doesn’t align with the name of the nic device itself in the LXD config.

4

Yes… after the container stop/start and the error, ip link showed the host machine had eno1 and eth1 but not eno2. I guess the internal logic of LXD needs to ‘undo’ the passthrough when the container is stopped … so the host again would show eno2 and when the container is started eno2 is once again available to be exclusively assigned.

–edit: When the process worked correctly, stopping the container with lxc stop it DID correctly return eno2 to the host, so that’s good.

–edit2:

Regadring my own question of creating the container with host physical nic, if the profile is:

devices:
  eth0:
    name: eth0
    nictype: physical
    parent: eno2
    type: nic

seems to have worked!

5

Continuing on before you guys replied, I removed the physical nic from the container config and rebooted. Then I made sure the container was stopped and added the physical nic again, started it, and then I did not get an error. Host loses eno2 (and does not get eth1) and container gets an eth1 which I added to 01-netcfg.yaml (hate) to make work.

So, I think that is all as expected.

Should I file a bug about adding physical nic to a running container and then restarting the container (lxd stop container / lxd start container) causes an error because the nic is not restored after stop?

6

If you are able to re-create the issue, if you could write down the order of the commands you run to re-create it then I can see if I can re-create it and if so will investigate it further. As @stgraber mentioned I’m currently working on ensuring that the MTU of a physical passthrough device is restored on container shutdown, so it sounds similar.

7

Hi Thomas:

I don’t have a suitable testing environment right now, but if you do, this would be quick enough to test – I was on a physical machine when I encountered this.

From memory, and some notes, the steps were:

Init a default LXD environment, with default NAT bridge setup
launch an ubuntu:18.04 container with default options*
(it is then running, I did not enter it)
issue the command to add physical nic lxc config device add webserver eth1 nic nictype=physical parent=eno2
lxd stop the container
lxd start the container

then error ocurrs. Error: Common start logic: Missing parent 'eno2' for nic 'eth1'

I think if the container had not been running when I added the physical nic the problem would not have occurred. In this case, if a user is foolish enough to attempt to add a device to a running container, maybe LXD should refuse to add it, with output ‘container must be stopped to do that’ sort of thing.

If adding ethernet nic, type ‘physical’, is supported, maybe lxd-readthedocs.io should be updated, as the only reference I see there to nic / physical is in the context of Infiniband devices.

Thanks for all your (Thomas, Stephane) hard work and responses to all the questions here… I have read many of them!

  • = ok, not really default options entirely, had this config:

    user.vendor-data: |
    #cloud-config
    package_upgrade: true
    users:
    - name: root
    ssh_authorized_keys:
    - ssh-ed25519 AAAA…
    - ssh-rsa AAAA…
    lock_passwd: false
    shell: /bin/bash

8

We were actually discussing just this situation on IRC #lxc-dev today.

Whilst adding a nic whilst a container is running (hot plugging) is in theory supported for all nic types except IPVLAN, there is indeed an issue with hot plugging a physical device after a container has started and then shutting down the container without removing it.

Currently what happens is that the physical nic will be correctly hot plugged into the running container and will work fine, but the underlying LXC library has no idea that this device exists in the container (only LXD does).

If you then shut down the container, LXD does not remove the devices that were hot plugged before getting LXC to shutdown the container.

This leaves it to the OS to clean up the physical device once the container’s namespace has been destroyed. This will leave the interface in the host’s namespace, but possibly with the name of the interface as it was in the container, eth1. This means that when you try and start the container again, the original name of eno2 is not there any more.

I need to discuss this issue more with @stgraber to see whether there is a solution to this or whether we just block hot plugging physical devices as you suggest.