How can I make a FIDO/U2F key available in an Incus Container?

Incus
1

@stgraber I have been trying to make a FIDO/U2F key available within an incus container in order to enable hardware key authentication via ssh:

ssh-keygen -t ed25519-sk

The problem is that the USB FIDO key is unreachable.

image
image771×127 12.8 KB

I have tried to connect the device from the host to the container “Test” with the following command and that does not seem to work.

incus config device add Test security-key usb vendorid=1050 productid=0407

I have also tried as a Unix-Hotplug and no luck with that either.

incus config device add Test security-key unix-hotplug vendorid=1050 productid=0407

Any ideas?

2

unix-hotplug should work fine, at least it did for me in the past, but note that it needs to then be hotplugged.

3

As mentioned earlier, I tried both USB and unix-hotplug.
Note that:

ssh-keygen -t ed25519-sk

works on the incus host. It just fails on the container. This is a YubiKey 5C.
Interestingly, the device shows but just doesn’t seem accessible.
Are there udev rules needed?
Should this work?
Is this a permission thing requiring some type of raw.id mapping?

ssh-keygen -t ed25519-sk -vvv
Generating public/private ed25519-sk key pair.
You may need to touch your authenticator to authorize key generation.
debug3: start_helper: started pid=714
debug3: ssh_msg_send: type 5
debug3: ssh_msg_recv entering
debug1: start_helper: starting /usr/lib/openssh/ssh-sk-helper 
debug1: sshsk_enroll: provider "internal", device "(null)", application "ssh:", userid "(null)", flags 0x01, challenge len 0
debug1: sshsk_enroll: using random challenge
debug1: sk_probe: 0 device(s) detected
debug1: ssh_sk_enroll: failed to find sk
debug1: sshsk_enroll: provider "internal" failure -4
debug1: ssh-sk-helper: Enrollment failed: device not found
debug1: main: reply len 8
debug3: ssh_msg_send: type 5
debug1: client_converse: helper returned error -60
debug3: reap_helper: pid=714
Key enrollment failed: device not found
4

So, and to me hotplug means I define the device while unplugged:

incus config device add Test security-key unix-hotplug vendorid=1050 productid=0407

Then, I plug it in and it should work from the container. Right? It’s not seeing it.

5 
stgraber@dakara:~/data/code/lxc/incus (stgraber/main)$ incus launch images:ubuntu/24.04 u1
Launching u1

stgraber@dakara:~/data/code/lxc/incus (stgraber/main)$ incus config device add u1 yubikey unix-hotplug vendorid=1050 productid=0406
Device yubikey added to u1

stgraber@dakara:~/data/code/lxc/incus (stgraber/main)$ incus exec u1 bash
root@u1:~# find /dev/
/dev/
/dev/pts
/dev/pts/1
/dev/pts/0
/dev/pts/ptmx
/dev/fuse
/dev/net
/dev/net/tun
/dev/zfs
/dev/mqueue
/dev/incus
/dev/incus/sock
/dev/.incus-mounts
find: ‘/dev/.incus-mounts’: Permission denied
/dev/full
/dev/null
/dev/random
/dev/tty
/dev/urandom
/dev/zero
/dev/ptmx
/dev/console
/dev/fd
/dev/stdin
/dev/stdout
/dev/stderr
/dev/.lxc-boot-id
/dev/shm
/dev/core
/dev/initctl
/dev/log

root@u1:~# find /dev/
/dev/
/dev/pts
/dev/pts/1
/dev/pts/0
/dev/pts/ptmx
/dev/fuse
/dev/net
/dev/net/tun
/dev/zfs
/dev/mqueue
/dev/incus
/dev/incus/sock
/dev/.incus-mounts
find: ‘/dev/.incus-mounts’: Permission denied
/dev/full
/dev/null
/dev/random
/dev/tty
/dev/urandom
/dev/zero
/dev/ptmx
/dev/console
/dev/fd
/dev/stdin
/dev/stdout
/dev/stderr
/dev/.lxc-boot-id
/dev/shm
/dev/core
/dev/initctl
/dev/log
/dev/bus
/dev/bus/usb
/dev/bus/usb/005
/dev/bus/usb/005/069
/dev/char
/dev/char/189:580
/dev/char/235:6
/dev/hidraw6

root@u1:~# ssh-keygen -t ed25519-sk -vvv
Generating public/private ed25519-sk key pair.
You may need to touch your authenticator to authorize key generation.
debug3: start_helper: started pid=351
debug3: ssh_msg_send: type 5
debug3: ssh_msg_recv entering
debug1: start_helper: starting /usr/lib/openssh/ssh-sk-helper 
debug1: sshsk_enroll: provider "internal", device "(null)", application "ssh:", userid "(null)", flags 0x01, challenge len 0
debug1: sshsk_enroll: using random challenge
debug1: sk_probe: 1 device(s) detected
debug1: sk_probe: selecting sk by touch
debug1: ssh_sk_enroll: using device /dev/hidraw6
debug1: ssh_sk_enroll: fido_dev_make_cred: FIDO_ERR_PIN_REQUIRED
debug1: sshsk_enroll: provider "internal" failure -3
debug1: ssh-sk-helper: Enrollment failed: incorrect passphrase supplied to decrypt private key
debug1: main: reply len 8
debug3: ssh_msg_send: type 5
debug1: client_converse: helper returned error -43
debug3: reap_helper: pid=351
Enter PIN for authenticator:

Note that mine is a 0406 so probably older version of the 5C than what you have.

7

Apparently only root has permissions. I am completely unable to ssh to the container using the Yubikey. There is some subtle permissions error in all this.

ssh -i ~/.ssh/id_ecdsa_sk  172.16.1.195
sign_and_send_pubkey: signing failed for ECDSA-SK "/home/scott/.ssh/id_ecdsa_sk" from agent: agent refused operation
8

So, “unix-hotplug” works, but ONLY if the “ssh-keygen -t ed25519-sk” is performed with sudo which is self defeating since it is only on the root account. I tried:

raw.lxc="lxc.apparmor.profile=unconfined"

and

security.privileged=true

thinking those would provide access to the hardware correctly, but that did not work.

9

Can’t you just set mode=0666 on the device?

10

The device, meaning???

sudo chmod 666 /dev/.incus-mounts
11

incus config device set Test security-key mode=0666

12 
incus launch images:ubuntu/24.04 Test 
incus config device add Test security-key unix-hotplug vendorid=1050 productid=0407
incus config device set Test security-key mode=0666

Device exists viewing from inside the container:

scott@Test:~$ lsusb
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 001 Device 002: ID 058f:6366 Alcor Micro Corp. Multi Flash Reader
Bus 001 Device 003: ID 046d:0aaf Logitech, Inc. Yeti X
Bus 001 Device 004: ID 046d:0823 Logitech, Inc. HD Webcam B910
Bus 001 Device 006: ID 0764:0501 Cyber Power System, Inc. CP1500 AVR UPS
Bus 001 Device 007: ID 0bda:5411 Realtek Semiconductor Corp. RTS5411 Hub
Bus 001 Device 008: ID 046d:c016 Logitech, Inc. Optical Wheel Mouse
Bus 001 Device 009: ID 045e:0024 Microsoft Corp. Trackball Explorer
Bus 001 Device 010: ID 0c45:7603 Microdia USB Keyboard
Bus 001 Device 016: ID 1050:0407 Yubico.com Yubikey 4/5 OTP+U2F+CCID
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 002 Device 002: ID 0bda:0411 Realtek Semiconductor Corp. Hub
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub

However, it’s not reachable:

scott@Test:~$ ssh-keygen -t ed25519-sk -vvv
Generating public/private ed25519-sk key pair.
You may need to touch your authenticator to authorize key generation.
debug3: start_helper: started pid=1088
debug3: ssh_msg_send: type 5
debug3: ssh_msg_recv entering
debug1: start_helper: starting /usr/lib/openssh/ssh-sk-helper 
debug1: sshsk_enroll: provider "internal", device "(null)", application "ssh:", userid "(null)", flags 0x01, challenge len 0
debug1: sshsk_enroll: using random challenge
debug1: sk_probe: 0 device(s) detected
debug1: ssh_sk_enroll: failed to find sk
debug1: sshsk_enroll: provider "internal" failure -4
debug1: ssh-sk-helper: Enrollment failed: device not found
debug1: main: reply len 8
debug3: ssh_msg_send: type 5
debug1: client_converse: helper returned error -60
debug3: reap_helper: pid=1088
Key enrollment failed: device not found
13 
stgraber@dakara:~$ incus launch images:ubuntu/24.04 u1
Launching u1
stgraber@dakara:~$ incus config device add u1 yubikey unix-hotplug vendorid=1050 productid=0406 mode=0666
Device yubikey added to u1

stgraber@dakara:~$ incus exec u1 bash
root@u1:~# su ubuntu
To run a command as administrator (user "root"), use "sudo <command>".
See "man sudo_root" for details.

ubuntu@u1:/root$ cd
ubuntu@u1:~$ ssh-keygen -t ed25519-sk -vvv
Generating public/private ed25519-sk key pair.
You may need to touch your authenticator to authorize key generation.
debug3: start_helper: started pid=376
debug3: ssh_msg_send: type 5
debug1: start_helper: starting /usr/lib/openssh/ssh-sk-helper 
debug3: ssh_msg_recv entering
debug1: sshsk_enroll: provider "internal", device "(null)", application "ssh:", userid "(null)", flags 0x01, challenge len 0
debug1: sshsk_enroll: using random challenge
debug1: sk_probe: 1 device(s) detected
debug1: sk_probe: selecting sk by touch
debug1: ssh_sk_enroll: using device /dev/hidraw6
debug1: ssh_sk_enroll: fido_dev_make_cred: FIDO_ERR_PIN_REQUIRED
debug1: sshsk_enroll: provider "internal" failure -3
debug1: ssh-sk-helper: Enrollment failed: incorrect passphrase supplied to decrypt private key
debug1: main: reply len 8
debug3: ssh_msg_send: type 5
debug1: client_converse: helper returned error -43
debug3: reap_helper: pid=376
Enter PIN for authenticator: 

ubuntu@u1:~$
14

I performed the exact same commands (adjusting for my device ID 0407). When I get to the keygen, my device is not found as user ubuntu, only as user root and yes I have the mode=0666.

ubuntu@u1:~$ ssh-keygen -t ed25519-sk -vvv
Generating public/private ed25519-sk key pair.
You may need to touch your authenticator to authorize key generation.
debug3: start_helper: started pid=367
debug3: ssh_msg_send: type 5
debug3: ssh_msg_recv entering
debug1: start_helper: starting /usr/lib/openssh/ssh-sk-helper 
debug1: sshsk_enroll: provider "internal", device "(null)", application "ssh:", userid "(null)", flags 0x01, challenge len 0
debug1: sshsk_enroll: using random challenge
debug1: sk_probe: 0 device(s) detected
debug1: ssh_sk_enroll: failed to find sk
debug1: sshsk_enroll: provider "internal" failure -4
debug1: ssh-sk-helper: Enrollment failed: device not found
debug1: main: reply len 8
debug3: ssh_msg_send: type 5
debug1: client_converse: helper returned error -60
debug3: reap_helper: pid=367
Key enrollment failed: device not found
ubuntu@u1:~$

Funny this works great as root. Could there be anything different on the Ubuntu Server at all? I am running the latest Incus 6.1.

15

Ok, my bad. I have to HOT-PLUG the device everytime I recreate the container and ONLY after adding the device to the container.

Thanks so much for your outstanding help and patience.

16

Do you mind writing a summary? I have a device I would like to test out.

17

I will once I get it working end to end. I will come back here and post.

1 Like