Hello and happy New Year! I have been using LXD to run system containers on my server for approximately one year, which has been a great! I also run libvirt on the same server, and am interested in converting my KVM/qemu VM builds to be accommodated by LXD.
Problem: My initial VM build (with LXD) was successful, but cannot connect to the VM after adding limits.memory.hugepages to the LXD’s configuration for the VM.
Note: I have previously configured hugepages on my system to support running a VM via libvirt, which appears to work fine, but I suspect that this configuration is problematic and/or insufficient for LXD’s needs. Or maybe it is fine, and i have a different issue… I will detail my hugepages configuration further below.
Basic information about the bare-metal host:
$ grep PRETTY_NAME /etc/os-release
PRETTY_NAME="Debian GNU/Linux 11 (bullseye)"
$ uname -a
Linux mountain 5.10.0-8-amd64 #1 SMP Debian 5.10.46-4 (2021-08-03) x86_64 GNU/Linux
$ lscpu | grep "Model name"
Model name: AMD Ryzen 9 5950X 16-Core Processor
My LXD installation was performed via snap, and qemu was installed via apt.
$ snap list
Name Version Rev Tracking Publisher Notes
core 16-2.52.1 11993 latest/stable canonical✓ core
core20 20211129 1270 latest/stable canonical✓ base
lxd 4.21 22147 latest/stable canonical✓ -
$ qemu-system-x86_64 --version
QEMU emulator version 5.2.0 (Debian 1:5.2+dfsg-11)
Copyright (c) 2003-2020 Fabrice Bellard and the QEMU Project developers
Here is the LXD build of the VM, first without hugepages. I am following this guide, with changes as applicable. As indicated below, this works fine.
$ lxc launch images:ubuntu/focal/cloud/amd64 furnace --vm
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Creating furnace
Starting furnace
$ lxc info furnace
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Name: furnace
Status: RUNNING
Type: virtual-machine
Architecture: x86_64
PID: 1221899
Created: 2021/12/31 13:43 EST
Last Used: 2021/12/31 13:43 EST
Resources:
Processes: 15
Disk usage:
root: 6.26MiB
CPU usage:
CPU usage (in seconds): 3
Memory usage:
Memory (current): 136.66MiB
Memory (peak): 160.94MiB
Network usage:
enp5s0:
Type: broadcast
State: UP
Host interface: tapef4100bc
MAC address: 00:16:3e:0f:53:57
MTU: 1500
Bytes received: 1.66kB
Bytes sent: 1.77kB
Packets received: 14
Packets sent: 14
IP addresses:
inet: 192.168.14.170/24 (global)
inet6: fe80::216:3eff:fe0f:5357/64 (link)
lo:
Type: loopback
State: UP
MTU: 65536
Bytes received: 5.92kB
Bytes sent: 5.92kB
Packets received: 80
Packets sent: 80
IP addresses:
inet: 127.0.0.1/8 (local)
inet6: ::1/128 (local)
$ lxc exec furnace -- su root
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
root@furnace:~# uname -a
Linux furnace 5.4.0-91-generic #102-Ubuntu SMP Fri Nov 5 16:31:28 UTC 2021 x86_64 x86_64 x86_64 GNU/Linux
However, if I try to add hugepages into the configuration, I cannot lxc exec into the VM anymore because of an issue with the lxd-agent. lxc console seems to work, but I have no viable login.
$ lxc init images:ubuntu/focal/cloud/amd64 furnace --vm
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Creating furnace
$ lxc info furnace
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Name: furnace
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2021/12/31 13:50 EST
$ lxc config set furnace limits.memory.hugepages=true
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
$ lxc config show furnace
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
architecture: x86_64
config:
image.architecture: amd64
image.description: Ubuntu focal amd64 (20211228_07:42)
image.os: Ubuntu
image.release: focal
image.serial: "20211228_07:42"
image.type: disk-kvm.img
image.variant: cloud
limits.memory.hugepages: "true"
volatile.apply_template: create
volatile.base_image: 3909be5d5c59409c001c40805c86bcb29ac787e10618a3c10ddfd425300d7adb
volatile.eth0.hwaddr: 00:16:3e:cd:73:63
volatile.uuid: 5985947b-2fff-4c91-ba0d-7ca3ad4aceb5
devices: {}
ephemeral: false
profiles:
- default
stateful: false
description: ""
$ lxc start furnace
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
$ lxc info furnace
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Name: furnace
Status: RUNNING
Type: virtual-machine
Architecture: x86_64
PID: 1236047
Created: 2021/12/31 13:50 EST
Last Used: 2021/12/31 13:54 EST
Resources:
Processes: -1
Disk usage:
root: 8.00KiB
Network usage:
eth0:
Type: broadcast
State: UP
Host interface: tape76c6878
MAC address: 00:16:3e:cd:73:63
MTU: 1500
Bytes received: 862B
Bytes sent: 164B
Packets received: 5
Packets sent: 2
IP addresses:
$ lxc exec furnace -- su root
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
Error: Failed to connect to lxd-agent
$ lxc console furnace
WARNING: cgroup v2 is not fully supported yet, proceeding with partial confinement
To detach from the console, press: <ctrl>+a q
furnace login:
Admittedly, my “configuration” of the VM to use hugepages seems pretty naive and so my first thought is that it is incomplete. Maybe there is something pretty obvious that am missing something in my LXD VM configuration. Any help here would be much appreciated.
Thank you!
