Running virtual machines with LXD 4.0

simos · February 28, 2021, 7:05pm

It’s the following.

$ lxc start win10 --console=vga
Error: Failed to start device "eth0": Failed to run: ip link add dev maca2b0d635 link enp3s0 type macvtap mode bridge: RTNETLINK answers: Device or resource busy
Try `lxc info --show-log win10` for more info
$ lxc info --show-log win10
Name: win10
Location: none
Remote: unix://
Architecture: x86_64
Created: 2021/02/27 17:50 UTC
Status: Stopped
Type: virtual-machine
Profiles: default, macvlan
Error: open /var/snap/lxd/common/lxd/logs/win10/qemu.log: no such file or directory

stgraber · February 28, 2021, 7:25pm

Ah, okay, so it’s looking like some kind of compatibility issue with macvlan/macvtap and your network driver.

muslu · March 1, 2021, 2:42pm

Hi all;

I tested Windows vm in LXC on Ubuntu 18.04 and its worked.

Note: remmina (for slice) or virt-viewer can connect vga.

apt install -yqq remmina-plugin-spice virt-viewer

snap install distrobuilder --classic --edge
apt install -y libguestfs-tools wimtools
distrobuilder repack-windows /home/muslu/win10.iso /home/muslu/win10_distrobuilder.iso
lxc init win10 --empty --vm -c security.secureboot=false
lxc config device override win10 root size=50GiB
lxc config device add win10 iso disk source=/home/muslu/win10_distrobuilder.iso boot.priority=10
lxc start win10 --console=vga

If install virt-viewer than vga windows will be open automaticly after lxc start.

makdosdogancan · March 4, 2021, 12:51pm

thanks Muslu. It’s very helpfull.

simos · March 4, 2021, 4:53pm

@muslu, which executable is provided from the wimtools package that is needed by distrobuilder?

muslu · March 5, 2021, 10:39am

Hi all;

Sorry for delay.

I wanted to test all the options.

When i test first time with my own pc (Ubuntu 18.04), i did get error for win parse and i installed wimtools.

i tested same code on my other pc (Ubuntu 18.04) but i did get more errors.

distrobuilder repack-windows /home/muslu/win10.iso /home/muslu/win10_distrobuilder.iso
2021/03/05 13:26:29 Mounting Windows ISO ...
2021/03/05 13:26:29 Mounting driver ISO ...
2021/03/05 13:26:29 Injecting drivers into boot.wim (index 2)...
2021/03/05 13:26:36 Injecting drivers into install.wim (index 1)...
Error: Failed to modify index 1 of "install.wim": Failed to inject drivers: Failed to edit Windows DRIVERS registry for driver "NetKVM": Failed to run: hivexregedit --merge --prefix='HKEY_LOCAL_MACHINE\DRIVERS' /var/cache/distrobuilder.507518749/wim/Windows/System32/config/DRIVERS: reg_import: cannot create \DriverDatabase\DeviceIds\pci\VEN_1AF4&DEV_1000 since parent \DriverDatabase\DeviceIds\pci\ does not exist at /usr/lib/x86_64-linux-gnu/perl5/5.26/Win/Hivex/Regedit.pm line 341, <FILE> line 5.
    Win::Hivex::Regedit::_merge_node(CODE(0x557773bab150), HASH(0x557773ca8f90), "\DriverDatabase\DeviceIds\pci\VEN_1AF4\x{26}DEV_1000", ARRAY(0x557773d2d190), ARRAY(0x557773d2d148)) called at /usr/lib/x86_64-linux-gnu/perl5/5.26/Win/Hivex/Regedit.pm line 196
    Win::Hivex::Regedit::reg_import(GLOB(0x557773d5d960), CODE(0x557773bab150)) called at /usr/bin/hivexregedit line 331

I will try again and report to you.

Thansk for everyting about lxc/lxd/lxcfs/distrobuilder and etc.

stgraber · March 5, 2021, 2:19pm

@monstermunchkin

TomvB · March 11, 2021, 4:58pm

@stgraber How can i limit the bandwidth?

like: lxc config device set VM eth0 limits.ingress 100Mbit

stgraber · March 11, 2021, 5:00pm

limits.ingress and limits.egress should work as expected, though note that they only work with bridges and cannot work with macvlan, physical or sr-iov.

TomvB · March 11, 2021, 7:36pm

Thanks! Using macvlan.

nvm: works now… using root partition only.

TomvB · March 12, 2021, 12:05pm

I have a new test server available for LXD.
CPU: Intel Xeon W3520

egrep -c ‘(vmx|svm)’ /proc/cpuinfo
8

kvm-ok
INFO: /dev/kvm exists
KVM acceleration can be used

Server OS: 20.04
2 disks in RAID1 with LVM
The VM starts after the launch (ipv6 only)

lxc launch ubuntu:20.04 test1 --vm
Then the VM stops (after a few seconds) and I can’t start the VM

Error:
Error: open /var/snap/lxd/common/lxd/logs/test1/qemu.log: no such file or directory

lxc info
driver: lxc | qemu
driver_version: 4.0.6 | 5.2.0
firewall: xtables
kernel: Linux
kernel_architecture: x86_64
kernel_features:

Any idea @stgraber

Update: I cant delete or start the VM after this. Reboot required to delete it.

After a long wait…
user@node:~$ lxc delete test1
Error: Error deleting storage volume: Error removing LVM logical volume: Failed to run: lvremove -f /dev/LXD/virtual-machines_test1.block: Logical volume LXD/virtual-machines_test1.block in use.

lxc start test1
Error: virtiofsd failed to bind socket within 10s
Try lxc info --show-log test1 for more info

stgraber · March 12, 2021, 8:23pm

Anything scary looking in dmesg? Sounds like that system isn’t having a good day.

TomvB · March 12, 2021, 8:37pm

Nope, nothing. Reinstallation doesn’t help either

Container:
t=2021-03-12T20:47:26+0000 lvl=info msg="Started container" action=start created=2021-03-12T11:04:26+0000 ephemeral=false instance=test instanceType=container project=default stateful=false used=2021-03-12T11:20:09+0000

VM:
t=2021-03-12T20:53:31+0000 lvl=warn msg="Error getting disk usage" err="strconv.ParseFloat: parsing \"\": invalid syntax" instance=test1 instanceType=virtual-machine project=default

QEMU log:

KVM internal error. Suberror: 3
extra data[0]: 800000ec
extra data[1]: 31
extra data[2]: 81
extra data[3]: 30db0
RAX=0000000000000000 RBX=0000000000000001 RCX=0000000000000001 RDX=00000000000001ca
RSI=ffff9823b5d1cba8 RDI=0000000000000001 RBP=ffffa46d80077e90 RSP=ffffa46d80077e78
R8 =000000001327186b R9 =0000000000000000 R10=0000000000002000 R11=0000000000002000
R12=0000000000000001 R13=ffff9823401c8c80 R14=0000000000000000 R15=0000000000000000
RIP=ffffffffadf54c14 RFL=00000246 [—Z-P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 0000000000000000 ffffffff 00c00000
CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA]
SS =0000 0000000000000000 ffffffff 00c00000
DS =0000 0000000000000000 ffffffff 00c00000
FS =0000 0000000000000000 ffffffff 00c00000
GS =0000 ffff9823b5d00000 ffffffff 00c00000
LDT=0000 0000000000000000 ffffffff 00c00000
TR =0040 fffffe0000036000 0000206f 00008b00 DPL=0 TSS64-busy
GDT= fffffe0000034000 0000007f
IDT= fffffe0000000000 00000fff
CR0=80050033 CR2=00000000ffffffff CR3=0000000029a0a000 CR4=000006a0
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
DR6=00000000ffff0ff0 DR7=0000000000000400
EFER=0000000000000d01
Code=00 85 c0 7e 07 0f 00 2d b6 c1 4b 00 fb f4 8b 05 74 8f 78 00 <65> 44 8b 25 14 b5 0b 52 85 c0 0f 8f 85 00 00 00 5b 41 5c 41 5d 5d c3 65 8b 05 fe b4 0b 52

Tested with HWE (without success)
Tested with Proxmox (VM/LXC) instead of Ubuntu 20.04 with LXD and it works.

@stgraber I’ll use proxmox for now. I don’t know why it is going wrong on ubuntu 20.04.

muslu · March 19, 2021, 8:29am

OK!.

distrobuilder is working like a magic.

stgraber · April 19, 2021, 1:47pm

I’ve updated the instructions to reference the work we did on distrobuilder and on LXD booting the resulting ISO, no more need for workarounds.

erik_lonroth · May 18, 2021, 3:55am

Are there any pre-built/community images of win10 around which would spare me the effort of creating it as part of a small test?

I was hoping to see if I could use this with gitlab-runner and lxd executors…

stgraber · May 18, 2021, 1:22pm

No, unfortunately it’s not legal to re-distribute a Windows ISO image (or VM image) which is why we can only provide the tool that generates it for yourself and can’t just make the result available as we do for our other images.

stgraber · May 27, 2021, 2:36pm

6 posts were split to a new topic: Unable to export/import Windows VM

stgraber · June 3, 2021, 6:56pm

3 posts were split to a new topic: VM not booting from Windows ISO

igoryonya · June 16, 2021, 4:33am

Is there or will be there a new tutorial for LXD version 4.0, as you have for LXD version 2.0?