Booting from the ISO and beginning the installation runs as expected, until the installer indicates that no hard drives are present. Going into the install ISO’s shell, I was able to confirm that the device does not exist.
Using the same set of instructions have worked for CentOS so far, and Windows 10 (when adding the appropriate windows drivers).
Is there drivers for FreeBSD I seem to be missing, or is there something I’m doing wrong here?
Thanks! I had meant to post an update on what I’ve found out and its given me new questions.
raw.qemu ignores the -bios flag in the LXC implementation.
Looking through qemu documentation and chatting it up with people on the discord, the hard disk needs to be a qcow2 format and defined in raw.qemu. I’ll be trying that shortly.
If this works, how do I ensure the manually created file can enjoy the migration features within the LXD cluster?
On them working with virtual UEFI systems, I’m not sure. Considering I’ve been just throwing crap at it to see what works, I don’t understand enough of the issue to open a proper bug report.
So my best guess here is the disks allocated by LXC is presented as an IDE device. Manually defining a manually created disk as virtio resolves the issue. This breaks the clustering and backup features I’m used to in LXD, but could be remediated by other means.
The below was added to the beginning of the raw.qemu:
LXC VM’s using qemu don’t like to listen the LXC. The only way I’ve had any luck with this so far is to define everything in the raw.qemu. Mainly disk drives and NICs.
Documentation on what to put in raw.qemu has been hard for me to find. The following link the only place I was able to find the relevant man page https://linux.die.net/man/1/qemu-kvm
LXD unpacks the image templates to raw format so they can be used on directly block devices (or loopback images). These raw disk devices are exposed to the VM as virtio-scsi-pci devices and not IDE.
Is there any configuration options to change how the LXC provided resources are presented?
My thought is the disks and NICs presented in the raw.qemu as virtio are being recognized by the OS, but the disk and NIC presented by LXC are not being seen as is. In the disk example I would assume passing virtio is significantly different from virtio-scsi-pci and is the reason why its working like that. Maybe changing how it attaches to the VM would remedy issues like this for pain in the rear OSes like FreeBSD.
I’ll give that a try and let you know the results. Friends more familiar with FreeBSD had indicated it was enabled by default in the current release.
In the ISO’s preboot enviorment lsdev reveals the disk and network interfaces without issue. Here I run “set virtio_scsi_load=“YES””.
I get into the installer environment and it sees the AHCI device as a Intel ICH9 SATA, but no disks.
I go into the installer’s shell and run “kldload virtio_scsi” and get that it’s already loaded.
The only errors I’m seeing are PCI related allocation errors.
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pcib1: <ACPI PCI-PCI bridge> mem 0xc1a46000-0xc1a46fff irq 21 at device 1.0 on pci0
pcib1: failed to allocate initial I/O port window: 0xa000-0xafff
pcib1: [GIANT-LOCKED]
pcib2: <PCI-PCI bridge> mem 0xc1a45000-0xc1a45fff irq 21 at device 1.1 on pci0
pcib2: Failed to allocate interrupt for PCI-e events
pcib3: <PCI-PCI bridge> mem 0xc1a44000-0xc1a44fff irq 21 at device 1.2 on pci0
pcib3: Failed to allocate interrupt for PCI-e events
pcib4: <PCI-PCI bridge> mem 0xc1a43000-0xc1a43fff irq 21 at device 1.3 on pci0
pcib4: Failed to allocate interrupt for PCI-e events
pcib5: <PCI-PCI bridge> mem 0xc1a42000-0xc1a42fff irq 21 at device 1.4 on pci0
pcib5: Failed to allocate interrupt for PCI-e events
I’m stumped, any ideas?
EDIT: The error and hardware presence (or lack thereof) is the same even without the "set virtio_scsi_load=“YES” being set. I’m just an idiot and didn’t look at the console startup messages.
So, I de-evolved and just kept changing the configuration for three hours to figure out how to make this work.
Given the current known issue with FreeBSD, the solution I’ve found is to specify “-machine pc-q35-2.6” in the raw.qemu. Here is my quick write up on how to install pFsense on LXC.
Now you should be able to install pFsense. Once you are done installing, shut down the VM then run the following. We are just removing the ISO and getting rid of the boot menu. sudo echo -n '-machine pc-q35-2.6 -device virtio-vga -vnc :2' | sudo lxc config set pfsense raw.qemu -
Connect via VNC to finish the setup. Once you are done setting up, shut down and run the below to get rid of VNC. sudo echo -n '-machine pc-q35-2.6' | sudo lxc config set pfsense raw.qemu -
Let me know if you guys spot any problems with this! I know using 2.6 is completely non-ideal, and the flag should be removed whenever FreeBSD fixes this bug, but it appears to be low on the priority list for them.
