LXD pfsense VM installation

cemzafer · December 1, 2021, 9:29pm

Hi,
I have tested with the new pfSense-CE-memstick-2.5.1-RELEASE-amd64.img image with lxd vm capability and it works great. Here are the steps as follows.

lxc init pfsense --empty --vm -c limits.cpu=1 -c limits.memory=2GB -c security.secureboot=false -n lxdbr0 # create an empty instance.
lxc config device override pfsense root size=32GB # resize the pfsense disk
echo -n '-machine pc-q35-2.6' | sudo lxc config set pfsense raw.qemu - # change the machine type with qemu parameter.
lxc config device add pfsense install disk source=/home/indiana/Downloads/pfSense-CE-memstick-2.5.1-RELEASE-amd64.img boot.priority=10 # Add boot disk image to the lxd vm
lxc start pfsense && lxc console pfsense --type=vga
Follows the pfsense installation parameters as UEFI and GPT and finished the installation.
lxc stop pfsense --force # stop the lxd vm
lxc config device set pfsense root boot.priority=15 # change the boot order
lxc start pfsense && lxc console pfsense --type=vga # start the instance again

LXD VM configuration of the pfsense.

architecture: x86_64
config:
  limits.cpu: "1"
  limits.memory: 2GB
  raw.qemu: -machine pc-q35-2.6
  security.secureboot: "false"
  volatile.eth0.host_name: tap2d254e0f
  volatile.eth0.hwaddr: 00:16:3e:a0:b8:cc
  volatile.last_state.power: RUNNING
  volatile.uuid: 33b9c90b-e4dc-49e3-a1a1-f8f1d088b041
  volatile.vsock_id: "114"
devices:
  eth0:
    name: eth0
    network: lxdbr0
    type: nic
  install:
    boot.priority: "10"
    source: /home/indiana/Downloads/pfSense-CE-memstick-2.5.1-RELEASE-amd64.img
    type: disk
  root:
    boot.priority: "15"
    path: /
    pool: zpool
    size: 32GB
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""

Not: This configuration has just one network interface to test the pfsense installation please feel free to add another interfaces. And this simple quick documentation is not new just gathering some of the information together. And lastly thanks to the lxd team to make this happen, keep up the good work. Some of the points can be inaccurate. Happy pfsensing.

tomp · December 2, 2021, 8:45am

Thanks! I’ve moved this to Tutorials.

ahmadqz · April 18, 2022, 4:55pm

Thanks! Worked fine for me, easy and straightforward.

cemzafer · April 18, 2022, 5:04pm

You are welcome.
Regards.

Andrew_Wilson · February 3, 2023, 5:55pm

Hi:

I tried this but it fails for me. My setup/steps/output:
lxd version 5.0.2 on a dual EPYC server with lots of ram/cpu/storage:

lxc init pfsense --empty --vm -c limits.cpu=4 -c limits.memory=8GB -c security.secureboot=false -n lxdbr0
lxc config device override pfsense root size=32GB
echo -n ‘-machine pc-q35-2.6’ | sudo lxc config set pfsense raw.qemu -
lxc config device add pfsense install disk source=/home/andrew/pfSense-CE-memstick-2.6.0-RELEASE-amd64.img boot.priority=10
lxc start pfsense

Error: Failed to run: forklimits limit=memlock:unlimited:unlimited fd=3 – /snap/lxd/24322/bin/qemu-system-x86_64 -S -name pfsense -uuid 7f7ed5ca-7201-49de-966d-9fc03ba7cbbe -daemonize -cpu host,hv_passthrough -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=allow,resourcecontrol=deny -readconfig /var/snap/lxd/common/lxd/logs/pfsense/qemu.conf -spice unix=on,disable-ticketing=on,addr=/var/snap/lxd/common/lxd/logs/pfsense/qemu.spice -pidfile /var/snap/lxd/common/lxd/logs/pfsense/qemu.pid -D /var/snap/lxd/common/lxd/logs/pfsense/qemu.log -smbios type=2,manufacturer=Canonical Ltd.,product=LXD -runas lxd -machine pc-q35-2.6: : Process exited with non-zero value 1

And when I run lxc info --show-log pfsense I get:

Name: pfsense
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2023/02/03 17:47 UTC

Log:

qemu-system-x86_64: Invalid SMP CPUs 256. The max CPUs supported by machine ‘pc-q35-2.6’ is 255

So it’s as if qemu is trying to load all my CPU’s into the VM (I do have 256 cores, but I only assign four of them). Any suggestions as to what I am doing wrong please? THanks in anticipation.

Andrew

cemzafer · February 3, 2023, 6:23pm

Hi @Andrew_Wilson, I’am not so sure but can you try like that.

echo -n ‘-machine pc-q35-2.6 -smp 2’ | sudo lxc config set pfsense raw.qemu -

Regards.

Andrew_Wilson · February 3, 2023, 6:29pm

Wow, super fast - thank you. I tried the command on a new pfsense vm and it also failed albeit with a new error:

Error: Failed to run: forklimits limit=memlock:unlimited:unlimited fd=3 – /snap/lxd/24322/bin/qemu-system-x86_64 -S -name pfsense -uuid f16ae3df-4b20-4440-a34f-2dd21e7a4fac -daemonize -cpu host,hv_passthrough -nographic -serial chardev:console -nodefaults -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=allow,resourcecontrol=deny -readconfig /var/snap/lxd/common/lxd/logs/pfsense/qemu.conf -spice unix=on,disable-ticketing=on,addr=/var/snap/lxd/common/lxd/logs/pfsense/qemu.spice -pidfile /var/snap/lxd/common/lxd/logs/pfsense/qemu.pid -D /var/snap/lxd/common/lxd/logs/pfsense/qemu.log -smbios type=2,manufacturer=Canonical Ltd.,product=LXD -runas lxd ‘-machine pc-q35-2.6 -smp 2’: : Process exited with non-zero value 1

And I get this from ‘lxc info --show-log pfsense’:

Name: pfsense
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2023/02/03 18:25 UTC

Log:

qemu-system-x86_64: ‘-machine: Could not open ‘‘-machine’: No such file or directory

cemzafer · February 3, 2023, 6:35pm

Hi again, this problem discussed on this thread, have a look at that post, please.
https://discuss.linuxcontainers.org/t/cant-start-vm-failed-to-run-forklimits-limit-memlockunlimited-lxd-4-0-9-lts-snap/15827
Regards.

Andrew_Wilson · February 3, 2023, 6:36pm

Many thanks!

cemzafer · February 3, 2023, 6:48pm

You are welcome.

macduff23 · May 2, 2023, 5:17pm

I’m getting err on line 3.
uname -r
5.4.0-148-generic
lxd 5.x
lxd git-976cd2b 24814 latest/edge canonical
root@shinto:/home/ubuntu# echo -n ‘-machine pc-q35-2.6’ | sudo lxc config set pfsense raw.qemu

Error: cannot set ‘pfsense’ to ‘raw.qemu’: unknown key

So I proceed anyway,able to complete the rest of the commands
However Lxd barks about…
root@shinto:/home/ubuntu# lxc start pfsense && lxc console pfsense --type=vga
LXD relies on either remote-viewer or spicy to provide VGA console access.
Those can’t be bundled with the LXD snap and so need to be manually installed.

“remote-viewer” usually comes in a package called virt-viewer
“spicy” usually comes in a package called spice-client-gtk
Install either of those and they will automatically start.
Alternatively you may use another SPICE client using the following URI:
spice+unix:///root/snap/lxd/common/config/sockets/2222276300.spice

fyi - for others that may attempt.

apt-get install virt-viewer -
which is a hefty package for a vm installing GTK libs

then
lxc start pfsense && lxc console pfsense --type=vga
Unable to init server: Could not connect: Connection refused

(remote-viewer:25364): Gtk-WARNING **: 17:36:55.399: cannot open display:…

I had this working as well at one time on bionic /iso. But I thinK also had kvm/virsh and some other libs installed… so maybe some by chance libs helped out with lxd.

cemzafer · May 3, 2023, 1:53am

Hi @macduff23,
The line you mentioned replaced with that line with dash at the end.
echo -n ‘-machine pc-q35-2.6’ | sudo lxc config set pfsense raw.qemu -

Regards.

pmarini · May 4, 2023, 5:03pm

Thanks! Worked for me perfectly too… nice!