Home Assistant OS (HA OS) VM vs system container in Incus

I have been running Home Assistant as a HA OS VM for a number of years in ESXi and then TrueNAS SCALE. I prefer having all of the supervised and other features that HA OS provide vs. other installation options HA supports.

Is it possible — or even desirable — to run HA OS as a system container? I’m trying to figure out what makes sense. My impression is that this probably wouldn’t work and that a VM would continue to be a better fit.

Background:

I recently found Incus, and it looks much closer to what I’ve been wanting for years than anything else I’ve come across.

Unified management of VMs and containers (now with system and app containers) with one UI sounds great for someone who is basically dabbling in all this stuff (primarily at home), especially when the mental “to do” list of things to tinker with only ever seems to grow.

Thanks!

Welcome!

Home Assistant (HA) Operating System (OS) is available as an image to run in a VM. It could be possible to set it up in a system container (compared to an Incus VM), though it would require to somehow convert the available image in a format that is usable for Incus system containers. It’s possible to do so, see the instructions at How to import physical or virtual machines to Incus instances - Incus documentation
And report back on the outcome.

The other option is to setup HAOS in an Incus VM.

The third option is to use the HA OCI container image from Linux - Home Assistant Incus supports OCI images, so it might be easily usable to install HA as such. This would probably be your last option because you cannot install HA plugins.

The fourth option is to create an Incus container image for Home Assistant.
There’s a tool for that, GitHub - lxc/distrobuilder: System container image builder for LXC and Incus and these are the images that are already available, https://images.linuxcontainers.org/
To create such an image, you need to make a special distrobuilder configuration like the ones found here, lxc-ci/images at main · lxc/lxc-ci · GitHub
If such an image becomes available, then it would be as easy as running the following, to get an installation of HA.

incus install images:ha/2024.9.2 myhomeassistant

Hello,

I ve tried the first option. Nothing fancy, just executed incus-migrate with the qcow2 image of hoas. It said the migration was successfull, but when i try to start the instance ( a container ), it says:

Error: Failed getting instance pool: Instance storage pool not found
Try incus info --show-log haos2 for more info

the log doesn t not help.

when i check the pool, i don t see the new container using it. It must be the problem, but i don t know how to solve it… Any suggestions?

I have incus 6 installed. qcow2 image support has been added in 6.6. Is there a way i could do a migration without installing from source?
i have debian bookworm.

GitHub - zabbly/incus: Incus package repository would get you the latest Incus on Debian 12.

Alternatively if it’s just incus-migrate that you need, you can fetch the binary build of that tool from the most recent release on Github.

yes i ve found that it was a standalone binary. I downloaded the 6.7 on github. But same result:

incus start haos3                                                                                                     ─╯
Error: Failed getting instance pool: Instance storage pool not found
Try `incus info --show-log haos3` for more info

I ll try to export it and import it. I don t get where the instance is located if not in the storage pool

incus storage show incus-vms

doesnt show in used_by section…

Maybe i can add the info of the storage with a command but i couldn t find anything about it.

I ll report with the result of the export import thing.

that was fast.

incus export haos3 haos3.tar.gz                                                                                       ─╯
Error: Create backup: Failed loading instance storage pool: Failed getting instance storage pool name: Failed getting instance pool: Instance storage pool not found

incus config show --expanded haos3

I have no idea where the instance could be… It took a several minutes to complete the migration.

incus config show --expanded haos3
architecture: x86_64
config:
  volatile.cloud-init.instance-id: d31d3765-ac4a-4d25-a25c-a1f54d13e9ca
  volatile.eth0.hwaddr: 00:16:3e:cf:c2:4e
  volatile.idmap.base: "0"
  volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
  volatile.last_state.idmap: '[]'
  volatile.uuid: d9c08536-f5d8-449e-b269-382e01ddfed9
  volatile.uuid.generation: d9c08536-f5d8-449e-b269-382e01ddfed9
devices:
  eth0:
    name: eth0
    nictype: bridged
    parent: incusbr0
    type: nic
  root:
    path: /
    pool: incus-vms
    type: disk
ephemeral: false
profiles:
- default
stateful: false
description: ""
incus storage show incus-vms
config:
  source: incus-vms
  volatile.initial_source: /dev/sda2
  zfs.pool_name: incus-vms
description: ""
name: incus-vms
driver: zfs
used_by:
- /1.0/instances/opnsense
- /1.0/instances/vpn-fr
- /1.0/profiles/default
status: Created
locations:
- none

That’s definitely a very weird one…

Can you show zfs list -t all?

The only thing i did without knowing if it was right is put “/” when it asks for a root filesystem. I migrate the image on the incus server.

zfs list -t all
NAME                                                                                                 USED  AVAIL     REFER  MOUNTPOINT
incus-vms                                                                                           6.52G  78.3G      192K  legacy
incus-vms/buckets                                                                                    192K  78.3G      192K  legacy
incus-vms/containers                                                                                2.31M  78.3G      192K  legacy
incus-vms/containers/vpn-fr                                                                         2.12M  78.3G     13.6M  legacy
incus-vms/custom                                                                                     192K  78.3G      192K  legacy
incus-vms/deleted                                                                                   13.7M  78.3G      192K  legacy
incus-vms/deleted/buckets                                                                            192K  78.3G      192K  legacy
incus-vms/deleted/containers                                                                         192K  78.3G      192K  legacy
incus-vms/deleted/custom                                                                             192K  78.3G      192K  legacy
incus-vms/deleted/images                                                                            12.8M  78.3G      192K  legacy
incus-vms/deleted/images/b0178f3ccb7c187d137eea25903e6f726657d86d02652e98866b2598ca2bd882           12.6M  78.3G     12.6M  legacy
incus-vms/deleted/images/b0178f3ccb7c187d137eea25903e6f726657d86d02652e98866b2598ca2bd882@readonly    16K      -     12.6M  -
incus-vms/deleted/virtual-machines                                                                   192K  78.3G      192K  legacy
incus-vms/images                                                                                     192K  78.3G      192K  legacy
incus-vms/virtual-machines                                                                          6.42G  78.3G      192K  legacy
incus-vms/virtual-machines/opnsense                                                                 7.85M   492M     7.85M  legacy
incus-vms/virtual-machines/opnsense.block                                                           6.41G  78.3G     6.41G 

Okay, so that confirms it’s definitely not there…
Is that a standalone system or a cluster?

Can you show ls -lh /var/lib/incus/containers/?

It s a standalone server.

total 0
lrwxrwxrwx 1 root root 56 Apr 22  2024 vpn-fr -> /var/lib/incus/storage-pools/incus-vms/containers/vpn-fr

Yeah, so definitely looks like the migration didn’t really happen…

Do you still have the source system and could give it another shot?
Basically deleting the instance that got created and trying a migration again.

I could try that, but i tried 3 times already so the 3 in haos3.

Only the last one have been made with incus-migrate 6.7

incus ls
+----------+---------+---------------------+------+-----------------+-----------+
|   NAME   |  STATE  |        IPV4         | IPV6 |      TYPE       | SNAPSHOTS |
+----------+---------+---------------------+------+-----------------+-----------+
| haos     | STOPPED |                     |      | CONTAINER       | 0         |
+----------+---------+---------------------+------+-----------------+-----------+
| haos2    | STOPPED |                     |      | CONTAINER       | 0         |
+----------+---------+---------------------+------+-----------------+-----------+
| haos3    | STOPPED |                     |      | CONTAINER       | 0         |
+----------+---------+---------------------+------+-----------------+-----------+
| opnsense | RUNNING | 192.168.0.1 (eth0)  |      | VIRTUAL-MACHINE | 0         |
+----------+---------+---------------------+------+-----------------+-----------+
| vpn-fr   | RUNNING | 192.168.x.x (wg)  |      | CONTAINER       | 0         |
|          |         | 192.168.0.50 (eth0) |      |                 |           |
+----------+---------+---------------------+------+-----------------+-----------+

Maybe i should try to convert the qcow2 into a raw image?

if you have time this is the image

https://github.com/home-assistant/operating-system/releases/download/13.2/haos_ova-13.2.qcow2.xz

Okay, that’s a VM image so you should import it as a VM.

root@dakara:~/Downloads# ls -lh *qcow2
-rw-r--r-- 1 stgraber domain admins 867M Nov 26 14:45 haos_ova-13.2.qcow2
root@dakara:~/Downloads# incus-migrate 
The local Incus server is the target [default=yes]: 
Would you like to create a container (1) or virtual-machine (2)?: 2
Project to create the instance in [default=default]: 
Name of the new instance: haos
Please provide the path to a disk, partition, or qcow2/raw/vmdk image file: haos_ova-13.2.qcow2
Does the VM support UEFI booting? [default=yes]: 
Does the VM support UEFI Secure Boot? [default=yes]: no

Instance to be created:
  Name: haos
  Project: default
  Type: virtual-machine
  Source: haos_ova-13.2.qcow2
  Source format: qcow2
  Config:
    security.secureboot: "false"

Additional overrides can be applied at this stage:
1) Begin the migration with the above configuration
2) Override profile list
3) Set additional configuration options
4) Change instance storage pool or volume size
5) Change instance network

Please pick one of the options above [default=1]: 1
Converting image "haos_ova-13.2.qcow2" to raw format before importing
Instance haos successfully created                
root@dakara:~/Downloads# incus start haos
root@dakara:~/Downloads# 

Oh sorry for the misunderstanding.

I ve read this sentence and understood it wrong

The tool can create both containers and virtual machines:

I thought that it could make a standalone container from a vm. that would be awesome but maybe impossible