Can't register storage pool with incus after import

IncusOS
,
1

Hey all!

I’m exploring IncusOS for use in my homelab and currently looking into disaster recovery: “what if the boot drive fails?”

The goal is to have most (if not all) services running as a Linux container, including my NAS.

The idea is to attach two ZFS volumes to the NAS: one for regular shares and one for time machine backups.

I created the pool, and volumes, like so:

# Create the pool using edit
$ incus admin os system storage edit

# Create a data volume
$ incus admin os system storage create-volume -d ‘{“pool”:“data”,“name”:“incus”,“use”:“incus”}’

# Register it with incus
$ incus storage create data zfs source=data/incus

# Create volumes
$ incus storage volume create data shares
$ incus storage volume create data timemachine

To test the disaster recovery I did a clean install of IncusOS and imported the existing encrypted data pool using the backed-up encryption key:

$ incus admin os system storage import-storage-pool \
             '{"name":"data","type":"zfs","encryption_key":"my-key"}'

The import succeeds and IncusOS is able to see the data pool and its existing volume:

$ incus admin os system storage show
WARNING: The IncusOS API and configuration is subject to change

config:
  scrub_schedule: 0 4 * * 0
state:
  drives:
   ...
  pools:
   ...
  - devices:
    - /dev/disk/by-id/ata-WDC_WD6005FFBX-68CASN0_WD-AM023HZT
    - /dev/disk/by-id/ata-WDC_WD6005FFBX-68CASN0_WD-AM02457T
    - /dev/disk/by-id/ata-WDC_WD6005FFBX-68CASN0_WD-AM02H87T
    - /dev/disk/by-id/ata-WDC_WD6005FFBX-68CASN0_WD-AM02K4DT
    encryption_key_status: available
    name: data
    pool_allocated_space_in_bytes: 9.05216e+06
    raw_pool_size_in_bytes: 2.4000277250048e+13
    state: ONLINE
    type: zfs-raidz1
    usable_pool_size_in_bytes: 1.7437701439488e+13
    volumes:
    - name: incus
      quota_in_bytes: 0
      usage_in_bytes: 3.678336e+06
      use: incus

After this import, incus does not know about the pool yet:

$ incus storage list
+-------+--------+--------------------------------------+---------+---------+
| NAME  | DRIVER |             DESCRIPTION              | USED BY |  STATE  |
+-------+--------+--------------------------------------+---------+---------+
| local | zfs    | Local storage pool (on system drive) | 4       | CREATED |
+-------+--------+--------------------------------------+---------+---------+

Trying to register the pool using incus storage create data zfs source=data/incus fails with Provided ZFS pool (or dataset) isn't empty and incus admin recover is not available on IncusOS.

I feel like I’m missing something but I can’t seem to figure out what. What am I doing wrong? Any help in figuring this out would be greatly appreciated!

2

What happens with incus admin recover? We did add remote support to it for the IncusOS case but we may well have missed something.

3

It’s not available, it seems:

$ incus admin
Description:
  Manage incus daemon

Usage:
  incus admin [command]

Available Commands:
  os          Manage IncusOS systems

Global Flags:
      --debug          Show all debug messages
      --force-local    Force using the local unix socket
  -h, --help           Print help
      --project        Override the source project
  -q, --quiet          Don't show progress information
      --sub-commands   Use with help or --help to view sub-commands
  -v, --verbose        Show all information messages
      --version        Print version number

Use "incus admin [command] --help" for more information about a command.

I’m running version 202604100425, which is the latest version for as far as I can tell. I have only the incus application installed.

4

What’s the incus version output on your system?
I’m also assuming you’re running the client on a non-Linux system?

5

Running incus version gives me:

Client version: 6.23
Server version: 6.23

You assumed correct! I’m running this on macOS :sweat_smile: Should the admin recover command be there?

Edit: I’ve installed the incus client on a Debian VM. Running incus admin shows a lot more subcommands than it does on macOS, including recover.

I then downloaded the x86 version of the macOS client to see if it could be arch related (long shot, but verifying doesn’t hurt). The x86 behaves the same as aarch64.

Running the code directly also behaves the same way: no recover subcommand. I’m now trying to figure out why the binary behaves differently between different platforms.

The admin_recover.go file has //go:build linux at the top, so it’s being excluded in the case of a macOS build. Is there any particular reason for this?

Edit 2: I’ve modified the code a bit to enable both the sql and recover subcommand for a macOS build. It took a bit of trial and error but it looks like the recovery was successful. I managed to recover the pool along with the two custom volumes!

6

Yeah, we need to re-enable a few commands on non-Linux.

7
1 Like