Lxd-to-incus failed - no such table: main.auth_groups_permissions

west17m · December 23, 2025, 3:14pm

I’m trying to migrate an lxd instance to incus using lxd-to-incus and the migration failed.

Here is, I think the pertenent line when incusd tried to start

Error: Failed to initialize global database: Failed to prepare statements: "\nDELETE FROM instances_snapshots WHERE instance_id = (SELECT instances.id FROM instances JOIN projects ON instances.project_id = projects.id WHERE projects.name = ? AND instances.name = ?) AND name = ?\n": no such table: main.auth_groups_permissions

Lxd-to-incus migration failed: Failed to initialize global database - #12 by disputin looked similar, but I couldn’t figure out how to fix the database. I would appreciate any assistance.

More details:

Host: gentoo (openrc); kernel 6.12.41; incus 6.0.5 (portage); zfs storage

lxd-to-incus.log

cat /var/log/lxd-to-incus.1186073.log
Source server: manual installation
Target server: openrc
Source server paths: &{daemon:/var/lib/lxd logs:/var/log/lxd cache:/var/cache/lxd}
Target server paths: &{daemon:/var/lib/incus logs:/var/log/incus cache:/var/cache/incus}
Rewrite SQL statements:
 - UPDATE profiles SET description='Default Incus profile' WHERE description='Default LXD profile';
 - UPDATE projects SET description='Default Incus project' WHERE description='Default LXD project';
 - DELETE FROM storage_volumes_config WHERE key='volatile.uuid';
 - DELETE FROM storage_volumes_snapshots_config WHERE key='volatile.uuid';
 - DELETE FROM instances_config WHERE key='volatile.uuid';
 - DELETE FROM instances_snapshots_config WHERE key='volatile.uuid';
 - CREATE TABLE certificates (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    fingerprint TEXT NOT NULL,
    type INTEGER NOT NULL,
    name TEXT NOT NULL,
    certificate TEXT NOT NULL,
    restricted INTEGER NOT NULL DEFAULT 0,
    UNIQUE (fingerprint)
);
CREATE TABLE "certificates_projects" (
    certificate_id INTEGER NOT NULL,
    project_id INTEGER NOT NULL,
    FOREIGN KEY (certificate_id) REFERENCES certificates (id) ON DELETE CASCADE,
    FOREIGN KEY (project_id) REFERENCES "projects" (id) ON DELETE CASCADE,
    UNIQUE (certificate_id, project_id)
);
 - DELETE FROM schema WHERE version < 73;
UPDATE schema SET version=69 WHERE version=73;
 - INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 1, name, json_extract(metadata, "$.cert"), 1 FROM identities WHERE type=1;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 1, name, json_extract(metadata, "$.cert"), 0 FROM identities WHERE type=2;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 2, name, json_extract(metadata, "$.cert"), 0 FROM identities WHERE type=3;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 3, name, json_extract(metadata, "$.cert"), 1 FROM identities WHERE type=4;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 3, name, json_extract(metadata, "$.cert"), 0 FROM identities WHERE type=6;
INSERT INTO certificates_projects (certificate_id, project_id) SELECT identity_id, project_id FROM identities_projects;
 - DROP TRIGGER IF EXISTS on_auth_group_delete;
DROP TRIGGER IF EXISTS on_cluster_group_delete;
DROP TRIGGER IF EXISTS on_identity_delete;
DROP TRIGGER IF EXISTS on_identity_provider_group_delete;
DROP TRIGGER IF EXISTS on_image_alias_delete;
DROP TRIGGER IF EXISTS on_image_delete;
DROP TRIGGER IF EXISTS on_instance_backup_delete;
DROP TRIGGER IF EXISTS on_instance_delete;
DROP TRIGGER IF EXISTS on_instance_snapshot_delete;
DROP TRIGGER IF EXISTS on_network_acl_delete;
DROP TRIGGER IF EXISTS on_network_delete;
DROP TRIGGER IF EXISTS on_network_zone_delete;
DROP TRIGGER IF EXISTS on_node_delete;
DROP TRIGGER IF EXISTS on_operation_delete;
DROP TRIGGER IF EXISTS on_profile_delete;
DROP TRIGGER IF EXISTS on_project_delete;
DROP TRIGGER IF EXISTS on_storage_bucket_delete;
DROP TRIGGER IF EXISTS on_storage_pool_delete;
DROP TRIGGER IF EXISTS on_storage_volume_backup_delete;
DROP TRIGGER IF EXISTS on_storage_volume_delete;
DROP TRIGGER IF EXISTS on_storage_volume_snapshot_delete;
DROP TRIGGER IF EXISTS on_warning_delete;
DROP TABLE IF EXISTS identities_projects;
DROP TABLE IF EXISTS auth_groups_permissions;
DROP TABLE IF EXISTS auth_groups_identity_provider_groups;
DROP TABLE IF EXISTS identities_auth_groups;
DROP TABLE IF EXISTS identity_provider_groups;
DROP TABLE IF EXISTS identities;
DROP TABLE IF EXISTS auth_groups;
Rewrite commands:
Migration started
Stopping the source server
Stopping the target server
Unmounting "/var/lib/incus/devlxd"
Unmounting "/var/lib/incus/shmounts"
Wiping the target server
Migrating the data
Moving data over
Migrating database files
Writing the database patch
Cleaning up target paths
Cleaning up path "/var/lib/incus/backups"
Cleaning up path "/var/lib/incus/images"
Cleaning up path "/var/lib/incus/devices"
Cleaning up path "/var/lib/incus/devlxd"
Cleaning up path "/var/lib/incus/security"
Cleaning up path "/var/lib/incus/shmounts"
Rewrite symlinks:
 - "/var/lib/incus/storage-pools/default/containers/ansible" to "/var/lib/incus/containers/ansible"
[snip - other containers]
 - "/var/lib/incus/storage-pools/default/containers-snapshots/www-maintenance" to "/var/lib/incus/snapshots/www-maintenance"
Rewrite symlinks:
Rewrite symlinks:
Starting the target server
Checking target server
ERROR: Get "http://unix.socket/1.0": dial unix /var/lib/incus/unix.socket: connect: no such file or directory

Both incus and incus-user show crashed, and I don’t get a meaningful error when restarting. Here is the error when manually running incusd –debug

incusd --debug
INFO   [2025-12-23T09:04:34-06:00] Starting up                                   mode=normal path=/var/lib/incus version=6.0.5
INFO   [2025-12-23T09:04:34-06:00] System idmap (root user):
INFO   [2025-12-23T09:04:34-06:00]  - u 0 1000000 1000000000
INFO   [2025-12-23T09:04:34-06:00]  - g 0 1000000 1000000000
INFO   [2025-12-23T09:04:34-06:00] Selected idmap:
INFO   [2025-12-23T09:04:34-06:00]  - u 0 1000000 1000000000
INFO   [2025-12-23T09:04:34-06:00]  - g 0 1000000 1000000000
WARNING[2025-12-23T09:04:34-06:00] AppArmor support has been disabled because of lack of kernel support
WARNING[2025-12-23T09:04:34-06:00] Unsupported CGroup setup detected, V1 controllers on top of V2 root
INFO   [2025-12-23T09:04:34-06:00] Kernel features:
INFO   [2025-12-23T09:04:34-06:00]  - closing multiple file descriptors efficiently: yes
INFO   [2025-12-23T09:04:34-06:00]  - netnsid-based network retrieval: yes
INFO   [2025-12-23T09:04:34-06:00]  - pidfds: yes
INFO   [2025-12-23T09:04:34-06:00]  - pidfds for threads: yes
INFO   [2025-12-23T09:04:34-06:00]  - core scheduling: no
INFO   [2025-12-23T09:04:34-06:00]  - uevent injection: yes
INFO   [2025-12-23T09:04:34-06:00]  - seccomp listener: yes
INFO   [2025-12-23T09:04:34-06:00]  - seccomp listener continue syscalls: yes
INFO   [2025-12-23T09:04:34-06:00]  - seccomp listener add file descriptors: yes
INFO   [2025-12-23T09:04:34-06:00]  - attach to namespaces via pidfds: yes
INFO   [2025-12-23T09:04:34-06:00]  - safe native terminal allocation: yes
INFO   [2025-12-23T09:04:34-06:00]  - unprivileged binfmt_misc: yes
INFO   [2025-12-23T09:04:34-06:00]  - unprivileged file capabilities: yes
INFO   [2025-12-23T09:04:34-06:00]  - cgroup layout: cgroup2
WARNING[2025-12-23T09:04:34-06:00]  - AppArmor support has been disabled, Disabled because of lack of kernel support
WARNING[2025-12-23T09:04:34-06:00]  - Couldn't find the CGroup memory swap accounting, swap limits will be ignored
INFO   [2025-12-23T09:04:34-06:00]  - idmapped mounts kernel support: yes
DEBUG  [2025-12-23T09:04:34-06:00] QMP monitor started                           path=/tmp/214993497
DEBUG  [2025-12-23T09:04:34-06:00] Failed adding block device during VM feature check  err="Failed adding block device: GenericError: Parameter 'aio' does not accept value 'io_uring'"
DEBUG  [2025-12-23T09:04:34-06:00] QMP monitor stopped                           path=/tmp/214993497
INFO   [2025-12-23T09:04:34-06:00] Instance type operational                     driver=qemu type=virtual-machine
INFO   [2025-12-23T09:04:34-06:00] Instance type operational                     driver=lxc type=container
WARNING[2025-12-23T09:04:34-06:00] Unable to access device nodes, likely running on a nodev mount
INFO   [2025-12-23T09:04:34-06:00] Initializing local database
DEBUG  [2025-12-23T09:04:34-06:00] Refreshing local trusted certificate cache
INFO   [2025-12-23T09:04:34-06:00] Set client certificate to server certificate  fingerprint=1849f5771334d86482e57dcf101446678dd9686f024da09d8b4e9cff52152e55
DEBUG  [2025-12-23T09:04:34-06:00] Initializing database gateway
INFO   [2025-12-23T09:04:34-06:00] Starting database node                        id=1 local=1 role=voter
INFO   [2025-12-23T09:04:35-06:00] Loading daemon configuration
INFO   [2025-12-23T09:04:35-06:00] Binding socket                                socket=/var/lib/incus/unix.socket type="REST API Unix socket"
INFO   [2025-12-23T09:04:35-06:00] Binding socket                                socket=/var/lib/incus/guestapi/sock type="devIncus socket"
INFO   [2025-12-23T09:04:35-06:00] Binding socket                                socket="host(2):34068" type="VM socket"
INFO   [2025-12-23T09:04:35-06:00] Initializing global database
INFO   [2025-12-23T09:04:35-06:00] Connecting to global database
DEBUG  [2025-12-23T09:04:35-06:00] Dqlite: attempt 1: server 1: connected
INFO   [2025-12-23T09:04:35-06:00] Connected to global database
ERROR  [2025-12-23T09:04:35-06:00] Failed to start the daemon                    err="Failed to initialize global database: Failed to prepare statements: \"\\nDELETE FROM instances_snapshots WHERE instance_id = (SELECT instances.id FROM instances JOIN projects ON instances.project_id = projects.id WHERE projects.name = ? AND instances.name = ?) AND name = ?\\n\": no such table: main.auth_groups_permissions"
INFO   [2025-12-23T09:04:35-06:00] Starting shutdown sequence                    signal=interrupt
DEBUG  [2025-12-23T09:04:35-06:00] Cancel ongoing or future gRPC connection attempts
DEBUG  [2025-12-23T09:04:35-06:00] Stop database gateway
INFO   [2025-12-23T09:04:35-06:00] Closing socket                                socket=/var/lib/incus/unix.socket type="REST API Unix socket"
INFO   [2025-12-23T09:04:35-06:00] Closing socket                                socket=/var/lib/incus/guestapi/sock type="devIncus socket"
INFO   [2025-12-23T09:04:35-06:00] Closing socket                                socket="host(2):34068" type="VM socket"
INFO   [2025-12-23T09:04:35-06:00] Not unmounting temporary filesystems (instances are still running)
INFO   [2025-12-23T09:04:35-06:00] Daemon stopped
Error: Failed to initialize global database: Failed to prepare statements: "\nDELETE FROM instances_snapshots WHERE instance_id = (SELECT instances.id FROM instances JOIN projects ON instances.project_id = projects.id WHERE projects.name = ? AND instances.name = ?) AND name = ?\n": no such table: main.auth_groups_permissions

stgraber · December 23, 2025, 3:26pm

What version of LXD is that?

west17m · December 23, 2025, 3:31pm

lxd 5.21.1 LTS

stgraber · December 23, 2025, 3:40pm

Do you have a file currently located at /var/lib/incus/database/patch.global.sql?
If so, what’s its content?

If not, we’ll want to re-create it from the value you showed in the log above.

west17m · December 23, 2025, 3:44pm

I do not see it. Here is what I do have. Thanks so much for responding so fast.

ls -1R /var/lib/incus/database/
/var/lib/incus/database/:
global
global.bak
local.db
local.db.bak

/var/lib/incus/database/global:
0000000000132097-0000000000132911
[snip data files]
0000000000140642-0000000000140642
db.bin
db.bin-wal
dqlite-lock
metadata1
snapshot-1-139264-21560708936
snapshot-1-139264-21560708936.meta
snapshot-1-140288-22406709064
snapshot-1-140288-22406709064.meta

/var/lib/incus/database/global.bak:
0000000000132097-0000000000132911
[snip data files]
0000000000140514-0000000000140641
db.bin
db.bin-wal
dqlite-lock
metadata1
open-1
open-2
snapshot-1-139264-21560708936
snapshot-1-139264-21560708936.meta
snapshot-1-140288-22406709064
snapshot-1-140288-22406709064.meta

stgraber · December 23, 2025, 4:24pm

Okay, can you:

Stop Incus
Restore /var/lib/incus/database/patch.global.sql with the content showed in the migration log
Start Incus
Post the startup log (likely will still fail, but should fail differently)

west17m · December 23, 2025, 4:39pm

Okay,

incus is stopped

/etc/init.d/incus status
 * status: stopped

patch.global.sql

UPDATE profiles SET description='Default Incus profile' WHERE description='Default LXD profile';
UPDATE projects SET description='Default Incus project' WHERE description='Default LXD project';
DELETE FROM storage_volumes_config WHERE key='volatile.uuid';
DELETE FROM storage_volumes_snapshots_config WHERE key='volatile.uuid';
DELETE FROM instances_config WHERE key='volatile.uuid';
DELETE FROM instances_snapshots_config WHERE key='volatile.uuid';
CREATE TABLE certificates (
    id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
    fingerprint TEXT NOT NULL,
    type INTEGER NOT NULL,
    name TEXT NOT NULL,
    certificate TEXT NOT NULL,
    restricted INTEGER NOT NULL DEFAULT 0,
    UNIQUE (fingerprint)
);
CREATE TABLE "certificates_projects" (
    certificate_id INTEGER NOT NULL,
    project_id INTEGER NOT NULL,
    FOREIGN KEY (certificate_id) REFERENCES certificates (id) ON DELETE CASCADE,
    FOREIGN KEY (project_id) REFERENCES "projects" (id) ON DELETE CASCADE,
    UNIQUE (certificate_id, project_id)
);
DELETE FROM schema WHERE version < 73;
UPDATE schema SET version=69 WHERE version=73;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 1, name, json_extract(metadata, "$.cert"), 1 FROM identities WHERE type=1;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 1, name, json_extract(metadata, "$.cert"), 0 FROM identities WHERE type=2;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 2, name, json_extract(metadata, "$.cert"), 0 FROM identities WHERE type=3;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 3, name, json_extract(metadata, "$.cert"), 1 FROM identities WHERE type=4;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 3, name, json_extract(metadata, "$.cert"), 0 FROM identities WHERE type=6;
INSERT INTO certificates_projects (certificate_id, project_id) SELECT identity_id, project_id FROM identities_projects;
DROP TRIGGER IF EXISTS on_auth_group_delete;
DROP TRIGGER IF EXISTS on_cluster_group_delete;
DROP TRIGGER IF EXISTS on_identity_delete;
DROP TRIGGER IF EXISTS on_identity_provider_group_delete;
DROP TRIGGER IF EXISTS on_image_alias_delete;
DROP TRIGGER IF EXISTS on_image_delete;
DROP TRIGGER IF EXISTS on_instance_backup_delete;
DROP TRIGGER IF EXISTS on_instance_delete;
DROP TRIGGER IF EXISTS on_instance_snapshot_delete;
DROP TRIGGER IF EXISTS on_network_acl_delete;
DROP TRIGGER IF EXISTS on_network_delete;
DROP TRIGGER IF EXISTS on_network_zone_delete;
DROP TRIGGER IF EXISTS on_node_delete;
DROP TRIGGER IF EXISTS on_operation_delete;
DROP TRIGGER IF EXISTS on_profile_delete;
DROP TRIGGER IF EXISTS on_project_delete;
DROP TRIGGER IF EXISTS on_storage_bucket_delete;
DROP TRIGGER IF EXISTS on_storage_pool_delete;
DROP TRIGGER IF EXISTS on_storage_volume_backup_delete;
DROP TRIGGER IF EXISTS on_storage_volume_delete;
DROP TRIGGER IF EXISTS on_storage_volume_snapshot_delete;
DROP TRIGGER IF EXISTS on_warning_delete;
DROP TABLE IF EXISTS identities_projects;
DROP TABLE IF EXISTS auth_groups_permissions;
DROP TABLE IF EXISTS auth_groups_identity_provider_groups;
DROP TABLE IF EXISTS identities_auth_groups;
DROP TABLE IF EXISTS identity_provider_groups;
DROP TABLE IF EXISTS identities;
DROP TABLE IF EXISTS auth_groups;

start incus

incusd --debug
INFO   [2025-12-23T10:37:45-06:00] Starting up                                   mode=normal path=/var/lib/incus version=6.0.5
INFO   [2025-12-23T10:37:45-06:00] System idmap (root user):
INFO   [2025-12-23T10:37:45-06:00]  - u 0 1000000 1000000000
INFO   [2025-12-23T10:37:45-06:00]  - g 0 1000000 1000000000
INFO   [2025-12-23T10:37:45-06:00] Selected idmap:
INFO   [2025-12-23T10:37:45-06:00]  - u 0 1000000 1000000000
INFO   [2025-12-23T10:37:45-06:00]  - g 0 1000000 1000000000
WARNING[2025-12-23T10:37:45-06:00] AppArmor support has been disabled because of lack of kernel support
WARNING[2025-12-23T10:37:45-06:00] Unsupported CGroup setup detected, V1 controllers on top of V2 root
INFO   [2025-12-23T10:37:45-06:00] Kernel features:
INFO   [2025-12-23T10:37:45-06:00]  - closing multiple file descriptors efficiently: yes
INFO   [2025-12-23T10:37:45-06:00]  - netnsid-based network retrieval: yes
INFO   [2025-12-23T10:37:45-06:00]  - pidfds: yes
INFO   [2025-12-23T10:37:45-06:00]  - pidfds for threads: yes
INFO   [2025-12-23T10:37:45-06:00]  - core scheduling: no
INFO   [2025-12-23T10:37:45-06:00]  - uevent injection: yes
INFO   [2025-12-23T10:37:45-06:00]  - seccomp listener: yes
INFO   [2025-12-23T10:37:45-06:00]  - seccomp listener continue syscalls: yes
INFO   [2025-12-23T10:37:45-06:00]  - seccomp listener add file descriptors: yes
INFO   [2025-12-23T10:37:45-06:00]  - attach to namespaces via pidfds: yes
INFO   [2025-12-23T10:37:45-06:00]  - safe native terminal allocation: yes
INFO   [2025-12-23T10:37:45-06:00]  - unprivileged binfmt_misc: yes
INFO   [2025-12-23T10:37:45-06:00]  - unprivileged file capabilities: yes
INFO   [2025-12-23T10:37:45-06:00]  - cgroup layout: cgroup2
WARNING[2025-12-23T10:37:45-06:00]  - AppArmor support has been disabled, Disabled because of lack of kernel support
WARNING[2025-12-23T10:37:45-06:00]  - Couldn't find the CGroup memory swap accounting, swap limits will be ignored
INFO   [2025-12-23T10:37:45-06:00]  - idmapped mounts kernel support: yes
INFO   [2025-12-23T10:37:45-06:00] Instance type operational                     driver=lxc type=container
DEBUG  [2025-12-23T10:37:45-06:00] QMP monitor started                           path=/tmp/172595828
DEBUG  [2025-12-23T10:37:45-06:00] Failed adding block device during VM feature check  err="Failed adding block device: GenericError: Parameter 'aio' does not accept value 'io_uring'"
DEBUG  [2025-12-23T10:37:45-06:00] QMP monitor stopped                           path=/tmp/172595828
INFO   [2025-12-23T10:37:45-06:00] Instance type operational                     driver=qemu type=virtual-machine
WARNING[2025-12-23T10:37:45-06:00] Unable to access device nodes, likely running on a nodev mount
INFO   [2025-12-23T10:37:45-06:00] Initializing local database
DEBUG  [2025-12-23T10:37:45-06:00] Refreshing local trusted certificate cache
INFO   [2025-12-23T10:37:45-06:00] Set client certificate to server certificate  fingerprint=1849f5771334d86482e57dcf101446678dd9686f024da09d8b4e9cff52152e55
DEBUG  [2025-12-23T10:37:45-06:00] Initializing database gateway
INFO   [2025-12-23T10:37:45-06:00] Starting database node                        id=1 local=1 role=voter
INFO   [2025-12-23T10:37:46-06:00] Loading daemon configuration
INFO   [2025-12-23T10:37:46-06:00] Binding socket                                socket="host(2):36893" type="VM socket"
INFO   [2025-12-23T10:37:46-06:00] Binding socket                                socket=/var/lib/incus/unix.socket type="REST API Unix socket"
INFO   [2025-12-23T10:37:46-06:00] Binding socket                                socket=/var/lib/incus/guestapi/sock type="devIncus socket"
INFO   [2025-12-23T10:37:46-06:00] Initializing global database
INFO   [2025-12-23T10:37:46-06:00] Connecting to global database
DEBUG  [2025-12-23T10:37:46-06:00] Dqlite: attempt 1: server 1: connected
INFO   [2025-12-23T10:37:46-06:00] Connected to global database
INFO   [2025-12-23T10:37:46-06:00] Updating the global schema. Backup made as "global.bak"
DEBUG  [2025-12-23T10:37:46-06:00] Running pre-update queries from file for global DB schema
DEBUG  [2025-12-23T10:37:46-06:00] Database error                                err="failed to execute queries from /var/lib/incus/database/patch.global.sql: table certificates already exists"
ERROR  [2025-12-23T10:37:46-06:00] Failed to start the daemon                    err="Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: table certificates already exists"
INFO   [2025-12-23T10:37:46-06:00] Starting shutdown sequence                    signal=interrupt
DEBUG  [2025-12-23T10:37:46-06:00] Cancel ongoing or future gRPC connection attempts
DEBUG  [2025-12-23T10:37:46-06:00] Stop database gateway
INFO   [2025-12-23T10:37:46-06:00] Closing socket                                socket=/var/lib/incus/unix.socket type="REST API Unix socket"
INFO   [2025-12-23T10:37:46-06:00] Closing socket                                socket=/var/lib/incus/guestapi/sock type="devIncus socket"
INFO   [2025-12-23T10:37:46-06:00] Closing socket                                socket="host(2):36893" type="VM socket"
INFO   [2025-12-23T10:37:46-06:00] Not unmounting temporary filesystems (instances are still running)
INFO   [2025-12-23T10:37:46-06:00] Daemon stopped
Error: Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: table certificates already exists

Should I rerun without creating the certificates tables in patch.global.sql?

stgraber · December 23, 2025, 5:35pm

Yeah, I was about to suggest that.

west17m · December 23, 2025, 5:46pm

remove create certificates lines

ERROR  [2025-12-23T11:38:51-06:00] Failed to start the daemon                    err="Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: table \"certificates_projects\" already exists"

removed certificates_projects lines

Now I get the new error

incusd --debug
INFO   [2025-12-23T11:39:50-06:00] Starting up
...
DEBUG  [2025-12-23T11:39:51-06:00] Running pre-update queries from file for global DB schema
DEBUG  [2025-12-23T11:39:51-06:00] Database error                                err="failed to execute queries from /var/lib/incus/database/patch.global.sql: no such table: identities"
ERROR  [2025-12-23T11:39:51-06:00] Failed to start the daemon                    err="Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: no such table: identities"
INFO   [2025-12-23T11:39:51-06:00] Starting shutdown sequence                    signal=interrupt

I removed the six inserts into identities

ERROR  [2025-12-23T11:42:37-06:00] Failed to start the daemon                    err="Failed to initialize global database: failed to ensure schema: failed to apply update 69: Failed adding description column to certificate: duplicate column name: description"

with this, patch.global.sql is also gone again

trying to starting incusd without the patch.global.sql crashes with the following

incusd --debug
INFO   [2025-12-23T11:44:06-06:00] Starting up
...
INFO   [2025-12-23T11:44:06-06:00] Starting database node                        id=1 local=1 role=voter
INFO   [2025-12-23T11:44:07-06:00] Loading daemon configuration
INFO   [2025-12-23T11:44:07-06:00] Binding socket                                socket=/var/lib/incus/unix.socket type="REST API Unix socket"
INFO   [2025-12-23T11:44:07-06:00] Binding socket                                socket=/var/lib/incus/guestapi/sock type="devIncus socket"
INFO   [2025-12-23T11:44:07-06:00] Binding socket                                socket="host(2):51862" type="VM socket"
INFO   [2025-12-23T11:44:07-06:00] Initializing global database
INFO   [2025-12-23T11:44:07-06:00] Connecting to global database
DEBUG  [2025-12-23T11:44:07-06:00] Dqlite: attempt 1: server 1: connected
INFO   [2025-12-23T11:44:07-06:00] Connected to global database
ERROR  [2025-12-23T11:44:07-06:00] Failed to start the daemon                    err="Failed to initialize global database: Failed to prepare statements: \"\\nDELETE FROM instances WHERE project_id = (SELECT projects.id FROM projects WHERE projects.name = ?) AND name = ?\\n\": no such table: main.auth_groups_permissions"
INFO   [2025-12-23T11:44:07-06:00] Starting shutdown sequence                    signal=interrupt
DEBUG  [2025-12-23T11:44:07-06:00] Cancel ongoing or future gRPC connection attempts
DEBUG  [2025-12-23T11:44:07-06:00] Stop database gateway

stgraber · December 23, 2025, 5:51pm

Okay, can you try sqlite3 /var/lib/incus/database/global/db.bin .dump and then look for auth_ in that dump.

Basically we need to figure out what remaining TRIGGER is left that references that dead table and then put a DROP TRIGGER xyz; in patch.global.sql to get rid of it.

stgraber · December 23, 2025, 5:52pm

In general Incus doesn’t use any of the on_XYZ triggers, so if any of those are left in your database for some reason, they need to go away.

stgraber · December 23, 2025, 5:53pm

Looking at our code, I see that version 6.0.5 is missing this one compared to Incus 6.20:

DROP TRIGGER IF EXISTS on_instance_snaphot_delete;

So that may be what’s causing you all this trouble.

west17m · December 23, 2025, 5:57pm

Here is the single instance of auth_groups_permissions

sqlite3 /var/lib/incus/database/global/db.bin .dump | grep -A 4 -B 4 -i auth
  END;
CREATE TRIGGER on_instance_snaphot_delete
        AFTER DELETE ON instances_snapshots
        BEGIN
        DELETE FROM auth_groups_permissions
                WHERE entity_type = 7
                AND entity_id = OLD.id;
        DELETE FROM warnings
                WHERE entity_type_code = 7

and here is the single trigger on_XYZ

sqlite3 /var/lib/incus/database/global/db.bin .dump | grep -A 4 -B 4 -i 'TRIGGER on_'
  BEGIN
    SELECT RAISE(FAIL,
    "invalid ID");
  END;
CREATE TRIGGER on_instance_snaphot_delete
        AFTER DELETE ON instances_snapshots
        BEGIN
        DELETE FROM auth_groups_permissions
                WHERE entity_type = 7

I’m not an sqlite expert, so I’m not sure what to change those to

west17m · December 23, 2025, 5:57pm

and i see now it’s the same chunk

stgraber · December 23, 2025, 5:57pm

Right, bingo, that lines up with the fix that’s missing in 6.0.5…

So a patch.global.sql with:

DROP TRIGGER IF EXISTS on_instance_snaphot_delete;

Should fix things.

west17m · December 23, 2025, 6:00pm

thank you! that did it

incus list
+---------------------------+---------+-----------------------+------+-----------+-----------+
|           NAME            |  STATE  |         IPV4          | IPV6 |   TYPE    | SNAPSHOTS |
+---------------------------+---------+-----------------------+------+-----------+-----------+
| ansible                   | RUNNING | 10.0.169.42 (eth0)    |      | CONTAINER | 3         |
+---------------------------+---------+-----------------------+------+-----------+-----------+
...

I’ll go confirm everything is working, but at least the service is up. Thank you!