Failed to migrate using lxd-to-incus

[lwh504530121]

root@zfi-PowerEdge-T640:/var/lib/incus/database# journalctl -u incus -n 30 | cat
11月 21 20:05:48 zfi-PowerEdge-T640 incusd[39165]: Error: Failed to initialize global database: Failed to prepare statements: “\nDELETE FROM projects WHERE name = ?\n”: no such table: main.auth_groups_permissions
11月 21 20:05:49 zfi-PowerEdge-T640 systemd[1]: incus.service: Main process exited, code=exited, status=1/FAILURE
11月 21 20:15:48 zfi-PowerEdge-T640 systemd[1]: incus.service: start-post operation timed out. Terminating.
11月 21 20:15:48 zfi-PowerEdge-T640 incus[39166]: Error: Daemon still not running after 600s timeout (Get “http://unix.socket/1.0”: EOF)
11月 21 20:15:48 zfi-PowerEdge-T640 systemd[1]: incus.service: Control process exited, code=exited, status=1/FAILURE
11月 21 20:15:48 zfi-PowerEdge-T640 systemd[1]: incus.service: Failed with result ‘exit-code’.
11月 21 20:15:48 zfi-PowerEdge-T640 systemd[1]: Failed to start incus.service - Incus - Main daemon.
11月 21 20:15:48 zfi-PowerEdge-T640 systemd[1]: incus.service: Scheduled restart job, restart counter is at 1.
11月 21 20:15:48 zfi-PowerEdge-T640 systemd[1]: Starting incus.service - Incus - Main daemon…
11月 21 20:15:49 zfi-PowerEdge-T640 incusd[89074]: time=“2025-11-21T20:15:49+08:00” level=error msg=“Failed to start the daemon” err=“Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: table certificates already exists”
11月 21 20:15:49 zfi-PowerEdge-T640 incusd[89074]: Error: Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: table certificates already exists
11月 21 20:15:49 zfi-PowerEdge-T640 systemd[1]: incus.service: Main process exited, code=exited, status=1/FAILURE
11月 21 20:25:48 zfi-PowerEdge-T640 incus[89075]: Error: Daemon still not running after 600s timeout (Get “http://unix.socket/1.0”: EOF)
11月 21 20:25:48 zfi-PowerEdge-T640 systemd[1]: incus.service: Control process exited, code=exited, status=1/FAILURE
11月 21 20:25:48 zfi-PowerEdge-T640 systemd[1]: incus.service: Failed with result ‘exit-code’.
11月 21 20:25:48 zfi-PowerEdge-T640 systemd[1]: Failed to start incus.service - Incus - Main daemon.
11月 21 20:25:49 zfi-PowerEdge-T640 systemd[1]: incus.service: Scheduled restart job, restart counter is at 2.
11月 21 20:25:49 zfi-PowerEdge-T640 systemd[1]: Starting incus.service - Incus - Main daemon…
11月 21 20:25:49 zfi-PowerEdge-T640 incusd[138260]: time=“2025-11-21T20:25:49+08:00” level=error msg=“Failed to start the daemon” err=“Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: table certificates already exists”
11月 21 20:25:49 zfi-PowerEdge-T640 incusd[138260]: Error: Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: table certificates already exists
11月 21 20:25:49 zfi-PowerEdge-T640 systemd[1]: incus.service: Main process exited, code=exited, status=1/FAILURE
11月 21 20:35:49 zfi-PowerEdge-T640 incus[138261]: Error: Daemon still not running after 600s timeout (Get “http://unix.socket/1.0”: EOF)
11月 21 20:35:49 zfi-PowerEdge-T640 systemd[1]: incus.service: Control process exited, code=exited, status=1/FAILURE
11月 21 20:35:49 zfi-PowerEdge-T640 systemd[1]: incus.service: Failed with result ‘exit-code’.
11月 21 20:35:49 zfi-PowerEdge-T640 systemd[1]: Failed to start incus.service - Incus - Main daemon.
11月 21 20:35:49 zfi-PowerEdge-T640 systemd[1]: incus.service: Scheduled restart job, restart counter is at 3.
11月 21 20:35:49 zfi-PowerEdge-T640 systemd[1]: Starting incus.service - Incus - Main daemon…
11月 21 20:35:50 zfi-PowerEdge-T640 incusd[187328]: time=“2025-11-21T20:35:50+08:00” level=error msg=“Failed to start the daemon” err=“Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: table certificates already exists”
11月 21 20:35:50 zfi-PowerEdge-T640 incusd[187328]: Error: Failed to initialize global database: failed to ensure schema: failed to execute queries from /var/lib/incus/database/patch.global.sql: table certificates already exists
11月 21 20:35:50 zfi-PowerEdge-T640 systemd[1]: incus.service: Main process exited, code=exited, status=1/FAILURE

patch.global.sql:

UPDATE profiles SET description=‘Default Incus profile’ WHERE description=‘Default LXD profile’;
UPDATE projects SET description=‘Default Incus project’ WHERE description=‘Default LXD project’;
CREATE TABLE certificates (
id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
fingerprint TEXT NOT NULL,
type INTEGER NOT NULL,
name TEXT NOT NULL,
certificate TEXT NOT NULL,
restricted INTEGER NOT NULL DEFAULT 0,
UNIQUE (fingerprint)
);
CREATE TABLE “certificates_projects” (
certificate_id INTEGER NOT NULL,
project_id INTEGER NOT NULL,
FOREIGN KEY (certificate_id) REFERENCES certificates (id) ON DELETE CASCADE,
FOREIGN KEY (project_id) REFERENCES “projects” (id) ON DELETE CASCADE,
UNIQUE (certificate_id, project_id)
);
DELETE FROM schema WHERE version < 73;
UPDATE schema SET version=69 WHERE version=73;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 1, name, json_extract(metadata, “$.cert”), 1 FROM identities WHERE type=1;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 1, name, json_extract(metadata, “$.cert”), 0 FROM identities WHERE type=2;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 2, name, json_extract(metadata, “$.cert”), 0 FROM identities WHERE type=3;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 3, name, json_extract(metadata, “$.cert”), 1 FROM identities WHERE type=4;
INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 3, name, json_extract(metadata, “$.cert”), 0 FROM identities WHERE type=6;
INSERT INTO certificates_projects (certificate_id, project_id) SELECT identity_id, project_id FROM identities_projects;
DROP TRIGGER IF EXISTS on_auth_group_delete;
DROP TRIGGER IF EXISTS on_cluster_group_delete;
DROP TRIGGER IF EXISTS on_identity_delete;
DROP TRIGGER IF EXISTS on_identity_provider_group_delete;
DROP TRIGGER IF EXISTS on_image_alias_delete;
DROP TRIGGER IF EXISTS on_image_delete;
DROP TRIGGER IF EXISTS on_instance_backup_delete;
DROP TRIGGER IF EXISTS on_instance_delete;
DROP TRIGGER IF EXISTS on_instance_snaphot_delete;
DROP TRIGGER IF EXISTS on_network_acl_delete;
DROP TRIGGER IF EXISTS on_network_delete;
DROP TRIGGER IF EXISTS on_network_zone_delete;
DROP TRIGGER IF EXISTS on_node_delete;
DROP TRIGGER IF EXISTS on_operation_delete;
DROP TRIGGER IF EXISTS on_profile_delete;
DROP TRIGGER IF EXISTS on_project_delete;
DROP TRIGGER IF EXISTS on_storage_bucket_delete;
DROP TRIGGER IF EXISTS on_storage_pool_delete;
DROP TRIGGER IF EXISTS on_storage_volume_backup_delete;
DROP TRIGGER IF EXISTS on_storage_volume_delete;
DROP TRIGGER IF EXISTS on_storage_volume_snapshot_delete;
DROP TRIGGER IF EXISTS on_warning_delete;
DROP TRIGGER IF EXISTS on_instance_snaphot_delete;
DROP TABLE IF EXISTS identities_projects;
DROP TABLE IF EXISTS auth_groups_permissions;
DROP TABLE IF EXISTS auth_groups_identity_provider_groups;
DROP TABLE IF EXISTS identities_auth_groups;
DROP TABLE IF EXISTS identity_provider_groups;
DROP TABLE IF EXISTS identities;
DROP TABLE IF EXISTS auth_groups;

incus version : 6.0.0

How should I proceed to recover the service?

thanks~~

[lwh504530121]

lxd-to-incus log:

root@zfi-PowerEdge-T640:/var/log# cat lxd-to-incus.189973.log
Source server: snap package
Target server: systemd
Source server paths: &{daemon:/var/snap/lxd/common/lxd logs:/var/snap/lxd/common/lxd/logs cache:/var/snap/lxd/common/lxd/cache}
Target server paths: &{daemon:/var/lib/incus logs:/var/log/incus cache:/var/cache/incus}
Rewrite SQL statements:

• UPDATE profiles SET description=‘Default Incus profile’ WHERE description=‘Default LXD profile’;
• UPDATE projects SET description=‘Default Incus project’ WHERE description=‘Default LXD project’;
• CREATE TABLE certificates (
  id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
  fingerprint TEXT NOT NULL,
  type INTEGER NOT NULL,
  name TEXT NOT NULL,
  certificate TEXT NOT NULL,
  restricted INTEGER NOT NULL DEFAULT 0,
  UNIQUE (fingerprint)
  );
  CREATE TABLE “certificates_projects” (
  certificate_id INTEGER NOT NULL,
  project_id INTEGER NOT NULL,
  FOREIGN KEY (certificate_id) REFERENCES certificates (id) ON DELETE CASCADE,
  FOREIGN KEY (project_id) REFERENCES “projects” (id) ON DELETE CASCADE,
  UNIQUE (certificate_id, project_id)
  );
• DELETE FROM schema WHERE version < 73;
  UPDATE schema SET version=69 WHERE version=73;
• INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 1, name, json_extract(metadata, “$.cert”), 1 FROM identities WHERE type=1;
  INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 1, name, json_extract(metadata, “$.cert”), 0 FROM identities WHERE type=2;
  INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 2, name, json_extract(metadata, “$.cert”), 0 FROM identities WHERE type=3;
  INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 3, name, json_extract(metadata, “$.cert”), 1 FROM identities WHERE type=4;
  INSERT INTO certificates (id, fingerprint, type, name, certificate, restricted) SELECT id, identifier, 3, name, json_extract(metadata, “$.cert”), 0 FROM identities WHERE type=6;
  INSERT INTO certificates_projects (certificate_id, project_id) SELECT identity_id, project_id FROM identities_projects;
• DROP TRIGGER on_auth_group_delete;
  DROP TRIGGER on_cluster_group_delete;
  DROP TRIGGER on_identity_delete;
  DROP TRIGGER on_identity_provider_group_delete;
  DROP TRIGGER on_image_alias_delete;
  DROP TRIGGER on_image_delete;
  DROP TRIGGER on_instance_backup_delete;
  DROP TRIGGER on_instance_delete;
  DROP TRIGGER on_instance_snaphot_delete;
  DROP TRIGGER on_network_acl_delete;
  DROP TRIGGER on_network_delete;
  DROP TRIGGER on_network_zone_delete;
  DROP TRIGGER on_node_delete;
  DROP TRIGGER on_operation_delete;
  DROP TRIGGER on_profile_delete;
  DROP TRIGGER on_project_delete;
  DROP TRIGGER on_storage_bucket_delete;
  DROP TRIGGER on_storage_pool_delete;
  DROP TRIGGER on_storage_volume_backup_delete;
  DROP TRIGGER on_storage_volume_delete;
  DROP TRIGGER on_storage_volume_snapshot_delete;
  DROP TRIGGER on_warning_delete;
  DROP TABLE identities_projects;
  DROP TABLE auth_groups_permissions;
  DROP TABLE auth_groups_identity_provider_groups;
  DROP TABLE identities_auth_groups;
  DROP TABLE identity_provider_groups;
  DROP TABLE identities;
  DROP TABLE auth_groups;
  Rewrite commands:
  Migration started
  Stopping the source server
  Stopping the target server
  Unmounting “/var/lib/incus/devlxd”
  Unmounting “/var/lib/incus/shmounts”
  Wiping the target server
  Migrating the data
  Moving data over
  Migrating database files
  Writing the database patch
  Cleaning up target paths
  Cleaning up path “/var/lib/incus/backups”
  Cleaning up path “/var/lib/incus/images”
  Cleaning up path “/var/lib/incus/devices”
  Cleaning up path “/var/lib/incus/devlxd”
  Cleaning up path “/var/lib/incus/security”
  Cleaning up path “/var/lib/incus/shmounts”
  Rewrite symlinks:
• “/var/lib/incus/storage-pools/zfs-lxc-pool/containers/vision-1804” to “/var/lib/incus/containers/vision-1804”
• “/var/lib/incus/storage-pools/zfs-lxc-pool/containers/vision-1804-1” to “/var/lib/incus/containers/vision-1804-1”
• “/var/lib/incus/storage-pools/zfs-lxc-pool/containers/vision-1804” to “/var/lib/incus/containers/vision-1804”
• “/var/lib/incus/storage-pools/zfs-lxc-pool/containers/vision-1804” to “/var/lib/incus/containers/vision-1804”
• “/var/lib/incus/storage-pools/zfs-lxc-pool/containers/vision-1804” to “/var/lib/incus/containers/vision-1804”
  Rewrite symlinks:
  Rewrite symlinks:
  Rewrite symlinks:
  Starting the target server
  ERROR: Failed to run: systemctl start incus.service incus.socket: exit status 1 (Job for incus.service failed because the control process exited with error code.
  See “systemctl status incus.service” and “journalctl -xeu incus.service” for details.)

[lwh504530121]

After I tried deleting patch.global.sql and executed systemctl restart incus.service, the error still occurred.
11月 21 21:06:24 zfi-PowerEdge-T640 systemd[1]: Starting incus.service - Incus - Main daemon…
11月 21 21:06:25 zfi-PowerEdge-T640 incusd[337888]: time=“2025-11-21T21:06:25+08:00” level=error msg=“Failed to start the daemon” err="Failed to initialize global database: Failed to prepare statements: "\nDELETE FROM instances WHERE project_id = (SELECT projects.id >
11月 21 21:06:25 zfi-PowerEdge-T640 incusd[337888]: Error: Failed to initialize global database: Failed to prepare statements: “\nDELETE FROM instances WHERE project_id = (SELECT projects.id FROM projects WHERE projects.name = ?) AND name = ?\n”: no such table: main.au>
11月 21 21:06:25 zfi-PowerEdge-T640 systemd[1]: incus.service: Main process exited, code=exited, status=1/FAILURE

[stgraber (Stéphane Graber)]

What was the LXD version used here?

Your output is getting truncated, can you maybe include the content of /var/log/incus/incusd.log

[lwh504530121]

lxd version is 6.5

logfile.log.txt (2.8 MB)

incus.log.txt (42.0 KB)

[lwh504530121]

Initially, my LXD version was 6.5. However, I found that this version was too high to use with the lxd-to-incus tool, so I downgraded it to 5.21. After the downgrade, I tested several commands such as lxc list and lxc start, which worked normally. Subsequently, I executed the lxd-to-incus migration, but the process failed. Now, LXD has been upgraded back to version 6.5 automatically, and all my containers are missing.

[stgraber (Stéphane Graber)]

Right, so that’s the problem.

As it turns out, there is a reason why lxd-to-incus will refuse to work on anything more recent than 5.21…

LXD and Incus do not support downgrading, it’s made pretty clear in both of their documentation, so downgrading to 5.21 just to bypass the validation check in lxd-to-incus will (and clearly has) only result in failure and potential data loss.

Your best bet at this point is to try to move the data from /var/lib/incus back to /var/snap/lxd/common/lxd, restore the database using the pre-migration backup we make and get LXD working.

Then once you do, use lxc export --export-version=1 for each of the instances you have, save those exports, then wipe LXD, do a clean install of Incus and use incus import to load the instances back in.

[lwh504530121]

@stgraber

降级到 5.21 只是为了绕过验证检查lxd-to-incus （而且显然已经发生过），只会导致失败和潜在的数据丢失。

I have already tried moving back from /var/lib/incus to /var/snap/lxd/common/lxd, but now I’m encountering an error:

bash

root@zfi-PowerEdge-T640:/var/log/lxd# lxc start ufi-vision-1804
Error: Unable to resolve container rootfs: lstat /var/lib/incus: no such file or directory
Try `lxc info --show-log ufi-vision-1804` for more info

bash

root@zfi-PowerEdge-T640:/var/log/lxd# lxc info --show-log ufi-vision-1804
Name: ufi-vision-1804
Status: STOPPED
Type: container
Architecture: x86_64
Created: 2024/04/01 09:38 CST
Last Used: 2025/06/10 10:46 CST

Log:

It seems that the configuration file is still pointing to the Incus path (/var/lib/incus). How can I resolve this issue?

@stgraber

降级到 5.21 只是为了绕过验证检查lxd-to-incus （而且显然已经发生过），只会导致失败和潜在的数据丢失。

I have already tried moving back from /var/lib/incus to /var/snap/lxd/common/lxd, but now I’m encountering an error:

bash

root@zfi-PowerEdge-T640:/var/log/lxd# lxc start ufi-vision-1804
Error: Unable to resolve container rootfs: lstat /var/lib/incus: no such file or directory
Try `lxc info --show-log ufi-vision-1804` for more info

bash

root@zfi-PowerEdge-T640:/var/log/lxd# lxc info --show-log ufi-vision-1804
Name: ufi-vision-1804
Status: STOPPED
Type: container
Architecture: x86_64
Created: 2024/04/01 09:38 CST
Last Used: 2025/06/10 10:46 CST

Log:

It seems that the configuration file is still pointing to the Incus path (/var/lib/incus). How can I resolve this issue?

[stgraber (Stéphane Graber)]

You’ll need to go fix the symlinks in /var/snap/lxd/common/lxd/containers and /var/snap/lxd/common/lxd/virtual-machines

[lwh504530121]

Thank you very much. Everything is working normally now, and it’s likely that the inability to execute commands was due to a damaged container. The reason I want to upgrade to Incus is that when I upgraded the host system from version 18.04 to 24.04 while using LXD 4.0, and also upgraded the containers accordingly, a problem emerged. One container is on version 24.04 and another on 22.04, and both of them run any command extremely slowly. For example, commands like apt update and htop take over ten minutes longer than usual to complete. However, containers on versions 20.04 and 18.04 are operating normally. I’ve also upgraded the LXD version to 5.21 and 6.0, but the issue persists. Could you please help me figure out why this is happening?