Error: Failed importing backup from ( lxc import)

Dear

I have to restore a container on a new server. The older server crashed.

I had lxc export of all my containers

First restoired the lxc config by this command

cat lxd.config.backup | lxd init --preseed

Then restoring the image but it fails when error Error:
Failed importing backup: Failed checking snapshots: Snapshot count in backup config and storage device are different: Backup snapshots mismatch

The lxc containers I am restoring are taken via an lxc export.

Not sure my container had snapshot , probably it had based on the error.

my goal is just to restore my container. ( I don 't care about any snapshots)

The version from LXD is newer then the server before not sure its related.

Hope you can help me.

root@vmi1400827:~# lxc import Proxy-backup-08-11-2023.tar.xz --debug
DEBUG  [2023-08-12T12:35:50+02:00] Connecting to a local LXD over a Unix socket 
DEBUG  [2023-08-12T12:35:50+02:00] Sending request to LXD                        etag= method=GET url="http://unix.socket/1.0"
DEBUG  [2023-08-12T12:35:50+02:00] Got response struct from LXD                 
DEBUG  [2023-08-12T12:35:50+02:00] 
	{
		"config": {},
		"api_extensions": [
			"storage_zfs_remove_snapshots",
			"container_host_shutdown_timeout",
			"container_stop_priority",
			"container_syscall_filtering",
			"auth_pki",
			"container_last_used_at",
			"etag",
			"patch",
			"usb_devices",
			"https_allowed_credentials",
			"image_compression_algorithm",
			"directory_manipulation",
			"container_cpu_time",
			"storage_zfs_use_refquota",
			"storage_lvm_mount_options",
			"network",
			"profile_usedby",
			"container_push",
			"container_exec_recording",
			"certificate_update",
			"container_exec_signal_handling",
			"gpu_devices",
			"container_image_properties",
			"migration_progress",
			"id_map",
			"network_firewall_filtering",
			"network_routes",
			"storage",
			"file_delete",
			"file_append",
			"network_dhcp_expiry",
			"storage_lvm_vg_rename",
			"storage_lvm_thinpool_rename",
			"network_vlan",
			"image_create_aliases",
			"container_stateless_copy",
			"container_only_migration",
			"storage_zfs_clone_copy",
			"unix_device_rename",
			"storage_lvm_use_thinpool",
			"storage_rsync_bwlimit",
			"network_vxlan_interface",
			"storage_btrfs_mount_options",
			"entity_description",
			"image_force_refresh",
			"storage_lvm_lv_resizing",
			"id_map_base",
			"file_symlinks",
			"container_push_target",
			"network_vlan_physical",
			"storage_images_delete",
			"container_edit_metadata",
			"container_snapshot_stateful_migration",
			"storage_driver_ceph",
			"storage_ceph_user_name",
			"resource_limits",
			"storage_volatile_initial_source",
			"storage_ceph_force_osd_reuse",
			"storage_block_filesystem_btrfs",
			"resources",
			"kernel_limits",
			"storage_api_volume_rename",
			"macaroon_authentication",
			"network_sriov",
			"console",
			"restrict_devlxd",
			"migration_pre_copy",
			"infiniband",
			"maas_network",
			"devlxd_events",
			"proxy",
			"network_dhcp_gateway",
			"file_get_symlink",
			"network_leases",
			"unix_device_hotplug",
			"storage_api_local_volume_handling",
			"operation_description",
			"clustering",
			"event_lifecycle",
			"storage_api_remote_volume_handling",
			"nvidia_runtime",
			"container_mount_propagation",
			"container_backup",
			"devlxd_images",
			"container_local_cross_pool_handling",
			"proxy_unix",
			"proxy_udp",
			"clustering_join",
			"proxy_tcp_udp_multi_port_handling",
			"network_state",
			"proxy_unix_dac_properties",
			"container_protection_delete",
			"unix_priv_drop",
			"pprof_http",
			"proxy_haproxy_protocol",
			"network_hwaddr",
			"proxy_nat",
			"network_nat_order",
			"container_full",
			"candid_authentication",
			"backup_compression",
			"candid_config",
			"nvidia_runtime_config",
			"storage_api_volume_snapshots",
			"storage_unmapped",
			"projects",
			"candid_config_key",
			"network_vxlan_ttl",
			"container_incremental_copy",
			"usb_optional_vendorid",
			"snapshot_scheduling",
			"snapshot_schedule_aliases",
			"container_copy_project",
			"clustering_server_address",
			"clustering_image_replication",
			"container_protection_shift",
			"snapshot_expiry",
			"container_backup_override_pool",
			"snapshot_expiry_creation",
			"network_leases_location",
			"resources_cpu_socket",
			"resources_gpu",
			"resources_numa",
			"kernel_features",
			"id_map_current",
			"event_location",
			"storage_api_remote_volume_snapshots",
			"network_nat_address",
			"container_nic_routes",
			"rbac",
			"cluster_internal_copy",
			"seccomp_notify",
			"lxc_features",
			"container_nic_ipvlan",
			"network_vlan_sriov",
			"storage_cephfs",
			"container_nic_ipfilter",
			"resources_v2",
			"container_exec_user_group_cwd",
			"container_syscall_intercept",
			"container_disk_shift",
			"storage_shifted",
			"resources_infiniband",
			"daemon_storage",
			"instances",
			"image_types",
			"resources_disk_sata",
			"clustering_roles",
			"images_expiry",
			"resources_network_firmware",
			"backup_compression_algorithm",
			"ceph_data_pool_name",
			"container_syscall_intercept_mount",
			"compression_squashfs",
			"container_raw_mount",
			"container_nic_routed",
			"container_syscall_intercept_mount_fuse",
			"container_disk_ceph",
			"virtual-machines",
			"image_profiles",
			"clustering_architecture",
			"resources_disk_id",
			"storage_lvm_stripes",
			"vm_boot_priority",
			"unix_hotplug_devices",
			"api_filtering",
			"instance_nic_network",
			"clustering_sizing",
			"firewall_driver",
			"projects_limits",
			"container_syscall_intercept_hugetlbfs",
			"limits_hugepages",
			"container_nic_routed_gateway",
			"projects_restrictions",
			"custom_volume_snapshot_expiry",
			"volume_snapshot_scheduling",
			"trust_ca_certificates",
			"snapshot_disk_usage",
			"clustering_edit_roles",
			"container_nic_routed_host_address",
			"container_nic_ipvlan_gateway",
			"resources_usb_pci",
			"resources_cpu_threads_numa",
			"resources_cpu_core_die",
			"api_os",
			"container_nic_routed_host_table",
			"container_nic_ipvlan_host_table",
			"container_nic_ipvlan_mode",
			"resources_system",
			"images_push_relay",
			"network_dns_search",
			"container_nic_routed_limits",
			"instance_nic_bridged_vlan",
			"network_state_bond_bridge",
			"usedby_consistency",
			"custom_block_volumes",
			"clustering_failure_domains",
			"resources_gpu_mdev",
			"console_vga_type",
			"projects_limits_disk",
			"network_type_macvlan",
			"network_type_sriov",
			"container_syscall_intercept_bpf_devices",
			"network_type_ovn",
			"projects_networks",
			"projects_networks_restricted_uplinks",
			"custom_volume_backup",
			"backup_override_name",
			"storage_rsync_compression",
			"network_type_physical",
			"network_ovn_external_subnets",
			"network_ovn_nat",
			"network_ovn_external_routes_remove",
			"tpm_device_type",
			"storage_zfs_clone_copy_rebase",
			"gpu_mdev",
			"resources_pci_iommu",
			"resources_network_usb",
			"resources_disk_address",
			"network_physical_ovn_ingress_mode",
			"network_ovn_dhcp",
			"network_physical_routes_anycast",
			"projects_limits_instances",
			"network_state_vlan",
			"instance_nic_bridged_port_isolation",
			"instance_bulk_state_change",
			"network_gvrp",
			"instance_pool_move",
			"gpu_sriov",
			"pci_device_type",
			"storage_volume_state",
			"network_acl",
			"migration_stateful",
			"disk_state_quota",
			"storage_ceph_features",
			"projects_compression",
			"projects_images_remote_cache_expiry",
			"certificate_project",
			"network_ovn_acl",
			"projects_images_auto_update",
			"projects_restricted_cluster_target",
			"images_default_architecture",
			"network_ovn_acl_defaults",
			"gpu_mig",
			"project_usage",
			"network_bridge_acl",
			"warnings",
			"projects_restricted_backups_and_snapshots",
			"clustering_join_token",
			"clustering_description",
			"server_trusted_proxy",
			"clustering_update_cert",
			"storage_api_project",
			"server_instance_driver_operational",
			"server_supported_storage_drivers",
			"event_lifecycle_requestor_address",
			"resources_gpu_usb",
			"clustering_evacuation",
			"network_ovn_nat_address",
			"network_bgp",
			"network_forward",
			"custom_volume_refresh",
			"network_counters_errors_dropped",
			"metrics",
			"image_source_project",
			"clustering_config",
			"network_peer",
			"linux_sysctl",
			"network_dns",
			"ovn_nic_acceleration",
			"certificate_self_renewal",
			"instance_project_move",
			"storage_volume_project_move",
			"cloud_init",
			"network_dns_nat",
			"database_leader",
			"instance_all_projects",
			"clustering_groups",
			"ceph_rbd_du",
			"instance_get_full",
			"qemu_metrics",
			"gpu_mig_uuid",
			"event_project",
			"clustering_evacuation_live",
			"instance_allow_inconsistent_copy",
			"network_state_ovn",
			"storage_volume_api_filtering",
			"image_restrictions",
			"storage_zfs_export",
			"network_dns_records",
			"storage_zfs_reserve_space",
			"network_acl_log",
			"storage_zfs_blocksize",
			"metrics_cpu_seconds",
			"instance_snapshot_never",
			"certificate_token",
			"instance_nic_routed_neighbor_probe",
			"event_hub",
			"agent_nic_config",
			"projects_restricted_intercept",
			"metrics_authentication",
			"images_target_project",
			"cluster_migration_inconsistent_copy",
			"cluster_ovn_chassis",
			"container_syscall_intercept_sched_setscheduler",
			"storage_lvm_thinpool_metadata_size",
			"storage_volume_state_total",
			"instance_file_head",
			"instances_nic_host_name",
			"image_copy_profile",
			"container_syscall_intercept_sysinfo",
			"clustering_evacuation_mode",
			"resources_pci_vpd",
			"qemu_raw_conf",
			"storage_cephfs_fscache",
			"network_load_balancer",
			"vsock_api",
			"instance_ready_state",
			"network_bgp_holdtime",
			"storage_volumes_all_projects",
			"metrics_memory_oom_total",
			"storage_buckets",
			"storage_buckets_create_credentials",
			"metrics_cpu_effective_total",
			"projects_networks_restricted_access",
			"storage_buckets_local",
			"loki",
			"acme",
			"internal_metrics",
			"cluster_join_token_expiry",
			"remote_token_expiry",
			"init_preseed",
			"storage_volumes_created_at",
			"cpu_hotplug",
			"projects_networks_zones",
			"network_txqueuelen",
			"cluster_member_state",
			"instances_placement_scriptlet",
			"storage_pool_source_wipe",
			"zfs_block_mode",
			"instance_generation_id",
			"disk_io_cache",
			"amd_sev",
			"storage_pool_loop_resize",
			"migration_vm_live",
			"ovn_nic_nesting",
			"oidc",
			"network_ovn_l3only",
			"ovn_nic_acceleration_vdpa",
			"cluster_healing",
			"instances_state_total",
			"auth_user",
			"security_csm",
			"instances_rebuild",
			"numa_cpu_placement",
			"custom_volume_iso",
			"network_allocations"
		],
		"api_status": "stable",
		"api_version": "1.0",
		"auth": "trusted",
		"public": false,
		"auth_methods": [
			"tls"
		],
		"auth_user_name": "root",
		"auth_user_method": "unix",
		"environment": {
			"addresses": [],
			"architectures": [
				"x86_64",
				"i686"
			],
			"certificate": "-----BEGIN CERTIFICATE-----\nMIICJTCCAaugAwIBAgIQAP4gPMCN4/9bVaE6T4Ud7jAKBggqhkjOPQQDAzA6MQww\nCgYDVQQKEwNMWEQxKjAoBgNVBAMMIXJvb3RAdm1pMTQwMDgyNy5jb250YWJvc2Vy\ndmVyLm5ldDAeFw0yMzA4MTIxMDI0MjhaFw0zMzA4MDkxMDI0MjhaMDoxDDAKBgNV\nBAoTA0xYRDEqMCgGA1UEAwwhcm9vdEB2bWkxNDAwODI3LmNvbnRhYm9zZXJ2ZXIu\nbmV0MHYwEAYHKoZIzj0CAQYFK4EEACIDYgAE5VgK3THvSBPRBAsLcXTJlExbkbcI\nzyQpC4mOV/weZYNdoZQSU1JIT1XLqLzcp7MPGPYVR01MmGIVVIw4gtmCbOdYCu+2\nDbmBxuX+k6cbxfhbw0dxo2szLG7qxE4FW/Myo3YwdDAOBgNVHQ8BAf8EBAMCBaAw\nEwYDVR0lBAwwCgYIKwYBBQUHAwEwDAYDVR0TAQH/BAIwADA/BgNVHREEODA2ghx2\nbWkxNDAwODI3LmNvbnRhYm9zZXJ2ZXIubmV0hwR/AAABhxAAAAAAAAAAAAAAAAAA\nAAABMAoGCCqGSM49BAMDA2gAMGUCMQC6nOoKzmY+6ieisXM6i72zJMcZFJmHC5ft\nZ7DphrM1KfAP01Bkb6nLKvm2LtekPv8CMEB7txQelKwiGnkOo8qrg8yE/4nOl174\nHpVCvstGtlUtOvCmrlssaFztSYCpNXy7YA==\n-----END CERTIFICATE-----\n",
			"certificate_fingerprint": "c8972714e9b23a59ccc714ec3fd12da7eac0b557248d2147de1c16beffc743ce",
			"driver": "lxc",
			"driver_version": "5.0.2",
			"firewall": "nftables",
			"kernel": "Linux",
			"kernel_architecture": "x86_64",
			"kernel_features": {
				"idmapped_mounts": "false",
				"netnsid_getifaddrs": "true",
				"seccomp_listener": "true",
				"seccomp_listener_continue": "true",
				"shiftfs": "false",
				"uevent_injection": "true",
				"unpriv_fscaps": "true"
			},
			"kernel_version": "5.4.0-105-generic",
			"lxc_features": {
				"cgroup2": "true",
				"core_scheduling": "true",
				"devpts_fd": "true",
				"idmapped_mounts_v2": "true",
				"mount_injection_file": "true",
				"network_gateway_device_route": "true",
				"network_ipvlan": "true",
				"network_l2proxy": "true",
				"network_phys_macvlan_mtu": "true",
				"network_veth_router": "true",
				"pidfd": "true",
				"seccomp_allow_deny_syntax": "true",
				"seccomp_notify": "true",
				"seccomp_proxy_send_notify_fd": "true"
			},
			"os_name": "Ubuntu",
			"os_version": "20.04",
			"project": "default",
			"server": "lxd",
			"server_clustered": false,
			"server_event_mode": "full-mesh",
			"server_name": "vmi1400827.contaboserver.net",
			"server_pid": 3089,
			"server_version": "5.16",
			"storage": "zfs",
			"storage_version": "0.8.3-1ubuntu12.13",
			"storage_supported_drivers": [
				{
					"Name": "cephobject",
					"Version": "17.2.5",
					"Remote": true
				},
				{
					"Name": "dir",
					"Version": "1",
					"Remote": false
				},
				{
					"Name": "lvm",
					"Version": "2.03.11(2) (2021-01-08) / 1.02.175 (2021-01-08) / 4.41.0",
					"Remote": false
				},
				{
					"Name": "zfs",
					"Version": "0.8.3-1ubuntu12.13",
					"Remote": false
				},
				{
					"Name": "btrfs",
					"Version": "5.16.2",
					"Remote": false
				},
				{
					"Name": "ceph",
					"Version": "17.2.5",
					"Remote": true
				},
				{
					"Name": "cephfs",
					"Version": "17.2.5",
					"Remote": true
				}
			]
		}
	} 
DEBUG  [2023-08-12T12:35:50+02:00] Connected to the websocket: ws://unix.socket/1.0/events 
DEBUG  [2023-08-12T12:35:50+02:00] Sending request to LXD                        etag= method=POST url="http://unix.socket/1.0/instances"
Importing instance: 100% (915.14MB/s)DEBUG  [2023-08-12T12:35:50+02:00] Got operation from LXD                       
DEBUG  [2023-08-12T12:35:50+02:00] 
	{
		"id": "b9d1db03-d9cd-4411-959b-631aecf9a088",
		"class": "task",
		"description": "Restoring backup",
		"created_at": "2023-08-12T12:35:50.655564138+02:00",
		"updated_at": "2023-08-12T12:35:50.655564138+02:00",
		"status": "Running",
		"status_code": 103,
		"resources": {
			"instances": [
				"/1.0/instances/Proxy"
			]
		},
		"metadata": null,
		"may_cancel": false,
		"err": "",
		"location": "none"
	} 
DEBUG  [2023-08-12T12:35:50+02:00] Sending request to LXD                        etag= method=GET url="http://unix.socket/1.0/operations/b9d1db03-d9cd-4411-959b-631aecf9a088"
DEBUG  [2023-08-12T12:35:50+02:00] Got response struct from LXD                 
DEBUG  [2023-08-12T12:35:50+02:00] 
	{
		"id": "b9d1db03-d9cd-4411-959b-631aecf9a088",
		"class": "task",
		"description": "Restoring backup",
		"created_at": "2023-08-12T12:35:50.655564138+02:00",
		"updated_at": "2023-08-12T12:35:50.655564138+02:00",
		"status": "Running",
		"status_code": 103,
		"resources": {
			"instances": [
				"/1.0/instances/Proxy"
			]
		},
		"metadata": null,
		"may_cancel": false,
		"err": "",
		"location": "none"
	} 
Error: Failed importing backup: Failed checking snapshots: Snapshot count in backup config and storage device are different: Backup snapshots mismatch

When unpack the exported container I have this file

# cd backup
root@vmi1400827:~/backup# ls
container.bin  index.yaml
root@vmi1400827:~/backup# cat index.yaml 
name: Proxy
backend: zfs
pool: default
optimized: true
optimized_header: false
type: container
config:
  container:
    architecture: x86_64
    config:
      image.architecture: amd64
      image.description: Ubuntu focal amd64 (20201211_08:35)
      image.os: Ubuntu
      image.release: focal
      image.serial: "20201211_08:35"
      image.type: squashfs
      image.variant: default
      raw.apparmor: mount fstype=nfs,
      volatile.base_image: 5557e843214fd5ff4b159d23628ced7e3cf6655b67773bb2f4bf15c61d584939
      volatile.eth0.host_name: veth5845fabc
      volatile.eth0.hwaddr: 00:16:3e:ed:36:b0
      volatile.idmap.base: "0"
      volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
      volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
      volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
      volatile.last_state.power: RUNNING
      volatile.uuid: 12df5e52-7062-4c29-af9e-7a145daa72c3
    devices:
      myport80:
        connect: tcp:10.196.47.103:80
        listen: tcp:0.0.0.0:80
        type: proxy
      myport443:
        connect: tcp:10.196.47.103:443
        listen: tcp:0.0.0.0:443
        type: proxy
      myport8080:
        connect: tcp:10.196.47.103:8080
        listen: tcp:0.0.0.0:8080
        type: proxy
    ephemeral: false
    profiles:
    - default
    stateful: false
    description: ""
    created_at: 2020-12-12T13:34:08.762546561+01:00
    expanded_config:
      image.architecture: amd64
      image.description: Ubuntu focal amd64 (20201211_08:35)
      image.os: Ubuntu
      image.release: focal
      image.serial: "20201211_08:35"
      image.type: squashfs
      image.variant: default
      raw.apparmor: mount fstype=nfs,
      volatile.base_image: 5557e843214fd5ff4b159d23628ced7e3cf6655b67773bb2f4bf15c61d584939
      volatile.eth0.host_name: veth5845fabc
      volatile.eth0.hwaddr: 00:16:3e:ed:36:b0
      volatile.idmap.base: "0"
      volatile.idmap.current: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
      volatile.idmap.next: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
      volatile.last_state.idmap: '[{"Isuid":true,"Isgid":false,"Hostid":1000000,"Nsid":0,"Maprange":1000000000},{"Isuid":false,"Isgid":true,"Hostid":1000000,"Nsid":0,"Maprange":1000000000}]'
      volatile.last_state.power: RUNNING
      volatile.uuid: 12df5e52-7062-4c29-af9e-7a145daa72c3
    expanded_devices:
      eth0:
        name: eth0
        network: lxdbr0
        type: nic
      myport80:
        connect: tcp:10.196.47.103:80
        listen: tcp:0.0.0.0:80
        type: proxy
      myport443:
        connect: tcp:10.196.47.103:443
        listen: tcp:0.0.0.0:443
        type: proxy
      myport8080:
        connect: tcp:10.196.47.103:8080
        listen: tcp:0.0.0.0:8080
        type: proxy
      root:
        path: /
        pool: default
        type: disk
    name: Proxy
    status: Running
    status_code: 103
    last_used_at: 2022-09-20T18:24:04.739331211Z
    location: none
    type: container
    project: default
  snapshots: []  
  pool:
    config:
      size: 30GB
      source: /var/snap/lxd/common/lxd/disks/default.img
      zfs.pool_name: default
    description: ""
    name: default
    driver: zfs
    used_by: []
    status: Created
    locations:
    - none
  profiles:
  - config: {}
    description: Default LXD profile
    devices:
      eth0:
        name: eth0
        network: lxdbr0
        type: nic
      root:
        path: /
        pool: default
        type: disk
    name: default
    used_by: []
  volume:
    config: {}
    description: ""
    name: Proxy
    type: container
    used_by: []
    location: none
    content_type: filesystem
    project: default
    created_at: 0001-01-01T00:00:00Z

Hi,
I have import and export a container which has a snapshot so what are the versions of previous lxd system and the latest one? And you can make the previous lxd system and the latest one to the same version then try again cos I have tried without problem on my lxd 5.15 system.
Regards.

Dear

Thx for your reply i need to test. Lxc export is taken from lxd 5.11 new server is running 5.16. As the old server luckly recovered :slight_smile: .

I was using 5.11 as I was expirencing the bug/issue

I will test and let you know.

However it would be good to know how we can take LXC backups/export unrelated to any lxd version.
I was thinking of making images to avoid this problem, not sure its the best way.

Thx

Tim

Dear

I have just tested with the same Lxd version (5.16):

If we take an export container X with 1 snapshot with the command below the import does not work on on a new storage pool

/snap/bin/lxc export containerX --optimized-storage --instance-only

However it works when:

  • Removing the snapshot before the export

  • Removing “-instance-only” from the export command

So the export command with option “–instance-only” can not be used on new storage pools. Maybe good to update documenation on this