Images in project not shown when client does lxc image list <remote>:

Hello everyone,

On a lxd server a new project foo is created. Next, a user and image is added to said project:

lxc project create foo

# adding new user with restricted access, here only granted to project foo
lxc config trust add client.crt --restricted --projects foo

# confirm, if user has been added successfully
lxc config trust list
lxc config trust show <fingerprint-of-user>

# copying image from remote into project foo
lxc image copy  images:alpine/3.13/amd64 local: --alias my-image --project foo

# checking, if an alpine image with name my-image can be found in project foo
lxc image list --project foo
+----------+--------------+--------+------------------------------------+--------------+-----------+--------+------------------------------+
|  ALIAS   | FINGERPRINT  | PUBLIC |            DESCRIPTION             | ARCHITECTURE |   TYPE    |  SIZE  |         UPLOAD DATE          |
+----------+--------------+--------+------------------------------------+--------------+-----------+--------+------------------------------+
| my-image | e096be4e1d49 | no     | Alpine 3.13 amd64 (20210613_13:00) | x86_64       | CONTAINER | 2.45MB | Jun 14, 2021 at 3:13pm (UTC) |
+----------+--------------+--------+------------------------------------+--------------+-----------+--------+------------------------------

Now, on the client I list all images available on the aforementioned server/remote:

lxc image list my-lxd-server:

+-------+-------------+--------+-------------+--------------+------+------+-------------+
| ALIAS | FINGERPRINT | PUBLIC | DESCRIPTION | ARCHITECTURE | TYPE | SIZE | UPLOAD DATE |
+-------+-------------+--------+-------------+--------------+------+------+-------------+

Since this client has access to project foo, I expected to see my-image as well. However, none of the images assigned to a project are listed, except the those of the default project.

Did I made a mistake while configuring the lxd server?

Hmm, I would have expected that too, what does lxc project list my-lxd-server: show on that client?

Hello @stgraber,

the output of that command is as follows:

lxc project list my-lxd-server: 
+------+--------+----------+-----------------+----------+-------------+---------+
| NAME | IMAGES | PROFILES | STORAGE VOLUMES | NETWORKS | DESCRIPTION | USED BY |
+------+--------+----------+-----------------+----------+-------------+---------+
| foo  | YES    | YES      | YES             | NO       |             | 2       |
+------+--------+----------+-----------------+----------+-------------+---------+

The project will be listed as expected.

I’ve removed the remote my-lxd-server from the client and added it again:

lxc remote add my-lxd-server https://192.168.XXX.XXX:8443 --protocol=lxd

The output of lxc project list my-lxd-server: remains unchanged.

[Edit:] Both, client and server run LXD 4.15

Ok, interesting, I think there’s something we can do to make lxc remote add slightly smarter in this case, matching the logic we have for candid/rbac authentication.

In any case, what you need to do to make things behave is lxc project switch my-lxd-server:foo as without doing that, you’re querying the default project which you don’t have access to (and so can only see public images).

This would be greatly appreciated. How would the logic you’ve in place for candid/rbac authentication affect the behavior of lxc remote add or the image query commands in general?

This did the job :+1:

My overall goal is to create a private LXD image server. The addition project restricted certificates in LXD 4.12 was very useful in this regard. This way a subset of images in the image store can be added to a project while access to it is restricted to authorized clients only. Now, the images can be queried as well.

Thanks again for your valuable help.

Next, I tried to copy my-image (private) from the project foo from the remote to the client:

lxc image copy  my-lxd-server:my-image local: 
Error: Failed remote image download:
 - https://10.147.96.1:8443: Failed to connect to LXD server "https://10.147.96.1:8443": Get "https://10.147.96.1:8443/1.0": Unable to connect to: 10.147.96.1:8443
 - https://[fd42:8342:9885:2003::1]:8443: Failed to connect to LXD server "https://[fd42:8342:9885:2003::1]:8443": Get "https://[fd42:8342:9885:2003::1]:8443/1.0": Unable to connect to: [fd42:8342:9885:2003::1]:8443
 - https://192.168.199.131:8443: not found

I’ve found older threads in this forum that described problems when copying private images but nothing recent nor open issues on Github.

According to the output of lxc image copy my-lxd-server:my-image local: --debug, the client can find the remote at 10.147.96.1, connect to it, resolve the alias my-image, request a download token and start the download …

lxc image copy  my-lxd-server:my-image local: --debug
DBUG[06-14|12:42:19] Connecting to a remote LXD over HTTPs 
DBUG[06-14|12:42:19] Sending request to LXD                   method=GET url=https://192.168.199.131:8443/1.0 etag=
DBUG[06-14|12:42:19] Got response struct from LXD 
DBUG[06-14|12:42:19] 
	{
		"config": null,
		"api_extensions": [
			"storage_zfs_remove_snapshots",
			"container_host_shutdown_timeout",
			"container_stop_priority",
			"container_syscall_filtering",
			"auth_pki",
			"container_last_used_at",
			"etag",
			"patch",
			"usb_devices",
			"https_allowed_credentials",
			"image_compression_algorithm",
			"directory_manipulation",
			"container_cpu_time",
			"storage_zfs_use_refquota",
			"storage_lvm_mount_options",
			"network",
			"profile_usedby",
			"container_push",
			"container_exec_recording",
			"certificate_update",
			"container_exec_signal_handling",
			"gpu_devices",
			"container_image_properties",
			"migration_progress",
			"id_map",
			"network_firewall_filtering",
			"network_routes",
			"storage",
			"file_delete",
			"file_append",
			"network_dhcp_expiry",
			"storage_lvm_vg_rename",
			"storage_lvm_thinpool_rename",
			"network_vlan",
			"image_create_aliases",
			"container_stateless_copy",
			"container_only_migration",
			"storage_zfs_clone_copy",
			"unix_device_rename",
			"storage_lvm_use_thinpool",
			"storage_rsync_bwlimit",
			"network_vxlan_interface",
			"storage_btrfs_mount_options",
			"entity_description",
			"image_force_refresh",
			"storage_lvm_lv_resizing",
			"id_map_base",
			"file_symlinks",
			"container_push_target",
			"network_vlan_physical",
			"storage_images_delete",
			"container_edit_metadata",
			"container_snapshot_stateful_migration",
			"storage_driver_ceph",
			"storage_ceph_user_name",
			"resource_limits",
			"storage_volatile_initial_source",
			"storage_ceph_force_osd_reuse",
			"storage_block_filesystem_btrfs",
			"resources",
			"kernel_limits",
			"storage_api_volume_rename",
			"macaroon_authentication",
			"network_sriov",
			"console",
			"restrict_devlxd",
			"migration_pre_copy",
			"infiniband",
			"maas_network",
			"devlxd_events",
			"proxy",
			"network_dhcp_gateway",
			"file_get_symlink",
			"network_leases",
			"unix_device_hotplug",
			"storage_api_local_volume_handling",
			"operation_description",
			"clustering",
			"event_lifecycle",
			"storage_api_remote_volume_handling",
			"nvidia_runtime",
			"container_mount_propagation",
			"container_backup",
			"devlxd_images",
			"container_local_cross_pool_handling",
			"proxy_unix",
			"proxy_udp",
			"clustering_join",
			"proxy_tcp_udp_multi_port_handling",
			"network_state",
			"proxy_unix_dac_properties",
			"container_protection_delete",
			"unix_priv_drop",
			"pprof_http",
			"proxy_haproxy_protocol",
			"network_hwaddr",
			"proxy_nat",
			"network_nat_order",
			"container_full",
			"candid_authentication",
			"backup_compression",
			"candid_config",
			"nvidia_runtime_config",
			"storage_api_volume_snapshots",
			"storage_unmapped",
			"projects",
			"candid_config_key",
			"network_vxlan_ttl",
			"container_incremental_copy",
			"usb_optional_vendorid",
			"snapshot_scheduling",
			"snapshot_schedule_aliases",
			"container_copy_project",
			"clustering_server_address",
			"clustering_image_replication",
			"container_protection_shift",
			"snapshot_expiry",
			"container_backup_override_pool",
			"snapshot_expiry_creation",
			"network_leases_location",
			"resources_cpu_socket",
			"resources_gpu",
			"resources_numa",
			"kernel_features",
			"id_map_current",
			"event_location",
			"storage_api_remote_volume_snapshots",
			"network_nat_address",
			"container_nic_routes",
			"rbac",
			"cluster_internal_copy",
			"seccomp_notify",
			"lxc_features",
			"container_nic_ipvlan",
			"network_vlan_sriov",
			"storage_cephfs",
			"container_nic_ipfilter",
			"resources_v2",
			"container_exec_user_group_cwd",
			"container_syscall_intercept",
			"container_disk_shift",
			"storage_shifted",
			"resources_infiniband",
			"daemon_storage",
			"instances",
			"image_types",
			"resources_disk_sata",
			"clustering_roles",
			"images_expiry",
			"resources_network_firmware",
			"backup_compression_algorithm",
			"ceph_data_pool_name",
			"container_syscall_intercept_mount",
			"compression_squashfs",
			"container_raw_mount",
			"container_nic_routed",
			"container_syscall_intercept_mount_fuse",
			"container_disk_ceph",
			"virtual-machines",
			"image_profiles",
			"clustering_architecture",
			"resources_disk_id",
			"storage_lvm_stripes",
			"vm_boot_priority",
			"unix_hotplug_devices",
			"api_filtering",
			"instance_nic_network",
			"clustering_sizing",
			"firewall_driver",
			"projects_limits",
			"container_syscall_intercept_hugetlbfs",
			"limits_hugepages",
			"container_nic_routed_gateway",
			"projects_restrictions",
			"custom_volume_snapshot_expiry",
			"volume_snapshot_scheduling",
			"trust_ca_certificates",
			"snapshot_disk_usage",
			"clustering_edit_roles",
			"container_nic_routed_host_address",
			"container_nic_ipvlan_gateway",
			"resources_usb_pci",
			"resources_cpu_threads_numa",
			"resources_cpu_core_die",
			"api_os",
			"container_nic_routed_host_table",
			"container_nic_ipvlan_host_table",
			"container_nic_ipvlan_mode",
			"resources_system",
			"images_push_relay",
			"network_dns_search",
			"container_nic_routed_limits",
			"instance_nic_bridged_vlan",
			"network_state_bond_bridge",
			"usedby_consistency",
			"custom_block_volumes",
			"clustering_failure_domains",
			"resources_gpu_mdev",
			"console_vga_type",
			"projects_limits_disk",
			"network_type_macvlan",
			"network_type_sriov",
			"container_syscall_intercept_bpf_devices",
			"network_type_ovn",
			"projects_networks",
			"projects_networks_restricted_uplinks",
			"custom_volume_backup",
			"backup_override_name",
			"storage_rsync_compression",
			"network_type_physical",
			"network_ovn_external_subnets",
			"network_ovn_nat",
			"network_ovn_external_routes_remove",
			"tpm_device_type",
			"storage_zfs_clone_copy_rebase",
			"gpu_mdev",
			"resources_pci_iommu",
			"resources_network_usb",
			"resources_disk_address",
			"network_physical_ovn_ingress_mode",
			"network_ovn_dhcp",
			"network_physical_routes_anycast",
			"projects_limits_instances",
			"network_state_vlan",
			"instance_nic_bridged_port_isolation",
			"instance_bulk_state_change",
			"network_gvrp",
			"instance_pool_move",
			"gpu_sriov",
			"pci_device_type",
			"storage_volume_state",
			"network_acl",
			"migration_stateful",
			"disk_state_quota",
			"storage_ceph_features",
			"projects_compression",
			"projects_images_remote_cache_expiry",
			"certificate_project",
			"network_ovn_acl",
			"projects_images_auto_update",
			"projects_restricted_cluster_target",
			"images_default_architecture",
			"network_ovn_acl_defaults",
			"gpu_mig",
			"project_usage",
			"network_bridge_acl",
			"warnings",
			"projects_restricted_backups_and_snapshots",
			"clustering_join_token",
			"clustering_description",
			"server_trusted_proxy"
		],
		"api_status": "stable",
		"api_version": "1.0",
		"auth": "trusted",
		"public": false,
		"auth_methods": [
			"tls"
		],
		"environment": {
			"addresses": [
				"192.168.199.131:8443",
				"10.147.96.1:8443",
				"[fd42:8342:9885:2003::1]:8443"
			],
			"architectures": [
				"x86_64",
				"i686"
			],
			"certificate": "-----BEGIN CERTIFICATE-----\nMIICDzCCAZWgAwIBAgIQS6LymZ9V/hhbKID08pj/rDAKBggqhkjOPQQDAzA4MRww\nGgYDVQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRgwFgYDVQQDDA9yb290QGx4ZC1z\nZXJ2ZXIwHhcNMjEwNjAyMTAxMTU2WhcNMzEwNTMxMTAxMTU2WjA4MRwwGgYDVQQK\nExNsaW51eGNvbnRhaW5lcnMub3JnMRgwFgYDVQQDDA9yb290QGx4ZC1zZXJ2ZXIw\ndjAQBgcqhkjOPQIBBgUrgQQAIgNiAATpNkGHtraND3jhuMTeuMQOCdTRxS+EhRX3\nNC6f1ZBD9wIsi9PPy1OOPqaIgxcKqXtR6PTOJH7Zr1Ksoh8PmQSYKFJSzd4/G0as\noAmgG/izH/8xSFN5n/EeUov1R9MMCeajZDBiMA4GA1UdDwEB/wQEAwIFoDATBgNV\nHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMC0GA1UdEQQmMCSCCmx4ZC1z\nZXJ2ZXKHBH8AAAGHEAAAAAAAAAAAAAAAAAAAAAEwCgYIKoZIzj0EAwMDaAAwZQIw\nDPb7+3Vkd7y1b+H/S9JiMr0T0BNijKUaF17CmKIAJhx48IDMKX20Q1n037H8wtvK\nAjEA1FuDi7aEPFq/Nc68sWTogIFf7VqBm6NNRuEi+JblaWItQZVrEvWlU9STs1WT\nKtNA\n-----END CERTIFICATE-----\n",
			"certificate_fingerprint": "2ac0e709d7a892490b13cc608aa2df79058392a338ea0cf52956c3f3a952427f",
			"driver": "qemu | lxc",
			"driver_version": "5.2.0 | 4.0.9",
			"firewall": "nftables",
			"kernel": "Linux",
			"kernel_architecture": "x86_64",
			"kernel_features": {
				"netnsid_getifaddrs": "true",
				"seccomp_listener": "true",
				"seccomp_listener_continue": "true",
				"shiftfs": "false",
				"uevent_injection": "true",
				"unpriv_fscaps": "true"
			},
			"kernel_version": "5.4.0-73-generic",
			"lxc_features": {
				"cgroup2": "true",
				"devpts_fd": "true",
				"idmapped_mounts_v2": "false",
				"mount_injection_file": "true",
				"network_gateway_device_route": "true",
				"network_ipvlan": "true",
				"network_l2proxy": "true",
				"network_phys_macvlan_mtu": "true",
				"network_veth_router": "true",
				"pidfd": "true",
				"seccomp_allow_deny_syntax": "true",
				"seccomp_notify": "true",
				"seccomp_proxy_send_notify_fd": "true"
			},
			"os_name": "Ubuntu",
			"os_version": "20.04",
			"project": "default",
			"server": "lxd",
			"server_clustered": false,
			"server_name": "lxdserver",
			"server_pid": 26712,
			"server_version": "4.15",
			"storage": "btrfs",
			"storage_version": "4.15.1"
		}
	} 
DBUG[06-14|12:42:19] Connecting to a local LXD over a Unix socket 
DBUG[06-14|12:42:19] Sending request to LXD                   method=GET url=http://unix.socket/1.0 etag=
DBUG[06-14|12:42:19] Got response struct from LXD 
DBUG[06-14|12:42:19] 
	{
		"config": {
			"core.trust_ca_certificates": "true"
		},
		"api_extensions": [
			"storage_zfs_remove_snapshots",
			"container_host_shutdown_timeout",
			"container_stop_priority",
			"container_syscall_filtering",
			"auth_pki",
			"container_last_used_at",
			"etag",
			"patch",
			"usb_devices",
			"https_allowed_credentials",
			"image_compression_algorithm",
			"directory_manipulation",
			"container_cpu_time",
			"storage_zfs_use_refquota",
			"storage_lvm_mount_options",
			"network",
			"profile_usedby",
			"container_push",
			"container_exec_recording",
			"certificate_update",
			"container_exec_signal_handling",
			"gpu_devices",
			"container_image_properties",
			"migration_progress",
			"id_map",
			"network_firewall_filtering",
			"network_routes",
			"storage",
			"file_delete",
			"file_append",
			"network_dhcp_expiry",
			"storage_lvm_vg_rename",
			"storage_lvm_thinpool_rename",
			"network_vlan",
			"image_create_aliases",
			"container_stateless_copy",
			"container_only_migration",
			"storage_zfs_clone_copy",
			"unix_device_rename",
			"storage_lvm_use_thinpool",
			"storage_rsync_bwlimit",
			"network_vxlan_interface",
			"storage_btrfs_mount_options",
			"entity_description",
			"image_force_refresh",
			"storage_lvm_lv_resizing",
			"id_map_base",
			"file_symlinks",
			"container_push_target",
			"network_vlan_physical",
			"storage_images_delete",
			"container_edit_metadata",
			"container_snapshot_stateful_migration",
			"storage_driver_ceph",
			"storage_ceph_user_name",
			"resource_limits",
			"storage_volatile_initial_source",
			"storage_ceph_force_osd_reuse",
			"storage_block_filesystem_btrfs",
			"resources",
			"kernel_limits",
			"storage_api_volume_rename",
			"macaroon_authentication",
			"network_sriov",
			"console",
			"restrict_devlxd",
			"migration_pre_copy",
			"infiniband",
			"maas_network",
			"devlxd_events",
			"proxy",
			"network_dhcp_gateway",
			"file_get_symlink",
			"network_leases",
			"unix_device_hotplug",
			"storage_api_local_volume_handling",
			"operation_description",
			"clustering",
			"event_lifecycle",
			"storage_api_remote_volume_handling",
			"nvidia_runtime",
			"container_mount_propagation",
			"container_backup",
			"devlxd_images",
			"container_local_cross_pool_handling",
			"proxy_unix",
			"proxy_udp",
			"clustering_join",
			"proxy_tcp_udp_multi_port_handling",
			"network_state",
			"proxy_unix_dac_properties",
			"container_protection_delete",
			"unix_priv_drop",
			"pprof_http",
			"proxy_haproxy_protocol",
			"network_hwaddr",
			"proxy_nat",
			"network_nat_order",
			"container_full",
			"candid_authentication",
			"backup_compression",
			"candid_config",
			"nvidia_runtime_config",
			"storage_api_volume_snapshots",
			"storage_unmapped",
			"projects",
			"candid_config_key",
			"network_vxlan_ttl",
			"container_incremental_copy",
			"usb_optional_vendorid",
			"snapshot_scheduling",
			"snapshot_schedule_aliases",
			"container_copy_project",
			"clustering_server_address",
			"clustering_image_replication",
			"container_protection_shift",
			"snapshot_expiry",
			"container_backup_override_pool",
			"snapshot_expiry_creation",
			"network_leases_location",
			"resources_cpu_socket",
			"resources_gpu",
			"resources_numa",
			"kernel_features",
			"id_map_current",
			"event_location",
			"storage_api_remote_volume_snapshots",
			"network_nat_address",
			"container_nic_routes",
			"rbac",
			"cluster_internal_copy",
			"seccomp_notify",
			"lxc_features",
			"container_nic_ipvlan",
			"network_vlan_sriov",
			"storage_cephfs",
			"container_nic_ipfilter",
			"resources_v2",
			"container_exec_user_group_cwd",
			"container_syscall_intercept",
			"container_disk_shift",
			"storage_shifted",
			"resources_infiniband",
			"daemon_storage",
			"instances",
			"image_types",
			"resources_disk_sata",
			"clustering_roles",
			"images_expiry",
			"resources_network_firmware",
			"backup_compression_algorithm",
			"ceph_data_pool_name",
			"container_syscall_intercept_mount",
			"compression_squashfs",
			"container_raw_mount",
			"container_nic_routed",
			"container_syscall_intercept_mount_fuse",
			"container_disk_ceph",
			"virtual-machines",
			"image_profiles",
			"clustering_architecture",
			"resources_disk_id",
			"storage_lvm_stripes",
			"vm_boot_priority",
			"unix_hotplug_devices",
			"api_filtering",
			"instance_nic_network",
			"clustering_sizing",
			"firewall_driver",
			"projects_limits",
			"container_syscall_intercept_hugetlbfs",
			"limits_hugepages",
			"container_nic_routed_gateway",
			"projects_restrictions",
			"custom_volume_snapshot_expiry",
			"volume_snapshot_scheduling",
			"trust_ca_certificates",
			"snapshot_disk_usage",
			"clustering_edit_roles",
			"container_nic_routed_host_address",
			"container_nic_ipvlan_gateway",
			"resources_usb_pci",
			"resources_cpu_threads_numa",
			"resources_cpu_core_die",
			"api_os",
			"container_nic_routed_host_table",
			"container_nic_ipvlan_host_table",
			"container_nic_ipvlan_mode",
			"resources_system",
			"images_push_relay",
			"network_dns_search",
			"container_nic_routed_limits",
			"instance_nic_bridged_vlan",
			"network_state_bond_bridge",
			"usedby_consistency",
			"custom_block_volumes",
			"clustering_failure_domains",
			"resources_gpu_mdev",
			"console_vga_type",
			"projects_limits_disk",
			"network_type_macvlan",
			"network_type_sriov",
			"container_syscall_intercept_bpf_devices",
			"network_type_ovn",
			"projects_networks",
			"projects_networks_restricted_uplinks",
			"custom_volume_backup",
			"backup_override_name",
			"storage_rsync_compression",
			"network_type_physical",
			"network_ovn_external_subnets",
			"network_ovn_nat",
			"network_ovn_external_routes_remove",
			"tpm_device_type",
			"storage_zfs_clone_copy_rebase",
			"gpu_mdev",
			"resources_pci_iommu",
			"resources_network_usb",
			"resources_disk_address",
			"network_physical_ovn_ingress_mode",
			"network_ovn_dhcp",
			"network_physical_routes_anycast",
			"projects_limits_instances",
			"network_state_vlan",
			"instance_nic_bridged_port_isolation",
			"instance_bulk_state_change",
			"network_gvrp",
			"instance_pool_move",
			"gpu_sriov",
			"pci_device_type",
			"storage_volume_state",
			"network_acl",
			"migration_stateful",
			"disk_state_quota",
			"storage_ceph_features",
			"projects_compression",
			"projects_images_remote_cache_expiry",
			"certificate_project",
			"network_ovn_acl",
			"projects_images_auto_update",
			"projects_restricted_cluster_target",
			"images_default_architecture",
			"network_ovn_acl_defaults",
			"gpu_mig",
			"project_usage",
			"network_bridge_acl",
			"warnings",
			"projects_restricted_backups_and_snapshots",
			"clustering_join_token",
			"clustering_description",
			"server_trusted_proxy"
		],
		"api_status": "stable",
		"api_version": "1.0",
		"auth": "trusted",
		"public": false,
		"auth_methods": [
			"tls"
		],
		"environment": {
			"addresses": [],
			"architectures": [
				"x86_64",
				"i686"
			],
			"certificate": "-----BEGIN CERTIFICATE-----\nMIICAzCCAYmgAwIBAgIQdmRHPTM+KMJFxXv1GnFqHDAKBggqhkjOPQQDAzA0MRww\nGgYDVQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRQwEgYDVQQDDAtyb290QHVidW50\ndTAeFw0yMTA1MDgxNDMzMTBaFw0zMTA1MDYxNDMzMTBaMDQxHDAaBgNVBAoTE2xp\nbnV4Y29udGFpbmVycy5vcmcxFDASBgNVBAMMC3Jvb3RAdWJ1bnR1MHYwEAYHKoZI\nzj0CAQYFK4EEACIDYgAEp+2rTfQNXIDUaPjg6w81o7kyB5XUCrLa+I+eecPfSnym\nmwCaDaF2weON4E893SZFFqkWbZq2Af7I9WvHb7BJwN5il3HYHGAEmLdgxCK9MF+K\nWQscuAUr1G3BkvjSHn5eo2AwXjAOBgNVHQ8BAf8EBAMCBaAwEwYDVR0lBAwwCgYI\nKwYBBQUHAwEwDAYDVR0TAQH/BAIwADApBgNVHREEIjAgggZ1YnVudHWHBH8AAAGH\nEAAAAAAAAAAAAAAAAAAAAAEwCgYIKoZIzj0EAwMDaAAwZQIxAPs+JTmtEeSfXfuR\nK1/v2F0cs085tX29CJ9tiNUoY4YdrsvJqi51GSHgAV8XATlhWQIwZPbx1RW4JfmC\nArz+Esb1ItZvzCpfTJ7oaNRh13BUbySc1jJEKBYiiolbstgRVCXt\n-----END CERTIFICATE-----\n",
			"certificate_fingerprint": "cc2527284ed0c26eecd230d0882797609b7eb20e4787352ec020df22532a31db",
			"driver": "lxc | qemu",
			"driver_version": "4.0.9 | 5.2.0",
			"firewall": "nftables",
			"kernel": "Linux",
			"kernel_architecture": "x86_64",
			"kernel_features": {
				"netnsid_getifaddrs": "true",
				"seccomp_listener": "true",
				"seccomp_listener_continue": "true",
				"shiftfs": "false",
				"uevent_injection": "true",
				"unpriv_fscaps": "true"
			},
			"kernel_version": "5.8.0-53-generic",
			"lxc_features": {
				"cgroup2": "true",
				"devpts_fd": "true",
				"idmapped_mounts_v2": "false",
				"mount_injection_file": "true",
				"network_gateway_device_route": "true",
				"network_ipvlan": "true",
				"network_l2proxy": "true",
				"network_phys_macvlan_mtu": "true",
				"network_veth_router": "true",
				"pidfd": "true",
				"seccomp_allow_deny_syntax": "true",
				"seccomp_notify": "true",
				"seccomp_proxy_send_notify_fd": "true"
			},
			"os_name": "Ubuntu",
			"os_version": "20.04",
			"project": "default",
			"server": "lxd",
			"server_clustered": false,
			"server_name": "ubuntu",
			"server_pid": 136688,
			"server_version": "4.15",
			"storage": "btrfs",
			"storage_version": "4.15.1"
		}
	} 
DBUG[06-14|12:42:19] Sending request to LXD                   method=GET url="https://192.168.199.131:8443/1.0/images/aliases/my-image?project=foo" etag=
DBUG[06-14|12:42:19] Got response struct from LXD 
DBUG[06-14|12:42:19] 
	{
		"description": "",
		"target": "e096be4e1d497891a08763e33a94f6d4bb785c47b69d24cc0a746dcc576a5fda",
		"name": "my-image",
		"type": "container"
	} 
DBUG[06-14|12:42:19] Sending request to LXD                   method=GET url="https://192.168.199.131:8443/1.0/images/e096be4e1d497891a08763e33a94f6d4bb785c47b69d24cc0a746dcc576a5fda?project=foo" etag=
DBUG[06-14|12:42:19] Got response struct from LXD 
DBUG[06-14|12:42:19] 
	{
		"auto_update": false,
		"properties": {
			"architecture": "amd64",
			"description": "Alpine 3.13 amd64 (20210613_13:00)",
			"os": "Alpine",
			"release": "3.13",
			"serial": "20210613_13:00",
			"type": "squashfs",
			"variant": "default"
		},
		"public": false,
		"expires_at": "1970-01-01T00:00:00Z",
		"profiles": [
			"default"
		],
		"aliases": [
			{
				"name": "my-image",
				"description": ""
			}
		],
		"architecture": "x86_64",
		"cached": false,
		"filename": "lxd.tar.xz",
		"fingerprint": "e096be4e1d497891a08763e33a94f6d4bb785c47b69d24cc0a746dcc576a5fda",
		"size": 2564996,
		"update_source": {
			"alias": "alpine/3.13/amd64",
			"certificate": "",
			"protocol": "simplestreams",
			"server": "https://images.linuxcontainers.org",
			"image_type": ""
		},
		"type": "container",
		"created_at": "2021-06-13T00:00:00Z",
		"last_used_at": "0001-01-01T00:00:00Z",
		"uploaded_at": "2021-06-14T15:13:46.763604717Z"
	} 
DBUG[06-14|12:42:19] Connected to the websocket: wss://192.168.199.131:8443/1.0/events?project=foo 
DBUG[06-14|12:42:19] Sending request to LXD                   method=POST url="https://192.168.199.131:8443/1.0/images/e096be4e1d497891a08763e33a94f6d4bb785c47b69d24cc0a746dcc576a5fda/secret?project=foo" etag=
DBUG[06-14|12:42:19] Got operation from LXD 
DBUG[06-14|12:42:19] 
	{
		"id": "20c7d399-2444-450b-b7e8-034c69ffd33d",
		"class": "token",
		"description": "Image download token",
		"created_at": "2021-06-14T19:42:19.327763697Z",
		"updated_at": "2021-06-14T19:42:19.327763697Z",
		"status": "Running",
		"status_code": 103,
		"resources": {
			"images": [
				"/1.0/images/e096be4e1d497891a08763e33a94f6d4bb785c47b69d24cc0a746dcc576a5fda"
			]
		},
		"metadata": {
			"secret": "9a5b433dc4e3806d628eadda1b5e493db4fca29317009ea3434e35a1cc0612e6"
		},
		"may_cancel": true,
		"err": "",
		"location": "none"
	} 
DBUG[06-14|12:42:19] Connected to the websocket: ws://unix.socket/1.0/events 
DBUG[06-14|12:42:19] Sending request to LXD                   method=POST url=http://unix.socket/1.0/images etag=
DBUG[06-14|12:42:19] 
	{
		"auto_update": false,
		"properties": null,
		"public": false,
		"expires_at": "0001-01-01T00:00:00Z",
		"profiles": null,
		"filename": "",
		"source": {
			"alias": "",
			"certificate": "-----BEGIN CERTIFICATE-----\nMIICDzCCAZWgAwIBAgIQS6LymZ9V/hhbKID08pj/rDAKBggqhkjOPQQDAzA4MRww\nGgYDVQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRgwFgYDVQQDDA9yb290QGx4ZC1z\nZXJ2ZXIwHhcNMjEwNjAyMTAxMTU2WhcNMzEwNTMxMTAxMTU2WjA4MRwwGgYDVQQK\nExNsaW51eGNvbnRhaW5lcnMub3JnMRgwFgYDVQQDDA9yb290QGx4ZC1zZXJ2ZXIw\ndjAQBgcqhkjOPQIBBgUrgQQAIgNiAATpNkGHtraND3jhuMTeuMQOCdTRxS+EhRX3\nNC6f1ZBD9wIsi9PPy1OOPqaIgxcKqXtR6PTOJH7Zr1Ksoh8PmQSYKFJSzd4/G0as\noAmgG/izH/8xSFN5n/EeUov1R9MMCeajZDBiMA4GA1UdDwEB/wQEAwIFoDATBgNV\nHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMC0GA1UdEQQmMCSCCmx4ZC1z\nZXJ2ZXKHBH8AAAGHEAAAAAAAAAAAAAAAAAAAAAEwCgYIKoZIzj0EAwMDaAAwZQIw\nDPb7+3Vkd7y1b+H/S9JiMr0T0BNijKUaF17CmKIAJhx48IDMKX20Q1n037H8wtvK\nAjEA1FuDi7aEPFq/Nc68sWTogIFf7VqBm6NNRuEi+JblaWItQZVrEvWlU9STs1WT\nKtNA\n-----END CERTIFICATE-----\n",
			"protocol": "lxd",
			"server": "https://192.168.199.131:8443",
			"image_type": "",
			"mode": "pull",
			"type": "image",
			"url": "",
			"name": "",
			"fingerprint": "e096be4e1d497891a08763e33a94f6d4bb785c47b69d24cc0a746dcc576a5fda",
			"secret": "9a5b433dc4e3806d628eadda1b5e493db4fca29317009ea3434e35a1cc0612e6"
		},
		"compression_algorithm": "",
		"aliases": null
	} 
DBUG[06-14|12:42:19] Got operation from LXD 
DBUG[06-14|12:42:19] 
	{
		"id": "c2e34e8e-94a4-4d7a-a50e-9bfa9ed33e41",
		"class": "task",
		"description": "Downloading image",
		"created_at": "2021-06-14T12:42:19.333924991-07:00",
		"updated_at": "2021-06-14T12:42:19.333924991-07:00",
		"status": "Running",
		"status_code": 103,
		"resources": null,
		"metadata": null,
		"may_cancel": false,
		"err": "",
		"location": "none"
	} 
DBUG[06-14|12:42:19] Sending request to LXD                   method=GET url=http://unix.socket/1.0/operations/c2e34e8e-94a4-4d7a-a50e-9bfa9ed33e41 etag=
DBUG[06-14|12:42:19] Got response struct from LXD 
DBUG[06-14|12:42:19] 
	{
		"id": "c2e34e8e-94a4-4d7a-a50e-9bfa9ed33e41",
		"class": "task",
		"description": "Downloading image",
		"created_at": "2021-06-14T12:42:19.333924991-07:00",
		"updated_at": "2021-06-14T12:42:19.333924991-07:00",
		"status": "Running",
		"status_code": 103,
		"resources": null,
		"metadata": null,
		"may_cancel": false,
		"err": "",
		"location": "none"
	} 
DBUG[06-14|12:42:19] Sending request to LXD                   method=POST url=http://unix.socket/1.0/images etag=
DBUG[06-14|12:42:19] 
	{
		"auto_update": false,
		"properties": null,
		"public": false,
		"expires_at": "0001-01-01T00:00:00Z",
		"profiles": null,
		"filename": "",
		"source": {
			"alias": "",
			"certificate": "-----BEGIN CERTIFICATE-----\nMIICDzCCAZWgAwIBAgIQS6LymZ9V/hhbKID08pj/rDAKBggqhkjOPQQDAzA4MRww\nGgYDVQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRgwFgYDVQQDDA9yb290QGx4ZC1z\nZXJ2ZXIwHhcNMjEwNjAyMTAxMTU2WhcNMzEwNTMxMTAxMTU2WjA4MRwwGgYDVQQK\nExNsaW51eGNvbnRhaW5lcnMub3JnMRgwFgYDVQQDDA9yb290QGx4ZC1zZXJ2ZXIw\ndjAQBgcqhkjOPQIBBgUrgQQAIgNiAATpNkGHtraND3jhuMTeuMQOCdTRxS+EhRX3\nNC6f1ZBD9wIsi9PPy1OOPqaIgxcKqXtR6PTOJH7Zr1Ksoh8PmQSYKFJSzd4/G0as\noAmgG/izH/8xSFN5n/EeUov1R9MMCeajZDBiMA4GA1UdDwEB/wQEAwIFoDATBgNV\nHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMC0GA1UdEQQmMCSCCmx4ZC1z\nZXJ2ZXKHBH8AAAGHEAAAAAAAAAAAAAAAAAAAAAEwCgYIKoZIzj0EAwMDaAAwZQIw\nDPb7+3Vkd7y1b+H/S9JiMr0T0BNijKUaF17CmKIAJhx48IDMKX20Q1n037H8wtvK\nAjEA1FuDi7aEPFq/Nc68sWTogIFf7VqBm6NNRuEi+JblaWItQZVrEvWlU9STs1WT\nKtNA\n-----END CERTIFICATE-----\n",
			"protocol": "lxd",
			"server": "https://10.147.96.1:8443",
			"image_type": "",
			"mode": "pull",
			"type": "image",
			"url": "",
			"name": "",
			"fingerprint": "e096be4e1d497891a08763e33a94f6d4bb785c47b69d24cc0a746dcc576a5fda",
			"secret": "9a5b433dc4e3806d628eadda1b5e493db4fca29317009ea3434e35a1cc0612e6"
		},
		"compression_algorithm": "",
		"aliases": null
	} 
DBUG[06-14|12:42:19] Got operation from LXD 
DBUG[06-14|12:42:19] 
	{
		"id": "ca461a8e-9d40-47d6-8a36-fe4b83d9e056",
		"class": "task",
		"description": "Downloading image",
		"created_at": "2021-06-14T12:42:19.388906446-07:00",
		"updated_at": "2021-06-14T12:42:19.388906446-07:00",
		"status": "Running",
		"status_code": 103,
		"resources": null,
		"metadata": null,
		"may_cancel": false,
		"err": "",
		"location": "none"
	} 
DBUG[06-14|12:42:19] Sending request to LXD                   method=GET url=http://unix.socket/1.0/operations/ca461a8e-9d40-47d6-8a36-fe4b83d9e056 etag=
DBUG[06-14|12:42:19] Got response struct from LXD 
DBUG[06-14|12:42:19] 
	{
		"id": "ca461a8e-9d40-47d6-8a36-fe4b83d9e056",
		"class": "task",
		"description": "Downloading image",
		"created_at": "2021-06-14T12:42:19.388906446-07:00",
		"updated_at": "2021-06-14T12:42:19.388906446-07:00",
		"status": "Running",
		"status_code": 103,
		"resources": null,
		"metadata": null,
		"may_cancel": false,
		"err": "",
		"location": "none"
	} 
DBUG[06-14|12:42:39] Sending request to LXD                   method=POST url=http://unix.socket/1.0/images etag=
DBUG[06-14|12:42:39] 
	{
		"auto_update": false,
		"properties": null,
		"public": false,
		"expires_at": "0001-01-01T00:00:00Z",
		"profiles": null,
		"filename": "",
		"source": {
			"alias": "",
			"certificate": "-----BEGIN CERTIFICATE-----\nMIICDzCCAZWgAwIBAgIQS6LymZ9V/hhbKID08pj/rDAKBggqhkjOPQQDAzA4MRww\nGgYDVQQKExNsaW51eGNvbnRhaW5lcnMub3JnMRgwFgYDVQQDDA9yb290QGx4ZC1z\nZXJ2ZXIwHhcNMjEwNjAyMTAxMTU2WhcNMzEwNTMxMTAxMTU2WjA4MRwwGgYDVQQK\nExNsaW51eGNvbnRhaW5lcnMub3JnMRgwFgYDVQQDDA9yb290QGx4ZC1zZXJ2ZXIw\ndjAQBgcqhkjOPQIBBgUrgQQAIgNiAATpNkGHtraND3jhuMTeuMQOCdTRxS+EhRX3\nNC6f1ZBD9wIsi9PPy1OOPqaIgxcKqXtR6PTOJH7Zr1Ksoh8PmQSYKFJSzd4/G0as\noAmgG/izH/8xSFN5n/EeUov1R9MMCeajZDBiMA4GA1UdDwEB/wQEAwIFoDATBgNV\nHSUEDDAKBggrBgEFBQcDATAMBgNVHRMBAf8EAjAAMC0GA1UdEQQmMCSCCmx4ZC1z\nZXJ2ZXKHBH8AAAGHEAAAAAAAAAAAAAAAAAAAAAEwCgYIKoZIzj0EAwMDaAAwZQIw\nDPb7+3Vkd7y1b+H/S9JiMr0T0BNijKUaF17CmKIAJhx48IDMKX20Q1n037H8wtvK\nAjEA1FuDi7aEPFq/Nc68sWTogIFf7VqBm6NNRuEi+JblaWItQZVrEvWlU9STs1WT\nKtNA\n-----END CERTIFICATE-----\n",
			"protocol": "lxd",
			"server": "https://[fd42:8342:9885:2003::1]:8443",
			"image_type": "",
			"mode": "pull",
			"type": "image",
			"url": "",
			"name": "",
			"fingerprint": "e096be4e1d497891a08763e33a94f6d4bb785c47b69d24cc0a746dcc576a5fda",
			"secret": "9a5b433dc4e3806d628eadda1b5e493db4fca29317009ea3434e35a1cc0612e6"
		},
		"compression_algorithm": "",
		"aliases": null
	} 
DBUG[06-14|12:42:39] Got operation from LXD 
DBUG[06-14|12:42:39] 
	{
		"id": "260439dd-ebdf-45bf-ae2f-013b40d77a35",
		"class": "task",
		"description": "Downloading image",
		"created_at": "2021-06-14T12:42:39.408774736-07:00",
		"updated_at": "2021-06-14T12:42:39.408774736-07:00",
		"status": "Running",
		"status_code": 103,
		"resources": null,
		"metadata": null,
		"may_cancel": false,
		"err": "",
		"location": "none"
	} 
DBUG[06-14|12:42:39] Sending request to LXD                   method=GET url=http://unix.socket/1.0/operations/260439dd-ebdf-45bf-ae2f-013b40d77a35 etag=
DBUG[06-14|12:42:39] Got response struct from LXD 
DBUG[06-14|12:42:39] 
	{
		"id": "260439dd-ebdf-45bf-ae2f-013b40d77a35",
		"class": "task",
		"description": "Downloading image",
		"created_at": "2021-06-14T12:42:39.408774736-07:00",
		"updated_at": "2021-06-14T12:42:39.408774736-07:00",
		"status": "Running",
		"status_code": 103,
		"resources": null,
		"metadata": null,
		"may_cancel": false,
		"err": "",
		"location": "none"
	} 
Error: Failed remote image download:
 - https://192.168.199.131:8443: not found
 - https://10.147.96.1:8443: Failed to connect to LXD server "https://10.147.96.1:8443": Get "https://10.147.96.1:8443/1.0": Unable to connect to: 10.147.96.1:8443
 - https://[fd42:8342:9885:2003::1]:8443: Failed to connect to LXD server "https://[fd42:8342:9885:2003::1]:8443": Get "https://[fd42:8342:9885:2003::1]:8443/1.0": Unable to connect to: [fd42:8342:9885:2003::1]:8443

Good one, adding to our list of stuff to fix :wink:

Basically we don’t seem to have a way in pull mode to tell the target server what project to pull from. So it’s going to be looking at the default project…

I suspect passing --mode=relay will fix it for now though.

1 Like

The logic we have in the candid/rbac case is to perform a project list at the time we add the remote. Then if the default project is available, we use that as usual, if it’s not and there’s only one other project, that one becomes the one we use, then if there are multiple projects and the default one isn’t one of them, we show a list to the user.

The same logic should be used for restricted certificates as the same scenarios can happen. I’ve added it to our list of tweaks to make.

Yep, with this additional flag it works like a charm.


The logic implemented for candid/rbac would really make it more intuitive for users who have to deal with remote(s) and multiple projects. I’m looking forward to see this features and fixes in an upcoming release.

Yeah, I build up a list on the side (Trello) with every little bug and annoyances that we notice during day to day usage or see mentioned here on the forum.

Then if I have a few minutes to spare here or there I may just go fix a few, if not, I process them all in batches every so often, filing Easy issues on Github which can then be picked up by anyone wanting to make an easy contribution to LXD.

1 Like