Overwriting device information during a container copy via the API

Jimbo · December 18, 2020, 7:18pm

I am copying a container via the API but I do not want it to copy over proxies, but when i overwrite the list of devices the new copy still has this device.

The manual states Config override / Optional list of devices the instance should have, so not sure if I am misunderstanding this.

This is what I send via POST to /instances

  Array
(
    [name] => foo
    [architecture] => x86_64
    [profiles] => Array
        (
            [0] => custom-default
            [1] => custom-nat
        )

    [ephemeral] => 
    [config] => Array
        (
            [image.architecture] => amd64
            [image.description] => Alpine 3.12 amd64 (20201217_13:00)
            [image.os] => Alpine
            [image.release] => 3.12
            [image.serial] => 20201217_13:00
            [image.type] => squashfs
            [limits.cpu] => 400
            [limits.memory] => 1GB
            [volatile.base_image] => bf419ffb3b7a600ea86d386558e52b0d56b7a064289d21a95032b2549528eafe
        )

    [stateful] => 
    [source] => Array
        (
            [type] => copy
            [certificate] => 
            [base-image] => bf419ffb3b7a600ea86d386558e52b0d56b7a064289d21a95032b2549528eafe
            [source] => alpine
            [live] => 
            [instance_only] => 1
        )

    [devices] => Array
        (
            [eth0] => Array
                (
                    [name] => eth0
                    [nictype] => bridged
                    [parent] => custombr0
                    [type] => nic
                )

            [root] => Array
                (
                    [path] => /
                    [pool] => default
                    [size] => 5GB
                    [type] => disk
                )

        )

    [type] => container
)

This new container has 3 devices, including this proxy which I removed.

 [my-proxy] => Array
    (
        [connect] => tcp:0.0.0.0:80
        [listen] => tcp:192.168.1.100:3080
        [nat] => true
        [type] => proxy
    )

Thanks in advance

tomp · December 21, 2020, 8:39pm

I believe the devices supplied in this option would add new devices or override devices that exist in the instance already (i.e you don’t have to provide a complete set of devices). I’m not sure its possible to specify that a device that exists should be removed.

@stgraber is this behaviour by design?

stgraber · December 21, 2020, 11:51pm

I would expect this to work like a PUT, so if the user passes null as devices, then we don’t perform any changes to the source. If something is passed, then we replace whatever we may be getting as part of migration with the provided data.

Jimbo · December 22, 2020, 1:21pm

Looking the GO source ( not a go programmer) it just overwrites individual settings for config or for a device, but you can’t actually override the devices or the config with something different?

stgraber · December 22, 2020, 3:15pm

Can you file a bug on GitHub so we don’t forget about this?

Canonical staff is already on holiday break until the 4th so it’s a bit best effort until then

tomp · December 22, 2020, 4:21pm

I think the code we are referring to is:

github.com

lxc/lxd/blob/ee2375faaf91a2031db459b5c97b225711371e55/lxd/instances_post.go#L457-L468


	if req.Devices == nil {
		req.Devices = make(map[string]map[string]string)
	}
	for key, value := range sourceDevices {
		_, exists := req.Devices[key]
		if exists {
			continue
		}
		req.Devices[key] = value
	}

If so, then as it has been 4 years since it was introduced I wonder if we should add a new field or URL parameter to specify the change in behaviour as otherwise we may find other users are depending on the current behaviour.

What do you think?

Jimbo · December 22, 2020, 7:17pm

May I suggest just waiting until next major release and do as a breaking change, that solves the dilemma and avoids having to have another option.

As a developer I found it pretty confusing as I wanted to make changes to containers due to start problems, but to be honest I am not sure if i should be changing the settings before the copy anyway (this is probably why people have not noticed it). Doing another request to make changes also seems reasonable. You have had to do via the command line anyway.

stgraber · December 22, 2020, 8:53pm

I’ll have to take a look but I suspect it’s a case of nobody directly exercising that code path. We do through the CLI but I’m pretty sure we’re pushing a full set of devices, in line with how I would expect this to have worked.

Partial data is only really a thing with PATCH.

tomp · January 4, 2021, 3:57pm

Confirmed that lxc gets all of source container’s devices first and then sends them as the target’s devices, e.g.

Source container config:

devices:
  p1:
    connect: tcp:127.0.0.1:1234
    listen: tcp:127.0.0.1:1234
    type: proxy
  p2:
    connect: tcp:127.0.0.1:1234
    listen: tcp:127.0.0.1:1236
    type: proxy
  root:
    path: /
    pool: zfs
    type: disk

lxc copy c1 c2
DBUG[01-04|15:52:50] Handling                                 ip=@ username=user protocol=unix method=POST url=/1.0/instances
DBUG[01-04|15:52:50] 
	{
		"architecture": "x86_64",
		"config": {
			"image.architecture": "amd64",
			"image.description": "Ubuntu focal amd64 (20210104_07:42)",
			"image.os": "Ubuntu",
			"image.release": "focal",
			"image.serial": "20210104_07:42",
			"image.type": "squashfs",
			"image.variant": "default",
			"volatile.base_image": "f80186e7f084a36bcabe2b7e82f107bc6a4be1a1731e34dccd558860ecbdd8d6"
		},
		"devices": {
			"p1": {
				"connect": "tcp:127.0.0.1:1234",
				"listen": "tcp:127.0.0.1:1234",
				"type": "proxy"
			},
			"p2": {
				"connect": "tcp:127.0.0.1:1234",
				"listen": "tcp:127.0.0.1:1236",
				"type": "proxy"
			},
			"root": {
				"path": "/",
				"pool": "zfs",
				"type": "disk"
			}
		},
		"ephemeral": false,
		"profiles": [
			"default"
		],
		"stateful": false,
		"description": "",
		"name": "c2",
		"source": {
			"type": "copy",
			"certificate": "",
			"base-image": "f80186e7f084a36bcabe2b7e82f107bc6a4be1a1731e34dccd558860ecbdd8d6",
			"source": "c1",
			"live": true
		},
		"instance_type": "",
		"type": "container"
	}

And if I supply the -d argument to modify the container’s devices, this appears to happen inside the lxc client:

lxc copy c1 c2 -d p1,listen=tcp:127.0.0.1:1235
DBUG[01-04|15:54:01] Handling                                 method=POST url=/1.0/instances ip=@ username=user protocol=unix
DBUG[01-04|15:54:01] 
	{
		"architecture": "x86_64",
		"config": {
			"image.architecture": "amd64",
			"image.description": "Ubuntu focal amd64 (20210104_07:42)",
			"image.os": "Ubuntu",
			"image.release": "focal",
			"image.serial": "20210104_07:42",
			"image.type": "squashfs",
			"image.variant": "default",
			"volatile.base_image": "f80186e7f084a36bcabe2b7e82f107bc6a4be1a1731e34dccd558860ecbdd8d6"
		},
		"devices": {
			"p1": {
				"connect": "tcp:127.0.0.1:1234",
				"listen": "tcp:127.0.0.1:1235",
				"type": "proxy"
			},
			"p2": {
				"connect": "tcp:127.0.0.1:1234",
				"listen": "tcp:127.0.0.1:1236",
				"type": "proxy"
			},
			"root": {
				"path": "/",
				"pool": "zfs",
				"type": "disk"
			}
		},
		"ephemeral": false,
		"profiles": [
			"default"
		],
		"stateful": false,
		"description": "",
		"name": "c2",
		"source": {
			"type": "copy",
			"certificate": "",
			"base-image": "f80186e7f084a36bcabe2b7e82f107bc6a4be1a1731e34dccd558860ecbdd8d6",
			"source": "c1",
			"live": true
		},
		"instance_type": "",
		"type": "container"
	}

So we could can change the code above to detect if req.Devices is non-nil and skip populating devices from source if supplied and only use the devices supplied by client.

Sound good @stgraber?