I was exploring setting a storage size on my containers which led to confusion.
I thought this would be a “one step” process, but it turned out to be more scary than that.
My initial thought was to simply do, this:
lxc config device set mycontainer root size 7GB
Error: Device from profile(s) cannot be modified for individual instance. Override device or modify profile instead
Right, I’ve seen this thing from adding network limits, so I tried:
lxc config device override scptest root size 7GB
Error: No value found in "size"
Hmm, whats going on here? Googled some and found an article that suggested to first “add” the root device (which feels super un-intuitive and very scary and makes little sense to me (since the disk presumably already is there?)
lxc config device add mycontainer root disk pool=lxdhosts path=/
Device root added to mycontainer
… then setting the size: lxc config device set scptest root size 7GB
Now, my question is why does this process needs to be this two step process, whats the logic behind this?
It feels very not intuitive and even a little bit scary.
Doing “add” then “set” is the same as doing a single “override” command, where it will add a device by copying it from the profile(s) and then modifying it with any of the overridden keys.
When making config changes. Its the resulting config that is applied to the instance, not the specific command run.
What I mean by this is say if you have an instance that has a root disk device with a size of 5GiB coming from its profile(s). Then running “lxc config device add” and using the same settings as the current profile(s) and only changing the size to say 7GiB. So the only effective change to the resulting config is that the root size changed from 5 to 7. So the root disk would just be resized even though you ran an “add” command.
And now, because the root disk is defined in the instance itself rather than its profile(s) it is insulated from any future changes to the root disk made in those profiles as they won’t take effect.
Equally doing “lxc config device remove” on that imaginary instance root device wouldn’t result in the instance having no root disk, but rather the current profile(s) root disk settings would then take effect again (shrinking the size back to 5GiB).
The resulting config is what is shown by “lxc config show --expanded”
The device add, set, unset, override and remove commands are really just conveniences for manually doing “lxc config edit”.
From the docs:
Profiles store a set of configuration options. They can contain instance options, devices and device options.
You can apply any number of profiles to an instance. They are applied in the order they are specified, so the last profile to specify a specific key takes precedence. However, instance-specific configuration always overrides the configuration coming from the profiles.
And
Every device entry is identified by a name unique to the instance.
Devices from profiles are applied to the instance in the order in which the profiles are assigned to the instance. Devices defined directly in the instance configuration are applied last. At each stage, if a device with the same name already exists from an earlier stage, the whole device entry is overridden by the latest definition.
So I think of the config as layers, with profiles being applied in order first and then the instance config last. At each stage selective devices can override devices from previous layers.