Behaviour of cluster member evacuation with attached storage volumes

LXD
1

In the following scenario:

  • LXD version 5.0.0
  • 3-nodes LXD cluster
  • separate storage pool for each node
  • c01 lives on node lxd-node-03 has an attached storage volume called vol01
  • c01 has cluster.evacuate = migrate 
$ lxc cluster evacuate lxd-node03 --force
Error: Failed to migrate instance: Copy instance operation failed: Failed instance creation: Failed creating instance record: Failed initialising instance: Failed to load device to add "vol01": Failed loading custom volume: Storage pool volume not found

Should evacuate also migrate the attached storage volumes to a container? Or the only option is to use a distributed storage pool?

2

Any ideas @monstermunchkin @stgraber ?

Sounds like a potential ordering issue, or should the instance have refused to migrate?

3

You’ve set cluster.evacuate to migrate which tells LXD to migrate the instance even if it believes it will fail.

Leaving it to the default value of auto would have had LXD determine that the instance is dependent on a local volume and instead performed a stop rather than migrate.

1 Like
4

Thanks for the reply.

This was my expectation that setting cluster.evacuate to migrate would migrate the container as a whole, also with the attached storage volumes.

Is this something possible?

I’m planning to try distributed storage pools nevertheless, in which storage volumes are not moved around.

5

It’s not currently possible and is made tricky by the fact that storage volumes can be shared with several instances, making such evacuation problematic/impossible in some situations.

6

Ok thanks