Problem with Parallel Container Copying

Incus
1

Hi,

I’m facing an issue where incus copy node1:origin-container new-container fails when copying a container to multiple nodes simultaneously. The second copy fails with:

Error: Failed migration on source: A matching non-reusable operation has now succeeded

I’m trying to run the command on node2 & node3. origin-container on node1 is stopped.

Setup

  • Nodes aren’t in cluster mode
  • PKI trust between them
  • ZFS backend
  • Incus 6.12
  • Ubuntu noble, latest 24.04
  • Zabbly kernel 6.10.14-zabbly+
  • Zabbly zfs 2.2.5-amd64-202408071636-ubuntu24.04

What I Tried

  • Manually ran zfs send | zfs receive on the same dataset to multiple nodes - works.
  • Checked Incus code - it seems to block parallel copies.

Questions

  • Is this a bug in Incus, or an intentional design (e.g., to avoid race conditions)?
  • Is this a ZFS limitation (e.g., lock during zfs send), or Incus-specific?
  • Can this be bypassed?

The current behavior causes slower replica(clone/copy) creation (have to wait for the first to end). Any insight would help!

1 Like