Is it possible to use lxc query to stream a container to stdout?

LXD
1

I want to get around the backup API limitation, as it requires to save the image twice as a file before archiving, is not quite optimized for speed.

Let’s assume we have two LXD servers available, h1 and h2. There is one container c1. lxc can be run on both of them locally as well as on the server in charge of backing up, which does not have LXD but instead has h1 and h2 remotes.

Containers are already automatically copied with --refresh from h1 to h2 and given the -backup suffix. They are stopped. Backups are to be done from h2.

I want to do something along the following lines:

  1. Set up container
    lxc query h2:/1.0/containers/c1-backup -X POST --wait --raw -d ‘{“name”: “c2-backup-stream”, “migration”: true, “live”: false}’

  2. Stream it
    lxc query h2:/1.0/containers -X POST --wait --raw -d ‘{“name”: “h2-backup-stream”, “architecture”: “x86_64”, “profiles”: [“default”], “ephemeral”: false, “source”: {“type”: “migration”, “mode”: “push”, “operation”: “https://h2:8443/1.0/operations/109ccbe5-0a7d-4b9d-8083-32d01df56b32”, “container_only”: true, “secrets”: {“control”: “my-secret-string”, “fs”: “my third secret”}}}’ > backupfile

I am not quite there yet and would appreciate any insight.

Where do I find the secrets or are they given for the session?
What are the final bits to get that stream in a format that I can lxc import later (or equivalent query command if format differs)?

2

From memory they are returned to the request that starts the operation

Doing my best to interpret the LXD source code (I could be wrong, im not an LXD dev) I dont think this technique isn’t going to work because LXD appears to handle it instance to instance (it doesnt go via the client) & it either

  • handles the transfers behind the scenes using rsync or the storage tools (zfs, ceph, CRIU etc)
    • Some setup & container details (presumably anything stored in the DB) is transferred over the API but not the actual filesystem
  • It uses some kind of web sockets (I dunno, CRIU? they threw them in there, probably header info, but I cant tell what they do, its been a long day)

You might be able todo the following;

  • Build an small program to accept the LXD host API’s calls to initiate an rsync transfer (container migration)
    • Im not sure how the auth works between hosts (or if there even is any - its probably all todo with control secrets you will have to dig)
  • Store the header info about container exchanged over the API
  • Initiate the rsync to your FS
  • combine header info & rsync’d FS into one “file”

To Restore you would just do the inverse

Of course the above only works for transfers that would qualify for rsync transferal, if you use zfs or some other storage device you will have to modify as applies

3

Thanks @turtle0x1!

Container streams can be routed via the client with mode: relay. That mode is not documented for LXD rather it is in lxc so I am guessing the query is the same at LXD, and the feature is in lxc which just runs two querys, one pull and one push.

For the secrets, the first query does not return anything - i.e. it does not return at all. Depending on mode, I could actually expect that command to return the container.

@stgraber mentions in this issue that the API should work for the purpose. So I am mostly trying to figure out if lxc query, rather than a purpose-built tool, can be used, how, and what the resulting format will be.

CRIU is for stateful (running) containers, and as such is optional, so it is likely easiest to start without and add it if it works later.

4

At least lxc operation show works to get the secrets

5

As per the discussion on the issue thread, this indeed requires a purpose-built tool. Case closed.

6

I had the feeling that would be the case!