[LXD] Bidirectional vsock interface for VMs

Specifications
#1
Project LXD
Status Draft
Author(s) @monstermunchkin
Approver(s) @stgraber @tomp
Release 5.3
Internal ID LX015

Abstract

Have LXD itself listen for VM sockets connections.

Rationale

The current state is that the lxd-agent listens on the VM socket inside of the VM once it’s up and running. This allows for requests from LXD to the VM. There’s currently no way of the VM talking to LXD.

If LXD were to listen on the vsock, the VM could send requests to it. This would e.g. enable the VM to notify LXD that it’s ready i.e. done booting.

Specification

Design

The LXD server will listen on the vsock. Clients (VMs) can connect to the vsock using their client certificate, and make requests through it. The server will be listening on port 1024 + CID (Context ID). The CID is unique for each VM, and can be retrieved from both the server and the client.

The current /dev/lxd handler in VMs will be replaced by this interface. The instance-data file containing various information can be removed as it will not be needed anymore.

API changes

No API changes.

CLI changes

No CLI changes.

Database changes

No database changes.

Upgrade handling

No upgrade handling.

Further information

TBD

1 Like
#2

Hmm, no, that’d conflict with a potential VM. Instead we should use 0 (hypervisor) or 2 (host) for this and have one listener per daemon, we can easily identify what instance is talking to us from the certificate they use.

We’ll also need to handle the case where we’re running multiple LXD daemons so the host can’t use the default port of 8443 (as will be the case in Jenkins) as well as the case where LXD is itself running inside of a VM and so likely can’t use 2 as the address but will likely have to use the address of the VM (its CID).

So we effectively need a way to communicate and update the host CID and port to the agent in the guest. We may be able to do that over the existing serial port and its ring buffer.