HOWTO use lxd-p2c

I had great difficulty in getting lxd-p2c to work so I have written some notes on the process which will hopefully help someone get up and running faster. Feel free to improve these notes, put them on the wiki or whatever.

LXD 3.x introduced lxd-p2c, a go program to ease the process of migrating an existing Linux server (most likely physical, but it could be another type of VM) into an LXD container. The LXD devs have decided not to bundle lxd-p2c in either their Ubuntu packages nor the LXD snap so it has to be manually built from the LXD github repo. It is expected that the source machine is likely to not have a version of go new enough to build lxd-p2c in which case it can be built on a newer machine and then the static build transferred over.

Building lxd-p2c

You should run the exact same version of lxd-p2c on the machine being containerised as is running on the lxd server, otherwise you may get rsync errors preventing the transfer starting.

If the machine to be containerised is running a 64 bit Linux OS and your lxd server is running LXD 3.8, for example, you would build lxd-p2c like so:

~$ go get github.com/lxc/lxd/lxd-p2c
~$ cd go/src/github.com/lxc/lxd
~/go/src/github.com/lxc/lxd$ git checkout lxd-3.8
~/go/src/github.com/lxc/lxd$ go get github.com/lxc/lxd/lxd-p2c

If the build completes successfully, the lxd-p2c binary should be under ~/go/bin .

If the machine to be containerised is running a 32 bit Linux OS and you’re bullding lxd-p2c on a 64 bit OS, you would do the same but the last command would be replaced with:

$ env GOOS=linux GOARCH=386 go get github.com/lxc/lxd/lxd-p2c

Preparing to run lxd-p2c

lxd-p2c depends upon TLS to work and so it will fail if the clocks of the machine being containerised and the LXD server are out of sync so its best to ensure NTP is enabled on both beforehand.

When you run lxd-p2c, it will request you enter an admin password before the transfer begins. This is the password created when you are configuring the bridge interface to be used by lxd when you run lxd init on the lxd server to do the initial configuration. In case you have forgot what that password was, it can be reset by running:

$ sudo lxc config set core.trust_password some-password

It is recommended to stop the active services (eg web servers, databases etc) before running lxd-p2c. Failure to do so could result in data loss or corruption. At the bare minimum you should at least stop the logging services before running lxd-p2c, which under recent releases of Ubuntu can be achieved by running:

$ sudo systemctl disable rsyslog
$ sudo systemctl stop rsyslog
$ sudo systemctl mask systemd-journald
$ sudo systemctl stop systemd-journald

Running lxd-p2c

Simply stopping rsyslog isn’t enough because systemd automatically restarts rsyslog when data is written to the system log socket.Running lxd-p2cIf you only have one filesystem / mount point to containerise (the root fs) and you wanted to create a container called test, then a simple invocation of lx2-p2c would look like:

$ sudo ./lxd-p2c https://lxdserver:8443 test /

Doing the same but omitting /var/cache by passing an rsync option would look like:

$ sudo ./lxd-p2c https://lxdserver:8443 test / --rsync-args="--exclude=/var/cache"
5 Likes

Excellent notes! Do you have any suggestions on how to handle this issue I’m encountering trying to migrate a 32 bit CentOS 5.11 system to LXD?

[root@localhost /]# ./lxd-p2c https://10.0.2.5:8443 centos5-test /

Error: Failed to setup the source: Failed to make / read-only: device or resource busy

I’ve tried shutting down various services and mounting / ro, but that won’t work either since /tmp is not on a separate partition.

Hi Brian

No, I’ve not encountered that error when using lxd-p2c but please let us know if you work out what causes it or how to fix it.

Thank you for these notes, great help.

Some additional notes:

  1. After the transfer was finished my container was having troubles, services not starting. It relates to this discussion: https://github.com/lxc/lxd/issues/4573 So you may need to clean out your /etc/fstab which I did and after reboot everything was working nicely.

  2. My original ubuntu 18 server had some networking setup, hardcoded in /etc/network/interfaces and was not using netplan. The container would not be assigned an IP
 So I removed the interfaces file and setup netplan as discussed here: https://vitux.com/how-to-configure-networking-with-netplan-on-ubuntu/ and rebooted the container and now networking is good and it is assigned an IP automatically.

  3. Before doing the transfer i did a service --status-all to see what services I had going and stopped most to quiet things down.

The lxd-p2c tool works great and made a difficult task much much easier. Cheers!

1 Like

I had the same fstab issue. Your comment helped me out. Thanks.

1 Like

I am unable to edit the OP to fix the build instructions but the process is now:

# snap install go --classic
# add-apt-repository ppa:dqlite/stable
# apt install -y build-essential acl libacl1-dev libcap-dev liblxc1 liblxc-dev libsqlite3-dev libudev-dev libuv1-dev libdqlite-dev
# go get github.com/lxc/lxd/lxd-p2c

The lxd-p2c binary should be ~/go/bin

@stgraber is there a good link for the pre-built images for lxd-p2c?

There is no pre-built version of lxd-p2c. It’s included with the LXD snap, but unlike lxd-migrate, it’s not possible to build lxd-p2c statically so we can’t just distribute a binary for it.

So, I have LXD installed via the Snap. However, I do not have a lxd-p2c command in the CLI. I also do not see a “lxc lxd-p2c”. Accessing lxd-p2c does not seem to be documented. I’d like to cover this utility on my channel. How exactly do I access it via the snap or is there any documentation, because I have not been able to locate it.

Oh, oops, sorry I got confused with lxc-to-lxd.

lxd-p2c is indeed the static one but that particular tool got deprecated and replaced by lxd-migrate.
You’ll find the static lxd-migrate builds attached to each release on Github.

Thanks. However, in googling lxd-migrate, I am not seeing any documentation and I can’t find any static images attached to releases on github, nor am I seeing it installed as a part of the lxd snap. Is this lxd-migrate tool the utility that can take a physical machine and create a lxd container from it? I sure would like to read about this, but the documentation is elusive even to a master “googler”.

You can find the latest edge build of lxd-migrate by using the Linux asset zip file produced with each successful push into the main branch, e.g.

https://github.com/canonical/lxd/actions?query=event%3Apush+branch%3Amain+is%3Asuccess

Then click into the most recent one and at the bottom you’ll find an Artifacts section.

Download the Linux zip file and lxd-migrate is in there.

You’ll also find it on the Assets section at the bottom of each release
https://github.com/canonical/lxd/releases

However note that lxd-migrate wasn’t working properly in LXD 5.15, so either wait until LXD 5.16 or use it from an edge build.

See also https://documentation.ubuntu.com/lxd/en/latest/howto/import_machines_to_instances/

@tomp After two days of digging and following through your links I found only the source code. I try to compile the “go” code and it is a giant fail each time. I was hoping for a utility as straight forward as vmware’s p2v utility. Clearly this is not that. For almost a year now I have been planning to add a video to my channel on how to use this tool, but it is subtlety hidden and it seems like a massive puzzle to use. In following your bunny trail, the best I have found is pull requests and the “main” branch only seems to give me either source code that won’t compile for me or developer comments. Honestly your links were very helpful for me to see all the work that has been done, but no binaries are to be found after two days of searching every possible link.

So if you browse here https://github.com/canonical/lxd/actions/runs/5608033153 (this just happens to be the latest build at the time of posting and was top of the list on the https://github.com/canonical/lxd/actions?query=event%3Apush+branch%3Amain+is%3Asuccess link I sent before), then scroll to the bottom there is a link to download the Linux artifacts as a ZIP file.

Inside that ZIP file is a static Linux binary called lxd-migrate, and the docs to use it are here:

https://documentation.ubuntu.com/lxd/en/latest/howto/import_machines_to_instances/

We also have an improvement to the docs on this page in the works here:

I will also be tagging the LXD 5.16 release very soon which will be found here https://github.com/canonical/lxd/releases and will similarly have downloads available for a statically built lxd-migrate binary for Linux. So alternatively look out for that.

I tried to migrate a test KVM Virtual Machine to a LXD container. The migrate completed without error. I used lxd-migrate 5.16. The resulting container had a broken network. I figured that could be fixed with Netplan. I performed:

systemctl enable systemd-networkd.service
systemctl start systemd-networkd.service
systemctl enable systemd-resolved.service
systemctl start systemd-resolved.service

Then I built an appropriate Netplan and applied it.
This did get me an IP address on my LAN after I applied the appropriate bridge profile.
I was able to ping it from the LAN.
Something is not right though.
I had both openssh-server and RDP server installed and they both rejected connections.
I connected to the shell with “lxc shell” and it appears that something is not right with systemctl in general. One thing I found was that /etc/fstab had entries for mounting root and the efi partition from the source VM. I ended up deleting the fstab and restarting the container.
That didn’t fix things either. I am really not sure what is broken exactly. The VM I test migrated was an ubuntu 22.04 desktop.

Any ideas?

Please can you post over at https://discourse.ubuntu.com/c/lxd/support/149 with the transcript of what you entered for lxd-migrate and lxc config show (instance) --expanded. Thanks