An updated version of this tutorial is available at https://discourse.ubuntu.com/t/building-custom-lxd-binaries-for-side-loading-into-an-existing-snap-installation/37327
This can be useful if you want to test a feature in a specific environment you already have set up, like a cluster or one with multiple projects, profiles, …, etc. You can place your newly compiled binaries in that environment as .debug files, restart the LXD daemon for the system to pick up these special binaries, test and then simply erase the .debug files and restart the daemon to revert back to normal.
Note that if you test a code change that modifies your database, you will have to revert your database manually when you go back to the previous state or wait for that feature to make it to the official store before you can remove your .debug files.
1. Use a builder container with the same version we are using to create official releases. This will ensure you have the same dependencies as those used in the published snaps. The table below lists the base containers we use in all channels.
|Snap Channel||Base Container|
Here my container is called “builder”:
lxc launch images:ubuntu/22.04 builder lxc shell builder
2. Inside the container, from instructions provided here
apt update apt -y upgrade
Then install the required packages except golang which we will install as a snap later.
apt -y install acl attr autoconf automake dnsmasq-base git libacl1-dev libcap-dev liblxc1 liblxc-dev libsqlite3-dev libtool libudev-dev liblz4-dev libuv1-dev make pkg-config rsync squashfs-tools tar tcl xz-utils ebtables apt -y install lvm2 thin-provisioning-tools apt -y install btrfs-progs apt -y install curl gettext jq sqlite3 socat bind9-dnsutils apt -y install shellcheck
apt -y install snapd snap install go --classic
git clone https://github.com/lxc/lxd
Build raft and dqlite
As per make’s instructions
export CGO_CFLAGS="-I/root/go/deps/raft/include/ -I/root/go/deps/dqlite/include/" export CGO_LDFLAGS="-L/root/go/deps/raft/.libs -L/root/go/deps/dqlite/.libs/" export LD_LIBRARY_PATH="/root/go/deps/raft/.libs/:/root/go/deps/dqlite/.libs/" export CGO_LDFLAGS_ALLOW="(-Wl,-wrap,pthread_create)|(-Wl,-z,now)"
Build lxd, lxc, … etc
Get binaries from your builder and add .debug extension
lxc file pull builder/root/go/bin/lxd lxd.debug lxc file pull builder/root/go/bin/lxc lxc.debug
Push binaries in your current working environment
sudo mv lx[cd].debug /var/snap/lxd/common
Restart LXD daemon
sudo systemctl reload snap.lxd.daemon
Check using top that lxd.debug is running
sudo rm /var/snap/lxd/common/lx[cd].debug sudo systemctl reload snap.lxd.daemon
Check using top that the regular lxd is running