I had a container which I exported as tarball and then imported as an image on another server.
Now whenever I launch a container with this imported Image it takes anywhere between 5-10 minutes to launch with majority of the time going to Image retrieval part even after it already being imported and ready to use.
Is there a possible way to reduce this time?
What storage backend is being used?
On one node I am using BTRFS and on another ZFS. But the time duration is similar in both.
One thing common in both setup is that I create a new storage pool for every new container that I launch.
That’ll be the issue then, the storage pool concept is designed to “pool” together a storage resource for multiple instances. Not to be used on a 1:1 basis for each instance.
The first time an instance is created on a storage pool, if the storage driver supports the concept of “optimised storage” (basically snapshots), then LXD will unpack the compressed instance image into a fresh “image” volume, which can take quite some time.
Then the initial instance, and each subsequent instance will be created as a snapshot from that initial image volume, which is normally much quicker than unpacking the image file every time.
This also has benefits from a storage perspective, as each instance will only take up the differences from the original image.
If you’re creating a new storage pool for each instance, then every instance will have to create the initial image volume on the storage pool, which is why you’re seeing this take so long every time. You’ll also lose the storage efficiency benefits.
5-10minutes seems quite long for a container, even for the first time, but I’m assuming this custom image is quite large.