Container publish too slow?


(Shantur Rathore) #1

Hi,

I am using lxd cluster with ceph. I have seen that whenever I do a “lxc publish” it takes ages to publish the container.
I always thought lxc publish would use the quick copy of rbd to create the image. Is this not the case?

Ps: The container just takes 21G on disk

~# rbd diff lxd/container_android-runner | awk ‘{ SUM += $2 } END { print SUM/1024/1024 " MB" }’
21007.4 MB

Cheers,
Shantur


(Stéphane Graber) #2

It’s not. All LXD images must exist as compressed tarballs in /var/lib/lxd/images, so when you publish a container, a tarball is generated from it, it’s then compressed and hashed to produce the image fingerprint. Then the first time you create a container from that image, the tarball will be imported into your CEPH storage pool as a RBD volume.

Note that tar doesn’t handle sparse files well, so if you have a 10GB sparse file in your container, the pre-compression tarball will contain this as 10GB of data regardless of how much of the sparse file is used.


(Trystan) #3

@stgraber

Any thoughts on the OCI spec for images instead of tarballs?

Not sure if it would be feasible but a lot of the imaging and snapshotting features of ceph really hit a wall when LXD exports/publishes.


(Fridolin Somers) #4

I use gzip compression but with the lowest level 1.

To allow that, I have created a small script /usr/local/bin/lgzip :
#!/bin/bash
gzip -1 $@

And defined in lxd config :
images.compression_algorithm: lgzip

Hope it helps you.


(Shantur Rathore) #5

@stgraber

I think the whole process of
container ceph rbd -> tarball -> hash -> /var/lib/lxd/images might be taking all that time. It would become very fast whenever if you guys think of implementing the image store in ceph as discussed earlier in another post.
For now, is it possible to show some kind of progress while it is being published?
That would really help in knowing if its going on or stuck.


(Jonathan Hatch) #6

I agree with this. Some sort of status would be nice. I can’t tell if I’m sitting here waiting for nothing or what.