I’m using LXD 3.0.0 on ChromeOS (their Crostini project). If I try to do a ‘lxd pull container/path/file.tgz /outside/path/’ and file.tgz is bigger than 4gb, it fails with a “No space left on device” regardless of how much space is actually at /outside/path/ (in my case about 400gb). As a workaround I’ve been splitting the file inside the container into 1gb chunks, lxd pulling them out, and then catting them together outside the container – which takes up twice the disk space (the pieces plus the reconstituted file) which is evidence that I really do have enough disk space.
What is the correct forum for reporting this? Should I go to https://github.com/lxc/lxd/issues or try and work through Chromium OS’s bug reporting process?
So it’s unfortunately kinda expected in this case and part of a security measure on the LXD daemon side. Effectively, we can’t just go read the path you’re providing through the filesystem as the container may be doing something nasty to try and trick us into reading a file outside of the container (using symlinks or bind-mounts).
So what LXD does instead is open a temporary file in /tmp of the LXD server, then attach to the container, drop privileges to the user inside the container, open the file you requested and copy the content to the temporary file it opened earlier. This effectively makes for a safe data copy from within the container to its host. The LXD daemon then reads that temporary file and serves it over the REST API which is what lxc file pull downloads and write in the destination path you provided. At the end of the transfer, the temporary file is deleted.
What this means though is that the largest file you can transfer through the LXD APi will be as large as the free space you have in /tmp of the LXD host.