How to set the container to can use the maximum resources of the physical machine

My physical machine’s resources information are below.
CPU: total 16 cores, total 32 threads.
memory: 32 GB.
hard drive: 1TB usable.

I want to make my lxd container ubuntu/18.04 can use the maximum resources of the physical machine. How can I do that?

By default, no CPU or memory limits are applied so that part is covered.
For storage, it depends on the storage driver, but generally we similarly don’t apply space limits unless configured to do so.

Where are the lxc container such as my-ubuntu all data stored?
In my physical machine, the “/” directory only has 37GB usable, and an additional hard drive has 1.1TB usable. I worry about that if lxc container data are stored below the “/” directory instead of the hard drive, so the space is not enough to use soon.

Here is command df -h result in my physical machine:

(base) admin1@admin1-S2600CO:~ $ df -h
filesystem        capacity  used  usable used% mount
udev             16G     0   16G    0% /dev
tmpfs           3.2G  9.8M  3.2G    1% /run
/dev/sdb1       198G  151G   37G   81% /
tmpfs            16G  364K   16G    1% /dev/shm
tmpfs           5.0M     0  5.0M    0% /run/lock
tmpfs            16G     0   16G    0% /sys/fs/cgroup
/dev/loop2       70M   70M     0  100% /snap/lxd/14954
/dev/loop1       28M   28M     0  100% /snap/snapd/7264
/dev/loop0       55M   55M     0  100% /snap/core18/1754
tmpfs           1.0M     0  1.0M    0% /var/snap/lxd/common/ns
tmpfs           3.2G   72K  3.2G    1% /run/user/1000
/dev/sda        3.6T  2.4T  1.1T   70% /media/admin1/bf1debbd-17d1-4f71-a00a-37f049fb597e

Do I need to do some configuration to let the lxc container data being stored in /dev/sda hard drive?

lxc storage show default

(base) admin1@admin1-S2600CO:~ $ lxc storage show default
  size: 39GB
  source: /var/snap/lxd/common/lxd/disks/default.img
  zfs.pool_name: default
description: ""
name: default
driver: zfs
- /1.0/containers/ubt-1
- /1.0/images/872233202ebb2d16a7a0e33ec5e08183e990c5007c156b5932acd656f8875861
- /1.0/profiles/default
status: Created
- none

Ok, so yeah, you should change that.

If you have a dedicated disk or partition, you should have LXD use that rather than setup loop based storage on your root partition.

On a clean system, you can tell LXD that during lxd init.

In your case, you can retrofit it with something like:

lxc delete ubt-1
lxc profile device remove default root
lxc storage delete default
lxc storage create default zfs source=/dev/sdX
lxc profile device add default root disk path=/ pool=default

Replacing /dev/sdX with a full drive or partition to use for storage.

I have other files in /dev/sda . So the commands you recommend will not delete or destroy my files in /dev/sda, right?
And when I finish typing those commands, I can just type lxc launch local:ubuntu18-tang ubt-1, without other parameters, right?

Oh, I see, it’s mounted at /media/admin1/…

It would have been best to partition that drive, splitting it between host storage and LXD but that’s probably a bit late for that…

So one option would be to use the dir storage backend, that will let you use your current drive as it is, but container creation will be slow, snapshots will be extremely slow and wasteful and you won’t be able to set quotas. If none of those matter to you, then you can do:

lxc storage create default dir source=/media/admin1/bf1debbd-17d1-4f71-a00a-37f049fb597e/lxd to have LXD use a directory on it for container storage.

So you mean that every time I am going to do lxd init on a machine, I should prepare a clean hard drive partition first? For my situation now,I only have 1 hard drive connecting to my machine, and there has been already 70% used. So I need to copy all those files to another mobile hard drive temporarily, and then I make the original hard drive into 2 partitions. One partition is going to store my old files back,and the other partition is for lxd pool. Is this procedure correct? And another question,is it enough to be only one partition for lxd pool in my machine? I mean maybe I will have several images and many containers in my one machine. So all those containers use 1 partition,is there any problem?

And I think maybe it’s risky for me to do partition with my only one hard drive. I am afraid of that there will be problem after the data migration. So I wonder is there technique in lxd/lxc just like data volume in docker tech? At present, my purpose of using lxc container is to do data analyzing. Thus in the container I will deploy many environments and software and personal codes. I think these things could be not greater than 30GB. But the input files and intermediate files,I guess they will occupy at least 100GB,and even 200GB maybe. So can I set up a sharing directory outside the container,just for storing those big data. If you know docker data volume tech, maybe you can understand my idea.

If you use other storage backends (like zfs and btrfs) in my understanding you can limit how big the storage size will be.

Update: Don’t try the below (striked out) method right now, it caused the author of this issue problems.
I solved the problem of to less space on host root, by adding a link.
So I simply linked the lxd-folder in /var to a folder on my second drive.
So “/var/snap/lxd/” is pointing to “/media/seconddrive/lxd_data2/lxd” for example.

As far as I remember I just copied all content to the new folder.

So you do the following:
1. Stop LXD.
2. copy the whole LXD-folder into the new location (see also Docs: Backup)
3. Then delete the old LXD-folder (should be empty now anyway).
4. Then apply the link.
5. Start LXD again.

Maybe @stgraber could say whether this method is ok :wink:.

Interesting boy! I will try it.


  • Maybe you could consider adding an option to set the folderpath of LXD in lxd-init?

Because some distros suggest to partition the harddrive during installation, and the resulting root-partition (or even a seperate var-partititon) is often too small.

  • Also a method to split the lxd folder would be convenient, so that:
    Some storage pools could be on one harddrive and other storage pools could be on a different harddrive.

    Update2: Now I am confused.
    Lets put it into a question:

  • So it is not possible to create a storage pool (with something different than “dir”-backend) on a second harddrive?
    Without creating a new partition on it?

  • Is this the right variable for setting the lxd folder:
    "LXD_DIR" - The LXD data directory
    And how do I set it?
    I guess via snap:
    sudo snap set lxd LXD_DIR=/media/example/lxd ?

This might be useful:

There is some error when I move the directory to other place.

(base) admin1@admin1-S2600CO:~ $ sudo mv /var/snap/lxd /media/admin1/work/tangmaomao/project/linux_container/lxc_storage_source
[sudo] admin1 password: 
mv: access '/var/snap/lxd/common/ns/shmounts' error: invalid argument
mv: access '/var/snap/lxd/common/ns/mntns' error: invalid argument

Then I rename the /var/snap/lxd to /var/snap/lxd_legacy_20200509 ,
create the symbolic link file as /var/snap/lxd .

Then I test the lxc command. Error appears.

(base) admin1@admin1-S2600CO:/var/snap $ lxc list
Error: Get "http://unix.socket/1.0": dial unix /var/snap/lxd/common/lxd/unix.socket: connect: connection refused

Well I told you it could be insecure.

Backup-Docs say:
is the folder for full backup.
So you would only need to move that folder I guess.

Did you stop and restart LXD?

Try the following (to revert what you did):

  1. Stop LXD (also daemon)
  2. Remove the link (only the link, not the target folder!)
  3. rename /var/snap/lxd_legacy_20200509 back to /var/snap/lxd
  4. Try to move all data back to /var/snap/lxd
  5. Restart LXD

This should revert everything.

If LXD then starts sucessful again, you can just leave it as it is.

If you are in doubt, wait for a better answer, maybe by @stgraber or some of the other devs.

Sry for your problems, but I warned you :slightly_frowning_face: .

I will look at it again tomorrow.

Got it. Thank you first

How did you solve it?
In case someone experiences the same problems.

I am trying. Please wait.

In any case, make a seperate backup/copy of
Or if it was already copied, from:
lxd/common/lxd (inside your copied path).

This way you can save the images, containers, configs etc.
Until we find a solution.