How to enable SWAP for LXD Containers on Ubuntu 22.04.2?

LXD
1

Hey everyone! I need some help configuring SWAP for LXD Containers on Ubuntu 22.04.2 with Linux 5.19 and LXD 5.14. Is this related to cgroupv2 only now with the latest Ubuntu/Linux versions? If it is, could someone guide me on the proper and easy way to get this done nowadays?

I’m curious about the current state of SWAP accounting in LXD/LXC with Ubuntu 22.04.2 and CGroupV2. I’ve come across various methods, but unfortunately, I haven’t been able to make it work on a fresh installation.

To give you an idea, here’s an example of my LXD containers configuration:

config:
  limits.cpu: "16"
  limits.memory: 64GB
  limits.memory.enforce: hard
  limits.memory.swap: "false"

The issue is that when I check the container, there doesn’t seem to be any swap space available. It shows a size of zero.

I’ve tried using the swapaccount Linux option and experimenting with different values for the other options, but so far, no luck in getting the swap working in the LXD containers.

I’m curious why it’s not as straightforward as something like this (for example):

limits.memory.swap: 32G

Any insights or suggestions would be greatly appreciated!

References:

Thanks in advance, cheers!

2

Any thoughts on this one @amikhalitsyn ?

3

Hey guys, any tips about this?

This problem is the only blocker for me to move forward with a nice LXD project!

4

Hi,

What is the reason you need to see the swap available inside the container?

I think this document may help to explain the reasons why swap allocation in containers is not straightforward:

https://github.com/lxc/lxcfs#swap-handling

But generally speaking one should increase the limits.memory to the amount of overall memory required and then let the OS manage swapping centrally. However you can use the other settings you mentioned to indicate to the OS the propensity to swapping it should have.

1 Like
5

We are also hitting this swap issue and it is very much a blocking issue for us. I will layout the scenario as much as possible.

Env:

  • Ubuntu 22.04 LXD host and containers
  • Using ubuntu 6.1.x mainline kernel
  • Hardware: Intel 13900k with 128G ram
  • Swap is enabled on host and set to 128G on fast nvme.

Usage:

  • We have a container for remote ide (Jetbrain gateway) per developer
  • Each container has access to a gpu which can also do training/inference.

Issue:

  • Container cannot see any swap regardless of toggle mentioned here or in other lxd posts/docs. Host is also showing zero swap usage from containers.
  • As such, we are getting linux process.fork() faults due to memory on the containers where there are some physical ram left and full 128g of swap available.
  • Whatever the developers are running, ide, python scripts, etc are using lots of ram and/or also allocating lots of virtual memory.

Without proper swap, a fully functional host with 100% swap available is brought to a halt unless we kill some processes to relief memory pressure. There are only 4 memory slots on this machine so we can’t expand further.

6

Using LXC with cgroups v2 instead of LXD, at least the “swap” line is being printed when running the “free” command inside a container.

But it always has a value of 0 (on “free” output), despite setting a non zero value with lxc-cgroup -n my-container memory.swap.high X, where X could be anything you want, like 128G.

At least I can confirm cgroups v2 is “aware” of it, because the command lxc-cgroup -n my-container memory.swap.high (to check the current defined value) returns the correct one (non-zero value).