I use rpi4 LXD cluster (5 nodes) for home lab.
I see strange container memory allocation behavior: container consumes much more memory if it starts first time on the LXD node. If container restarts or is implicitly stopped and then started memory consumption reduces significantly.
It’s not so visible for containers with small memory footprint, but may become a problem with more heavier containers. Memory allocation (for 1st time started containers) is stable for long time (weeks).
Below are several examples. I deployed 3 instances from 3 different images. You can see memory consumption after 1st container starts and then container after it’s restarted.
You can see the memory ration ratio for:
- alpine/3.16 (test-a) = 2.3 (5.75MiB/2.49MiB)
- alpine/3.16/cloud (test-a-c) = 9.8 (32.49MiB/3.32MiB)
- ubuntu/22.04 (test-u) = 3.1(100.79MiB/32.33MiB)
1st start after LXD host started
alpine/3.16
$ lxc info test-a
Name: test-a
Status: RUNNING
Type: container
Architecture: aarch64
Location: picl-01
PID: 1462
Created: 2022/11/15 21:40 PST
Last Used: 2022/11/15 21:46 PST
Resources:
Processes: 4
CPU usage:
CPU usage (in seconds): 3
Memory usage:
Memory (current): 5.75MiB
Memory (peak): 7.58MiB
Network usage:
lo:
Type: loopback
State: UP
MTU: 65536
Bytes received: 0B
Bytes sent: 0B
Packets received: 0
Packets sent: 0
IP addresses:
inet: 127.0.0.1/8 (local)
inet6: ::1/128 (local)
alpine/3.16/cloud
$ lxc info test-a-c
Name: test-a-c
Status: RUNNING
Type: container
Architecture: aarch64
Location: picl-01
PID: 1744
Created: 2022/11/15 21:40 PST
Last Used: 2022/11/15 21:46 PST
Resources:
Processes: 4
CPU usage:
CPU usage (in seconds): 12
Memory usage:
Memory (current): 32.49MiB
Memory (peak): 56.79MiB
Network usage:
lo:
Type: loopback
State: UP
MTU: 65536
Bytes received: 0B
Bytes sent: 0B
Packets received: 0
Packets sent: 0
IP addresses:
inet: 127.0.0.1/8 (local)
inet6: ::1/128 (local)
ubuntu/22.04
$ lxc info test-u
Name: test-u
Status: RUNNING
Type: container
Architecture: aarch64
Location: picl-01
PID: 2036
Created: 2022/11/15 21:37 PST
Last Used: 2022/11/15 21:46 PST
Resources:
Processes: 13
CPU usage:
CPU usage (in seconds): 4
Memory usage:
Memory (current): 100.79MiB
Memory (peak): 103.43MiB
Network usage:
lo:
Type: loopback
State: UP
MTU: 65536
Bytes received: 0B
Bytes sent: 0B
Packets received: 0
Packets sent: 0
IP addresses:
inet: 127.0.0.1/8 (local)
inet6: ::1/128 (local)
After containers’ restart
alpine/3.16
$ lxc info test-a
Name: test-a
Status: RUNNING
Type: container
Architecture: aarch64
Location: picl-01
PID: 3794
Created: 2022/11/15 21:40 PST
Last Used: 2022/11/15 21:49 PST
Resources:
Processes: 4
CPU usage:
CPU usage (in seconds): 3
Memory usage:
Memory (current): 2.49MiB
Memory (peak): 4.70MiB
Network usage:
lo:
Type: loopback
State: UP
MTU: 65536
Bytes received: 0B
Bytes sent: 0B
Packets received: 0
Packets sent: 0
IP addresses:
inet: 127.0.0.1/8 (local)
inet6: ::1/128 (local)
alpine/3.16/cloud
$ lxc info test-a-c
Name: test-a-c
Status: RUNNING
Type: container
Architecture: aarch64
Location: picl-01
PID: 3872
Created: 2022/11/15 21:40 PST
Last Used: 2022/11/15 21:49 PST
Resources:
Processes: 4
CPU usage:
CPU usage (in seconds): 10
Memory usage:
Memory (current): 3.32MiB
Memory (peak): 28.12MiB
Network usage:
lo:
Type: loopback
State: UP
MTU: 65536
Bytes received: 0B
Bytes sent: 0B
Packets received: 0
Packets sent: 0
IP addresses:
inet: 127.0.0.1/8 (local)
inet6: ::1/128 (local)
ubuntu/22.04
$ lxc info test-u
Name: test-u
Status: RUNNING
Type: container
Architecture: aarch64
Location: picl-01
PID: 3588
Created: 2022/11/15 21:37 PST
Last Used: 2022/11/15 21:49 PST
Resources:
Processes: 13
CPU usage:
CPU usage (in seconds): 3
Memory usage:
Memory (current): 32.33MiB
Memory (peak): 34.93MiB
Network usage:
lo:
Type: loopback
State: UP
MTU: 65536
Bytes received: 0B
Bytes sent: 0B
Packets received: 0
Packets sent: 0
IP addresses:
inet: 127.0.0.1/8 (local)
inet6: ::1/128 (local)
Steps to reproduce:
- Restart LXD/LXC host (optional)
- Deploy container (lxc init)
- Start container and check memory allocation using ‘lxc list -c Lm’ and 'lxc info ’
- Restart container and check memory allocation using ‘lxc list -c Lm’ and 'lxc info ’
- Repeat step 4 in some time
- Restart LXD/LXC host and repeat steps 3 - 5