I’m not sure if I didn’t search enough but I can’t seem to make blkio limits work on Ceph RBD volumes.
I first tried to set limits in the container config using config edit however that did not appear to work.
I then created a new volume manually and attached it and edited the config again. This did not give any errors but it also is not limiting my blkio as far as I can tell (by using dd and inspecting the created blkio rules)
/sys/fs/cgroup/blkio/lxc.payload/NAME, I’d expect to see read_bps_device and write_bps_device populated in there.
It would also be interesting to restart the container and then look at /var/snap/lxd/common/lxd/logs/NAME/lxc.conf as I would also expect to see some lxc.cgroup lines in there to match.
I think your initial suspicion might be correct. I build lxd from source to get some debug lines in and found a few issues.
The initial problem is here it tries to check if the device path maran-test exists. Which it obviously does not since it’s not a real path but a storage volume. If I remove the check all-together it still won’t work properly, it does create a blkio cgroup record but it’s for the wrong major:minor identifier.
I’m not comfortable enough with the project yet to say how to solve this, I guess some translation from the Ceph rbd name is required, if you know a solution I would be happy to try and provide a PR for it.
Is there some way to apply the limits.write/read to the ‘root’ volume? I wonder if that would work but I couldn’t work out where to set the limits on the rootfs. (not sure the right terminology here but basically the volume that’s mounted on / inside the container)