Hi there,
I’m using LXD Clustering to run my project using haproxy as load balancer, running projects on other containers using the <container_name>.lxd:80 for the backend server on haproxy.cfg. The thing is that since CRIU is not supported, i have to stop my containers using lxc stop and then move them to another host in order to manage load. But this procedure produces a lot of down time for me, is there any recommended method to perform host migration of containers without producing downtime, thanks.
At the moment I think your best option would be to use ceph as storage backend for the rootfs of your containers. At that point lxc move should be basically instantaneous. You still have the stop/start overhead though.
I’ve tried that but ceph storage doesn’t allow for storage quotas and limits right, is there a way to do that. Although it would be fine even without.
And just a thought is it possible to share a volume across two containers, so that i can register them both for one service on the backend of HAProxy and perform a stop/start on one of them at a time so that there’ll always be one to handle the processes, this will make the service highly available.
Do you mean share a custom volume? Can you describe a bit better what your use case would be? (like what would the two containers do with the shared data)
The only way I can think of would be with ceph, if I understand correctly ceph does not support mapping/mounting the same rbd block device on two different nodes. Perhaps cephfs would allow that, and there might be support for cephfs in the coming future.
I’m afraid I can’t help you more, probably @stgraber or @brauner will have more to say.
Well let me be straight with you, i’m running a Laravel App with PostgreSQL as my database backend. And when the load on the containers get too high i would like to automatically create a new node join it to the cluster and move the containers to it without causing any downtime, so if i had two containers sharing the same block storage i can make that the default storage location for these services and have two containers running on this service writing and reading from it. So that when i perform a migration using lxc stop/move/start the other container will be still processing my requests and writing to the storage volume. using the HAProxy load balancer. Is a solution like this possible, would i really need CEPHFS for this?
Okay, one thing that is still not clear to me: let’s say you have 2 HAProxy backends of a certain Laravel app, each one running in its own container. As far as Laravel alone is concerned those two containers are stateless. Don’t you need a third container running the PostgreSQL process, with the other two connecting to it?
Or is our setup such that you run 2 PostgreSQL processes each one inside the same container as the Laveral stateless up? I don’t see how that’s possible, so I’m still confused about your setup and what exactly you want to achieve.
Well it’s like this i’ll be using ElephantSQL or something like that (DBaaS) for the PostgreSQL part and would be using the containers to execute php threads. But the filesystem should remain the same since there is a file upload section for the app, such as profile pictures, gallery and etc, that goes unto the public folder, i understand that my solution might not be entirely possible, but my goal here is to establish a lxc based highly available laravel system, do you have any recommendations besides this, i am all ears, thanks.
With approach you’re taking, the only way I see this possible is by using a distributed file system (e.g. ceph), so the file upload folder can be shared across containers residing on different network hosts. That would allow you to implement your HA strategy (have 2 containers, shut one done and move it, shut the other down and move it too).
AFAIU there’s no support for that in LXD itself yet (but cephfs has been considered as possible feature), so you’d have to implement it out of bad.
Another option would be to store those files in the DBaaS directly, but I have no idea if the Laravel framework has support for that (e.g. having a SQL backend as opposed to a file system backend). I’d also suggest to check what’s the recommended solution that Laravel suggest for this use case, if there’s one.
No i have been considering using AWS S3, Backblaze b2 instead for this purpose. But still support for CephFS seems like an interesting feature, considering that most applications would benefit from this. If i’m correct this is similar to what is known as “Persistent Volumes” in kubernetes, right?
I’d be a bit surprised if PVs in K8S can be shared between pods, so I’d say if different. You’ll have to check the docs, please let me know if I’m mistaken
Right, again I don’t think it’s possible to share the volume across pods (which would be effectively what you need, because pods might be on different hosts), but I’m no expert either.
Ok so guess we’ll have to wait for Stephane’s reply, but i would like to request one last thing if that’s ok from you, since Ceph doesn’t allow for storage quotas, is there an alternative solution to limiting my storage quota on individual containers, maybe a linux app or something like that?
That configuration looks like it’d only ever work if both containers are on the same host, the diagram seems to indicate that it’s what’s going on too.
CEPH does support quotas in that it will get you a block device of the size you’ve set in LXD.
Note that depending on the filesystem you put on top of it, it may not be shrinkable (xfs) and that growing/shrinking operations will usually require a container reboot.
