We as an Iaas/Paas provider (small scale) tried to offer LXD to clients over the past years as a replacement or completion to classic VM’s , projects were welcome as sort of isolated units per client.
Not sure the experience will be of interest here.
Definitely interesting to us.
@sdeziel and myself run a small-ish cloud based on LXD for a non-profit and its partners. Only a few hundred instances at this point though but running on top of OVN and Ceph, all on recent AMD Epyc with a good chunk of CPU, RAM and storage on each compute node.
It’s been working quite well so far, creating projects for each individual user or group of users and using restrictions to give them a limited mount of resources and access to the cluster.
Great.
Before starting with specifics, would like to state the obvious, as reason and motivation for choosing LXD.
Resources
The fact that instance size is a fraction of any comparable virtual-machine, makes it agile, saving time and resources for deployments, backups, cloning, moving around.
It enables a much higher density of isolated applications on same hardware.
Concept of usage
An average vm, due to high cost, is usually packed with all sorts of services in one. Where with lxd you can afford dedicated application instances, each either facing public or only one exposed to public and many workers serving it from background through private network/bridge. This is an enhancement of availability and security.
IPv4 vs. IPv6
Another advantage came up surprisingly. The slow adoption and sometimes rejection of IP6 implementation.
IPv4 is exhausted and expensive, yet any technique seems to be welcomed to squeeze the very last bit of it, rather jumping to IP6.
Considering the global reach of IP6 well below 40% (20 y since Introduction) let presume IP4 will still hang around for a while.
VM’s work best with a dedicated IP4 (common cloud practice), with LXD you can practically split a unique IP4 to cover almost 65000 services in background by NAT or even more by proxying.
Was about creating an overview of Hardware and Infrastructure involved in IaaS, unfortunately work load came up.
Will try to update soon.
This signature dominating LXD throughout design.
It is perfect to use as single end-user, own Infrastructure or just through it as subsidized resources at a community for free use…
But as for commercial use, we learnt it hard way that in the end we will always subsidize.
The concept of Limitations, restrictions on everything and on many levels is the exact opposite of a commercial use.
Once customer in Shop, the least thing you want to do is limiting him!
Instead you need robust measurements tools of resources usage.
It is quite a challenge, despite projects, a lot is still shared and hidden everywhere.
So we ended up, customer paying for 24K referenced ZFS container and we paid 300M Image which that reference was based on, for snapshots, for backups, for backup snapshots, traffic, IOPS. (Sure there is a config for ZFS size measurement, not helped much)
Fetching all those information over multiple servers requires endless API queries per customer/project/container and the project has its footprint everywhere spread over storage pools/networks.
When you imagine some clients require billing by hourly usage, than the severity of accounting task gets clear.
Still struggling with collecting customer movements and usage as as a persistent continues logging of events per Project into a file or DB.
Unfortunately (understandably) most -known to me- third-party GUI or management going sync with LXD -for private use only- concept.