Working for a company called AtCore for a department which develops a custom in-memory query based database for the Travel industry. It searches billions of travel related datum within a few 10’s of ms and returns queries live for frontend websites to display to end customers in real time.
The original product was vertically scaled… That’s been broken down into much small shards of data which are combined to give the original result. The system runs in LXD instances and dynamic instances are orchestrated using python scripts using the pylxd module