Exciting things to come for incus power users

I’ve been developing my own cloud platform that I’m calling “PiF” (Public Internet Fork) for the last few years with incus at the core. Many times I have come up against snags in development only to find that incus has shipped a new feature or fix matching my exact need at the same time. So, even though I’ve been mostly building in stealth - It feels like my efforts are in line with the needs of others. This makes me confident that my overall solution will be well received by the community.

That said, I still wanted to start sharing some of the things I’ve been working on in case anyone out there might be interested in becoming an early beta tester. The following utilize Incus’ powerful core functionality, incorporates many bleeding edge features, and elegantly wraps them together into one cohesive solution. I’m providing a broad description of a few things I have going so far:

S3 Native backups:
Automatically establishes a backup file on a remote S3 bucket for every launched container (via Restic). Stopping a container immediately backs it up to a remote S3 bucket. Starting a container first pulls the latest backup from S3. Stop a container on host A and start it on Host B. Your containers live on S3 and run on any host you want when needed.

Incus Billing System:
Show an hourly billing rate column in the “incus list” command. Define the monthly cost for your server, total available ram/cpu/ssd, and your profit margin - this gets reflected to an incus user as an hourly cost that changes based on the cpu/ram/ssd they provision to each instance. Every incus project gets its own “total hourly billing rate” that adds up all running instances together and a ledger file that defines their account balance/usages.

Incus multi-node mesh:
Automatically installs and configures headscale/tailscale (community version) onto every launched container. Ping containers by name and reach them regardless of them being local or on a remote node. “Incus list” shows containers from all hosts. Sidesteps incus’s native clustering functionality for a more scalable solution that still feels native

Production SSL, Auth, and reverse proxy:
Creates a wildcard SSL via lets encrypt, a Tinyauth server, and dedicated nginx instance for every incus project created. Every launched container automatically creates an nginx config for the container that points to container:8000. It also handles auth redirect as well. So when you launch a container named “containerA” and run an app on port 8000 - it’s automatically available at - https://containerA.project1.domain.com. Visiting the url first brings you to auth login and redirects you to the url after validation. Deleting a container deletes the associated nginx config.

If any of these features sound interesting, let me know and I can make it a priority to share demo videos and more info. If you have been working on similar things and need a hand, feel free to reach out. Thanks!

Release early, release often …

That’s what people do who wanna scare away users with broken stuff that doesn’t upgrade without breaking changes.

hey @AboveSound

The features you describe sound great! Do you plan to release it as open source or something else?

I’m mostly interested in S3 backup and recovery.

I’m the Maintainer of https://docs.incus-compose.org/ there I go a different way with proxying, Caddy for incus-compose · Issue #138 · lxc/incus-compose · GitHub

Maybe we can work together on parts, would be nice.

Bit of a long reply here but in short; yes and yes.

I plan for everything mentioned above besides the billing solution to be released as individual open-source (MIT License) components that can be installed alongside any incus installation.

My end-goal is to make enterprise grade networking, compute, and data storage so incredibly accessible that they become standardized commodities with standardized pricing and SLA’s; In turn, allowing them to be traded in one open marketplace made up of a large collection of providers. Think AWS, digital ocean, ect competing for the same customers in the same place on one unified standard - the playing field is leveled when it comes to pure compute, storage, and bandwidth - leaving SLA’s and unique product experiences as the only differentiator left to win customers. The cost and experience of buying/using compute will have been standardized across the board.

With enough host adoption, what I’m building has the ability to disrupt the biz model of current hyperecalers and eventually allows consumers to cut out many levels of middlemen involved in products like Dropbox or square space. Today, Incus on its own accomplishes much of this. The vision for PiF is removing some of the last barriers of entry - removing the need for sysadmins or developers in the loop at all in many use cases.

But, yes, there will be a commercial offering designed around a lot of this functionality between now and then.

My background is in digital transformation for SMB’s, so I’ve developed much of this platform around that. Cutting out as many of the hard costs that businesses face (hosting, SaaS, and IT) as possible is the initial value proposition I’m leading with. But I very much value open source (obviously) and want to make meaningful contributions to incus and the other projects I’m utilizing along the way (JuiceFS, Rclone, to name a few).

I still have to eat at the same time :slight_smile: