A homelab is a dedicated server at your premises where you install several self-hosted services, such as media server, DVR software, network ad-blocking, remote access, backups, etc. Typically you put these services in VMs.
Can you give a series of topics that would fit into the theme of using Incus for a homelab? The idea is to produce a set of tutorials, and put them together within the theme of a homelab. Most likely the list will be long; sort the list by importance. The most common software for a task, for example, pi-hole for network adblocking.
When I add the content from your reply, I’ll click on the of your reply.
Running Keycloak for incus authentication and authorization. How to integrate this with SSO for other services would also be nice.
Setting up incus to use Let’s Encrypt certificates
Integrate keycloak and let’s encrypt with incus canonical ui
Show how to use incus integration with ansible/terraform/open tofu
Set up incus to serve Prometheus/Grafana/Loki (metrics)
Set up a basic Ceph cluster and integration with incus. Can it integrate easily with microceph/microovn?
ZFS Management, how to deal with issues when something goes wrong.
Material for those with related prior experience
There are prospective users with related prior experience. In their case, it is easier to get them to adopt Incus if certain tutorials take into account their prior experience.
Use Incus instances for compiling/testing software. As a way to keep the host clean of unnecessary development packages or binaries. Also, mention --ephemeral. Show best practices on how to use/move the binaries from within the instance in a convenient way.
I’m not a LXC contributor (yet), but I would like to give my 2c. I hope its useful and I will try to update it regularly.
Nextcloud: Server for transparent file sync between multiple computers, with automatic file revision history (with possibility of rollback)
Coder (also known as Code Server): Visual Studio Code directly accessible through your browser and already installed on your development environment Linux server
AppFlowy or Affine Pro: high quality open source alternatives to Notion, allowing you to self host your own “Notion” server, with complete privacy and without limited plans and their costs
And the number one, all time winner : how to run Docker in a LXC. Worth updating the guidance given the changes in ZFS, and previously the information was a bit disperate.
I use Caddy as a reverse proxy that also takes care of TLS.
For DNS, I have used CoreDNS and Unbound. I am not sure if they are the best options for all people, but my use case is to expose local services to my local network. I am not blocking ads or trackers. I am still iterating on the topic.
I also have an Incus network that uses a bridge on the host of the incus service that bridges directly to my local network. This way, DNS and DHCP is taken care of by my router.
Also, consider Podman. I use it instead of Docker. It has a lot of benefits.
There are some interesting things which would be directly talked to incus which would be quite nice. Among these:
Running Keycloak for incus authentication and authorization. How to integrate this with SSO for other services would also be nice.
Setting up incus to use Let’s Encrypt certificates
Integrate keycloak and let’s encrypt with incus canonical ui
Show how to use incus integration with ansible/terraform/open tofu
Set up incus to serve Prometheus/Grafana/Loki
Set up a basic Ceph cluster and integration with incus. Can it integrate easily with microceph/microovn?
The points above are que nice since and most are que general to be used with many other services. And it’s stuff directly related to incus usage as well.
Other than that, for firewall, I’m using OpenWRT in an unprivileged container. Linux based, same interface as wireless routers/access points is que nice and no need to run in a VM.
I’ve seen some questions on using Docker inside containers. Some even concerning running Keycloak in Docker, for example.
Jellyfin is an awesome media player. It can be run with GPU passthrough in an unprivileged container and the configuration is super easy compared to proxmox, for example.
Some form of backup system for containers would be super nice.
A lot of people run these things in proxmox VMs. Showing how to run them in unprivileged containers is awesome. And incus ui is super important for these people as well.
There’s a very nice video from @stgraber on how to expose lxd services. It would be nice to remake that for incus and also give some extra information on setting up a bridge which he describes as one of the best methods, but did not show up how to set it up in at least one distro.
Podman indeed had benefits on Docker when running on the host machine. Once you’re isolated inside an incus container, those benefits are mostly irrelevant and Docker with compose and extensive community far outweighs it.
Also, considering that Incus may get at some point support for OCI images.
I’ll leave it on the list, it looks low priority, unless there is a compelling use-case.
I think I have updated the top post with all points, up to this point (pun not intended).
I use Podman at work. One nice thing about it is that you can easily wrap containers into a systemd service. You can also manage them in the same way as you would on Kubernetes. This makes it easy to migrate them to a production Kubernetes cluster. With that you don’t really need Docker compose.
But, yeah, that is all work related stuff. Maybe not so important for homelabs.
All the options I gave are self-hosted DNS. They mostly leverage Unbound under the hood but “the people” have spoken and they like a nice GUI. I’m afraid - and I say this as someone who actually owned the grasshopper book on DNS and BIND - the days of BIND in homelabs are over except for the diehards. All three options also do DHCP, by the way, and adblocking through dynamic lists.
Ah Docker, my old nemesis. The problem is exactly that there doesn’t seem to be just one post that tells you what to do. Next point is that the changes in ZFS 2.2 allow the use of overlay2 in a more performant way which has never been properly documented or Incus or LXD (that I have seen) :
eg Reddit - Dive into anything
Lots of folks say it can now be used and there is general carousing, but no one explains how to enable it properly in one shot. So many confusing options and pitfalls :
And why not cover shiftfs too :
I see some evangelists here pushing Podman, but unfortunately the whole homelab crowd is basically not interested. It’s a shame, but there it is. I would also expect that anyone wanting Podman would be capable of deploying it themselves, whereas the typical Docker crowd will require some handholding - even me, post the ZFS changes. For example, where should the the persisted data for the Docker containers be stored after you’ve configured ZFS delegates? Is it different for volumes and mounts (presumably not)?
Since Ceph was mentioned, I would like to suggest Linstor.
In my opinion, its strategy is simpler and more performant than Ceph in the specific context of distributed high-available storage for LXC containers and virtual machines.
I still think a native storage driver for Linstor would be interesting to get down the line.
There’s also been similar interest around Gluster.
I’m currently doing some work to support LVM on top of a shared block volume, that work is cleaning up a bunch of assumptions made by Incus around remote filesystems and should make it even easier for others to be added down the line.
That said, for anything we add support to, we’ll want there to be a solid community around it, stable releases, good security handling, …
That’s the one. And a comprehensive video should mention the second one as well.
On the podman side of things, it might not have been the correct place to even start that discussion. But both are run inside incus in the same way and talking about both in the same light has its merits. But with that discussion out of the way, focusing on Docker has a higher crowd interest in general.
Any of these are doable. Anything that is served through a Web browser is OK. With Incus, your instances
can have any type of network access (protected on a private bridge, appearing on the LAN, exposed to the Internet).
can have access to other instances (one instance is the service, another is the MySQL server, as if you have multiple servers)
IncusUI currently replicates well the command-line incus tool.
At some point in the future I expect that it would be possible to setup any of these through a UI like IncusUI. But how would we be able to get there? I think that the first stage would be to figure out the steps to do these manually.
at home we are already running all services on incus and even migrated the QNAP NAS to boot with ubuntu 22 and incus to act as a backup sink for incus and other zfs datasets. Main 24/7 server is a odroidm1 with mirrored ssds, consuming < 10 W, running armbian and incus on zfs with following containers:
samba4 active directory
samba4 fileservers
nextcloud
nginx as reverse proxy
tvheadend to stream and record from SAT using a DIGIBIT Twin
another system at hetzner having more memory and cpu power is running
Joplin (sync shared notes and webclippings)
Zimbra (groupware)
jitsi-meet (web meetings)
In our office we recently managed to successfully migrate all VMs from ESXi to incus containers among others:
Grav CMS
Redmine (project management with integrated tickets, wiki, git repos)
Jenkins
Artifactory
Alfresco DMS
EspoCRM
as VMs:
pfsense (any box connected directly to the internet should have something like a pfsense not to expose containers directly)
zulip chat
Win10 and Win11 Testsystems
On the company servers we also use custom, independent zfs pools to be mounted into the containers and we replicate the datasets via zrepl to other locations.
The migration of the windows VMs was an unexpectedly steep learning curve but now it’s really great to just clone a snapshot in seconds and delete it after testing. This is even much easier and faster than before in vmware! What I don’t want to miss anymore is the zfs backend under the hood.
I could help creating and managing documentation, how to set these systems up but we should focus on the incus specifics since we don’t want to replicate existing how to’s and maintain them.
What I’m am still looking for are tools and how to’s to create and manage images via simplestreams. If we could create such a howto, it may promote the exchange of incus container images for specific use cases!
of your reply.
For example, where should the the persisted data for the Docker containers be stored after you’ve configured ZFS delegates? Is it different for volumes and mounts (presumably not)?