FARM: Disposable Forgejo Actions runners with Incus VMs

I’ve been working on a small project called FARM:

FARM runs Forgejo Actions jobs in disposable Incus virtual machines.

The basic flow is:

queued job → FARM → Incus VM → one-job runner → cleanup

Each VM registers one ephemeral Forgejo Actions runner, accepts at most one job, and is deleted afterwards together with its runner registration.

FARM supports:

  • repository, organization, user, and global runner pools
  • queue-based scaling
  • optional idle capacity
  • startup, concurrency, idle, and lifetime limits
  • cleanup of stale runners and VMs
  • a small read-only operations interface

I built it because I wanted my Forgejo Actions jobs to run in disposable VMs instead of keeping long-running runners around. Incus was already managing the VMs in my infrastructure, so it seemed like a good fit.

FARM manages Incus virtual machines only. It does not create or manage containers.

Feedback is welcome, especially from people running CI workloads on Incus.

I am also running Incus and Forgejo in my homelab. This is a great idea to save on resources. All the best.

Given how close this is in concept and possibly in design to GARM and GARM already supports both Github and Gitea, did you look into what it would have taken to extend GARM to support Forgejo? Or is that something that upstream GARM isn’t interested in?

Why VMs, not unprivileged containers as well?

I have not really looked into GARM before starting FARM.

From what I understand now, GARM mainly uses a push model based on webhooks. Forgejo currently seems to be missing some of the API/webhook functionality needed for that approach.

FARM uses a polling model instead. The FARM controller periodically asks the Forgejo API whether workflow jobs are waiting and then scales the Incus runner VMs accordingly.

And to be fair, part of the motivation was also simply that I wanted a small project to learn how to build something on top of Incus :slight_smile:

Mainly because Forgejo Actions can execute third-party code inside the runner environment, for example actions used to check out code or set up tooling.

I don’t want to trust that code more than necessary. Unprivileged containers provide good isolation, but they still share the host kernel and container escapes are possible.

For CI workloads, I therefore prefer the stronger isolation boundary of a VM, especially since the VM is disposable and removed after a single job.

Today I Learnt that GARM is listed at Third party tools and integrations - Incus documentation

GARM supports two modes, matching what Github itself supports. One is the webhook mechanism, the other is a long poll based mechanism that they use for runner scale sets.

So there’s precedent for there being different ways to learn about needed runners.

Sure, but you could also have an outer wrapper VM, inside which the containers are run. You’d get a lower setup/teardown overhead, much better RAM pooling, and still have reasonable container-to-container isolation.

Looks like there is a new plugin capability in Forgejo Runners that could be interesting: Forgejo Runner v13.1.0 is available — Forgejo