Incus behind CloudFlare closing down due to CF timeout


I have an Incus environment running that is working well for my needs, except for one issue that I haven’t found a good solution for. My Incus server and client versions are 0.5.1, per incus version.

I have my domain protected by CloudFlare, and CloudFlare puts a 100-second timeout on websites that don’t seem to be doing anything. So, about 103 seconds into my Incus instance, if the user hasn’t typed anything into the terminal (but is just reading the instructions on the screen), the terminal shuts down and gives a “Reconnect” button.

The Reconnect button works fine. But I have a lot of instructional text on my site explaining the nature of what I’ve got running in the terminal, and I don’t want the user to have to keep clicking “Reconnect” every time they’ve spent 100 seconds reading. But I also don’t want to either turn off CloudFlare proxying for the site, nor pay for enterprise-level CloudFlare service (which lets you control that 100-second timeout).

The environment alive in my web-accessible terminal has no network, so it can’t “ping” the outside world to demonstrate that it’s alive. I’ve tried tacking a minute-interval REST call to the end of the xterm.js file to fetch …/api/1.0 against the Incus API, but that doesn’t help at all; the terminal still closes down after 100 seconds.

Has anyone else found a way around this? Thanks for any guidance!

It’s the websocket connection that’s likely tripping the timeout.

The Incus websocket API actually has a keepalive mechanism enabled which does keep it active, however a lot of proxy solution, quite possibly including Cloudflare’s will monitor the outer and inner parts of the websocket, so even though the outer part is sending a frequent keepalive, they’ll still shut down the session if nothing is sent on the inner connection. As the inner connection is directly attached to the terminal, you can’t exactly send stuff over it without impacting the user experience.

(Equivalent in haproxy would be the tunnel timeout which you usually want to be set to a very high value to handle things like incus exec without disconnections, whereas the normal connection timeouts can remain low)

Thank you for the reply, Stéphane.

What I think I’m reading in that is: “Nope! There’s not any good solution that anyone has come across yet, but you can maybe have the system print some output to the terminal every minute to keep it alive.”

Am I reading that correctly?

Yeah, you can try that, but also keep in mind that there are two websockets used for Incus exec sessions, one is the user-visible terminal, the other is a control connection, either closing will disconnect the whole thing and I’m not sure how kind Incus is to random stuff being sent on the control one :slight_smile:

Though you mention the xterm.js client, is that using incus-demo-server or are you doing your own thing? If incus-demo-server then I’m not sure that we have the keepalive bits set on it so maybe that would help in your case.