Thank you @stgraber . That indeed fixes it!
In the meantime, it might be good to mention this on the image server thread as the fact that I was using LXD 5.20 was the reason I had to migrate before Incus LTS comes out.
It was a bit confusing to still not be able to pull images afterward.