Ah, Error: The 'default' profile cannot be deleted - #3 by stgraber seems to have helped. 
After doing this config change, I managed to delete the default storage:
foo@some-host:~$ sudo incus storage delete default
Storage pool default deleted
…and I could now run sudo incus admin init successfully. Very happy about this!
@stgraber - what’s the feeling with incus in general, are you supposed to use sudo incus if you want to operate on the “global” level (in the default project) and never just incus as your own user? If so, this seems a bit different in philosophy vs LXD, where you could just adduser <foo> lxd and be able to run things in the “global” context even as your own user.