Error on cluster formation when trying MicroCloud

I want to try MicroCloud and I’ve prepared 3 VMs. I’ve installed microcloud, lxd and microceph on them and start microcloud but I got this error:

root@mc01:~# microcloud init
Please choose the address MicroCloud will be listening on [default=]: 
Scanning for eligible servers...
Press enter to end scanning for servers
 Found "mc03" at ""
 Found "mc02" at ""

Ending scan
Initializing a new cluster
 Local MicroCloud is ready
 Local LXD is ready
 Local MicroCeph is ready
Error: Failed to issue LXD token for peer "mc02": Failed converting token operation to join token: Operation expiresAt is type <nil> not string

Have anyone experienced the same error? Any tips?

Any ideas @stgraber ?

Seems like the token isn’t being added?

That’s odd.

@masnax any ideas?

Seems to be a LXD version issue from looking at the error. The token is missing the expiresAt key, which is internally expected when validating a join token. This key was added to LXD’s join tokens about 2 months ago.

The error shouldn’t happen if you can refresh the LXD snap.

I am having what I believe might be a similar issue. At first I thought apparmor might be getting in the way, disabled, no joy.

My situation:

3 fresh VM’s of Ubuntu 22.04 running in LXD on Ubuntu 22.04. Each VM has a disk (/dev/sdb) exposed to them, each 150 gigs. install all snaps, refresh all snaps - everything is up to date. /etc/hosts has hostnames of remotes and locals in it.

Fire up microcloud init, it finds the other VM’s, finds the storage, wipes the storage. lxd is initialized on all three nodes, ceph is initialized on the first node (where I am running microcloud init), but then failure with -

Error: Peer "03" failed to join the cluster: Failed to update cluster status of services: Failed to join "MicroCloud" cluster: Failed to join cluster with the given join token

After this error, I am left with:
lxd, microcephd, microcloudd running on 01, 02, and 03
ceph mon, mgr, and mds running on 01, (NOT running on 02, 03)
storage is partitioned on 01, (not partitioned on 02, 03)

01 thinks it is clustered and the database leader in lxd
02,03 are not part of a cluster

Where do I start to look for the problem?