By way of maas 3.0 stable, I’m creating LXD virtual machines with multiple disks.
For reference:
maas vm-host compose 11 storage=‘10(default),11(default),12(default)’
It appears LXD is generating the incorrect scsi-ids/BootIndex for the disks defined by maas.
The boot priority only affects well, the boot priority, it doesn’t affect the order in which we attach the device to the bus.
For bus ordering, LXD simply uses alphabetical ordering. As your root disk goes after disk-1 and disk-2 in alphabetical order, it’s correctly getting sdc.
In general, I’d very very strongly recommend against making any assumptions on the /dev/sdX name. The probing order isn’t guaranteed by Linux so even if the drives were in the expected order on the bus, a probing delay at boot can result in a different order.
Instead you really should use one of the stable /dev/disk entries which will work regardless of what name the kernel may give the device at boot time.
To keep everyone in the loop, I’ve logged the maas bug:
FYI @tomp there’s a note in there about the scsi id for the disk devices. I understand why you have it match the bootindex, but it is slightly strange when a nic has bootindex=0, then you don’t have any disks with scsi-id=0.
IMO it would be nice to decouple the two especially as a modification to the boot priority would currently renumber the devices, not that we should be relying on such a thing (i.e. don’t mount fs by /dev/sdX), but I don’t think an end user would have the expectation that the two are linked.
Also @tomp if you have the background on why boot.priority:0 was set on disk devices in maas (my guess is the NIC bootindex bug I’ve referenced), I’d appreciate your confirmation on the LP bug above.
IMO it would be nice to decouple the two especially as a modification to the boot priority would currently renumber the devices
Do you mean that we should decouple the NICs from influencing (causing a ‘hole’) in the SCSI-ID sequence? As opposed to decoupling the bootindex from the SCSI-ID?
@tomp sorry, yes I’m contradicting myself a little there!
Yes, decoupling the NICs so that we don’t have a hole in the SCSI-ID sequence would be ideal. This just appears to be an artifact of the LXD implementation. Of course the scsi-id could still be based on the bootindex, it could just be re-numbered so that the first disk’s id is always 0, regardless of how many other boot devices may be added before (e.g. iso, nics).
To speak to my first point, having the boot sequence boot scsi-ids, 0, 1, 2 etc. in sequence is an elegant property that makes the boot sequence easier to understand and I think is a reasonable limitation to have compared to raw qemu configuration, where you could have an out of order boot sequence.