I`am struggling now the second day by trying to install incusOS from the iso image on Hardware without TPM and on VM with or without vTPM
I tried nearly all settings on the image download page and tried to follow this documentation:
Hardware issue:
first boot => Secureboot keys are installed
second boot from iso: server hangs without any useful error message:
I tried really all settings during the iso image configuration: Withouth TPM including Secure Boot, without everything, with wipe disk and without wipe disk. RH2288 V3 and RH2488V5 (Both without TPM Module)
Then I switched to VMware to create a VM with TPM Module and without.
Same game but different. VMware doesn`t recognis the iso image as a boot device:
“CD Rom not found”
I`am working now over 20 years with several hypervisors and Linux OS but never came to a point where I had to open a topic in a community forum to be able to install an operating system
I really want to use this but when I need to spend days to get it installed, how long would it take to get it stable for a production environment to replace some fully automated VMware clusters?
Answers to possible questions:
yes, I followed the instructions, yes the VM has TPM 2.0 and Secureboot and advanced settings enabled and I uploaded the .der files
both hardware servers have a raid configured but “wipe disk” was enabled during image creation
no I cannot use the flash tool because we are air gapped and its a pain to build this within an air gapped environment
On the hardware side, you managed to get through the installation fine, but then got stuck there on first boot? If so, that suggests something funky going on with systemd-repart as that’s what’s likely to be running at that point of the boot process. Could be something weird going on with the storage maybe? Another option would normally be a TPM issue but since you’re using the software TPM that shouldn’t really be possible.
On the VMware side, that error is very weird and isn’t something I’ve personally ever run into with the many VMware deployments we’ve done lately. I’d probably start by triple checking that ISO looks correct (should be 3.2GB large and be the ISO not the USB image), obviously need to make sure that the ISO is attached to the VMware drive and that the drive is connected to the VM. I’m sure you already checked but it’s very easy to miss the “Connect at boot” checkbox
Anyway, if the ISO looks good, maybe share the VMware config here so I or one of the folks who are doing a lot of IncusOS on VMware deployments (@stephendotcarter, @cylebas) may help.
Hi Stephane,
thanks for your reply. I downloaded nearly 10 different images (always iso for installation purposes). I tried within our company via proxy and also private without any proxy and ssl injection.
I uploaded the iso image to the datastores and also tried to mount them via client devices (vmware console). This behavior is really strange and blieve me, when I say, I work with VMware daily since Version 3.5 I will check tomorrow further.
Probably also triple check that the VM is configured to use UEFI and not regular BIOS as you’d be getting that error on regular BIOS I think (ISO is GPT formatted).
I have seen this behavior “CD-ROM not found” when the Secure Boot keys were either not placed in the VM’s folder alongside the .vmx file or not properly defined (exact name) in the Advanced Parameters before the VM was powered on for the first time.
Moving or modifying the Secure Boot keys in the VM’s folder, editing the Advanced Parameters, or recreating the virtual disk after the initial power-on will not help.
You may need to create a new VM, do not power it on after creation, and ensure that all required steps like adding the TPM module, placing the Secure Boot keys in the VM folder, configuring the Advanced Parameters, and enabling Secure Boot are completed before the first power-on.
thanks for testing. I put the files to the folder where the vmx file is located but I will try and double check / recreate everything again- just to be sure.
Status HW Installation: unchanged. I have no starting point of debugging and it takes a lot of time to boot the HW several times to mount the remote iso image. I guess its an issue with the internal usb devices of the Huawei hardware for the OS FS but Its hard to know without any logs or debug output.
I have no empty newer hardware with TPM available at the moment but it would be nice to get it running for performance testing and alua settings and all this stuff.
Right, you mentioned the system is completely offline. Installing in that kind of environment is slightly more involved.
IncusOS itself doesn’t actually include Incus or any “primary” application out of the box.
That gets downloaded on first boot which is failing in your case as you’re in an offline environment.
In offline environments you need to use a secondary drive that includes the full update from the image server so it can install from there. We actually have some built-in logic in the image customizer but it’s not currently exposed in the web UI. I’ll see if I can easily surface that so you can grab the second ISO and get that machine fully installed (you’ll need to attach the second ISO and then reboot to have it picked up).
For your physical server. If at all possible, try to have someone plug a USB stick with the image to eliminate any virtual media weirdness. We’ve generally seen virtual media work very well with IncusOS, even in rather extreme environments (100kB/s and 300ms latency type thing) but all implementations are different and we’ve never dealt with Huawei.
Thanks for your response. I downloaded now both iso images.
I came out that the V3 Server have SD Cards installed, so this will for sure not work. So I switched to the V5 Server with m2 SCSI devices.
I downloaded a new image from the customizer where I also added the ip, wipe disk and device type = scsi.
I also removed all secure boot keys (factory reset) and documented the keys.
We normally don`t have issues with iBMC images and I really do not want to run with an usb stick to the DC.
Currently the V5 server is not booting from the iso image. Will do some further testing but it feels similar to the VM where the iso image just won`t work without the keys uploaded first and tpm enabled. The response time to the management interface is below 1ms
I will download some other images with different configuration settings to find out what the issue is.
You can always turn off Secure Boot and TPM (if present) on the server and try to boot.
You’ll get an error from IncusOS pretty early on in boot, but it should at least boot that far.
If it doesn’t, then the issue isn’t related to Secure Boot or TPM, but may be related to your BMC’s virtual media choking on a complex GPT partitioned ISO image. We’ve definitely seen servers do that before (ASUS).
If that’s the case, check if the server has separate “virtual cd-rom” and “virtual usb”. For “virtual cdrom” you’d want the ISO image (2048 bytes sectors), for “virtual usb” you’d want the USB image (512 bytes sectors).
If you only have one kind of virtual media on this platform, you could still give it a shot with the other image type, in case the BMC does some weird magic internally.
I wanna give an update for the community on this topic.
I was not able to boot any image without Secureboot enabled on any Huawei Server V3, V5, V7 or a Virtual Machine and I tested it several annoying days.
Very frustrating when no image and no option was working to boot the iso image.
The VM is running with Secureboot and TPM without a problem (well documented).
How I solved it on xFusion / Huawei V7 with your support:
mount the IncusOS iso image and boot into the Bios
Administer Secureboot, Enroll KEK and DB (PK not required), select the certificate files (keys/KEK and keys/DB/2025 & 2026 files) from the ISO Image and save.
Save and Boot to start the installer. Reboot with second stable-image iso
Include the PFX file to the browser and ~/.config/incus/ to use the incus client as well
Hint: you have to load it to the personal certificate store and not the local cert store
I was never able to get it running without TPM or Secureboot, even with several images I downloaded with different options. I guess it is the same reason why you can`t boot the iso image to a VM without uploading the secureboot certificates to the VM folder, where the .vmx file is. At the end it should also work without secureboot certificates if you disable the option in the image
Incus looks good and the api seems to be really fast. I will have several more questions, but this topic is solved.
