Hi, I am trying to better understand how AMD EPYC CPU’s memory and register encryption behave under incus. I have several EPYC CPU servers under my control - 1st, 2nd and 3rd-gen. I can enable both SEV and SEV-ES in the bios and activate via grub entries - that all seems to work as expected, e.g. an EPYC ROME CPU with memory encryption enabled in the bios:
andrew@skywalker:~$ sudo dmesg | grep -i sev
[ 0.000000] Command line: BOOT_IMAGE=/vmlinuz-6.1.0-25-amd64 root=/dev/mapper/skywalker–vg-root ro mem_encrypt=on kvm_amd.sev=1 kvm_amd.sev_es=1
[ 0.016664] Kernel command line: BOOT_IMAGE=/vmlinuz-6.1.0-25-amd64 root=/dev/mapper/skywalker–vg-root ro mem_encrypt=on kvm_amd.sev=1 kvm_amd.sev_es=1
[ 100.804918] ccp 0000:4a:00.1: sev enabled
[ 100.856562] ccp 0000:4a:00.1: firmware: direct-loading firmware amd/amd_sev_fam17h_model31h.sbin
[ 100.882669] ccp 0000:4a:00.1: SEV firmware update successful
[ 100.955501] ccp 0000:4a:00.1: SEV API:0.24 build:20
[ 101.123351] SEV enabled (ASIDs 256 - 509)
[ 101.123366] SEV-ES enabled (ASIDs 1 - 255)
So, if I set security.sev=true in an incus vm (be it from iso or images:), I can boot it up and drop to a shell via the agent, I can see the expected:
root@SEV-ES:~# dmesg | grep -i sev
[ 0.067721] Memory Encryption Features active: AMD SEV
[ 1.813811] systemd[1]: Hostname set to SEV-ES.
[ 2.688072] SVM: KVM is unsupported when running as an SEV guest
So far so good. Now if I reset security.sev and “instead” enable security.sev.policy.es=true & restart the VM, I get the following (unexpected) output from the agent terminal:
andrew@skywalker:~$ incus config set SEV-ES security.sev=
andrew@skywalker:~$ incus config set SEV-ES security.sev.policy.es=true
andrew@skywalker:~$ incus start SEV-ES
andrew@skywalker:~$ incus exec SEV-ES bash
root@SEV-ES:~# dmesg | grep -i sev
[ 1.891571] systemd[1]: Hostname set to SEV-ES
Nothing about memory encryption?
So I tried also setting both security.sev=true and security.sev.policy.es=true and the VM fails to boot:
andrew@skywalker:~$ incus config set SEV-ES security.sev=true
andrew@skywalker:~$ incus config set SEV-ES security.sev.policy.es=true
andrew@skywalker:~$ incus start SEV-ES
Error: Failed setting reboot action: Failed setting actions: Monitor is disconnected
Try incus info --show-log SEV-ES for more info
andrew@skywalker:~$ incus info --show-log SEV-ES
Name: SEV-ES
Status: STOPPED
Type: virtual-machine
Architecture: x86_64
Created: 2024/10/05 09:44 EDT
Last Used: 2024/10/05 13:41 EDT
Log:
cpus are not resettable, terminating
Googling the above output gives me little additional information and I quickly get a little lost.
I believed that security.sev=true gives memory only encryption, and security.sev.policy.es=true gives both memory and register encryption (based on googling), and to me that seems logical. Is my understanding off or am I misinterpreting the (lack of) output?
There’s not much to find on this via google - an old lxd 5.13 release/video (useful, thanks Stephane!) and a little in the docs. Memory encryption itself seems very easy to implement via incus and it is certainly more than I will ever need, I am just a little surprised at the output I got via a vm with security.sev.policy.es=true (i.e. nothing evident inside the vm) and the fact that it failed to start with security.sev=true and security.sev.policy.es=true.
I did all of this with Debian 12 vm’s. Whether installed via iso or from images:, I got the same output.
Can anyone point me to a better understanding of this, and what if anything I am doing wrong; why does (i) security.sev.policy.se=true seem to turn memory (and register) encryption off inside the vm (ii) setting security.sev=true works as expected (i.e. memory only encryption), and (iii) setting both to true results in vm failing to start?
THANK YOU
Andrew
