CPU burst : Windows VM

Savin_Savin · September 16, 2020, 1:17pm

Hello,
I have a windows VM running on a remote linux host. I need to know if there any possibility that the windows VM can use more CPU core ( for a shot period) than it reserved?
Is there any way to allow CPU burst to the VM? Can you help me on this?

Thank you!!

Savin_Savin · September 22, 2020, 3:53pm

also Is possible to allow memory overcommitting with LXD virtual machines?

stgraber · September 22, 2020, 5:02pm

For CPU you have two options, neither of which can be changed after start.

You just set a number of CPU cores and let the scheduler on the host figure out where to run it at any given time and deal with other tasks running on there.
You set a specific set of host CPUs to expose to the VM, this then gives you a bit more control over VM allocation on physical CPU but avoids the scheduler on the host so if you pin to busy cores, you’ll get something slower.

For memory, you must select how much memory you want the VM to get.
This should handle over-committing in that the memory will only be consumed from the host when used in the VM, so you can have a bunch of VMs totalling more than the total memory of your host and have them run fine so long as they don’t all use what they were configured with.

Or you use hugepages in which case the memory is fully dedicated to the VM.

Savin_Savin · September 23, 2020, 1:22pm

Is it possible to allow memory overcommitting on windows VMs?
I have a windows VM with the memory limit 8GB. When the windows VM is in running state it used complete physical memory allocated to the VM from the host, not only use what its needed. The screenshot shows the usage:-

memory

stgraber · September 23, 2020, 1:43pm

Right, in this case, Windows used all the memory and never gave any back to the host.
That behavior is really up to Windows, Linux can’t free memory that it doesn’t know has been freeed.

Make sure you have all the relevant drivers installed on Windows, especially the balloon driver which is used to release memory back to the host when not in use in the VM.

Savin_Savin · September 23, 2020, 4:17pm

@stgraber yeah, balloon driver and all required drivers are installed.
but cannot update these PCI drivers from virtio drivers.
Is because of this the windows VM reserve the complete physical memory allocated to VM?

drivers

stgraber · September 23, 2020, 4:59pm

On top of the driver, there’s also a balloon service (BLNSVR) which must be installed and running for memory to be given back to the host

Savin_Savin · September 24, 2020, 12:46pm

hello @stgraber
Installed the Balloon Service and now its running in the windows VM, But it still reserve the physical memory allocated to the VM from the host
Is there any way to get back memory to host when not in use in the VM?

Savin_Savin · September 28, 2020, 10:25am

All the required drivers and services are installed in the windows VM, especially Balloon driver and Balloon service. But still the windows VM reserve the same amount of memory limit from the host. Can anyone help me on this?

stgraber · September 29, 2020, 6:34pm

@tomp is working on live memory reconfiguration which you’ll soon be able to use to force the balloon to inflate inside the VM, releasing some amount of memory to the host.

That should work for cases where the guest doesn’t already voluntarily release memory back to the host through itself causing the balloon to inflate.

tomp · September 30, 2020, 9:38am

Pull request is here:

Savin_Savin · September 30, 2020, 10:31am

okay thanks !! @stgraber @tomp waiting for your updates

Savin_Savin · October 14, 2020, 9:09am

Hi @tomp with lxd 4.6, also the windows VM reserves the memory from the host. How long will take for the update to fix this?

tomp · October 14, 2020, 9:16am

I believe @stgraber will be doing the next release in the next couple of days.

When that is released you should be able to reduce the memory using lxc config set <instance> limits.memory=<size> and regrow it to original boot size limit (if needed) using the same command with a larger value.

Savin_Savin · October 14, 2020, 9:31am

thanks for the faster response @tomp and also waiting for updates

stgraber · October 14, 2020, 1:10pm

Worth noting that this got backported into the snap already, so the 4.6 stable snap can do it already.

Savin_Savin · October 23, 2020, 11:24am

Hello,
I have updated to lxd 4.7 and when doing the live shrinking and re-grow of memory limits to the windows virtual machines facing some issues.
when I decrease the memory size, it makes only change to the lxd config and the changes does not reflect inside the windows VM. It requires a restart to make the changes.
After the restart of the windows VM, I cannot increase the memory size. When trying to increase getting the below error

Error: Failed updating memory limit: Cannot increase memory size beyond boot time size when VM is running

Also I need to know if memory overcommitting is possible in windows VM?
When I start the windows VM, it reserve the complete physical memory allocated to the VM. But I can see that LInux VMs uses only what its needed. The screenshot of memory usage is attached below.

can you help me on this ?
Thank you !

Savin_Savin · October 26, 2020, 10:27am

@stgraber @tomp can you have a look at this?

stgraber · October 26, 2020, 2:59pm

LXD is working as designed.

stgraber@castiana:~$ ps aux | grep win10
lxd      2354294  133 26.1 5293592 4249144 ?     Sl   10:19  10:46 /snap/lxd/18007/bin/qemu-system-x86_64 -S -name win10 -uuid 6b701e0d-8bf9-496f-85ad-386e67f5c47f -daemonize -cpu host -nographic -serial chardev:console -nodefaults -no-reboot -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=deny,resourcecontrol=deny -readconfig /var/snap/lxd/common/lxd/logs/win10/qemu.conf -pidfile /var/snap/lxd/common/lxd/logs/win10/qemu.pid -D /var/snap/lxd/common/lxd/logs/win10/qemu.log -chroot /var/snap/lxd/common/lxd/virtual-machines/win10 -smbios type=2,manufacturer=Canonical Ltd.,product=LXD -runas lxd
stgraber 2379700  4.1  0.1 1316400 20428 pts/0   Sl+  10:21   0:15 /var/snap/lxd/common/lxc.debug console win10 --type=vga

stgraber@castiana:~$ lxc config set win10 limits.memory 2GiB
Error: Failed updating memory limit: Failed setting memory to 2147483648 bytes (currently 3013607424 bytes) as it was taking too long
stgraber@castiana:~$ lxc config set win10 limits.memory 2GiB

stgraber@castiana:~$ ps aux | grep win10
lxd      2354294  129 13.3 5293592 2168876 ?     Sl   10:19  11:07 /snap/lxd/18007/bin/qemu-system-x86_64 -S -name win10 -uuid 6b701e0d-8bf9-496f-85ad-386e67f5c47f -daemonize -cpu host -nographic -serial chardev:console -nodefaults -no-reboot -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=deny,resourcecontrol=deny -readconfig /var/snap/lxd/common/lxd/logs/win10/qemu.conf -pidfile /var/snap/lxd/common/lxd/logs/win10/qemu.pid -D /var/snap/lxd/common/lxd/logs/win10/qemu.log -chroot /var/snap/lxd/common/lxd/virtual-machines/win10 -smbios type=2,manufacturer=Canonical Ltd.,product=LXD -runas lxd
stgraber 2379700  4.0  0.1 1316400 20428 pts/0   Sl+  10:21   0:15 /var/snap/lxd/common/lxc.debug console win10 --type=vga

stgraber@castiana:~$ lxc config set win10 limits.memory 4GiB

stgraber@castiana:~$ ps aux | grep win10
lxd      2354294  128 26.1 5293592 4246812 ?     Sl   10:19  11:14 /snap/lxd/18007/bin/qemu-system-x86_64 -S -name win10 -uuid 6b701e0d-8bf9-496f-85ad-386e67f5c47f -daemonize -cpu host -nographic -serial chardev:console -nodefaults -no-reboot -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=deny,resourcecontrol=deny -readconfig /var/snap/lxd/common/lxd/logs/win10/qemu.conf -pidfile /var/snap/lxd/common/lxd/logs/win10/qemu.pid -D /var/snap/lxd/common/lxd/logs/win10/qemu.log -chroot /var/snap/lxd/common/lxd/virtual-machines/win10 -smbios type=2,manufacturer=Canonical Ltd.,product=LXD -runas lxd
stgraber 2379700  3.9  0.1 1316400 20428 pts/0   Sl+  10:21   0:16 /var/snap/lxd/common/lxc.debug console win10 --type=vga

The Windows VM has mapped about 4GiB of RAM in this case, the initial shrink times out before we force it to vacate the memory, doing it again succeeds, we can see the VM indeed being down to 2GiB, then resettting the limit causes Windows to notice and immediately re-map the rest of the RAM.

That’s a different behavior than Linux because Linux doesn’t seem to actively map/mess with memory it’s not actively using, whereas Windows apparently is.

There may be more memory settings that can be done within Windows to change that behavior, but that’s certainly out of our knowledge and LXD here is definitely doing as expected. It’s inflating the balloon in the VM to cause memory to be freed and deflating it when requested. The difference is that on a Linux VM, this doesn’t then cause the OS to immediately consume the added system memory.

Savin_Savin · November 3, 2020, 8:08am

hello @stgraber okay
here initially the windows VM is mapped to 8GB of RAM and then changed to 6GB, But the windows VM shows 8GB on the task manager, remains unchanged

memory

Also when trying to change memory to 8GB, getting error.

root@cpu-5280:~# lxc config show windows | grep memory
  limits.memory: 8GB
root@cpu-5280:~# ps aux | grep windows
nobody     561 11.5 11.9 8653404 7864628 ?     Sl   07:29   0:58 /bin/qemu-system-x86_64 -S -name windows -uuid c1d30ebf-0136-4ff7-9de2-3bdfe995f848 -daemonize -cpu host -nographic -serial chardev:console -nodefaults -no-reboot -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=deny,resourcecontrol=deny -readconfig /var/log/lxd/windows/qemu.conf -pidfile /var/log/lxd/windows/qemu.pid -D /var/log/lxd/windows/qemu.log -chroot /var/lib/lxd/virtual-machines/windows -smbios type=2,manufacturer=Canonical Ltd.,product=LXD -runas nobody
root     15415  0.0  0.0   6144   884 pts/0    S+   07:38   0:00 grep windows
root@cpu-5280:~# lxc config set windows limits.memory 6GB
root@cpu-5280:~# lxc config show windows | grep memory
  limits.memory: 6GB
root@cpu-5280:~# ps aux | grep windows
nobody     561 10.7  8.9 8653404 5912168 ?     Sl   07:29   1:02 /bin/qemu-system-x86_64 -S -name windows -uuid c1d30ebf-0136-4ff7-9de2-3bdfe995f848 -daemonize -cpu host -nographic -serial chardev:console -nodefaults -no-reboot -no-user-config -sandbox on,obsolete=deny,elevateprivileges=allow,spawn=deny,resourcecontrol=deny -readconfig /var/log/lxd/windows/qemu.conf -pidfile /var/log/lxd/windows/qemu.pid -D /var/log/lxd/windows/qemu.log -chroot /var/lib/lxd/virtual-machines/windows -smbios type=2,manufacturer=Canonical Ltd.,product=LXD -runas nobody
root     16233  0.0  0.0   6144   880 pts/0    S+   07:39   0:00 grep windows
root@cpu-5280:~# lxc config set windows limits.memory 8GB
Error: Failed updating memory limit: Cannot increase memory size beyond boot time size when VM is running