[RFC PATCH qemu-server] memory: deallocate hugepages on cleanup

[Maximiliano Sandoval <m.sandoval@proxmox.com>]

Right before VM startup the hugepages_allocate call would increase the number of
persistent hugepages (at
/sys/kernel/mm/hugepages/hugepages-{size}kB/nr_hugepages) based on the guest and
host numa topologies, then the VM is started, and immediately after the
hugepages_pre_deallocate would set the nr_hugepages back to the original value.

>From the HugeTLB Pages documentation [1]:

Caveat: Shrinking the persistent huge page pool via ``nr_hugepages`` such that
it becomes less than the number of huge pages in use will convert the balance
of the in-use huge pages to surplus huge pages.  This will occur even if
the number of surplus pages would exceed the overcommit value.  As long as
this condition holds--that is, until ``nr_hugepages+nr_overcommit_hugepages`` is
increased sufficiently, or the surplus huge pages go out of use and are freed--
no more surplus huge pages will be allowed to be allocated.

In a setup where nr_overcommit_hugepages is 0, if the KVM process frees its
hugepages, the number of surplus hugepages can potentially be above
`nr_overcommit_hugepages` and no new surplus will be allowed to be allocated by
the process.

This can be reproduced as follows:

On a setup with 1 numa node, create a Linux VM, then configure it as follows:

$ qm set <vmid> --numa 1 --hugepages 2 --balloon 1

If one then starts the VM, the nr_hugepages would decrease monotonically and the
VM would get stuck during its boot process.

We address this by deferring the de-allocation until qm cleanup.

[1] https://www.kernel.org/doc/html/latest/admin-guide/mm/hugetlbpage.html

Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
---
 src/PVE/CLI/qm.pm     | 8 ++++++++
 src/PVE/QemuServer.pm | 3 ---
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/src/PVE/CLI/qm.pm b/src/PVE/CLI/qm.pm
index b903c1f1..8a88fb48 100755
--- a/src/PVE/CLI/qm.pm
+++ b/src/PVE/CLI/qm.pm
@@ -1137,6 +1137,14 @@ __PACKAGE__->register_method({
                 die "skipping cleanup - 'rollback' lock is present\n"
                     if $conf->{lock} && $conf->{lock} eq 'rollback';
 
+                my $hotplug_features =
+                    PVE::QemuServer::parse_hotplug_features(defined($conf->{hotplug}) ? $conf->{hotplug} : '1');
+                my $hugepages_topology =
+                    PVE::QemuServer::Memory::hugepages_topology($conf, $hotplug_features->{memory});
+
+                PVE::QemuServer::Memory::hugepages_pre_deallocate($hugepages_topology)
+                    if !$conf->{keephugepages};
+
                 if (!$clean) {
                     # we have to cleanup the tap devices after a crash
 
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 59a37098..23d5aa9f 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -5840,9 +5840,6 @@ sub vm_start_nolock {
                     if !$conf->{keephugepages};
                 die $err;
             }
-
-            PVE::QemuServer::Memory::hugepages_pre_deallocate($hugepages_topology)
-                if !$conf->{keephugepages};
         };
         eval { PVE::QemuServer::Memory::hugepages_update_locked($code); };
 
-- 
2.47.3