From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 7777C1FF0E7 for ; Fri, 10 Jul 2026 14:51:24 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id A02922143B; Fri, 10 Jul 2026 14:51:23 +0200 (CEST) From: Maximiliano Sandoval To: pve-devel@lists.proxmox.com Subject: [RFC PATCH qemu-server] memory: deallocate hugepages on cleanup Date: Fri, 10 Jul 2026 14:51:15 +0200 Message-ID: <20260710125119.422202-1-m.sandoval@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1783687868107 X-SPAM-LEVEL: Spam detection results: 0 DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: OMBKWHDPDHIGSTROQWN52BHKPRPKAWWQ X-Message-ID-Hash: OMBKWHDPDHIGSTROQWN52BHKPRPKAWWQ X-MailFrom: m.sandoval@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Right before VM startup the hugepages_allocate call would increase the number of persistent hugepages (at /sys/kernel/mm/hugepages/hugepages-{size}kB/nr_hugepages) based on the guest and host numa topologies, then the VM is started, and immediately after the hugepages_pre_deallocate would set the nr_hugepages back to the original value. >>From the HugeTLB Pages documentation [1]: Caveat: Shrinking the persistent huge page pool via ``nr_hugepages`` such that it becomes less than the number of huge pages in use will convert the balance of the in-use huge pages to surplus huge pages. This will occur even if the number of surplus pages would exceed the overcommit value. As long as this condition holds--that is, until ``nr_hugepages+nr_overcommit_hugepages`` is increased sufficiently, or the surplus huge pages go out of use and are freed-- no more surplus huge pages will be allowed to be allocated. In a setup where nr_overcommit_hugepages is 0, if the KVM process frees its hugepages, the number of surplus hugepages can potentially be above `nr_overcommit_hugepages` and no new surplus will be allowed to be allocated by the process. This can be reproduced as follows: On a setup with 1 numa node, create a Linux VM, then configure it as follows: $ qm set --numa 1 --hugepages 2 --balloon 1 If one then starts the VM, the nr_hugepages would decrease monotonically and the VM would get stuck during its boot process. We address this by deferring the de-allocation until qm cleanup. [1] https://www.kernel.org/doc/html/latest/admin-guide/mm/hugetlbpage.html Signed-off-by: Maximiliano Sandoval --- src/PVE/CLI/qm.pm | 8 ++++++++ src/PVE/QemuServer.pm | 3 --- 2 files changed, 8 insertions(+), 3 deletions(-) diff --git a/src/PVE/CLI/qm.pm b/src/PVE/CLI/qm.pm index b903c1f1..8a88fb48 100755 --- a/src/PVE/CLI/qm.pm +++ b/src/PVE/CLI/qm.pm @@ -1137,6 +1137,14 @@ __PACKAGE__->register_method({ die "skipping cleanup - 'rollback' lock is present\n" if $conf->{lock} && $conf->{lock} eq 'rollback'; + my $hotplug_features = + PVE::QemuServer::parse_hotplug_features(defined($conf->{hotplug}) ? $conf->{hotplug} : '1'); + my $hugepages_topology = + PVE::QemuServer::Memory::hugepages_topology($conf, $hotplug_features->{memory}); + + PVE::QemuServer::Memory::hugepages_pre_deallocate($hugepages_topology) + if !$conf->{keephugepages}; + if (!$clean) { # we have to cleanup the tap devices after a crash diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm index 59a37098..23d5aa9f 100644 --- a/src/PVE/QemuServer.pm +++ b/src/PVE/QemuServer.pm @@ -5840,9 +5840,6 @@ sub vm_start_nolock { if !$conf->{keephugepages}; die $err; } - - PVE::QemuServer::Memory::hugepages_pre_deallocate($hugepages_topology) - if !$conf->{keephugepages}; }; eval { PVE::QemuServer::Memory::hugepages_update_locked($code); }; -- 2.47.3