From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id C89C51FF141 for ; Mon, 13 Apr 2026 15:11:20 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B413824D50; Mon, 13 Apr 2026 15:12:07 +0200 (CEST) Message-ID: <8fada82b-f24e-48a4-97ab-9cbfd432b11a@proxmox.com> Date: Mon, 13 Apr 2026 15:12:03 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH pve-qemu 1/2] PVE: use tcmalloc as the memory allocator To: Kefu Chai , pve-devel@lists.proxmox.com References: <20260410043027.3621673-1-k.chai@proxmox.com> <20260410043027.3621673-2-k.chai@proxmox.com> Content-Language: en-US From: Fiona Ebner In-Reply-To: <20260410043027.3621673-2-k.chai@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1776085849107 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.007 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [meson.build] Message-ID-Hash: IZIBLRR5JFCWLFPRD33QYVQ6L4J6DDCS X-Message-ID-Hash: IZIBLRR5JFCWLFPRD33QYVQ6L4J6DDCS X-MailFrom: f.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: It's preparation for using tcmalloc, not actually using it. I'd prefer a title like "add patch to support using tcmalloc as the memory allocator". Am 10.04.26 um 6:30 AM schrieb Kefu Chai: > Add allocator-aware memory release in the backup completion path: > since tcmalloc does not provide glibc's malloc_trim(), use the > tcmalloc-specific MallocExtension_ReleaseFreeMemory() API instead. > This function walks tcmalloc's page heap free span lists and calls > madvise(MADV_DONTNEED) -- it does not walk allocated memory or compact > the heap, so latency impact is negligible. > > Also adds a CONFIG_TCMALLOC meson define so the conditional compilation > in pve-backup.c can detect the allocator choice. > > Signed-off-by: Kefu Chai > --- > ...use-tcmalloc-as-the-memory-allocator.patch | 77 +++++++++++++++++++ > debian/patches/series | 1 + > 2 files changed, 78 insertions(+) > create mode 100644 debian/patches/pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch > > diff --git a/debian/patches/pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch b/debian/patches/pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch > new file mode 100644 > index 0000000..719d522 > --- /dev/null > +++ b/debian/patches/pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch > @@ -0,0 +1,77 @@ > +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001 > +From: Kefu Chai > +Date: Thu, 9 Apr 2026 17:29:10 +0800 > +Subject: [PATCH] PVE: use tcmalloc as the memory allocator Similar here. Also, it is specific to backup, I'd go for something like "PVE-Backup: support using tcmalloc as the memory allocator". > + > +Use tcmalloc (from gperftools) as the memory allocator for improved > +performance with workloads that create many small, short-lived > +allocations -- particularly Ceph/librbd I/O paths. > + > +tcmalloc uses per-thread caches and size-class freelists that handle > +this allocation pattern more efficiently than glibc's allocator. Ceph > +benchmarks show ~50% IOPS improvement on 16KB random reads. > + > +Since tcmalloc does not provide glibc's malloc_trim(), use the > +tcmalloc-specific MallocExtension_ReleaseFreeMemory() API to release > +cached memory back to the OS after backup completion. This function > +walks tcmalloc's page heap free span lists and calls > +madvise(MADV_DONTNEED) -- it does not walk allocated memory or compact > +the heap, so latency impact is negligible. > + > +Historical context: > +- tcmalloc was originally enabled in 2015 but removed due to > + performance issues with gperftools 2.2's default settings (low > + TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES and aggressive decommit > + disabled). These issues were resolved in gperftools 2.4+. > +- jemalloc replaced tcmalloc but was removed in 2020 because it didn't > + release memory allocated from Rust (proxmox-backup-qemu) back to the > + OS. The allocator-specific release API addresses this. > +- PVE 9 ships gperftools 2.16, so the old tuning issues are moot. > + > +Signed-off-by: Kefu Chai > +--- > + meson.build | 1 + > + pve-backup.c | 8 ++++++++ > + 2 files changed, 9 insertions(+) > + > +diff --git a/meson.build b/meson.build > +index 0b28d2ec39..c6de2464d6 100644 > +--- a/meson.build > ++++ b/meson.build > +@@ -2567,6 +2567,7 @@ config_host_data.set('CONFIG_CRYPTO_SM4', crypto_sm4.found()) > + config_host_data.set('CONFIG_CRYPTO_SM3', crypto_sm3.found()) > + config_host_data.set('CONFIG_HOGWEED', hogweed.found()) > + config_host_data.set('CONFIG_MALLOC_TRIM', has_malloc_trim) > ++config_host_data.set('CONFIG_TCMALLOC', get_option('malloc') == 'tcmalloc') > + config_host_data.set('CONFIG_ZSTD', zstd.found()) > + config_host_data.set('CONFIG_QPL', qpl.found()) > + config_host_data.set('CONFIG_UADK', uadk.found()) > +diff --git a/pve-backup.c b/pve-backup.c > +index ad0f8668fd..d5556f152b 100644 > +--- a/pve-backup.c > ++++ b/pve-backup.c > +@@ -19,6 +19,8 @@ > + > + #if defined(CONFIG_MALLOC_TRIM) > + #include > ++#elif defined(CONFIG_TCMALLOC) > ++#include > + #endif > + > + #include > +@@ -303,6 +305,12 @@ static void coroutine_fn pvebackup_co_cleanup(void) > + * Won't happen by default if there is fragmentation. > + */ > + malloc_trim(4 * 1024 * 1024); > ++#elif defined(CONFIG_TCMALLOC) > ++ /* > ++ * Release free memory from tcmalloc's page cache back to the OS. This is > ++ * allocator-aware and efficiently returns cached spans via madvise(). > ++ */ > ++ MallocExtension_ReleaseFreeMemory(); > + #endif > + } > + > +-- > +2.47.3 > + > diff --git a/debian/patches/series b/debian/patches/series > index 8ed0c52..468df6c 100644 > --- a/debian/patches/series > +++ b/debian/patches/series > @@ -81,3 +81,4 @@ pve/0044-PVE-backup-get-device-info-allow-caller-to-specify-f.patch > pve/0045-PVE-backup-implement-backup-access-setup-and-teardow.patch > pve/0046-PVE-backup-prepare-for-the-switch-to-using-blockdev-.patch > pve/0047-savevm-async-reuse-migration-blocker-check-for-snaps.patch > +pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch