Re: [PATCH pve-qemu 1/2] PVE: use tcmalloc as the memory allocator

[Fiona Ebner <f.ebner@proxmox.com>]

It's preparation for using tcmalloc, not actually using it. I'd prefer a
title like "add patch to support using tcmalloc as the memory allocator".

Am 10.04.26 um 6:30 AM schrieb Kefu Chai:
> Add allocator-aware memory release in the backup completion path:
> since tcmalloc does not provide glibc's malloc_trim(), use the
> tcmalloc-specific MallocExtension_ReleaseFreeMemory() API instead.
> This function walks tcmalloc's page heap free span lists and calls
> madvise(MADV_DONTNEED) -- it does not walk allocated memory or compact
> the heap, so latency impact is negligible.
> 
> Also adds a CONFIG_TCMALLOC meson define so the conditional compilation
> in pve-backup.c can detect the allocator choice.
> 
> Signed-off-by: Kefu Chai <k.chai@proxmox.com>
> ---
>  ...use-tcmalloc-as-the-memory-allocator.patch | 77 +++++++++++++++++++
>  debian/patches/series                         |  1 +
>  2 files changed, 78 insertions(+)
>  create mode 100644 debian/patches/pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch
> 
> diff --git a/debian/patches/pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch b/debian/patches/pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch
> new file mode 100644
> index 0000000..719d522
> --- /dev/null
> +++ b/debian/patches/pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch
> @@ -0,0 +1,77 @@
> +From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
> +From: Kefu Chai <k.chai@proxmox.com>
> +Date: Thu, 9 Apr 2026 17:29:10 +0800
> +Subject: [PATCH] PVE: use tcmalloc as the memory allocator

Similar here. Also, it is specific to backup, I'd go for something like
"PVE-Backup: support using tcmalloc as the memory allocator".

> +
> +Use tcmalloc (from gperftools) as the memory allocator for improved
> +performance with workloads that create many small, short-lived
> +allocations -- particularly Ceph/librbd I/O paths.
> +
> +tcmalloc uses per-thread caches and size-class freelists that handle
> +this allocation pattern more efficiently than glibc's allocator. Ceph
> +benchmarks show ~50% IOPS improvement on 16KB random reads.
> +
> +Since tcmalloc does not provide glibc's malloc_trim(), use the
> +tcmalloc-specific MallocExtension_ReleaseFreeMemory() API to release
> +cached memory back to the OS after backup completion. This function
> +walks tcmalloc's page heap free span lists and calls
> +madvise(MADV_DONTNEED) -- it does not walk allocated memory or compact
> +the heap, so latency impact is negligible.
> +
> +Historical context:
> +- tcmalloc was originally enabled in 2015 but removed due to
> +  performance issues with gperftools 2.2's default settings (low
> +  TCMALLOC_MAX_TOTAL_THREAD_CACHE_BYTES and aggressive decommit
> +  disabled). These issues were resolved in gperftools 2.4+.
> +- jemalloc replaced tcmalloc but was removed in 2020 because it didn't
> +  release memory allocated from Rust (proxmox-backup-qemu) back to the
> +  OS. The allocator-specific release API addresses this.
> +- PVE 9 ships gperftools 2.16, so the old tuning issues are moot.
> +
> +Signed-off-by: Kefu Chai <k.chai@proxmox.com>
> +---
> + meson.build  | 1 +
> + pve-backup.c | 8 ++++++++
> + 2 files changed, 9 insertions(+)
> +
> +diff --git a/meson.build b/meson.build
> +index 0b28d2ec39..c6de2464d6 100644
> +--- a/meson.build
> ++++ b/meson.build
> +@@ -2567,6 +2567,7 @@ config_host_data.set('CONFIG_CRYPTO_SM4', crypto_sm4.found())
> + config_host_data.set('CONFIG_CRYPTO_SM3', crypto_sm3.found())
> + config_host_data.set('CONFIG_HOGWEED', hogweed.found())
> + config_host_data.set('CONFIG_MALLOC_TRIM', has_malloc_trim)
> ++config_host_data.set('CONFIG_TCMALLOC', get_option('malloc') == 'tcmalloc')
> + config_host_data.set('CONFIG_ZSTD', zstd.found())
> + config_host_data.set('CONFIG_QPL', qpl.found())
> + config_host_data.set('CONFIG_UADK', uadk.found())
> +diff --git a/pve-backup.c b/pve-backup.c
> +index ad0f8668fd..d5556f152b 100644
> +--- a/pve-backup.c
> ++++ b/pve-backup.c
> +@@ -19,6 +19,8 @@
> + 
> + #if defined(CONFIG_MALLOC_TRIM)
> + #include <malloc.h>
> ++#elif defined(CONFIG_TCMALLOC)
> ++#include <gperftools/malloc_extension_c.h>
> + #endif
> + 
> + #include <proxmox-backup-qemu.h>
> +@@ -303,6 +305,12 @@ static void coroutine_fn pvebackup_co_cleanup(void)
> +      * Won't happen by default if there is fragmentation.
> +      */
> +     malloc_trim(4 * 1024 * 1024);
> ++#elif defined(CONFIG_TCMALLOC)
> ++    /*
> ++     * Release free memory from tcmalloc's page cache back to the OS. This is
> ++     * allocator-aware and efficiently returns cached spans via madvise().
> ++     */
> ++    MallocExtension_ReleaseFreeMemory();
> + #endif
> + }
> + 
> +-- 
> +2.47.3
> +
> diff --git a/debian/patches/series b/debian/patches/series
> index 8ed0c52..468df6c 100644
> --- a/debian/patches/series
> +++ b/debian/patches/series
> @@ -81,3 +81,4 @@ pve/0044-PVE-backup-get-device-info-allow-caller-to-specify-f.patch
>  pve/0045-PVE-backup-implement-backup-access-setup-and-teardow.patch
>  pve/0046-PVE-backup-prepare-for-the-switch-to-using-blockdev-.patch
>  pve/0047-savevm-async-reuse-migration-blocker-check-for-snaps.patch
> +pve/0048-PVE-use-tcmalloc-as-the-memory-allocator.patch