From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id C242C92C20 for ; Thu, 23 Mar 2023 07:58:25 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id A13F32E047 for ; Thu, 23 Mar 2023 07:57:55 +0100 (CET) Received: from bastionodiso.odiso.net (bastionodiso.odiso.net [185.151.191.93]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Thu, 23 Mar 2023 07:57:54 +0100 (CET) Received: from kvmformation3.odiso.net (formationkvm3.odiso.net [10.3.94.12]) by bastionodiso.odiso.net (Postfix) with ESMTP id C696780C8; Thu, 23 Mar 2023 07:57:53 +0100 (CET) Received: by kvmformation3.odiso.net (Postfix, from userid 0) id B5FAF2418E6; Thu, 23 Mar 2023 07:57:53 +0100 (CET) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Date: Thu, 23 Mar 2023 07:57:48 +0100 Message-Id: <20230323065750.17230-2-aderumier@odiso.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20230323065750.17230-1-aderumier@odiso.com> References: <20230323065750.17230-1-aderumier@odiso.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.060 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% HEADER_FROM_DIFFERENT_DOMAINS 0.25 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods NO_DNS_FOR_FROM 0.001 Envelope sender has no MX or A DNS records SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Subject: [pve-devel] [PATCH-SERIES qemu, qemu-server, manager 1/1] add patch: add memory allocator detection and use malloc_trim for glibc only X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Mar 2023 06:58:25 -0000 Detect if a different allocator than glibc malloc is linked with LD_PRELOAD, and call malloc_trim() only for glibc malloc This patch is mostly copy/paste from haproxy https://github.com/haproxy/haproxy/blob/master/src/pool.c Signed-off-by: Alexandre Derumier --- ...-and-call-malloc_trim-only-for-glibc.patch | 163 ++++++++++++++++++ debian/patches/series | 1 + 2 files changed, 164 insertions(+) create mode 100644 debian/patches/pve/0063-detect-allocator-and-call-malloc_trim-only-for-glibc.patch diff --git a/debian/patches/pve/0063-detect-allocator-and-call-malloc_trim-only-for-glibc.patch b/debian/patches/pve/0063-detect-allocator-and-call-malloc_trim-only-for-glibc.patch new file mode 100644 index 0000000..aacb6f4 --- /dev/null +++ b/debian/patches/pve/0063-detect-allocator-and-call-malloc_trim-only-for-glibc.patch @@ -0,0 +1,163 @@ +From 9bd1fdf160b0c5801c7f09fc02d15024cd838b39 Mon Sep 17 00:00:00 2001 +From: Alexandre Derumier +Date: Wed, 22 Mar 2023 17:51:35 +0100 +Subject: [PATCH] detect allocator and call malloc_trim only for glibc + +Allow to detect if a different allocator is dynamicaly loaded +with LD_PRELOAD , and don't call malloc_trim in this case. + +mostly copy/paste from haproxy code + +https://github.com/haproxy/haproxy/blob/master/src/pool.c +Signed-off-by: Alexandre Derumier +--- + include/qemu/rcu.h | 19 ++++++++++++ + util/rcu.c | 76 +++++++++++++++++++++++++++++++++++++++++++++- + 2 files changed, 94 insertions(+), 1 deletion(-) + +diff --git a/include/qemu/rcu.h b/include/qemu/rcu.h +index b063c6fde8..7ab8b899f6 100644 +--- a/include/qemu/rcu.h ++++ b/include/qemu/rcu.h +@@ -196,6 +196,25 @@ G_DEFINE_AUTOPTR_CLEANUP_FUNC(RCUReadAuto, rcu_read_auto_unlock) + void rcu_add_force_rcu_notifier(Notifier *n); + void rcu_remove_force_rcu_notifier(Notifier *n); + ++/* This macro may be used to block constant propagation that lets the compiler ++ * detect a possible NULL dereference on a variable resulting from an explicit ++ * assignment in an impossible check. Sometimes a function is called which does ++ * safety checks and returns NULL if safe conditions are not met. The place ++ * where it's called cannot hit this condition and dereferencing the pointer ++ * without first checking it will make the compiler emit a warning about a ++ * "potential null pointer dereference" which is hard to work around. This ++ * macro "washes" the pointer and prevents the compiler from emitting tests ++ * branching to undefined instructions. It may only be used when the developer ++ * is absolutely certain that the conditions are guaranteed and that the ++ * pointer passed in argument cannot be NULL by design. ++ */ ++#define ALREADY_CHECKED(p) do { asm("" : "=rm"(p) : "0"(p)); } while (0) ++ ++/* same as above but to be used to pass the input value to the output but ++ * without letting the compiler know about its initial properties. ++ */ ++#define DISGUISE(v) ({ typeof(v) __v = (v); ALREADY_CHECKED(__v); __v; }) ++ + #ifdef __cplusplus + } + #endif +diff --git a/util/rcu.c b/util/rcu.c +index b6d6c71cff..5dc0fa5f58 100644 +--- a/util/rcu.c ++++ b/util/rcu.c +@@ -35,6 +35,13 @@ + #if defined(CONFIG_MALLOC_TRIM) + #include + #endif ++#if (defined(__GNU_LIBRARY__) && (__GLIBC__ > 2 || __GLIBC__ == 2 && __GLIBC_MINOR__ >= 8)) ++#define GLIBC_HAVE_MALLOC_TRIM ++#endif ++/* glibc 2.33 provides mallinfo2() that overcomes mallinfo()'s type limitations */ ++#if (defined(__GNU_LIBRARY__) && (__GLIBC__ > 2 || __GLIBC__ == 2 && __GLIBC_MINOR__ >= 33)) ++#define GLIBC_HAVE_MALLINFO2 ++#endif + + /* + * Global grace period counter. Bit 0 is always one in rcu_gp_ctr. +@@ -50,6 +57,9 @@ static int in_drain_call_rcu; + static QemuMutex rcu_registry_lock; + static QemuMutex rcu_sync_lock; + ++static int using_default_allocator = 1; ++static int(*my_mallctl)(const char *, void *, size_t *, void *, size_t) = NULL; ++ + /* + * Check whether a quiescent state was crossed between the beginning of + * update_counter_and_wait and now. +@@ -256,7 +266,8 @@ static void *call_rcu_thread(void *opaque) + n = qatomic_read(&rcu_call_count); + if (n == 0) { + #if defined(CONFIG_MALLOC_TRIM) +- malloc_trim(4 * 1024 * 1024); ++ if (!using_default_allocator) ++ malloc_trim(4 * 1024 * 1024); + #endif + qemu_event_wait(&rcu_call_ready_event); + } +@@ -445,11 +456,74 @@ static void rcu_init_child(void) + } + #endif + ++/* Tries to retrieve the address of the first occurrence symbol . ++ * Note that NULL in return is not always an error as a symbol may have that ++ * address in special situations. ++ */ ++static void *get_sym_curr_addr(const char *name) ++{ ++ void *ptr = NULL; ++ ++#ifdef RTLD_DEFAULT ++ if (!build_is_static) ++ ptr = dlsym(RTLD_DEFAULT, name); ++#endif ++ return ptr; ++} ++ ++/* check if we're using the same allocator as the one that provides ++ * malloc_trim() and mallinfo(). The principle is that on glibc, both ++ * malloc_trim() and mallinfo() are provided, and using mallinfo() we ++ * can check if malloc() is performed through glibc or any other one ++ * the executable was linked against (e.g. jemalloc). Prior to this we ++ * have to check whether we're running on jemalloc by verifying if the ++ * mallctl() function is provided. Its pointer will be used later. ++ */ ++static void detect_allocator(void) ++{ ++#if defined(__ELF__) ++ extern int mallctl(const char *, void *, size_t *, void *, size_t) __attribute__((weak)); ++ ++ my_mallctl = mallctl; ++#endif ++ if (!my_mallctl) ++ my_mallctl = get_sym_curr_addr("mallctl"); ++ ++ using_default_allocator = (my_mallctl == NULL); ++ ++ if (!my_mallctl) { ++#if defined(GLIBC_HAVE_MALLOC_TRIM) ++ ++#ifdef GLIBC_HAVE_MALLINFO2 ++ struct mallinfo2 mi1, mi2; ++ void *ptr; ++ mi1 = mallinfo2(); ++ ptr = DISGUISE(malloc(1)); ++ mi2 = mallinfo2(); ++ ++#else ++ struct mallinfo mi1, mi2; ++ void *ptr; ++ mi1 = mallinfo(); ++ ptr = DISGUISE(malloc(1)); ++ mi2 = mallinfo(); ++ ++#endif ++ ++ free(DISGUISE(ptr)); ++ using_default_allocator = !!memcmp(&mi1, &mi2, sizeof(mi1)); ++#endif ++ } ++} ++ + static void __attribute__((__constructor__)) rcu_init(void) + { + smp_mb_global_init(); + #ifdef CONFIG_POSIX + pthread_atfork(rcu_init_lock, rcu_init_unlock, rcu_init_child); ++#endif ++#if defined(CONFIG_MALLOC_TRIM) ++ detect_allocator(); + #endif + rcu_init_complete(); + } +-- +2.30.2 + diff --git a/debian/patches/series b/debian/patches/series index 4e8ddd6..7715e13 100644 --- a/debian/patches/series +++ b/debian/patches/series @@ -94,3 +94,4 @@ pve/0059-vma-create-support-64KiB-unaligned-input-images.patch pve/0060-vma-create-avoid-triggering-assertion-in-error-case.patch pve/0061-block-alloc-track-avoid-premature-break.patch pve/0062-PVE-Backup-allow-passing-max-workers-performance-set.patch +pve/0063-detect-allocator-and-call-malloc_trim-only-for-glibc.patch \ No newline at end of file -- 2.30.2