all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox-backup v3] Set MMAP_THRESHOLD to a fixed value (128K)
@ 2022-01-26  6:10 Dietmar Maurer
  2022-01-26 13:27 ` [pbs-devel] applied: " Thomas Lamprecht
  0 siblings, 1 reply; 2+ messages in thread
From: Dietmar Maurer @ 2022-01-26  6:10 UTC (permalink / raw)
  To: pbs-devel

This reduces RSS size by factor 10-20 when running backups.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---

Changes since v2: add it to proxmox-restore-daemon

Changes since v1: use libc bindings


 pbs-tools/src/lib.rs               | 10 ++++++++++
 proxmox-backup-client/src/main.rs  |  2 ++
 proxmox-restore-daemon/src/main.rs |  3 +++
 src/bin/proxmox-backup-api.rs      |  2 ++
 src/bin/proxmox-backup-proxy.rs    |  3 +++
 5 files changed, 20 insertions(+)

diff --git a/pbs-tools/src/lib.rs b/pbs-tools/src/lib.rs
index 053b3c0c..fa1c2c50 100644
--- a/pbs-tools/src/lib.rs
+++ b/pbs-tools/src/lib.rs
@@ -8,3 +8,13 @@ pub mod sha;
 pub mod ticket;
 
 pub mod async_lru_cache;
+
+/// Set MMAP_THRESHOLD to a fixed value (128K)
+///
+/// This reduces RSS size by factor 10-20 when running backups (not
+/// sure why - anything between 128K and 1M works).
+pub fn setup_libc_malloc_opts() {
+    unsafe {
+        libc::mallopt(libc::M_MMAP_THRESHOLD, 4096*32);
+    }
+}
diff --git a/proxmox-backup-client/src/main.rs b/proxmox-backup-client/src/main.rs
index 50682637..a51d5bef 100644
--- a/proxmox-backup-client/src/main.rs
+++ b/proxmox-backup-client/src/main.rs
@@ -1465,6 +1465,8 @@ impl ReadAt for BufferedDynamicReadAt {
 
 fn main() {
 
+    pbs_tools::setup_libc_malloc_opts();
+
     let backup_cmd_def = CliCommand::new(&API_METHOD_CREATE_BACKUP)
         .arg_param(&["backupspec"])
         .completion_cb("repository", complete_repository)
diff --git a/proxmox-restore-daemon/src/main.rs b/proxmox-restore-daemon/src/main.rs
index c2be942b..74d7b147 100644
--- a/proxmox-restore-daemon/src/main.rs
+++ b/proxmox-restore-daemon/src/main.rs
@@ -38,6 +38,9 @@ lazy_static! {
 
 /// This is expected to be run by 'proxmox-file-restore' within a mini-VM
 fn main() -> Result<(), Error> {
+
+    pbs_tools::setup_libc_malloc_opts();
+
     if !Path::new(VM_DETECT_FILE).exists() {
         bail!(
             "This binary is not supposed to be run manually, use 'proxmox-file-restore' instead."
diff --git a/src/bin/proxmox-backup-api.rs b/src/bin/proxmox-backup-api.rs
index e6fc5f23..ee037a3b 100644
--- a/src/bin/proxmox-backup-api.rs
+++ b/src/bin/proxmox-backup-api.rs
@@ -19,6 +19,8 @@ use proxmox_backup::auth_helpers::*;
 use proxmox_backup::config;
 
 fn main() {
+    pbs_tools::setup_libc_malloc_opts();
+
     proxmox_backup::tools::setup_safe_path_env();
 
     if let Err(err) = proxmox_async::runtime::main(run()) {
diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index 523966cf..13025271 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -73,6 +73,9 @@ use proxmox_backup::server::do_verification_job;
 use proxmox_backup::server::do_prune_job;
 
 fn main() -> Result<(), Error> {
+
+    pbs_tools::setup_libc_malloc_opts();
+
     proxmox_backup::tools::setup_safe_path_env();
 
     let backup_uid = pbs_config::backup_user()?.uid;
-- 
2.30.2





^ permalink raw reply	[flat|nested] 2+ messages in thread

* [pbs-devel] applied: [PATCH proxmox-backup v3] Set MMAP_THRESHOLD to a fixed value (128K)
  2022-01-26  6:10 [pbs-devel] [PATCH proxmox-backup v3] Set MMAP_THRESHOLD to a fixed value (128K) Dietmar Maurer
@ 2022-01-26 13:27 ` Thomas Lamprecht
  0 siblings, 0 replies; 2+ messages in thread
From: Thomas Lamprecht @ 2022-01-26 13:27 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Dietmar Maurer

On 26.01.22 07:10, Dietmar Maurer wrote:
> This reduces RSS size by factor 10-20 when running backups.
> 
> Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
> ---
> 
> Changes since v2: add it to proxmox-restore-daemon
> 
> Changes since v1: use libc bindings
> 
> 
>  pbs-tools/src/lib.rs               | 10 ++++++++++
>  proxmox-backup-client/src/main.rs  |  2 ++
>  proxmox-restore-daemon/src/main.rs |  3 +++
>  src/bin/proxmox-backup-api.rs      |  2 ++
>  src/bin/proxmox-backup-proxy.rs    |  3 +++
>  5 files changed, 20 insertions(+)
> 
>

applied, with comment and commit message expanded a bit (could be surely still
even better or have more info, but it did not want to write whole essay's into
it :-)), thanks!




^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2022-01-26 13:27 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-26  6:10 [pbs-devel] [PATCH proxmox-backup v3] Set MMAP_THRESHOLD to a fixed value (128K) Dietmar Maurer
2022-01-26 13:27 ` [pbs-devel] applied: " Thomas Lamprecht

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal