From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 1BDD41FF13E for ; Wed, 01 Jul 2026 16:05:14 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 4EC852147B; Wed, 01 Jul 2026 16:05:06 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox-backup v3 00/15] fix 7642: avoid expensive uid/gid lookups for lock- and config-files Date: Wed, 1 Jul 2026 16:03:57 +0200 Message-ID: <20260701140412.200920-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1782914659441 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.044 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: NEGNOAEVWVF4C2XPAWGQPUNBIHRHXC4F X-Message-ID-Hash: NEGNOAEVWVF4C2XPAWGQPUNBIHRHXC4F X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: As reported by a user in the community forum [0] and the thereof resulting bugzilla entry [1], phase 2 of garbage collection currently produces avoidable syscalls and parsing overhead, reading /etc/nsswitch.conf and /etc/passwd for each per-chunk file locking call. This is however not limited to garbage collection, but rather affects file locking in general. The backup_user() lookup was identified as the culprit. This patch series fixes this by switching to proxmox-product-config helpers for config file operations and file locks instead, which does use the user as parsed once on process init. [0] https://forum.proxmox.com/threads/s3-gc-phase-2-nsswitch-passwd-overhead.183709/ [1] https://bugzilla.proxmox.com/show_bug.cgi?id=7642 Change since version 2: - Extend previous patch series by refactoring also other callers of pbs_config::backup_user() and use proxmox-product-config helpers where possible Change since version 1 (thanks @Fabian for feedback): - Instead of fixing this for S3 file locking only, do it for all file locks using the open_backup_lockfile(). - Use proxmox-product-config helpers and init user as required Note: Above changes significantly escalated the scope of the patches, as they do now touch any config update as well and in the calling process the init() must have been called once before. proxmox-backup: Christian Ebner (15): bin: api: early init proxmox-product-config bin: daily update: refactor to use proxmox-product-config pbs-config: use proxmox-product-config::replace_secret_config() pbs-config: use proxmox-product-config::replace_config() fix #7642: avoid expensive user lookups on file locking pbs-config: use proxmox-product-config helpers pbs-config: drop backup_group helper, use users gid instead pbs-datastore: use proxmox-product-config cached backup user pbs-datastore: use general helpers for file lock create options server: auth helpers: use proxmox-product-config create options helpers api: subscription: use proxmox-product-config create options tape: use proxmox-product-config helper for user lookup tape: use proxmox-product-config lock file create options tape: use proxmox-product-config to generate create options tree-wide: use proxmox-product-config::get_api_user for user lookup pbs-config/Cargo.toml | 1 + pbs-config/src/acl.rs | 5 +- pbs-config/src/config_version_cache.rs | 2 +- pbs-config/src/datastore.rs | 5 +- pbs-config/src/domains.rs | 5 +- pbs-config/src/drive.rs | 5 +- pbs-config/src/encryption_keys.rs | 9 +-- pbs-config/src/lib.rs | 56 ++----------------- pbs-config/src/media_pool.rs | 5 +- pbs-config/src/metrics.rs | 3 +- pbs-config/src/node.rs | 3 +- pbs-config/src/notifications.rs | 8 ++- pbs-config/src/prune.rs | 5 +- pbs-config/src/remote.rs | 3 +- pbs-config/src/s3.rs | 5 +- pbs-config/src/sync.rs | 5 +- pbs-config/src/tape_job.rs | 5 +- pbs-config/src/token_shadow.rs | 8 +-- pbs-config/src/traffic_control.rs | 5 +- pbs-config/src/user.rs | 5 +- pbs-config/src/verify.rs | 5 +- pbs-datastore/Cargo.toml | 1 + pbs-datastore/src/chunk_store.rs | 27 +++++---- pbs-datastore/src/datastore.rs | 17 +++--- pbs-datastore/src/move_journal.rs | 11 +--- pbs-datastore/src/task_tracking.rs | 7 +-- src/api2/admin/datastore.rs | 2 +- src/api2/admin/s3.rs | 4 +- src/api2/config/datastore.rs | 2 +- src/api2/config/s3.rs | 2 +- src/api2/node/subscription.rs | 11 +--- src/auth_helpers.rs | 27 ++------- src/bin/proxmox-backup-api.rs | 5 +- src/bin/proxmox-backup-proxy.rs | 5 +- src/bin/proxmox-daily-update.rs | 12 ++-- src/bin/proxmox-tape.rs | 8 +++ .../proxmox_backup_manager/subscription.rs | 4 +- src/bin/sg-tape-cmd.rs | 5 +- src/config/mod.rs | 9 +-- src/server/jobstate.rs | 4 +- src/server/metric_collection/mod.rs | 2 +- src/server/metric_collection/pull_metrics.rs | 2 +- src/server/metric_collection/rrd.rs | 2 +- src/server/mod.rs | 6 +- src/server/notifications/mod.rs | 4 +- src/tape/changer/mod.rs | 2 +- src/tape/drive/mod.rs | 11 +--- src/tape/encryption_keys.rs | 5 +- src/tape/inventory.rs | 6 +- src/tape/media_catalog.rs | 14 ++--- src/tape/media_catalog_cache.rs | 10 +--- src/tape/mod.rs | 8 +-- src/tape/test/alloc_writable_media.rs | 17 ++++++ src/traffic_control_cache.rs | 2 +- 54 files changed, 171 insertions(+), 236 deletions(-) Summary over all repositories: 54 files changed, 171 insertions(+), 236 deletions(-) -- Generated by murpp 0.11.0