From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 983A01FF13F for ; Thu, 18 Jun 2026 14:01:46 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id BD40617128; Thu, 18 Jun 2026 14:01:45 +0200 (CEST) Date: Thu, 18 Jun 2026 14:01:38 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= Subject: Re: [PATCH proxmox-backup 0/4] fix 7642: s3: avoid expensive uid/gid lookups To: Christian Ebner , pbs-devel@lists.proxmox.com References: <20260601123124.461765-1-c.ebner@proxmox.com> In-Reply-To: <20260601123124.461765-1-c.ebner@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.17.0 (https://github.com/astroidmail/astroid) Message-Id: <1781783962.ps1il2f5oc.astroid@yuna.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1781784042095 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.054 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com,mod.rs,lib.rs,datastore.rs,pull.rs,verify.rs] Message-ID-Hash: UN264ORPREULF2D6YPUM3NAGJRJ5DPN2 X-Message-ID-Hash: UN264ORPREULF2D6YPUM3NAGJRJ5DPN2 X-MailFrom: f.gruenbichler@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On June 1, 2026 2:31 pm, Christian Ebner wrote: > As reported by a user in the community forum [0] and the thereof > resulting bugzilla entry [1], phase 2 of garbage collection currently > produces avoidable syscalls and parsing overhead, reading > /etc/nsswitch.conf and /etc/passwd for each per-chunk file locking > call. This is however not limited to garbage collection, but rather > affects per-chunk file locking in general. The backup_user() lookup > was identified as the culprit. >=20 > This patch series fixes the overhead by only reading the user once at > the start of garbage collection or whenever a s3 backend instance is > created (which already performs the user lookup anyways), storing it > for the rest of the backend lifetime. couldn't we implement this with less churn, and fix it for *all* call sites, by caching the resolved user? if it changes during the execution of our daemon we have a problem anyway, so might as well treat it as static (initialized in a lazy fashion)? same for priv_user and backup_group.. >=20 > [0] https://forum.proxmox.com/threads/s3-gc-phase-2-nsswitch-passwd-overh= ead.183709/ > [1] https://bugzilla.proxmox.com/show_bug.cgi?id=3D7642 >=20 > Christian Ebner (4): > local datastore cache: combine same module use statements > datastore/api: s3: wrap s3 client into s3 backend type > datastore: s3: avoid double calls to rather expensive backup_user() > fix #7642: s3: avoid repeated user lookup for per-chunk file locking >=20 > pbs-config/src/lib.rs | 30 +- > pbs-datastore/src/backup_info.rs | 64 +++-- > pbs-datastore/src/chunk_store.rs | 33 ++- > pbs-datastore/src/datastore.rs | 262 +++++++++++------- > pbs-datastore/src/lib.rs | 2 +- > pbs-datastore/src/local_chunk_reader.rs | 19 +- > .../src/local_datastore_lru_cache.rs | 14 +- > src/api2/config/datastore.rs | 62 ++--- > src/api2/reader/mod.rs | 13 +- > src/backup/verify.rs | 4 +- > src/server/pull.rs | 10 +- > 11 files changed, 304 insertions(+), 209 deletions(-) >=20 > --=20 > 2.47.3 >=20 >=20 >=20 >=20 >=20 >=20