From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id BC9921FF13C for ; Thu, 05 Mar 2026 10:07:09 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 999691DC70; Thu, 5 Mar 2026 10:08:14 +0100 (CET) Message-ID: <6c4d3524-6da0-4a42-ae0f-3b981bf85880@proxmox.com> Date: Thu, 5 Mar 2026 10:08:10 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH proxmox-backup] datastore: don't skip empty namespaces on s3 refresh To: =?UTF-8?Q?Fabian_Gr=C3=BCnbichler?= , pbs-devel@lists.proxmox.com References: <20260304103938.42167-1-h.laimer@proxmox.com> <1772700882.3w2c1gwglr.astroid@yuna.none> Content-Language: en-US From: Hannes Laimer In-Reply-To: <1772700882.3w2c1gwglr.astroid@yuna.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1772701664111 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.989 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.018 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.703 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 1.386 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 2XH5OJDZICFKL2YFVEYWHSWOUKYKFH55 X-Message-ID-Hash: 2XH5OJDZICFKL2YFVEYWHSWOUKYKFH55 X-MailFrom: h.laimer@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 2026-03-05 09:59, Fabian Grünbichler wrote: > On March 4, 2026 11:39 am, Hannes Laimer wrote: >> Without this empty namespaces present on s3 won't be visible locally. >> >> Signed-off-by: Hannes Laimer >> --- >> pbs-datastore/src/datastore.rs | 9 +++++++++ >> 1 file changed, 9 insertions(+) >> >> diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs >> index 7ad3d917..4749319a 100644 >> --- a/pbs-datastore/src/datastore.rs >> +++ b/pbs-datastore/src/datastore.rs >> @@ -2535,6 +2535,15 @@ impl DataStore { >> format!("failed to strip store context prefix {store_prefix} for {object_key}") >> })?; >> if object_path.ends_with(NAMESPACE_MARKER_FILENAME) { >> + // Ensure the marker's parent directory exists so the namespace is visible locally >> + let ns_dir = tmp_base.join(object_path); >> + if let Some(parent) = ns_dir.parent() { >> + proxmox_sys::fs::create_path( >> + parent, >> + Some(dir_create_options), >> + Some(dir_create_options), >> + )?; >> + } > > this is exactly the same code that is done a few lines below (modulo > naming of the variable), could we just move this if with the continue > and the `info!` statement below it a few lines down and have the same > effect? > yes, I kind of though there might be some value in stating explicitly "this is what we do for namespaces" and "this for everything else". But tbh since these are the exact same there is no point in doing that, I'll send a v2, thanks! > diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs > index 7ad3d917d..ce09a72f1 100644 > --- a/pbs-datastore/src/datastore.rs > +++ b/pbs-datastore/src/datastore.rs > @@ -2534,21 +2534,23 @@ impl DataStore { > let object_path = object_path.strip_prefix(&store_prefix).with_context(|| { > format!("failed to strip store context prefix {store_prefix} for {object_key}") > })?; > - if object_path.ends_with(NAMESPACE_MARKER_FILENAME) { > - continue; > - } > - > - info!("Fetching object {object_path}"); > > let file_path = tmp_base.join(object_path); > if let Some(parent) = file_path.parent() { > proxmox_sys::fs::create_path( > parent, > Some(dir_create_options), > Some(dir_create_options), > )?; > } > > + if object_path.ends_with(NAMESPACE_MARKER_FILENAME) { > + // namespace created above even if empty > + continue; > + } > + > + info!("Fetching object {object_path}"); > + > let mut target_file = tokio::fs::OpenOptions::new() > .write(true) > .create(true) > > >> continue; >> } >> >> -- >> 2.47.3 >> >> >> >> >> >>