From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id A7DE31FF185 for ; Mon, 21 Jul 2025 16:00:54 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 08F6E10B34; Mon, 21 Jul 2025 16:02:05 +0200 (CEST) Message-ID: Date: Mon, 21 Jul 2025 16:01:31 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: Proxmox Backup Server development discussion , Hannes Laimer References: <20250719125035.9926-1-c.ebner@proxmox.com> <20250719125035.9926-22-c.ebner@proxmox.com> Content-Language: en-US, de-DE From: Christian Ebner In-Reply-To: X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1753106484551 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.045 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [PATCH proxmox-backup v9 18/46] datastore: create namespace marker in s3 backend X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Cc: pbs-devel Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" On 7/21/25 3:52 PM, Hannes Laimer wrote: > On Sat Jul 19, 2025 at 2:50 PM CEST, Christian Ebner wrote: >> The S3 object store only allows to store objects, referenced by their >> key. For backup namespaces datastores however use directories, so >> they cannot be represented as one to one mapping. >> >> Instead, create an empty marker file for each namespace and operate >> based on that. >> >> Signed-off-by: Christian Ebner >> --- >> changes since version 8: >> - use upload_on_replace_with_retry >> >> pbs-datastore/src/datastore.rs | 10 ++++++++++ >> 1 file changed, 10 insertions(+) >> >> diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs >> index bc829c5b8..5bb4e1777 100644 >> --- a/pbs-datastore/src/datastore.rs >> +++ b/pbs-datastore/src/datastore.rs >> @@ -44,6 +44,7 @@ static DATASTORE_MAP: LazyLock>>> = >> LazyLock::new(|| Mutex::new(HashMap::new())); >> >> const GROUP_NOTES_FILE_NAME: &str = "notes"; >> +const NAMESPACE_MARKER_FILENAME: &str = ".namespace"; >> >> /// checks if auth_id is owner, or, if owner is a token, if >> /// auth_id is the user of the token >> @@ -613,6 +614,15 @@ impl DataStore { >> // construct ns before mkdir to enforce max-depth and name validity >> let ns = BackupNamespace::from_parent_ns(parent, name)?; >> >> + if let DatastoreBackend::S3(s3_client) = self.backend()? { >> + let object_key = crate::s3::object_key_from_path(&ns.path(), NAMESPACE_MARKER_FILENAME) >> + .context("invalid namespace marker object key")?; > > > hmmm, I didn't test this, but with our NS max depth of 7 (IIRC) and the > linux max filename length of ~250 this "could" be larger than the 1024 > bytes an object key can be. Maybe we should check for that, even though > I doubt this would ever be a problem.. Okay, can add a check to cover that as well. _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel