From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pbs-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id E898E1FF17F
for <inbox@lore.proxmox.com>; Mon, 19 May 2025 13:48:09 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 9345E8D4F;
Mon, 19 May 2025 13:48:05 +0200 (CEST)
From: Christian Ebner <c.ebner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Date: Mon, 19 May 2025 13:46:40 +0200
Message-Id: <20250519114640.303640-40-c.ebner@proxmox.com>
X-Mailer: git-send-email 2.39.5
In-Reply-To: <20250519114640.303640-1-c.ebner@proxmox.com>
References: <20250519114640.303640-1-c.ebner@proxmox.com>
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.032 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pbs-devel] [RFC proxmox-backup 39/39] datastore: create namespace
marker in S3 backend
X-BeenThere: pbs-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox Backup Server development discussion
<pbs-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/>
List-Post: <mailto:pbs-devel@lists.proxmox.com>
List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>,
<mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox Backup Server development discussion
<pbs-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pbs-devel-bounces@lists.proxmox.com
Sender: "pbs-devel" <pbs-devel-bounces@lists.proxmox.com>
The S3 object store only allows to store objects, referenced by their
key. For backup namespaces datastores however use directories, so
they cannot be represented as one to one mapping.
Instead, create an empty marker file for each namespace and operate
based on that.
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
pbs-datastore/src/datastore.rs | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)
diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
index 9c8b7de03..a45b21413 100644
--- a/pbs-datastore/src/datastore.rs
+++ b/pbs-datastore/src/datastore.rs
@@ -635,6 +635,25 @@ impl DataStore {
// construct ns before mkdir to enforce max-depth and name validity
let ns = BackupNamespace::from_parent_ns(parent, name)?;
+ if let DatastoreBackend::S3(s3_client) = self.backend()? {
+ let mut marker = ns.path();
+ marker.push(".namespace");
+ let namespace_marker = marker
+ .to_str()
+ .ok_or_else(|| format_err!("unexpected namespace path"))?;
+
+ let response = proxmox_async::runtime::block_on(
+ s3_client.put_object(namespace_marker.into(), hyper::body::Body::empty()),
+ )?;
+ match response {
+ PutObjectResponse::NeedsRetry => bail!("failed to create namespace, needs retry"),
+ PutObjectResponse::PreconditionFailed => {
+ bail!("failed to create namespace, precondition failed")
+ }
+ PutObjectResponse::Success(_) => (),
+ }
+ }
+
let mut ns_full_path = self.base_path();
ns_full_path.push(ns.path());
@@ -645,6 +664,19 @@ impl DataStore {
/// Returns if the given namespace exists on the datastore
pub fn namespace_exists(&self, ns: &BackupNamespace) -> bool {
+ if let DatastoreBackend::S3(s3_client) = self.backend().unwrap() {
+ let mut marker = ns.path();
+ marker.push(".namespace");
+ let namespace_marker = marker
+ .to_str()
+ .ok_or_else(|| format_err!("unexpected namespace path"))
+ .unwrap();
+
+ let response =
+ proxmox_async::runtime::block_on(s3_client.head_object(namespace_marker.into()))
+ .unwrap();
+ return response.is_some();
+ }
let mut path = self.base_path();
path.push(ns.path());
path.exists()
--
2.39.5
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel