[pbs-devel] [PATCH proxmox-backup] datastore: destroy: delete s3 in-use marker unconditionally

* [pbs-devel] [PATCH proxmox-backup] datastore: destroy: delete s3 in-use marker unconditionally
@ 2025-07-30 15:23 Christian Ebner
  2025-07-31 10:18 ` [pbs-devel] applied: " Thomas Lamprecht
  2025-07-31 11:26 ` [pbs-devel] " Lukas Wagner
  0 siblings, 2 replies; 3+ messages in thread
From: Christian Ebner @ 2025-07-30 15:23 UTC (permalink / raw)
  To: pbs-devel

Currently, the in-use marker object for datastores backed by s3
backend is only removed together with the data when all data is
requested to be removed.

It the data should be preserved, the in-use marker should however
also be removed, allowing to reuse the datastore on the same or
a different host without force overwriting the marker.

Therefore, unconditionally clear the in-use marker from the S3
object store. In order to use the predefined marker filename
constant move it so it can be used during destruction as well.

Suggested-by: Lukas Wagner <l.wagner@proxmox.com>
Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
---
 pbs-datastore/src/datastore.rs | 11 +++++++++++
 pbs-datastore/src/lib.rs       |  2 +-
 src/api2/config/datastore.rs   |  6 +++---
 3 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
index 3e53a670b..57a90971a 100644
--- a/pbs-datastore/src/datastore.rs
+++ b/pbs-datastore/src/datastore.rs
@@ -53,6 +53,8 @@ static DATASTORE_MAP: LazyLock<Mutex<HashMap<String, Arc<DataStoreImpl>>>> =
 pub const GROUP_NOTES_FILE_NAME: &str = "notes";
 /// Filename to store backup group owner
 pub const GROUP_OWNER_FILE_NAME: &str = "owner";
+/// Filename for in-use marker stored on S3 object store backend
+pub const S3_DATASTORE_IN_USE_MARKER: &str = ".in-use";
 const NAMESPACE_MARKER_FILENAME: &str = ".namespace";
 
 /// checks if auth_id is owner, or, if owner is a token, if
@@ -2160,6 +2162,15 @@ impl DataStore {
             if ok {
                 remove(".chunks", &mut ok);
             }
+        } else if let (_backend, Some(s3_client)) =
+            Self::s3_client_and_backend_from_datastore_config(&datastore_config)?
+        {
+
+            // Only delete in-use marker so datastore can be re-imported
+            let object_key = S3ObjectKey::try_from(S3_DATASTORE_IN_USE_MARKER)
+                .context("failed to generate in-use marker object key")?;
+            proxmox_async::runtime::block_on(s3_client.delete_object(object_key))
+                .context("failed to delete in-use marker")?;
         }
 
         // now the config
diff --git a/pbs-datastore/src/lib.rs b/pbs-datastore/src/lib.rs
index b9eb035c2..849078a8f 100644
--- a/pbs-datastore/src/lib.rs
+++ b/pbs-datastore/src/lib.rs
@@ -204,7 +204,7 @@ pub use store_progress::StoreProgress;
 mod datastore;
 pub use datastore::{
     check_backup_owner, ensure_datastore_is_mounted, get_datastore_mount_status, DataStore,
-    DatastoreBackend,
+    DatastoreBackend, S3_DATASTORE_IN_USE_MARKER,
 };
 
 mod hierarchy;
diff --git a/src/api2/config/datastore.rs b/src/api2/config/datastore.rs
index 1b56fd276..14532e5ec 100644
--- a/src/api2/config/datastore.rs
+++ b/src/api2/config/datastore.rs
@@ -32,15 +32,15 @@ use crate::api2::config::tape_backup_job::{delete_tape_backup_job, list_tape_bac
 use crate::api2::config::verify::delete_verification_job;
 use pbs_config::CachedUserInfo;
 
-use pbs_datastore::{get_datastore_mount_status, DataStore, DatastoreBackend};
+use pbs_datastore::{
+    get_datastore_mount_status, DataStore, DatastoreBackend, S3_DATASTORE_IN_USE_MARKER,
+};
 use proxmox_rest_server::WorkerTask;
 use proxmox_s3_client::S3ObjectKey;
 
 use crate::server::jobstate;
 use crate::tools::disks::unmount_by_mountpoint;
 
-const S3_DATASTORE_IN_USE_MARKER: &str = ".in-use";
-
 #[derive(Default, serde::Deserialize, serde::Serialize)]
 #[serde(rename_all = "kebab-case")]
 struct InUseContent {
-- 
2.47.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 3+ messages in thread