public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: "Fabian Grünbichler" <f.gruenbichler@proxmox.com>
To: Proxmox Backup Server development discussion
	<pbs-devel@lists.proxmox.com>
Subject: Re: [pbs-devel] [PATCH proxmox-backup 02/17] api/datastore: move group notes setting to the datastore
Date: Mon, 03 Nov 2025 15:51:23 +0100	[thread overview]
Message-ID: <1762174415.18lfpr185h.astroid@yuna.none> (raw)
In-Reply-To: <20251103113120.239455-3-c.ebner@proxmox.com>

On November 3, 2025 12:31 pm, Christian Ebner wrote:
> In an effort to abstract away the datastore backend related logic
> from the api, move the set_group_notes to a helper method on the
> datastore.
> 
> The new helper method now also assures that the exclusive lock on
> the backup group is acquired before updating the notes, in order to
> avoid possible race conditions, e.g. with backup group destruction.
> 
> Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
> ---
>  pbs-datastore/src/datastore.rs | 26 ++++++++++++++++++++++++++
>  src/api2/admin/datastore.rs    | 17 ++++-------------
>  2 files changed, 30 insertions(+), 13 deletions(-)
> 
> diff --git a/pbs-datastore/src/datastore.rs b/pbs-datastore/src/datastore.rs
> index 127ba1c81..45f315aeb 100644
> --- a/pbs-datastore/src/datastore.rs
> +++ b/pbs-datastore/src/datastore.rs
> @@ -2418,4 +2418,30 @@ impl DataStore {
>              .map_err(|err| format_err!("{err:#}"))?;
>          Ok((backend_type, Some(s3_client)))
>      }
> +
> +    /// Creates or updates the notes associated with a backup group.
> +    /// Acquires exclusive lock on the backup group.
> +    pub fn set_group_notes(
> +        self: &Arc<Self>,
> +        notes: String,
> +        backup_group: BackupGroup,
> +    ) -> Result<(), Error> {
> +        let _lock = backup_group.lock().context("failed to lock backup group")?;

this takes an exclusive lock on group, which means all sorts of other
operations (including creating new snapshots?) are blocked while it is
held

> +
> +        if let DatastoreBackend::S3(s3_client) = self.backend()? {
> +            let mut path = backup_group.backup_ns().path();
> +            path.push(backup_group.group().to_string());
> +            let object_key = crate::s3::object_key_from_path(&path, "notes")
> +                .context("invalid owner file object key")?;
> +            let data = hyper::body::Bytes::copy_from_slice(notes.as_bytes());
> +            let _is_duplicate = proxmox_async::runtime::block_on(
> +                s3_client.upload_replace_with_retry(object_key, data),

but this can take a while, right? FWIW, the same is true of setting the
backup owner..

> +            )
> +            .context("failed to set notes on s3 backend")?;
> +        }
> +        let notes_path = self.group_notes_path(backup_group.backup_ns(), backup_group.group());
> +        replace_file(notes_path, notes.as_bytes(), CreateOptions::new(), false)
> +            .context("failed to replace group notes file")?;
> +        Ok(())
> +    }
>  }
> diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
> index d192ee390..131cdae51 100644
> --- a/src/api2/admin/datastore.rs
> +++ b/src/api2/admin/datastore.rs
> @@ -2009,19 +2009,10 @@ pub fn set_group_notes(
>          &backup_group,
>      )?;
>  
> -    if let DatastoreBackend::S3(s3_client) = datastore.backend()? {
> -        let mut path = ns.path();
> -        path.push(backup_group.to_string());
> -        let object_key = pbs_datastore::s3::object_key_from_path(&path, "notes")
> -            .context("invalid owner file object key")?;
> -        let data = hyper::body::Bytes::copy_from_slice(notes.as_bytes());
> -        let _is_duplicate =
> -            proxmox_async::runtime::block_on(s3_client.upload_replace_with_retry(object_key, data))
> -                .context("failed to set notes on s3 backend")?;
> -    }
> -    let notes_path = datastore.group_notes_path(&ns, &backup_group);
> -    replace_file(notes_path, notes.as_bytes(), CreateOptions::new(), false)?;
> -
> +    let backup_group = datastore.backup_group(ns, backup_group);
> +    datastore
> +        .set_group_notes(notes, backup_group)
> +        .map_err(|err| format_err!("failed to set group notes - {err:#?}"))?;
>      Ok(())
>  }
>  
> -- 
> 2.47.3
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
> 


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


  reply	other threads:[~2025-11-03 14:51 UTC|newest]

Thread overview: 31+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-11-03 11:31 [pbs-devel] [PATCH proxmox-backup 00/17] fix chunk upload/insert, rename corrupt chunks and GC race conditions for s3 backend Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 01/17] sync: pull: instantiate backend only once per sync job Christian Ebner
2025-11-03 14:51   ` Fabian Grünbichler
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 02/17] api/datastore: move group notes setting to the datastore Christian Ebner
2025-11-03 14:51   ` Fabian Grünbichler [this message]
2025-11-04  8:51     ` Christian Ebner
2025-11-04  9:13       ` Fabian Grünbichler
2025-11-04  9:37         ` Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 03/17] api/datastore: move snapshot deletion into dedicated datastore helper Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 04/17] api/datastore: move backup log upload by implementing " Christian Ebner
2025-11-03 14:51   ` Fabian Grünbichler
2025-11-04  8:47     ` Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 05/17] api/datastore: add dedicated datastore helper to set snapshot notes Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 06/17] datastore: refactor chunk insert based on backend Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 07/17] verify: rename corrupted to corrupt in log output and function names Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 08/17] verify/datastore: make rename corrupt chunk a datastore helper method Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 09/17] datastore: refactor rename_corrupt_chunk error handling Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 10/17] datastore: implement per-chunk file locking helper for s3 backend Christian Ebner
2025-11-03 14:51   ` Fabian Grünbichler
2025-11-04  8:45     ` Christian Ebner
2025-11-04  9:01       ` Fabian Grünbichler
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 11/17] datastore: acquire chunk store mutex lock when renaming corrupt chunk Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 12/17] datastore: get per-chunk file lock for chunk rename on s3 backend Christian Ebner
2025-11-03 14:51   ` Fabian Grünbichler
2025-11-04  8:33     ` Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 13/17] fix #6961: datastore: verify: evict corrupt chunks from in-memory LRU cache Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 14/17] datastore: add locking to protect against races on chunk insert for s3 Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 15/17] GC: fix race with chunk upload/insert on s3 backends Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 16/17] GC: lock chunk marker before cleanup in phase 3 " Christian Ebner
2025-11-03 11:31 ` [pbs-devel] [PATCH proxmox-backup 17/17] datastore: GC: drop overly verbose info message during s3 chunk sweep Christian Ebner
2025-11-04 13:08 ` [pbs-devel] superseded: [PATCH proxmox-backup 00/17] fix chunk upload/insert, rename corrupt chunks and GC race conditions for s3 backend Christian Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1762174415.18lfpr185h.astroid@yuna.none \
    --to=f.gruenbichler@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal