public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Christian Ebner <c.ebner@proxmox.com>
To: Hannes Laimer <h.laimer@proxmox.com>, pbs-devel@lists.proxmox.com
Subject: Re: [PATCH proxmox-backup v4 0/7] fixes #6195: add support for moving groups and namespaces
Date: Tue, 17 Mar 2026 14:47:43 +0100	[thread overview]
Message-ID: <47cc7466-ec85-4a32-99d7-d8dedf7fadf5@proxmox.com> (raw)
In-Reply-To: <20260311151315.133637-1-h.laimer@proxmox.com>

On 3/11/26 4:12 PM, Hannes Laimer wrote:
> Implements moving both backup groups and namespaces within a datastore.
> This also adds namespace locking, this allows moves to happen within a
> datastore that is otherwise in-use.
> 
> 
> # Namespace locking
> 
> To make move_group and move_namespace safe against concurrent
> operations, a namespace-level locking scheme is introduced in the first
> patch.
> 
> Lock files live at:
>    /run/proxmox-backup/locks/<store>/<ns:colon:encoded>/.ns-lock
> 
> Two lock modes are used, both non-blocking (timeout=0):
> 
> - Shared: held by operations that read from or write into a namespace
>    (backup, pull/push sync, verify per snapshot, prune per group).
>    Acquiring a shared lock on ns also acquires shared locks on all
>    non-root ancestors, so an exclusive lock on any ancestor blocks
>    all active operations below it.
> 
> - Exclusive: held by operations that structurally modify a namespace
>    (move_namespace, move_group on the group lock).
>    Acquiring an exclusive lock on ns also acquires shared locks on all
>    non-root ancestors, mirroring the shared variant so that two
>    concurrent structural operations on related namespaces contend
>    correctly.
> 
> Locking up the ancestor chain rather than down the subtree keeps the
> cost O(depth), bounded by MAX_NAMESPACE_DEPTH (8).
> 
> Verify and prune skip gracefully when a namespace lock cannot be
> acquired, since a concurrent move is a transient condition.
> create_locked_backup_group now returns (owner, ns_guard, group_guard),
> all callers updated.
> 
> # Moving
> 
> ## Groups
>    1. lock source ns (shared), lock target ns (shared), lock group (exclusive)
>    2. create target type directory
>    3. FS: rename group directory
>       S3: copy objects, rename cache, delete source objects
> 
> ## Namespace
>    1. lock source ns (exclusive), lock target ns (exclusive)
>    2. FS: rename namespace directory
>       S3: - create target ns dirs and markers, both on S3 and local cache
>           - move groups one by one, if copying fails the group stays at
>             the source and can be moved afterwards manually
>           - clean up empty source ns dirs


Finally managed to do some testing (as promised( today. Major things tested:

- Moving of namespaces on regular and S3 backed datastores
- Namespaces cannot be moved to themself
- Groups cannot be moved to namespace already containing same group
- Renamed namespaces by move to same parent with different name
- Checked namespace/group cannot move while ongoing backup
- Moving externally flocked namespace/group is not possible, including 
shared lock on parents

Did not run into issues.

What I did lack tough is an easy way to move namespaces/groups via the CLI.

Other than that this is in a pretty good shape already!




      parent reply	other threads:[~2026-03-17 13:47 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-03-11 15:13 Hannes Laimer
2026-03-11 15:13 ` [PATCH proxmox-backup v4 1/7] datastore: add namespace-level locking Hannes Laimer
2026-03-12 15:43   ` Christian Ebner
2026-03-13  7:40     ` Hannes Laimer
2026-03-13  7:56       ` Christian Ebner
2026-03-17 13:03     ` Christian Ebner
2026-03-11 15:13 ` [PATCH proxmox-backup v4 2/7] datastore: add move_group Hannes Laimer
2026-03-12 16:08   ` Christian Ebner
2026-03-13  7:28     ` Hannes Laimer
2026-03-13  7:52       ` Christian Ebner
2026-03-11 15:13 ` [PATCH proxmox-backup v4 3/7] datastore: add move_namespace Hannes Laimer
2026-03-11 15:13 ` [PATCH proxmox-backup v4 4/7] api: add PUT endpoint for move_group Hannes Laimer
2026-03-12 16:17   ` Christian Ebner
2026-03-11 15:13 ` [PATCH proxmox-backup v4 5/7] api: add PUT endpoint for move_namespace Hannes Laimer
2026-03-12 16:19   ` Christian Ebner
2026-03-11 15:13 ` [PATCH proxmox-backup v4 6/7] ui: add move group action Hannes Laimer
2026-03-17 11:43   ` Christian Ebner
2026-03-17 11:48     ` Hannes Laimer
2026-03-11 15:13 ` [PATCH proxmox-backup v4 7/7] ui: add move namespace action Hannes Laimer
2026-03-12 16:21 ` [PATCH proxmox-backup v4 0/7] fixes #6195: add support for moving groups and namespaces Christian Ebner
2026-03-17 13:47 ` Christian Ebner [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=47cc7466-ec85-4a32-99d7-d8dedf7fadf5@proxmox.com \
    --to=c.ebner@proxmox.com \
    --cc=h.laimer@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal