all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [PATCH proxmox-backup v5 0/9] fixes #6195: add support for moving groups and namespaces
@ 2026-03-19 16:13 Hannes Laimer
  2026-03-19 16:13 ` [PATCH proxmox-backup v5 1/9] ui: show empty groups Hannes Laimer
                   ` (8 more replies)
  0 siblings, 9 replies; 10+ messages in thread
From: Hannes Laimer @ 2026-03-19 16:13 UTC (permalink / raw)
  To: pbs-devel

Implements moving both backup groups and namespaces within a datastore.
This also adds namespace locking, this allows moves to happen within a
datastore that is otherwise in-use.


# Namespace locking

To make move_group and move_namespace safe against concurrent
operations, a namespace-level locking scheme is introduced in the first
patch.

Locks are acquired on the directory itself at:
  /run/proxmox-backup/locks/<store>/<ns:colon:encoded>

Two lock modes are used:

- Shared: held by operations that read from or write into a namespace
  (backup, pull/push sync, verify per snapshot, prune per group,
  move_group on both source and target ns).
  Acquiring a shared lock on ns also acquires shared locks on all
  non-root ancestors, so an exclusive lock on any ancestor blocks
  all active operations below it.
  Uses a 2 second timeout so that worker tasks briefly wait for a
  concurrent move to finish rather than immediately skipping.

- Exclusive: held by operations that structurally modify a namespace
  (move_namespace, delete_namespace). Non-blocking (timeout=0).
  Acquiring an exclusive lock on ns also acquires shared locks on all
  non-root ancestors, mirroring the shared variant so that two
  concurrent structural operations on related namespaces contend
  correctly.

Locking up the ancestor chain rather than down the subtree keeps the
cost O(depth), bounded by MAX_NAMESPACE_DEPTH (8).

Verify and prune skip gracefully when a namespace lock cannot be
acquired, since a concurrent move is a transient condition.
create_locked_backup_group now returns (owner, ns_guard, group_guard),
all callers updated.

# Moving

## Groups
  1. lock source ns (shared), lock target ns (shared), lock group (exclusive)
  2. create target type directory
  3. FS: rename group directory
     S3: copy objects, rename cache, delete source objects

## Namespace
  1. lock source ns (exclusive), lock target ns (exclusive)
  2. FS: rename namespace directory
     S3: - create target ns dirs and markers, both on S3 and local cache
         - move groups one by one, if copying fails the group stays at
           the source and can be moved afterwards manually
         - clean up empty source ns dirs


v5, thanks @Chris!:
 - lock dir instead of `.ns-lock` file
 - explicitly drop ns lock guards in specific order
 - improve cleanup of partially failed s3 moves, we now create the local
   empty dir+owner file before we start copying s3 objects, if any of
   the s3 ops fail, the dir stays behind and can be deleted through the
   UI(which also triggers a prefix cleanup on the s3 storage)
 - update parameters for `DataStore::lookup_datastore()`
 - ui: re-ordered actions, `move` now next to `verify`
 - ui: add move to right-click context menu
 - ui: show empty groups in the UI
 - add cli commands for both ns and group moves
 - add 2s ns lock timeout for worker tasks

*note*: given the UI change to show empty groups it could make sense to
not auto-delete a group if the last snapshot is deleted. For this series
though that is not relevant since we just need empty groups to be
deletable through the UI for partially failed s3 moves



Hannes Laimer (9):
  ui: show empty groups
  datastore: add namespace-level locking
  datastore: add move_group
  datastore: add move_namespace
  api: add PUT endpoint for move_group
  api: add PUT endpoint for move_namespace
  ui: add move group action
  ui: add move namespace action
  cli: add move-namespace and move-group commands

 pbs-datastore/src/backup_info.rs            | 261 ++++++++++++++-
 pbs-datastore/src/datastore.rs              | 345 +++++++++++++++++++-
 src/api2/admin/datastore.rs                 |  78 ++++-
 src/api2/admin/namespace.rs                 |  86 ++++-
 src/api2/backup/environment.rs              |   4 +
 src/api2/backup/mod.rs                      |  14 +-
 src/api2/tape/restore.rs                    |   9 +-
 src/backup/verify.rs                        |  19 +-
 src/bin/proxmox_backup_manager/datastore.rs |  85 ++++-
 src/server/prune_job.rs                     |  11 +
 src/server/pull.rs                          |   8 +-
 src/server/push.rs                          |   6 +
 www/Makefile                                |   2 +
 www/datastore/Content.js                    | 134 +++++++-
 www/form/NamespaceSelector.js               |  11 +
 www/window/GroupMove.js                     |  56 ++++
 www/window/NamespaceMove.js                 |  79 +++++
 17 files changed, 1166 insertions(+), 42 deletions(-)
 create mode 100644 www/window/GroupMove.js
 create mode 100644 www/window/NamespaceMove.js

-- 
2.47.3





^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2026-03-19 16:14 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2026-03-19 16:13 [PATCH proxmox-backup v5 0/9] fixes #6195: add support for moving groups and namespaces Hannes Laimer
2026-03-19 16:13 ` [PATCH proxmox-backup v5 1/9] ui: show empty groups Hannes Laimer
2026-03-19 16:13 ` [PATCH proxmox-backup v5 2/9] datastore: add namespace-level locking Hannes Laimer
2026-03-19 16:13 ` [PATCH proxmox-backup v5 3/9] datastore: add move_group Hannes Laimer
2026-03-19 16:13 ` [PATCH proxmox-backup v5 4/9] datastore: add move_namespace Hannes Laimer
2026-03-19 16:13 ` [PATCH proxmox-backup v5 5/9] api: add PUT endpoint for move_group Hannes Laimer
2026-03-19 16:13 ` [PATCH proxmox-backup v5 6/9] api: add PUT endpoint for move_namespace Hannes Laimer
2026-03-19 16:13 ` [PATCH proxmox-backup v5 7/9] ui: add move group action Hannes Laimer
2026-03-19 16:13 ` [PATCH proxmox-backup v5 8/9] ui: add move namespace action Hannes Laimer
2026-03-19 16:13 ` [PATCH proxmox-backup v5 9/9] cli: add move-namespace and move-group commands Hannes Laimer

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal