From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id C4A051FF13B for ; Wed, 22 Apr 2026 15:40:27 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id A2C5520533; Wed, 22 Apr 2026 15:40:27 +0200 (CEST) From: Hannes Laimer To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox-backup v8 00/13] fixes #6195: add support for moving groups and namespaces Date: Wed, 22 Apr 2026 15:39:38 +0200 Message-ID: <20260422133951.192862-1-h.laimer@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1776865106550 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.081 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [datastore.rs,lib.rs,sync.rs,namespace.rs,pull.rs,push.rs] Message-ID-Hash: Z6DHFOWTY3SIPAXTJL5ATB4DFKXBBOPG X-Message-ID-Hash: Z6DHFOWTY3SIPAXTJL5ATB4DFKXBBOPG X-MailFrom: h.laimer@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Add support for moving backup groups and entire namespace subtrees to a different location within the same datastore. Groups are moved with exclusive per-group and per-snapshot locking. For S3, objects are copied to the target prefix before deleting the source. Namespace moves process groups individually, deferring and retrying lock conflicts once, so partially failed moves can be completed with move-group. v8, thanks @Chris and @Fabian: - shared check_move_group/check_move_namespace - have BackupGroup::destroy consume the group lock so move can hand over cleanup instead of duplicating it - split remove_namespace into flat and recursive variants, and lift the empty-type-dir cleanup so the flat path prunes after move - lift check_namespace_depth_limit into pbs-datastore - move snapshot-move to BackupDir::move_to with typed target and same-store assertion - check_merge_invariants: take the target group directly, stream min/first-overlap instead of collecting, include both namespaces in the error - reuse create_locked_backup_group (new non-pub variant) for the target group, differentiating soft/hard errors - rename MoveGroupError -> BackupGroupOpError - move_notes_to: S3 upload first, local rename second, with upload failures propagated as fatal instead of warn+continue - source-ns cleanup: distinguish depth-limited (silent) from concurrent (warn, named) children - iter_backup_groups errors recorded per-ns instead of aborting - running moved-group counter in the task log, final summary splits fatal from lock(rerun may succeed) - docs: fix merge-groups default, document permission asymmetry, note stacked destructive defaults - cli: --ns, --target-ns completion - new patch 5/13, thanks @Chris!: fix data-loss race between move-group/move-namespace and a concurrent GC phase 1 with a per-datastore write-ahead journal at /run/proxmox-backup/locks//move-journal v7, thanks @Fabian and @Arthur!: - allow moving into existing target namespaces - merge groups with the same name when owner matches and source snapshots are strictly newer than target snapshots - lock and move snapshots in batches(512) to avoid FD exhaustion - assert new-style locking upfront - pre-create target group dir with source owner - use remove_dir for source cleanup, log errors instead of swallowing - rename 'new-ns' to 'target-ns', use dedicated POST endpoints /move-group and /move-namespace - relax move-group permissions: DATASTORE_PRUNE + ownership on source, DATASTORE_BACKUP + ownership on target - add max-depth, delete-source, and merge-group/merge-groups flags - add docs section and online help link for move dialogs - make ns and target-ns required in CLI, no need to keep the empty -> root thing since this is new v6, thanks @Fabian and @Dominik!: - drop ns locks, lock everything directly, like we do for delete - ui: disable prune for empty groups - ui: dont render verification status for empty groups v5, thanks @Chris!: - lock dir instead of .ns-lock file - improve cleanup of partially failed s3 moves - ui: show empty groups, re-order actions, add context menu - add cli commands for both ns and group moves Hannes Laimer (13): ui: show empty groups datastore: lift check_namespace_depth_limit to pbs-datastore datastore: have BackupGroup::destroy consume the group lock datastore: split remove_namespace into flat and recursive variants datastore: add move journal for coordinating with gc phase 1 datastore: add move-group datastore: add move-namespace docs: add section on moving namespaces and groups api: add POST endpoint for move-group api: add POST endpoint for move-namespace ui: add move group action ui: add move namespace action cli: add move-namespace and move-group commands docs/storage.rst | 60 +- pbs-datastore/src/backup_info.rs | 269 ++++++- pbs-datastore/src/datastore.rs | 732 ++++++++++++++++++-- pbs-datastore/src/lib.rs | 7 +- pbs-datastore/src/move_journal.rs | 149 ++++ src/api2/admin/datastore.rs | 94 +++ src/api2/admin/namespace.rs | 85 ++- src/bin/proxmox_backup_manager/datastore.rs | 113 ++- src/server/pull.rs | 16 +- src/server/push.rs | 7 +- src/server/sync.rs | 21 - www/Makefile | 2 + www/OnlineHelpInfo.js | 4 + www/datastore/Content.js | 167 ++++- www/form/NamespaceSelector.js | 11 + www/window/GroupMove.js | 71 ++ www/window/NamespaceMove.js | 136 ++++ 17 files changed, 1792 insertions(+), 152 deletions(-) create mode 100644 pbs-datastore/src/move_journal.rs create mode 100644 www/window/GroupMove.js create mode 100644 www/window/NamespaceMove.js -- 2.47.3