From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id C7DC91FF138 for ; Tue, 21 Jul 2026 12:37:30 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id D924D214FA; Tue, 21 Jul 2026 12:37:23 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox{,-backup} v3 00/10] keep datastore config unlocked during long running operations Date: Tue, 21 Jul 2026 12:36:33 +0200 Message-ID: <20260721103643.333028-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1784630220858 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.247 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_LOW -0.7 Sender listed at https://www.dnswl.org/, low trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: FC7RVMMXGYYUFXVUJ5VP7JUPLY7J4MLD X-Message-ID-Hash: FC7RVMMXGYYUFXVUJ5VP7JUPLY7J4MLD X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Currently an s3-refresh for datastores backed by s3, mount and unmount for removalbe datastores as well as datastore creation (independent of datastore type) run while holding the datastore config lock for consistency. However, this also blocks config access by all unrelated datastores. Instead, implement per-datastore locks for maintenance-mode changes, and rely on block unexpected maintenance-mode changes during ongoing operations. This allows to concurrently create datastores, run s3 refresh and perform mount/unmount without blocking config access longer than necessary. Based on a report in the community forum: https://forum.proxmox.com/threads/183244/ Changes since version 2: - Rebased onto current master Changes since version 1 (thanks @Fabian for review): - Heavily reworked locking helpers for s3-refresh and mount/unmount operations - Include new maintenance type create, used to unblock concurrent datastore creation - Refactor and code cleanup, especially with respect to chunk store scope restriction proxmox: Christian Ebner (1): pbs-api-types: add datastore create maintenance-mode type pbs-api-types/src/datastore.rs | 8 ++++++++ pbs-api-types/src/maintenance.rs | 4 ++++ 2 files changed, 12 insertions(+) proxmox-backup: Christian Ebner (9): api: config: rearrange independent code block for datastore creation api/datastore: refactor datastore creation helper logic datastore: restrict chunk store scope to pbs-datastore crate datastore: move lock files base path constant to central location datastore: move file lock helper to centralized place datastore: create lockdir with correct mode for backup user access api/datastore: use maintenance-mode lock to protect against changes api: config: unlocked s3 bucket access check for datastore creation datastore: protect datastore creation by maintenance-mode pbs-datastore/src/backup_info.rs | 36 +--- pbs-datastore/src/chunk_store.rs | 22 +-- pbs-datastore/src/datastore.rs | 135 +++++++++++++- pbs-datastore/src/lib.rs | 70 +++++++- .../src/local_datastore_lru_cache.rs | 2 +- pbs-datastore/src/move_journal.rs | 2 +- proxmox-backup-client/src/main.rs | 15 +- src/api2/admin/datastore.rs | 65 +++---- src/api2/config/datastore.rs | 168 ++++-------------- src/api2/helpers.rs | 77 +++++++- src/api2/node/disks/directory.rs | 27 ++- src/api2/node/disks/zfs.rs | 30 +++- 12 files changed, 399 insertions(+), 250 deletions(-) Summary over all repositories: 14 files changed, 411 insertions(+), 250 deletions(-) -- Generated by murpp 0.11.0