From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 311541FF13B for ; Wed, 06 May 2026 18:57:44 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 11CEC1E2D; Wed, 6 May 2026 18:57:44 +0200 (CEST) From: Christian Ebner To: pbs-devel@lists.proxmox.com Subject: [PATCH proxmox{,-backup} v2 00/10] keep datastore config unlock during long running operations Date: Wed, 6 May 2026 18:56:41 +0200 Message-ID: <20260506165651.1322947-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1778086518412 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.070 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Message-ID-Hash: UJ2O4MYP6FH3754PWWKOTI52YL7FR7PB X-Message-ID-Hash: UJ2O4MYP6FH3754PWWKOTI52YL7FR7PB X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Currently an s3-refresh for datastores backed by s3, mount and unmount for removalbe datastores as well as datastore creation (independent of datastore type) run while holding the datastore config lock for consistency. However, this also blocks config access by all unrelated datastores. Instead, implement per-datastore locks for maintenance-mode changes, and rely on block unexpected maintenance-mode changes during ongoing operations. This allows to concurrently create datastores, run s3 refresh and perform mount/unmount without blocking config access longer than necessary. Based on a report in the community forum: https://forum.proxmox.com/threads/183244/ Changes since version 1 (thanks @Fabian for review): - Fully reworked locking helpers for s3-refresh and mount/unmount operations - Include new maintenance type create, used to unblock concurrent datastore creation - Refactor and code cleanup, especially with respect to chunk store scope restriction proxmox: Christian Ebner (1): pbs-api-types: add datastore create maintenance-mode type pbs-api-types/src/datastore.rs | 8 ++++++++ pbs-api-types/src/maintenance.rs | 4 ++++ 2 files changed, 12 insertions(+) proxmox-backup: Christian Ebner (9): api: config: rearrange independent code block for datastore creation api/datastore: refactor datastore creation helper logic datastore: restrict chunk store scope to pbs-datastore crate datastore: move lock files base path constant to central location datastore: move file lock helper to centralized place datastore: create lockdir with correct mode for backup user access api/datastore: use maintenance-mode lock to protect against changes api: config: unlocked s3 bucket access check for datastore creation datastore: protect datastore creation by maintenance-mode pbs-datastore/src/backup_info.rs | 36 +--- pbs-datastore/src/chunk_store.rs | 22 +-- pbs-datastore/src/datastore.rs | 127 ++++++++++++- pbs-datastore/src/lib.rs | 70 +++++++- .../src/local_datastore_lru_cache.rs | 2 +- pbs-datastore/src/move_journal.rs | 2 +- proxmox-backup-client/src/main.rs | 15 +- src/api2/admin/datastore.rs | 67 +++---- src/api2/config/datastore.rs | 167 ++++-------------- src/api2/helpers.rs | 77 +++++++- src/api2/node/disks/directory.rs | 27 ++- src/api2/node/disks/zfs.rs | 30 +++- 12 files changed, 395 insertions(+), 247 deletions(-) Summary over all repositories: 14 files changed, 407 insertions(+), 247 deletions(-) -- Generated by murpp 0.11.0