From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 6B8991FF0C0 for ; Wed, 18 Nov 2026 14:32:17 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 77D012174F; Wed, 18 Nov 2026 14:30:31 +0100 (CET) From: Thomas Lamprecht To: pve-devel@lists.proxmox.com Subject: [PATCH v2 storage 03/20] add backup concurrency option to storages Date: Wed, 18 Nov 2026 14:18:27 +0100 Message-ID: <20261118132947.1093069-4-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261118132947.1093069-1-t.lamprecht@proxmox.com> References: <20261118132947.1093069-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1795008592194 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.171 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: YOZNVGVJDY5QSUQ6BKFA3FDLL7BOD6FQ X-Message-ID-Hash: YOZNVGVJDY5QSUQ6BKFA3FDLL7BOD6FQ X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Allows limiting how many guest backups use a storage at the same time, be it as backup target or because guest disks live on it. A backup server or a Ceph cluster can thus be protected from more backup load than it can handle, independent of how many nodes and jobs take part. For shared storages the limit counts for the whole cluster, for local storages on each node. The ESXi import storage is not a backup source or target and does not get the option. The option is declared in the base plugin, as the backup code reads it from storages of any type, so plugins that isolate their properties can offer it as well. Third-party plugins need to list it in their options() to support it. Signed-off-by: Thomas Lamprecht --- changes v1 -> v2: - rebase onto the property isolation changes; the option is now in the pinned lists of the plugin property test src/PVE/Storage/BTRFSPlugin.pm | 1 + src/PVE/Storage/CIFSPlugin.pm | 1 + src/PVE/Storage/CephFSPlugin.pm | 1 + src/PVE/Storage/DirPlugin.pm | 1 + src/PVE/Storage/ISCSIDirectPlugin.pm | 1 + src/PVE/Storage/ISCSIPlugin.pm | 1 + src/PVE/Storage/LVMPlugin.pm | 1 + src/PVE/Storage/LvmThinPlugin.pm | 1 + src/PVE/Storage/NFSPlugin.pm | 1 + src/PVE/Storage/PBSPlugin.pm | 1 + src/PVE/Storage/Plugin.pm | 9 +++++++++ src/PVE/Storage/RBDPlugin.pm | 1 + src/PVE/Storage/ZFSPlugin.pm | 1 + src/PVE/Storage/ZFSPoolPlugin.pm | 1 + src/test/plugin_properties_test.pm | 5 +++-- 15 files changed, 25 insertions(+), 2 deletions(-) diff --git a/src/PVE/Storage/BTRFSPlugin.pm b/src/PVE/Storage/BTRFSPlugin.pm index 72cb39b7..3f5181fd 100644 --- a/src/PVE/Storage/BTRFSPlugin.pm +++ b/src/PVE/Storage/BTRFSPlugin.pm @@ -83,6 +83,7 @@ sub options { return { path => { fixed => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, shared => { optional => 1 }, disable => { optional => 1 }, 'prune-backups' => { optional => 1 }, diff --git a/src/PVE/Storage/CIFSPlugin.pm b/src/PVE/Storage/CIFSPlugin.pm index 15d5a22d..594ec1de 100644 --- a/src/PVE/Storage/CIFSPlugin.pm +++ b/src/PVE/Storage/CIFSPlugin.pm @@ -189,6 +189,7 @@ sub options { share => { fixed => 1 }, subdir => { optional => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, disable => { optional => 1 }, 'prune-backups' => { optional => 1 }, 'max-protected-backups' => { optional => 1 }, diff --git a/src/PVE/Storage/CephFSPlugin.pm b/src/PVE/Storage/CephFSPlugin.pm index 39b9686b..19a1f7ab 100644 --- a/src/PVE/Storage/CephFSPlugin.pm +++ b/src/PVE/Storage/CephFSPlugin.pm @@ -164,6 +164,7 @@ sub options { 'content-dirs' => { optional => 1 }, monhost => { optional => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, subdir => { optional => 1 }, disable => { optional => 1 }, options => { optional => 1 }, diff --git a/src/PVE/Storage/DirPlugin.pm b/src/PVE/Storage/DirPlugin.pm index 397d4fe7..56c5d785 100644 --- a/src/PVE/Storage/DirPlugin.pm +++ b/src/PVE/Storage/DirPlugin.pm @@ -76,6 +76,7 @@ sub options { path => { fixed => 1 }, 'content-dirs' => { optional => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, shared => { optional => 1 }, disable => { optional => 1 }, 'prune-backups' => { optional => 1 }, diff --git a/src/PVE/Storage/ISCSIDirectPlugin.pm b/src/PVE/Storage/ISCSIDirectPlugin.pm index d57bc704..f1d133ba 100644 --- a/src/PVE/Storage/ISCSIDirectPlugin.pm +++ b/src/PVE/Storage/ISCSIDirectPlugin.pm @@ -77,6 +77,7 @@ sub options { portal => { fixed => 1 }, target => { fixed => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, disable => { optional => 1 }, content => { optional => 1 }, bwlimit => { optional => 1 }, diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm index 1e349cde..9c7b20a4 100644 --- a/src/PVE/Storage/ISCSIPlugin.pm +++ b/src/PVE/Storage/ISCSIPlugin.pm @@ -416,6 +416,7 @@ sub options { portal => { fixed => 1 }, target => { fixed => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, disable => { optional => 1 }, content => { optional => 1 }, bwlimit => { optional => 1 }, diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm index 3cdab9e4..60bc8743 100644 --- a/src/PVE/Storage/LVMPlugin.pm +++ b/src/PVE/Storage/LVMPlugin.pm @@ -949,6 +949,7 @@ sub options { return { vgname => { fixed => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, shared => { optional => 1 }, disable => { optional => 1 }, saferemove => { optional => 1 }, diff --git a/src/PVE/Storage/LvmThinPlugin.pm b/src/PVE/Storage/LvmThinPlugin.pm index 67f7bb2f..74817e77 100644 --- a/src/PVE/Storage/LvmThinPlugin.pm +++ b/src/PVE/Storage/LvmThinPlugin.pm @@ -63,6 +63,7 @@ sub options { thinpool => { fixed => 1 }, vgname => { fixed => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, disable => { optional => 1 }, content => { optional => 1 }, bwlimit => { optional => 1 }, diff --git a/src/PVE/Storage/NFSPlugin.pm b/src/PVE/Storage/NFSPlugin.pm index fdc32c01..cf42b619 100644 --- a/src/PVE/Storage/NFSPlugin.pm +++ b/src/PVE/Storage/NFSPlugin.pm @@ -119,6 +119,7 @@ sub options { server => { fixed => 1 }, export => { fixed => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, disable => { optional => 1 }, 'prune-backups' => { optional => 1 }, 'max-protected-backups' => { optional => 1 }, diff --git a/src/PVE/Storage/PBSPlugin.pm b/src/PVE/Storage/PBSPlugin.pm index cf826237..01f8c936 100644 --- a/src/PVE/Storage/PBSPlugin.pm +++ b/src/PVE/Storage/PBSPlugin.pm @@ -85,6 +85,7 @@ sub options { namespace => { optional => 1 }, port => { optional => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, disable => { optional => 1 }, content => { optional => 1 }, username => { optional => 1 }, diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm index 724c2d35..2095e7eb 100644 --- a/src/PVE/Storage/Plugin.pm +++ b/src/PVE/Storage/Plugin.pm @@ -217,6 +217,15 @@ my $defaultData = { default => "Unlimited for users with Datastore.Allocate privilege, 5 for other users", }, + 'backup-concurrency' => { + description => + "Maximal number of guest backups using this storage at the same time," + . " either as backup target or for guest disks. Counted for the whole cluster if" + . " the storage is shared, for each node otherwise.", + type => 'integer', + minimum => 1, + optional => 1, + }, shared => { title => "Shared", description => diff --git a/src/PVE/Storage/RBDPlugin.pm b/src/PVE/Storage/RBDPlugin.pm index 26a1cf81..f4bc702f 100644 --- a/src/PVE/Storage/RBDPlugin.pm +++ b/src/PVE/Storage/RBDPlugin.pm @@ -482,6 +482,7 @@ sub properties { sub options { return { nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, disable => { optional => 1 }, monhost => { optional => 1 }, pool => { optional => 1 }, diff --git a/src/PVE/Storage/ZFSPlugin.pm b/src/PVE/Storage/ZFSPlugin.pm index 732069e0..69316e7d 100644 --- a/src/PVE/Storage/ZFSPlugin.pm +++ b/src/PVE/Storage/ZFSPlugin.pm @@ -247,6 +247,7 @@ sub properties { sub options { return { nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, disable => { optional => 1 }, portal => { fixed => 1 }, target => { fixed => 1 }, diff --git a/src/PVE/Storage/ZFSPoolPlugin.pm b/src/PVE/Storage/ZFSPoolPlugin.pm index 1085db94..6c47dc48 100644 --- a/src/PVE/Storage/ZFSPoolPlugin.pm +++ b/src/PVE/Storage/ZFSPoolPlugin.pm @@ -106,6 +106,7 @@ sub options { blocksize => { optional => 1 }, sparse => { optional => 1 }, nodes => { optional => 1 }, + 'backup-concurrency' => { optional => 1 }, disable => { optional => 1 }, content => { optional => 1 }, bwlimit => { optional => 1 }, diff --git a/src/test/plugin_properties_test.pm b/src/test/plugin_properties_test.pm index d8e336d1..25c1a023 100644 --- a/src/test/plugin_properties_test.pm +++ b/src/test/plugin_properties_test.pm @@ -12,6 +12,7 @@ use PVE::Storage::Plugin; # none can be removed, renamed, narrowed or left unexposed by an in-tree plugin that isolates. my $available = { authsupported => 'string', + 'backup-concurrency' => 'integer minimum=1', base => 'string:pve-volume-id', blocksize => 'string:pve-storage-zfs-blocksize', bwlimit => 'string:{clone,default,migration,move,restore}', @@ -137,8 +138,8 @@ for my $name (sort keys $available->%*) { # adding one can break third-party plugins. A property belongs here only if methods of the base # plugin or code outside of the plugins read it from storages of any type. my $base = [qw( - base bwlimit content content-dirs create-base-path create-subdirs disable format - max-protected-backups mkdir nodes path preallocation prune-backups shared + backup-concurrency base bwlimit content content-dirs create-base-path create-subdirs disable + format max-protected-backups mkdir nodes path preallocation prune-backups shared snapshot-as-volume-chain storage type )]; my $declared = PVE::Storage::Plugin->base_properties(); -- 2.47.3