From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 290BA1FF0C0 for ; Wed, 18 Nov 2026 14:32:04 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id EDA7721722; Wed, 18 Nov 2026 14:30:21 +0100 (CET) From: Thomas Lamprecht To: pve-devel@lists.proxmox.com Subject: [PATCH v2 manager 18/20] api: backup info: add endpoints for cluster-wide backup slots Date: Wed, 18 Nov 2026 14:18:42 +0100 Message-ID: <20261118132947.1093069-19-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261118132947.1093069-1-t.lamprecht@proxmox.com> References: <20261118132947.1093069-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1795008593158 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.172 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: QMQAHENUM2BQP2R7DZLLDDQOV77ZNSZD X-Message-ID-Hash: QMQAHENUM2BQP2R7DZLLDDQOV77ZNSZD X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: List the guest backups that hold or wait for cluster-wide backup slots, to see where the slots went when guest backups wait. Slots of an offline node stay taken, as its guest backups might still be running. Allow releasing them once it is clear that these backups stopped, for example because the node is powered off. Otherwise the node releases them itself when it is back online. Signed-off-by: Thomas Lamprecht --- changes v1 -> v2: - list slots without going through pvedaemon, the table is no longer in priv/ [Fiona] - use signatures and no multi-line postfix condition, like patch 16 - say which permission the guest list needs, and the unit of 'since' PVE/API2/Cluster/BackupInfo.pm | 106 ++++++++++++++++++++++++++++++++- PVE/VZDump/ClusterSlots.pm | 11 ++++ 2 files changed, 115 insertions(+), 2 deletions(-) diff --git a/PVE/API2/Cluster/BackupInfo.pm b/PVE/API2/Cluster/BackupInfo.pm index 4ef59ac3d..df487c7c5 100644 --- a/PVE/API2/Cluster/BackupInfo.pm +++ b/PVE/API2/Cluster/BackupInfo.pm @@ -9,10 +9,11 @@ use PVE::Tools qw(extract_param); use PVE::Cluster qw(cfs_lock_file cfs_read_file cfs_write_file); use PVE::RESTHandler; use PVE::RPCEnvironment; -use PVE::JSONSchema; +use PVE::JSONSchema qw(get_standard_option); use PVE::Storage; use PVE::Exception qw(raise_param_exc); use PVE::VZDump; +use PVE::VZDump::ClusterSlots; use PVE::VZDump::Common; use base qw(PVE::RESTHandler); @@ -57,11 +58,112 @@ __PACKAGE__->register_method({ }, code => sub { return [ - { subdir => 'not-backed-up' }, + { subdir => 'not-backed-up' }, { subdir => 'slots' }, ]; }, }); +__PACKAGE__->register_method({ + name => 'get_backup_slots', + path => 'slots', + method => 'GET', + description => "Shows guest backups that hold or wait for cluster-wide backup slots.", + permissions => { + description => + "Only lists guests where you have 'VM.Audit' permissions on '/vms/'.", + check => ['perm', '/', ['Sys.Audit']], + }, + parameters => { + additionalProperties => 0, + properties => {}, + }, + returns => { + type => 'array', + items => { + type => 'object', + properties => { + node => get_standard_option('pve-node'), + vmid => get_standard_option('pve-vmid'), + upid => { + type => 'string', + description => 'Backup task the guest backup belongs to.', + }, + state => { + type => 'string', + enum => ['running', 'waiting'], + description => + 'Whether the guest backup holds its slots or waits for them.', + }, + since => { + type => 'integer', + description => + 'When the guest backup got its slots or started waiting for them (UNIX epoch).', + }, + storages => { + type => 'array', + items => get_standard_option('pve-storage-id'), + description => + 'Shared storages with a backup concurrency limit that the guest' + . ' backup uses.', + }, + }, + }, + }, + code => sub { + my $rpcenv = PVE::RPCEnvironment::get(); + my $user = $rpcenv->get_user(); + + my $state = PVE::VZDump::ClusterSlots::read_state(); + + my $result = []; + for my $kind (qw(holders waiters)) { + for my $entry (values $state->{$kind}->%*) { + next if !$rpcenv->check($user, "/vms/$entry->{vmid}", ['VM.Audit'], 1); + + push $result->@*, + { + node => $entry->{node}, + vmid => int($entry->{vmid}), + upid => $entry->{upid}, + state => $kind eq 'holders' ? 'running' : 'waiting', + since => int($kind eq 'holders' ? $entry->{granted} : $entry->{queued}), + storages => [map { m!^storage/(.+)$! ? $1 : () } $entry->{resources}->@*], + }; + } + } + + return $result; + }, +}); + +__PACKAGE__->register_method({ + name => 'release_backup_slots', + path => 'slots/{node}', + method => 'DELETE', + protected => 1, + description => + "Release the cluster-wide backup slots of an offline node. They stay taken because its" + . " guest backups might still be running, so only release them once you are sure that" + . " these stopped, for example because the node is powered off.", + permissions => { + check => ['perm', '/', ['Sys.Modify']], + }, + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + }, + }, + returns => { type => 'null' }, + code => sub { + my ($param) = @_; + + PVE::VZDump::ClusterSlots::release_node($param->{node}); + + return; + }, +}); + __PACKAGE__->register_method({ name => 'get_guests_not_in_backup', path => 'not-backed-up', diff --git a/PVE/VZDump/ClusterSlots.pm b/PVE/VZDump/ClusterSlots.pm index 9d199481d..02895c127 100644 --- a/PVE/VZDump/ClusterSlots.pm +++ b/PVE/VZDump/ClusterSlots.pm @@ -282,4 +282,15 @@ sub cleanup_local_entries($running) { }); } +sub release_node($node) { + update_state(sub($state) { + # checked under the lock, so the node cannot rejoin and take new slots meanwhile + if (get_members()->{$node}->{online}) { + die "cannot release backup slots of node '$node' - it is online\n"; + } + + remove_entries($state, sub($entry) { $entry->{node} eq $node }); + }); +} + 1; -- 2.47.3