From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id F3A2284E9F for ; Thu, 16 Dec 2021 13:12:39 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id BBDE51E47C for ; Thu, 16 Dec 2021 13:12:39 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 2A0D31E44E for ; Thu, 16 Dec 2021 13:12:38 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 0384B45310 for ; Thu, 16 Dec 2021 13:12:38 +0100 (CET) From: Fabian Ebner To: pve-devel@lists.proxmox.com Date: Thu, 16 Dec 2021 13:12:26 +0100 Message-Id: <20211216121233.162288-5-f.ebner@proxmox.com> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20211216121233.162288-1-f.ebner@proxmox.com> References: <20211216121233.162288-1-f.ebner@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.155 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [vzdump.pm] Subject: [pve-devel] [PATCH manager 2/6] vzdump: backup limit: only count unprotected backups X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Dec 2021 12:12:40 -0000 since they are the ones relevant for pruning and protected backups have their own separate limit. Since get_backup_file_list is only used in places where the unprotected backups are needed, adapt the helper accordingly. If there is a storage, use PVE::Storage::volume_list to count the unprotected backups. This avoids a direct invocation of run_client_cmd for PBS and the limit check can also work for external storage plugins which might not be dir-based or name the backups differently. Signed-off-by: Fabian Ebner --- Dependency bump for storage is needed for it to filter by backup type and not just ID, and also since we'd like to have the limiting behavior for protected there, before making the limit weaker here. PVE/VZDump.pm | 27 ++++++++++++--------------- 1 file changed, 12 insertions(+), 15 deletions(-) diff --git a/PVE/VZDump.pm b/PVE/VZDump.pm index daaaf0e3..8e20c320 100644 --- a/PVE/VZDump.pm +++ b/PVE/VZDump.pm @@ -684,16 +684,17 @@ sub compressor_info { } } -sub get_backup_file_list { +sub get_unprotected_backup_file_list { my ($dir, $bkname) = @_; my $bklist = []; foreach my $fn (<$dir/${bkname}-*>) { my $archive_info = eval { PVE::Storage::archive_info($fn) } // {}; if ($archive_info->{is_std_name}) { - my $filename = $archive_info->{filename}; + my $path = "$dir/$archive_info->{filename}"; + next if -e PVE::Storage::protection_file_path($path); my $backup = { - 'path' => "$dir/$filename", + 'path' => $path, 'ctime' => $archive_info->{ctime}, }; push @{$bklist}, $backup; @@ -773,13 +774,15 @@ sub exec_backup_task { if ($backup_limit && !$opts->{remove}) { my $count; - if ($self->{opts}->{pbs}) { - my $res = PVE::Storage::PBSPlugin::run_client_cmd($opts->{scfg}, $opts->{storage}, 'snapshots', $pbs_group_name); - $count = scalar(@$res); + if (my $storeid = $opts->{storage}) { + my $backups = PVE::Storage::volume_list($cfg, $storeid, $vmid, 'backup'); + $count = grep { + !$_->{protected} && (!$_->{subtype} || $_->{subtype} eq $vmtype) + } $backups->@*; } else { - my $bklist = get_backup_file_list($opts->{dumpdir}, $bkname); - $count = scalar(@$bklist); + $count = scalar(get_unprotected_backup_file_list($opts->{dumpdir}, $bkname)->@*); } + die "There is a max backup limit of $backup_limit enforced by the". " target storage or the vzdump parameters.". " Either increase the limit or delete old backup(s).\n" @@ -989,13 +992,7 @@ sub exec_backup_task { debugmsg ('info', "prune older backups with retention: $keepstr", $logfd); my $pruned = 0; if (!defined($opts->{storage})) { - my $bklist = get_backup_file_list($opts->{dumpdir}, $bkname); - - for my $prune_entry ($bklist->@*) { - if (-e PVE::Storage::protection_file_path($prune_entry->{path})) { - $prune_entry->{mark} = 'protected'; - } - } + my $bklist = get_unprotected_backup_file_list($opts->{dumpdir}, $bkname); PVE::Storage::prune_mark_backup_group($bklist, $prune_options); -- 2.30.2