From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id D39FA623D6 for ; Mon, 23 Nov 2020 13:33:37 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id CF6E128A4E for ; Mon, 23 Nov 2020 13:33:37 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 91ECA28A44 for ; Mon, 23 Nov 2020 13:33:36 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 5EA5D43F0A for ; Mon, 23 Nov 2020 13:33:36 +0100 (CET) From: Fabian Ebner To: pve-devel@lists.proxmox.com Date: Mon, 23 Nov 2020 13:33:08 +0100 Message-Id: <20201123123310.13898-1-f.ebner@proxmox.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.010 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_SHORT 0.001 Use of a URL Shortener for very short URL RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [storage.pm, gnu.org, plugin.pm, pbsplugin.pm] Subject: [pve-devel] [PATCH storage 1/3] prune: introduce keep-all option X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Nov 2020 12:33:37 -0000 useful to have an alternative to the old maxfiles = 0. There has to be a way for vzdump to distinguish between: 1. use the /etc/vzdump.conf default (when no options are configured for the storage) 2. use no limit (when keep-all=1) Signed-off-by: Fabian Ebner --- A mutual dependency bump between storage and manager is needed for this and patch #3. PVE/Storage.pm | 9 ++++++--- PVE/Storage/PBSPlugin.pm | 15 +++++++++++---- PVE/Storage/Plugin.pm | 20 +++++++++++++++++++- 3 files changed, 36 insertions(+), 8 deletions(-) diff --git a/PVE/Storage.pm b/PVE/Storage.pm index bd6e15e..d613469 100755 --- a/PVE/Storage.pm +++ b/PVE/Storage.pm @@ -41,11 +41,11 @@ use PVE::Storage::DRBDPlugin; use PVE::Storage::PBSPlugin; # Storage API version. Increment it on changes in storage API interface. -use constant APIVER => 8; +use constant APIVER => 9; # Age is the number of versions we're backward compatible with. # This is like having 'current=APIVER' and age='APIAGE' in libtool, # see https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html -use constant APIAGE => 7; +use constant APIAGE => 8; # load standard plugins PVE::Storage::DirPlugin->register(); @@ -1622,7 +1622,10 @@ my $prune_mark = sub { sub prune_mark_backup_group { my ($backup_group, $keep) = @_; - if (!scalar(grep {$_ > 0} values %{$keep})) { + my $keep_all = delete $keep->{'keep-all'}; + + if ($keep_all || !scalar(grep {$_ > 0} values %{$keep})) { + $keep = { 'keep-all' => 1 } if $keep_all; foreach my $prune_entry (@{$backup_group}) { $prune_entry->{mark} = 'keep'; } diff --git a/PVE/Storage/PBSPlugin.pm b/PVE/Storage/PBSPlugin.pm index 2e6d3f6..ef9bc79 100644 --- a/PVE/Storage/PBSPlugin.pm +++ b/PVE/Storage/PBSPlugin.pm @@ -313,10 +313,17 @@ sub prune_backups { } my @param; - foreach my $opt (keys %{$keep}) { - next if $keep->{$opt} == 0; - push @param, "--$opt"; - push @param, "$keep->{$opt}"; + + my $keep_all = delete $keep->{'keep-all'}; + + if (!$keep_all) { + foreach my $opt (keys %{$keep}) { + next if $keep->{$opt} == 0; + push @param, "--$opt"; + push @param, "$keep->{$opt}"; + } + } else { # no need to pass anything to PBS + $keep = { 'keep-all' => 1 }; } push @param, '--dry-run' if $dryrun; diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm index fe56864..391f441 100644 --- a/PVE/Storage/Plugin.pm +++ b/PVE/Storage/Plugin.pm @@ -52,6 +52,11 @@ my %prune_option = ( ); our $prune_backups_format = { + 'keep-all' => { + type => 'boolean', + description => 'Keep all backups. Conflicts with the other options when true.', + optional => 1, + }, 'keep-last' => { %prune_option, description => 'Keep the last backups.', @@ -82,7 +87,20 @@ our $prune_backups_format = { 'than one backup for a single year, only the latest one is kept.' }, }; -PVE::JSONSchema::register_format('prune-backups', $prune_backups_format); +PVE::JSONSchema::register_format('prune-backups', $prune_backups_format, \&validate_prune_backups); +sub validate_prune_backups { + my ($prune_backups) = @_; + + my $keep_all = delete $prune_backups->{'keep-all'}; + + if (!scalar(grep {$_ > 0} values %{$prune_backups})) { + $prune_backups = { 'keep-all' => 1 }; + } elsif ($keep_all) { + die "keep-all cannot be set together with other options.\n"; + } + + return $prune_backups; +} register_standard_option('prune-backups', { description => "The retention options with shorter intervals are processed first " . "with --keep-last being the very first one. Each option covers a " . -- 2.20.1