From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 21985B115 for ; Wed, 6 Apr 2022 12:42:41 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 14BA529673 for ; Wed, 6 Apr 2022 12:42:41 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 9615A2966A for ; Wed, 6 Apr 2022 12:42:39 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 5C4B141E5F for ; Wed, 6 Apr 2022 12:42:39 +0200 (CEST) Date: Wed, 06 Apr 2022 12:42:14 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox VE development discussion References: <20220329125324.120737-1-f.ebner@proxmox.com> <20220329125324.120737-2-f.ebner@proxmox.com> In-Reply-To: <20220329125324.120737-2-f.ebner@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.15.0 (https://github.com/astroidmail/astroid) Message-Id: <1649241726.hobppve4kh.astroid@nora.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL -0.074 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URI_NOVOWEL 0.5 URI hostname has long non-vowel sequence Subject: [pve-devel] applied: [PATCH v2 storage 1/1] plugins: allow limiting the number of protected backups per guest X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Apr 2022 10:42:41 -0000 On March 29, 2022 2:53 pm, Fabian Ebner wrote: > The ability to mark backups as protected broke the implicit assumption > in vzdump that remove=3D1 and current number of backups being the limit > (i.e. sum of all keep options) will result in a backup being removed. >=20 > Introduce a new storage property 'max-protected-backups' to limit the > number of protected backups per guest. Use 5 as a default value, as it > should cover most use cases, while still not having too big of a > potential overhead in many scenarios. >=20 > For external plugins that do not return the backup subtype in > list_volumes, all protected backups with the same ID will count > towards the limit. >=20 > An alternative would be to count the protected backups when pruning. > While that would avoid the need for a new property, it would break the > current semantics of protected backups being ignored for pruning. It > also would be less flexible, e.g. for PBS, it can make sense to have > both keep-all=3D1 and a limit for the number of protected snapshots on > the PVE side. >=20 > Signed-off-by: Fabian Ebner > --- >=20 > Changes from v1: > * Use different default depending on privilege level. >=20 > PVE/Storage.pm | 35 ++++++++++++++++++++++++++++++++++ > PVE/Storage/BTRFSPlugin.pm | 3 ++- > PVE/Storage/CIFSPlugin.pm | 1 + > PVE/Storage/CephFSPlugin.pm | 1 + > PVE/Storage/DirPlugin.pm | 1 + > PVE/Storage/GlusterfsPlugin.pm | 1 + > PVE/Storage/NFSPlugin.pm | 1 + > PVE/Storage/PBSPlugin.pm | 1 + > PVE/Storage/Plugin.pm | 7 +++++++ > 9 files changed, 50 insertions(+), 1 deletion(-) >=20 > diff --git a/PVE/Storage.pm b/PVE/Storage.pm > index 6112991..1a4454c 100755 > --- a/PVE/Storage.pm > +++ b/PVE/Storage.pm > @@ -211,6 +211,17 @@ sub storage_can_replicate { > return $plugin->storage_can_replicate($scfg, $storeid, $format); > } > =20 > +sub get_max_protected_backups { > + my ($scfg, $storeid) =3D @_; > + > + return $scfg->{'max-protected-backups'} if defined($scfg->{'max-prot= ected-backups'}); > + > + my $rpcenv =3D PVE::RPCEnvironment::get(); > + my $authuser =3D $rpcenv->get_user(); > + > + return $rpcenv->check($authuser, "/storage/$storeid", ['Datastore.Al= locate'], 1) ? -1 : 5; > +} > + > sub storage_ids { > my ($cfg) =3D @_; > =20 > @@ -240,6 +251,30 @@ sub update_volume_attribute { > my $scfg =3D storage_config($cfg, $storeid); > my $plugin =3D PVE::Storage::Plugin->lookup($scfg->{type}); > =20 > + my ($vtype, undef, $vmid) =3D $plugin->parse_volname($volname); > + my $max_protected_backups =3D get_max_protected_backups($scfg, $stor= eid); > + > + if ( > + $vtype eq 'backup' > + && $vmid > + && $attribute eq 'protected' > + && $value > + && !$plugin->get_volume_attribute($scfg, $storeid, $volname, 'protected= ') > + && $max_protected_backups > -1 # -1 is unlimited > + ) { > + my $backups =3D $plugin->list_volumes($storeid, $scfg, $vmid, ['backup'= ]); > + my ($backup_type) =3D map { $_->{subtype} } grep { $_->{volid} eq $voli= d } $backups->@*; > + > + my $protected_count =3D grep { > + $_->{protected} && (!$backup_type || ($_->{subtype} && $_->{subtype= } eq $backup_type)) > + } $backups->@*; > + > + if ($max_protected_backups <=3D $protected_count) { > + die "The number of protected backups per guest is limited to $max_p= rotected_backups ". > + "on storage '$storeid'\n"; > + } > + } > + > return $plugin->update_volume_attribute($scfg, $storeid, $volname, $= attribute, $value); > } > =20 > diff --git a/PVE/Storage/BTRFSPlugin.pm b/PVE/Storage/BTRFSPlugin.pm > index c8caa41..7dac34b 100644 > --- a/PVE/Storage/BTRFSPlugin.pm > +++ b/PVE/Storage/BTRFSPlugin.pm > @@ -67,7 +67,8 @@ sub options { > shared =3D> { optional =3D> 1 }, > disable =3D> { optional =3D> 1 }, > maxfiles =3D> { optional =3D> 1 }, > - 'prune-backups'=3D> { optional =3D> 1 }, > + 'prune-backups' =3D> { optional =3D> 1 }, > + 'max-protected-backups' =3D> { optional =3D> 1 }, > content =3D> { optional =3D> 1 }, > format =3D> { optional =3D> 1 }, > is_mountpoint =3D> { optional =3D> 1 }, > diff --git a/PVE/Storage/CIFSPlugin.pm b/PVE/Storage/CIFSPlugin.pm > index d5efa5f..982040a 100644 > --- a/PVE/Storage/CIFSPlugin.pm > +++ b/PVE/Storage/CIFSPlugin.pm > @@ -134,6 +134,7 @@ sub options { > disable =3D> { optional =3D> 1 }, > maxfiles =3D> { optional =3D> 1 }, > 'prune-backups' =3D> { optional =3D> 1 }, > + 'max-protected-backups' =3D> { optional =3D> 1 }, > content =3D> { optional =3D> 1 }, > format =3D> { optional =3D> 1 }, > username =3D> { optional =3D> 1 }, > diff --git a/PVE/Storage/CephFSPlugin.pm b/PVE/Storage/CephFSPlugin.pm > index f75c1b8..4976747 100644 > --- a/PVE/Storage/CephFSPlugin.pm > +++ b/PVE/Storage/CephFSPlugin.pm > @@ -155,6 +155,7 @@ sub options { > maxfiles =3D> { optional =3D> 1 }, > keyring =3D> { optional =3D> 1 }, > 'prune-backups' =3D> { optional =3D> 1 }, > + 'max-protected-backups' =3D> { optional =3D> 1 }, > 'fs-name' =3D> { optional =3D> 1 }, > }; > } > diff --git a/PVE/Storage/DirPlugin.pm b/PVE/Storage/DirPlugin.pm > index c60818b..1baad63 100644 > --- a/PVE/Storage/DirPlugin.pm > +++ b/PVE/Storage/DirPlugin.pm > @@ -58,6 +58,7 @@ sub options { > disable =3D> { optional =3D> 1 }, > maxfiles =3D> { optional =3D> 1 }, > 'prune-backups' =3D> { optional =3D> 1 }, > + 'max-protected-backups' =3D> { optional =3D> 1 }, > content =3D> { optional =3D> 1 }, > format =3D> { optional =3D> 1 }, > mkdir =3D> { optional =3D> 1 }, > diff --git a/PVE/Storage/GlusterfsPlugin.pm b/PVE/Storage/GlusterfsPlugin= .pm > index d8d2b88..ad386d2 100644 > --- a/PVE/Storage/GlusterfsPlugin.pm > +++ b/PVE/Storage/GlusterfsPlugin.pm > @@ -133,6 +133,7 @@ sub options { > disable =3D> { optional =3D> 1 }, > maxfiles =3D> { optional =3D> 1 }, > 'prune-backups' =3D> { optional =3D> 1 }, > + 'max-protected-backups' =3D> { optional =3D> 1 }, > content =3D> { optional =3D> 1 }, > format =3D> { optional =3D> 1 }, > mkdir =3D> { optional =3D> 1 }, > diff --git a/PVE/Storage/NFSPlugin.pm b/PVE/Storage/NFSPlugin.pm > index 0400c93..5bd7313 100644 > --- a/PVE/Storage/NFSPlugin.pm > +++ b/PVE/Storage/NFSPlugin.pm > @@ -85,6 +85,7 @@ sub options { > disable =3D> { optional =3D> 1 }, > maxfiles =3D> { optional =3D> 1 }, > 'prune-backups' =3D> { optional =3D> 1 }, > + 'max-protected-backups' =3D> { optional =3D> 1 }, > options =3D> { optional =3D> 1 }, > content =3D> { optional =3D> 1 }, > format =3D> { optional =3D> 1 }, > diff --git a/PVE/Storage/PBSPlugin.pm b/PVE/Storage/PBSPlugin.pm > index 4b3f349..687901f 100644 > --- a/PVE/Storage/PBSPlugin.pm > +++ b/PVE/Storage/PBSPlugin.pm > @@ -73,6 +73,7 @@ sub options { > 'master-pubkey' =3D> { optional =3D> 1 }, > maxfiles =3D> { optional =3D> 1 }, > 'prune-backups' =3D> { optional =3D> 1 }, > + 'max-protected-backups' =3D> { optional =3D> 1 }, > fingerprint =3D> { optional =3D> 1 }, > }; > } > diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm > index 6673409..521e3e9 100644 > --- a/PVE/Storage/Plugin.pm > +++ b/PVE/Storage/Plugin.pm > @@ -155,6 +155,13 @@ my $defaultData =3D { > optional =3D> 1, > }, > 'prune-backups' =3D> get_standard_option('prune-backups'), > + 'max-protected-backups' =3D> { > + description =3D> "Maximal number of protected backups per guest. Us= e '-1' for unlimited.", > + type =3D> 'integer', > + minimum =3D> -1, > + optional =3D> 1, > + default =3D> "Unlimited for users with Datastore.Allocate privilege= , 5 for other users", > + }, > shared =3D> { > description =3D> "Mark storage as shared.", > type =3D> 'boolean', > --=20 > 2.30.2 >=20 >=20 >=20 > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel >=20 >=20 >=20