From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <f.gruenbichler@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 21985B115
for <pve-devel@lists.proxmox.com>; Wed, 6 Apr 2022 12:42:41 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 14BA529673
for <pve-devel@lists.proxmox.com>; Wed, 6 Apr 2022 12:42:41 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[94.136.29.106])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id 9615A2966A
for <pve-devel@lists.proxmox.com>; Wed, 6 Apr 2022 12:42:39 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 5C4B141E5F
for <pve-devel@lists.proxmox.com>; Wed, 6 Apr 2022 12:42:39 +0200 (CEST)
Date: Wed, 06 Apr 2022 12:42:14 +0200
From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= <f.gruenbichler@proxmox.com>
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
References: <20220329125324.120737-1-f.ebner@proxmox.com>
<20220329125324.120737-2-f.ebner@proxmox.com>
In-Reply-To: <20220329125324.120737-2-f.ebner@proxmox.com>
MIME-Version: 1.0
User-Agent: astroid/0.15.0 (https://github.com/astroidmail/astroid)
Message-Id: <1649241726.hobppve4kh.astroid@nora.none>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.074 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
T_SCC_BODY_TEXT_LINE -0.01 -
URI_NOVOWEL 0.5 URI hostname has long non-vowel sequence
Subject: [pve-devel] applied: [PATCH v2 storage 1/1] plugins: allow limiting
the number of protected backups per guest
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Wed, 06 Apr 2022 10:42:41 -0000
On March 29, 2022 2:53 pm, Fabian Ebner wrote:
> The ability to mark backups as protected broke the implicit assumption
> in vzdump that remove=3D1 and current number of backups being the limit
> (i.e. sum of all keep options) will result in a backup being removed.
>=20
> Introduce a new storage property 'max-protected-backups' to limit the
> number of protected backups per guest. Use 5 as a default value, as it
> should cover most use cases, while still not having too big of a
> potential overhead in many scenarios.
>=20
> For external plugins that do not return the backup subtype in
> list_volumes, all protected backups with the same ID will count
> towards the limit.
>=20
> An alternative would be to count the protected backups when pruning.
> While that would avoid the need for a new property, it would break the
> current semantics of protected backups being ignored for pruning. It
> also would be less flexible, e.g. for PBS, it can make sense to have
> both keep-all=3D1 and a limit for the number of protected snapshots on
> the PVE side.
>=20
> Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
> ---
>=20
> Changes from v1:
> * Use different default depending on privilege level.
>=20
> PVE/Storage.pm | 35 ++++++++++++++++++++++++++++++++++
> PVE/Storage/BTRFSPlugin.pm | 3 ++-
> PVE/Storage/CIFSPlugin.pm | 1 +
> PVE/Storage/CephFSPlugin.pm | 1 +
> PVE/Storage/DirPlugin.pm | 1 +
> PVE/Storage/GlusterfsPlugin.pm | 1 +
> PVE/Storage/NFSPlugin.pm | 1 +
> PVE/Storage/PBSPlugin.pm | 1 +
> PVE/Storage/Plugin.pm | 7 +++++++
> 9 files changed, 50 insertions(+), 1 deletion(-)
>=20
> diff --git a/PVE/Storage.pm b/PVE/Storage.pm
> index 6112991..1a4454c 100755
> --- a/PVE/Storage.pm
> +++ b/PVE/Storage.pm
> @@ -211,6 +211,17 @@ sub storage_can_replicate {
> return $plugin->storage_can_replicate($scfg, $storeid, $format);
> }
> =20
> +sub get_max_protected_backups {
> + my ($scfg, $storeid) =3D @_;
> +
> + return $scfg->{'max-protected-backups'} if defined($scfg->{'max-prot=
ected-backups'});
> +
> + my $rpcenv =3D PVE::RPCEnvironment::get();
> + my $authuser =3D $rpcenv->get_user();
> +
> + return $rpcenv->check($authuser, "/storage/$storeid", ['Datastore.Al=
locate'], 1) ? -1 : 5;
> +}
> +
> sub storage_ids {
> my ($cfg) =3D @_;
> =20
> @@ -240,6 +251,30 @@ sub update_volume_attribute {
> my $scfg =3D storage_config($cfg, $storeid);
> my $plugin =3D PVE::Storage::Plugin->lookup($scfg->{type});
> =20
> + my ($vtype, undef, $vmid) =3D $plugin->parse_volname($volname);
> + my $max_protected_backups =3D get_max_protected_backups($scfg, $stor=
eid);
> +
> + if (
> + $vtype eq 'backup'
> + && $vmid
> + && $attribute eq 'protected'
> + && $value
> + && !$plugin->get_volume_attribute($scfg, $storeid, $volname, 'protected=
')
> + && $max_protected_backups > -1 # -1 is unlimited
> + ) {
> + my $backups =3D $plugin->list_volumes($storeid, $scfg, $vmid, ['backup'=
]);
> + my ($backup_type) =3D map { $_->{subtype} } grep { $_->{volid} eq $voli=
d } $backups->@*;
> +
> + my $protected_count =3D grep {
> + $_->{protected} && (!$backup_type || ($_->{subtype} && $_->{subtype=
} eq $backup_type))
> + } $backups->@*;
> +
> + if ($max_protected_backups <=3D $protected_count) {
> + die "The number of protected backups per guest is limited to $max_p=
rotected_backups ".
> + "on storage '$storeid'\n";
> + }
> + }
> +
> return $plugin->update_volume_attribute($scfg, $storeid, $volname, $=
attribute, $value);
> }
> =20
> diff --git a/PVE/Storage/BTRFSPlugin.pm b/PVE/Storage/BTRFSPlugin.pm
> index c8caa41..7dac34b 100644
> --- a/PVE/Storage/BTRFSPlugin.pm
> +++ b/PVE/Storage/BTRFSPlugin.pm
> @@ -67,7 +67,8 @@ sub options {
> shared =3D> { optional =3D> 1 },
> disable =3D> { optional =3D> 1 },
> maxfiles =3D> { optional =3D> 1 },
> - 'prune-backups'=3D> { optional =3D> 1 },
> + 'prune-backups' =3D> { optional =3D> 1 },
> + 'max-protected-backups' =3D> { optional =3D> 1 },
> content =3D> { optional =3D> 1 },
> format =3D> { optional =3D> 1 },
> is_mountpoint =3D> { optional =3D> 1 },
> diff --git a/PVE/Storage/CIFSPlugin.pm b/PVE/Storage/CIFSPlugin.pm
> index d5efa5f..982040a 100644
> --- a/PVE/Storage/CIFSPlugin.pm
> +++ b/PVE/Storage/CIFSPlugin.pm
> @@ -134,6 +134,7 @@ sub options {
> disable =3D> { optional =3D> 1 },
> maxfiles =3D> { optional =3D> 1 },
> 'prune-backups' =3D> { optional =3D> 1 },
> + 'max-protected-backups' =3D> { optional =3D> 1 },
> content =3D> { optional =3D> 1 },
> format =3D> { optional =3D> 1 },
> username =3D> { optional =3D> 1 },
> diff --git a/PVE/Storage/CephFSPlugin.pm b/PVE/Storage/CephFSPlugin.pm
> index f75c1b8..4976747 100644
> --- a/PVE/Storage/CephFSPlugin.pm
> +++ b/PVE/Storage/CephFSPlugin.pm
> @@ -155,6 +155,7 @@ sub options {
> maxfiles =3D> { optional =3D> 1 },
> keyring =3D> { optional =3D> 1 },
> 'prune-backups' =3D> { optional =3D> 1 },
> + 'max-protected-backups' =3D> { optional =3D> 1 },
> 'fs-name' =3D> { optional =3D> 1 },
> };
> }
> diff --git a/PVE/Storage/DirPlugin.pm b/PVE/Storage/DirPlugin.pm
> index c60818b..1baad63 100644
> --- a/PVE/Storage/DirPlugin.pm
> +++ b/PVE/Storage/DirPlugin.pm
> @@ -58,6 +58,7 @@ sub options {
> disable =3D> { optional =3D> 1 },
> maxfiles =3D> { optional =3D> 1 },
> 'prune-backups' =3D> { optional =3D> 1 },
> + 'max-protected-backups' =3D> { optional =3D> 1 },
> content =3D> { optional =3D> 1 },
> format =3D> { optional =3D> 1 },
> mkdir =3D> { optional =3D> 1 },
> diff --git a/PVE/Storage/GlusterfsPlugin.pm b/PVE/Storage/GlusterfsPlugin=
.pm
> index d8d2b88..ad386d2 100644
> --- a/PVE/Storage/GlusterfsPlugin.pm
> +++ b/PVE/Storage/GlusterfsPlugin.pm
> @@ -133,6 +133,7 @@ sub options {
> disable =3D> { optional =3D> 1 },
> maxfiles =3D> { optional =3D> 1 },
> 'prune-backups' =3D> { optional =3D> 1 },
> + 'max-protected-backups' =3D> { optional =3D> 1 },
> content =3D> { optional =3D> 1 },
> format =3D> { optional =3D> 1 },
> mkdir =3D> { optional =3D> 1 },
> diff --git a/PVE/Storage/NFSPlugin.pm b/PVE/Storage/NFSPlugin.pm
> index 0400c93..5bd7313 100644
> --- a/PVE/Storage/NFSPlugin.pm
> +++ b/PVE/Storage/NFSPlugin.pm
> @@ -85,6 +85,7 @@ sub options {
> disable =3D> { optional =3D> 1 },
> maxfiles =3D> { optional =3D> 1 },
> 'prune-backups' =3D> { optional =3D> 1 },
> + 'max-protected-backups' =3D> { optional =3D> 1 },
> options =3D> { optional =3D> 1 },
> content =3D> { optional =3D> 1 },
> format =3D> { optional =3D> 1 },
> diff --git a/PVE/Storage/PBSPlugin.pm b/PVE/Storage/PBSPlugin.pm
> index 4b3f349..687901f 100644
> --- a/PVE/Storage/PBSPlugin.pm
> +++ b/PVE/Storage/PBSPlugin.pm
> @@ -73,6 +73,7 @@ sub options {
> 'master-pubkey' =3D> { optional =3D> 1 },
> maxfiles =3D> { optional =3D> 1 },
> 'prune-backups' =3D> { optional =3D> 1 },
> + 'max-protected-backups' =3D> { optional =3D> 1 },
> fingerprint =3D> { optional =3D> 1 },
> };
> }
> diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm
> index 6673409..521e3e9 100644
> --- a/PVE/Storage/Plugin.pm
> +++ b/PVE/Storage/Plugin.pm
> @@ -155,6 +155,13 @@ my $defaultData =3D {
> optional =3D> 1,
> },
> 'prune-backups' =3D> get_standard_option('prune-backups'),
> + 'max-protected-backups' =3D> {
> + description =3D> "Maximal number of protected backups per guest. Us=
e '-1' for unlimited.",
> + type =3D> 'integer',
> + minimum =3D> -1,
> + optional =3D> 1,
> + default =3D> "Unlimited for users with Datastore.Allocate privilege=
, 5 for other users",
> + },
> shared =3D> {
> description =3D> "Mark storage as shared.",
> type =3D> 'boolean',
> --=20
> 2.30.2
>=20
>=20
>=20
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>=20
>=20
>=20