From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <f.ebner@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 3C44F6184C
for <pve-devel@lists.proxmox.com>; Thu, 9 Jul 2020 14:46:31 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 31CE5128D6
for <pve-devel@lists.proxmox.com>; Thu, 9 Jul 2020 14:46:01 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[212.186.127.180])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id DBC3C1286D
for <pve-devel@lists.proxmox.com>; Thu, 9 Jul 2020 14:45:55 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 9FA8D44141
for <pve-devel@lists.proxmox.com>; Thu, 9 Jul 2020 14:45:55 +0200 (CEST)
From: Fabian Ebner <f.ebner@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Thu, 9 Jul 2020 14:45:41 +0200
Message-Id: <20200709124547.2913-2-f.ebner@proxmox.com>
X-Mailer: git-send-email 2.20.1
In-Reply-To: <20200709124547.2913-1-f.ebner@proxmox.com>
References: <20200709124547.2913-1-f.ebner@proxmox.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.150 Adjusted score from AWL reputation of From: address
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/,
medium trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH v4 storage 1/7] Introduce prune-backups property
for directory-based storages
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Thu, 09 Jul 2020 12:46:31 -0000
Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
Changes from v3:
* use property string validator
PVE/Storage/CIFSPlugin.pm | 1 +
PVE/Storage/CephFSPlugin.pm | 1 +
PVE/Storage/DirPlugin.pm | 5 +--
PVE/Storage/GlusterfsPlugin.pm | 5 +--
PVE/Storage/NFSPlugin.pm | 5 +--
PVE/Storage/PBSPlugin.pm | 1 +
PVE/Storage/Plugin.pm | 59 +++++++++++++++++++++++++++++++++-
7 files changed, 70 insertions(+), 7 deletions(-)
diff --git a/PVE/Storage/CIFSPlugin.pm b/PVE/Storage/CIFSPlugin.pm
index 72ec757..6edbc9d 100644
--- a/PVE/Storage/CIFSPlugin.pm
+++ b/PVE/Storage/CIFSPlugin.pm
@@ -134,6 +134,7 @@ sub options {
nodes => { optional => 1 },
disable => { optional => 1 },
maxfiles => { optional => 1 },
+ 'prune-backups' => { optional => 1 },
content => { optional => 1 },
format => { optional => 1 },
username => { optional => 1 },
diff --git a/PVE/Storage/CephFSPlugin.pm b/PVE/Storage/CephFSPlugin.pm
index 6575f4f..880ec05 100644
--- a/PVE/Storage/CephFSPlugin.pm
+++ b/PVE/Storage/CephFSPlugin.pm
@@ -150,6 +150,7 @@ sub options {
fuse => { optional => 1 },
bwlimit => { optional => 1 },
maxfiles => { optional => 1 },
+ 'prune-backups' => { optional => 1 },
};
}
diff --git a/PVE/Storage/DirPlugin.pm b/PVE/Storage/DirPlugin.pm
index 39760a8..3c81d24 100644
--- a/PVE/Storage/DirPlugin.pm
+++ b/PVE/Storage/DirPlugin.pm
@@ -49,10 +49,11 @@ sub properties {
sub options {
return {
path => { fixed => 1 },
- nodes => { optional => 1 },
+ nodes => { optional => 1 },
shared => { optional => 1 },
disable => { optional => 1 },
- maxfiles => { optional => 1 },
+ maxfiles => { optional => 1 },
+ 'prune-backups' => { optional => 1 },
content => { optional => 1 },
format => { optional => 1 },
mkdir => { optional => 1 },
diff --git a/PVE/Storage/GlusterfsPlugin.pm b/PVE/Storage/GlusterfsPlugin.pm
index 70ea4fc..2dd414d 100644
--- a/PVE/Storage/GlusterfsPlugin.pm
+++ b/PVE/Storage/GlusterfsPlugin.pm
@@ -129,9 +129,10 @@ sub options {
server2 => { optional => 1 },
volume => { fixed => 1 },
transport => { optional => 1 },
- nodes => { optional => 1 },
+ nodes => { optional => 1 },
disable => { optional => 1 },
- maxfiles => { optional => 1 },
+ maxfiles => { optional => 1 },
+ 'prune-backups' => { optional => 1 },
content => { optional => 1 },
format => { optional => 1 },
mkdir => { optional => 1 },
diff --git a/PVE/Storage/NFSPlugin.pm b/PVE/Storage/NFSPlugin.pm
index 82b0c5f..6abb24b 100644
--- a/PVE/Storage/NFSPlugin.pm
+++ b/PVE/Storage/NFSPlugin.pm
@@ -79,9 +79,10 @@ sub options {
path => { fixed => 1 },
server => { fixed => 1 },
export => { fixed => 1 },
- nodes => { optional => 1 },
+ nodes => { optional => 1 },
disable => { optional => 1 },
- maxfiles => { optional => 1 },
+ maxfiles => { optional => 1 },
+ 'prune-backups' => { optional => 1 },
options => { optional => 1 },
content => { optional => 1 },
format => { optional => 1 },
diff --git a/PVE/Storage/PBSPlugin.pm b/PVE/Storage/PBSPlugin.pm
index b236f6c..87b09f2 100644
--- a/PVE/Storage/PBSPlugin.pm
+++ b/PVE/Storage/PBSPlugin.pm
@@ -55,6 +55,7 @@ sub options {
password => { optional => 1 },
'encryption-key' => { optional => 1 },
maxfiles => { optional => 1 },
+ 'prune-backups' => { optional => 1 },
fingerprint => { optional => 1 },
};
}
diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm
index 7f04e85..6e321ae 100644
--- a/PVE/Storage/Plugin.pm
+++ b/PVE/Storage/Plugin.pm
@@ -10,7 +10,7 @@ use File::Basename;
use File::stat qw();
use PVE::Tools qw(run_command);
-use PVE::JSONSchema qw(get_standard_option);
+use PVE::JSONSchema qw(get_standard_option register_standard_option);
use PVE::Cluster qw(cfs_register_file);
use JSON;
@@ -43,6 +43,62 @@ cfs_register_file ('storage.cfg',
sub { __PACKAGE__->parse_config(@_); },
sub { __PACKAGE__->write_config(@_); });
+my %prune_option = (
+ optional => 1,
+ type => 'integer', minimum => '0',
+ format_description => 'N',
+);
+
+my $prune_backups_format = {
+ 'keep-last' => {
+ %prune_option,
+ description => 'Keep the last <N> backups.',
+ },
+ 'keep-hourly' => {
+ %prune_option,
+ description => 'Keep backups for the last <N> different hours. If there is more' .
+ 'than one backup for a single hour, only the latest one is kept.'
+ },
+ 'keep-daily' => {
+ %prune_option,
+ description => 'Keep backups for the last <N> different days. If there is more' .
+ 'than one backup for a single day, only the latest one is kept.'
+ },
+ 'keep-weekly' => {
+ %prune_option,
+ description => 'Keep backups for the last <N> different weeks. If there is more' .
+ 'than one backup for a single week, only the latest one is kept.'
+ },
+ 'keep-monthly' => {
+ %prune_option,
+ description => 'Keep backups for the last <N> different months. If there is more' .
+ 'than one backup for a single month, only the latest one is kept.'
+ },
+ 'keep-yearly' => {
+ %prune_option,
+ description => 'Keep backups for the last <N> different years. If there is more' .
+ 'than one backup for a single year, only the latest one is kept.'
+ },
+};
+PVE::JSONSchema::register_format('prune-backups', $prune_backups_format, \&validate_prune_backups);
+sub validate_prune_backups {
+ my ($keep) = @_;
+
+ die "at least one keep-option must be set and positive\n"
+ if !grep { $_ } values %{$keep};
+
+ return $keep;
+}
+register_standard_option('prune-backups', {
+ description => "The retention options with shorter intervals are processed first " .
+ "with --keep-last being the very first one. Each option covers a " .
+ "specific period of time. We say that backups within this period " .
+ "are covered by this option. The next option does not take care " .
+ "of already covered backups and only considers older backups.",
+ optional => 1,
+ type => 'string',
+ format => 'prune-backups',
+});
my $defaultData = {
propertyList => {
@@ -68,6 +124,7 @@ my $defaultData = {
minimum => 0,
optional => 1,
},
+ 'prune-backups' => get_standard_option('prune-backups'),
shared => {
description => "Mark storage as shared.",
type => 'boolean',
--
2.20.1