[pve-devel] [PATCH-SERIES] unify vzdump exclude-path behavior

[pve-devel] [PATCH-SERIES] unify vzdump exclude-path behavior

[Fabian Ebner]

across different modes and also with a pbs storage. For suspend mode relative
patterns are already working. To not break that behavior, and because pxar
and proxmox-backup-client also support relative patterns, allow them for the
other modes as well.


pve-guest-common:

Fabian Ebner (1):
  vzdump: update exclude-path description

 PVE/VZDump/Common.pm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)


pve-docs:

Fabian Ebner (1):
  vdzump: extend examples to also cover relative patterns

 vzdump.adoc | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)


pve-container:

Fabian Ebner (3):
  vzdump: turn findexcl into its own variable
  vzdump: allow relative exclude patterns for snapshot and stop mode
  vzdump: pass along exclude patterns to proxmox-backup-client

 src/PVE/VZDump/LXC.pm | 20 ++++++++++++++++++--
 1 file changed, 18 insertions(+), 2 deletions(-)

-- 
2.20.1

[pve-devel] [PATCH guest-common 1/5] vzdump: update exclude-path description

[Fabian Ebner]

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
 PVE/VZDump/Common.pm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/PVE/VZDump/Common.pm b/PVE/VZDump/Common.pm
index 63a4689..389473b 100644
--- a/PVE/VZDump/Common.pm
+++ b/PVE/VZDump/Common.pm
@@ -128,7 +128,9 @@ my $confdesc = {
     },
     'exclude-path' => {
 	type => 'string', format => 'string-alist',
-	description => "Exclude certain files/directories (shell globs).",
+	description => "Exclude certain files/directories (shell globs)." .
+	    " Paths starting with '/' are anchored to the container's root, " .
+	    " other paths match relative to each subdirectory.",
 	optional => 1,
     },
     mailto => {
-- 
2.20.1

[pve-devel] [PATCH docs 2/5] vdzump: extend examples to also cover relative patterns

[Fabian Ebner]

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
 vzdump.adoc | 11 ++++++++++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/vzdump.adoc b/vzdump.adoc
index 1c39680..a9a7b2f 100644
--- a/vzdump.adoc
+++ b/vzdump.adoc
@@ -288,7 +288,16 @@ You can also manually specify (additional) exclude paths, for example:
 
  # vzdump 777 --exclude-path /tmp/ --exclude-path '/var/foo*'
 
-(only excludes tmp directories)
+excludes the directory `/tmp/` and any file or directory named `/var/foo`,
+`/var/foobar`, and so on.
+
+Paths that do not start with a `/` are not anchored to the container's root,
+but will match relative to any subdirectory. For example:
+
+ # vzdump 777 --exclude-path bar
+
+excludes any file or directoy named `/bar`, `/var/bar`, `/var/foo/bar`, and
+so on, but not `/bar2`.
 
 Configuration files are also stored inside the backup archive
 (in `./etc/vzdump/`) and will be correctly restored.
-- 
2.20.1

[pve-devel] [PATCH container 3/5] vzdump: turn findexcl into its own variable

[Fabian Ebner]

for easier readability.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
 src/PVE/VZDump/LXC.pm | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/src/PVE/VZDump/LXC.pm b/src/PVE/VZDump/LXC.pm
index 05d8495..c02c6e1 100644
--- a/src/PVE/VZDump/LXC.pm
+++ b/src/PVE/VZDump/LXC.pm
@@ -369,6 +369,7 @@ sub archive {
     my $tmpdir = $task->{tmpdir};
 
     my $userns_cmd = $task->{userns_cmd};
+    my $findexcl = $self->{vzdump}->{findexcl};
 
     if ($self->{vzdump}->{opts}->{pbs}) {
 
@@ -427,7 +428,7 @@ sub archive {
 	push @$tar, "--directory=$snapdir";
 	push @$tar, '--no-anchored', '--exclude=lost+found' if $userns_cmd;
 	push @$tar, '--anchored';
-	push @$tar, map { "--exclude=.$_" } @{$self->{vzdump}->{findexcl}};
+	push @$tar, map { "--exclude=.$_" } @{$findexcl};
 
 	push @$tar, @sources;
 
-- 
2.20.1

[pve-devel] [PATCH container 4/5] vzdump: allow relative exclude patterns for snapshot and stop mode

[Fabian Ebner]

to make the behavior consistent across modes.

For suspend mode, relative patterns worked for a long time, because the
exclusion already happens when rsync copies the data during an earlier stage of
the backup.

For the other two methods, the way the patterns are passed to tar (after the
'--anchored' option and prefixed with a dot) meant that relative patterns
had no effect previously.

Users which have a relative exclude path by accident (if it's not by accident
then this fixes the behavior) and did not use suspend mode (if they did use
suspend mode, they hopefully would have noticed the unintended exclusion then)
will be affected by this change.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---

The alternative way to make things consistent is to warn/die on relative
patterns and not pass along relative patterns to rsync. That would be a breaking
change for the (likely few) existing users of relative patterns.

With the approach from the patch, the behavior is also consisten with pxar/manual
usage of proxmox-backup-client. And the next patch further unifies the behavior for
non-manual usage of the backup client.

 src/PVE/VZDump/LXC.pm | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/src/PVE/VZDump/LXC.pm b/src/PVE/VZDump/LXC.pm
index c02c6e1..eb07430 100644
--- a/src/PVE/VZDump/LXC.pm
+++ b/src/PVE/VZDump/LXC.pm
@@ -426,9 +426,23 @@ sub archive {
 	push @$tar, "--directory=$tmpdir", './etc/vzdump/pct.conf';
 	push @$tar, "./etc/vzdump/pct.fw" if $task->{fw};
 	push @$tar, "--directory=$snapdir";
-	push @$tar, '--no-anchored', '--exclude=lost+found' if $userns_cmd;
+
+	my @findexcl_no_anchored = ();
+	my @findexcl_anchored = ();
+	foreach my $pattern (@{$findexcl}) {
+	    if ($pattern !~ m|^/|) {
+		push @findexcl_no_anchored, $pattern;
+	    } else {
+		push @findexcl_anchored, $pattern;
+	    }
+	}
+
+	push @$tar, '--no-anchored';
+	push @$tar, '--exclude=lost+found' if $userns_cmd;
+	push @$tar, map { "--exclude=$_" } @findexcl_no_anchored;
+
 	push @$tar, '--anchored';
-	push @$tar, map { "--exclude=.$_" } @{$findexcl};
+	push @$tar, map { "--exclude=.$_" } @findexcl_anchored;
 
 	push @$tar, @sources;
 
-- 
2.20.1

[pve-devel] [PATCH container 5/5] vzdump: pass along exclude patterns to proxmox-backup-client

[Fabian Ebner]

to make the behavior consistent across modes.

Previously vzdump's exclude-path option only had an effect for suspend mode
backups, as then the exclusion already happens when rsync copies the data
during an earlier stage in the backup.

Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
---
 src/PVE/VZDump/LXC.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/PVE/VZDump/LXC.pm b/src/PVE/VZDump/LXC.pm
index eb07430..b7f7463 100644
--- a/src/PVE/VZDump/LXC.pm
+++ b/src/PVE/VZDump/LXC.pm
@@ -389,6 +389,7 @@ sub archive {
 	}
 
 	push @$param, '--skip-lost-and-found' if $userns_cmd;
+	push @$param, map { "--exclude=$_" } @$findexcl;
 
 	push @$param, '--backup-type', 'ct';
 	push @$param, '--backup-id', $vmid;
-- 
2.20.1

[pve-devel] applied-series: [PATCH-SERIES] unify vzdump exclude-path behavior

[Fabian Grünbichler]

might be worthy of a prominent changelog entry to avoid surprises.

On November 20, 2020 3:50 pm, Fabian Ebner wrote:
> across different modes and also with a pbs storage. For suspend mode relative
> patterns are already working. To not break that behavior, and because pxar
> and proxmox-backup-client also support relative patterns, allow them for the
> other modes as well.
> 
> 
> pve-guest-common:
> 
> Fabian Ebner (1):
>   vzdump: update exclude-path description
> 
>  PVE/VZDump/Common.pm | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)
> 
> 
> pve-docs:
> 
> Fabian Ebner (1):
>   vdzump: extend examples to also cover relative patterns
> 
>  vzdump.adoc | 11 ++++++++++-
>  1 file changed, 10 insertions(+), 1 deletion(-)
> 
> 
> pve-container:
> 
> Fabian Ebner (3):
>   vzdump: turn findexcl into its own variable
>   vzdump: allow relative exclude patterns for snapshot and stop mode
>   vzdump: pass along exclude patterns to proxmox-backup-client
> 
>  src/PVE/VZDump/LXC.pm | 20 ++++++++++++++++++--
>  1 file changed, 18 insertions(+), 2 deletions(-)
> 
> -- 
> 2.20.1
> 
> 
> 
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
> 
> 
>