[pve-devel] [PATCH-SERIES guest-common/docs v3] add pre/post/failed snapshot hooks

[pve-devel] [PATCH-SERIES guest-common/docs v3] add pre/post/failed snapshot hooks

[Stefan Hanreich]

This patch series introduces the pre/post/failed-snapshot hooks that run before/
after a snapshot is taken, or after failing to take a snapshot.

I used the new example script from pve-docs as template for my test hookscripts.

What I tested:
* Normal snapshotting, without VM state, without hookscript
  * snapshot works, no hooks executed
* Normal snapshotting, with VM state, without hookscript
  * snapshot works, no hooks executed
* Normal snapshotting, without VM state, with hookscript
  * snapshot works, pre/post hooks work
* Normal snapshotting, with VM state, with hookscript
  * snapshot works, pre/post hooks work
* Taking snapshot with existing name, with hookscript
  * fails, no hookscripts get executed
* Failed at wrong storage config, with hookscript
  * pre/failed get executed, lock gets released
* Failed at taking RAM Snapshot (simulated with monkey-patched die), with hookscript
  * pre/failed get executed, lock gets released
* pre/post hookscript that detaches/attaches unsnapshottable disk (without --skiplock)
  * snapshotting fails, attach/detach fails, pre/failed get executed, lock released
* pre/post hookscript that detaches/attaches unsnapshottable disk (with --skiplock)
  * snapshotting works, attach/detach works, pre/post get executed
  * restoring works, detached disk is detached after restoring
* pre-snapshot hookscript exits with code > 0
  * pre/failed-snapshot get executed, snapshot fails, lock gets released
* post-snapshot hookscript exits with code > 0
  * pre/post/failed-snapshot get executed, snapshot fails, lock gets released
* Taking snapshot of template
  * fails, no hookscripts get executed
* execute commands in VM in pre/post via qm guest exec
  * snapshot succeeds, commands get executed, pre/post executed

Changes from v2 (thanks fabian!):
* added guards, so cfs_update() only gets called when necessary
* added PVE_SNAPSHOT_PHASE envvar to failed-snapshot as indicator
  when the failure occured

Changes from v1:
* added failed-snapshot hook that runs after a failed snapshot
  * this enables users to revert any changes made in pre-snapshot hooks
  in case of errors
* running cfs_update() after every hookscript invocation
* adjusted the call sites of exec_hookscript()
  * particularly interesting for pre-snapshot since some checks now run
  before the hook runs
* VM/CT config is now locked during hookscript execution

Thanks to Fiona and Fabian for their valuable input/help!

pve-guest-common:

Stefan Hanreich (1):
  examples: add pre/post/failed-snapshot hooks to example hookscript

 examples/guest-example-hookscript.pl | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)


pve-docs:

Stefan Hanreich (1):
  partial fix #2530: snapshots: add pre/post/failed-snapshot hooks

 src/PVE/AbstractConfig.pm | 76 +++++++++++++++++++++++++++++++++++----
 1 file changed, 70 insertions(+), 6 deletions(-)

-- 
2.30.2

[pve-devel] [PATCH pve-docs v3 1/1] examples: add pre/post/failed-snapshot hooks to example hookscript

[Stefan Hanreich]

Added a section for each new snapshot hook to the example hookscript,
as well as a short comment explaining when the respective section gets
executed. Additionally added documentation for the different possible
values of the envvar PVE_SNAPSHOT_PHASE.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 examples/guest-example-hookscript.pl | 26 ++++++++++++++++++++++++++
 1 file changed, 26 insertions(+)

diff --git a/examples/guest-example-hookscript.pl b/examples/guest-example-hookscript.pl
index adeed59..51c1d61 100755
--- a/examples/guest-example-hookscript.pl
+++ b/examples/guest-example-hookscript.pl
@@ -54,6 +54,32 @@ if ($phase eq 'pre-start') {
 
     print "$vmid stopped. Doing cleanup.\n";
 
+} elsif ($phase eq 'pre-snapshot') {
+
+    # Phase 'pre-snapshot' will be executed before taking a snapshot of
+    # the guest (via UI or CLI)
+
+    print "$vmid will be snapshotted.\n";
+
+} elsif ($phase eq 'post-snapshot') {
+
+    # Phase 'post-snapshot' will be executed after taking a snapshot of
+    # the guest (via UI or CLI)
+
+    print "$vmid has been successfully snapshotted.\n";
+
+} elsif ($phase eq 'failed-snapshot') {
+
+    # Phase 'failed-snapshot' will be executed when taking a snapshot of
+    # the guest fails and 'pre-snapshot' already ran (via UI or CLI)
+    # Envvar PVE_SNAPSHOT_PHASE can be one of the following:
+    # - pre-snapshot (when pre-snapshot hook failed)
+    # - prepare (when preparation step failed)
+    # - snapshot (when snapshotting the disks failed)
+    # - post-snapshot (when post-snapshot hook failed)
+
+    print "$vmid snapshot failed in phase $ENV{PVE_SNAPSHOT_PHASE}.\n";
+
 } else {
     die "got unknown phase '$phase'\n";
 }
-- 
2.30.2

[pve-devel] [PATCH pve-guest-common v3 1/1] partial fix #2530: snapshots: add pre/post/failed-snapshot hooks

[Stefan Hanreich]

This commit adds hooks to the snapshotting process, which can be used to
run additional setup scripts to prepare the VM for snapshotting.

Examples for use cases include:
- forcing processes to flush their writes
- blocking processes from writing
- altering the configuration of the VM to make snapshotting possible

The prepare step has been split into two parts, so the configuration can
be locked a bit earlier during the snapshotting process. Doing it this
way ensures that the configuration is already locked during the
pre-snapshot hook. Because of this split, the VM config gets written in
two stages now, rather than one.

In case of failure during the preparation step - after the lock is
written - error handling has been added so the lock gets released
properly. The failed-snapshot hook runs when the snapshot or a snapshot
hook fails, but only if the pre-snapshot hook ran already. This enables
users to revert any changes done during the pre-snapshot hookscript.

The preparation step assumes that the hook does not convert the current
VM into a template, which is why the basic checks are not re-run after
the pre-snapshot hook. The storage check runs after the pre-snapshot
hook, because the hook might get used to setup the storage for
snapshotting. If the hook would run after the storage checks, this
becomes impossible.

cfs_update() gets called after every invocation of a hookscript, since
it is impossible to know which changes get made by the hookscript.
Doing this ensures that we see the updated state of the CFS after the
hookscript got invoked.

Signed-off-by: Stefan Hanreich <s.hanreich@proxmox.com>
---
 src/PVE/AbstractConfig.pm | 76 +++++++++++++++++++++++++++++++++++----
 1 file changed, 70 insertions(+), 6 deletions(-)

diff --git a/src/PVE/AbstractConfig.pm b/src/PVE/AbstractConfig.pm
index a0c0bc6..9763c0c 100644
--- a/src/PVE/AbstractConfig.pm
+++ b/src/PVE/AbstractConfig.pm
@@ -704,14 +704,28 @@ sub __snapshot_apply_config {
     return $newconf;
 }
 
+sub __snapshot_failed {
+    my ($class, $vmid, $phase) = @_;
+
+    my $conf = $class->load_config($vmid);
+
+    return if !$conf->{hookscript};
+
+    {
+	local $ENV{'PVE_SNAPSHOT_PHASE'} = $phase;
+	PVE::GuestHelpers::exec_hookscript($conf, $vmid, "failed-snapshot");
+    }
+
+    PVE::Cluster::cfs_update();
+}
+
 # Prepares the configuration for snapshotting.
 sub __snapshot_prepare {
     my ($class, $vmid, $snapname, $save_vmstate, $comment) = @_;
 
     my $snap;
 
-    my $updatefn =  sub {
-
+    my $run_checks = sub {
 	my $conf = $class->load_config($vmid);
 
 	die "you can't take a snapshot if it's a template\n"
@@ -721,15 +735,21 @@ sub __snapshot_prepare {
 
 	$conf->{lock} = 'snapshot';
 
-	my $snapshots = $conf->{snapshots};
-
 	die "snapshot name '$snapname' already used\n"
-	    if defined($snapshots->{$snapname});
+	    if defined($conf->{snapshots}->{$snapname});
 
+	$class->write_config($vmid, $conf);
+    };
+
+    my $updatefn = sub {
+	my $conf = $class->load_config($vmid);
 	my $storecfg = PVE::Storage::config();
+
 	die "snapshot feature is not available\n"
 	    if !$class->has_feature('snapshot', $conf, $storecfg, undef, undef, $snapname eq 'vzdump');
 
+	my $snapshots = $conf->{snapshots};
+
 	for my $snap (sort keys %$snapshots) {
 	    my $parent_name = $snapshots->{$snap}->{parent} // '';
 	    if ($snapname eq $parent_name) {
@@ -753,7 +773,33 @@ sub __snapshot_prepare {
 	$class->write_config($vmid, $conf);
     };
 
-    $class->lock_config($vmid, $updatefn);
+    $class->lock_config($vmid, $run_checks);
+
+    my $conf = $class->load_config($vmid);
+
+    if ($conf->{hookscript}) {
+	eval {
+	    PVE::GuestHelpers::exec_hookscript($conf, $vmid, "pre-snapshot", 1);
+	};
+	my $err = $@;
+
+	PVE::Cluster::cfs_update();
+
+	if ($err) {
+	    $class->__snapshot_failed($vmid, 'pre-snapshot');
+	    $class->remove_lock($vmid, 'snapshot');
+	    die $err;
+	}
+    }
+
+    eval {
+	$class->lock_config($vmid, $updatefn);
+    };
+    if (my $err = $@) {
+	$class->__snapshot_failed($vmid, 'prepare');
+	$class->remove_lock($vmid, 'snapshot');
+	die $err;
+    }
 
     return $snap;
 }
@@ -837,11 +883,29 @@ sub snapshot_create {
 
     if ($err) {
 	warn "snapshot create failed: starting cleanup\n";
+
 	eval { $class->snapshot_delete($vmid, $snapname, 1, $drivehash); };
 	warn "$@" if $@;
+
+	$class->__snapshot_failed($vmid, 'snapshot');
+
 	die "$err\n";
     }
 
+    if ($conf->{hookscript}) {
+	eval {
+	    PVE::GuestHelpers::exec_hookscript($conf, $vmid, "post-snapshot", 1);
+	};
+	$err = $@;
+
+	PVE::Cluster::cfs_update();
+
+	if ($err) {
+	    warn $err;
+	    $class->__snapshot_failed($vmid, 'post-snapshot');
+	}
+    }
+
     $class->__snapshot_commit($vmid, $snapname);
 }
 
-- 
2.30.2

Re: [pve-devel] [PATCH-SERIES guest-common/docs v3] add pre/post/failed snapshot hooks

[Stefan Hanreich]

On 1/23/23 16:58, Stefan Hanreich wrote:
> This patch series introduces the pre/post/failed-snapshot hooks that run before/
> after a snapshot is taken, or after failing to take a snapshot.
> 
> I used the new example script from pve-docs as template for my test hookscripts.
> 
> What I tested:
> * Normal snapshotting, without VM state, without hookscript
>    * snapshot works, no hooks executed
> * Normal snapshotting, with VM state, without hookscript
>    * snapshot works, no hooks executed
> * Normal snapshotting, without VM state, with hookscript
>    * snapshot works, pre/post hooks work
> * Normal snapshotting, with VM state, with hookscript
>    * snapshot works, pre/post hooks work
> * Taking snapshot with existing name, with hookscript
>    * fails, no hookscripts get executed
> * Failed at wrong storage config, with hookscript
>    * pre/failed get executed, lock gets released
> * Failed at taking RAM Snapshot (simulated with monkey-patched die), with hookscript
>    * pre/failed get executed, lock gets released
> * pre/post hookscript that detaches/attaches unsnapshottable disk (without --skiplock)
>    * snapshotting fails, attach/detach fails, pre/failed get executed, lock released
> * pre/post hookscript that detaches/attaches unsnapshottable disk (with --skiplock)
>    * snapshotting works, attach/detach works, pre/post get executed
>    * restoring works, detached disk is detached after restoring
> * pre-snapshot hookscript exits with code > 0
>    * pre/failed-snapshot get executed, snapshot fails, lock gets released
> * post-snapshot hookscript exits with code > 0
>    * pre/post/failed-snapshot get executed, snapshot fails, lock gets released
> * Taking snapshot of template
>    * fails, no hookscripts get executed
> * execute commands in VM in pre/post via qm guest exec
>    * snapshot succeeds, commands get executed, pre/post executed
> 
> Changes from v2 (thanks fabian!):
> * added guards, so cfs_update() only gets called when necessary
> * added PVE_SNAPSHOT_PHASE envvar to failed-snapshot as indicator
>    when the failure occured
> 
> Changes from v1:
> * added failed-snapshot hook that runs after a failed snapshot
>    * this enables users to revert any changes made in pre-snapshot hooks
>    in case of errors
> * running cfs_update() after every hookscript invocation
> * adjusted the call sites of exec_hookscript()
>    * particularly interesting for pre-snapshot since some checks now run
>    before the hook runs
> * VM/CT config is now locked during hookscript execution
> 
> Thanks to Fiona and Fabian for their valuable input/help!
> 
> pve-guest-common:
> 

flipped this heading with pve-docs

> Stefan Hanreich (1):
>    examples: add pre/post/failed-snapshot hooks to example hookscript
> 
>   examples/guest-example-hookscript.pl | 26 ++++++++++++++++++++++++++
>   1 file changed, 26 insertions(+)
> 
> 
> pve-docs:

^ see above

> 
> Stefan Hanreich (1):
>    partial fix #2530: snapshots: add pre/post/failed-snapshot hooks
> 
>   src/PVE/AbstractConfig.pm | 76 +++++++++++++++++++++++++++++++++++----
>   1 file changed, 70 insertions(+), 6 deletions(-)
>