public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error
@ 2026-11-18 12:21 Fiona Ebner
  2026-11-18 12:21 ` [PATCH qemu-server 1/3] " Fiona Ebner
                   ` (3 more replies)
  0 siblings, 4 replies; 5+ messages in thread
From: Fiona Ebner @ 2026-11-18 12:21 UTC (permalink / raw)
  To: pve-devel

The loop in the 'after-unfreeze' hook phase relies on the 'bytes'
value from the result of 'query-savevm' disappearing when the
'savevm-end' command is done. This does not happen in case of an
error status currently. Use a timeout of 35 seconds instead to ensure
that the loop stays bounded.

This phase was enabled by commit 568b295c ("fix #7697: snapshot hook
name mismatch") so this is an unintended regression.

Further improve the situation by having 'query-savevm' report if
'savevm-end' is still active and qemu-server checking for that.

qemu-server:

Fiona Ebner (2):
  fix #8331: snapshot: fix regression with endless loop after error
  snapshot: directly check for savevm-end status

 src/PVE/QemuConfig.pm | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)


qemu:

Fiona Ebner (1):
  async snapshot: keep track of savevm-end status

 migration/savevm-async.c | 13 +++++++++++++
 qapi/migration.json      |  6 +++++-
 2 files changed, 18 insertions(+), 1 deletion(-)


Summary over all repositories:
  3 files changed, 30 insertions(+), 3 deletions(-)

-- 
Generated by git-murpp 0.5.0




^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH qemu-server 1/3] fix #8331: snapshot: fix regression with endless loop after error
  2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner
@ 2026-11-18 12:21 ` Fiona Ebner
  2026-11-18 12:21 ` [PATCH qemu-server 2/3] snapshot: directly check for savevm-end status Fiona Ebner
                   ` (2 subsequent siblings)
  3 siblings, 0 replies; 5+ messages in thread
From: Fiona Ebner @ 2026-11-18 12:21 UTC (permalink / raw)
  To: pve-devel

The loop in the 'after-unfreeze' hook phase relies on the 'bytes'
value from the result of 'query-savevm' disappearing when the
'savevm-end' command is done. This does not happen in case of an
error status currently. Use a timeout of 35 seconds instead to ensure
that the loop stays bounded.

A further improvement requires QEMU to report more information with
'query-savevm'.

Fixes: 568b295c ("fix #7697: snapshot hook name mismatch")
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/QemuConfig.pm | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/src/PVE/QemuConfig.pm b/src/PVE/QemuConfig.pm
index d89a62e6..f35a49eb 100644
--- a/src/PVE/QemuConfig.pm
+++ b/src/PVE/QemuConfig.pm
@@ -390,11 +390,16 @@ sub __snapshot_create_vol_snapshots_hook {
             };
             warn $@ if $@;
         } elsif ($hook eq "after-unfreeze") {
-            # savevm-end is async, we need to wait
+            # savevm-end is async and waits for up to 30 seconds for closing the target.
+            my $timeout = 35;
+            my $start_time = time();
             for (;;) {
                 my $stat = mon_cmd($vmid, "query-savevm");
                 if (!$stat->{bytes}) {
                     last;
+                } elsif ((time() - $start_time) >= $timeout) {
+                    warn "timeout waiting for 'savevm-end' to complete\n";
+                    last;
                 } else {
                     print "waiting for QEMU to finish the 'savevm-end' command...\n";
                     sleep(1);
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH qemu-server 2/3] snapshot: directly check for savevm-end status
  2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner
  2026-11-18 12:21 ` [PATCH qemu-server 1/3] " Fiona Ebner
@ 2026-11-18 12:21 ` Fiona Ebner
  2026-11-18 12:21 ` [PATCH qemu 3/3] async snapshot: keep track of " Fiona Ebner
  2026-11-18 21:02 ` applied: [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Thomas Lamprecht
  3 siblings, 0 replies; 5+ messages in thread
From: Fiona Ebner @ 2026-11-18 12:21 UTC (permalink / raw)
  To: pve-devel

Since pve-qemu >= 11.1.3-3, 'query-savevm' reports if there is an
active 'savevm-end' handler. Check this status directly, while keeping
the proxy check for the presence of 'bytes' for compatibility with
older QEMU binaries. Checking for 'bytes' did not work as a sentinel
in case the snapshot ended with an error status, as the field was
always present.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

NOTE: the version in the commit message needs to be adapted depending
on the pve-qemu-kvm version for which patch 3/3 is applied.

 src/PVE/QemuConfig.pm | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/src/PVE/QemuConfig.pm b/src/PVE/QemuConfig.pm
index f35a49eb..a8279ac6 100644
--- a/src/PVE/QemuConfig.pm
+++ b/src/PVE/QemuConfig.pm
@@ -395,9 +395,14 @@ sub __snapshot_create_vol_snapshots_hook {
             my $start_time = time();
             for (;;) {
                 my $stat = mon_cmd($vmid, "query-savevm");
-                if (!$stat->{bytes}) {
-                    last;
-                } elsif ((time() - $start_time) >= $timeout) {
+
+                if (defined($stat->{'savevm-end-active'})) {
+                    last if !$stat->{'savevm-end-active'};
+                } else { # Fallback for older QEMU binaries that did not report savevm-end-active
+                    last if !$stat->{bytes};
+                }
+
+                if ((time() - $start_time) >= $timeout) {
                     warn "timeout waiting for 'savevm-end' to complete\n";
                     last;
                 } else {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH qemu 3/3] async snapshot: keep track of savevm-end status
  2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner
  2026-11-18 12:21 ` [PATCH qemu-server 1/3] " Fiona Ebner
  2026-11-18 12:21 ` [PATCH qemu-server 2/3] snapshot: directly check for savevm-end status Fiona Ebner
@ 2026-11-18 12:21 ` Fiona Ebner
  2026-11-18 21:02 ` applied: [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Thomas Lamprecht
  3 siblings, 0 replies; 5+ messages in thread
From: Fiona Ebner @ 2026-11-18 12:21 UTC (permalink / raw)
  To: pve-devel

If a snapshot ends with an error, qemu-server had no way to detect
when it's safe to consider 'savevm-end' done. The result of
'query-savevm' only changes when the status is set to SAVE_STATE_DONE,
not reporting the 'bytes' value anymore, and qemu-server looked at
that. Keep track of the 'savevm-end' state and report it to allow
distinguishing between 'savevm-end' being done or still waiting for
the target file close in case of an error.

Checking for an already active 'savevm-end' invocation at the
beginning of qmp_savevm_end() fixes a crash: when a second
'savevm-end' QMP call came in, while a previous one was still sleeping
in target_close_wait, the following assertion would trigger:
> kvm: ../util/qemu-coroutine-sleep.c:77: qemu_co_sleep: \
> Assertion `prev == NULL' failed.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 migration/savevm-async.c | 13 +++++++++++++
 qapi/migration.json      |  6 +++++-
 2 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/migration/savevm-async.c b/migration/savevm-async.c
index 3ab0bd4539..e9037173af 100644
--- a/migration/savevm-async.c
+++ b/migration/savevm-async.c
@@ -60,6 +60,7 @@ static struct SnapshotState {
     Coroutine *co;
     QemuCoSleep target_close_wait;
     IOThread *iothread;
+    bool savevm_end_active;
 } snap_state;
 
 static bool savevm_aborted(void)
@@ -77,6 +78,8 @@ SaveVMInfo *qmp_query_savevm(Error **errp)
     SaveVMInfo *info = g_malloc0(sizeof(*info));
     struct SnapshotState *s = &snap_state;
 
+    info->savevm_end_active = s->savevm_end_active;
+
     if (s->state != SAVE_STATE_DONE) {
         info->has_bytes = true;
         info->bytes = s->bs_pos;
@@ -498,6 +501,7 @@ static void coroutine_fn wait_for_close_co(void *opaque)
                                 "qmp_savevm_end");
             /* we cannot assume the snapshot finished in this case, so leave the
              * state alone - caller has to figure something out */
+            snap_state.savevm_end_active = false;
             return;
         }
     } else {
@@ -511,6 +515,8 @@ static void coroutine_fn wait_for_close_co(void *opaque)
         snap_state.state = SAVE_STATE_DONE;
     }
 
+    snap_state.savevm_end_active = false;
+
     DPRINTF("savevm-end: cleanup done\n");
 }
 
@@ -521,6 +527,13 @@ void qmp_savevm_end(Error **errp)
         return;
     }
 
+    if (snap_state.savevm_end_active) {
+        error_setg(errp, "savevm-end handler already running");
+        return;
+    }
+
+    snap_state.savevm_end_active = true;
+
     Coroutine *wait_for_close = qemu_coroutine_create(wait_for_close_co, NULL);
 
     if (snap_state.state == SAVE_STATE_ACTIVE) {
diff --git a/qapi/migration.json b/qapi/migration.json
index 0794a3a7a3..d7504a0d4d 100644
--- a/qapi/migration.json
+++ b/qapi/migration.json
@@ -354,11 +354,15 @@
 #
 # @bytes: total amount of data transfered
 #
+# @savevm-end-active: Whether a @savevm-end handler is currently
+#     active.
+#
 # Since: 1.3
 ##
 { 'struct': 'SaveVMInfo',
   'data': {'*status': 'str', '*error': 'str',
-           '*total-time': 'int', '*bytes': 'int'} }
+           '*total-time': 'int', '*bytes': 'int',
+           'savevm-end-active': 'bool'} }
 
 ##
 # @query-savevm:
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 5+ messages in thread

* applied: [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error
  2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner
                   ` (2 preceding siblings ...)
  2026-11-18 12:21 ` [PATCH qemu 3/3] async snapshot: keep track of " Fiona Ebner
@ 2026-11-18 21:02 ` Thomas Lamprecht
  3 siblings, 0 replies; 5+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 21:02 UTC (permalink / raw)
  To: pve-devel, Fiona Ebner

On Wed, 18 Nov 2026 13:21:46 +0100, Fiona Ebner wrote:
> The loop in the 'after-unfreeze' hook phase relies on the 'bytes'
> value from the result of 'query-savevm' disappearing when the
> 'savevm-end' command is done. This does not happen in case of an
> error status currently. Use a timeout of 35 seconds instead to ensure
> that the loop stays bounded.
> 
> This phase was enabled by commit 568b295c ("fix #7697: snapshot hook
> name mismatch") so this is an unintended regression.
> 
> [...]

Applied, thanks!

[qemu-server] applied to qemu-server.git (master)
[1/2] fix #8331: snapshot: fix regression with endless loop after error
      https://git.proxmox.com/?p=qemu-server.git;a=commitdiff;h=610bd239a85f54053a2708abab5be633120abc52
[2/2] snapshot: directly check for savevm-end status
      https://git.proxmox.com/?p=qemu-server.git;a=commitdiff;h=a906ed74688d825549dc50ec6cbce10d1cfa152e

[qemu] applied to git/pve-qemu (master)
[1/1] async snapshot: keep track of savevm-end status
      commit: 4494236f6b1da3955766707e327ba53c892284ea




^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2026-11-18 21:02 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner
2026-11-18 12:21 ` [PATCH qemu-server 1/3] " Fiona Ebner
2026-11-18 12:21 ` [PATCH qemu-server 2/3] snapshot: directly check for savevm-end status Fiona Ebner
2026-11-18 12:21 ` [PATCH qemu 3/3] async snapshot: keep track of " Fiona Ebner
2026-11-18 21:02 ` applied: [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Thomas Lamprecht

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal