* [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error
@ 2026-11-18 12:21 Fiona Ebner
2026-11-18 12:21 ` [PATCH qemu-server 1/3] " Fiona Ebner
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Fiona Ebner @ 2026-11-18 12:21 UTC (permalink / raw)
To: pve-devel
The loop in the 'after-unfreeze' hook phase relies on the 'bytes'
value from the result of 'query-savevm' disappearing when the
'savevm-end' command is done. This does not happen in case of an
error status currently. Use a timeout of 35 seconds instead to ensure
that the loop stays bounded.
This phase was enabled by commit 568b295c ("fix #7697: snapshot hook
name mismatch") so this is an unintended regression.
Further improve the situation by having 'query-savevm' report if
'savevm-end' is still active and qemu-server checking for that.
qemu-server:
Fiona Ebner (2):
fix #8331: snapshot: fix regression with endless loop after error
snapshot: directly check for savevm-end status
src/PVE/QemuConfig.pm | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
qemu:
Fiona Ebner (1):
async snapshot: keep track of savevm-end status
migration/savevm-async.c | 13 +++++++++++++
qapi/migration.json | 6 +++++-
2 files changed, 18 insertions(+), 1 deletion(-)
Summary over all repositories:
3 files changed, 30 insertions(+), 3 deletions(-)
--
Generated by git-murpp 0.5.0
^ permalink raw reply [flat|nested] 5+ messages in thread* [PATCH qemu-server 1/3] fix #8331: snapshot: fix regression with endless loop after error 2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner @ 2026-11-18 12:21 ` Fiona Ebner 2026-11-18 12:21 ` [PATCH qemu-server 2/3] snapshot: directly check for savevm-end status Fiona Ebner ` (2 subsequent siblings) 3 siblings, 0 replies; 5+ messages in thread From: Fiona Ebner @ 2026-11-18 12:21 UTC (permalink / raw) To: pve-devel The loop in the 'after-unfreeze' hook phase relies on the 'bytes' value from the result of 'query-savevm' disappearing when the 'savevm-end' command is done. This does not happen in case of an error status currently. Use a timeout of 35 seconds instead to ensure that the loop stays bounded. A further improvement requires QEMU to report more information with 'query-savevm'. Fixes: 568b295c ("fix #7697: snapshot hook name mismatch") Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> --- src/PVE/QemuConfig.pm | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/PVE/QemuConfig.pm b/src/PVE/QemuConfig.pm index d89a62e6..f35a49eb 100644 --- a/src/PVE/QemuConfig.pm +++ b/src/PVE/QemuConfig.pm @@ -390,11 +390,16 @@ sub __snapshot_create_vol_snapshots_hook { }; warn $@ if $@; } elsif ($hook eq "after-unfreeze") { - # savevm-end is async, we need to wait + # savevm-end is async and waits for up to 30 seconds for closing the target. + my $timeout = 35; + my $start_time = time(); for (;;) { my $stat = mon_cmd($vmid, "query-savevm"); if (!$stat->{bytes}) { last; + } elsif ((time() - $start_time) >= $timeout) { + warn "timeout waiting for 'savevm-end' to complete\n"; + last; } else { print "waiting for QEMU to finish the 'savevm-end' command...\n"; sleep(1); -- 2.47.3 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH qemu-server 2/3] snapshot: directly check for savevm-end status 2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner 2026-11-18 12:21 ` [PATCH qemu-server 1/3] " Fiona Ebner @ 2026-11-18 12:21 ` Fiona Ebner 2026-11-18 12:21 ` [PATCH qemu 3/3] async snapshot: keep track of " Fiona Ebner 2026-11-18 21:02 ` applied: [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Thomas Lamprecht 3 siblings, 0 replies; 5+ messages in thread From: Fiona Ebner @ 2026-11-18 12:21 UTC (permalink / raw) To: pve-devel Since pve-qemu >= 11.1.3-3, 'query-savevm' reports if there is an active 'savevm-end' handler. Check this status directly, while keeping the proxy check for the presence of 'bytes' for compatibility with older QEMU binaries. Checking for 'bytes' did not work as a sentinel in case the snapshot ended with an error status, as the field was always present. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> --- NOTE: the version in the commit message needs to be adapted depending on the pve-qemu-kvm version for which patch 3/3 is applied. src/PVE/QemuConfig.pm | 11 ++++++++--- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/src/PVE/QemuConfig.pm b/src/PVE/QemuConfig.pm index f35a49eb..a8279ac6 100644 --- a/src/PVE/QemuConfig.pm +++ b/src/PVE/QemuConfig.pm @@ -395,9 +395,14 @@ sub __snapshot_create_vol_snapshots_hook { my $start_time = time(); for (;;) { my $stat = mon_cmd($vmid, "query-savevm"); - if (!$stat->{bytes}) { - last; - } elsif ((time() - $start_time) >= $timeout) { + + if (defined($stat->{'savevm-end-active'})) { + last if !$stat->{'savevm-end-active'}; + } else { # Fallback for older QEMU binaries that did not report savevm-end-active + last if !$stat->{bytes}; + } + + if ((time() - $start_time) >= $timeout) { warn "timeout waiting for 'savevm-end' to complete\n"; last; } else { -- 2.47.3 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PATCH qemu 3/3] async snapshot: keep track of savevm-end status 2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner 2026-11-18 12:21 ` [PATCH qemu-server 1/3] " Fiona Ebner 2026-11-18 12:21 ` [PATCH qemu-server 2/3] snapshot: directly check for savevm-end status Fiona Ebner @ 2026-11-18 12:21 ` Fiona Ebner 2026-11-18 21:02 ` applied: [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Thomas Lamprecht 3 siblings, 0 replies; 5+ messages in thread From: Fiona Ebner @ 2026-11-18 12:21 UTC (permalink / raw) To: pve-devel If a snapshot ends with an error, qemu-server had no way to detect when it's safe to consider 'savevm-end' done. The result of 'query-savevm' only changes when the status is set to SAVE_STATE_DONE, not reporting the 'bytes' value anymore, and qemu-server looked at that. Keep track of the 'savevm-end' state and report it to allow distinguishing between 'savevm-end' being done or still waiting for the target file close in case of an error. Checking for an already active 'savevm-end' invocation at the beginning of qmp_savevm_end() fixes a crash: when a second 'savevm-end' QMP call came in, while a previous one was still sleeping in target_close_wait, the following assertion would trigger: > kvm: ../util/qemu-coroutine-sleep.c:77: qemu_co_sleep: \ > Assertion `prev == NULL' failed. Signed-off-by: Fiona Ebner <f.ebner@proxmox.com> --- migration/savevm-async.c | 13 +++++++++++++ qapi/migration.json | 6 +++++- 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/migration/savevm-async.c b/migration/savevm-async.c index 3ab0bd4539..e9037173af 100644 --- a/migration/savevm-async.c +++ b/migration/savevm-async.c @@ -60,6 +60,7 @@ static struct SnapshotState { Coroutine *co; QemuCoSleep target_close_wait; IOThread *iothread; + bool savevm_end_active; } snap_state; static bool savevm_aborted(void) @@ -77,6 +78,8 @@ SaveVMInfo *qmp_query_savevm(Error **errp) SaveVMInfo *info = g_malloc0(sizeof(*info)); struct SnapshotState *s = &snap_state; + info->savevm_end_active = s->savevm_end_active; + if (s->state != SAVE_STATE_DONE) { info->has_bytes = true; info->bytes = s->bs_pos; @@ -498,6 +501,7 @@ static void coroutine_fn wait_for_close_co(void *opaque) "qmp_savevm_end"); /* we cannot assume the snapshot finished in this case, so leave the * state alone - caller has to figure something out */ + snap_state.savevm_end_active = false; return; } } else { @@ -511,6 +515,8 @@ static void coroutine_fn wait_for_close_co(void *opaque) snap_state.state = SAVE_STATE_DONE; } + snap_state.savevm_end_active = false; + DPRINTF("savevm-end: cleanup done\n"); } @@ -521,6 +527,13 @@ void qmp_savevm_end(Error **errp) return; } + if (snap_state.savevm_end_active) { + error_setg(errp, "savevm-end handler already running"); + return; + } + + snap_state.savevm_end_active = true; + Coroutine *wait_for_close = qemu_coroutine_create(wait_for_close_co, NULL); if (snap_state.state == SAVE_STATE_ACTIVE) { diff --git a/qapi/migration.json b/qapi/migration.json index 0794a3a7a3..d7504a0d4d 100644 --- a/qapi/migration.json +++ b/qapi/migration.json @@ -354,11 +354,15 @@ # # @bytes: total amount of data transfered # +# @savevm-end-active: Whether a @savevm-end handler is currently +# active. +# # Since: 1.3 ## { 'struct': 'SaveVMInfo', 'data': {'*status': 'str', '*error': 'str', - '*total-time': 'int', '*bytes': 'int'} } + '*total-time': 'int', '*bytes': 'int', + 'savevm-end-active': 'bool'} } ## # @query-savevm: -- 2.47.3 ^ permalink raw reply related [flat|nested] 5+ messages in thread
* applied: [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error 2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner ` (2 preceding siblings ...) 2026-11-18 12:21 ` [PATCH qemu 3/3] async snapshot: keep track of " Fiona Ebner @ 2026-11-18 21:02 ` Thomas Lamprecht 3 siblings, 0 replies; 5+ messages in thread From: Thomas Lamprecht @ 2026-11-18 21:02 UTC (permalink / raw) To: pve-devel, Fiona Ebner On Wed, 18 Nov 2026 13:21:46 +0100, Fiona Ebner wrote: > The loop in the 'after-unfreeze' hook phase relies on the 'bytes' > value from the result of 'query-savevm' disappearing when the > 'savevm-end' command is done. This does not happen in case of an > error status currently. Use a timeout of 35 seconds instead to ensure > that the loop stays bounded. > > This phase was enabled by commit 568b295c ("fix #7697: snapshot hook > name mismatch") so this is an unintended regression. > > [...] Applied, thanks! [qemu-server] applied to qemu-server.git (master) [1/2] fix #8331: snapshot: fix regression with endless loop after error https://git.proxmox.com/?p=qemu-server.git;a=commitdiff;h=610bd239a85f54053a2708abab5be633120abc52 [2/2] snapshot: directly check for savevm-end status https://git.proxmox.com/?p=qemu-server.git;a=commitdiff;h=a906ed74688d825549dc50ec6cbce10d1cfa152e [qemu] applied to git/pve-qemu (master) [1/1] async snapshot: keep track of savevm-end status commit: 4494236f6b1da3955766707e327ba53c892284ea ^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2026-11-18 21:02 UTC | newest] Thread overview: 5+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-11-18 12:21 [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Fiona Ebner 2026-11-18 12:21 ` [PATCH qemu-server 1/3] " Fiona Ebner 2026-11-18 12:21 ` [PATCH qemu-server 2/3] snapshot: directly check for savevm-end status Fiona Ebner 2026-11-18 12:21 ` [PATCH qemu 3/3] async snapshot: keep track of " Fiona Ebner 2026-11-18 21:02 ` applied: [PATCH-SERIES qemu-server/qemu 0/3] fix #8331: snapshot: fix regression with endless loop after error Thomas Lamprecht
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox