[PATCH-SERIES qemu/qemu-server/docs/manager 0/5] vm: snapshot: improve consistency and documentation

[PATCH-SERIES qemu/qemu-server/docs/manager 0/5] vm: snapshot: improve consistency and documentation

[Fiona Ebner]

The VM was resumed before the disk snapshots were created in case a
snapshot with RAM was taken. This is changed to rely on the savevm-end
QMP handler to resume the VM after the disk snapshots are created. See
the first patch for details.

In the docs, there was no dedicated section about VM snapshots yet,
except the subsection for the configuration. Create a common intro for
snapshots and hibernation, since they share some commonalities and add
documentation about the snapshot operations.

In particular, documentation about what ensures consistency between
multiple disks is added.


The pve-manager patch build depends on new doc-generator, depends on
new docs.


qemu:

Fiona Ebner (1):
  async snapshot: improve disk consistency by only calling vm_start() in
    savevm-end handler

 ...olerate-known-seSparse-reserved1-bit.patch |  8 +++---
 ...async-for-background-state-snapshots.patch | 27 ++++++++++++-------
 ...add-optional-buffer-size-to-QEMUFile.patch |  6 ++---
 ...se-migration-blocker-check-for-snaps.patch |  8 +++---
 4 files changed, 30 insertions(+), 19 deletions(-)


qemu-server:

Fiona Ebner (1):
  snapshot: document how consistency between multiple disks is achieved

 src/PVE/QemuConfig.pm | 7 +++++++
 1 file changed, 7 insertions(+)


docs:

Fiona Ebner (2):
  pvesm: add anchor for storage types
  qm: document snapshot operations

 pvesm.adoc |  2 +-
 qm.adoc    | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++---
 2 files changed, 68 insertions(+), 5 deletions(-)


manager:

Fiona Ebner (1):
  ui: qemu: online help: link to new section for snapshots

 www/manager6/qemu/Config.js | 1 +
 1 file changed, 1 insertion(+)


Summary over all repositories:
  8 files changed, 106 insertions(+), 24 deletions(-)

-- 
Generated by git-murpp 0.5.0

[PATCH qemu 1/5] async snapshot: improve disk consistency by only calling vm_start() in savevm-end handler

[Fiona Ebner]

A snapshot with VM state is created as follows in Proxmox VE:
1. qemu-server calls savevm-start and waits for query-savevm to return
   status 'completed'.
2. process_savevm_co() runs until state saving can converge.
3. blockdevs in IO threads are flushed async as an optimization.
4. The process_savevm_finalize() bottom half is scheduled and runs.
5. The VM is stopped, which drains and flushes all IO.
6. The state is set to 'completed'.
7a. The VM is started if it was running before.
7b. qemu-server sees status 'completed'.
8. qemu-server creates all the disk snapshots.
9. qemu-server calls savevm-end.

Since vm_start() is the very next command after setting the state to
'completed', in practice, step 7a will always happen before step 8,
which is problematic for consistency. The VM should only be started
again after the disk snapshots are taken. The functionality for
restoring the run state is already in place in qmp_savevm_end(), so
rely on that.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 ...olerate-known-seSparse-reserved1-bit.patch |  8 +++---
 ...async-for-background-state-snapshots.patch | 27 ++++++++++++-------
 ...add-optional-buffer-size-to-QEMUFile.patch |  6 ++---
 ...se-migration-blocker-check-for-snaps.patch |  8 +++---
 4 files changed, 30 insertions(+), 19 deletions(-)

diff --git a/debian/patches/extra/0004-block-vmdk-tolerate-known-seSparse-reserved1-bit.patch b/debian/patches/extra/0004-block-vmdk-tolerate-known-seSparse-reserved1-bit.patch
index bd04ec222e..1ebe382bd7 100644
--- a/debian/patches/extra/0004-block-vmdk-tolerate-known-seSparse-reserved1-bit.patch
+++ b/debian/patches/extra/0004-block-vmdk-tolerate-known-seSparse-reserved1-bit.patch
@@ -32,10 +32,12 @@ unknown reserved1 bit or a set reserved2 is still refused.
 
 Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
 ---
- block/vmdk.c                                 |   3 +-
- .../tests/vmdk-sesparse-reserved             | 105 +++++++++++++++++
- .../tests/vmdk-sesparse-reserved.out         |  14 +++
+ block/vmdk.c                                  |   3 +-
+ .../qemu-iotests/tests/vmdk-sesparse-reserved | 105 ++++++++++++++++++
+ .../tests/vmdk-sesparse-reserved.out          |  14 +++
  3 files changed, 121 insertions(+), 1 deletion(-)
+ create mode 100755 tests/qemu-iotests/tests/vmdk-sesparse-reserved
+ create mode 100644 tests/qemu-iotests/tests/vmdk-sesparse-reserved.out
 
 diff --git a/block/vmdk.c b/block/vmdk.c
 index cd8b4ec7c8..7f8fdf0727 100644
diff --git a/debian/patches/pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch b/debian/patches/pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
index 192f3abf46..0e1be00f71 100644
--- a/debian/patches/pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
+++ b/debian/patches/pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
@@ -35,7 +35,8 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
      improve runstate preservation
      use dedicated iothread for state file to avoid deadlock, bug #6262
      add parameter to skip vm start to be used for hibernation
-     rebase for 11.0.0]
+     rebase for 11.0.0
+     defer vm_start() to qmp_savevm_end() in case of success]
 Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
 ---
  hmp-commands-info.hx         |  13 +
@@ -43,13 +44,13 @@ Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
  include/migration/snapshot.h |   2 +
  include/monitor/hmp.h        |   3 +
  migration/meson.build        |   1 +
- migration/savevm-async.c     | 597 +++++++++++++++++++++++++++++++++++
+ migration/savevm-async.c     | 605 +++++++++++++++++++++++++++++++++++
  monitor/hmp-cmds.c           |  38 +++
  qapi/migration.json          |  34 ++
  qapi/misc.json               |  25 ++
  qemu-options.hx              |  12 +
  system/vl.c                  |  10 +
- 11 files changed, 752 insertions(+)
+ 11 files changed, 760 insertions(+)
  create mode 100644 migration/savevm-async.c
 
 diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
@@ -147,10 +148,10 @@ index 0222d5ea6e..90d62d5723 100644
  ), gnutls, zlib)
 diff --git a/migration/savevm-async.c b/migration/savevm-async.c
 new file mode 100644
-index 0000000000..2a860f11a1
+index 0000000000..6356557e52
 --- /dev/null
 +++ b/migration/savevm-async.c
-@@ -0,0 +1,597 @@
+@@ -0,0 +1,605 @@
 +#include "qemu/osdep.h"
 +#include "migration/channel-savevm-async.h"
 +#include "migration/migration.h"
@@ -385,11 +386,19 @@ index 0000000000..2a860f11a1
 +        save_snapshot_error("process_savevm_cleanup: invalid state: %d",
 +                            snap_state.state);
 +    }
-+    if (snap_state.vm_needs_start) {
-+        if (!should_skip_vm_start()) {
-+            vm_start();
++
++    /*
++     * In case of success, qemu-server still needs to create the disk snapshots
++     * while the VM is stopped. qmp_savevm_end() will restore the run state
++     * later.
++     */
++    if (snap_state.state != SAVE_STATE_COMPLETED) {
++        if (snap_state.vm_needs_start) {
++            if (!should_skip_vm_start()) {
++                vm_start();
++            }
++            snap_state.vm_needs_start = false;
 +        }
-+        snap_state.vm_needs_start = false;
 +    }
 +
 +    DPRINTF("timing: process_savevm_finalize (full) took %ld ms\n",
diff --git a/debian/patches/pve/0018-PVE-add-optional-buffer-size-to-QEMUFile.patch b/debian/patches/pve/0018-PVE-add-optional-buffer-size-to-QEMUFile.patch
index ab8aac16a5..9b615b0752 100644
--- a/debian/patches/pve/0018-PVE-add-optional-buffer-size-to-QEMUFile.patch
+++ b/debian/patches/pve/0018-PVE-add-optional-buffer-size-to-QEMUFile.patch
@@ -184,10 +184,10 @@ index a390554208..eda093b16a 100644
  
  G_DEFINE_AUTOPTR_CLEANUP_FUNC(QEMUFile, qemu_fclose)
 diff --git a/migration/savevm-async.c b/migration/savevm-async.c
-index 2a860f11a1..f5a4819e27 100644
+index 6356557e52..1902a842c1 100644
 --- a/migration/savevm-async.c
 +++ b/migration/savevm-async.c
-@@ -418,7 +418,7 @@ void qmp_savevm_start(const char *statefile, bool has_skip_vm_start,
+@@ -426,7 +426,7 @@ void qmp_savevm_start(const char *statefile, bool has_skip_vm_start,
  
      QIOChannel *ioc = QIO_CHANNEL(qio_channel_savevm_async_new(snap_state.target,
                                                                 &snap_state.bs_pos));
@@ -196,7 +196,7 @@ index 2a860f11a1..f5a4819e27 100644
  
      if (!snap_state.file) {
          error_setg(errp, "failed to open '%s'", statefile);
-@@ -555,7 +555,8 @@ int load_snapshot_from_blockdev(const char *filename, Error **errp)
+@@ -563,7 +563,8 @@ int load_snapshot_from_blockdev(const char *filename, Error **errp)
      bdrv_op_block_all(bs, blocker);
  
      /* restore the VM state */
diff --git a/debian/patches/pve/0046-savevm-async-reuse-migration-blocker-check-for-snaps.patch b/debian/patches/pve/0046-savevm-async-reuse-migration-blocker-check-for-snaps.patch
index 0c4cef7f09..30ed1eafdb 100644
--- a/debian/patches/pve/0046-savevm-async-reuse-migration-blocker-check-for-snaps.patch
+++ b/debian/patches/pve/0046-savevm-async-reuse-migration-blocker-check-for-snaps.patch
@@ -61,10 +61,10 @@ Message-ID: <20250618102531.57444-1-f.ebner@proxmox.com>
  5 files changed, 29 insertions(+), 4 deletions(-)
 
 diff --git a/block/vmdk.c b/block/vmdk.c
-index cd8b4ec7c8..1f64ab1766 100644
+index 7f8fdf0727..5385a09fe5 100644
 --- a/block/vmdk.c
 +++ b/block/vmdk.c
-@@ -1404,9 +1404,7 @@ static int vmdk_open(BlockDriverState *bs, QDict *options, int flags,
+@@ -1405,9 +1405,7 @@ static int vmdk_open(BlockDriverState *bs, QDict *options, int flags,
      qemu_co_mutex_init(&s->lock);
  
      /* Disable migration when VMDK images are used */
@@ -136,10 +136,10 @@ index b6888daced..80eb0dcd1f 100644
  bool migration_in_postcopy(void);
  bool migration_postcopy_is_alive(MigrationStatus state);
 diff --git a/migration/savevm-async.c b/migration/savevm-async.c
-index f5a4819e27..41376406eb 100644
+index 1902a842c1..150b2dac15 100644
 --- a/migration/savevm-async.c
 +++ b/migration/savevm-async.c
-@@ -384,7 +384,7 @@ void qmp_savevm_start(const char *statefile, bool has_skip_vm_start,
+@@ -392,7 +392,7 @@ void qmp_savevm_start(const char *statefile, bool has_skip_vm_start,
          return;
      }
  
-- 
2.47.3

[PATCH qemu-server 2/5] snapshot: document how consistency between multiple disks is achieved

[Fiona Ebner]

Document how the savevm-start and savevm-end QMP handlers affect the
run-state of the VM and the block layer and how consistency between
multiple disks is ensured.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/QemuConfig.pm | 7 +++++++
 1 file changed, 7 insertions(+)

diff --git a/src/PVE/QemuConfig.pm b/src/PVE/QemuConfig.pm
index 26f0fda2..cadcd9e9 100644
--- a/src/PVE/QemuConfig.pm
+++ b/src/PVE/QemuConfig.pm
@@ -321,6 +321,13 @@ sub __snapshot_create_vol_snapshots_hook {
     if ($running) {
         my $storecfg = PVE::Storage::config();
 
+        # NOTE savevm-start is called even if no state is saved to ensure consistency between
+        # multiple disks. The savevm-start QMP handler does a vm_stop() which drains and flushes all
+        # block devices via QEMU's block layer. If the state is saved, vm_stop() is called when
+        # saving the state converges (right before the status from query-savevm becomes
+        # 'completed'). The savevm-end QMP handler resumes the VM if it was running (and not
+        # paused/suspended/etc.) before.
+
         if ($hook eq "before") {
             if ($snap->{vmstate}) {
                 my $path = PVE::Storage::path($storecfg, $snap->{vmstate});
-- 
2.47.3

[PATCH docs 3/5] pvesm: add anchor for storage types

[Fiona Ebner]

In preparation to link to it from the QEMU chapter. Use the already
existing auto-generated name, so that deep web links continue working.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 pvesm.adoc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/pvesm.adoc b/pvesm.adoc
index 5bd24b2..c31ae4d 100644
--- a/pvesm.adoc
+++ b/pvesm.adoc
@@ -42,7 +42,7 @@ The storage library (package `libpve-storage-perl`) uses a flexible
 plugin system to provide a common interface to all storage types. This
 can be easily adopted to include further storage types in the future.
 
-
+[[_storage_types]]
 Storage Types
 -------------
 
-- 
2.47.3

[PATCH docs 4/5] qm: document snapshot operations

[Fiona Ebner]

Add a common intro for snapshots and hibernation to describe the
differences and commonalities.

Document why snapshots with multiple disks are consistent. There
already is a 'qm_snapshots' anchor, but that is a subsection for the
configuration chapter, so use 'qm_snapshot_operations' for the new
one.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 qm.adoc | 71 +++++++++++++++++++++++++++++++++++++++++++++++++++++----
 1 file changed, 67 insertions(+), 4 deletions(-)

diff --git a/qm.adoc b/qm.adoc
index 93372c5..003cdc6 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -2127,9 +2127,67 @@ It will be called during various phases of the guests lifetime.
 For an example and documentation see the example script under
 `/usr/share/pve-docs/examples/guest-example-hookscript.pl`.
 
+[[qm_snapshots_and_hibernation]]
+Snapshots and Hibernation
+-------------------------
+
+You can use snapshots and hibernation to preserve the current state of the VM
+configuration, disks, device state and RAM. Hibernation stops and locks the VM
+and you can resume the VM from the saved state later. Snapshots allow the VM to
+continue running and being modified normally and you can rollback to a snapshot
+at any later time. Passing through host devices to the VM and certain other
+configuration settings might make preserving the state impossible and thus block
+snapshots and hibernation. Certain storages like RBD allow arbitrary snapshot
+trees, other storages like ZFS only allow linear snapshot chains and rollback to
+the latest snapshot. Check the list of xref:_storage_types[Storage Types]
+for snapshots support.
+
+[[qm_snapshot_operations]]
+Snapshots
+~~~~~~~~~
+
+Create a new snapshot of the current configuration and disk state with:
+
+----
+# qm snapshot ID SNAPSHOT_NAME
+----
+
+For a running VM, you may additionally include the device state and RAM with:
+
+----
+# qm snapshot ID SNAPSHOT_NAME --vmstate 1
+----
+
+This creates a new volume on the xref:qm_vmstatestorage[state storage] and
+allows to restore the live state of the VM upon rollback.
+
+NOTE: Proxmox VE ensures consistency between multiple disks by pausing the vCPUs
+and completing and flushing all pending IO via the QEMU block layer before the
+disk snapshots are created.
+
+Delete a snapshot with:
+
+----
+# qm delsnapshot ID SNAPSHOT_NAME
+----
+
+Roll back to a snapshot with:
+
+----
+# qm rollback ID SNAPSHOT_NAME
+----
+
+WARNING: Rollback destroys the current state of the VM (even if running) and
+disks!
+
+The VM configuration and the state of the VM disks are changed back to the point
+in time when the snapshot was taken. If the VM state is included in the
+snapshot, the VM is live after rollback and continues running from the point
+in time when the snapshot was taken.
+
 [[qm_hibernate]]
 Hibernation
------------
+~~~~~~~~~~~
 
 You can suspend a VM to disk with the GUI option `Hibernate` or with
 
@@ -2142,9 +2200,14 @@ and the VM gets stopped. On the next start, the memory content will be
 loaded and the VM can continue where it was left off.
 
 [[qm_vmstatestorage]]
-.State storage selection
-If no target storage for the memory is given, it will be automatically
-chosen, the first of:
+State storage selection
+~~~~~~~~~~~~~~~~~~~~~~~
+
+To the save the RAM and device state of the virtual machine, a separate volume
+is created. For hibernation, use the `--statestorage` option to specify the
+target storage for the state volume. For snapshots or if not provided for
+hibernation, the target storage for the state volume is automatically chosen,
+the first of:
 
 1. The storage `vmstatestorage` from the VM config.
 2. The first shared storage from any VM disk.
-- 
2.47.3

[PATCH manager 5/5] ui: qemu: online help: link to new section for snapshots

[Fiona Ebner]

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

Build depends on new doc-generator, depends on new docs.

 www/manager6/qemu/Config.js | 1 +
 1 file changed, 1 insertion(+)

diff --git a/www/manager6/qemu/Config.js b/www/manager6/qemu/Config.js
index 842d35de..8b997f2a 100644
--- a/www/manager6/qemu/Config.js
+++ b/www/manager6/qemu/Config.js
@@ -381,6 +381,7 @@ Ext.define('PVE.qemu.Config', {
                 type: 'qemu',
                 xtype: 'pveGuestSnapshotTree',
                 itemId: 'snapshot',
+                onlineHelp: 'qm_snapshots_and_hibernation',
             });
         }
 
-- 
2.47.3