* [pve-devel] [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2
@ 2021-02-11 16:11 Stefan Reiter
2021-02-11 16:11 ` [pve-devel] [PATCH v2 pve-qemu 2/5] add PBS master key support Stefan Reiter
` (4 more replies)
0 siblings, 5 replies; 7+ messages in thread
From: Stefan Reiter @ 2021-02-11 16:11 UTC (permalink / raw)
To: pve-devel
Lots of patches touched and some slight changes to the build process
since QEMU switched to meson as their build system. Functionality-wise
very little rebasing required.
New patches introduced:
* pve/0058: to fix VMA backups and clean up some code in general with
new 5.2 features now available to us (namely coroutine-enabled QMP).
* extra/0002: don't build man pages for guest agent when disabled
* extra/0003: fix live-migration with hugepages
* 0017 and 0018 are adjusted to fix snapshot abort and improve
snap performance a bit
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
v2:
* update Makefile to detect in-tree builds in qemu/
* add fixes for snapshot abort and fix QEMUFile buffer size increase
* add agent man page and live-migration fix (also accepted upstream)
Known issues:
* Snapshots with the vmstate target on ceph via librbd are slow - this is not a
new bug however. An attempt at a fix is included via the changes to the
QEMUFile patch, but it only improves the situtation slightly if at all.
I'm still looking into this one, but it should not be a blocker for 5.2 I
believe.
Makefile | 3 +
debian/control | 1 +
...u-img-convert-Don-t-pre-zero-images.patch} | 4 +-
...-always-align-copied-region-to-clust.patch | 34 -
...ll-man-page-if-guest-agent-is-disabl.patch | 38 ++
...heck-page-size-match-if-RAM-postcopy.patch | 31 +
...sb-fix-setup_len-init-CVE-2020-14364.patch | 87 ---
...k-file-change-locking-default-to-off.patch | 2 +-
...djust-network-script-path-to-etc-kvm.patch | 17 +-
...he-CPU-model-to-kvm64-32-instead-of-.patch | 4 +-
...ui-spice-default-to-pve-certificates.patch | 4 +-
.../0005-PVE-Config-smm_available-false.patch | 4 +-
...lock-rbd-disable-rbd_cache_writethro.patch | 4 +-
.../0008-PVE-Up-qmp-add-get_link_status.patch | 6 +-
...return-success-on-info-without-snaps.patch | 4 +-
...dd-add-osize-and-read-from-to-stdin-.patch | 18 +-
...E-Up-qemu-img-dd-add-isize-parameter.patch | 12 +-
...PVE-Up-qemu-img-dd-add-n-skip_create.patch | 10 +-
...virtio-balloon-improve-query-balloon.patch | 22 +-
.../0015-PVE-qapi-modify-query-machines.patch | 12 +-
.../0016-PVE-qapi-modify-spice-query.patch | 8 +-
...sync-for-background-state-snapshots.patch} | 584 +++++++++--------
...dd-optional-buffer-size-to-QEMUFile.patch} | 32 +-
...add-the-zeroinit-block-driver-filter.patch | 36 +-
...-Add-dummy-id-command-line-parameter.patch | 10 +-
...t-target-i386-disable-LINT0-after-re.patch | 2 +-
...le-posix-make-locking-optiono-on-cre.patch | 20 +-
...3-PVE-monitor-disable-oob-capability.patch | 4 +-
...sed-balloon-qemu-4-0-config-size-fal.patch | 4 +-
...E-Allow-version-code-in-machine-type.patch | 36 +-
...VE-Backup-add-vma-backup-format-code.patch | 83 +--
...-Backup-add-backup-dump-block-driver.patch | 36 +-
...ckup-proxmox-backup-patches-for-qemu.patch | 127 ++--
...estore-new-command-to-restore-from-p.patch | 39 +-
...d-support-for-sync-bitmap-mode-never.patch | 76 +--
...-support-for-conditional-and-always-.patch | 10 +-
...check-for-bitmap-mode-without-bitmap.patch | 6 +-
...-to-bdrv_dirty_bitmap_merge_internal.patch | 6 +-
...5-iotests-add-test-for-bitmap-mirror.patch | 2 +-
.../0036-mirror-move-some-checks-to-qmp.patch | 8 +-
...irty-bitmap-tracking-for-incremental.patch | 18 +-
...name-incremental-to-use-dirty-bitmap.patch | 8 +-
...ckup-add-compress-and-encrypt-option.patch | 10 +-
...k-driver-to-map-backup-archives-into.patch | 80 +--
...dd-query_proxmox_support-QMP-command.patch | 6 +-
...issing-crypt-and-compress-parameters.patch | 2 +-
...rite-callback-with-big-blocks-correc.patch | 2 +-
...-block-handling-to-PBS-dump-callback.patch | 2 +-
...E-add-query-pbs-bitmap-info-QMP-call.patch | 14 +-
...ct-stderr-to-journal-when-daemonized.patch | 40 +-
...d-sequential-job-transaction-support.patch | 2 +-
...-transaction-to-synchronize-job-stat.patch | 2 +-
...ore-coroutines-and-don-t-block-on-fi.patch | 6 +-
...n-up-error-handling-for-create_backu.patch | 2 +-
...grate-dirty-bitmap-state-via-savevm.patch} | 39 +-
...dirty-bitmap-fix-larger-granularity-.patch | 33 -
...irty-bitmap-migrate-other-bitmaps-e.patch} | 6 +-
...multiple-CREATED-jobs-in-sequential.patch} | 2 +-
...ll-back-to-open-iscsi-initiatorname.patch} | 5 +-
...routine-QMP-for-backup-cancel_backup.patch | 597 ++++++++++++++++++
debian/patches/series | 20 +-
debian/pve-qemu-kvm.install | 2 -
debian/rules | 12 +-
qemu | 2 +-
64 files changed, 1436 insertions(+), 922 deletions(-)
rename debian/patches/extra/{0002-Revert-qemu-img-convert-Don-t-pre-zero-images.patch => 0001-Revert-qemu-img-convert-Don-t-pre-zero-images.patch} (91%)
delete mode 100644 debian/patches/extra/0001-block-block-copy-always-align-copied-region-to-clust.patch
create mode 100644 debian/patches/extra/0002-docs-don-t-install-man-page-if-guest-agent-is-disabl.patch
create mode 100644 debian/patches/extra/0003-migration-only-check-page-size-match-if-RAM-postcopy.patch
delete mode 100644 debian/patches/extra/0003-usb-fix-setup_len-init-CVE-2020-14364.patch
rename debian/patches/pve/{0017-PVE-internal-snapshot-async.patch => 0017-PVE-add-savevm-async-for-background-state-snapshots.patch} (84%)
rename debian/patches/pve/{0018-add-optional-buffer-size-to-QEMUFile.patch => 0018-PVE-add-optional-buffer-size-to-QEMUFile.patch} (87%)
rename debian/patches/pve/{0055-PVE-Migrate-dirty-bitmap-state-via-savevm.patch => 0054-PVE-Migrate-dirty-bitmap-state-via-savevm.patch} (90%)
delete mode 100644 debian/patches/pve/0054-migration-block-dirty-bitmap-fix-larger-granularity-.patch
rename debian/patches/pve/{0056-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch => 0055-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch} (89%)
rename debian/patches/pve/{0057-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch => 0056-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch} (97%)
rename debian/patches/pve/{0058-PVE-fall-back-to-open-iscsi-initiatorname.patch => 0057-PVE-fall-back-to-open-iscsi-initiatorname.patch} (97%)
create mode 100644 debian/patches/pve/0058-PVE-Use-coroutine-QMP-for-backup-cancel_backup.patch
diff --git a/Makefile b/Makefile
index 497c6f3..688a3d1 100644
--- a/Makefile
+++ b/Makefile
@@ -19,6 +19,9 @@ submodule:
test -f "${SRCDIR}/configure" || git submodule update --init --recursive
$(BUILDDIR): keycodemapdb | submodule
+ # check if qemu/ was used for a build
+ # if so, please run 'make distclean' in the submodule and try again
+ test ! -f $(SRCDIR)/build/config.status
rm -rf $(BUILDDIR)
cp -a $(SRCDIR) $(BUILDDIR)
cp -a debian $(BUILDDIR)/debian
diff --git a/debian/control b/debian/control
index ee913e8..696d064 100644
--- a/debian/control
+++ b/debian/control
@@ -28,6 +28,7 @@ Build-Depends: autotools-dev,
libsystemd-dev,
libusb-1.0-0-dev (>= 1.0.17-1),
libusbredirparser-dev (>= 0.6-2),
+ meson,
python3-minimal,
python3-sphinx,
quilt,
diff --git a/debian/patches/extra/0002-Revert-qemu-img-convert-Don-t-pre-zero-images.patch b/debian/patches/extra/0001-Revert-qemu-img-convert-Don-t-pre-zero-images.patch
similarity index 91%
rename from debian/patches/extra/0002-Revert-qemu-img-convert-Don-t-pre-zero-images.patch
rename to debian/patches/extra/0001-Revert-qemu-img-convert-Don-t-pre-zero-images.patch
index 604feb3..0e7331f 100644
--- a/debian/patches/extra/0002-Revert-qemu-img-convert-Don-t-pre-zero-images.patch
+++ b/debian/patches/extra/0001-Revert-qemu-img-convert-Don-t-pre-zero-images.patch
@@ -12,10 +12,10 @@ https://bugzilla.proxmox.com/show_bug.cgi?id=3002
1 file changed, 9 insertions(+)
diff --git a/qemu-img.c b/qemu-img.c
-index 9635e9c001..c7884d248a 100644
+index 8bdea40b58..f9050bfaad 100644
--- a/qemu-img.c
+++ b/qemu-img.c
-@@ -2079,6 +2079,15 @@ static int convert_do_copy(ImgConvertState *s)
+@@ -2104,6 +2104,15 @@ static int convert_do_copy(ImgConvertState *s)
s->has_zero_init = bdrv_has_zero_init(blk_bs(s->target));
}
diff --git a/debian/patches/extra/0001-block-block-copy-always-align-copied-region-to-clust.patch b/debian/patches/extra/0001-block-block-copy-always-align-copied-region-to-clust.patch
deleted file mode 100644
index fe8589e..0000000
--- a/debian/patches/extra/0001-block-block-copy-always-align-copied-region-to-clust.patch
+++ /dev/null
@@ -1,34 +0,0 @@
-From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
-From: Stefan Reiter <s.reiter@proxmox.com>
-Date: Thu, 6 Aug 2020 15:43:58 +0200
-Subject: [PATCH] block/block-copy: always align copied region to cluster size
-
-Since commit 42ac214406e0 (block/block-copy: refactor task creation)
-block_copy_task_create calculates the area to be copied via
-bdrv_dirty_bitmap_next_dirty_area, but that can return an unaligned byte
-count if the image's last cluster end is not aligned to the bitmap's
-granularity.
-
-Always ALIGN_UP the resulting bytes value to satisfy block_copy_do_copy,
-which requires the 'bytes' parameter to be aligned to cluster size.
-
-Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
----
- block/block-copy.c | 3 +++
- 1 file changed, 3 insertions(+)
-
-diff --git a/block/block-copy.c b/block/block-copy.c
-index f7428a7c08..a30b9097ef 100644
---- a/block/block-copy.c
-+++ b/block/block-copy.c
-@@ -142,6 +142,9 @@ static BlockCopyTask *block_copy_task_create(BlockCopyState *s,
- return NULL;
- }
-
-+ assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
-+ bytes = QEMU_ALIGN_UP(bytes, s->cluster_size);
-+
- /* region is dirty, so no existent tasks possible in it */
- assert(!find_conflicting_task(s, offset, bytes));
-
diff --git a/debian/patches/extra/0002-docs-don-t-install-man-page-if-guest-agent-is-disabl.patch b/debian/patches/extra/0002-docs-don-t-install-man-page-if-guest-agent-is-disabl.patch
new file mode 100644
index 0000000..d5fb596
--- /dev/null
+++ b/debian/patches/extra/0002-docs-don-t-install-man-page-if-guest-agent-is-disabl.patch
@@ -0,0 +1,38 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Stefan Reiter <s.reiter@proxmox.com>
+Date: Thu, 28 Jan 2021 15:19:51 +0100
+Subject: [PATCH] docs: don't install man page if guest agent is disabled
+
+No sense outputting the qemu-ga and qemu-ga-ref man pages when the guest
+agent binary itself is disabled. This mirrors behaviour from before the
+meson switch.
+
+Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
+---
+ docs/meson.build | 6 ++++--
+ 1 file changed, 4 insertions(+), 2 deletions(-)
+
+diff --git a/docs/meson.build b/docs/meson.build
+index ebd85d59f9..cc6f5007f8 100644
+--- a/docs/meson.build
++++ b/docs/meson.build
+@@ -46,6 +46,8 @@ if build_docs
+ meson.source_root() / 'docs/sphinx/qmp_lexer.py',
+ qapi_gen_depends ]
+
++ have_ga = have_tools and config_host.has_key('CONFIG_GUEST_AGENT')
++
+ configure_file(output: 'index.html',
+ input: files('index.html.in'),
+ configuration: {'VERSION': meson.project_version()},
+@@ -53,8 +55,8 @@ if build_docs
+ manuals = [ 'devel', 'interop', 'tools', 'specs', 'system', 'user' ]
+ man_pages = {
+ 'interop' : {
+- 'qemu-ga.8': (have_tools ? 'man8' : ''),
+- 'qemu-ga-ref.7': 'man7',
++ 'qemu-ga.8': (have_ga ? 'man8' : ''),
++ 'qemu-ga-ref.7': (have_ga ? 'man7' : ''),
+ 'qemu-qmp-ref.7': 'man7',
+ },
+ 'tools': {
diff --git a/debian/patches/extra/0003-migration-only-check-page-size-match-if-RAM-postcopy.patch b/debian/patches/extra/0003-migration-only-check-page-size-match-if-RAM-postcopy.patch
new file mode 100644
index 0000000..8c2548b
--- /dev/null
+++ b/debian/patches/extra/0003-migration-only-check-page-size-match-if-RAM-postcopy.patch
@@ -0,0 +1,31 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Stefan Reiter <s.reiter@proxmox.com>
+Date: Thu, 4 Feb 2021 17:06:19 +0100
+Subject: [PATCH] migration: only check page size match if RAM postcopy is
+ enabled
+
+Postcopy may also be advised for dirty-bitmap migration only, in which
+case the remote page size will not be available and we'll instead read
+bogus data, blocking migration with a mismatch error if the VM uses
+hugepages.
+
+Fixes: 58110f0acb ("migration: split common postcopy out of ram postcopy")
+Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
+Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
+---
+ migration/ram.c | 2 +-
+ 1 file changed, 1 insertion(+), 1 deletion(-)
+
+diff --git a/migration/ram.c b/migration/ram.c
+index 7811cde643..6ace15261c 100644
+--- a/migration/ram.c
++++ b/migration/ram.c
+@@ -3521,7 +3521,7 @@ static int ram_load_precopy(QEMUFile *f)
+ }
+ }
+ /* For postcopy we need to check hugepage sizes match */
+- if (postcopy_advised &&
++ if (postcopy_advised && migrate_postcopy_ram() &&
+ block->page_size != qemu_host_page_size) {
+ uint64_t remote_page_size = qemu_get_be64(f);
+ if (remote_page_size != block->page_size) {
diff --git a/debian/patches/extra/0003-usb-fix-setup_len-init-CVE-2020-14364.patch b/debian/patches/extra/0003-usb-fix-setup_len-init-CVE-2020-14364.patch
deleted file mode 100644
index 57b99a4..0000000
--- a/debian/patches/extra/0003-usb-fix-setup_len-init-CVE-2020-14364.patch
+++ /dev/null
@@ -1,87 +0,0 @@
-From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
-From: Gerd Hoffmann <kraxel@redhat.com>
-Date: Tue, 25 Aug 2020 07:36:36 +0200
-Subject: [PATCH] usb: fix setup_len init (CVE-2020-14364)
-
-Store calculated setup_len in a local variable, verify it, and only
-write it to the struct (USBDevice->setup_len) in case it passed the
-sanity checks.
-
-This prevents other code (do_token_{in,out} functions specifically)
-from working with invalid USBDevice->setup_len values and overrunning
-the USBDevice->setup_buf[] buffer.
-
-Fixes: CVE-2020-14364
-Signed-off-by: Gerd Hoffmann <kraxel@redhat.com>
-Tested-by: Gonglei <arei.gonglei@huawei.com>
-Reviewed-by: Li Qiang <liq3ea@gmail.com>
-Message-id: 20200825053636.29648-1-kraxel@redhat.com
-(cherry picked from commit b946434f2659a182afc17e155be6791ebfb302eb)
-Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
----
- hw/usb/core.c | 16 ++++++++++------
- 1 file changed, 10 insertions(+), 6 deletions(-)
-
-diff --git a/hw/usb/core.c b/hw/usb/core.c
-index 5abd128b6b..5234dcc73f 100644
---- a/hw/usb/core.c
-+++ b/hw/usb/core.c
-@@ -129,6 +129,7 @@ void usb_wakeup(USBEndpoint *ep, unsigned int stream)
- static void do_token_setup(USBDevice *s, USBPacket *p)
- {
- int request, value, index;
-+ unsigned int setup_len;
-
- if (p->iov.size != 8) {
- p->status = USB_RET_STALL;
-@@ -138,14 +139,15 @@ static void do_token_setup(USBDevice *s, USBPacket *p)
- usb_packet_copy(p, s->setup_buf, p->iov.size);
- s->setup_index = 0;
- p->actual_length = 0;
-- s->setup_len = (s->setup_buf[7] << 8) | s->setup_buf[6];
-- if (s->setup_len > sizeof(s->data_buf)) {
-+ setup_len = (s->setup_buf[7] << 8) | s->setup_buf[6];
-+ if (setup_len > sizeof(s->data_buf)) {
- fprintf(stderr,
- "usb_generic_handle_packet: ctrl buffer too small (%d > %zu)\n",
-- s->setup_len, sizeof(s->data_buf));
-+ setup_len, sizeof(s->data_buf));
- p->status = USB_RET_STALL;
- return;
- }
-+ s->setup_len = setup_len;
-
- request = (s->setup_buf[0] << 8) | s->setup_buf[1];
- value = (s->setup_buf[3] << 8) | s->setup_buf[2];
-@@ -259,26 +261,28 @@ static void do_token_out(USBDevice *s, USBPacket *p)
- static void do_parameter(USBDevice *s, USBPacket *p)
- {
- int i, request, value, index;
-+ unsigned int setup_len;
-
- for (i = 0; i < 8; i++) {
- s->setup_buf[i] = p->parameter >> (i*8);
- }
-
- s->setup_state = SETUP_STATE_PARAM;
-- s->setup_len = (s->setup_buf[7] << 8) | s->setup_buf[6];
- s->setup_index = 0;
-
- request = (s->setup_buf[0] << 8) | s->setup_buf[1];
- value = (s->setup_buf[3] << 8) | s->setup_buf[2];
- index = (s->setup_buf[5] << 8) | s->setup_buf[4];
-
-- if (s->setup_len > sizeof(s->data_buf)) {
-+ setup_len = (s->setup_buf[7] << 8) | s->setup_buf[6];
-+ if (setup_len > sizeof(s->data_buf)) {
- fprintf(stderr,
- "usb_generic_handle_packet: ctrl buffer too small (%d > %zu)\n",
-- s->setup_len, sizeof(s->data_buf));
-+ setup_len, sizeof(s->data_buf));
- p->status = USB_RET_STALL;
- return;
- }
-+ s->setup_len = setup_len;
-
- if (p->pid == USB_TOKEN_OUT) {
- usb_packet_copy(p, s->data_buf, s->setup_len);
diff --git a/debian/patches/pve/0001-PVE-Config-block-file-change-locking-default-to-off.patch b/debian/patches/pve/0001-PVE-Config-block-file-change-locking-default-to-off.patch
index cf9b54c..9d7bec1 100644
--- a/debian/patches/pve/0001-PVE-Config-block-file-change-locking-default-to-off.patch
+++ b/debian/patches/pve/0001-PVE-Config-block-file-change-locking-default-to-off.patch
@@ -14,7 +14,7 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/block/file-posix.c b/block/file-posix.c
-index 9a00d4190a..bb72e1e5ca 100644
+index d5fd1dbcd2..bda3e606dc 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -508,7 +508,7 @@ static QemuOptsList raw_runtime_opts = {
diff --git a/debian/patches/pve/0002-PVE-Config-Adjust-network-script-path-to-etc-kvm.patch b/debian/patches/pve/0002-PVE-Config-Adjust-network-script-path-to-etc-kvm.patch
index 6068532..17e6fa5 100644
--- a/debian/patches/pve/0002-PVE-Config-Adjust-network-script-path-to-etc-kvm.patch
+++ b/debian/patches/pve/0002-PVE-Config-Adjust-network-script-path-to-etc-kvm.patch
@@ -5,22 +5,21 @@ Subject: [PATCH] PVE: [Config] Adjust network script path to /etc/kvm/
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
- include/net/net.h | 5 +++--
- 1 file changed, 3 insertions(+), 2 deletions(-)
+ include/net/net.h | 4 ++--
+ 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/include/net/net.h b/include/net/net.h
-index e7ef42d62b..4fd1144e58 100644
+index 778fc787ca..fb2db6bb75 100644
--- a/include/net/net.h
+++ b/include/net/net.h
-@@ -209,8 +209,9 @@ void netdev_add(QemuOpts *opts, Error **errp);
+@@ -210,8 +210,8 @@ void netdev_add(QemuOpts *opts, Error **errp);
int net_hub_id_for_client(NetClientState *nc, int *id);
NetClientState *net_hub_port_find(int hub_id);
--#define DEFAULT_NETWORK_SCRIPT "/etc/qemu-ifup"
--#define DEFAULT_NETWORK_DOWN_SCRIPT "/etc/qemu-ifdown"
-+#define DEFAULT_NETWORK_SCRIPT "/etc/kvm/kvm-ifup"
-+#define DEFAULT_NETWORK_DOWN_SCRIPT "/etc/kvm/kvm-ifdown"
-+
+-#define DEFAULT_NETWORK_SCRIPT CONFIG_SYSCONFDIR "/qemu-ifup"
+-#define DEFAULT_NETWORK_DOWN_SCRIPT CONFIG_SYSCONFDIR "/qemu-ifdown"
++#define DEFAULT_NETWORK_SCRIPT CONFIG_SYSCONFDIR "/kvm/kvm-ifup"
++#define DEFAULT_NETWORK_DOWN_SCRIPT CONFIG_SYSCONFDIR "/kvm/kvm-ifdown"
#define DEFAULT_BRIDGE_HELPER CONFIG_QEMU_HELPERDIR "/qemu-bridge-helper"
#define DEFAULT_BRIDGE_INTERFACE "br0"
diff --git a/debian/patches/pve/0003-PVE-Config-set-the-CPU-model-to-kvm64-32-instead-of-.patch b/debian/patches/pve/0003-PVE-Config-set-the-CPU-model-to-kvm64-32-instead-of-.patch
index c906ed0..3b08513 100644
--- a/debian/patches/pve/0003-PVE-Config-set-the-CPU-model-to-kvm64-32-instead-of-.patch
+++ b/debian/patches/pve/0003-PVE-Config-set-the-CPU-model-to-kvm64-32-instead-of-.patch
@@ -10,10 +10,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/target/i386/cpu.h b/target/i386/cpu.h
-index e1a5c174dc..8973d3160f 100644
+index 88e8586f8f..93563ee0c2 100644
--- a/target/i386/cpu.h
+++ b/target/i386/cpu.h
-@@ -1975,9 +1975,9 @@ uint64_t cpu_get_tsc(CPUX86State *env);
+@@ -1973,9 +1973,9 @@ uint64_t cpu_get_tsc(CPUX86State *env);
#define CPU_RESOLVING_TYPE TYPE_X86_CPU
#ifdef TARGET_X86_64
diff --git a/debian/patches/pve/0004-PVE-Config-ui-spice-default-to-pve-certificates.patch b/debian/patches/pve/0004-PVE-Config-ui-spice-default-to-pve-certificates.patch
index fb4ee8f..f02a0f8 100644
--- a/debian/patches/pve/0004-PVE-Config-ui-spice-default-to-pve-certificates.patch
+++ b/debian/patches/pve/0004-PVE-Config-ui-spice-default-to-pve-certificates.patch
@@ -9,10 +9,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/ui/spice-core.c b/ui/spice-core.c
-index ecc2ec2c55..ca04965ead 100644
+index eea52f5389..d09ee7f09e 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
-@@ -668,32 +668,35 @@ void qemu_spice_init(void)
+@@ -667,32 +667,35 @@ static void qemu_spice_init(void)
if (tls_port) {
x509_dir = qemu_opt_get(opts, "x509-dir");
diff --git a/debian/patches/pve/0005-PVE-Config-smm_available-false.patch b/debian/patches/pve/0005-PVE-Config-smm_available-false.patch
index ff8fde9..59fd521 100644
--- a/debian/patches/pve/0005-PVE-Config-smm_available-false.patch
+++ b/debian/patches/pve/0005-PVE-Config-smm_available-false.patch
@@ -10,10 +10,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/hw/i386/x86.c b/hw/i386/x86.c
-index 67bee1bcb8..d954bf77b9 100644
+index 5944fc44ed..31b481b4e9 100644
--- a/hw/i386/x86.c
+++ b/hw/i386/x86.c
-@@ -856,7 +856,7 @@ bool x86_machine_is_smm_enabled(X86MachineState *x86ms)
+@@ -1115,7 +1115,7 @@ bool x86_machine_is_smm_enabled(const X86MachineState *x86ms)
if (tcg_enabled() || qtest_enabled()) {
smm_available = true;
} else if (kvm_enabled()) {
diff --git a/debian/patches/pve/0007-PVE-Config-rbd-block-rbd-disable-rbd_cache_writethro.patch b/debian/patches/pve/0007-PVE-Config-rbd-block-rbd-disable-rbd_cache_writethro.patch
index 3bf4925..7180411 100644
--- a/debian/patches/pve/0007-PVE-Config-rbd-block-rbd-disable-rbd_cache_writethro.patch
+++ b/debian/patches/pve/0007-PVE-Config-rbd-block-rbd-disable-rbd_cache_writethro.patch
@@ -18,10 +18,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+)
diff --git a/block/rbd.c b/block/rbd.c
-index 688074c64b..8ae39abb46 100644
+index 9bd2bce716..c7195a2342 100644
--- a/block/rbd.c
+++ b/block/rbd.c
-@@ -651,6 +651,8 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
+@@ -609,6 +609,8 @@ static int qemu_rbd_connect(rados_t *cluster, rados_ioctx_t *io_ctx,
rados_conf_set(*cluster, "rbd_cache", "false");
}
diff --git a/debian/patches/pve/0008-PVE-Up-qmp-add-get_link_status.patch b/debian/patches/pve/0008-PVE-Up-qmp-add-get_link_status.patch
index cae1f1b..4930618 100644
--- a/debian/patches/pve/0008-PVE-Up-qmp-add-get_link_status.patch
+++ b/debian/patches/pve/0008-PVE-Up-qmp-add-get_link_status.patch
@@ -11,10 +11,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
3 files changed, 43 insertions(+)
diff --git a/net/net.c b/net/net.c
-index bbaedb3c7a..9de23ec834 100644
+index 6a2c3d9567..a1e9514fb8 100644
--- a/net/net.c
+++ b/net/net.c
-@@ -1276,6 +1276,33 @@ void hmp_info_network(Monitor *mon, const QDict *qdict)
+@@ -1277,6 +1277,33 @@ void hmp_info_network(Monitor *mon, const QDict *qdict)
}
}
@@ -49,7 +49,7 @@ index bbaedb3c7a..9de23ec834 100644
{
NetClientState *nc;
diff --git a/qapi/net.json b/qapi/net.json
-index ddb113e5e5..eb3b785984 100644
+index a3a1336001..b8092c4e20 100644
--- a/qapi/net.json
+++ b/qapi/net.json
@@ -35,6 +35,21 @@
diff --git a/debian/patches/pve/0010-PVE-Up-qemu-img-return-success-on-info-without-snaps.patch b/debian/patches/pve/0010-PVE-Up-qemu-img-return-success-on-info-without-snaps.patch
index 8a4ca3a..d15050e 100644
--- a/debian/patches/pve/0010-PVE-Up-qemu-img-return-success-on-info-without-snaps.patch
+++ b/debian/patches/pve/0010-PVE-Up-qemu-img-return-success-on-info-without-snaps.patch
@@ -9,10 +9,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/qemu-img.c b/qemu-img.c
-index 5308773811..45aa024acc 100644
+index f9050bfaad..7e6666b5f7 100644
--- a/qemu-img.c
+++ b/qemu-img.c
-@@ -2955,7 +2955,8 @@ static int img_info(int argc, char **argv)
+@@ -3022,7 +3022,8 @@ static int img_info(int argc, char **argv)
list = collect_image_info_list(image_opts, filename, fmt, chain,
force_share);
if (!list) {
diff --git a/debian/patches/pve/0011-PVE-Up-qemu-img-dd-add-osize-and-read-from-to-stdin-.patch b/debian/patches/pve/0011-PVE-Up-qemu-img-dd-add-osize-and-read-from-to-stdin-.patch
index 3b73980..07a877e 100644
--- a/debian/patches/pve/0011-PVE-Up-qemu-img-dd-add-osize-and-read-from-to-stdin-.patch
+++ b/debian/patches/pve/0011-PVE-Up-qemu-img-dd-add-osize-and-read-from-to-stdin-.patch
@@ -37,7 +37,7 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 files changed, 121 insertions(+), 74 deletions(-)
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
-index b89c019b76..91d18a4819 100644
+index b3620f29e5..e70ef3dc91 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -58,9 +58,9 @@ SRST
@@ -53,10 +53,10 @@ index b89c019b76..91d18a4819 100644
DEF("info", img_info,
diff --git a/qemu-img.c b/qemu-img.c
-index 45aa024acc..af54d0896e 100644
+index 7e6666b5f7..44cf942bd2 100644
--- a/qemu-img.c
+++ b/qemu-img.c
-@@ -4819,10 +4819,12 @@ static int img_bitmap(int argc, char **argv)
+@@ -4897,10 +4897,12 @@ static int img_bitmap(int argc, char **argv)
#define C_IF 04
#define C_OF 010
#define C_SKIP 020
@@ -69,7 +69,7 @@ index 45aa024acc..af54d0896e 100644
};
struct DdIo {
-@@ -4898,6 +4900,19 @@ static int img_dd_skip(const char *arg,
+@@ -4976,6 +4978,19 @@ static int img_dd_skip(const char *arg,
return 0;
}
@@ -89,7 +89,7 @@ index 45aa024acc..af54d0896e 100644
static int img_dd(int argc, char **argv)
{
int ret = 0;
-@@ -4938,6 +4953,7 @@ static int img_dd(int argc, char **argv)
+@@ -5016,6 +5031,7 @@ static int img_dd(int argc, char **argv)
{ "if", img_dd_if, C_IF },
{ "of", img_dd_of, C_OF },
{ "skip", img_dd_skip, C_SKIP },
@@ -97,7 +97,7 @@ index 45aa024acc..af54d0896e 100644
{ NULL, NULL, 0 }
};
const struct option long_options[] = {
-@@ -5016,8 +5032,13 @@ static int img_dd(int argc, char **argv)
+@@ -5094,8 +5110,13 @@ static int img_dd(int argc, char **argv)
arg = NULL;
}
@@ -113,7 +113,7 @@ index 45aa024acc..af54d0896e 100644
ret = -1;
goto out;
}
-@@ -5029,85 +5050,101 @@ static int img_dd(int argc, char **argv)
+@@ -5107,85 +5128,101 @@ static int img_dd(int argc, char **argv)
goto out;
}
@@ -279,7 +279,7 @@ index 45aa024acc..af54d0896e 100644
}
if (dd.flags & C_SKIP && (in.offset > INT64_MAX / in.bsz ||
-@@ -5125,11 +5162,17 @@ static int img_dd(int argc, char **argv)
+@@ -5203,11 +5240,17 @@ static int img_dd(int argc, char **argv)
for (out_pos = 0; in_pos < size; block_count++) {
int in_ret, out_ret;
@@ -301,7 +301,7 @@ index 45aa024acc..af54d0896e 100644
}
if (in_ret < 0) {
error_report("error while reading from input image file: %s",
-@@ -5139,9 +5182,13 @@ static int img_dd(int argc, char **argv)
+@@ -5217,9 +5260,13 @@ static int img_dd(int argc, char **argv)
}
in_pos += in_ret;
diff --git a/debian/patches/pve/0012-PVE-Up-qemu-img-dd-add-isize-parameter.patch b/debian/patches/pve/0012-PVE-Up-qemu-img-dd-add-isize-parameter.patch
index edcb94e..ff2ff47 100644
--- a/debian/patches/pve/0012-PVE-Up-qemu-img-dd-add-isize-parameter.patch
+++ b/debian/patches/pve/0012-PVE-Up-qemu-img-dd-add-isize-parameter.patch
@@ -15,10 +15,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 25 insertions(+), 3 deletions(-)
diff --git a/qemu-img.c b/qemu-img.c
-index af54d0896e..0f1d464392 100644
+index 44cf942bd2..5ce60e8a45 100644
--- a/qemu-img.c
+++ b/qemu-img.c
-@@ -4820,11 +4820,13 @@ static int img_bitmap(int argc, char **argv)
+@@ -4898,11 +4898,13 @@ static int img_bitmap(int argc, char **argv)
#define C_OF 010
#define C_SKIP 020
#define C_OSIZE 040
@@ -32,7 +32,7 @@ index af54d0896e..0f1d464392 100644
};
struct DdIo {
-@@ -4913,6 +4915,19 @@ static int img_dd_osize(const char *arg,
+@@ -4991,6 +4993,19 @@ static int img_dd_osize(const char *arg,
return 0;
}
@@ -52,7 +52,7 @@ index af54d0896e..0f1d464392 100644
static int img_dd(int argc, char **argv)
{
int ret = 0;
-@@ -4927,12 +4942,14 @@ static int img_dd(int argc, char **argv)
+@@ -5005,12 +5020,14 @@ static int img_dd(int argc, char **argv)
int c, i;
const char *out_fmt = "raw";
const char *fmt = NULL;
@@ -68,7 +68,7 @@ index af54d0896e..0f1d464392 100644
};
struct DdIo in = {
.bsz = 512, /* Block size is by default 512 bytes */
-@@ -4954,6 +4971,7 @@ static int img_dd(int argc, char **argv)
+@@ -5032,6 +5049,7 @@ static int img_dd(int argc, char **argv)
{ "of", img_dd_of, C_OF },
{ "skip", img_dd_skip, C_SKIP },
{ "osize", img_dd_osize, C_OSIZE },
@@ -76,7 +76,7 @@ index af54d0896e..0f1d464392 100644
{ NULL, NULL, 0 }
};
const struct option long_options[] = {
-@@ -5160,14 +5178,18 @@ static int img_dd(int argc, char **argv)
+@@ -5238,14 +5256,18 @@ static int img_dd(int argc, char **argv)
in.buf = g_new(uint8_t, in.bsz);
diff --git a/debian/patches/pve/0013-PVE-Up-qemu-img-dd-add-n-skip_create.patch b/debian/patches/pve/0013-PVE-Up-qemu-img-dd-add-n-skip_create.patch
index aca640b..3230c04 100644
--- a/debian/patches/pve/0013-PVE-Up-qemu-img-dd-add-n-skip_create.patch
+++ b/debian/patches/pve/0013-PVE-Up-qemu-img-dd-add-n-skip_create.patch
@@ -9,10 +9,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 14 insertions(+), 9 deletions(-)
diff --git a/qemu-img.c b/qemu-img.c
-index 0f1d464392..9635e9c001 100644
+index 5ce60e8a45..86bfd0288b 100644
--- a/qemu-img.c
+++ b/qemu-img.c
-@@ -4944,7 +4944,7 @@ static int img_dd(int argc, char **argv)
+@@ -5022,7 +5022,7 @@ static int img_dd(int argc, char **argv)
const char *fmt = NULL;
int64_t size = 0, readsize = 0;
int64_t block_count = 0, out_pos, in_pos;
@@ -21,7 +21,7 @@ index 0f1d464392..9635e9c001 100644
struct DdInfo dd = {
.flags = 0,
.count = 0,
-@@ -4982,7 +4982,7 @@ static int img_dd(int argc, char **argv)
+@@ -5060,7 +5060,7 @@ static int img_dd(int argc, char **argv)
{ 0, 0, 0, 0 }
};
@@ -30,7 +30,7 @@ index 0f1d464392..9635e9c001 100644
if (c == EOF) {
break;
}
-@@ -5002,6 +5002,9 @@ static int img_dd(int argc, char **argv)
+@@ -5080,6 +5080,9 @@ static int img_dd(int argc, char **argv)
case 'h':
help();
break;
@@ -40,7 +40,7 @@ index 0f1d464392..9635e9c001 100644
case 'U':
force_share = true;
break;
-@@ -5142,13 +5145,15 @@ static int img_dd(int argc, char **argv)
+@@ -5220,13 +5223,15 @@ static int img_dd(int argc, char **argv)
size - in.bsz * in.offset, &error_abort);
}
diff --git a/debian/patches/pve/0014-PVE-virtio-balloon-improve-query-balloon.patch b/debian/patches/pve/0014-PVE-virtio-balloon-improve-query-balloon.patch
index dd83122..4bd835f 100644
--- a/debian/patches/pve/0014-PVE-virtio-balloon-improve-query-balloon.patch
+++ b/debian/patches/pve/0014-PVE-virtio-balloon-improve-query-balloon.patch
@@ -10,11 +10,11 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
hw/virtio/virtio-balloon.c | 33 +++++++++++++++++++++++++++++++--
monitor/hmp-cmds.c | 30 +++++++++++++++++++++++++++++-
- qapi/misc.json | 22 +++++++++++++++++++++-
+ qapi/machine.json | 22 +++++++++++++++++++++-
3 files changed, 81 insertions(+), 4 deletions(-)
diff --git a/hw/virtio/virtio-balloon.c b/hw/virtio/virtio-balloon.c
-index 22cb5df717..6513adb0a6 100644
+index b22b5beda3..6e581439bf 100644
--- a/hw/virtio/virtio-balloon.c
+++ b/hw/virtio/virtio-balloon.c
@@ -805,8 +805,37 @@ static uint64_t virtio_balloon_get_features(VirtIODevice *vdev, uint64_t f,
@@ -58,10 +58,10 @@ index 22cb5df717..6513adb0a6 100644
static void virtio_balloon_to_target(void *opaque, ram_addr_t target)
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
-index ae4b6a4246..6e26ea2cd0 100644
+index 65d8ff4849..705f08a8f1 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
-@@ -660,7 +660,35 @@ void hmp_info_balloon(Monitor *mon, const QDict *qdict)
+@@ -695,7 +695,35 @@ void hmp_info_balloon(Monitor *mon, const QDict *qdict)
return;
}
@@ -98,13 +98,13 @@ index ae4b6a4246..6e26ea2cd0 100644
qapi_free_BalloonInfo(info);
}
-diff --git a/qapi/misc.json b/qapi/misc.json
-index 9d32820dc1..44b1fb6fa7 100644
---- a/qapi/misc.json
-+++ b/qapi/misc.json
-@@ -226,10 +226,30 @@
- #
- # @actual: the number of bytes the balloon currently contains
+diff --git a/qapi/machine.json b/qapi/machine.json
+index 7c9a263778..3e59199280 100644
+--- a/qapi/machine.json
++++ b/qapi/machine.json
+@@ -1205,10 +1205,30 @@
+ # @actual: the logical size of the VM in bytes
+ # Formula used: logical_vm_size = vm_ram_size - balloon_size
#
+# @last_update: time when stats got updated from guest
+#
diff --git a/debian/patches/pve/0015-PVE-qapi-modify-query-machines.patch b/debian/patches/pve/0015-PVE-qapi-modify-query-machines.patch
index efb2984..8b403fa 100644
--- a/debian/patches/pve/0015-PVE-qapi-modify-query-machines.patch
+++ b/debian/patches/pve/0015-PVE-qapi-modify-query-machines.patch
@@ -13,7 +13,7 @@ Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
2 files changed, 9 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
-index 963088b798..32f630549e 100644
+index 5362c80a18..3fcb82ce2f 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -234,6 +234,12 @@ MachineInfoList *qmp_query_machines(Error **errp)
@@ -30,10 +30,10 @@ index 963088b798..32f630549e 100644
info->default_cpu_type = g_strdup(mc->default_cpu_type);
info->has_default_cpu_type = true;
diff --git a/qapi/machine.json b/qapi/machine.json
-index 481b1f07ec..268044a34b 100644
+index 3e59199280..dfc1a49d3c 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
-@@ -342,6 +342,8 @@
+@@ -318,6 +318,8 @@
#
# @is-default: whether the machine is default
#
@@ -42,12 +42,12 @@ index 481b1f07ec..268044a34b 100644
# @cpu-max: maximum number of CPUs supported by the machine type
# (since 1.5.0)
#
-@@ -361,7 +363,7 @@
+@@ -339,7 +341,7 @@
##
{ 'struct': 'MachineInfo',
'data': { 'name': 'str', '*alias': 'str',
- '*is-default': 'bool', 'cpu-max': 'int',
+ '*is-default': 'bool', '*is-current': 'bool', 'cpu-max': 'int',
'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool',
- 'deprecated': 'bool', '*default-cpu-type': 'str' } }
-
+ 'deprecated': 'bool', '*default-cpu-type': 'str',
+ '*default-ram-id': 'str' } }
diff --git a/debian/patches/pve/0016-PVE-qapi-modify-spice-query.patch b/debian/patches/pve/0016-PVE-qapi-modify-spice-query.patch
index e600590..437fc8e 100644
--- a/debian/patches/pve/0016-PVE-qapi-modify-spice-query.patch
+++ b/debian/patches/pve/0016-PVE-qapi-modify-spice-query.patch
@@ -12,10 +12,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 files changed, 8 insertions(+)
diff --git a/qapi/ui.json b/qapi/ui.json
-index 9d6721037f..af87b18db9 100644
+index 6c7b33cb72..39ff301d1e 100644
--- a/qapi/ui.json
+++ b/qapi/ui.json
-@@ -214,11 +214,14 @@
+@@ -215,11 +215,14 @@
#
# @channels: a list of @SpiceChannel for each active spice channel
#
@@ -31,10 +31,10 @@ index 9d6721037f..af87b18db9 100644
'if': 'defined(CONFIG_SPICE)' }
diff --git a/ui/spice-core.c b/ui/spice-core.c
-index ca04965ead..243466c13d 100644
+index d09ee7f09e..da3d2644d1 100644
--- a/ui/spice-core.c
+++ b/ui/spice-core.c
-@@ -539,6 +539,11 @@ SpiceInfo *qmp_query_spice(Error **errp)
+@@ -538,6 +538,11 @@ static SpiceInfo *qmp_query_spice_real(Error **errp)
micro = SPICE_SERVER_VERSION & 0xff;
info->compiled_version = g_strdup_printf("%d.%d.%d", major, minor, micro);
diff --git a/debian/patches/pve/0017-PVE-internal-snapshot-async.patch b/debian/patches/pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
similarity index 84%
rename from debian/patches/pve/0017-PVE-internal-snapshot-async.patch
rename to debian/patches/pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
index 817aad4..e70e69a 100644
--- a/debian/patches/pve/0017-PVE-internal-snapshot-async.patch
+++ b/debian/patches/pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
@@ -1,9 +1,7 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Dietmar Maurer <dietmar@proxmox.com>
Date: Mon, 6 Apr 2020 12:16:46 +0200
-Subject: [PATCH] PVE: internal snapshot async
-
-Truncate at 1024 boundary (Fabian Ebner will send a patch for stable)
+Subject: [PATCH] PVE: add savevm-async for background state snapshots
Put qemu_savevm_state_{header,setup} into the main loop and the rest
of the iteration into a coroutine. The former need to lock the
@@ -11,40 +9,37 @@ iothread (and we can't unlock it in the coroutine), and the latter
can't deal with being in a separate thread, so a coroutine it must
be.
+Truncate output file at 1024 boundary.
+
+Do not block the VM and save the state on aborting a snapshot, as the
+snapshot will be invalid anyway.
+
+Also, when aborting, wait for the target file to be closed, otherwise a
+client might run into race-conditions when trying to remove the file
+still opened by QEMU.
+
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
+[improve aborting]
+Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
- Makefile.objs | 1 +
hmp-commands-info.hx | 13 +
- hmp-commands.hx | 32 +++
- include/block/aio.h | 10 +
+ hmp-commands.hx | 33 ++
include/migration/snapshot.h | 1 +
include/monitor/hmp.h | 5 +
+ migration/meson.build | 1 +
+ migration/savevm-async.c | 591 +++++++++++++++++++++++++++++++++++
monitor/hmp-cmds.c | 57 ++++
- qapi/migration.json | 34 +++
- qapi/misc.json | 32 +++
+ qapi/migration.json | 34 ++
+ qapi/misc.json | 32 ++
qemu-options.hx | 12 +
- savevm-async.c | 542 +++++++++++++++++++++++++++++++++++
softmmu/vl.c | 10 +
- util/async.c | 30 ++
- 13 files changed, 779 insertions(+)
- create mode 100644 savevm-async.c
+ 11 files changed, 789 insertions(+)
+ create mode 100644 migration/savevm-async.c
-diff --git a/Makefile.objs b/Makefile.objs
-index d22b3b45d7..a1307c12a8 100644
---- a/Makefile.objs
-+++ b/Makefile.objs
-@@ -46,6 +46,7 @@ common-obj-y += bootdevice.o iothread.o
- common-obj-y += dump/
- common-obj-y += job-qmp.o
- common-obj-y += monitor/
-+common-obj-y += savevm-async.o
- common-obj-y += net/
- common-obj-y += qdev-monitor.o
- common-obj-$(CONFIG_WIN32) += os-win32.o
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
-index 30209e3903..ae8ff21789 100644
+index 117ba25f91..b3b797ca28 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -580,6 +580,19 @@ SRST
@@ -68,10 +63,10 @@ index 30209e3903..ae8ff21789 100644
.name = "balloon",
.args_type = "",
diff --git a/hmp-commands.hx b/hmp-commands.hx
-index 60f395c276..2b58ac4a1c 100644
+index ff2d7aa8f3..d294c234a5 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
-@@ -1829,3 +1829,35 @@ ERST
+@@ -1866,3 +1866,36 @@ ERST
.flags = "p",
},
@@ -105,36 +100,9 @@ index 60f395c276..2b58ac4a1c 100644
+ .args_type = "",
+ .params = "",
+ .help = "Resume VM after snaphot.",
-+ .cmd = hmp_savevm_end,
++ .cmd = hmp_savevm_end,
++ .coroutine = true,
+ },
-diff --git a/include/block/aio.h b/include/block/aio.h
-index b2f703fa3f..c37617b404 100644
---- a/include/block/aio.h
-+++ b/include/block/aio.h
-@@ -17,6 +17,7 @@
- #ifdef CONFIG_LINUX_IO_URING
- #include <liburing.h>
- #endif
-+#include "qemu/coroutine.h"
- #include "qemu/queue.h"
- #include "qemu/event_notifier.h"
- #include "qemu/thread.h"
-@@ -654,6 +655,15 @@ static inline bool aio_node_check(AioContext *ctx, bool is_external)
- */
- void aio_co_schedule(AioContext *ctx, struct Coroutine *co);
-
-+/**
-+ * aio_co_reschedule_self:
-+ * @new_ctx: the new context
-+ *
-+ * Move the currently running coroutine to new_ctx. If the coroutine is already
-+ * running in new_ctx, do nothing.
-+ */
-+void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx);
-+
- /**
- * aio_co_wake:
- * @co: the coroutine
diff --git a/include/migration/snapshot.h b/include/migration/snapshot.h
index c85b6ec75b..4411b7121d 100644
--- a/include/migration/snapshot.h
@@ -147,7 +115,7 @@ index c85b6ec75b..4411b7121d 100644
#endif
diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
-index c986cfd28b..243952d32f 100644
+index ed2913fd18..4e06f89e8e 100644
--- a/include/monitor/hmp.h
+++ b/include/monitor/hmp.h
@@ -25,6 +25,7 @@ void hmp_info_status(Monitor *mon, const QDict *qdict);
@@ -169,191 +137,24 @@ index c986cfd28b..243952d32f 100644
void hmp_sendkey(Monitor *mon, const QDict *qdict);
void hmp_screendump(Monitor *mon, const QDict *qdict);
void hmp_chardev_add(Monitor *mon, const QDict *qdict);
-diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
-index 6e26ea2cd0..280bb447a6 100644
---- a/monitor/hmp-cmds.c
-+++ b/monitor/hmp-cmds.c
-@@ -1904,6 +1904,63 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict)
- hmp_handle_error(mon, err);
- }
-
-+void hmp_savevm_start(Monitor *mon, const QDict *qdict)
-+{
-+ Error *errp = NULL;
-+ const char *statefile = qdict_get_try_str(qdict, "statefile");
-+
-+ qmp_savevm_start(statefile != NULL, statefile, &errp);
-+ hmp_handle_error(mon, errp);
-+}
-+
-+void hmp_snapshot_drive(Monitor *mon, const QDict *qdict)
-+{
-+ Error *errp = NULL;
-+ const char *name = qdict_get_str(qdict, "name");
-+ const char *device = qdict_get_str(qdict, "device");
-+
-+ qmp_snapshot_drive(device, name, &errp);
-+ hmp_handle_error(mon, errp);
-+}
-+
-+void hmp_delete_drive_snapshot(Monitor *mon, const QDict *qdict)
-+{
-+ Error *errp = NULL;
-+ const char *name = qdict_get_str(qdict, "name");
-+ const char *device = qdict_get_str(qdict, "device");
-+
-+ qmp_delete_drive_snapshot(device, name, &errp);
-+ hmp_handle_error(mon, errp);
-+}
-+
-+void hmp_savevm_end(Monitor *mon, const QDict *qdict)
-+{
-+ Error *errp = NULL;
-+
-+ qmp_savevm_end(&errp);
-+ hmp_handle_error(mon, errp);
-+}
-+
-+void hmp_info_savevm(Monitor *mon, const QDict *qdict)
-+{
-+ SaveVMInfo *info;
-+ info = qmp_query_savevm(NULL);
-+
-+ if (info->has_status) {
-+ monitor_printf(mon, "savevm status: %s\n", info->status);
-+ monitor_printf(mon, "total time: %" PRIu64 " milliseconds\n",
-+ info->total_time);
-+ } else {
-+ monitor_printf(mon, "savevm status: not running\n");
-+ }
-+ if (info->has_bytes) {
-+ monitor_printf(mon, "Bytes saved: %"PRIu64"\n", info->bytes);
-+ }
-+ if (info->has_error) {
-+ monitor_printf(mon, "Error: %s\n", info->error);
-+ }
-+}
-+
- void hmp_info_iothreads(Monitor *mon, const QDict *qdict)
- {
- IOThreadInfoList *info_list = qmp_query_iothreads(NULL);
-diff --git a/qapi/migration.json b/qapi/migration.json
-index ea53b23dca..c556257544 100644
---- a/qapi/migration.json
-+++ b/qapi/migration.json
-@@ -225,6 +225,40 @@
- '*compression': 'CompressionStats',
- '*socket-address': ['SocketAddress'] } }
-
-+##
-+# @SaveVMInfo:
-+#
-+# Information about current migration process.
-+#
-+# @status: string describing the current savevm status.
-+# This can be 'active', 'completed', 'failed'.
-+# If this field is not returned, no savevm process
-+# has been initiated
-+#
-+# @error: string containing error message is status is failed.
-+#
-+# @total-time: total amount of milliseconds since savevm started.
-+# If savevm has ended, it returns the total save time
-+#
-+# @bytes: total amount of data transfered
-+#
-+# Since: 1.3
-+##
-+{ 'struct': 'SaveVMInfo',
-+ 'data': {'*status': 'str', '*error': 'str',
-+ '*total-time': 'int', '*bytes': 'int'} }
-+
-+##
-+# @query-savevm:
-+#
-+# Returns information about current savevm process.
-+#
-+# Returns: @SaveVMInfo
-+#
-+# Since: 1.3
-+##
-+{ 'command': 'query-savevm', 'returns': 'SaveVMInfo' }
-+
- ##
- # @query-migrate:
- #
-diff --git a/qapi/misc.json b/qapi/misc.json
-index 44b1fb6fa7..9895899f8b 100644
---- a/qapi/misc.json
-+++ b/qapi/misc.json
-@@ -1168,6 +1168,38 @@
- ##
- { 'command': 'query-fdsets', 'returns': ['FdsetInfo'] }
-
-+##
-+# @savevm-start:
-+#
-+# Prepare for snapshot and halt VM. Save VM state to statefile.
-+#
-+##
-+{ 'command': 'savevm-start', 'data': { '*statefile': 'str' } }
-+
-+##
-+# @snapshot-drive:
-+#
-+# Create an internal drive snapshot.
-+#
-+##
-+{ 'command': 'snapshot-drive', 'data': { 'device': 'str', 'name': 'str' } }
-+
-+##
-+# @delete-drive-snapshot:
-+#
-+# Delete a drive snapshot.
-+#
-+##
-+{ 'command': 'delete-drive-snapshot', 'data': { 'device': 'str', 'name': 'str' } }
-+
-+##
-+# @savevm-end:
-+#
-+# Resume VM after a snapshot.
-+#
-+##
-+{ 'command': 'savevm-end' }
-+
- ##
- # @AcpiTableOptions:
- #
-diff --git a/qemu-options.hx b/qemu-options.hx
-index 708583b4ce..d32995cc50 100644
---- a/qemu-options.hx
-+++ b/qemu-options.hx
-@@ -3866,6 +3866,18 @@ SRST
- Start right away with a saved state (``loadvm`` in monitor)
- ERST
-
-+DEF("loadstate", HAS_ARG, QEMU_OPTION_loadstate, \
-+ "-loadstate file\n" \
-+ " start right away with a saved state\n",
-+ QEMU_ARCH_ALL)
-+SRST
-+``-loadstate file``
-+ Start right away with a saved state. This option does not rollback
-+ disk state like @code{loadvm}, so user must make sure that disk
-+ have correct state. @var{file} can be any valid device URL. See the section
-+ for "Device URL Syntax" for more information.
-+ERST
-+
- #ifndef _WIN32
- DEF("daemonize", 0, QEMU_OPTION_daemonize, \
- "-daemonize daemonize QEMU after initializing\n", QEMU_ARCH_ALL)
-diff --git a/savevm-async.c b/savevm-async.c
+diff --git a/migration/meson.build b/migration/meson.build
+index 980e37865c..e62b79b60f 100644
+--- a/migration/meson.build
++++ b/migration/meson.build
+@@ -23,6 +23,7 @@ softmmu_ss.add(files(
+ 'multifd-zlib.c',
+ 'postcopy-ram.c',
+ 'savevm.c',
++ 'savevm-async.c',
+ 'socket.c',
+ 'tls.c',
+ ))
+diff --git a/migration/savevm-async.c b/migration/savevm-async.c
new file mode 100644
-index 0000000000..f918e18dce
+index 0000000000..4e345c1a7d
--- /dev/null
-+++ b/savevm-async.c
-@@ -0,0 +1,542 @@
++++ b/migration/savevm-async.c
+@@ -0,0 +1,591 @@
+#include "qemu/osdep.h"
+#include "migration/migration.h"
+#include "migration/savevm.h"
@@ -372,6 +173,7 @@ index 0000000000..f918e18dce
+#include "qapi/qapi-commands-misc.h"
+#include "qapi/qapi-commands-block.h"
+#include "qemu/cutils.h"
++#include "qemu/timer.h"
+#include "qemu/main-loop.h"
+#include "qemu/rcu.h"
+
@@ -408,8 +210,15 @@ index 0000000000..f918e18dce
+ int64_t total_time;
+ QEMUBH *finalize_bh;
+ Coroutine *co;
++ QemuCoSleepState *target_close_wait;
+} snap_state;
+
++static bool savevm_aborted(void)
++{
++ return snap_state.state == SAVE_STATE_CANCELLED ||
++ snap_state.state == SAVE_STATE_ERROR;
++}
++
+SaveVMInfo *qmp_query_savevm(Error **errp)
+{
+ SaveVMInfo *info = g_malloc0(sizeof(*info));
@@ -462,17 +271,23 @@ index 0000000000..f918e18dce
+ }
+
+ if (snap_state.target) {
-+ /* try to truncate, but ignore errors (will fail on block devices).
-+ * note1: bdrv_read() need whole blocks, so we need to round up
-+ * note2: PVE requires 1024 (BDRV_SECTOR_SIZE*2) alignment
-+ */
-+ size_t size = QEMU_ALIGN_UP(snap_state.bs_pos, BDRV_SECTOR_SIZE*2);
-+ blk_truncate(snap_state.target, size, false, PREALLOC_MODE_OFF, 0, NULL);
++ if (!savevm_aborted()) {
++ /* try to truncate, but ignore errors (will fail on block devices).
++ * note1: bdrv_read() need whole blocks, so we need to round up
++ * note2: PVE requires 1024 (BDRV_SECTOR_SIZE*2) alignment
++ */
++ size_t size = QEMU_ALIGN_UP(snap_state.bs_pos, BDRV_SECTOR_SIZE*2);
++ blk_truncate(snap_state.target, size, false, PREALLOC_MODE_OFF, 0, NULL);
++ }
+ blk_op_unblock_all(snap_state.target, snap_state.blocker);
+ error_free(snap_state.blocker);
+ snap_state.blocker = NULL;
+ blk_unref(snap_state.target);
+ snap_state.target = NULL;
++
++ if (snap_state.target_close_wait) {
++ qemu_co_sleep_wake(snap_state.target_close_wait);
++ }
+ }
+
+ return ret;
@@ -558,6 +373,8 @@ index 0000000000..f918e18dce
+ AioContext *iohandler_ctx = iohandler_get_aio_context();
+ MigrationState *ms = migrate_get_current();
+
++ bool aborted = savevm_aborted();
++
+#ifdef DEBUG_SAVEVM_STATE
+ int64_t start_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
+#endif
@@ -579,10 +396,13 @@ index 0000000000..f918e18dce
+ save_snapshot_error("vm_stop_force_state error %d", ret);
+ }
+
-+ (void)qemu_savevm_state_complete_precopy(snap_state.file, false, false);
-+ ret = qemu_file_get_error(snap_state.file);
-+ if (ret < 0) {
-+ save_snapshot_error("qemu_savevm_state_iterate error %d", ret);
++ if (!aborted) {
++ /* skip state saving if we aborted, snapshot will be invalid anyway */
++ (void)qemu_savevm_state_complete_precopy(snap_state.file, false, false);
++ ret = qemu_file_get_error(snap_state.file);
++ if (ret < 0) {
++ save_snapshot_error("qemu_savevm_state_iterate error %d", ret);
++ }
+ }
+
+ DPRINTF("state saving complete\n");
@@ -591,7 +411,7 @@ index 0000000000..f918e18dce
+
+ /* clear migration state */
+ migrate_set_state(&ms->state, MIGRATION_STATUS_SETUP,
-+ ret ? MIGRATION_STATUS_FAILED : MIGRATION_STATUS_COMPLETED);
++ ret || aborted ? MIGRATION_STATUS_FAILED : MIGRATION_STATUS_COMPLETED);
+ ms->to_dst_file = NULL;
+
+ qemu_savevm_state_cleanup();
@@ -601,6 +421,9 @@ index 0000000000..f918e18dce
+ save_snapshot_error("save_snapshot_cleanup error %d", ret);
+ } else if (snap_state.state == SAVE_STATE_ACTIVE) {
+ snap_state.state = SAVE_STATE_COMPLETED;
++ } else if (aborted) {
++ save_snapshot_error("process_savevm_cleanup: found aborted state: %d",
++ snap_state.state);
+ } else {
+ save_snapshot_error("process_savevm_cleanup: invalid state: %d",
+ snap_state.state);
@@ -790,11 +613,14 @@ index 0000000000..f918e18dce
+
+ if (snap_state.saved_vm_running) {
+ vm_start();
++ snap_state.saved_vm_running = false;
+ }
+}
+
-+void qmp_savevm_end(Error **errp)
++void coroutine_fn qmp_savevm_end(Error **errp)
+{
++ int64_t timeout;
++
+ if (snap_state.state == SAVE_STATE_DONE) {
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
+ "VM snapshot not started\n");
@@ -803,14 +629,38 @@ index 0000000000..f918e18dce
+
+ if (snap_state.state == SAVE_STATE_ACTIVE) {
+ snap_state.state = SAVE_STATE_CANCELLED;
-+ return;
++ goto wait_for_close;
+ }
+
+ if (snap_state.saved_vm_running) {
+ vm_start();
++ snap_state.saved_vm_running = false;
+ }
+
+ snap_state.state = SAVE_STATE_DONE;
++
++wait_for_close:
++ if (!snap_state.target) {
++ DPRINTF("savevm-end: no target file open\n");
++ return;
++ }
++
++ /* wait until cleanup is done before returning, this ensures that after this
++ * call exits the statefile will be closed and can be removed immediately */
++ DPRINTF("savevm-end: waiting for cleanup\n");
++ timeout = 30L * 1000 * 1000 * 1000;
++ qemu_co_sleep_ns_wakeable(QEMU_CLOCK_REALTIME, timeout,
++ &snap_state.target_close_wait);
++ snap_state.target_close_wait = NULL;
++ if (snap_state.target) {
++ save_snapshot_error("timeout waiting for target file close in "
++ "qmp_savevm_end");
++ /* we cannot assume the snapshot finished in this case, so leave the
++ * state alone - caller has to figure something out */
++ return;
++ }
++
++ DPRINTF("savevm-end: cleanup done\n");
+}
+
+// FIXME: Deprecated
@@ -896,11 +746,190 @@ index 0000000000..f918e18dce
+ }
+ return ret;
+}
+diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
+index 705f08a8f1..77ab152aab 100644
+--- a/monitor/hmp-cmds.c
++++ b/monitor/hmp-cmds.c
+@@ -1949,6 +1949,63 @@ void hmp_info_memory_devices(Monitor *mon, const QDict *qdict)
+ hmp_handle_error(mon, err);
+ }
+
++void hmp_savevm_start(Monitor *mon, const QDict *qdict)
++{
++ Error *errp = NULL;
++ const char *statefile = qdict_get_try_str(qdict, "statefile");
++
++ qmp_savevm_start(statefile != NULL, statefile, &errp);
++ hmp_handle_error(mon, errp);
++}
++
++void hmp_snapshot_drive(Monitor *mon, const QDict *qdict)
++{
++ Error *errp = NULL;
++ const char *name = qdict_get_str(qdict, "name");
++ const char *device = qdict_get_str(qdict, "device");
++
++ qmp_snapshot_drive(device, name, &errp);
++ hmp_handle_error(mon, errp);
++}
++
++void hmp_delete_drive_snapshot(Monitor *mon, const QDict *qdict)
++{
++ Error *errp = NULL;
++ const char *name = qdict_get_str(qdict, "name");
++ const char *device = qdict_get_str(qdict, "device");
++
++ qmp_delete_drive_snapshot(device, name, &errp);
++ hmp_handle_error(mon, errp);
++}
++
++void coroutine_fn hmp_savevm_end(Monitor *mon, const QDict *qdict)
++{
++ Error *errp = NULL;
++
++ qmp_savevm_end(&errp);
++ hmp_handle_error(mon, errp);
++}
++
++void hmp_info_savevm(Monitor *mon, const QDict *qdict)
++{
++ SaveVMInfo *info;
++ info = qmp_query_savevm(NULL);
++
++ if (info->has_status) {
++ monitor_printf(mon, "savevm status: %s\n", info->status);
++ monitor_printf(mon, "total time: %" PRIu64 " milliseconds\n",
++ info->total_time);
++ } else {
++ monitor_printf(mon, "savevm status: not running\n");
++ }
++ if (info->has_bytes) {
++ monitor_printf(mon, "Bytes saved: %"PRIu64"\n", info->bytes);
++ }
++ if (info->has_error) {
++ monitor_printf(mon, "Error: %s\n", info->error);
++ }
++}
++
+ void hmp_info_iothreads(Monitor *mon, const QDict *qdict)
+ {
+ IOThreadInfoList *info_list = qmp_query_iothreads(NULL);
+diff --git a/qapi/migration.json b/qapi/migration.json
+index 3c75820527..cb3627884c 100644
+--- a/qapi/migration.json
++++ b/qapi/migration.json
+@@ -242,6 +242,40 @@
+ '*compression': 'CompressionStats',
+ '*socket-address': ['SocketAddress'] } }
+
++##
++# @SaveVMInfo:
++#
++# Information about current migration process.
++#
++# @status: string describing the current savevm status.
++# This can be 'active', 'completed', 'failed'.
++# If this field is not returned, no savevm process
++# has been initiated
++#
++# @error: string containing error message is status is failed.
++#
++# @total-time: total amount of milliseconds since savevm started.
++# If savevm has ended, it returns the total save time
++#
++# @bytes: total amount of data transfered
++#
++# Since: 1.3
++##
++{ 'struct': 'SaveVMInfo',
++ 'data': {'*status': 'str', '*error': 'str',
++ '*total-time': 'int', '*bytes': 'int'} }
++
++##
++# @query-savevm:
++#
++# Returns information about current savevm process.
++#
++# Returns: @SaveVMInfo
++#
++# Since: 1.3
++##
++{ 'command': 'query-savevm', 'returns': 'SaveVMInfo' }
++
+ ##
+ # @query-migrate:
+ #
+diff --git a/qapi/misc.json b/qapi/misc.json
+index 40df513856..4f5333d960 100644
+--- a/qapi/misc.json
++++ b/qapi/misc.json
+@@ -476,6 +476,38 @@
+ ##
+ { 'command': 'query-fdsets', 'returns': ['FdsetInfo'] }
+
++##
++# @savevm-start:
++#
++# Prepare for snapshot and halt VM. Save VM state to statefile.
++#
++##
++{ 'command': 'savevm-start', 'data': { '*statefile': 'str' } }
++
++##
++# @snapshot-drive:
++#
++# Create an internal drive snapshot.
++#
++##
++{ 'command': 'snapshot-drive', 'data': { 'device': 'str', 'name': 'str' } }
++
++##
++# @delete-drive-snapshot:
++#
++# Delete a drive snapshot.
++#
++##
++{ 'command': 'delete-drive-snapshot', 'data': { 'device': 'str', 'name': 'str' } }
++
++##
++# @savevm-end:
++#
++# Resume VM after a snapshot.
++#
++##
++{ 'command': 'savevm-end', 'coroutine': true }
++
+ ##
+ # @CommandLineParameterType:
+ #
+diff --git a/qemu-options.hx b/qemu-options.hx
+index 104632ea34..c1352312c2 100644
+--- a/qemu-options.hx
++++ b/qemu-options.hx
+@@ -3903,6 +3903,18 @@ SRST
+ Start right away with a saved state (``loadvm`` in monitor)
+ ERST
+
++DEF("loadstate", HAS_ARG, QEMU_OPTION_loadstate, \
++ "-loadstate file\n" \
++ " start right away with a saved state\n",
++ QEMU_ARCH_ALL)
++SRST
++``-loadstate file``
++ Start right away with a saved state. This option does not rollback
++ disk state like @code{loadvm}, so user must make sure that disk
++ have correct state. @var{file} can be any valid device URL. See the section
++ for "Device URL Syntax" for more information.
++ERST
++
+ #ifndef _WIN32
+ DEF("daemonize", 0, QEMU_OPTION_daemonize, \
+ "-daemonize daemonize QEMU after initializing\n", QEMU_ARCH_ALL)
diff --git a/softmmu/vl.c b/softmmu/vl.c
-index 4eb9d1f7fd..670b7e427c 100644
+index e6e0ad5a92..03152c816c 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
-@@ -2844,6 +2844,7 @@ void qemu_init(int argc, char **argv, char **envp)
+@@ -2878,6 +2878,7 @@ void qemu_init(int argc, char **argv, char **envp)
int optind;
const char *optarg;
const char *loadvm = NULL;
@@ -908,7 +937,7 @@ index 4eb9d1f7fd..670b7e427c 100644
MachineClass *machine_class;
const char *cpu_option;
const char *vga_model = NULL;
-@@ -3408,6 +3409,9 @@ void qemu_init(int argc, char **argv, char **envp)
+@@ -3439,6 +3440,9 @@ void qemu_init(int argc, char **argv, char **envp)
case QEMU_OPTION_loadvm:
loadvm = optarg;
break;
@@ -918,7 +947,7 @@ index 4eb9d1f7fd..670b7e427c 100644
case QEMU_OPTION_full_screen:
dpy.has_full_screen = true;
dpy.full_screen = true;
-@@ -4464,6 +4468,12 @@ void qemu_init(int argc, char **argv, char **envp)
+@@ -4478,6 +4482,12 @@ void qemu_init(int argc, char **argv, char **envp)
autostart = 0;
exit(1);
}
@@ -931,44 +960,3 @@ index 4eb9d1f7fd..670b7e427c 100644
}
if (replay_mode != REPLAY_MODE_NONE) {
replay_vmstate_init();
-diff --git a/util/async.c b/util/async.c
-index 1319eee3bc..b68e73f488 100644
---- a/util/async.c
-+++ b/util/async.c
-@@ -559,6 +559,36 @@ void aio_co_schedule(AioContext *ctx, Coroutine *co)
- aio_context_unref(ctx);
- }
-
-+typedef struct AioCoRescheduleSelf {
-+ Coroutine *co;
-+ AioContext *new_ctx;
-+} AioCoRescheduleSelf;
-+
-+static void aio_co_reschedule_self_bh(void *opaque)
-+{
-+ AioCoRescheduleSelf *data = opaque;
-+ aio_co_schedule(data->new_ctx, data->co);
-+}
-+
-+void coroutine_fn aio_co_reschedule_self(AioContext *new_ctx)
-+{
-+ AioContext *old_ctx = qemu_get_current_aio_context();
-+
-+ if (old_ctx != new_ctx) {
-+ AioCoRescheduleSelf data = {
-+ .co = qemu_coroutine_self(),
-+ .new_ctx = new_ctx,
-+ };
-+ /*
-+ * We can't directly schedule the coroutine in the target context
-+ * because this would be racy: The other thread could try to enter the
-+ * coroutine before it has yielded in this one.
-+ */
-+ aio_bh_schedule_oneshot(old_ctx, aio_co_reschedule_self_bh, &data);
-+ qemu_coroutine_yield();
-+ }
-+}
-+
- void aio_co_wake(struct Coroutine *co)
- {
- AioContext *ctx;
diff --git a/debian/patches/pve/0018-add-optional-buffer-size-to-QEMUFile.patch b/debian/patches/pve/0018-PVE-add-optional-buffer-size-to-QEMUFile.patch
similarity index 87%
rename from debian/patches/pve/0018-add-optional-buffer-size-to-QEMUFile.patch
rename to debian/patches/pve/0018-PVE-add-optional-buffer-size-to-QEMUFile.patch
index efa45af..d3e7b73 100644
--- a/debian/patches/pve/0018-add-optional-buffer-size-to-QEMUFile.patch
+++ b/debian/patches/pve/0018-PVE-add-optional-buffer-size-to-QEMUFile.patch
@@ -1,31 +1,35 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Wolfgang Bumiller <w.bumiller@proxmox.com>
Date: Mon, 4 May 2020 11:05:08 +0200
-Subject: [PATCH] add optional buffer size to QEMUFile
+Subject: [PATCH] PVE: add optional buffer size to QEMUFile
So we can use a 4M buffer for savevm-async which should
increase performance storing the state onto ceph.
Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
+[increase max IOV count in QEMUFile to actually write more data]
+Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
- migration/qemu-file.c | 36 ++++++++++++++++++++++++------------
- migration/qemu-file.h | 1 +
- savevm-async.c | 4 ++--
- 3 files changed, 27 insertions(+), 14 deletions(-)
+ migration/qemu-file.c | 38 +++++++++++++++++++++++++-------------
+ migration/qemu-file.h | 1 +
+ migration/savevm-async.c | 4 ++--
+ 3 files changed, 28 insertions(+), 15 deletions(-)
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
-index be21518c57..a4d2e2c8ff 100644
+index be21518c57..1926b5202c 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
-@@ -30,7 +30,7 @@
+@@ -30,8 +30,8 @@
#include "trace.h"
#include "qapi/error.h"
-#define IO_BUF_SIZE 32768
+-#define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64)
+#define DEFAULT_IO_BUF_SIZE 32768
- #define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64)
++#define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 256)
struct QEMUFile {
+ const QEMUFileOps *ops;
@@ -45,7 +45,8 @@ struct QEMUFile {
when reading */
int buf_index;
@@ -159,11 +163,11 @@ index a9b6d6ccb7..8752d27c74 100644
void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks);
int qemu_get_fd(QEMUFile *f);
int qemu_fclose(QEMUFile *f);
-diff --git a/savevm-async.c b/savevm-async.c
-index f918e18dce..156b7a030e 100644
---- a/savevm-async.c
-+++ b/savevm-async.c
-@@ -392,7 +392,7 @@ void qmp_savevm_start(bool has_statefile, const char *statefile, Error **errp)
+diff --git a/migration/savevm-async.c b/migration/savevm-async.c
+index 4e345c1a7d..8a17ec1f74 100644
+--- a/migration/savevm-async.c
++++ b/migration/savevm-async.c
+@@ -414,7 +414,7 @@ void qmp_savevm_start(bool has_statefile, const char *statefile, Error **errp)
goto restart;
}
@@ -172,7 +176,7 @@ index f918e18dce..156b7a030e 100644
if (!snap_state.file) {
error_set(errp, ERROR_CLASS_GENERIC_ERROR, "failed to open '%s'", statefile);
-@@ -514,7 +514,7 @@ int load_snapshot_from_blockdev(const char *filename, Error **errp)
+@@ -563,7 +563,7 @@ int load_snapshot_from_blockdev(const char *filename, Error **errp)
blk_op_block_all(be, blocker);
/* restore the VM state */
diff --git a/debian/patches/pve/0019-PVE-block-add-the-zeroinit-block-driver-filter.patch b/debian/patches/pve/0019-PVE-block-add-the-zeroinit-block-driver-filter.patch
index 7c24b16..ce302e8 100644
--- a/debian/patches/pve/0019-PVE-block-add-the-zeroinit-block-driver-filter.patch
+++ b/debian/patches/pve/0019-PVE-block-add-the-zeroinit-block-driver-filter.patch
@@ -5,29 +5,29 @@ Subject: [PATCH] PVE: block: add the zeroinit block driver filter
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
- block/Makefile.objs | 1 +
- block/zeroinit.c | 198 ++++++++++++++++++++++++++++++++++++++++++++
- 2 files changed, 199 insertions(+)
+ block/meson.build | 1 +
+ block/zeroinit.c | 196 ++++++++++++++++++++++++++++++++++++++++++++++
+ 2 files changed, 197 insertions(+)
create mode 100644 block/zeroinit.c
-diff --git a/block/Makefile.objs b/block/Makefile.objs
-index 19c6f371c9..d1a9227b8f 100644
---- a/block/Makefile.objs
-+++ b/block/Makefile.objs
-@@ -11,6 +11,7 @@ block-obj-$(CONFIG_QED) += qed.o qed-l2-cache.o qed-table.o qed-cluster.o
- block-obj-$(CONFIG_QED) += qed-check.o
- block-obj-y += vhdx.o vhdx-endian.o vhdx-log.o
- block-obj-y += quorum.o
-+block-obj-y += zeroinit.o
- block-obj-y += blkdebug.o blkverify.o blkreplay.o
- block-obj-$(CONFIG_PARALLELS) += parallels.o
- block-obj-y += blklogwrites.o
+diff --git a/block/meson.build b/block/meson.build
+index 5dcc1e5cce..c10d544864 100644
+--- a/block/meson.build
++++ b/block/meson.build
+@@ -39,6 +39,7 @@ block_ss.add(files(
+ 'vmdk.c',
+ 'vpc.c',
+ 'write-threshold.c',
++ 'zeroinit.c',
+ ), zstd, zlib)
+
+ softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
diff --git a/block/zeroinit.c b/block/zeroinit.c
new file mode 100644
-index 0000000000..4fbb80eab0
+index 0000000000..5529627f7e
--- /dev/null
+++ b/block/zeroinit.c
-@@ -0,0 +1,198 @@
+@@ -0,0 +1,196 @@
+/*
+ * Filter to fake a zero-initialized block device.
+ *
@@ -212,8 +212,6 @@ index 0000000000..4fbb80eab0
+
+ .bdrv_has_zero_init = zeroinit_has_zero_init,
+
-+ .bdrv_co_block_status = bdrv_co_block_status_from_file,
-+
+ .bdrv_co_pdiscard = zeroinit_co_pdiscard,
+
+ .bdrv_co_truncate = zeroinit_co_truncate,
diff --git a/debian/patches/pve/0020-PVE-Add-dummy-id-command-line-parameter.patch b/debian/patches/pve/0020-PVE-Add-dummy-id-command-line-parameter.patch
index aa56372..73ded0c 100644
--- a/debian/patches/pve/0020-PVE-Add-dummy-id-command-line-parameter.patch
+++ b/debian/patches/pve/0020-PVE-Add-dummy-id-command-line-parameter.patch
@@ -14,10 +14,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 files changed, 11 insertions(+)
diff --git a/qemu-options.hx b/qemu-options.hx
-index d32995cc50..abfde19ce0 100644
+index c1352312c2..9a0cb6780e 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
-@@ -914,6 +914,9 @@ DEFHEADING()
+@@ -906,6 +906,9 @@ DEFHEADING()
DEFHEADING(Block device options:)
@@ -28,10 +28,10 @@ index d32995cc50..abfde19ce0 100644
"-fda/-fdb file use 'file' as floppy disk 0/1 image\n", QEMU_ARCH_ALL)
DEF("fdb", HAS_ARG, QEMU_OPTION_fdb, "", QEMU_ARCH_ALL)
diff --git a/softmmu/vl.c b/softmmu/vl.c
-index 670b7e427c..366e30e594 100644
+index 03152c816c..da204d24f0 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
-@@ -2832,6 +2832,7 @@ static void create_default_memdev(MachineState *ms, const char *path)
+@@ -2866,6 +2866,7 @@ static char *find_datadir(void)
void qemu_init(int argc, char **argv, char **envp)
{
int i;
@@ -39,7 +39,7 @@ index 670b7e427c..366e30e594 100644
int snapshot, linux_boot;
const char *initrd_filename;
const char *kernel_filename, *kernel_cmdline;
-@@ -3530,6 +3531,13 @@ void qemu_init(int argc, char **argv, char **envp)
+@@ -3557,6 +3558,13 @@ void qemu_init(int argc, char **argv, char **envp)
exit(1);
}
break;
diff --git a/debian/patches/pve/0021-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch b/debian/patches/pve/0021-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch
index f6d39fa..bc3d98b 100644
--- a/debian/patches/pve/0021-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch
+++ b/debian/patches/pve/0021-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch
@@ -11,7 +11,7 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 9 insertions(+)
diff --git a/hw/intc/apic_common.c b/hw/intc/apic_common.c
-index 81addd6390..c2026b07c5 100644
+index 502e94effc..590ef6ec8e 100644
--- a/hw/intc/apic_common.c
+++ b/hw/intc/apic_common.c
@@ -278,6 +278,15 @@ static void apic_reset_common(DeviceState *dev)
diff --git a/debian/patches/pve/0022-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch b/debian/patches/pve/0022-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch
index 271e5df..52c0046 100644
--- a/debian/patches/pve/0022-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch
+++ b/debian/patches/pve/0022-PVE-Up-Config-file-posix-make-locking-optiono-on-cre.patch
@@ -13,10 +13,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
2 files changed, 43 insertions(+), 21 deletions(-)
diff --git a/block/file-posix.c b/block/file-posix.c
-index bb72e1e5ca..914bd1f367 100644
+index bda3e606dc..037839622e 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
-@@ -2390,6 +2390,7 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
+@@ -2388,6 +2388,7 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
int fd;
uint64_t perm, shared;
int result = 0;
@@ -24,7 +24,7 @@ index bb72e1e5ca..914bd1f367 100644
/* Validate options and set default values */
assert(options->driver == BLOCKDEV_DRIVER_FILE);
-@@ -2431,19 +2432,22 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
+@@ -2428,19 +2429,22 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
perm = BLK_PERM_WRITE | BLK_PERM_RESIZE;
shared = BLK_PERM_ALL & ~BLK_PERM_RESIZE;
@@ -59,7 +59,7 @@ index bb72e1e5ca..914bd1f367 100644
}
/* Clear the file by truncating it to 0 */
-@@ -2497,13 +2501,15 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
+@@ -2494,13 +2498,15 @@ raw_co_create(BlockdevCreateOptions *options, Error **errp)
}
out_unlock:
@@ -82,7 +82,7 @@ index bb72e1e5ca..914bd1f367 100644
}
out_close:
-@@ -2528,6 +2534,7 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv,
+@@ -2525,6 +2531,7 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv,
PreallocMode prealloc;
char *buf = NULL;
Error *local_err = NULL;
@@ -90,7 +90,7 @@ index bb72e1e5ca..914bd1f367 100644
/* Skip file: protocol prefix */
strstart(filename, "file:", &filename);
-@@ -2550,6 +2557,18 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv,
+@@ -2547,6 +2554,18 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv,
return -EINVAL;
}
@@ -109,7 +109,7 @@ index bb72e1e5ca..914bd1f367 100644
options = (BlockdevCreateOptions) {
.driver = BLOCKDEV_DRIVER_FILE,
.u.file = {
-@@ -2561,6 +2580,8 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv,
+@@ -2558,6 +2577,8 @@ static int coroutine_fn raw_co_create_opts(BlockDriver *drv,
.nocow = nocow,
.has_extent_size_hint = has_extent_size_hint,
.extent_size_hint = extent_size_hint,
@@ -118,7 +118,7 @@ index bb72e1e5ca..914bd1f367 100644
},
};
return raw_co_create(&options, errp);
-@@ -3107,7 +3128,7 @@ static int raw_check_perm(BlockDriverState *bs, uint64_t perm, uint64_t shared,
+@@ -3104,7 +3125,7 @@ static int raw_check_perm(BlockDriverState *bs, uint64_t perm, uint64_t shared,
}
/* Copy locks to the new fd */
@@ -128,10 +128,10 @@ index bb72e1e5ca..914bd1f367 100644
false, errp);
if (ret < 0) {
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index 197bdc1c36..ea5fae22ae 100644
+index 04ad80bc1e..7957b9867d 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -4178,7 +4178,8 @@
+@@ -4203,7 +4203,8 @@
'size': 'size',
'*preallocation': 'PreallocMode',
'*nocow': 'bool',
diff --git a/debian/patches/pve/0023-PVE-monitor-disable-oob-capability.patch b/debian/patches/pve/0023-PVE-monitor-disable-oob-capability.patch
index c509f78..f51c112 100644
--- a/debian/patches/pve/0023-PVE-monitor-disable-oob-capability.patch
+++ b/debian/patches/pve/0023-PVE-monitor-disable-oob-capability.patch
@@ -18,10 +18,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/monitor/qmp.c b/monitor/qmp.c
-index d433ceae5b..a16cf3532d 100644
+index b42f8c6af3..2e37d11bd3 100644
--- a/monitor/qmp.c
+++ b/monitor/qmp.c
-@@ -409,8 +409,7 @@ void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp)
+@@ -466,8 +466,7 @@ void monitor_init_qmp(Chardev *chr, bool pretty, Error **errp)
qemu_chr_fe_set_echo(&mon->common.chr, true);
/* Note: we run QMP monitor in I/O thread when @chr supports that */
diff --git a/debian/patches/pve/0024-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch b/debian/patches/pve/0024-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch
index a7b4c00..e183b57 100644
--- a/debian/patches/pve/0024-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch
+++ b/debian/patches/pve/0024-PVE-Compat-4.0-used-balloon-qemu-4-0-config-size-fal.patch
@@ -26,10 +26,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/hw/core/machine.c b/hw/core/machine.c
-index 8d1a90c6cf..413902777e 100644
+index d0408049b5..5b38cf9356 100644
--- a/hw/core/machine.c
+++ b/hw/core/machine.c
-@@ -66,7 +66,8 @@ GlobalProperty hw_compat_4_0[] = {
+@@ -78,7 +78,8 @@ GlobalProperty hw_compat_4_0[] = {
{ "virtio-vga", "edid", "false" },
{ "virtio-gpu-device", "edid", "false" },
{ "virtio-device", "use-started", "false" },
diff --git a/debian/patches/pve/0025-PVE-Allow-version-code-in-machine-type.patch b/debian/patches/pve/0025-PVE-Allow-version-code-in-machine-type.patch
index 3cf1e2f..d4b350b 100644
--- a/debian/patches/pve/0025-PVE-Allow-version-code-in-machine-type.patch
+++ b/debian/patches/pve/0025-PVE-Allow-version-code-in-machine-type.patch
@@ -13,12 +13,12 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
hw/core/machine-qmp-cmds.c | 6 ++++++
include/hw/boards.h | 2 ++
- qapi/machine.json | 3 ++-
+ qapi/machine.json | 4 +++-
softmmu/vl.c | 15 ++++++++++++++-
- 4 files changed, 24 insertions(+), 2 deletions(-)
+ 4 files changed, 25 insertions(+), 2 deletions(-)
diff --git a/hw/core/machine-qmp-cmds.c b/hw/core/machine-qmp-cmds.c
-index 32f630549e..71e19db4e1 100644
+index 3fcb82ce2f..7868241bd5 100644
--- a/hw/core/machine-qmp-cmds.c
+++ b/hw/core/machine-qmp-cmds.c
@@ -238,6 +238,12 @@ MachineInfoList *qmp_query_machines(Error **errp)
@@ -35,10 +35,10 @@ index 32f630549e..71e19db4e1 100644
if (mc->default_cpu_type) {
diff --git a/include/hw/boards.h b/include/hw/boards.h
-index 426ce5f625..3bce25a25f 100644
+index a49e3a6b44..8e0a8c5571 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
-@@ -170,6 +170,8 @@ struct MachineClass {
+@@ -165,6 +165,8 @@ struct MachineClass {
const char *desc;
const char *deprecation_reason;
@@ -48,24 +48,32 @@ index 426ce5f625..3bce25a25f 100644
void (*reset)(MachineState *state);
void (*wakeup)(MachineState *state);
diff --git a/qapi/machine.json b/qapi/machine.json
-index 268044a34b..7a811a5860 100644
+index dfc1a49d3c..32fc674042 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
-@@ -365,7 +365,8 @@
- 'data': { 'name': 'str', '*alias': 'str',
+@@ -337,6 +337,8 @@
+ #
+ # @default-ram-id: the default ID of initial RAM memory backend (since 5.2)
+ #
++# @pve-version: custom PVE version suffix specified as 'machine+pveN'
++#
+ # Since: 1.2.0
+ ##
+ { 'struct': 'MachineInfo',
+@@ -344,7 +346,7 @@
'*is-default': 'bool', '*is-current': 'bool', 'cpu-max': 'int',
'hotpluggable-cpus': 'bool', 'numa-mem-supported': 'bool',
-- 'deprecated': 'bool', '*default-cpu-type': 'str' } }
-+ 'deprecated': 'bool', '*default-cpu-type': 'str',
-+ '*pve-version': 'str' } }
+ 'deprecated': 'bool', '*default-cpu-type': 'str',
+- '*default-ram-id': 'str' } }
++ '*default-ram-id': 'str', '*pve-version': 'str' } }
##
# @query-machines:
diff --git a/softmmu/vl.c b/softmmu/vl.c
-index 366e30e594..16aa2186b0 100644
+index da204d24f0..5b5512128e 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
-@@ -2322,6 +2322,8 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
+@@ -2325,6 +2325,8 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
{
MachineClass *mc;
GSList *el;
@@ -74,7 +82,7 @@ index 366e30e594..16aa2186b0 100644
if (is_help_option(name)) {
printf("Supported machines are:\n");
-@@ -2338,12 +2340,23 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
+@@ -2341,12 +2343,23 @@ static MachineClass *machine_parse(const char *name, GSList *machines)
exit(0);
}
diff --git a/debian/patches/pve/0026-PVE-Backup-add-vma-backup-format-code.patch b/debian/patches/pve/0026-PVE-Backup-add-vma-backup-format-code.patch
index bbc4562..4271660 100644
--- a/debian/patches/pve/0026-PVE-Backup-add-vma-backup-format-code.patch
+++ b/debian/patches/pve/0026-PVE-Backup-add-vma-backup-format-code.patch
@@ -4,51 +4,54 @@ Date: Mon, 6 Apr 2020 12:16:57 +0200
Subject: [PATCH] PVE-Backup: add vma backup format code
---
- Makefile | 3 +-
- Makefile.objs | 1 +
- vma-reader.c | 857 ++++++++++++++++++++++++++++++++++++++++++++++++++
- vma-writer.c | 790 ++++++++++++++++++++++++++++++++++++++++++++++
- vma.c | 839 ++++++++++++++++++++++++++++++++++++++++++++++++
- vma.h | 150 +++++++++
- 6 files changed, 2639 insertions(+), 1 deletion(-)
+ block/meson.build | 2 +
+ meson.build | 5 +
+ vma-reader.c | 857 ++++++++++++++++++++++++++++++++++++++++++++++
+ vma-writer.c | 790 ++++++++++++++++++++++++++++++++++++++++++
+ vma.c | 839 +++++++++++++++++++++++++++++++++++++++++++++
+ vma.h | 150 ++++++++
+ 6 files changed, 2643 insertions(+)
create mode 100644 vma-reader.c
create mode 100644 vma-writer.c
create mode 100644 vma.c
create mode 100644 vma.h
-diff --git a/Makefile b/Makefile
-index 13dd708c4a..7b8c17ce2d 100644
---- a/Makefile
-+++ b/Makefile
-@@ -479,7 +479,7 @@ dummy := $(call unnest-vars,, \
+diff --git a/block/meson.build b/block/meson.build
+index c10d544864..feffbc8623 100644
+--- a/block/meson.build
++++ b/block/meson.build
+@@ -42,6 +42,8 @@ block_ss.add(files(
+ 'zeroinit.c',
+ ), zstd, zlib)
- include $(SRC_PATH)/tests/Makefile.include
++block_ss.add(files('../vma-writer.c'), libuuid)
++
+ softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
--all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
-+all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) vma$(EXESUF) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
+ block_ss.add(when: 'CONFIG_QCOW1', if_true: files('qcow.c'))
+diff --git a/meson.build b/meson.build
+index e3386196ba..d5b660516b 100644
+--- a/meson.build
++++ b/meson.build
+@@ -725,6 +725,8 @@ keyutils = dependency('libkeyutils', required: false,
- qemu-version.h: FORCE
- $(call quiet-command, \
-@@ -602,6 +602,7 @@ qemu-img$(EXESUF): qemu-img.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io
- qemu-nbd$(EXESUF): qemu-nbd.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
- qemu-io$(EXESUF): qemu-io.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
- qemu-storage-daemon$(EXESUF): qemu-storage-daemon.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(chardev-obj-y) $(io-obj-y) $(qom-obj-y) $(storage-daemon-obj-y) $(COMMON_LDADDS)
-+vma$(EXESUF): vma.o vma-reader.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
+ has_gettid = cc.has_function('gettid')
- qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o $(COMMON_LDADDS)
++libuuid = cc.find_library('uuid', required: true)
++
+ # Malloc tests
-diff --git a/Makefile.objs b/Makefile.objs
-index a1307c12a8..ade7b17a69 100644
---- a/Makefile.objs
-+++ b/Makefile.objs
-@@ -17,6 +17,7 @@ block-obj-y = block/ nbd/ scsi/
- block-obj-y += block.o blockjob.o job.o
- block-obj-y += qemu-io-cmds.o
- block-obj-$(CONFIG_REPLICATION) += replication.o
-+block-obj-y += vma-writer.o
-
- block-obj-m = block/
+ malloc = []
+@@ -1907,6 +1909,9 @@ if have_tools
+ qemu_nbd = executable('qemu-nbd', files('qemu-nbd.c'),
+ dependencies: [blockdev, qemuutil], install: true)
++ vma = executable('vma', files('vma.c', 'vma-reader.c'),
++ dependencies: [authz, block, crypto, io, qom], install: true)
++
+ subdir('storage-daemon')
+ subdir('contrib/rdmacm-mux')
+ subdir('contrib/elf2dmp')
diff --git a/vma-reader.c b/vma-reader.c
new file mode 100644
index 0000000000..2b1d1cdab3
@@ -914,7 +917,7 @@ index 0000000000..2b1d1cdab3
+
diff --git a/vma-writer.c b/vma-writer.c
new file mode 100644
-index 0000000000..f5d2c5d23c
+index 0000000000..11d8321ffd
--- /dev/null
+++ b/vma-writer.c
@@ -0,0 +1,790 @@
@@ -1213,20 +1216,20 @@ index 0000000000..f5d2c5d23c
+
+ if ((stat(filename, &st) == 0) && S_ISFIFO(st.st_mode)) {
+ oflags = O_NONBLOCK|O_WRONLY;
-+ vmaw->fd = qemu_open(filename, oflags, 0644);
++ vmaw->fd = qemu_open(filename, oflags, errp);
+ } else if (strstart(filename, "/dev/fdset/", &tmp_id_str)) {
+ oflags = O_NONBLOCK|O_WRONLY;
-+ vmaw->fd = qemu_open(filename, oflags, 0644);
++ vmaw->fd = qemu_open(filename, oflags, errp);
+ } else if (strstart(filename, "/dev/fdname/", &tmp_id_str)) {
-+ vmaw->fd = monitor_get_fd(cur_mon, tmp_id_str, errp);
++ vmaw->fd = monitor_get_fd(monitor_cur(), tmp_id_str, errp);
+ if (vmaw->fd < 0) {
+ goto err;
+ }
+ /* try to use O_NONBLOCK */
+ fcntl(vmaw->fd, F_SETFL, fcntl(vmaw->fd, F_GETFL)|O_NONBLOCK);
+ } else {
-+ oflags = O_NONBLOCK|O_DIRECT|O_WRONLY|O_CREAT|O_EXCL;
-+ vmaw->fd = qemu_open(filename, oflags, 0644);
++ oflags = O_NONBLOCK|O_DIRECT|O_WRONLY|O_EXCL;
++ vmaw->fd = qemu_create(filename, oflags, 0644, errp);
+ }
+
+ if (vmaw->fd < 0) {
diff --git a/debian/patches/pve/0027-PVE-Backup-add-backup-dump-block-driver.patch b/debian/patches/pve/0027-PVE-Backup-add-backup-dump-block-driver.patch
index cfd297b..8692f8e 100644
--- a/debian/patches/pve/0027-PVE-Backup-add-backup-dump-block-driver.patch
+++ b/debian/patches/pve/0027-PVE-Backup-add-backup-dump-block-driver.patch
@@ -8,26 +8,14 @@ Subject: [PATCH] PVE-Backup: add backup-dump block driver
- block/backup.c - backup-job-create: also consider source cluster size
- job.c: make job_should_pause non-static
---
- block/Makefile.objs | 1 +
block/backup-dump.c | 168 ++++++++++++++++++++++++++++++++++++++
block/backup.c | 23 ++----
+ block/meson.build | 1 +
include/block/block_int.h | 30 +++++++
job.c | 3 +-
5 files changed, 206 insertions(+), 19 deletions(-)
create mode 100644 block/backup-dump.c
-diff --git a/block/Makefile.objs b/block/Makefile.objs
-index d1a9227b8f..9ea0477d0b 100644
---- a/block/Makefile.objs
-+++ b/block/Makefile.objs
-@@ -33,6 +33,7 @@ block-obj-$(CONFIG_CURL) += curl.o
- block-obj-$(CONFIG_RBD) += rbd.o
- block-obj-$(CONFIG_GLUSTERFS) += gluster.o
- block-obj-$(CONFIG_LIBSSH) += ssh.o
-+block-obj-y += backup-dump.o
- block-obj-y += accounting.o dirty-bitmap.o
- block-obj-y += write-threshold.o
- block-obj-y += backup.o
diff --git a/block/backup-dump.c b/block/backup-dump.c
new file mode 100644
index 0000000000..93d7f46950
@@ -203,7 +191,7 @@ index 0000000000..93d7f46950
+ return bs;
+}
diff --git a/block/backup.c b/block/backup.c
-index 4f13bb20a5..cd42236b79 100644
+index 9afa0bf3b4..3df3d532d5 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -32,24 +32,6 @@
@@ -231,7 +219,7 @@ index 4f13bb20a5..cd42236b79 100644
static const BlockJobDriver backup_job_driver;
static void backup_progress_bytes_callback(int64_t bytes, void *opaque)
-@@ -422,6 +404,11 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
+@@ -423,6 +405,11 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
goto error;
}
@@ -243,11 +231,23 @@ index 4f13bb20a5..cd42236b79 100644
/*
* If source is in backing chain of target assume that target is going to be
* used for "image fleecing", i.e. it should represent a kind of snapshot of
+diff --git a/block/meson.build b/block/meson.build
+index feffbc8623..2507af1168 100644
+--- a/block/meson.build
++++ b/block/meson.build
+@@ -4,6 +4,7 @@ block_ss.add(files(
+ 'aio_task.c',
+ 'amend.c',
+ 'backup.c',
++ 'backup-dump.c',
+ 'backup-top.c',
+ 'blkdebug.c',
+ 'blklogwrites.c',
diff --git a/include/block/block_int.h b/include/block/block_int.h
-index 38dec0275b..1efb1f527c 100644
+index 95d9333be1..2645e53282 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
-@@ -62,6 +62,36 @@
+@@ -63,6 +63,36 @@
#define BLOCK_PROBE_BUF_SIZE 512
@@ -285,7 +285,7 @@ index 38dec0275b..1efb1f527c 100644
BDRV_TRACKED_READ,
BDRV_TRACKED_WRITE,
diff --git a/job.c b/job.c
-index 53be57a3a0..b8139c80a4 100644
+index 8fecf38960..f9884e7d9d 100644
--- a/job.c
+++ b/job.c
@@ -269,7 +269,8 @@ static bool job_started(Job *job)
diff --git a/debian/patches/pve/0028-PVE-Backup-proxmox-backup-patches-for-qemu.patch b/debian/patches/pve/0028-PVE-Backup-proxmox-backup-patches-for-qemu.patch
index 7896692..cb8334e 100644
--- a/debian/patches/pve/0028-PVE-Backup-proxmox-backup-patches-for-qemu.patch
+++ b/debian/patches/pve/0028-PVE-Backup-proxmox-backup-patches-for-qemu.patch
@@ -4,77 +4,47 @@ Date: Mon, 6 Apr 2020 12:16:59 +0200
Subject: [PATCH] PVE-Backup: proxmox backup patches for qemu
---
- Makefile | 1 +
- Makefile.objs | 2 +
- Makefile.target | 2 +-
+ block/meson.build | 5 +
block/monitor/block-hmp-cmds.c | 33 ++
blockdev.c | 1 +
hmp-commands-info.hx | 13 +
hmp-commands.hx | 29 +
include/block/block_int.h | 2 +-
include/monitor/hmp.h | 3 +
+ meson.build | 1 +
monitor/hmp-cmds.c | 44 ++
proxmox-backup-client.c | 176 ++++++
proxmox-backup-client.h | 59 ++
pve-backup.c | 955 +++++++++++++++++++++++++++++++++
qapi/block-core.json | 109 ++++
qapi/common.json | 13 +
- qapi/misc.json | 13 -
- 16 files changed, 1440 insertions(+), 15 deletions(-)
+ qapi/machine.json | 15 +-
+ 15 files changed, 1444 insertions(+), 14 deletions(-)
create mode 100644 proxmox-backup-client.c
create mode 100644 proxmox-backup-client.h
create mode 100644 pve-backup.c
-diff --git a/Makefile b/Makefile
-index 7b8c17ce2d..aec216968d 100644
---- a/Makefile
-+++ b/Makefile
-@@ -602,6 +602,7 @@ qemu-img$(EXESUF): qemu-img.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io
- qemu-nbd$(EXESUF): qemu-nbd.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
- qemu-io$(EXESUF): qemu-io.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
- qemu-storage-daemon$(EXESUF): qemu-storage-daemon.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(chardev-obj-y) $(io-obj-y) $(qom-obj-y) $(storage-daemon-obj-y) $(COMMON_LDADDS)
-+qemu-storage-daemon$(EXESUF): LIBS += -lproxmox_backup_qemu
- vma$(EXESUF): vma.o vma-reader.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
+diff --git a/block/meson.build b/block/meson.build
+index 2507af1168..dfae565db3 100644
+--- a/block/meson.build
++++ b/block/meson.build
+@@ -44,6 +44,11 @@ block_ss.add(files(
+ ), zstd, zlib)
- qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o $(COMMON_LDADDS)
-diff --git a/Makefile.objs b/Makefile.objs
-index ade7b17a69..240eb503f2 100644
---- a/Makefile.objs
-+++ b/Makefile.objs
-@@ -33,6 +33,7 @@ endif # CONFIG_SOFTMMU or CONFIG_TOOLS
+ block_ss.add(files('../vma-writer.c'), libuuid)
++block_ss.add(files(
++ '../proxmox-backup-client.c',
++ '../pve-backup.c',
++), libproxmox_backup_qemu)
++
- storage-daemon-obj-y = block/ monitor/ qapi/ qom/ storage-daemon/
- storage-daemon-obj-y += blockdev.o blockdev-nbd.o iothread.o job-qmp.o
-+storage-daemon-obj-y += proxmox-backup-client.o pve-backup.o
- storage-daemon-obj-$(CONFIG_WIN32) += os-win32.o
- storage-daemon-obj-$(CONFIG_POSIX) += os-posix.o
+ softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
-@@ -44,6 +45,7 @@ storage-daemon-obj-$(CONFIG_POSIX) += os-posix.o
- ifeq ($(CONFIG_SOFTMMU),y)
- common-obj-y = blockdev.o blockdev-nbd.o block/
- common-obj-y += bootdevice.o iothread.o
-+common-obj-y += proxmox-backup-client.o pve-backup.o
- common-obj-y += dump/
- common-obj-y += job-qmp.o
- common-obj-y += monitor/
-diff --git a/Makefile.target b/Makefile.target
-index ffa2657269..fd3fd6d2a7 100644
---- a/Makefile.target
-+++ b/Makefile.target
-@@ -159,7 +159,7 @@ obj-y += hw/
- obj-y += monitor/
- obj-y += qapi/
- obj-y += migration/ram.o
--LIBS := $(libs_softmmu) $(LIBS)
-+LIBS := $(libs_softmmu) $(LIBS) -lproxmox_backup_qemu
-
- # Hardware support
- ifeq ($(TARGET_NAME), sparc64)
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
-index 4c8c375172..d485c3ac79 100644
+index d15a2be827..9ba7c774a2 100644
--- a/block/monitor/block-hmp-cmds.c
+++ b/block/monitor/block-hmp-cmds.c
-@@ -1011,3 +1011,36 @@ void hmp_info_snapshots(Monitor *mon, const QDict *qdict)
+@@ -1012,3 +1012,36 @@ void hmp_info_snapshots(Monitor *mon, const QDict *qdict)
g_free(sn_tab);
g_free(global_snapshots);
}
@@ -112,7 +82,7 @@ index 4c8c375172..d485c3ac79 100644
+ hmp_handle_error(mon, error);
+}
diff --git a/blockdev.c b/blockdev.c
-index 3848a9c8ab..681da7c8b6 100644
+index fe6fb5dc1d..bae80b9177 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -36,6 +36,7 @@
@@ -124,7 +94,7 @@ index 3848a9c8ab..681da7c8b6 100644
#include "monitor/monitor.h"
#include "qemu/error-report.h"
diff --git a/hmp-commands-info.hx b/hmp-commands-info.hx
-index ae8ff21789..da16499f8d 100644
+index b3b797ca28..295e14e64f 100644
--- a/hmp-commands-info.hx
+++ b/hmp-commands-info.hx
@@ -513,6 +513,19 @@ SRST
@@ -148,10 +118,10 @@ index ae8ff21789..da16499f8d 100644
{
.name = "usernet",
diff --git a/hmp-commands.hx b/hmp-commands.hx
-index 2b58ac4a1c..9e58b6a5fc 100644
+index d294c234a5..0c6b944850 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
-@@ -97,6 +97,35 @@ ERST
+@@ -98,6 +98,35 @@ ERST
SRST
``block_stream``
Copy data from a backing file into a block device.
@@ -188,10 +158,10 @@ index 2b58ac4a1c..9e58b6a5fc 100644
{
diff --git a/include/block/block_int.h b/include/block/block_int.h
-index 1efb1f527c..8dda6f769d 100644
+index 2645e53282..9fa282ff54 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
-@@ -64,7 +64,7 @@
+@@ -65,7 +65,7 @@
typedef int BackupDumpFunc(void *opaque, uint64_t offset, uint64_t bytes, const void *buf);
@@ -201,7 +171,7 @@ index 1efb1f527c..8dda6f769d 100644
uint64_t byte_size,
BackupDumpFunc *dump_cb,
diff --git a/include/monitor/hmp.h b/include/monitor/hmp.h
-index 243952d32f..892a6064be 100644
+index 4e06f89e8e..10f52bd92a 100644
--- a/include/monitor/hmp.h
+++ b/include/monitor/hmp.h
@@ -30,6 +30,7 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict);
@@ -221,11 +191,23 @@ index 243952d32f..892a6064be 100644
void hmp_migrate(Monitor *mon, const QDict *qdict);
void hmp_device_add(Monitor *mon, const QDict *qdict);
void hmp_device_del(Monitor *mon, const QDict *qdict);
+diff --git a/meson.build b/meson.build
+index d5b660516b..3094f98c47 100644
+--- a/meson.build
++++ b/meson.build
+@@ -726,6 +726,7 @@ keyutils = dependency('libkeyutils', required: false,
+ has_gettid = cc.has_function('gettid')
+
+ libuuid = cc.find_library('uuid', required: true)
++libproxmox_backup_qemu = cc.find_library('proxmox_backup_qemu', required: true)
+
+ # Malloc tests
+
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
-index 280bb447a6..0e2d166552 100644
+index 77ab152aab..182e79c943 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
-@@ -192,6 +192,50 @@ void hmp_info_mice(Monitor *mon, const QDict *qdict)
+@@ -195,6 +195,50 @@ void hmp_info_mice(Monitor *mon, const QDict *qdict)
qapi_free_MouseInfoList(mice_list);
}
@@ -1485,10 +1467,10 @@ index 0000000000..55441eb9d1
+ return task.result;
+}
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index ea5fae22ae..69db270b1a 100644
+index 7957b9867d..be67dc3376 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -754,6 +754,115 @@
+@@ -745,6 +745,115 @@
{ 'command': 'query-block', 'returns': ['BlockInfo'] }
@@ -1544,12 +1526,12 @@ index ea5fae22ae..69db270b1a 100644
+# @format: format of the backup file
+#
+# @config-file: a configuration file to include into
-+# the backup archive.
++# the backup archive.
+#
+# @speed: the maximum speed, in bytes per second
+#
+# @devlist: list of block device names (separated by ',', ';'
-+# or ':'). By default the backup includes all writable block devices.
++# or ':'). By default the backup includes all writable block devices.
+#
+# @password: backup server passsword (required for format 'pbs')
+#
@@ -1625,13 +1607,22 @@ index 716712d4b3..556dab79e1 100644
+# Notes: If no UUID was specified for the guest, a null UUID is returned.
+##
+{ 'struct': 'UuidInfo', 'data': {'UUID': 'str'} }
-diff --git a/qapi/misc.json b/qapi/misc.json
-index 9895899f8b..75dff1b306 100644
---- a/qapi/misc.json
-+++ b/qapi/misc.json
-@@ -130,19 +130,6 @@
+diff --git a/qapi/machine.json b/qapi/machine.json
+index 32fc674042..145f1a4fa2 100644
+--- a/qapi/machine.json
++++ b/qapi/machine.json
+@@ -4,6 +4,8 @@
+ # This work is licensed under the terms of the GNU GPL, version 2 or later.
+ # See the COPYING file in the top-level directory.
+
++{ 'include': 'common.json' }
++
##
- { 'command': 'query-kvm', 'returns': 'KvmInfo' }
+ # = Machines
+ ##
+@@ -406,19 +408,6 @@
+ ##
+ { 'command': 'query-target', 'returns': 'TargetInfo' }
-##
-# @UuidInfo:
diff --git a/debian/patches/pve/0029-PVE-Backup-pbs-restore-new-command-to-restore-from-p.patch b/debian/patches/pve/0029-PVE-Backup-pbs-restore-new-command-to-restore-from-p.patch
index 02e78c5..d7b0f39 100644
--- a/debian/patches/pve/0029-PVE-Backup-pbs-restore-new-command-to-restore-from-p.patch
+++ b/debian/patches/pve/0029-PVE-Backup-pbs-restore-new-command-to-restore-from-p.patch
@@ -6,33 +6,26 @@ Subject: [PATCH] PVE-Backup: pbs-restore - new command to restore from proxmox
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
- Makefile | 4 +-
+ meson.build | 4 +
pbs-restore.c | 217 ++++++++++++++++++++++++++++++++++++++++++++++++++
- 2 files changed, 220 insertions(+), 1 deletion(-)
+ 2 files changed, 221 insertions(+)
create mode 100644 pbs-restore.c
-diff --git a/Makefile b/Makefile
-index aec216968d..b73da29f24 100644
---- a/Makefile
-+++ b/Makefile
-@@ -479,7 +479,7 @@ dummy := $(call unnest-vars,, \
-
- include $(SRC_PATH)/tests/Makefile.include
-
--all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) vma$(EXESUF) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
-+all: $(DOCS) $(if $(BUILD_DOCS),sphinxdocs) $(TOOLS) vma$(EXESUF) pbs-restore$(EXESUF) $(HELPERS-y) recurse-all modules $(vhost-user-json-y)
-
- qemu-version.h: FORCE
- $(call quiet-command, \
-@@ -604,6 +604,8 @@ qemu-io$(EXESUF): qemu-io.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-o
- qemu-storage-daemon$(EXESUF): qemu-storage-daemon.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(chardev-obj-y) $(io-obj-y) $(qom-obj-y) $(storage-daemon-obj-y) $(COMMON_LDADDS)
- qemu-storage-daemon$(EXESUF): LIBS += -lproxmox_backup_qemu
- vma$(EXESUF): vma.o vma-reader.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
-+pbs-restore$(EXESUF): pbs-restore.o $(authz-obj-y) $(block-obj-y) $(crypto-obj-y) $(io-obj-y) $(qom-obj-y) $(COMMON_LDADDS)
-+pbs-restore$(EXESUF): LIBS += -lproxmox_backup_qemu
-
- qemu-bridge-helper$(EXESUF): qemu-bridge-helper.o $(COMMON_LDADDS)
+diff --git a/meson.build b/meson.build
+index 3094f98c47..6f1fafee14 100644
+--- a/meson.build
++++ b/meson.build
+@@ -1913,6 +1913,10 @@ if have_tools
+ vma = executable('vma', files('vma.c', 'vma-reader.c'),
+ dependencies: [authz, block, crypto, io, qom], install: true)
++ pbs_restore = executable('pbs-restore', files('pbs-restore.c'),
++ dependencies: [authz, block, crypto, io, qom,
++ libproxmox_backup_qemu], install: true)
++
+ subdir('storage-daemon')
+ subdir('contrib/rdmacm-mux')
+ subdir('contrib/elf2dmp')
diff --git a/pbs-restore.c b/pbs-restore.c
new file mode 100644
index 0000000000..4bf37ef1fa
diff --git a/debian/patches/pve/0031-drive-mirror-add-support-for-sync-bitmap-mode-never.patch b/debian/patches/pve/0031-drive-mirror-add-support-for-sync-bitmap-mode-never.patch
index f09b953..3e324a3 100644
--- a/debian/patches/pve/0031-drive-mirror-add-support-for-sync-bitmap-mode-never.patch
+++ b/debian/patches/pve/0031-drive-mirror-add-support-for-sync-bitmap-mode-never.patch
@@ -35,10 +35,10 @@ Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
5 files changed, 145 insertions(+), 29 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
-index e8e8844afc..100e828639 100644
+index 8e1ad6eceb..97843992c2 100644
--- a/block/mirror.c
+++ b/block/mirror.c
-@@ -49,7 +49,7 @@ typedef struct MirrorBlockJob {
+@@ -50,7 +50,7 @@ typedef struct MirrorBlockJob {
BlockDriverState *to_replace;
/* Used to block operations on the drive-mirror-replace target */
Error *replace_blocker;
@@ -47,7 +47,7 @@ index e8e8844afc..100e828639 100644
BlockMirrorBackingMode backing_mode;
/* Whether the target image requires explicit zero-initialization */
bool zero_target;
-@@ -64,6 +64,8 @@ typedef struct MirrorBlockJob {
+@@ -65,6 +65,8 @@ typedef struct MirrorBlockJob {
size_t buf_size;
int64_t bdev_length;
unsigned long *cow_bitmap;
@@ -56,17 +56,17 @@ index e8e8844afc..100e828639 100644
BdrvDirtyBitmap *dirty_bitmap;
BdrvDirtyBitmapIter *dbi;
uint8_t *buf;
-@@ -676,7 +678,8 @@ static int mirror_exit_common(Job *job)
+@@ -677,7 +679,8 @@ static int mirror_exit_common(Job *job)
bdrv_child_refresh_perms(mirror_top_bs, mirror_top_bs->backing,
&error_abort);
if (!abort && s->backing_mode == MIRROR_SOURCE_BACKING_CHAIN) {
- BlockDriverState *backing = s->is_none_mode ? src : s->base;
+ BlockDriverState *backing;
+ backing = s->sync_mode == MIRROR_SYNC_MODE_NONE ? src : s->base;
- if (backing_bs(target_bs) != backing) {
- bdrv_set_backing_hd(target_bs, backing, &local_err);
- if (local_err) {
-@@ -771,6 +774,16 @@ static void mirror_abort(Job *job)
+ BlockDriverState *unfiltered_target = bdrv_skip_filters(target_bs);
+
+ if (bdrv_cow_bs(unfiltered_target) != backing) {
+@@ -774,6 +777,16 @@ static void mirror_abort(Job *job)
assert(ret == 0);
}
@@ -83,7 +83,7 @@ index e8e8844afc..100e828639 100644
static void coroutine_fn mirror_throttle(MirrorBlockJob *s)
{
int64_t now = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
-@@ -952,7 +965,8 @@ static int coroutine_fn mirror_run(Job *job, Error **errp)
+@@ -955,7 +968,8 @@ static int coroutine_fn mirror_run(Job *job, Error **errp)
mirror_free_init(s);
s->last_pause_ns = qemu_clock_get_ns(QEMU_CLOCK_REALTIME);
@@ -93,7 +93,7 @@ index e8e8844afc..100e828639 100644
ret = mirror_dirty_init(s);
if (ret < 0 || job_is_cancelled(&s->common.job)) {
goto immediate_exit;
-@@ -1184,6 +1198,7 @@ static const BlockJobDriver mirror_job_driver = {
+@@ -1188,6 +1202,7 @@ static const BlockJobDriver mirror_job_driver = {
.run = mirror_run,
.prepare = mirror_prepare,
.abort = mirror_abort,
@@ -101,7 +101,7 @@ index e8e8844afc..100e828639 100644
.pause = mirror_pause,
.complete = mirror_complete,
},
-@@ -1199,6 +1214,7 @@ static const BlockJobDriver commit_active_job_driver = {
+@@ -1203,6 +1218,7 @@ static const BlockJobDriver commit_active_job_driver = {
.run = mirror_run,
.prepare = mirror_prepare,
.abort = mirror_abort,
@@ -109,7 +109,7 @@ index e8e8844afc..100e828639 100644
.pause = mirror_pause,
.complete = mirror_complete,
},
-@@ -1547,7 +1563,10 @@ static BlockJob *mirror_start_job(
+@@ -1550,7 +1566,10 @@ static BlockJob *mirror_start_job(
BlockCompletionFunc *cb,
void *opaque,
const BlockJobDriver *driver,
@@ -121,7 +121,7 @@ index e8e8844afc..100e828639 100644
bool auto_complete, const char *filter_node_name,
bool is_mirror, MirrorCopyMode copy_mode,
Error **errp)
-@@ -1560,10 +1579,39 @@ static BlockJob *mirror_start_job(
+@@ -1563,10 +1582,39 @@ static BlockJob *mirror_start_job(
Error *local_err = NULL;
int ret;
@@ -163,7 +163,7 @@ index e8e8844afc..100e828639 100644
assert(is_power_of_2(granularity));
if (buf_size < 0) {
-@@ -1667,7 +1715,9 @@ static BlockJob *mirror_start_job(
+@@ -1705,7 +1753,9 @@ static BlockJob *mirror_start_job(
s->replaces = g_strdup(replaces);
s->on_source_error = on_source_error;
s->on_target_error = on_target_error;
@@ -174,7 +174,7 @@ index e8e8844afc..100e828639 100644
s->backing_mode = backing_mode;
s->zero_target = zero_target;
s->copy_mode = copy_mode;
-@@ -1687,6 +1737,18 @@ static BlockJob *mirror_start_job(
+@@ -1726,6 +1776,18 @@ static BlockJob *mirror_start_job(
bdrv_disable_dirty_bitmap(s->dirty_bitmap);
}
@@ -193,7 +193,7 @@ index e8e8844afc..100e828639 100644
ret = block_job_add_bdrv(&s->common, "source", bs, 0,
BLK_PERM_WRITE_UNCHANGED | BLK_PERM_WRITE |
BLK_PERM_CONSISTENT_READ,
-@@ -1740,6 +1802,9 @@ fail:
+@@ -1803,6 +1865,9 @@ fail:
if (s->dirty_bitmap) {
bdrv_release_dirty_bitmap(s->dirty_bitmap);
}
@@ -203,7 +203,7 @@ index e8e8844afc..100e828639 100644
job_early_fail(&s->common.job);
}
-@@ -1757,29 +1822,23 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
+@@ -1820,29 +1885,23 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, const char *replaces,
int creation_flags, int64_t speed,
uint32_t granularity, int64_t buf_size,
@@ -227,7 +227,7 @@ index e8e8844afc..100e828639 100644
- return;
- }
- is_none_mode = mode == MIRROR_SYNC_MODE_NONE;
- base = mode == MIRROR_SYNC_MODE_TOP ? backing_bs(bs) : NULL;
+ base = mode == MIRROR_SYNC_MODE_TOP ? bdrv_backing_chain_next(bs) : NULL;
mirror_start_job(job_id, bs, creation_flags, target, replaces,
speed, granularity, buf_size, backing_mode, zero_target,
on_source_error, on_target_error, unmap, NULL, NULL,
@@ -238,7 +238,7 @@ index e8e8844afc..100e828639 100644
}
BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
-@@ -1805,7 +1864,8 @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
+@@ -1868,7 +1927,8 @@ BlockJob *commit_active_start(const char *job_id, BlockDriverState *bs,
job_id, bs, creation_flags, base, NULL, speed, 0, 0,
MIRROR_LEAVE_BACKING_CHAIN, false,
on_error, on_error, true, cb, opaque,
@@ -249,10 +249,10 @@ index e8e8844afc..100e828639 100644
&local_err);
if (local_err) {
diff --git a/blockdev.c b/blockdev.c
-index 681da7c8b6..02d58e7645 100644
+index bae80b9177..c79e081f57 100644
--- a/blockdev.c
+++ b/blockdev.c
-@@ -2876,6 +2876,10 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
+@@ -2931,6 +2931,10 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
BlockDriverState *target,
bool has_replaces, const char *replaces,
enum MirrorSyncMode sync,
@@ -263,15 +263,15 @@ index 681da7c8b6..02d58e7645 100644
BlockMirrorBackingMode backing_mode,
bool zero_target,
bool has_speed, int64_t speed,
-@@ -2894,6 +2898,7 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
- Error **errp)
+@@ -2950,6 +2954,7 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
{
+ BlockDriverState *unfiltered_bs;
int job_flags = JOB_DEFAULT;
+ BdrvDirtyBitmap *bitmap = NULL;
if (!has_speed) {
speed = 0;
-@@ -2948,6 +2953,29 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
+@@ -3004,6 +3009,29 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
sync = MIRROR_SYNC_MODE_FULL;
}
@@ -298,10 +298,10 @@ index 681da7c8b6..02d58e7645 100644
+ }
+ }
+
- if (has_replaces) {
- BlockDriverState *to_replace_bs;
- AioContext *replace_aio_context;
-@@ -2985,8 +3013,8 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
+ if (!has_replaces) {
+ /* We want to mirror from @bs, but keep implicit filters on top */
+ unfiltered_bs = bdrv_skip_implicit_filters(bs);
+@@ -3050,8 +3078,8 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
* and will allow to check whether the node still exist at mirror completion
*/
mirror_start(job_id, bs, target,
@@ -312,7 +312,7 @@ index 681da7c8b6..02d58e7645 100644
on_source_error, on_target_error, unmap, filter_node_name,
copy_mode, errp);
}
-@@ -3127,6 +3155,8 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
+@@ -3196,6 +3224,8 @@ void qmp_drive_mirror(DriveMirror *arg, Error **errp)
blockdev_mirror_common(arg->has_job_id ? arg->job_id : NULL, bs, target_bs,
arg->has_replaces, arg->replaces, arg->sync,
@@ -321,7 +321,7 @@ index 681da7c8b6..02d58e7645 100644
backing_mode, zero_target,
arg->has_speed, arg->speed,
arg->has_granularity, arg->granularity,
-@@ -3148,6 +3178,8 @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
+@@ -3217,6 +3247,8 @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
const char *device, const char *target,
bool has_replaces, const char *replaces,
MirrorSyncMode sync,
@@ -330,7 +330,7 @@ index 681da7c8b6..02d58e7645 100644
bool has_speed, int64_t speed,
bool has_granularity, uint32_t granularity,
bool has_buf_size, int64_t buf_size,
-@@ -3197,7 +3229,8 @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
+@@ -3266,7 +3298,8 @@ void qmp_blockdev_mirror(bool has_job_id, const char *job_id,
}
blockdev_mirror_common(has_job_id ? job_id : NULL, bs, target_bs,
@@ -341,10 +341,10 @@ index 681da7c8b6..02d58e7645 100644
has_granularity, granularity,
has_buf_size, buf_size,
diff --git a/include/block/block_int.h b/include/block/block_int.h
-index 8dda6f769d..279bd4ab61 100644
+index 9fa282ff54..1bd4b64522 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
-@@ -1245,7 +1245,9 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
+@@ -1260,7 +1260,9 @@ void mirror_start(const char *job_id, BlockDriverState *bs,
BlockDriverState *target, const char *replaces,
int creation_flags, int64_t speed,
uint32_t granularity, int64_t buf_size,
@@ -356,10 +356,10 @@ index 8dda6f769d..279bd4ab61 100644
BlockdevOnError on_source_error,
BlockdevOnError on_target_error,
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index 69db270b1a..9db8e26517 100644
+index be67dc3376..9054db608c 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -2064,10 +2064,19 @@
+@@ -2080,10 +2080,19 @@
# (all the disk, only the sectors allocated in the topmost image, or
# only new I/O).
#
@@ -380,7 +380,7 @@ index 69db270b1a..9db8e26517 100644
#
# @buf-size: maximum amount of data in flight from source to
# target (since 1.4).
-@@ -2105,7 +2114,9 @@
+@@ -2121,7 +2130,9 @@
{ 'struct': 'DriveMirror',
'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
'*format': 'str', '*node-name': 'str', '*replaces': 'str',
@@ -391,7 +391,7 @@ index 69db270b1a..9db8e26517 100644
'*speed': 'int', '*granularity': 'uint32',
'*buf-size': 'int', '*on-source-error': 'BlockdevOnError',
'*on-target-error': 'BlockdevOnError',
-@@ -2372,10 +2383,19 @@
+@@ -2389,10 +2400,19 @@
# (all the disk, only the sectors allocated in the topmost image, or
# only new I/O).
#
@@ -412,7 +412,7 @@ index 69db270b1a..9db8e26517 100644
#
# @buf-size: maximum amount of data in flight from source to
# target
-@@ -2424,7 +2444,8 @@
+@@ -2441,7 +2461,8 @@
{ 'command': 'blockdev-mirror',
'data': { '*job-id': 'str', 'device': 'str', 'target': 'str',
'*replaces': 'str',
diff --git a/debian/patches/pve/0032-drive-mirror-add-support-for-conditional-and-always-.patch b/debian/patches/pve/0032-drive-mirror-add-support-for-conditional-and-always-.patch
index 63753d5..5a87360 100644
--- a/debian/patches/pve/0032-drive-mirror-add-support-for-conditional-and-always-.patch
+++ b/debian/patches/pve/0032-drive-mirror-add-support-for-conditional-and-always-.patch
@@ -23,10 +23,10 @@ Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
1 file changed, 18 insertions(+), 6 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
-index 100e828639..7d3c3252f3 100644
+index 97843992c2..d1cce079da 100644
--- a/block/mirror.c
+++ b/block/mirror.c
-@@ -653,8 +653,6 @@ static int mirror_exit_common(Job *job)
+@@ -654,8 +654,6 @@ static int mirror_exit_common(Job *job)
bdrv_unfreeze_backing_chain(mirror_top_bs, target_bs);
}
@@ -35,7 +35,7 @@ index 100e828639..7d3c3252f3 100644
/* Make sure that the source BDS doesn't go away during bdrv_replace_node,
* before we can call bdrv_drained_end */
bdrv_ref(src);
-@@ -752,6 +750,18 @@ static int mirror_exit_common(Job *job)
+@@ -755,6 +753,18 @@ static int mirror_exit_common(Job *job)
blk_set_perm(bjob->blk, 0, BLK_PERM_ALL, &error_abort);
blk_insert_bs(bjob->blk, mirror_top_bs, &error_abort);
@@ -54,7 +54,7 @@ index 100e828639..7d3c3252f3 100644
bs_opaque->job = NULL;
bdrv_drained_end(src);
-@@ -1589,10 +1599,6 @@ static BlockJob *mirror_start_job(
+@@ -1592,10 +1602,6 @@ static BlockJob *mirror_start_job(
" sync mode",
MirrorSyncMode_str(sync_mode));
return NULL;
@@ -65,7 +65,7 @@ index 100e828639..7d3c3252f3 100644
}
} else if (bitmap) {
error_setg(errp,
-@@ -1609,6 +1615,12 @@ static BlockJob *mirror_start_job(
+@@ -1612,6 +1618,12 @@ static BlockJob *mirror_start_job(
return NULL;
}
granularity = bdrv_dirty_bitmap_granularity(bitmap);
diff --git a/debian/patches/pve/0033-mirror-add-check-for-bitmap-mode-without-bitmap.patch b/debian/patches/pve/0033-mirror-add-check-for-bitmap-mode-without-bitmap.patch
index 607c90b..97077f3 100644
--- a/debian/patches/pve/0033-mirror-add-check-for-bitmap-mode-without-bitmap.patch
+++ b/debian/patches/pve/0033-mirror-add-check-for-bitmap-mode-without-bitmap.patch
@@ -15,10 +15,10 @@ Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
1 file changed, 3 insertions(+)
diff --git a/blockdev.c b/blockdev.c
-index 02d58e7645..0d480f02c7 100644
+index c79e081f57..827f004069 100644
--- a/blockdev.c
+++ b/blockdev.c
-@@ -2974,6 +2974,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
+@@ -3030,6 +3030,9 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_ALLOW_RO, errp)) {
return;
}
@@ -27,4 +27,4 @@ index 02d58e7645..0d480f02c7 100644
+ return;
}
- if (has_replaces) {
+ if (!has_replaces) {
diff --git a/debian/patches/pve/0034-mirror-switch-to-bdrv_dirty_bitmap_merge_internal.patch b/debian/patches/pve/0034-mirror-switch-to-bdrv_dirty_bitmap_merge_internal.patch
index 1ea8b1e..8b3c1e2 100644
--- a/debian/patches/pve/0034-mirror-switch-to-bdrv_dirty_bitmap_merge_internal.patch
+++ b/debian/patches/pve/0034-mirror-switch-to-bdrv_dirty_bitmap_merge_internal.patch
@@ -15,10 +15,10 @@ Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
-index 7d3c3252f3..fb12ccb932 100644
+index d1cce079da..e6140cf018 100644
--- a/block/mirror.c
+++ b/block/mirror.c
-@@ -756,8 +756,8 @@ static int mirror_exit_common(Job *job)
+@@ -759,8 +759,8 @@ static int mirror_exit_common(Job *job)
job->ret == 0 && ret == 0)) {
/* Success; synchronize copy back to sync. */
bdrv_clear_dirty_bitmap(s->sync_bitmap, NULL);
@@ -29,7 +29,7 @@ index 7d3c3252f3..fb12ccb932 100644
}
}
bdrv_release_dirty_bitmap(s->dirty_bitmap);
-@@ -1754,8 +1754,8 @@ static BlockJob *mirror_start_job(
+@@ -1793,8 +1793,8 @@ static BlockJob *mirror_start_job(
}
if (s->sync_mode == MIRROR_SYNC_MODE_BITMAP) {
diff --git a/debian/patches/pve/0035-iotests-add-test-for-bitmap-mirror.patch b/debian/patches/pve/0035-iotests-add-test-for-bitmap-mirror.patch
index f7ac056..07a66d6 100644
--- a/debian/patches/pve/0035-iotests-add-test-for-bitmap-mirror.patch
+++ b/debian/patches/pve/0035-iotests-add-test-for-bitmap-mirror.patch
@@ -3434,7 +3434,7 @@ index 0000000000..9b7408b6d6
+{"error": {"class": "GenericError", "desc": "bitmap-mode must be specified if a bitmap is provided"}}
+
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
-index 025ed5238d..bee527c012 100644
+index 2960dff728..952dceba1f 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -270,6 +270,7 @@
diff --git a/debian/patches/pve/0036-mirror-move-some-checks-to-qmp.patch b/debian/patches/pve/0036-mirror-move-some-checks-to-qmp.patch
index cd96a02..bfbb49f 100644
--- a/debian/patches/pve/0036-mirror-move-some-checks-to-qmp.patch
+++ b/debian/patches/pve/0036-mirror-move-some-checks-to-qmp.patch
@@ -18,10 +18,10 @@ Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
3 files changed, 70 insertions(+), 59 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
-index fb12ccb932..dfce442e97 100644
+index e6140cf018..3a08239a78 100644
--- a/block/mirror.c
+++ b/block/mirror.c
-@@ -1589,31 +1589,13 @@ static BlockJob *mirror_start_job(
+@@ -1592,31 +1592,13 @@ static BlockJob *mirror_start_job(
Error *local_err = NULL;
int ret;
@@ -59,10 +59,10 @@ index fb12ccb932..dfce442e97 100644
if (bitmap_mode != BITMAP_SYNC_MODE_NEVER) {
diff --git a/blockdev.c b/blockdev.c
-index 0d480f02c7..be87d65c02 100644
+index 827f004069..e2f826ca62 100644
--- a/blockdev.c
+++ b/blockdev.c
-@@ -2953,7 +2953,36 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
+@@ -3009,7 +3009,36 @@ static void blockdev_mirror_common(const char *job_id, BlockDriverState *bs,
sync = MIRROR_SYNC_MODE_FULL;
}
diff --git a/debian/patches/pve/0037-PVE-Backup-Add-dirty-bitmap-tracking-for-incremental.patch b/debian/patches/pve/0037-PVE-Backup-Add-dirty-bitmap-tracking-for-incremental.patch
index c974060..b5c2dab 100644
--- a/debian/patches/pve/0037-PVE-Backup-Add-dirty-bitmap-tracking-for-incremental.patch
+++ b/debian/patches/pve/0037-PVE-Backup-Add-dirty-bitmap-tracking-for-incremental.patch
@@ -29,10 +29,10 @@ Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
6 files changed, 134 insertions(+), 23 deletions(-)
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
-index d485c3ac79..fdc85a5c0e 100644
+index 9ba7c774a2..056d14deee 100644
--- a/block/monitor/block-hmp-cmds.c
+++ b/block/monitor/block-hmp-cmds.c
-@@ -1038,6 +1038,7 @@ void hmp_backup(Monitor *mon, const QDict *qdict)
+@@ -1039,6 +1039,7 @@ void hmp_backup(Monitor *mon, const QDict *qdict)
false, NULL, // PBS fingerprint
false, NULL, // PBS backup-id
false, 0, // PBS backup-time
@@ -41,10 +41,10 @@ index d485c3ac79..fdc85a5c0e 100644
false, NULL, false, NULL, !!devlist,
devlist, qdict_haskey(qdict, "speed"), speed, &error);
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
-index 0e2d166552..3ff014d32a 100644
+index 182e79c943..604026bb37 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
-@@ -218,19 +218,42 @@ void hmp_info_backup(Monitor *mon, const QDict *qdict)
+@@ -221,19 +221,42 @@ void hmp_info_backup(Monitor *mon, const QDict *qdict)
monitor_printf(mon, "End time: %s", ctime(&info->end_time));
}
@@ -397,10 +397,10 @@ index d40f3f2fd6..d50f03a050 100644
qemu_mutex_unlock(&backup_state.stat.lock);
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index 9db8e26517..6cad1e0e38 100644
+index 9054db608c..aadd5329b3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -767,8 +767,13 @@
+@@ -758,8 +758,13 @@
#
# @total: total amount of bytes involved in the backup process
#
@@ -414,7 +414,7 @@ index 9db8e26517..6cad1e0e38 100644
# @zero-bytes: amount of 'zero' bytes detected.
#
# @start-time: time (epoch) when backup job started.
-@@ -781,8 +786,8 @@
+@@ -772,8 +777,8 @@
#
##
{ 'struct': 'BackupStatus',
@@ -425,7 +425,7 @@ index 9db8e26517..6cad1e0e38 100644
'*start-time': 'int', '*end-time': 'int',
'*backup-file': 'str', '*uuid': 'str' } }
-@@ -825,6 +830,8 @@
+@@ -816,6 +821,8 @@
#
# @backup-time: backup timestamp (Unix epoch, required for format 'pbs')
#
@@ -434,7 +434,7 @@ index 9db8e26517..6cad1e0e38 100644
# Returns: the uuid of the backup job
#
##
-@@ -835,6 +842,7 @@
+@@ -826,6 +833,7 @@
'*fingerprint': 'str',
'*backup-id': 'str',
'*backup-time': 'int',
diff --git a/debian/patches/pve/0038-PVE-backup-rename-incremental-to-use-dirty-bitmap.patch b/debian/patches/pve/0038-PVE-backup-rename-incremental-to-use-dirty-bitmap.patch
index f1e2f55..56b9c32 100644
--- a/debian/patches/pve/0038-PVE-backup-rename-incremental-to-use-dirty-bitmap.patch
+++ b/debian/patches/pve/0038-PVE-backup-rename-incremental-to-use-dirty-bitmap.patch
@@ -94,10 +94,10 @@ index d50f03a050..7bf54b4c5d 100644
.format = format,
.has_config_file = has_config_file,
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index 6cad1e0e38..e00e577c6c 100644
+index aadd5329b3..d4e1c98c50 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -767,7 +767,7 @@
+@@ -758,7 +758,7 @@
#
# @total: total amount of bytes involved in the backup process
#
@@ -106,7 +106,7 @@ index 6cad1e0e38..e00e577c6c 100644
# in the backup process which are marked dirty.
#
# @transferred: amount of bytes already backed up.
-@@ -830,7 +830,7 @@
+@@ -821,7 +821,7 @@
#
# @backup-time: backup timestamp (Unix epoch, required for format 'pbs')
#
@@ -115,7 +115,7 @@ index 6cad1e0e38..e00e577c6c 100644
#
# Returns: the uuid of the backup job
#
-@@ -842,7 +842,7 @@
+@@ -833,7 +833,7 @@
'*fingerprint': 'str',
'*backup-id': 'str',
'*backup-time': 'int',
diff --git a/debian/patches/pve/0042-PVE-fixup-pbs-backup-add-compress-and-encrypt-option.patch b/debian/patches/pve/0042-PVE-fixup-pbs-backup-add-compress-and-encrypt-option.patch
index 90cb3e1..601df5c 100644
--- a/debian/patches/pve/0042-PVE-fixup-pbs-backup-add-compress-and-encrypt-option.patch
+++ b/debian/patches/pve/0042-PVE-fixup-pbs-backup-add-compress-and-encrypt-option.patch
@@ -10,10 +10,10 @@ Subject: [PATCH] PVE: fixup pbs backup, add compress and encrypt options
3 files changed, 21 insertions(+), 2 deletions(-)
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
-index fdc85a5c0e..43aa87487b 100644
+index 056d14deee..46c63b1cf9 100644
--- a/block/monitor/block-hmp-cmds.c
+++ b/block/monitor/block-hmp-cmds.c
-@@ -1038,7 +1038,9 @@ void hmp_backup(Monitor *mon, const QDict *qdict)
+@@ -1039,7 +1039,9 @@ void hmp_backup(Monitor *mon, const QDict *qdict)
false, NULL, // PBS fingerprint
false, NULL, // PBS backup-id
false, 0, // PBS backup-time
@@ -78,10 +78,10 @@ index 1cd9d31d7c..bfb648d6b5 100644
.speed = speed,
.errp = errp,
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index e00e577c6c..a177fea6cd 100644
+index d4e1c98c50..0fda1e3fd3 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -832,6 +832,10 @@
+@@ -823,6 +823,10 @@
#
# @use-dirty-bitmap: use dirty bitmap to detect incremental changes since last job (optional for format 'pbs')
#
@@ -92,7 +92,7 @@ index e00e577c6c..a177fea6cd 100644
# Returns: the uuid of the backup job
#
##
-@@ -843,6 +847,8 @@
+@@ -834,6 +838,8 @@
'*backup-id': 'str',
'*backup-time': 'int',
'*use-dirty-bitmap': 'bool',
diff --git a/debian/patches/pve/0043-PVE-Add-PBS-block-driver-to-map-backup-archives-into.patch b/debian/patches/pve/0043-PVE-Add-PBS-block-driver-to-map-backup-archives-into.patch
index 0ff3b1f..4c31eaa 100644
--- a/debian/patches/pve/0043-PVE-Add-PBS-block-driver-to-map-backup-archives-into.patch
+++ b/debian/patches/pve/0043-PVE-Add-PBS-block-driver-to-map-backup-archives-into.patch
@@ -7,30 +7,28 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
[error cleanups, file_open implementation]
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
---
- block/Makefile.objs | 2 +
+ block/meson.build | 3 +
block/pbs.c | 271 +++++++++++++++++++++++++++++++++++++++++++
- configure | 10 ++
+ configure | 9 ++
+ meson.build | 1 +
qapi/block-core.json | 14 ++-
- 4 files changed, 296 insertions(+), 1 deletion(-)
+ 5 files changed, 297 insertions(+), 1 deletion(-)
create mode 100644 block/pbs.c
-diff --git a/block/Makefile.objs b/block/Makefile.objs
-index 9ea0477d0b..28fb3b7f7c 100644
---- a/block/Makefile.objs
-+++ b/block/Makefile.objs
-@@ -5,6 +5,7 @@ block-obj-$(CONFIG_CLOOP) += cloop.o
- block-obj-$(CONFIG_BOCHS) += bochs.o
- block-obj-$(CONFIG_VVFAT) += vvfat.o
- block-obj-$(CONFIG_DMG) += dmg.o
-+block-obj-$(CONFIG_PBS_BDRV) += pbs.o
+diff --git a/block/meson.build b/block/meson.build
+index dfae565db3..a070060e53 100644
+--- a/block/meson.build
++++ b/block/meson.build
+@@ -49,6 +49,9 @@ block_ss.add(files(
+ '../pve-backup.c',
+ ), libproxmox_backup_qemu)
+
++block_ss.add(when: 'CONFIG_PBS_BDRV', if_true: files('pbs.c'))
++block_ss.add(when: 'CONFIG_PBS_BDRV', if_true: libproxmox_backup_qemu)
++
+
+ softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c'))
- block-obj-y += qcow2.o qcow2-refcount.o qcow2-cluster.o qcow2-snapshot.o qcow2-cache.o qcow2-bitmap.o qcow2-threads.o
- block-obj-$(CONFIG_QED) += qed.o qed-l2-cache.o qed-table.o qed-cluster.o
-@@ -75,3 +76,4 @@ io_uring.o-cflags := $(LINUX_IO_URING_CFLAGS)
- io_uring.o-libs := $(LINUX_IO_URING_LIBS)
- parallels.o-cflags := $(LIBXML2_CFLAGS)
- parallels.o-libs := $(LIBXML2_LIBS)
-+pbs.o-libs := -lproxmox_backup_qemu
diff --git a/block/pbs.c b/block/pbs.c
new file mode 100644
index 0000000000..1481a2bfd1
@@ -309,18 +307,18 @@ index 0000000000..1481a2bfd1
+
+block_init(bdrv_pbs_init);
diff --git a/configure b/configure
-index 2acc4d1465..3fc80d0c2c 100755
+index 18c26e0389..33d9933871 100755
--- a/configure
+++ b/configure
-@@ -510,6 +510,7 @@ vvfat="yes"
+@@ -436,6 +436,7 @@ vvfat="yes"
qed="yes"
parallels="yes"
- sheepdog="yes"
+ sheepdog="no"
+pbs_bdrv="yes"
libxml2=""
debug_mutex="no"
libpmem=""
-@@ -1576,6 +1577,10 @@ for opt do
+@@ -1461,6 +1462,10 @@ for opt do
;;
--enable-sheepdog) sheepdog="yes"
;;
@@ -331,24 +329,16 @@ index 2acc4d1465..3fc80d0c2c 100755
--disable-vhost-user) vhost_user="no"
;;
--enable-vhost-user) vhost_user="yes"
-@@ -1936,6 +1941,7 @@ disabled with --disable-FEATURE, default is enabled if available:
+@@ -1843,6 +1848,7 @@ disabled with --disable-FEATURE, default is enabled if available:
qed qed image format support
parallels parallels image format support
- sheepdog sheepdog block driver support
+ sheepdog sheepdog block driver support (deprecated)
+ pbs-bdrv Proxmox backup server read-only block driver support
crypto-afalg Linux AF_ALG crypto backend driver
capstone capstone disassembler support
debug-mutex mutex debugging support
-@@ -7009,6 +7015,7 @@ echo "vvfat support $vvfat"
- echo "qed support $qed"
- echo "parallels support $parallels"
- echo "sheepdog support $sheepdog"
-+echo "pbs-bdrv support $pbs_bdrv"
- echo "capstone $capstone"
- echo "libpmem support $libpmem"
- echo "libdaxctl support $libdaxctl"
-@@ -7885,6 +7892,9 @@ fi
- if test "$sheepdog" = "yes" ; then
+@@ -6682,6 +6688,9 @@ if test "$sheepdog" = "yes" ; then
+ add_to deprecated_features "sheepdog"
echo "CONFIG_SHEEPDOG=y" >> $config_host_mak
fi
+if test "$pbs_bdrv" = "yes" ; then
@@ -357,11 +347,23 @@ index 2acc4d1465..3fc80d0c2c 100755
if test "$pty_h" = "yes" ; then
echo "HAVE_PTY_H=y" >> $config_host_mak
fi
+diff --git a/meson.build b/meson.build
+index 6f1fafee14..4d156d35ce 100644
+--- a/meson.build
++++ b/meson.build
+@@ -2199,6 +2199,7 @@ summary_info += {'vvfat support': config_host.has_key('CONFIG_VVFAT')}
+ summary_info += {'qed support': config_host.has_key('CONFIG_QED')}
+ summary_info += {'parallels support': config_host.has_key('CONFIG_PARALLELS')}
+ summary_info += {'sheepdog support': config_host.has_key('CONFIG_SHEEPDOG')}
++summary_info += {'PBS bdrv support': config_host.has_key('CONFIG_PBS_BDRV')}
+ summary_info += {'capstone': capstone_opt == 'disabled' ? false : capstone_opt}
+ summary_info += {'libpmem support': config_host.has_key('CONFIG_LIBPMEM')}
+ summary_info += {'libdaxctl support': config_host.has_key('CONFIG_LIBDAXCTL')}
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index a177fea6cd..f782c2cf96 100644
+index 0fda1e3fd3..553112d998 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -2951,7 +2951,7 @@
+@@ -2975,7 +2975,7 @@
'luks', 'nbd', 'nfs', 'null-aio', 'null-co', 'nvme', 'parallels',
'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd',
{ 'name': 'replication', 'if': 'defined(CONFIG_REPLICATION)' },
@@ -370,7 +372,7 @@ index a177fea6cd..f782c2cf96 100644
'ssh', 'throttle', 'vdi', 'vhdx', 'vmdk', 'vpc', 'vvfat' ] }
##
-@@ -3015,6 +3015,17 @@
+@@ -3039,6 +3039,17 @@
{ 'struct': 'BlockdevOptionsNull',
'data': { '*size': 'int', '*latency-ns': 'uint64', '*read-zeroes': 'bool' } }
@@ -388,7 +390,7 @@ index a177fea6cd..f782c2cf96 100644
##
# @BlockdevOptionsNVMe:
#
-@@ -4121,6 +4132,7 @@
+@@ -4148,6 +4159,7 @@
'nfs': 'BlockdevOptionsNfs',
'null-aio': 'BlockdevOptionsNull',
'null-co': 'BlockdevOptionsNull',
diff --git a/debian/patches/pve/0044-PVE-add-query_proxmox_support-QMP-command.patch b/debian/patches/pve/0044-PVE-add-query_proxmox_support-QMP-command.patch
index e2dc20f..a9cbe84 100644
--- a/debian/patches/pve/0044-PVE-add-query_proxmox_support-QMP-command.patch
+++ b/debian/patches/pve/0044-PVE-add-query_proxmox_support-QMP-command.patch
@@ -16,7 +16,7 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 files changed, 33 insertions(+)
diff --git a/pve-backup.c b/pve-backup.c
-index bfb648d6b5..6bf138cfc6 100644
+index bfb648d6b5..ba9d0d8a86 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -1051,3 +1051,11 @@ BackupStatus *qmp_query_backup(Error **errp)
@@ -32,10 +32,10 @@ index bfb648d6b5..6bf138cfc6 100644
+ return ret;
+}
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index f782c2cf96..1ed5987c88 100644
+index 553112d998..e0a0a60354 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -877,6 +877,31 @@
+@@ -868,6 +868,31 @@
##
{ 'command': 'backup-cancel' }
diff --git a/debian/patches/pve/0045-pbs-fix-missing-crypt-and-compress-parameters.patch b/debian/patches/pve/0045-pbs-fix-missing-crypt-and-compress-parameters.patch
index 34b0f51..d4a03be 100644
--- a/debian/patches/pve/0045-pbs-fix-missing-crypt-and-compress-parameters.patch
+++ b/debian/patches/pve/0045-pbs-fix-missing-crypt-and-compress-parameters.patch
@@ -9,7 +9,7 @@ Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
-index 6bf138cfc6..cd3a132d8b 100644
+index ba9d0d8a86..e1dcf10a45 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -958,6 +958,8 @@ UuidInfo *qmp_backup(
diff --git a/debian/patches/pve/0046-PVE-handle-PBS-write-callback-with-big-blocks-correc.patch b/debian/patches/pve/0046-PVE-handle-PBS-write-callback-with-big-blocks-correc.patch
index d45a455..7457eb6 100644
--- a/debian/patches/pve/0046-PVE-handle-PBS-write-callback-with-big-blocks-correc.patch
+++ b/debian/patches/pve/0046-PVE-handle-PBS-write-callback-with-big-blocks-correc.patch
@@ -17,7 +17,7 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
1 file changed, 22 insertions(+), 8 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
-index cd3a132d8b..f14273645a 100644
+index e1dcf10a45..3eba85506a 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -67,6 +67,7 @@ opts_init(pvebackup_init);
diff --git a/debian/patches/pve/0047-PVE-add-zero-block-handling-to-PBS-dump-callback.patch b/debian/patches/pve/0047-PVE-add-zero-block-handling-to-PBS-dump-callback.patch
index 930ec8f..3bb6b35 100644
--- a/debian/patches/pve/0047-PVE-add-zero-block-handling-to-PBS-dump-callback.patch
+++ b/debian/patches/pve/0047-PVE-add-zero-block-handling-to-PBS-dump-callback.patch
@@ -20,7 +20,7 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
1 file changed, 9 insertions(+), 5 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
-index f14273645a..bd802c6205 100644
+index 3eba85506a..40c2697b37 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -8,6 +8,7 @@
diff --git a/debian/patches/pve/0048-PVE-add-query-pbs-bitmap-info-QMP-call.patch b/debian/patches/pve/0048-PVE-add-query-pbs-bitmap-info-QMP-call.patch
index 54f1dfa..828bf4e 100644
--- a/debian/patches/pve/0048-PVE-add-query-pbs-bitmap-info-QMP-call.patch
+++ b/debian/patches/pve/0048-PVE-add-query-pbs-bitmap-info-QMP-call.patch
@@ -14,10 +14,10 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
3 files changed, 159 insertions(+), 42 deletions(-)
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
-index 3ff014d32a..c3227a1498 100644
+index 604026bb37..95f4e7f5c1 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
-@@ -195,6 +195,7 @@ void hmp_info_mice(Monitor *mon, const QDict *qdict)
+@@ -198,6 +198,7 @@ void hmp_info_mice(Monitor *mon, const QDict *qdict)
void hmp_info_backup(Monitor *mon, const QDict *qdict)
{
BackupStatus *info;
@@ -25,7 +25,7 @@ index 3ff014d32a..c3227a1498 100644
info = qmp_query_backup(NULL);
-@@ -225,26 +226,29 @@ void hmp_info_backup(Monitor *mon, const QDict *qdict)
+@@ -228,26 +229,29 @@ void hmp_info_backup(Monitor *mon, const QDict *qdict)
// this should not happen normally
monitor_printf(mon, "Total size: %d\n", 0);
} else {
@@ -68,7 +68,7 @@ index 3ff014d32a..c3227a1498 100644
info->zero_bytes, zero_per);
diff --git a/pve-backup.c b/pve-backup.c
-index bd802c6205..2db4a62580 100644
+index 40c2697b37..1e7b92a950 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -46,6 +46,7 @@ static struct PVEBackupState {
@@ -357,10 +357,10 @@ index bd802c6205..2db4a62580 100644
return ret;
}
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index 1ed5987c88..03fc0af99b 100644
+index e0a0a60354..b368371e8e 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -885,11 +885,14 @@
+@@ -876,11 +876,14 @@
# @pbs-dirty-bitmap: True if dirty-bitmap-incremental backups to PBS are
# supported.
#
@@ -375,7 +375,7 @@ index 1ed5987c88..03fc0af99b 100644
'pbs-library-version': 'str' } }
##
-@@ -902,6 +905,59 @@
+@@ -893,6 +896,59 @@
##
{ 'command': 'query-proxmox-support', 'returns': 'ProxmoxSupportStatus' }
diff --git a/debian/patches/pve/0049-PVE-redirect-stderr-to-journal-when-daemonized.patch b/debian/patches/pve/0049-PVE-redirect-stderr-to-journal-when-daemonized.patch
index 1d54282..7fb3c4f 100644
--- a/debian/patches/pve/0049-PVE-redirect-stderr-to-journal-when-daemonized.patch
+++ b/debian/patches/pve/0049-PVE-redirect-stderr-to-journal-when-daemonized.patch
@@ -1,6 +1,6 @@
From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
From: Stefan Reiter <s.reiter@proxmox.com>
-Date: Tue, 30 Jun 2020 13:10:10 +0200
+Date: Tue, 12 Jan 2021 14:12:20 +0100
Subject: [PATCH] PVE: redirect stderr to journal when daemonized
QEMU uses the logging for error messages usually, so LOG_ERR is most
@@ -8,24 +8,32 @@ fitting.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
- Makefile.objs | 1 +
- os-posix.c | 7 +++++--
- 2 files changed, 6 insertions(+), 2 deletions(-)
+ meson.build | 2 ++
+ os-posix.c | 7 +++++--
+ 2 files changed, 7 insertions(+), 2 deletions(-)
-diff --git a/Makefile.objs b/Makefile.objs
-index 240eb503f2..c7ba4e11e7 100644
---- a/Makefile.objs
-+++ b/Makefile.objs
-@@ -54,6 +54,7 @@ common-obj-y += net/
- common-obj-y += qdev-monitor.o
- common-obj-$(CONFIG_WIN32) += os-win32.o
- common-obj-$(CONFIG_POSIX) += os-posix.o
-+os-posix.o-libs := -lsystemd
+diff --git a/meson.build b/meson.build
+index 4d156d35ce..737ea9e5d7 100644
+--- a/meson.build
++++ b/meson.build
+@@ -726,6 +726,7 @@ keyutils = dependency('libkeyutils', required: false,
+ has_gettid = cc.has_function('gettid')
- common-obj-$(CONFIG_LINUX) += fsdev/
+ libuuid = cc.find_library('uuid', required: true)
++libsystemd = cc.find_library('systemd', required: true)
+ libproxmox_backup_qemu = cc.find_library('proxmox_backup_qemu', required: true)
+ # Malloc tests
+@@ -1539,6 +1540,7 @@ blockdev_ss.add(files(
+ # os-posix.c contains POSIX-specific functions used by qemu-storage-daemon,
+ # os-win32.c does not
+ blockdev_ss.add(when: 'CONFIG_POSIX', if_true: files('os-posix.c'))
++blockdev_ss.add(when: 'CONFIG_POSIX', if_true: libsystemd)
+ softmmu_ss.add(when: 'CONFIG_WIN32', if_true: [files('os-win32.c')])
+
+ common_ss.add(files('cpus-common.c'))
diff --git a/os-posix.c b/os-posix.c
-index 3572db3f44..b45dde63ac 100644
+index 1de2839554..ac4f652923 100644
--- a/os-posix.c
+++ b/os-posix.c
@@ -28,6 +28,8 @@
@@ -37,7 +45,7 @@ index 3572db3f44..b45dde63ac 100644
#include "qemu-common.h"
/* Needed early for CONFIG_BSD etc. */
-@@ -312,9 +314,10 @@ void os_setup_post(void)
+@@ -288,9 +290,10 @@ void os_setup_post(void)
dup2(fd, 0);
dup2(fd, 1);
diff --git a/debian/patches/pve/0050-PVE-Add-sequential-job-transaction-support.patch b/debian/patches/pve/0050-PVE-Add-sequential-job-transaction-support.patch
index 20531d3..24f8bfb 100644
--- a/debian/patches/pve/0050-PVE-Add-sequential-job-transaction-support.patch
+++ b/debian/patches/pve/0050-PVE-Add-sequential-job-transaction-support.patch
@@ -33,7 +33,7 @@ index 32aabb1c60..f7a6a0926a 100644
* Release a reference that was previously acquired with job_txn_add_job or
* job_txn_new. If it's the last reference to the object, it will be freed.
diff --git a/job.c b/job.c
-index b8139c80a4..97ee97a192 100644
+index f9884e7d9d..8f06e05fbf 100644
--- a/job.c
+++ b/job.c
@@ -72,6 +72,8 @@ struct JobTxn {
diff --git a/debian/patches/pve/0051-PVE-Backup-Use-a-transaction-to-synchronize-job-stat.patch b/debian/patches/pve/0051-PVE-Backup-Use-a-transaction-to-synchronize-job-stat.patch
index daa2498..bb3b026 100644
--- a/debian/patches/pve/0051-PVE-Backup-Use-a-transaction-to-synchronize-job-stat.patch
+++ b/debian/patches/pve/0051-PVE-Backup-Use-a-transaction-to-synchronize-job-stat.patch
@@ -16,7 +16,7 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
1 file changed, 49 insertions(+), 118 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
-index 2db4a62580..b52f4a9364 100644
+index 1e7b92a950..f3df43eb8c 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -52,6 +52,7 @@ static struct PVEBackupState {
diff --git a/debian/patches/pve/0052-PVE-Backup-Use-more-coroutines-and-don-t-block-on-fi.patch b/debian/patches/pve/0052-PVE-Backup-Use-more-coroutines-and-don-t-block-on-fi.patch
index 49213d9..b215d3d 100644
--- a/debian/patches/pve/0052-PVE-Backup-Use-more-coroutines-and-don-t-block-on-fi.patch
+++ b/debian/patches/pve/0052-PVE-Backup-Use-more-coroutines-and-don-t-block-on-fi.patch
@@ -38,7 +38,7 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
2 files changed, 95 insertions(+), 58 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
-index b52f4a9364..4402c0cb17 100644
+index f3df43eb8c..f3b8ce1f3a 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -33,7 +33,9 @@ const char *PBS_BITMAP_NAME = "pbs-incremental-dirty-bitmap";
@@ -359,10 +359,10 @@ index b52f4a9364..4402c0cb17 100644
qemu_mutex_unlock(&backup_state.stat.lock);
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index 03fc0af99b..29650896e2 100644
+index b368371e8e..b90d6abe64 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -784,12 +784,15 @@
+@@ -775,12 +775,15 @@
#
# @uuid: uuid for this backup job
#
diff --git a/debian/patches/pve/0053-PVE-fix-and-clean-up-error-handling-for-create_backu.patch b/debian/patches/pve/0053-PVE-fix-and-clean-up-error-handling-for-create_backu.patch
index 873fd37..92dc3e0 100644
--- a/debian/patches/pve/0053-PVE-fix-and-clean-up-error-handling-for-create_backu.patch
+++ b/debian/patches/pve/0053-PVE-fix-and-clean-up-error-handling-for-create_backu.patch
@@ -22,7 +22,7 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
1 file changed, 54 insertions(+), 25 deletions(-)
diff --git a/pve-backup.c b/pve-backup.c
-index 4402c0cb17..c7cde0fb0e 100644
+index f3b8ce1f3a..ded90cb822 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -50,6 +50,7 @@ static struct PVEBackupState {
diff --git a/debian/patches/pve/0055-PVE-Migrate-dirty-bitmap-state-via-savevm.patch b/debian/patches/pve/0054-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
similarity index 90%
rename from debian/patches/pve/0055-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
rename to debian/patches/pve/0054-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
index 1c64645..a4ee6bc 100644
--- a/debian/patches/pve/0055-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
+++ b/debian/patches/pve/0054-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
@@ -14,12 +14,12 @@ safe migration is possible and makes sense.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
include/migration/misc.h | 3 ++
- migration/Makefile.objs | 1 +
+ migration/meson.build | 2 +
migration/pbs-state.c | 106 +++++++++++++++++++++++++++++++++++++++
pve-backup.c | 1 +
qapi/block-core.json | 6 +++
softmmu/vl.c | 1 +
- 6 files changed, 118 insertions(+)
+ 6 files changed, 119 insertions(+)
create mode 100644 migration/pbs-state.c
diff --git a/include/migration/misc.h b/include/migration/misc.h
@@ -34,18 +34,21 @@ index 34e7d75713..f83816dd3c 100644
+void pbs_state_mig_init(void);
+
#endif
-diff --git a/migration/Makefile.objs b/migration/Makefile.objs
-index 0fc619e380..20b3792599 100644
---- a/migration/Makefile.objs
-+++ b/migration/Makefile.objs
-@@ -9,6 +9,7 @@ common-obj-y += qjson.o
- common-obj-y += block-dirty-bitmap.o
- common-obj-y += multifd.o
- common-obj-y += multifd-zlib.o
-+common-obj-y += pbs-state.o
- common-obj-$(CONFIG_ZSTD) += multifd-zstd.o
+diff --git a/migration/meson.build b/migration/meson.build
+index e62b79b60f..b90a04aa75 100644
+--- a/migration/meson.build
++++ b/migration/meson.build
+@@ -7,8 +7,10 @@ migration_files = files(
+ 'qemu-file-channel.c',
+ 'qemu-file.c',
+ 'qjson.c',
++ 'pbs-state.c',
+ )
+ softmmu_ss.add(migration_files)
++softmmu_ss.add(libproxmox_backup_qemu)
- common-obj-$(CONFIG_RDMA) += rdma.o
+ softmmu_ss.add(files(
+ 'block-dirty-bitmap.c',
diff --git a/migration/pbs-state.c b/migration/pbs-state.c
new file mode 100644
index 0000000000..29f2b3860d
@@ -159,7 +162,7 @@ index 0000000000..29f2b3860d
+ NULL);
+}
diff --git a/pve-backup.c b/pve-backup.c
-index c7cde0fb0e..f65f1dda26 100644
+index ded90cb822..6b25292ba1 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -1130,5 +1130,6 @@ ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp)
@@ -170,10 +173,10 @@ index c7cde0fb0e..f65f1dda26 100644
return ret;
}
diff --git a/qapi/block-core.json b/qapi/block-core.json
-index 29650896e2..0da4b35028 100644
+index b90d6abe64..dee3d87efe 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
-@@ -890,12 +890,18 @@
+@@ -881,12 +881,18 @@
#
# @query-bitmap-info: True if the 'query-pbs-bitmap-info' QMP call is supported.
#
@@ -193,10 +196,10 @@ index 29650896e2..0da4b35028 100644
##
diff --git a/softmmu/vl.c b/softmmu/vl.c
-index 16aa2186b0..88b13871fd 100644
+index 5b5512128e..6721889fee 100644
--- a/softmmu/vl.c
+++ b/softmmu/vl.c
-@@ -4288,6 +4288,7 @@ void qemu_init(int argc, char **argv, char **envp)
+@@ -4304,6 +4304,7 @@ void qemu_init(int argc, char **argv, char **envp)
blk_mig_init();
ram_mig_init();
dirty_bitmap_mig_init();
diff --git a/debian/patches/pve/0054-migration-block-dirty-bitmap-fix-larger-granularity-.patch b/debian/patches/pve/0054-migration-block-dirty-bitmap-fix-larger-granularity-.patch
deleted file mode 100644
index a05478b..0000000
--- a/debian/patches/pve/0054-migration-block-dirty-bitmap-fix-larger-granularity-.patch
+++ /dev/null
@@ -1,33 +0,0 @@
-From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
-From: Stefan Reiter <s.reiter@proxmox.com>
-Date: Thu, 22 Oct 2020 17:34:17 +0200
-Subject: [PATCH] migration/block-dirty-bitmap: fix larger granularity bitmaps
-
-sectors_per_chunk is a 64 bit integer, but the calculation would be done
-in 32 bits, leading to an overflow for coarse bitmap granularities.
-
-If that results in the value 0, it leads to a hang where no progress is
-made but send_bitmap_bits is constantly called with nr_sectors being 0.
-
-Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
-Reviewed-by: Eric Blake <eblake@redhat.com>
-Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
----
- migration/block-dirty-bitmap.c | 3 ++-
- 1 file changed, 2 insertions(+), 1 deletion(-)
-
-diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
-index 784330ebe1..5bf0d9fbc6 100644
---- a/migration/block-dirty-bitmap.c
-+++ b/migration/block-dirty-bitmap.c
-@@ -334,8 +334,9 @@ static int add_bitmaps_to_list(DBMSaveState *s, BlockDriverState *bs,
- dbms->node_name = bs_name;
- dbms->bitmap = bitmap;
- dbms->total_sectors = bdrv_nb_sectors(bs);
-- dbms->sectors_per_chunk = CHUNK_SIZE * 8 *
-+ dbms->sectors_per_chunk = CHUNK_SIZE * 8LLU *
- bdrv_dirty_bitmap_granularity(bitmap) >> BDRV_SECTOR_BITS;
-+ assert(dbms->sectors_per_chunk != 0);
- if (bdrv_dirty_bitmap_enabled(bitmap)) {
- dbms->flags |= DIRTY_BITMAP_MIG_START_FLAG_ENABLED;
- }
diff --git a/debian/patches/pve/0056-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch b/debian/patches/pve/0055-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch
similarity index 89%
rename from debian/patches/pve/0056-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch
rename to debian/patches/pve/0055-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch
index 3154291..775f992 100644
--- a/debian/patches/pve/0056-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch
+++ b/debian/patches/pve/0055-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch
@@ -18,10 +18,10 @@ Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/migration/block-dirty-bitmap.c b/migration/block-dirty-bitmap.c
-index 5bf0d9fbc6..1070c19181 100644
+index c61d382be8..26e4e5c99c 100644
--- a/migration/block-dirty-bitmap.c
+++ b/migration/block-dirty-bitmap.c
-@@ -323,7 +323,7 @@ static int add_bitmaps_to_list(DBMSaveState *s, BlockDriverState *bs,
+@@ -534,7 +534,7 @@ static int add_bitmaps_to_list(DBMSaveState *s, BlockDriverState *bs,
if (bdrv_dirty_bitmap_check(bitmap, BDRV_BITMAP_DEFAULT, &local_err)) {
error_report_err(local_err);
@@ -29,4 +29,4 @@ index 5bf0d9fbc6..1070c19181 100644
+ continue;
}
- bdrv_ref(bs);
+ if (bitmap_aliases) {
diff --git a/debian/patches/pve/0057-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch b/debian/patches/pve/0056-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch
similarity index 97%
rename from debian/patches/pve/0057-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch
rename to debian/patches/pve/0056-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch
index 0b5e601..0e30326 100644
--- a/debian/patches/pve/0057-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch
+++ b/debian/patches/pve/0056-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch
@@ -20,7 +20,7 @@ Signed-off-by: Wolfgang Bumiller <w.bumiller@proxmox.com>
1 file changed, 7 insertions(+)
diff --git a/job.c b/job.c
-index 97ee97a192..51984e557c 100644
+index 8f06e05fbf..05b7797e82 100644
--- a/job.c
+++ b/job.c
@@ -1035,6 +1035,13 @@ int job_finish_sync(Job *job, void (*finish)(Job *, Error **errp), Error **errp)
diff --git a/debian/patches/pve/0058-PVE-fall-back-to-open-iscsi-initiatorname.patch b/debian/patches/pve/0057-PVE-fall-back-to-open-iscsi-initiatorname.patch
similarity index 97%
rename from debian/patches/pve/0058-PVE-fall-back-to-open-iscsi-initiatorname.patch
rename to debian/patches/pve/0057-PVE-fall-back-to-open-iscsi-initiatorname.patch
index 9f0e822..f4c4652 100644
--- a/debian/patches/pve/0058-PVE-fall-back-to-open-iscsi-initiatorname.patch
+++ b/debian/patches/pve/0057-PVE-fall-back-to-open-iscsi-initiatorname.patch
@@ -20,7 +20,7 @@ Signed-off-by: Fabian Ebner <f.ebner@proxmox.com>
1 file changed, 30 insertions(+)
diff --git a/block/iscsi.c b/block/iscsi.c
-index bd2122a3a4..56437d8b99 100644
+index e30a7e3606..6c70bbe351 100644
--- a/block/iscsi.c
+++ b/block/iscsi.c
@@ -1374,12 +1374,42 @@ static char *get_initiator_name(QemuOpts *opts)
@@ -66,6 +66,3 @@ index bd2122a3a4..56437d8b99 100644
uuid_info = qmp_query_uuid(NULL);
if (strcmp(uuid_info->UUID, UUID_NONE) == 0) {
name = qemu_get_vm_name();
---
-2.20.1
-
diff --git a/debian/patches/pve/0058-PVE-Use-coroutine-QMP-for-backup-cancel_backup.patch b/debian/patches/pve/0058-PVE-Use-coroutine-QMP-for-backup-cancel_backup.patch
new file mode 100644
index 0000000..a18d478
--- /dev/null
+++ b/debian/patches/pve/0058-PVE-Use-coroutine-QMP-for-backup-cancel_backup.patch
@@ -0,0 +1,597 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Stefan Reiter <s.reiter@proxmox.com>
+Date: Tue, 26 Jan 2021 15:45:30 +0100
+Subject: [PATCH] PVE: Use coroutine QMP for backup/cancel_backup
+
+Finally turn backup QMP calls into coroutines, now that it's possible.
+This has the benefit that calls are asynchronous to the main loop, i.e.
+long running operations like connecting to a PBS server will no longer
+hang the VM.
+
+Additionally, it allows us to get rid of block_on_coroutine_fn, which
+was always a hacky workaround.
+
+While we're already spring cleaning, also remove the QmpBackupTask
+struct, since we can now put the 'prepare' function directly into
+qmp_backup and thus no longer need those giant walls of text.
+
+(Note that for our patches to work with 5.2.0 this change is actually
+required, otherwise monitor_get_fd() fails as we're not in a QMP
+coroutine, but one we start ourselves - we could of course set the
+monitor for that coroutine ourselves, but let's just fix it the right
+way instead)
+
+Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
+---
+ block/monitor/block-hmp-cmds.c | 4 +-
+ hmp-commands.hx | 2 +
+ proxmox-backup-client.c | 31 -----
+ pve-backup.c | 232 ++++++++++-----------------------
+ qapi/block-core.json | 4 +-
+ 5 files changed, 77 insertions(+), 196 deletions(-)
+
+diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
+index 46c63b1cf9..11c84d5508 100644
+--- a/block/monitor/block-hmp-cmds.c
++++ b/block/monitor/block-hmp-cmds.c
+@@ -1013,7 +1013,7 @@ void hmp_info_snapshots(Monitor *mon, const QDict *qdict)
+ g_free(global_snapshots);
+ }
+
+-void hmp_backup_cancel(Monitor *mon, const QDict *qdict)
++void coroutine_fn hmp_backup_cancel(Monitor *mon, const QDict *qdict)
+ {
+ Error *error = NULL;
+
+@@ -1022,7 +1022,7 @@ void hmp_backup_cancel(Monitor *mon, const QDict *qdict)
+ hmp_handle_error(mon, error);
+ }
+
+-void hmp_backup(Monitor *mon, const QDict *qdict)
++void coroutine_fn hmp_backup(Monitor *mon, const QDict *qdict)
+ {
+ Error *error = NULL;
+
+diff --git a/hmp-commands.hx b/hmp-commands.hx
+index 0c6b944850..54de3f80e6 100644
+--- a/hmp-commands.hx
++++ b/hmp-commands.hx
+@@ -108,6 +108,7 @@ ERST
+ "\n\t\t\t Use -d to dump data into a directory instead"
+ "\n\t\t\t of using VMA format.",
+ .cmd = hmp_backup,
++ .coroutine = true,
+ },
+
+ SRST
+@@ -121,6 +122,7 @@ ERST
+ .params = "",
+ .help = "cancel the current VM backup",
+ .cmd = hmp_backup_cancel,
++ .coroutine = true,
+ },
+
+ SRST
+diff --git a/proxmox-backup-client.c b/proxmox-backup-client.c
+index 4ce7bc0b5e..0923037dec 100644
+--- a/proxmox-backup-client.c
++++ b/proxmox-backup-client.c
+@@ -5,37 +5,6 @@
+
+ /* Proxmox Backup Server client bindings using coroutines */
+
+-typedef struct BlockOnCoroutineWrapper {
+- AioContext *ctx;
+- CoroutineEntry *entry;
+- void *entry_arg;
+- bool finished;
+-} BlockOnCoroutineWrapper;
+-
+-static void coroutine_fn block_on_coroutine_wrapper(void *opaque)
+-{
+- BlockOnCoroutineWrapper *wrapper = opaque;
+- wrapper->entry(wrapper->entry_arg);
+- wrapper->finished = true;
+- aio_wait_kick();
+-}
+-
+-void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg)
+-{
+- assert(!qemu_in_coroutine());
+-
+- AioContext *ctx = qemu_get_current_aio_context();
+- BlockOnCoroutineWrapper wrapper = {
+- .finished = false,
+- .entry = entry,
+- .entry_arg = entry_arg,
+- .ctx = ctx,
+- };
+- Coroutine *wrapper_co = qemu_coroutine_create(block_on_coroutine_wrapper, &wrapper);
+- aio_co_enter(ctx, wrapper_co);
+- AIO_WAIT_WHILE(ctx, !wrapper.finished);
+-}
+-
+ // This is called from another thread, so we use aio_co_schedule()
+ static void proxmox_backup_schedule_wake(void *data) {
+ CoCtxData *waker = (CoCtxData *)data;
+diff --git a/pve-backup.c b/pve-backup.c
+index 6b25292ba1..f7597ae55c 100644
+--- a/pve-backup.c
++++ b/pve-backup.c
+@@ -357,7 +357,7 @@ static void job_cancel_bh(void *opaque) {
+ aio_co_enter(data->ctx, data->co);
+ }
+
+-static void coroutine_fn pvebackup_co_cancel(void *opaque)
++void coroutine_fn qmp_backup_cancel(Error **errp)
+ {
+ Error *cancel_err = NULL;
+ error_setg(&cancel_err, "backup canceled");
+@@ -394,11 +394,6 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+ }
+
+-void qmp_backup_cancel(Error **errp)
+-{
+- block_on_coroutine_fn(pvebackup_co_cancel, NULL);
+-}
+-
+ // assumes the caller holds backup_mutex
+ static int coroutine_fn pvebackup_co_add_config(
+ const char *file,
+@@ -531,50 +526,27 @@ static void create_backup_jobs_bh(void *opaque) {
+ aio_co_enter(data->ctx, data->co);
+ }
+
+-typedef struct QmpBackupTask {
+- const char *backup_file;
+- bool has_password;
+- const char *password;
+- bool has_keyfile;
+- const char *keyfile;
+- bool has_key_password;
+- const char *key_password;
+- bool has_backup_id;
+- const char *backup_id;
+- bool has_backup_time;
+- const char *fingerprint;
+- bool has_fingerprint;
+- int64_t backup_time;
+- bool has_use_dirty_bitmap;
+- bool use_dirty_bitmap;
+- bool has_format;
+- BackupFormat format;
+- bool has_config_file;
+- const char *config_file;
+- bool has_firewall_file;
+- const char *firewall_file;
+- bool has_devlist;
+- const char *devlist;
+- bool has_compress;
+- bool compress;
+- bool has_encrypt;
+- bool encrypt;
+- bool has_speed;
+- int64_t speed;
+- Error **errp;
+- UuidInfo *result;
+-} QmpBackupTask;
+-
+-static void coroutine_fn pvebackup_co_prepare(void *opaque)
++UuidInfo coroutine_fn *qmp_backup(
++ const char *backup_file,
++ bool has_password, const char *password,
++ bool has_keyfile, const char *keyfile,
++ bool has_key_password, const char *key_password,
++ bool has_fingerprint, const char *fingerprint,
++ bool has_backup_id, const char *backup_id,
++ bool has_backup_time, int64_t backup_time,
++ bool has_use_dirty_bitmap, bool use_dirty_bitmap,
++ bool has_compress, bool compress,
++ bool has_encrypt, bool encrypt,
++ bool has_format, BackupFormat format,
++ bool has_config_file, const char *config_file,
++ bool has_firewall_file, const char *firewall_file,
++ bool has_devlist, const char *devlist,
++ bool has_speed, int64_t speed, Error **errp)
+ {
+ assert(qemu_in_coroutine());
+
+ qemu_co_mutex_lock(&backup_state.backup_mutex);
+
+- QmpBackupTask *task = opaque;
+-
+- task->result = NULL; // just to be sure
+-
+ BlockBackend *blk;
+ BlockDriverState *bs = NULL;
+ const char *backup_dir = NULL;
+@@ -591,17 +563,17 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ const char *firewall_name = "qemu-server.fw";
+
+ if (backup_state.di_list) {
+- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
++ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
+ "previous backup not finished");
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+- return;
++ return NULL;
+ }
+
+ /* Todo: try to auto-detect format based on file name */
+- BackupFormat format = task->has_format ? task->format : BACKUP_FORMAT_VMA;
++ format = has_format ? format : BACKUP_FORMAT_VMA;
+
+- if (task->has_devlist) {
+- devs = g_strsplit_set(task->devlist, ",;:", -1);
++ if (has_devlist) {
++ devs = g_strsplit_set(devlist, ",;:", -1);
+
+ gchar **d = devs;
+ while (d && *d) {
+@@ -609,14 +581,14 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ if (blk) {
+ bs = blk_bs(blk);
+ if (!bdrv_is_inserted(bs)) {
+- error_setg(task->errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
++ error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
+ goto err;
+ }
+ PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
+ di->bs = bs;
+ di_list = g_list_append(di_list, di);
+ } else {
+- error_set(task->errp, ERROR_CLASS_DEVICE_NOT_FOUND,
++ error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+ "Device '%s' not found", *d);
+ goto err;
+ }
+@@ -639,7 +611,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ }
+
+ if (!di_list) {
+- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
++ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
+ goto err;
+ }
+
+@@ -649,13 +621,13 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ while (l) {
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
+ l = g_list_next(l);
+- if (bdrv_op_is_blocked(di->bs, BLOCK_OP_TYPE_BACKUP_SOURCE, task->errp)) {
++ if (bdrv_op_is_blocked(di->bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
+ goto err;
+ }
+
+ ssize_t size = bdrv_getlength(di->bs);
+ if (size < 0) {
+- error_setg_errno(task->errp, -di->size, "bdrv_getlength failed");
++ error_setg_errno(errp, -di->size, "bdrv_getlength failed");
+ goto err;
+ }
+ di->size = size;
+@@ -682,47 +654,44 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ }
+
+ if (format == BACKUP_FORMAT_PBS) {
+- if (!task->has_password) {
+- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'password'");
++ if (!has_password) {
++ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'password'");
+ goto err_mutex;
+ }
+- if (!task->has_backup_id) {
+- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-id'");
++ if (!has_backup_id) {
++ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-id'");
+ goto err_mutex;
+ }
+- if (!task->has_backup_time) {
+- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-time'");
++ if (!has_backup_time) {
++ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-time'");
+ goto err_mutex;
+ }
+
+ int dump_cb_block_size = PROXMOX_BACKUP_DEFAULT_CHUNK_SIZE; // Hardcoded (4M)
+ firewall_name = "fw.conf";
+
+- bool use_dirty_bitmap = task->has_use_dirty_bitmap && task->use_dirty_bitmap;
+-
+-
+ char *pbs_err = NULL;
+ pbs = proxmox_backup_new(
+- task->backup_file,
+- task->backup_id,
+- task->backup_time,
++ backup_file,
++ backup_id,
++ backup_time,
+ dump_cb_block_size,
+- task->has_password ? task->password : NULL,
+- task->has_keyfile ? task->keyfile : NULL,
+- task->has_key_password ? task->key_password : NULL,
+- task->has_compress ? task->compress : true,
+- task->has_encrypt ? task->encrypt : task->has_keyfile,
+- task->has_fingerprint ? task->fingerprint : NULL,
++ has_password ? password : NULL,
++ has_keyfile ? keyfile : NULL,
++ has_key_password ? key_password : NULL,
++ has_compress ? compress : true,
++ has_encrypt ? encrypt : has_keyfile,
++ has_fingerprint ? fingerprint : NULL,
+ &pbs_err);
+
+ if (!pbs) {
+- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
++ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
+ "proxmox_backup_new failed: %s", pbs_err);
+ proxmox_backup_free_error(pbs_err);
+ goto err_mutex;
+ }
+
+- int connect_result = proxmox_backup_co_connect(pbs, task->errp);
++ int connect_result = proxmox_backup_co_connect(pbs, errp);
+ if (connect_result < 0)
+ goto err_mutex;
+
+@@ -741,9 +710,9 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ BdrvDirtyBitmap *bitmap = bdrv_find_dirty_bitmap(di->bs, PBS_BITMAP_NAME);
+ bool expect_only_dirty = false;
+
+- if (use_dirty_bitmap) {
++ if (has_use_dirty_bitmap && use_dirty_bitmap) {
+ if (bitmap == NULL) {
+- bitmap = bdrv_create_dirty_bitmap(di->bs, dump_cb_block_size, PBS_BITMAP_NAME, task->errp);
++ bitmap = bdrv_create_dirty_bitmap(di->bs, dump_cb_block_size, PBS_BITMAP_NAME, errp);
+ if (!bitmap) {
+ goto err_mutex;
+ }
+@@ -773,12 +742,12 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ }
+ }
+
+- int dev_id = proxmox_backup_co_register_image(pbs, devname, di->size, expect_only_dirty, task->errp);
++ int dev_id = proxmox_backup_co_register_image(pbs, devname, di->size, expect_only_dirty, errp);
+ if (dev_id < 0) {
+ goto err_mutex;
+ }
+
+- if (!(di->target = bdrv_backup_dump_create(dump_cb_block_size, di->size, pvebackup_co_dump_pbs_cb, di, task->errp))) {
++ if (!(di->target = bdrv_backup_dump_create(dump_cb_block_size, di->size, pvebackup_co_dump_pbs_cb, di, errp))) {
+ goto err_mutex;
+ }
+
+@@ -792,10 +761,10 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ backup_state.stat.bitmap_list = g_list_append(backup_state.stat.bitmap_list, info);
+ }
+ } else if (format == BACKUP_FORMAT_VMA) {
+- vmaw = vma_writer_create(task->backup_file, uuid, &local_err);
++ vmaw = vma_writer_create(backup_file, uuid, &local_err);
+ if (!vmaw) {
+ if (local_err) {
+- error_propagate(task->errp, local_err);
++ error_propagate(errp, local_err);
+ }
+ goto err_mutex;
+ }
+@@ -806,25 +775,25 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
+ l = g_list_next(l);
+
+- if (!(di->target = bdrv_backup_dump_create(VMA_CLUSTER_SIZE, di->size, pvebackup_co_dump_vma_cb, di, task->errp))) {
++ if (!(di->target = bdrv_backup_dump_create(VMA_CLUSTER_SIZE, di->size, pvebackup_co_dump_vma_cb, di, errp))) {
+ goto err_mutex;
+ }
+
+ const char *devname = bdrv_get_device_name(di->bs);
+ di->dev_id = vma_writer_register_stream(vmaw, devname, di->size);
+ if (di->dev_id <= 0) {
+- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
++ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
+ "register_stream failed");
+ goto err_mutex;
+ }
+ }
+ } else if (format == BACKUP_FORMAT_DIR) {
+- if (mkdir(task->backup_file, 0640) != 0) {
+- error_setg_errno(task->errp, errno, "can't create directory '%s'\n",
+- task->backup_file);
++ if (mkdir(backup_file, 0640) != 0) {
++ error_setg_errno(errp, errno, "can't create directory '%s'\n",
++ backup_file);
+ goto err_mutex;
+ }
+- backup_dir = task->backup_file;
++ backup_dir = backup_file;
+
+ l = di_list;
+ while (l) {
+@@ -838,34 +807,34 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ bdrv_img_create(di->targetfile, "raw", NULL, NULL, NULL,
+ di->size, flags, false, &local_err);
+ if (local_err) {
+- error_propagate(task->errp, local_err);
++ error_propagate(errp, local_err);
+ goto err_mutex;
+ }
+
+ di->target = bdrv_open(di->targetfile, NULL, NULL, flags, &local_err);
+ if (!di->target) {
+- error_propagate(task->errp, local_err);
++ error_propagate(errp, local_err);
+ goto err_mutex;
+ }
+ }
+ } else {
+- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "unknown backup format");
++ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "unknown backup format");
+ goto err_mutex;
+ }
+
+
+ /* add configuration file to archive */
+- if (task->has_config_file) {
+- if (pvebackup_co_add_config(task->config_file, config_name, format, backup_dir,
+- vmaw, pbs, task->errp) != 0) {
++ if (has_config_file) {
++ if (pvebackup_co_add_config(config_file, config_name, format, backup_dir,
++ vmaw, pbs, errp) != 0) {
+ goto err_mutex;
+ }
+ }
+
+ /* add firewall file to archive */
+- if (task->has_firewall_file) {
+- if (pvebackup_co_add_config(task->firewall_file, firewall_name, format, backup_dir,
+- vmaw, pbs, task->errp) != 0) {
++ if (has_firewall_file) {
++ if (pvebackup_co_add_config(firewall_file, firewall_name, format, backup_dir,
++ vmaw, pbs, errp) != 0) {
+ goto err_mutex;
+ }
+ }
+@@ -883,7 +852,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ if (backup_state.stat.backup_file) {
+ g_free(backup_state.stat.backup_file);
+ }
+- backup_state.stat.backup_file = g_strdup(task->backup_file);
++ backup_state.stat.backup_file = g_strdup(backup_file);
+
+ uuid_copy(backup_state.stat.uuid, uuid);
+ uuid_unparse_lower(uuid, backup_state.stat.uuid_str);
+@@ -898,7 +867,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+
+ qemu_mutex_unlock(&backup_state.stat.lock);
+
+- backup_state.speed = (task->has_speed && task->speed > 0) ? task->speed : 0;
++ backup_state.speed = (has_speed && speed > 0) ? speed : 0;
+
+ backup_state.vmaw = vmaw;
+ backup_state.pbs = pbs;
+@@ -908,8 +877,6 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ uuid_info = g_malloc0(sizeof(*uuid_info));
+ uuid_info->UUID = uuid_str;
+
+- task->result = uuid_info;
+-
+ /* Run create_backup_jobs_bh outside of coroutine (in BH) but keep
+ * backup_mutex locked. This is fine, a CoMutex can be held across yield
+ * points, and we'll release it as soon as the BH reschedules us.
+@@ -923,7 +890,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ qemu_coroutine_yield();
+
+ if (local_err) {
+- error_propagate(task->errp, local_err);
++ error_propagate(errp, local_err);
+ goto err;
+ }
+
+@@ -936,7 +903,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
+ /* start the first job in the transaction */
+ job_txn_start_seq(backup_state.txn);
+
+- return;
++ return uuid_info;
+
+ err_mutex:
+ qemu_mutex_unlock(&backup_state.stat.lock);
+@@ -967,7 +934,7 @@ err:
+ if (vmaw) {
+ Error *err = NULL;
+ vma_writer_close(vmaw, &err);
+- unlink(task->backup_file);
++ unlink(backup_file);
+ }
+
+ if (pbs) {
+@@ -978,65 +945,8 @@ err:
+ rmdir(backup_dir);
+ }
+
+- task->result = NULL;
+-
+ qemu_co_mutex_unlock(&backup_state.backup_mutex);
+- return;
+-}
+-
+-UuidInfo *qmp_backup(
+- const char *backup_file,
+- bool has_password, const char *password,
+- bool has_keyfile, const char *keyfile,
+- bool has_key_password, const char *key_password,
+- bool has_fingerprint, const char *fingerprint,
+- bool has_backup_id, const char *backup_id,
+- bool has_backup_time, int64_t backup_time,
+- bool has_use_dirty_bitmap, bool use_dirty_bitmap,
+- bool has_compress, bool compress,
+- bool has_encrypt, bool encrypt,
+- bool has_format, BackupFormat format,
+- bool has_config_file, const char *config_file,
+- bool has_firewall_file, const char *firewall_file,
+- bool has_devlist, const char *devlist,
+- bool has_speed, int64_t speed, Error **errp)
+-{
+- QmpBackupTask task = {
+- .backup_file = backup_file,
+- .has_password = has_password,
+- .password = password,
+- .has_keyfile = has_keyfile,
+- .keyfile = keyfile,
+- .has_key_password = has_key_password,
+- .key_password = key_password,
+- .has_fingerprint = has_fingerprint,
+- .fingerprint = fingerprint,
+- .has_backup_id = has_backup_id,
+- .backup_id = backup_id,
+- .has_backup_time = has_backup_time,
+- .backup_time = backup_time,
+- .has_use_dirty_bitmap = has_use_dirty_bitmap,
+- .use_dirty_bitmap = use_dirty_bitmap,
+- .has_compress = has_compress,
+- .compress = compress,
+- .has_encrypt = has_encrypt,
+- .encrypt = encrypt,
+- .has_format = has_format,
+- .format = format,
+- .has_config_file = has_config_file,
+- .config_file = config_file,
+- .has_firewall_file = has_firewall_file,
+- .firewall_file = firewall_file,
+- .has_devlist = has_devlist,
+- .devlist = devlist,
+- .has_speed = has_speed,
+- .speed = speed,
+- .errp = errp,
+- };
+-
+- block_on_coroutine_fn(pvebackup_co_prepare, &task);
+-
+- return task.result;
++ return NULL;
+ }
+
+ BackupStatus *qmp_query_backup(Error **errp)
+diff --git a/qapi/block-core.json b/qapi/block-core.json
+index dee3d87efe..82133e2bee 100644
+--- a/qapi/block-core.json
++++ b/qapi/block-core.json
+@@ -847,7 +847,7 @@
+ '*config-file': 'str',
+ '*firewall-file': 'str',
+ '*devlist': 'str', '*speed': 'int' },
+- 'returns': 'UuidInfo' }
++ 'returns': 'UuidInfo', 'coroutine': true }
+
+ ##
+ # @query-backup:
+@@ -869,7 +869,7 @@
+ # Notes: This command succeeds even if there is no backup process running.
+ #
+ ##
+-{ 'command': 'backup-cancel' }
++{ 'command': 'backup-cancel', 'coroutine': true }
+
+ ##
+ # @ProxmoxSupportStatus:
diff --git a/debian/patches/series b/debian/patches/series
index 1ef7185..40bef93 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -1,6 +1,6 @@
-extra/0001-block-block-copy-always-align-copied-region-to-clust.patch
-extra/0002-Revert-qemu-img-convert-Don-t-pre-zero-images.patch
-extra/0003-usb-fix-setup_len-init-CVE-2020-14364.patch
+extra/0001-Revert-qemu-img-convert-Don-t-pre-zero-images.patch
+extra/0002-docs-don-t-install-man-page-if-guest-agent-is-disabl.patch
+extra/0003-migration-only-check-page-size-match-if-RAM-postcopy.patch
pve/0001-PVE-Config-block-file-change-locking-default-to-off.patch
pve/0002-PVE-Config-Adjust-network-script-path-to-etc-kvm.patch
pve/0003-PVE-Config-set-the-CPU-model-to-kvm64-32-instead-of-.patch
@@ -17,8 +17,8 @@ pve/0013-PVE-Up-qemu-img-dd-add-n-skip_create.patch
pve/0014-PVE-virtio-balloon-improve-query-balloon.patch
pve/0015-PVE-qapi-modify-query-machines.patch
pve/0016-PVE-qapi-modify-spice-query.patch
-pve/0017-PVE-internal-snapshot-async.patch
-pve/0018-add-optional-buffer-size-to-QEMUFile.patch
+pve/0017-PVE-add-savevm-async-for-background-state-snapshots.patch
+pve/0018-PVE-add-optional-buffer-size-to-QEMUFile.patch
pve/0019-PVE-block-add-the-zeroinit-block-driver-filter.patch
pve/0020-PVE-Add-dummy-id-command-line-parameter.patch
pve/0021-PVE-Config-Revert-target-i386-disable-LINT0-after-re.patch
@@ -54,8 +54,8 @@ pve/0050-PVE-Add-sequential-job-transaction-support.patch
pve/0051-PVE-Backup-Use-a-transaction-to-synchronize-job-stat.patch
pve/0052-PVE-Backup-Use-more-coroutines-and-don-t-block-on-fi.patch
pve/0053-PVE-fix-and-clean-up-error-handling-for-create_backu.patch
-pve/0054-migration-block-dirty-bitmap-fix-larger-granularity-.patch
-pve/0055-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
-pve/0056-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch
-pve/0057-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch
-pve/0058-PVE-fall-back-to-open-iscsi-initiatorname.patch
+pve/0054-PVE-Migrate-dirty-bitmap-state-via-savevm.patch
+pve/0055-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch
+pve/0056-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch
+pve/0057-PVE-fall-back-to-open-iscsi-initiatorname.patch
+pve/0058-PVE-Use-coroutine-QMP-for-backup-cancel_backup.patch
diff --git a/debian/pve-qemu-kvm.install b/debian/pve-qemu-kvm.install
index 3ac471a..723d548 100644
--- a/debian/pve-qemu-kvm.install
+++ b/debian/pve-qemu-kvm.install
@@ -1,6 +1,4 @@
# install the userspace utilities
-vma usr/bin/
-pbs-restore usr/bin/
debian/kvm-ifup etc/kvm/
debian/kvm-ifdown etc/kvm/
diff --git a/debian/rules b/debian/rules
index 57e1c91..9e72986 100755
--- a/debian/rules
+++ b/debian/rules
@@ -23,6 +23,9 @@ destdir := $(CURDIR)/debian/$(PACKAGE)
flagfile := $(destdir)/usr/share/kvm/recognized-CPUID-flags-x86_64
+# default QEMU out-of-tree build directory is ./build
+BUILDDIR=build
+
CFLAGS = -Wall
ifneq (,$(findstring noopt,$(DEB_BUILD_OPTIONS)))
@@ -31,7 +34,7 @@ else
CFLAGS += -O2
endif
-config.status: configure
+${BUILDDIR}/config.status: configure
dh_testdir
# Add here commands to configure the package.
@@ -42,7 +45,7 @@ config.status: configure
--prefix=/usr \
--sysconfdir=/etc \
--target-list=$(ARCH)-softmmu,aarch64-softmmu \
- --with-confsuffix="/kvm" \
+ --with-suffix="kvm" \
--with-pkgversion="${DEB_SOURCE}_${DEB_VERSION_UPSTREAM}" \
--audio-drv-list="alsa" \
--datadir=/usr/share \
@@ -73,7 +76,7 @@ config.status: configure
build: build-stamp
-build-stamp: config.status
+build-stamp: ${BUILDDIR}/config.status
dh_testdir
# Add here commands to compile the package.
@@ -121,6 +124,9 @@ install: build
rm $(destdir)/usr/share/kvm/u-boot.e500
# remove Aplha files
rm $(destdir)/usr/share/kvm/palcode-clipper
+ # remove RISC-V files
+ rm $(destdir)/usr/share/kvm/opensbi-riscv32-generic-fw_dynamic.elf
+ rm $(destdir)/usr/share/kvm/opensbi-riscv64-generic-fw_dynamic.elf
# Remove things we don't package at all, would be a "kvm-dev" package
rm -Rf $(destdir)/usr/include/linux/
diff --git a/qemu b/qemu
index d0ed6a6..553032d 160000
--- a/qemu
+++ b/qemu
@@ -1 +1 @@
-Subproject commit d0ed6a69d399ae193959225cdeaa9382746c91cc
+Subproject commit 553032db17440f8de011390e5a1cfddd13751b0b
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [pve-devel] [PATCH v2 pve-qemu 2/5] add PBS master key support
2021-02-11 16:11 [pve-devel] [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2 Stefan Reiter
@ 2021-02-11 16:11 ` Stefan Reiter
2021-02-12 14:35 ` [pve-devel] applied: " Thomas Lamprecht
2021-02-11 16:11 ` [pve-devel] [PATCH INFO v2 qemu 3/5] PVE: Use coroutine QMP for backup/cancel_backup Stefan Reiter
` (3 subsequent siblings)
4 siblings, 1 reply; 7+ messages in thread
From: Stefan Reiter @ 2021-02-11 16:11 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
Rebased from Fabian's series. Thus of course also needs the updated version of
libproxmox-backup-qemu to build and run.
See: https://lists.proxmox.com/pipermail/pve-devel/2021-February/046945.html
.../pve/0059-PBS-add-master-key-support.patch | 97 +++++++++++++++++++
debian/patches/series | 1 +
2 files changed, 98 insertions(+)
create mode 100644 debian/patches/pve/0059-PBS-add-master-key-support.patch
diff --git a/debian/patches/pve/0059-PBS-add-master-key-support.patch b/debian/patches/pve/0059-PBS-add-master-key-support.patch
new file mode 100644
index 0000000..52e600c
--- /dev/null
+++ b/debian/patches/pve/0059-PBS-add-master-key-support.patch
@@ -0,0 +1,97 @@
+From 0000000000000000000000000000000000000000 Mon Sep 17 00:00:00 2001
+From: Stefan Reiter <s.reiter@proxmox.com>
+Date: Wed, 10 Feb 2021 11:07:06 +0100
+Subject: [PATCH] PBS: add master key support
+MIME-Version: 1.0
+Content-Type: text/plain; charset=UTF-8
+Content-Transfer-Encoding: 8bit
+
+this requires a new enough libproxmox-backup-qemu0, and allows querying
+from the PVE side to avoid QMP calls with unsupported parameters.
+
+Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
+Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
+---
+ block/monitor/block-hmp-cmds.c | 1 +
+ pve-backup.c | 3 +++
+ qapi/block-core.json | 7 +++++++
+ 3 files changed, 11 insertions(+)
+
+diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
+index 11c84d5508..0932deb28c 100644
+--- a/block/monitor/block-hmp-cmds.c
++++ b/block/monitor/block-hmp-cmds.c
+@@ -1036,6 +1036,7 @@ void coroutine_fn hmp_backup(Monitor *mon, const QDict *qdict)
+ false, NULL, // PBS password
+ false, NULL, // PBS keyfile
+ false, NULL, // PBS key_password
++ false, NULL, // PBS master_keyfile
+ false, NULL, // PBS fingerprint
+ false, NULL, // PBS backup-id
+ false, 0, // PBS backup-time
+diff --git a/pve-backup.c b/pve-backup.c
+index f7597ae55c..0ecadf6ce6 100644
+--- a/pve-backup.c
++++ b/pve-backup.c
+@@ -531,6 +531,7 @@ UuidInfo coroutine_fn *qmp_backup(
+ bool has_password, const char *password,
+ bool has_keyfile, const char *keyfile,
+ bool has_key_password, const char *key_password,
++ bool has_master_keyfile, const char *master_keyfile,
+ bool has_fingerprint, const char *fingerprint,
+ bool has_backup_id, const char *backup_id,
+ bool has_backup_time, int64_t backup_time,
+@@ -679,6 +680,7 @@ UuidInfo coroutine_fn *qmp_backup(
+ has_password ? password : NULL,
+ has_keyfile ? keyfile : NULL,
+ has_key_password ? key_password : NULL,
++ has_master_keyfile ? master_keyfile : NULL,
+ has_compress ? compress : true,
+ has_encrypt ? encrypt : has_keyfile,
+ has_fingerprint ? fingerprint : NULL,
+@@ -1041,5 +1043,6 @@ ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp)
+ ret->pbs_dirty_bitmap = true;
+ ret->query_bitmap_info = true;
+ ret->pbs_dirty_bitmap_migration = true;
++ ret->pbs_masterkey = true;
+ return ret;
+ }
+diff --git a/qapi/block-core.json b/qapi/block-core.json
+index 82133e2bee..be3d6a0d37 100644
+--- a/qapi/block-core.json
++++ b/qapi/block-core.json
+@@ -818,6 +818,8 @@
+ #
+ # @key-password: password for keyfile (optional for format 'pbs')
+ #
++# @master-keyfile: PEM-formatted master public keyfile (optional for format 'pbs')
++#
+ # @fingerprint: server cert fingerprint (optional for format 'pbs')
+ #
+ # @backup-id: backup ID (required for format 'pbs')
+@@ -837,6 +839,7 @@
+ '*password': 'str',
+ '*keyfile': 'str',
+ '*key-password': 'str',
++ '*master-keyfile': 'str',
+ '*fingerprint': 'str',
+ '*backup-id': 'str',
+ '*backup-time': 'int',
+@@ -886,6 +889,9 @@
+ # migration cap if this is false/unset may lead
+ # to crashes on migration!
+ #
++# @pbs-masterkey: True if the QMP backup call supports the 'master_keyfile'
++# parameter.
++#
+ # @pbs-library-version: Running version of libproxmox-backup-qemu0 library.
+ #
+ ##
+@@ -893,6 +899,7 @@
+ 'data': { 'pbs-dirty-bitmap': 'bool',
+ 'query-bitmap-info': 'bool',
+ 'pbs-dirty-bitmap-migration': 'bool',
++ 'pbs-masterkey': 'bool',
+ 'pbs-library-version': 'str' } }
+
+ ##
diff --git a/debian/patches/series b/debian/patches/series
index 40bef93..60567e8 100644
--- a/debian/patches/series
+++ b/debian/patches/series
@@ -59,3 +59,4 @@ pve/0055-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch
pve/0056-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch
pve/0057-PVE-fall-back-to-open-iscsi-initiatorname.patch
pve/0058-PVE-Use-coroutine-QMP-for-backup-cancel_backup.patch
+pve/0059-PBS-add-master-key-support.patch
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [pve-devel] [PATCH INFO v2 qemu 3/5] PVE: Use coroutine QMP for backup/cancel_backup
2021-02-11 16:11 [pve-devel] [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2 Stefan Reiter
2021-02-11 16:11 ` [pve-devel] [PATCH v2 pve-qemu 2/5] add PBS master key support Stefan Reiter
@ 2021-02-11 16:11 ` Stefan Reiter
2021-02-11 16:11 ` [pve-devel] [PATCH INFO v2 qemu 4/5] PVE: increase max IOV count in QEMUFile Stefan Reiter
` (2 subsequent siblings)
4 siblings, 0 replies; 7+ messages in thread
From: Stefan Reiter @ 2021-02-11 16:11 UTC (permalink / raw)
To: pve-devel
Finally turn backup QMP calls into coroutines, now that it's possible.
This has the benefit that calls are asynchronous to the main loop, i.e.
long running operations like connecting to a PBS server will no longer
hang the VM.
Additionally, it allows us to get rid of block_on_coroutine_fn, which
was always a hacky workaround.
While we're already spring cleaning, also remove the QmpBackupTask
struct, since we can now put the 'prepare' function directly into
qmp_backup and thus no longer need those giant walls of text.
(Note that for our patches to work with 5.2.0 this change is actually
required, otherwise monitor_get_fd() fails as we're not in a QMP
coroutine, but one we start ourselves - we could of course set the
monitor for that coroutine ourselves, but let's just fix it the right
way instead)
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
Included in first patch.
No change from v1.
block/monitor/block-hmp-cmds.c | 4 +-
hmp-commands.hx | 2 +
proxmox-backup-client.c | 31 -----
pve-backup.c | 232 ++++++++++-----------------------
qapi/block-core.json | 4 +-
5 files changed, 77 insertions(+), 196 deletions(-)
diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c
index 46c63b1cf9..11c84d5508 100644
--- a/block/monitor/block-hmp-cmds.c
+++ b/block/monitor/block-hmp-cmds.c
@@ -1013,7 +1013,7 @@ void hmp_info_snapshots(Monitor *mon, const QDict *qdict)
g_free(global_snapshots);
}
-void hmp_backup_cancel(Monitor *mon, const QDict *qdict)
+void coroutine_fn hmp_backup_cancel(Monitor *mon, const QDict *qdict)
{
Error *error = NULL;
@@ -1022,7 +1022,7 @@ void hmp_backup_cancel(Monitor *mon, const QDict *qdict)
hmp_handle_error(mon, error);
}
-void hmp_backup(Monitor *mon, const QDict *qdict)
+void coroutine_fn hmp_backup(Monitor *mon, const QDict *qdict)
{
Error *error = NULL;
diff --git a/hmp-commands.hx b/hmp-commands.hx
index 70257ef7ca..b7639d189d 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -108,6 +108,7 @@ ERST
"\n\t\t\t Use -d to dump data into a directory instead"
"\n\t\t\t of using VMA format.",
.cmd = hmp_backup,
+ .coroutine = true,
},
SRST
@@ -121,6 +122,7 @@ ERST
.params = "",
.help = "cancel the current VM backup",
.cmd = hmp_backup_cancel,
+ .coroutine = true,
},
SRST
diff --git a/proxmox-backup-client.c b/proxmox-backup-client.c
index 4ce7bc0b5e..0923037dec 100644
--- a/proxmox-backup-client.c
+++ b/proxmox-backup-client.c
@@ -5,37 +5,6 @@
/* Proxmox Backup Server client bindings using coroutines */
-typedef struct BlockOnCoroutineWrapper {
- AioContext *ctx;
- CoroutineEntry *entry;
- void *entry_arg;
- bool finished;
-} BlockOnCoroutineWrapper;
-
-static void coroutine_fn block_on_coroutine_wrapper(void *opaque)
-{
- BlockOnCoroutineWrapper *wrapper = opaque;
- wrapper->entry(wrapper->entry_arg);
- wrapper->finished = true;
- aio_wait_kick();
-}
-
-void block_on_coroutine_fn(CoroutineEntry *entry, void *entry_arg)
-{
- assert(!qemu_in_coroutine());
-
- AioContext *ctx = qemu_get_current_aio_context();
- BlockOnCoroutineWrapper wrapper = {
- .finished = false,
- .entry = entry,
- .entry_arg = entry_arg,
- .ctx = ctx,
- };
- Coroutine *wrapper_co = qemu_coroutine_create(block_on_coroutine_wrapper, &wrapper);
- aio_co_enter(ctx, wrapper_co);
- AIO_WAIT_WHILE(ctx, !wrapper.finished);
-}
-
// This is called from another thread, so we use aio_co_schedule()
static void proxmox_backup_schedule_wake(void *data) {
CoCtxData *waker = (CoCtxData *)data;
diff --git a/pve-backup.c b/pve-backup.c
index 6b25292ba1..f7597ae55c 100644
--- a/pve-backup.c
+++ b/pve-backup.c
@@ -357,7 +357,7 @@ static void job_cancel_bh(void *opaque) {
aio_co_enter(data->ctx, data->co);
}
-static void coroutine_fn pvebackup_co_cancel(void *opaque)
+void coroutine_fn qmp_backup_cancel(Error **errp)
{
Error *cancel_err = NULL;
error_setg(&cancel_err, "backup canceled");
@@ -394,11 +394,6 @@ static void coroutine_fn pvebackup_co_cancel(void *opaque)
qemu_co_mutex_unlock(&backup_state.backup_mutex);
}
-void qmp_backup_cancel(Error **errp)
-{
- block_on_coroutine_fn(pvebackup_co_cancel, NULL);
-}
-
// assumes the caller holds backup_mutex
static int coroutine_fn pvebackup_co_add_config(
const char *file,
@@ -531,50 +526,27 @@ static void create_backup_jobs_bh(void *opaque) {
aio_co_enter(data->ctx, data->co);
}
-typedef struct QmpBackupTask {
- const char *backup_file;
- bool has_password;
- const char *password;
- bool has_keyfile;
- const char *keyfile;
- bool has_key_password;
- const char *key_password;
- bool has_backup_id;
- const char *backup_id;
- bool has_backup_time;
- const char *fingerprint;
- bool has_fingerprint;
- int64_t backup_time;
- bool has_use_dirty_bitmap;
- bool use_dirty_bitmap;
- bool has_format;
- BackupFormat format;
- bool has_config_file;
- const char *config_file;
- bool has_firewall_file;
- const char *firewall_file;
- bool has_devlist;
- const char *devlist;
- bool has_compress;
- bool compress;
- bool has_encrypt;
- bool encrypt;
- bool has_speed;
- int64_t speed;
- Error **errp;
- UuidInfo *result;
-} QmpBackupTask;
-
-static void coroutine_fn pvebackup_co_prepare(void *opaque)
+UuidInfo coroutine_fn *qmp_backup(
+ const char *backup_file,
+ bool has_password, const char *password,
+ bool has_keyfile, const char *keyfile,
+ bool has_key_password, const char *key_password,
+ bool has_fingerprint, const char *fingerprint,
+ bool has_backup_id, const char *backup_id,
+ bool has_backup_time, int64_t backup_time,
+ bool has_use_dirty_bitmap, bool use_dirty_bitmap,
+ bool has_compress, bool compress,
+ bool has_encrypt, bool encrypt,
+ bool has_format, BackupFormat format,
+ bool has_config_file, const char *config_file,
+ bool has_firewall_file, const char *firewall_file,
+ bool has_devlist, const char *devlist,
+ bool has_speed, int64_t speed, Error **errp)
{
assert(qemu_in_coroutine());
qemu_co_mutex_lock(&backup_state.backup_mutex);
- QmpBackupTask *task = opaque;
-
- task->result = NULL; // just to be sure
-
BlockBackend *blk;
BlockDriverState *bs = NULL;
const char *backup_dir = NULL;
@@ -591,17 +563,17 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
const char *firewall_name = "qemu-server.fw";
if (backup_state.di_list) {
- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
"previous backup not finished");
qemu_co_mutex_unlock(&backup_state.backup_mutex);
- return;
+ return NULL;
}
/* Todo: try to auto-detect format based on file name */
- BackupFormat format = task->has_format ? task->format : BACKUP_FORMAT_VMA;
+ format = has_format ? format : BACKUP_FORMAT_VMA;
- if (task->has_devlist) {
- devs = g_strsplit_set(task->devlist, ",;:", -1);
+ if (has_devlist) {
+ devs = g_strsplit_set(devlist, ",;:", -1);
gchar **d = devs;
while (d && *d) {
@@ -609,14 +581,14 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
if (blk) {
bs = blk_bs(blk);
if (!bdrv_is_inserted(bs)) {
- error_setg(task->errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
+ error_setg(errp, QERR_DEVICE_HAS_NO_MEDIUM, *d);
goto err;
}
PVEBackupDevInfo *di = g_new0(PVEBackupDevInfo, 1);
di->bs = bs;
di_list = g_list_append(di_list, di);
} else {
- error_set(task->errp, ERROR_CLASS_DEVICE_NOT_FOUND,
+ error_set(errp, ERROR_CLASS_DEVICE_NOT_FOUND,
"Device '%s' not found", *d);
goto err;
}
@@ -639,7 +611,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
}
if (!di_list) {
- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "empty device list");
goto err;
}
@@ -649,13 +621,13 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
while (l) {
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
- if (bdrv_op_is_blocked(di->bs, BLOCK_OP_TYPE_BACKUP_SOURCE, task->errp)) {
+ if (bdrv_op_is_blocked(di->bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
goto err;
}
ssize_t size = bdrv_getlength(di->bs);
if (size < 0) {
- error_setg_errno(task->errp, -di->size, "bdrv_getlength failed");
+ error_setg_errno(errp, -di->size, "bdrv_getlength failed");
goto err;
}
di->size = size;
@@ -682,47 +654,44 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
}
if (format == BACKUP_FORMAT_PBS) {
- if (!task->has_password) {
- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'password'");
+ if (!has_password) {
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'password'");
goto err_mutex;
}
- if (!task->has_backup_id) {
- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-id'");
+ if (!has_backup_id) {
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-id'");
goto err_mutex;
}
- if (!task->has_backup_time) {
- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-time'");
+ if (!has_backup_time) {
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "missing parameter 'backup-time'");
goto err_mutex;
}
int dump_cb_block_size = PROXMOX_BACKUP_DEFAULT_CHUNK_SIZE; // Hardcoded (4M)
firewall_name = "fw.conf";
- bool use_dirty_bitmap = task->has_use_dirty_bitmap && task->use_dirty_bitmap;
-
-
char *pbs_err = NULL;
pbs = proxmox_backup_new(
- task->backup_file,
- task->backup_id,
- task->backup_time,
+ backup_file,
+ backup_id,
+ backup_time,
dump_cb_block_size,
- task->has_password ? task->password : NULL,
- task->has_keyfile ? task->keyfile : NULL,
- task->has_key_password ? task->key_password : NULL,
- task->has_compress ? task->compress : true,
- task->has_encrypt ? task->encrypt : task->has_keyfile,
- task->has_fingerprint ? task->fingerprint : NULL,
+ has_password ? password : NULL,
+ has_keyfile ? keyfile : NULL,
+ has_key_password ? key_password : NULL,
+ has_compress ? compress : true,
+ has_encrypt ? encrypt : has_keyfile,
+ has_fingerprint ? fingerprint : NULL,
&pbs_err);
if (!pbs) {
- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
"proxmox_backup_new failed: %s", pbs_err);
proxmox_backup_free_error(pbs_err);
goto err_mutex;
}
- int connect_result = proxmox_backup_co_connect(pbs, task->errp);
+ int connect_result = proxmox_backup_co_connect(pbs, errp);
if (connect_result < 0)
goto err_mutex;
@@ -741,9 +710,9 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
BdrvDirtyBitmap *bitmap = bdrv_find_dirty_bitmap(di->bs, PBS_BITMAP_NAME);
bool expect_only_dirty = false;
- if (use_dirty_bitmap) {
+ if (has_use_dirty_bitmap && use_dirty_bitmap) {
if (bitmap == NULL) {
- bitmap = bdrv_create_dirty_bitmap(di->bs, dump_cb_block_size, PBS_BITMAP_NAME, task->errp);
+ bitmap = bdrv_create_dirty_bitmap(di->bs, dump_cb_block_size, PBS_BITMAP_NAME, errp);
if (!bitmap) {
goto err_mutex;
}
@@ -773,12 +742,12 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
}
}
- int dev_id = proxmox_backup_co_register_image(pbs, devname, di->size, expect_only_dirty, task->errp);
+ int dev_id = proxmox_backup_co_register_image(pbs, devname, di->size, expect_only_dirty, errp);
if (dev_id < 0) {
goto err_mutex;
}
- if (!(di->target = bdrv_backup_dump_create(dump_cb_block_size, di->size, pvebackup_co_dump_pbs_cb, di, task->errp))) {
+ if (!(di->target = bdrv_backup_dump_create(dump_cb_block_size, di->size, pvebackup_co_dump_pbs_cb, di, errp))) {
goto err_mutex;
}
@@ -792,10 +761,10 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
backup_state.stat.bitmap_list = g_list_append(backup_state.stat.bitmap_list, info);
}
} else if (format == BACKUP_FORMAT_VMA) {
- vmaw = vma_writer_create(task->backup_file, uuid, &local_err);
+ vmaw = vma_writer_create(backup_file, uuid, &local_err);
if (!vmaw) {
if (local_err) {
- error_propagate(task->errp, local_err);
+ error_propagate(errp, local_err);
}
goto err_mutex;
}
@@ -806,25 +775,25 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data;
l = g_list_next(l);
- if (!(di->target = bdrv_backup_dump_create(VMA_CLUSTER_SIZE, di->size, pvebackup_co_dump_vma_cb, di, task->errp))) {
+ if (!(di->target = bdrv_backup_dump_create(VMA_CLUSTER_SIZE, di->size, pvebackup_co_dump_vma_cb, di, errp))) {
goto err_mutex;
}
const char *devname = bdrv_get_device_name(di->bs);
di->dev_id = vma_writer_register_stream(vmaw, devname, di->size);
if (di->dev_id <= 0) {
- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR,
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR,
"register_stream failed");
goto err_mutex;
}
}
} else if (format == BACKUP_FORMAT_DIR) {
- if (mkdir(task->backup_file, 0640) != 0) {
- error_setg_errno(task->errp, errno, "can't create directory '%s'\n",
- task->backup_file);
+ if (mkdir(backup_file, 0640) != 0) {
+ error_setg_errno(errp, errno, "can't create directory '%s'\n",
+ backup_file);
goto err_mutex;
}
- backup_dir = task->backup_file;
+ backup_dir = backup_file;
l = di_list;
while (l) {
@@ -838,34 +807,34 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
bdrv_img_create(di->targetfile, "raw", NULL, NULL, NULL,
di->size, flags, false, &local_err);
if (local_err) {
- error_propagate(task->errp, local_err);
+ error_propagate(errp, local_err);
goto err_mutex;
}
di->target = bdrv_open(di->targetfile, NULL, NULL, flags, &local_err);
if (!di->target) {
- error_propagate(task->errp, local_err);
+ error_propagate(errp, local_err);
goto err_mutex;
}
}
} else {
- error_set(task->errp, ERROR_CLASS_GENERIC_ERROR, "unknown backup format");
+ error_set(errp, ERROR_CLASS_GENERIC_ERROR, "unknown backup format");
goto err_mutex;
}
/* add configuration file to archive */
- if (task->has_config_file) {
- if (pvebackup_co_add_config(task->config_file, config_name, format, backup_dir,
- vmaw, pbs, task->errp) != 0) {
+ if (has_config_file) {
+ if (pvebackup_co_add_config(config_file, config_name, format, backup_dir,
+ vmaw, pbs, errp) != 0) {
goto err_mutex;
}
}
/* add firewall file to archive */
- if (task->has_firewall_file) {
- if (pvebackup_co_add_config(task->firewall_file, firewall_name, format, backup_dir,
- vmaw, pbs, task->errp) != 0) {
+ if (has_firewall_file) {
+ if (pvebackup_co_add_config(firewall_file, firewall_name, format, backup_dir,
+ vmaw, pbs, errp) != 0) {
goto err_mutex;
}
}
@@ -883,7 +852,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
if (backup_state.stat.backup_file) {
g_free(backup_state.stat.backup_file);
}
- backup_state.stat.backup_file = g_strdup(task->backup_file);
+ backup_state.stat.backup_file = g_strdup(backup_file);
uuid_copy(backup_state.stat.uuid, uuid);
uuid_unparse_lower(uuid, backup_state.stat.uuid_str);
@@ -898,7 +867,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
qemu_mutex_unlock(&backup_state.stat.lock);
- backup_state.speed = (task->has_speed && task->speed > 0) ? task->speed : 0;
+ backup_state.speed = (has_speed && speed > 0) ? speed : 0;
backup_state.vmaw = vmaw;
backup_state.pbs = pbs;
@@ -908,8 +877,6 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
uuid_info = g_malloc0(sizeof(*uuid_info));
uuid_info->UUID = uuid_str;
- task->result = uuid_info;
-
/* Run create_backup_jobs_bh outside of coroutine (in BH) but keep
* backup_mutex locked. This is fine, a CoMutex can be held across yield
* points, and we'll release it as soon as the BH reschedules us.
@@ -923,7 +890,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
qemu_coroutine_yield();
if (local_err) {
- error_propagate(task->errp, local_err);
+ error_propagate(errp, local_err);
goto err;
}
@@ -936,7 +903,7 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque)
/* start the first job in the transaction */
job_txn_start_seq(backup_state.txn);
- return;
+ return uuid_info;
err_mutex:
qemu_mutex_unlock(&backup_state.stat.lock);
@@ -967,7 +934,7 @@ err:
if (vmaw) {
Error *err = NULL;
vma_writer_close(vmaw, &err);
- unlink(task->backup_file);
+ unlink(backup_file);
}
if (pbs) {
@@ -978,65 +945,8 @@ err:
rmdir(backup_dir);
}
- task->result = NULL;
-
qemu_co_mutex_unlock(&backup_state.backup_mutex);
- return;
-}
-
-UuidInfo *qmp_backup(
- const char *backup_file,
- bool has_password, const char *password,
- bool has_keyfile, const char *keyfile,
- bool has_key_password, const char *key_password,
- bool has_fingerprint, const char *fingerprint,
- bool has_backup_id, const char *backup_id,
- bool has_backup_time, int64_t backup_time,
- bool has_use_dirty_bitmap, bool use_dirty_bitmap,
- bool has_compress, bool compress,
- bool has_encrypt, bool encrypt,
- bool has_format, BackupFormat format,
- bool has_config_file, const char *config_file,
- bool has_firewall_file, const char *firewall_file,
- bool has_devlist, const char *devlist,
- bool has_speed, int64_t speed, Error **errp)
-{
- QmpBackupTask task = {
- .backup_file = backup_file,
- .has_password = has_password,
- .password = password,
- .has_keyfile = has_keyfile,
- .keyfile = keyfile,
- .has_key_password = has_key_password,
- .key_password = key_password,
- .has_fingerprint = has_fingerprint,
- .fingerprint = fingerprint,
- .has_backup_id = has_backup_id,
- .backup_id = backup_id,
- .has_backup_time = has_backup_time,
- .backup_time = backup_time,
- .has_use_dirty_bitmap = has_use_dirty_bitmap,
- .use_dirty_bitmap = use_dirty_bitmap,
- .has_compress = has_compress,
- .compress = compress,
- .has_encrypt = has_encrypt,
- .encrypt = encrypt,
- .has_format = has_format,
- .format = format,
- .has_config_file = has_config_file,
- .config_file = config_file,
- .has_firewall_file = has_firewall_file,
- .firewall_file = firewall_file,
- .has_devlist = has_devlist,
- .devlist = devlist,
- .has_speed = has_speed,
- .speed = speed,
- .errp = errp,
- };
-
- block_on_coroutine_fn(pvebackup_co_prepare, &task);
-
- return task.result;
+ return NULL;
}
BackupStatus *qmp_query_backup(Error **errp)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index dee3d87efe..82133e2bee 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -847,7 +847,7 @@
'*config-file': 'str',
'*firewall-file': 'str',
'*devlist': 'str', '*speed': 'int' },
- 'returns': 'UuidInfo' }
+ 'returns': 'UuidInfo', 'coroutine': true }
##
# @query-backup:
@@ -869,7 +869,7 @@
# Notes: This command succeeds even if there is no backup process running.
#
##
-{ 'command': 'backup-cancel' }
+{ 'command': 'backup-cancel', 'coroutine': true }
##
# @ProxmoxSupportStatus:
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [pve-devel] [PATCH INFO v2 qemu 4/5] PVE: increase max IOV count in QEMUFile
2021-02-11 16:11 [pve-devel] [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2 Stefan Reiter
2021-02-11 16:11 ` [pve-devel] [PATCH v2 pve-qemu 2/5] add PBS master key support Stefan Reiter
2021-02-11 16:11 ` [pve-devel] [PATCH INFO v2 qemu 3/5] PVE: Use coroutine QMP for backup/cancel_backup Stefan Reiter
@ 2021-02-11 16:11 ` Stefan Reiter
2021-02-11 16:11 ` [pve-devel] [PATCH INFO v2 qemu 5/5] PVE: savevm-async: improve snapshot abort Stefan Reiter
2021-02-12 14:35 ` [pve-devel] applied: [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2 Thomas Lamprecht
4 siblings, 0 replies; 7+ messages in thread
From: Stefan Reiter @ 2021-02-11 16:11 UTC (permalink / raw)
To: pve-devel
might help with RBD performance for snapshots, since it writes larger
data buffers at once
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
Included in first patch, squashed into existing patches.
New in v2.
migration/qemu-file.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index a4d2e2c8ff..1926b5202c 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -31,7 +31,7 @@
#include "qapi/error.h"
#define DEFAULT_IO_BUF_SIZE 32768
-#define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 64)
+#define MAX_IOV_SIZE MIN_CONST(IOV_MAX, 256)
struct QEMUFile {
const QEMUFileOps *ops;
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [pve-devel] [PATCH INFO v2 qemu 5/5] PVE: savevm-async: improve snapshot abort
2021-02-11 16:11 [pve-devel] [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2 Stefan Reiter
` (2 preceding siblings ...)
2021-02-11 16:11 ` [pve-devel] [PATCH INFO v2 qemu 4/5] PVE: increase max IOV count in QEMUFile Stefan Reiter
@ 2021-02-11 16:11 ` Stefan Reiter
2021-02-12 14:35 ` [pve-devel] applied: [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2 Thomas Lamprecht
4 siblings, 0 replies; 7+ messages in thread
From: Stefan Reiter @ 2021-02-11 16:11 UTC (permalink / raw)
To: pve-devel
Do not block the VM and save the state on aborting a snapshot, as the
snapshot will be invalid anyway.
Also, when aborting, wait for the target file to be closed, otherwise a
client might run into race-conditions when trying to remove the file
still opened by QEMU.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
Included in first patch, squashed into existing patches.
New in v2.
hmp-commands.hx | 3 +-
migration/savevm-async.c | 75 +++++++++++++++++++++++++++++++++-------
monitor/hmp-cmds.c | 2 +-
qapi/misc.json | 2 +-
4 files changed, 66 insertions(+), 16 deletions(-)
diff --git a/hmp-commands.hx b/hmp-commands.hx
index b7639d189d..54de3f80e6 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1927,5 +1927,6 @@ ERST
.args_type = "",
.params = "",
.help = "Resume VM after snaphot.",
- .cmd = hmp_savevm_end,
+ .cmd = hmp_savevm_end,
+ .coroutine = true,
},
diff --git a/migration/savevm-async.c b/migration/savevm-async.c
index 156b7a030e..8a17ec1f74 100644
--- a/migration/savevm-async.c
+++ b/migration/savevm-async.c
@@ -16,6 +16,7 @@
#include "qapi/qapi-commands-misc.h"
#include "qapi/qapi-commands-block.h"
#include "qemu/cutils.h"
+#include "qemu/timer.h"
#include "qemu/main-loop.h"
#include "qemu/rcu.h"
@@ -52,8 +53,15 @@ static struct SnapshotState {
int64_t total_time;
QEMUBH *finalize_bh;
Coroutine *co;
+ QemuCoSleepState *target_close_wait;
} snap_state;
+static bool savevm_aborted(void)
+{
+ return snap_state.state == SAVE_STATE_CANCELLED ||
+ snap_state.state == SAVE_STATE_ERROR;
+}
+
SaveVMInfo *qmp_query_savevm(Error **errp)
{
SaveVMInfo *info = g_malloc0(sizeof(*info));
@@ -106,17 +114,23 @@ static int save_snapshot_cleanup(void)
}
if (snap_state.target) {
- /* try to truncate, but ignore errors (will fail on block devices).
- * note1: bdrv_read() need whole blocks, so we need to round up
- * note2: PVE requires 1024 (BDRV_SECTOR_SIZE*2) alignment
- */
- size_t size = QEMU_ALIGN_UP(snap_state.bs_pos, BDRV_SECTOR_SIZE*2);
- blk_truncate(snap_state.target, size, false, PREALLOC_MODE_OFF, 0, NULL);
+ if (!savevm_aborted()) {
+ /* try to truncate, but ignore errors (will fail on block devices).
+ * note1: bdrv_read() need whole blocks, so we need to round up
+ * note2: PVE requires 1024 (BDRV_SECTOR_SIZE*2) alignment
+ */
+ size_t size = QEMU_ALIGN_UP(snap_state.bs_pos, BDRV_SECTOR_SIZE*2);
+ blk_truncate(snap_state.target, size, false, PREALLOC_MODE_OFF, 0, NULL);
+ }
blk_op_unblock_all(snap_state.target, snap_state.blocker);
error_free(snap_state.blocker);
snap_state.blocker = NULL;
blk_unref(snap_state.target);
snap_state.target = NULL;
+
+ if (snap_state.target_close_wait) {
+ qemu_co_sleep_wake(snap_state.target_close_wait);
+ }
}
return ret;
@@ -202,6 +216,8 @@ static void process_savevm_finalize(void *opaque)
AioContext *iohandler_ctx = iohandler_get_aio_context();
MigrationState *ms = migrate_get_current();
+ bool aborted = savevm_aborted();
+
#ifdef DEBUG_SAVEVM_STATE
int64_t start_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
#endif
@@ -223,10 +239,13 @@ static void process_savevm_finalize(void *opaque)
save_snapshot_error("vm_stop_force_state error %d", ret);
}
- (void)qemu_savevm_state_complete_precopy(snap_state.file, false, false);
- ret = qemu_file_get_error(snap_state.file);
- if (ret < 0) {
- save_snapshot_error("qemu_savevm_state_iterate error %d", ret);
+ if (!aborted) {
+ /* skip state saving if we aborted, snapshot will be invalid anyway */
+ (void)qemu_savevm_state_complete_precopy(snap_state.file, false, false);
+ ret = qemu_file_get_error(snap_state.file);
+ if (ret < 0) {
+ save_snapshot_error("qemu_savevm_state_iterate error %d", ret);
+ }
}
DPRINTF("state saving complete\n");
@@ -235,7 +254,7 @@ static void process_savevm_finalize(void *opaque)
/* clear migration state */
migrate_set_state(&ms->state, MIGRATION_STATUS_SETUP,
- ret ? MIGRATION_STATUS_FAILED : MIGRATION_STATUS_COMPLETED);
+ ret || aborted ? MIGRATION_STATUS_FAILED : MIGRATION_STATUS_COMPLETED);
ms->to_dst_file = NULL;
qemu_savevm_state_cleanup();
@@ -245,6 +264,9 @@ static void process_savevm_finalize(void *opaque)
save_snapshot_error("save_snapshot_cleanup error %d", ret);
} else if (snap_state.state == SAVE_STATE_ACTIVE) {
snap_state.state = SAVE_STATE_COMPLETED;
+ } else if (aborted) {
+ save_snapshot_error("process_savevm_cleanup: found aborted state: %d",
+ snap_state.state);
} else {
save_snapshot_error("process_savevm_cleanup: invalid state: %d",
snap_state.state);
@@ -434,11 +456,14 @@ restart:
if (snap_state.saved_vm_running) {
vm_start();
+ snap_state.saved_vm_running = false;
}
}
-void qmp_savevm_end(Error **errp)
+void coroutine_fn qmp_savevm_end(Error **errp)
{
+ int64_t timeout;
+
if (snap_state.state == SAVE_STATE_DONE) {
error_set(errp, ERROR_CLASS_GENERIC_ERROR,
"VM snapshot not started\n");
@@ -447,14 +472,38 @@ void qmp_savevm_end(Error **errp)
if (snap_state.state == SAVE_STATE_ACTIVE) {
snap_state.state = SAVE_STATE_CANCELLED;
- return;
+ goto wait_for_close;
}
if (snap_state.saved_vm_running) {
vm_start();
+ snap_state.saved_vm_running = false;
}
snap_state.state = SAVE_STATE_DONE;
+
+wait_for_close:
+ if (!snap_state.target) {
+ DPRINTF("savevm-end: no target file open\n");
+ return;
+ }
+
+ /* wait until cleanup is done before returning, this ensures that after this
+ * call exits the statefile will be closed and can be removed immediately */
+ DPRINTF("savevm-end: waiting for cleanup\n");
+ timeout = 30L * 1000 * 1000 * 1000;
+ qemu_co_sleep_ns_wakeable(QEMU_CLOCK_REALTIME, timeout,
+ &snap_state.target_close_wait);
+ snap_state.target_close_wait = NULL;
+ if (snap_state.target) {
+ save_snapshot_error("timeout waiting for target file close in "
+ "qmp_savevm_end");
+ /* we cannot assume the snapshot finished in this case, so leave the
+ * state alone - caller has to figure something out */
+ return;
+ }
+
+ DPRINTF("savevm-end: cleanup done\n");
}
// FIXME: Deprecated
diff --git a/monitor/hmp-cmds.c b/monitor/hmp-cmds.c
index f47fc01c10..95f4e7f5c1 100644
--- a/monitor/hmp-cmds.c
+++ b/monitor/hmp-cmds.c
@@ -2049,7 +2049,7 @@ void hmp_delete_drive_snapshot(Monitor *mon, const QDict *qdict)
hmp_handle_error(mon, errp);
}
-void hmp_savevm_end(Monitor *mon, const QDict *qdict)
+void coroutine_fn hmp_savevm_end(Monitor *mon, const QDict *qdict)
{
Error *errp = NULL;
diff --git a/qapi/misc.json b/qapi/misc.json
index b7e132698e..4f5333d960 100644
--- a/qapi/misc.json
+++ b/qapi/misc.json
@@ -506,7 +506,7 @@
# Resume VM after a snapshot.
#
##
-{ 'command': 'savevm-end' }
+{ 'command': 'savevm-end', 'coroutine': true }
##
# @CommandLineParameterType:
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [pve-devel] applied: [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2
2021-02-11 16:11 [pve-devel] [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2 Stefan Reiter
` (3 preceding siblings ...)
2021-02-11 16:11 ` [pve-devel] [PATCH INFO v2 qemu 5/5] PVE: savevm-async: improve snapshot abort Stefan Reiter
@ 2021-02-12 14:35 ` Thomas Lamprecht
4 siblings, 0 replies; 7+ messages in thread
From: Thomas Lamprecht @ 2021-02-12 14:35 UTC (permalink / raw)
To: Proxmox VE development discussion, Stefan Reiter
On 11.02.21 17:11, Stefan Reiter wrote:
> Lots of patches touched and some slight changes to the build process
> since QEMU switched to meson as their build system. Functionality-wise
> very little rebasing required.
>
> New patches introduced:
> * pve/0058: to fix VMA backups and clean up some code in general with
> new 5.2 features now available to us (namely coroutine-enabled QMP).
> * extra/0002: don't build man pages for guest agent when disabled
> * extra/0003: fix live-migration with hugepages
> * 0017 and 0018 are adjusted to fix snapshot abort and improve
> snap performance a bit
>
> Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
> ---
>
> v2:
> * update Makefile to detect in-tree builds in qemu/
> * add fixes for snapshot abort and fix QEMUFile buffer size increase
> * add agent man page and live-migration fix (also accepted upstream)
>
> Known issues:
> * Snapshots with the vmstate target on ceph via librbd are slow - this is not a
> new bug however. An attempt at a fix is included via the changes to the
> QEMUFile patch, but it only improves the situtation slightly if at all.
> I'm still looking into this one, but it should not be a blocker for 5.2 I
> believe.
>
>
> Makefile | 3 +
> debian/control | 1 +
> ...u-img-convert-Don-t-pre-zero-images.patch} | 4 +-
> ...-always-align-copied-region-to-clust.patch | 34 -
> ...ll-man-page-if-guest-agent-is-disabl.patch | 38 ++
> ...heck-page-size-match-if-RAM-postcopy.patch | 31 +
> ...sb-fix-setup_len-init-CVE-2020-14364.patch | 87 ---
> ...k-file-change-locking-default-to-off.patch | 2 +-
> ...djust-network-script-path-to-etc-kvm.patch | 17 +-
> ...he-CPU-model-to-kvm64-32-instead-of-.patch | 4 +-
> ...ui-spice-default-to-pve-certificates.patch | 4 +-
> .../0005-PVE-Config-smm_available-false.patch | 4 +-
> ...lock-rbd-disable-rbd_cache_writethro.patch | 4 +-
> .../0008-PVE-Up-qmp-add-get_link_status.patch | 6 +-
> ...return-success-on-info-without-snaps.patch | 4 +-
> ...dd-add-osize-and-read-from-to-stdin-.patch | 18 +-
> ...E-Up-qemu-img-dd-add-isize-parameter.patch | 12 +-
> ...PVE-Up-qemu-img-dd-add-n-skip_create.patch | 10 +-
> ...virtio-balloon-improve-query-balloon.patch | 22 +-
> .../0015-PVE-qapi-modify-query-machines.patch | 12 +-
> .../0016-PVE-qapi-modify-spice-query.patch | 8 +-
> ...sync-for-background-state-snapshots.patch} | 584 +++++++++--------
> ...dd-optional-buffer-size-to-QEMUFile.patch} | 32 +-
> ...add-the-zeroinit-block-driver-filter.patch | 36 +-
> ...-Add-dummy-id-command-line-parameter.patch | 10 +-
> ...t-target-i386-disable-LINT0-after-re.patch | 2 +-
> ...le-posix-make-locking-optiono-on-cre.patch | 20 +-
> ...3-PVE-monitor-disable-oob-capability.patch | 4 +-
> ...sed-balloon-qemu-4-0-config-size-fal.patch | 4 +-
> ...E-Allow-version-code-in-machine-type.patch | 36 +-
> ...VE-Backup-add-vma-backup-format-code.patch | 83 +--
> ...-Backup-add-backup-dump-block-driver.patch | 36 +-
> ...ckup-proxmox-backup-patches-for-qemu.patch | 127 ++--
> ...estore-new-command-to-restore-from-p.patch | 39 +-
> ...d-support-for-sync-bitmap-mode-never.patch | 76 +--
> ...-support-for-conditional-and-always-.patch | 10 +-
> ...check-for-bitmap-mode-without-bitmap.patch | 6 +-
> ...-to-bdrv_dirty_bitmap_merge_internal.patch | 6 +-
> ...5-iotests-add-test-for-bitmap-mirror.patch | 2 +-
> .../0036-mirror-move-some-checks-to-qmp.patch | 8 +-
> ...irty-bitmap-tracking-for-incremental.patch | 18 +-
> ...name-incremental-to-use-dirty-bitmap.patch | 8 +-
> ...ckup-add-compress-and-encrypt-option.patch | 10 +-
> ...k-driver-to-map-backup-archives-into.patch | 80 +--
> ...dd-query_proxmox_support-QMP-command.patch | 6 +-
> ...issing-crypt-and-compress-parameters.patch | 2 +-
> ...rite-callback-with-big-blocks-correc.patch | 2 +-
> ...-block-handling-to-PBS-dump-callback.patch | 2 +-
> ...E-add-query-pbs-bitmap-info-QMP-call.patch | 14 +-
> ...ct-stderr-to-journal-when-daemonized.patch | 40 +-
> ...d-sequential-job-transaction-support.patch | 2 +-
> ...-transaction-to-synchronize-job-stat.patch | 2 +-
> ...ore-coroutines-and-don-t-block-on-fi.patch | 6 +-
> ...n-up-error-handling-for-create_backu.patch | 2 +-
> ...grate-dirty-bitmap-state-via-savevm.patch} | 39 +-
> ...dirty-bitmap-fix-larger-granularity-.patch | 33 -
> ...irty-bitmap-migrate-other-bitmaps-e.patch} | 6 +-
> ...multiple-CREATED-jobs-in-sequential.patch} | 2 +-
> ...ll-back-to-open-iscsi-initiatorname.patch} | 5 +-
> ...routine-QMP-for-backup-cancel_backup.patch | 597 ++++++++++++++++++
> debian/patches/series | 20 +-
> debian/pve-qemu-kvm.install | 2 -
> debian/rules | 12 +-
> qemu | 2 +-
> 64 files changed, 1436 insertions(+), 922 deletions(-)
> rename debian/patches/extra/{0002-Revert-qemu-img-convert-Don-t-pre-zero-images.patch => 0001-Revert-qemu-img-convert-Don-t-pre-zero-images.patch} (91%)
> delete mode 100644 debian/patches/extra/0001-block-block-copy-always-align-copied-region-to-clust.patch
> create mode 100644 debian/patches/extra/0002-docs-don-t-install-man-page-if-guest-agent-is-disabl.patch
> create mode 100644 debian/patches/extra/0003-migration-only-check-page-size-match-if-RAM-postcopy.patch
> delete mode 100644 debian/patches/extra/0003-usb-fix-setup_len-init-CVE-2020-14364.patch
> rename debian/patches/pve/{0017-PVE-internal-snapshot-async.patch => 0017-PVE-add-savevm-async-for-background-state-snapshots.patch} (84%)
> rename debian/patches/pve/{0018-add-optional-buffer-size-to-QEMUFile.patch => 0018-PVE-add-optional-buffer-size-to-QEMUFile.patch} (87%)
> rename debian/patches/pve/{0055-PVE-Migrate-dirty-bitmap-state-via-savevm.patch => 0054-PVE-Migrate-dirty-bitmap-state-via-savevm.patch} (90%)
> delete mode 100644 debian/patches/pve/0054-migration-block-dirty-bitmap-fix-larger-granularity-.patch
> rename debian/patches/pve/{0056-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch => 0055-migration-block-dirty-bitmap-migrate-other-bitmaps-e.patch} (89%)
> rename debian/patches/pve/{0057-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch => 0056-PVE-fix-aborting-multiple-CREATED-jobs-in-sequential.patch} (97%)
> rename debian/patches/pve/{0058-PVE-fall-back-to-open-iscsi-initiatorname.patch => 0057-PVE-fall-back-to-open-iscsi-initiatorname.patch} (97%)
> create mode 100644 debian/patches/pve/0058-PVE-Use-coroutine-QMP-for-backup-cancel_backup.patch
>
>
applied, thanks!
^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-02-12 14:35 UTC | newest]
Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-11 16:11 [pve-devel] [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2 Stefan Reiter
2021-02-11 16:11 ` [pve-devel] [PATCH v2 pve-qemu 2/5] add PBS master key support Stefan Reiter
2021-02-12 14:35 ` [pve-devel] applied: " Thomas Lamprecht
2021-02-11 16:11 ` [pve-devel] [PATCH INFO v2 qemu 3/5] PVE: Use coroutine QMP for backup/cancel_backup Stefan Reiter
2021-02-11 16:11 ` [pve-devel] [PATCH INFO v2 qemu 4/5] PVE: increase max IOV count in QEMUFile Stefan Reiter
2021-02-11 16:11 ` [pve-devel] [PATCH INFO v2 qemu 5/5] PVE: savevm-async: improve snapshot abort Stefan Reiter
2021-02-12 14:35 ` [pve-devel] applied: [PATCH v2 pve-qemu 1/5] Update to QEMU 5.2 Thomas Lamprecht
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox