From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 0682B9C84 for ; Fri, 4 Aug 2023 15:05:09 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DC65DF46B for ; Fri, 4 Aug 2023 15:04:38 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Fri, 4 Aug 2023 15:04:37 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 9638440C0B for ; Fri, 4 Aug 2023 15:04:37 +0200 (CEST) From: Fiona Ebner To: pve-devel@lists.proxmox.com Date: Fri, 4 Aug 2023 15:04:33 +0200 Message-Id: <20230804130433.66703-1-f.ebner@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.044 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [meson.build] Subject: [pve-devel] [PATCH qemu] backup: drop broken BACKUP_FORMAT_DIR X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2023 13:05:09 -0000 Since upstream QEMU 8.0, it's no longer possible to call bdrv_img_create() from a coroutine anymore, meaning a backup with the directory format would crash the QEMU instance. The feature is only exposed via the monitor and was intended to be experimental. There were no user reports about the breakage and it only was noticed during the rebase for QEMU 8.1, because other parts of the backup code needed adaptation and I decided to check the BACKUP_FORMAT_DIR case too. It should not stay in a broken state of course, but avoid the maintenance cost and just make it a removed feature for Proxmox VE 8 retroactively. Signed-off-by: Fiona Ebner --- ...ckup-Proxmox-backup-patches-for-QEMU.patch | 92 +++++-------------- ...k-driver-to-map-backup-archives-into.patch | 8 +- ...igrate-dirty-bitmap-state-via-savevm.patch | 10 +- 3 files changed, 32 insertions(+), 78 deletions(-) diff --git a/debian/patches/pve/0030-PVE-Backup-Proxmox-backup-patches-for-QEMU.patch b/debian/patches/pve/0030-PVE-Backup-Proxmox-backup-patches-for-QEMU.patch index 3753eff..1f88285 100644 --- a/debian/patches/pve/0030-PVE-Backup-Proxmox-backup-patches-for-QEMU.patch +++ b/debian/patches/pve/0030-PVE-Backup-Proxmox-backup-patches-for-QEMU.patch @@ -83,20 +83,20 @@ Signed-off-by: Wolfgang Bumiller Signed-off-by: Fiona Ebner --- block/meson.build | 5 + - block/monitor/block-hmp-cmds.c | 40 ++ + block/monitor/block-hmp-cmds.c | 39 ++ blockdev.c | 1 + hmp-commands-info.hx | 14 + - hmp-commands.hx | 31 + + hmp-commands.hx | 29 + include/monitor/hmp.h | 3 + meson.build | 1 + monitor/hmp-cmds.c | 72 +++ proxmox-backup-client.c | 146 +++++ proxmox-backup-client.h | 60 ++ - pve-backup.c | 1097 ++++++++++++++++++++++++++++++++ - qapi/block-core.json | 226 +++++++ + pve-backup.c | 1051 ++++++++++++++++++++++++++++++++ + qapi/block-core.json | 229 +++++++ qapi/common.json | 13 + qapi/machine.json | 15 +- - 14 files changed, 1711 insertions(+), 13 deletions(-) + 14 files changed, 1665 insertions(+), 13 deletions(-) create mode 100644 proxmox-backup-client.c create mode 100644 proxmox-backup-client.h create mode 100644 pve-backup.c @@ -118,10 +118,10 @@ index f580f95395..5bcebb934b 100644 softmmu_ss.add(when: 'CONFIG_TCG', if_true: files('blkreplay.c')) softmmu_ss.add(files('block-ram-registrar.c')) diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c -index ca2599de44..636509b83e 100644 +index ca2599de44..6efe28cef5 100644 --- a/block/monitor/block-hmp-cmds.c +++ b/block/monitor/block-hmp-cmds.c -@@ -1029,3 +1029,43 @@ void hmp_change_medium(Monitor *mon, const char *device, const char *target, +@@ -1029,3 +1029,42 @@ void hmp_change_medium(Monitor *mon, const char *device, const char *target, qmp_blockdev_change_medium(device, NULL, target, arg, true, force, !!read_only, read_only_mode, errp); } @@ -139,7 +139,6 @@ index ca2599de44..636509b83e 100644 +{ + Error *error = NULL; + -+ int dir = qdict_get_try_bool(qdict, "directory", 0); + const char *backup_file = qdict_get_str(qdict, "backupfile"); + const char *devlist = qdict_get_try_str(qdict, "devlist"); + int64_t speed = qdict_get_try_int(qdict, "speed", 0); @@ -157,7 +156,7 @@ index ca2599de44..636509b83e 100644 + false, false, // PBS use-dirty-bitmap + false, false, // PBS compress + false, false, // PBS encrypt -+ true, dir ? BACKUP_FORMAT_DIR : BACKUP_FORMAT_VMA, ++ true, BACKUP_FORMAT_VMA, + NULL, NULL, + devlist, qdict_haskey(qdict, "speed"), speed, + false, 0, // BackupPerf max-workers @@ -203,10 +202,10 @@ index a166bff3d5..4b75966c2e 100644 { .name = "usernet", diff --git a/hmp-commands.hx b/hmp-commands.hx -index d9f9f42d11..775518fb09 100644 +index d9f9f42d11..ddb9678dc3 100644 --- a/hmp-commands.hx +++ b/hmp-commands.hx -@@ -101,6 +101,37 @@ ERST +@@ -101,6 +101,35 @@ ERST SRST ``block_stream`` Copy data from a backing file into a block device. @@ -214,11 +213,9 @@ index d9f9f42d11..775518fb09 100644 + + { + .name = "backup", -+ .args_type = "directory:-d,backupfile:s,speed:o?,devlist:s?", -+ .params = "[-d] backupfile [speed [devlist]]", -+ .help = "create a VM Backup." -+ "\n\t\t\t Use -d to dump data into a directory instead" -+ "\n\t\t\t of using VMA format.", ++ .args_type = "backupfile:s,speed:o?,devlist:s?", ++ .params = "backupfile [speed [devlist]]", ++ .help = "create a VM backup (VMA format).", + .cmd = hmp_backup, + .coroutine = true, + }, @@ -587,10 +584,10 @@ index 0000000000..8cbf645b2c +#endif /* PROXMOX_BACKUP_CLIENT_H */ diff --git a/pve-backup.c b/pve-backup.c new file mode 100644 -index 0000000000..dd72ee0ed6 +index 0000000000..884ca9239c --- /dev/null +++ b/pve-backup.c -@@ -0,0 +1,1097 @@ +@@ -0,0 +1,1051 @@ +#include "proxmox-backup-client.h" +#include "vma.h" + @@ -1020,7 +1017,6 @@ index 0000000000..dd72ee0ed6 + const char *file, + const char *name, + BackupFormat format, -+ const char *backup_dir, + VmaWriter *vmaw, + ProxmoxBackupHandle *pbs, + Error **errp) @@ -1046,13 +1042,6 @@ index 0000000000..dd72ee0ed6 + } else if (format == BACKUP_FORMAT_PBS) { + if (proxmox_backup_co_add_config(pbs, name, (unsigned char *)cdata, clen, errp) < 0) + goto err; -+ } else if (format == BACKUP_FORMAT_DIR) { -+ char config_path[PATH_MAX]; -+ snprintf(config_path, PATH_MAX, "%s/%s", backup_dir, name); -+ if (!g_file_set_contents(config_path, cdata, clen, &err)) { -+ error_setg(errp, "unable to write config file '%s'", config_path); -+ goto err; -+ } + } + + out: @@ -1188,7 +1177,6 @@ index 0000000000..dd72ee0ed6 + + BlockBackend *blk; + BlockDriverState *bs = NULL; -+ const char *backup_dir = NULL; + Error *local_err = NULL; + uuid_t uuid; + VmaWriter *vmaw = NULL; @@ -1428,54 +1416,21 @@ index 0000000000..dd72ee0ed6 + goto err_mutex; + } + } -+ } else if (format == BACKUP_FORMAT_DIR) { -+ if (mkdir(backup_file, 0640) != 0) { -+ error_setg_errno(errp, errno, "can't create directory '%s'\n", -+ backup_file); -+ goto err_mutex; -+ } -+ backup_dir = backup_file; -+ -+ l = di_list; -+ while (l) { -+ PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data; -+ l = g_list_next(l); -+ -+ const char *devname = bdrv_get_device_name(di->bs); -+ snprintf(di->targetfile, PATH_MAX, "%s/%s.raw", backup_dir, devname); -+ -+ int flags = BDRV_O_RDWR; -+ bdrv_img_create(di->targetfile, "raw", NULL, NULL, NULL, -+ di->size, flags, false, &local_err); -+ if (local_err) { -+ error_propagate(errp, local_err); -+ goto err_mutex; -+ } -+ -+ di->target = bdrv_co_open(di->targetfile, NULL, NULL, flags, &local_err); -+ if (!di->target) { -+ error_propagate(errp, local_err); -+ goto err_mutex; -+ } -+ } + } else { + error_set(errp, ERROR_CLASS_GENERIC_ERROR, "unknown backup format"); + goto err_mutex; + } + -+ + /* add configuration file to archive */ + if (config_file) { -+ if (pvebackup_co_add_config(config_file, config_name, format, backup_dir, -+ vmaw, pbs, errp) != 0) { ++ if (pvebackup_co_add_config(config_file, config_name, format, vmaw, pbs, errp) != 0) { + goto err_mutex; + } + } + + /* add firewall file to archive */ + if (firewall_file) { -+ if (pvebackup_co_add_config(firewall_file, firewall_name, format, backup_dir, -+ vmaw, pbs, errp) != 0) { ++ if (pvebackup_co_add_config(firewall_file, firewall_name, format, vmaw, pbs, errp) != 0) { + goto err_mutex; + } + } @@ -1588,10 +1543,6 @@ index 0000000000..dd72ee0ed6 + backup_state.pbs = NULL; + } + -+ if (backup_dir) { -+ rmdir(backup_dir); -+ } -+ + qemu_co_mutex_unlock(&backup_state.backup_mutex); + return NULL; +} @@ -1689,10 +1640,10 @@ index 0000000000..dd72ee0ed6 + return ret; +} diff --git a/qapi/block-core.json b/qapi/block-core.json -index 542add004b..4ec70acf95 100644 +index 542add004b..985859ddee 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json -@@ -835,6 +835,232 @@ +@@ -835,6 +835,235 @@ { 'command': 'query-block', 'returns': ['BlockInfo'], 'allow-preconfig': true } @@ -1742,9 +1693,12 @@ index 542add004b..4ec70acf95 100644 +# An enumeration of supported backup formats. +# +# @vma: Proxmox vma backup format ++# ++# @pbs: Proxmox backup server format ++# +## +{ 'enum': 'BackupFormat', -+ 'data': [ 'vma', 'dir', 'pbs' ] } ++ 'data': [ 'vma', 'pbs' ] } + +## +# @backup: diff --git a/debian/patches/pve/0032-PVE-Add-PBS-block-driver-to-map-backup-archives-into.patch b/debian/patches/pve/0032-PVE-Add-PBS-block-driver-to-map-backup-archives-into.patch index 81d33d3..102cfcf 100644 --- a/debian/patches/pve/0032-PVE-Add-PBS-block-driver-to-map-backup-archives-into.patch +++ b/debian/patches/pve/0032-PVE-Add-PBS-block-driver-to-map-backup-archives-into.patch @@ -403,10 +403,10 @@ index 32ab849ce6..69afe3441b 100644 summary_info += {'libdaxctl support': libdaxctl} summary_info += {'libudev': libudev} diff --git a/qapi/block-core.json b/qapi/block-core.json -index 4ec70acf95..47118bf83e 100644 +index 985859ddee..d601fb4ab2 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json -@@ -3301,6 +3301,7 @@ +@@ -3304,6 +3304,7 @@ 'parallels', 'preallocate', 'qcow', 'qcow2', 'qed', 'quorum', 'raw', 'rbd', { 'name': 'replication', 'if': 'CONFIG_REPLICATION' }, @@ -414,7 +414,7 @@ index 4ec70acf95..47118bf83e 100644 'ssh', 'throttle', 'vdi', 'vhdx', { 'name': 'virtio-blk-vfio-pci', 'if': 'CONFIG_BLKIO' }, { 'name': 'virtio-blk-vhost-user', 'if': 'CONFIG_BLKIO' }, -@@ -3377,6 +3378,17 @@ +@@ -3380,6 +3381,17 @@ { 'struct': 'BlockdevOptionsNull', 'data': { '*size': 'int', '*latency-ns': 'uint64', '*read-zeroes': 'bool' } } @@ -432,7 +432,7 @@ index 4ec70acf95..47118bf83e 100644 ## # @BlockdevOptionsNVMe: # -@@ -4750,6 +4762,7 @@ +@@ -4753,6 +4765,7 @@ 'nfs': 'BlockdevOptionsNfs', 'null-aio': 'BlockdevOptionsNull', 'null-co': 'BlockdevOptionsNull', diff --git a/debian/patches/pve/0034-PVE-Migrate-dirty-bitmap-state-via-savevm.patch b/debian/patches/pve/0034-PVE-Migrate-dirty-bitmap-state-via-savevm.patch index 9437869..9a00ce6 100644 --- a/debian/patches/pve/0034-PVE-Migrate-dirty-bitmap-state-via-savevm.patch +++ b/debian/patches/pve/0034-PVE-Migrate-dirty-bitmap-state-via-savevm.patch @@ -175,10 +175,10 @@ index 0000000000..887e998b9e + NULL); +} diff --git a/pve-backup.c b/pve-backup.c -index dd72ee0ed6..cb5312fff3 100644 +index 884ca9239c..717393f4a3 100644 --- a/pve-backup.c +++ b/pve-backup.c -@@ -1090,6 +1090,7 @@ ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp) +@@ -1044,6 +1044,7 @@ ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp) ret->pbs_library_version = g_strdup(proxmox_backup_qemu_version()); ret->pbs_dirty_bitmap = true; ret->pbs_dirty_bitmap_savevm = true; @@ -187,10 +187,10 @@ index dd72ee0ed6..cb5312fff3 100644 ret->pbs_masterkey = true; ret->backup_max_workers = true; diff --git a/qapi/block-core.json b/qapi/block-core.json -index 47118bf83e..809f3c61bc 100644 +index d601fb4ab2..16be1e02ec 100644 --- a/qapi/block-core.json +++ b/qapi/block-core.json -@@ -984,6 +984,11 @@ +@@ -987,6 +987,11 @@ # @pbs-dirty-bitmap-savevm: True if 'dirty-bitmaps' migration capability can # safely be set for savevm-async. # @@ -202,7 +202,7 @@ index 47118bf83e..809f3c61bc 100644 # @pbs-masterkey: True if the QMP backup call supports the 'master_keyfile' # parameter. # -@@ -994,6 +999,7 @@ +@@ -997,6 +1002,7 @@ 'data': { 'pbs-dirty-bitmap': 'bool', 'query-bitmap-info': 'bool', 'pbs-dirty-bitmap-savevm': 'bool', -- 2.39.2