From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 76FF991F82 for ; Mon, 10 Oct 2022 12:54:18 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 606BA2E46 for ; Mon, 10 Oct 2022 12:54:18 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 10 Oct 2022 12:54:17 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 4495F44829 for ; Mon, 10 Oct 2022 12:54:17 +0200 (CEST) Date: Mon, 10 Oct 2022 12:54:16 +0200 From: Wolfgang Bumiller To: Fiona Ebner Cc: pve-devel@lists.proxmox.com Message-ID: <20221010105416.dcle7icji4qsjsii@casey.proxmox.com> References: <20221003135211.183340-1-f.ebner@proxmox.com> <20221003135211.183340-2-f.ebner@proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20221003135211.183340-2-f.ebner@proxmox.com> X-SPAM-LEVEL: Spam detection results: 0 AWL 0.256 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] applied: [PATCH qemu 1/1] PVE Backup: allow passing max-workers performance setting X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Oct 2022 10:54:18 -0000 applied this one, thanks On Mon, Oct 03, 2022 at 03:52:04PM +0200, Fiona Ebner wrote: > For query-proxmox-support, add an indication that it's possible to use > the setting. > > For now, the other two BackupPerf settings are not exposed: > > * use-copy-range: would need to be implemented by the backup-dump > block driver first, and in fact, the default for backup was changed, > because it wasn't as fast for backup in QEMU, see commit > 6a30f663d4c0b3c45a544d541e0c4e214b2473a1. > > * max-chunk: enforced to be at least the backup cluster size, which is > 4 MiB for PBS and otherwise maximum of source and target cluster size. > And block-copy has a maximum buffer size of 1 MiB, so setting a larger > max-chunk doesn't even have an effect. To make the setting sensibly > usable the check would need to be removed and optionally the > block-copy max buffer size would need to be bumped. I tried doing just > that, and tested different source/target combinations with different > max-chunk settings, but there were no noticable improvements over the > default "unlimited" (resulting in 1 MiB for block-copy). > > Signed-off-by: Fiona Ebner > --- > block/monitor/block-hmp-cmds.c | 4 +++- > pve-backup.c | 18 +++++++++++++----- > qapi/block-core.json | 9 +++++++-- > 3 files changed, 23 insertions(+), 8 deletions(-) > > diff --git a/block/monitor/block-hmp-cmds.c b/block/monitor/block-hmp-cmds.c > index 0502f42be6..cc231ec3f2 100644 > --- a/block/monitor/block-hmp-cmds.c > +++ b/block/monitor/block-hmp-cmds.c > @@ -1049,7 +1049,9 @@ void coroutine_fn hmp_backup(Monitor *mon, const QDict *qdict) > false, false, // PBS encrypt > true, dir ? BACKUP_FORMAT_DIR : BACKUP_FORMAT_VMA, > false, NULL, false, NULL, !!devlist, > - devlist, qdict_haskey(qdict, "speed"), speed, &error); > + devlist, qdict_haskey(qdict, "speed"), speed, > + false, 0, // BackupPerf max-workers > + &error); > > hmp_handle_error(mon, error); > } > diff --git a/pve-backup.c b/pve-backup.c > index 2e22030eec..e9aa7e0f49 100644 > --- a/pve-backup.c > +++ b/pve-backup.c > @@ -55,6 +55,7 @@ static struct PVEBackupState { > bool starting; > } stat; > int64_t speed; > + BackupPerf perf; > VmaWriter *vmaw; > ProxmoxBackupHandle *pbs; > GList *di_list; > @@ -492,8 +493,6 @@ static void create_backup_jobs_bh(void *opaque) { > } > backup_state.txn = job_txn_new_seq(); > > - BackupPerf perf = { .max_workers = 16 }; > - > /* create and start all jobs (paused state) */ > GList *l = backup_state.di_list; > while (l) { > @@ -513,8 +512,9 @@ static void create_backup_jobs_bh(void *opaque) { > > BlockJob *job = backup_job_create( > NULL, di->bs, di->target, backup_state.speed, sync_mode, di->bitmap, > - bitmap_mode, false, NULL, &perf, BLOCKDEV_ON_ERROR_REPORT, BLOCKDEV_ON_ERROR_REPORT, > - JOB_DEFAULT, pvebackup_complete_cb, di, backup_state.txn, &local_err); > + bitmap_mode, false, NULL, &backup_state.perf, BLOCKDEV_ON_ERROR_REPORT, > + BLOCKDEV_ON_ERROR_REPORT, JOB_DEFAULT, pvebackup_complete_cb, di, backup_state.txn, > + &local_err); > > di->job = job; > if (job) { > @@ -584,7 +584,9 @@ UuidInfo coroutine_fn *qmp_backup( > bool has_config_file, const char *config_file, > bool has_firewall_file, const char *firewall_file, > bool has_devlist, const char *devlist, > - bool has_speed, int64_t speed, Error **errp) > + bool has_speed, int64_t speed, > + bool has_max_workers, int64_t max_workers, > + Error **errp) > { > assert(qemu_in_coroutine()); > > @@ -914,6 +916,11 @@ UuidInfo coroutine_fn *qmp_backup( > > backup_state.speed = (has_speed && speed > 0) ? speed : 0; > > + backup_state.perf = (BackupPerf){ .max_workers = 16 }; > + if (has_max_workers) { > + backup_state.perf.max_workers = max_workers; > + } > + > backup_state.vmaw = vmaw; > backup_state.pbs = pbs; > > @@ -1089,5 +1096,6 @@ ProxmoxSupportStatus *qmp_query_proxmox_support(Error **errp) > ret->pbs_dirty_bitmap_migration = true; > ret->query_bitmap_info = true; > ret->pbs_masterkey = true; > + ret->backup_max_workers = true; > return ret; > } > diff --git a/qapi/block-core.json b/qapi/block-core.json > index 89875f309c..43281aca79 100644 > --- a/qapi/block-core.json > +++ b/qapi/block-core.json > @@ -833,6 +833,8 @@ > # > # @encrypt: use encryption ((optional for format 'pbs', defaults to true if there is a keyfile) > # > +# @max-workers: see @BackupPerf for details. Default 16. > +# > # Returns: the uuid of the backup job > # > ## > @@ -851,7 +853,9 @@ > '*format': 'BackupFormat', > '*config-file': 'str', > '*firewall-file': 'str', > - '*devlist': 'str', '*speed': 'int' }, > + '*devlist': 'str', > + '*speed': 'int', > + '*max-workers': 'int' }, > 'returns': 'UuidInfo', 'coroutine': true } > > ## > @@ -906,7 +910,8 @@ > 'pbs-dirty-bitmap-savevm': 'bool', > 'pbs-dirty-bitmap-migration': 'bool', > 'pbs-masterkey': 'bool', > - 'pbs-library-version': 'str' } } > + 'pbs-library-version': 'str', > + 'backup-max-workers': 'bool' } } > > ## > # @query-proxmox-support: > -- > 2.30.2