From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 665D965386 for ; Wed, 22 Jul 2020 15:48:07 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 54FC621023 for ; Wed, 22 Jul 2020 15:48:07 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 14E2421018 for ; Wed, 22 Jul 2020 15:48:06 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id D39AB43305 for ; Wed, 22 Jul 2020 15:48:05 +0200 (CEST) From: Stefan Reiter To: pbs-devel@lists.proxmox.com Date: Wed, 22 Jul 2020 15:47:59 +0200 Message-Id: <20200722134759.9371-1-s.reiter@proxmox.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.020 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pbs-devel] [PATCH v2 qemu] PVE: handle PBS write callback with big blocks correctly X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jul 2020 13:48:07 -0000 Under certain conditions QEMU will push more than the given blocksize into the callback at once. Handle it like VMA does, by iterating the data until all is written. The block size is stored per backup device to be used in the callback. This avoids relying on PROXMOX_BACKUP_DEFAULT_CHUNK_SIZE, in case it is made configurable in the future. Signed-off-by: Stefan Reiter --- v2: * save chunk size explicitly to avoid depending on PROXMOX_BACKUP_DEFAULT_CHUNK_SIZE pve-backup.c | 30 ++++++++++++++++++++++-------- 1 file changed, 22 insertions(+), 8 deletions(-) diff --git a/pve-backup.c b/pve-backup.c index 77eb475563..40d8136f1a 100644 --- a/pve-backup.c +++ b/pve-backup.c @@ -67,6 +67,7 @@ opts_init(pvebackup_init); typedef struct PVEBackupDevInfo { BlockDriverState *bs; size_t size; + uint64_t block_size; uint8_t dev_id; bool completed; char targetfile[PATH_MAX]; @@ -147,17 +148,28 @@ pvebackup_co_dump_pbs_cb( return -1; } - pbs_res = proxmox_backup_co_write_data(backup_state.pbs, di->dev_id, buf, start, size, &local_err); - qemu_co_mutex_unlock(&backup_state.dump_callback_mutex); + uint64_t transferred = 0; + uint64_t reused = 0; + while (transferred < size) { + uint64_t left = size - transferred; + uint64_t to_transfer = left < di->block_size ? left : di->block_size; - if (pbs_res < 0) { - pvebackup_propagate_error(local_err); - return pbs_res; - } else { - size_t reused = (pbs_res == 0) ? size : 0; - pvebackup_add_transfered_bytes(size, !buf ? size : 0, reused); + pbs_res = proxmox_backup_co_write_data(backup_state.pbs, di->dev_id, + buf ? buf + transferred : NULL, start + transferred, to_transfer, &local_err); + transferred += to_transfer; + + if (pbs_res < 0) { + pvebackup_propagate_error(local_err); + qemu_co_mutex_unlock(&backup_state.dump_callback_mutex); + return pbs_res; + } + + reused += pbs_res == 0 ? to_transfer : 0; } + qemu_co_mutex_unlock(&backup_state.dump_callback_mutex); + pvebackup_add_transfered_bytes(size, !buf ? size : 0, reused); + return size; } @@ -730,6 +742,8 @@ static void coroutine_fn pvebackup_co_prepare(void *opaque) PVEBackupDevInfo *di = (PVEBackupDevInfo *)l->data; l = g_list_next(l); + di->block_size = dump_cb_block_size; + const char *devname = bdrv_get_device_name(di->bs); BdrvDirtyBitmap *bitmap = bdrv_find_dirty_bitmap(di->bs, PBS_BITMAP_NAME); -- 2.20.1