From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 6A2AF6A0B0 for ; Tue, 11 Aug 2020 11:30:59 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 5FB3A1B835 for ; Tue, 11 Aug 2020 11:30:29 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 78C091B82B for ; Tue, 11 Aug 2020 11:30:28 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 365D044585 for ; Tue, 11 Aug 2020 11:30:28 +0200 (CEST) Date: Tue, 11 Aug 2020 11:30:21 +0200 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox Backup Server development discussion References: <20200722134759.9371-1-s.reiter@proxmox.com> In-Reply-To: <20200722134759.9371-1-s.reiter@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.15.0 (https://github.com/astroidmail/astroid) Message-Id: <1597138212.pdm6uyrhme.astroid@nora.none> Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL 0.042 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: [pbs-devel] applied: [PATCH v2 qemu] PVE: handle PBS write callback with big blocks correctly X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Aug 2020 09:30:59 -0000 On July 22, 2020 3:47 pm, Stefan Reiter wrote: > Under certain conditions QEMU will push more than the given blocksize > into the callback at once. Handle it like VMA does, by iterating the > data until all is written. >=20 > The block size is stored per backup device to be used in the callback. > This avoids relying on PROXMOX_BACKUP_DEFAULT_CHUNK_SIZE, in case it is > made configurable in the future. >=20 > Signed-off-by: Stefan Reiter > --- >=20 > v2: > * save chunk size explicitly to avoid depending on > PROXMOX_BACKUP_DEFAULT_CHUNK_SIZE >=20 > pve-backup.c | 30 ++++++++++++++++++++++-------- > 1 file changed, 22 insertions(+), 8 deletions(-) >=20 > diff --git a/pve-backup.c b/pve-backup.c > index 77eb475563..40d8136f1a 100644 > --- a/pve-backup.c > +++ b/pve-backup.c > @@ -67,6 +67,7 @@ opts_init(pvebackup_init); > typedef struct PVEBackupDevInfo { > BlockDriverState *bs; > size_t size; > + uint64_t block_size; > uint8_t dev_id; > bool completed; > char targetfile[PATH_MAX]; > @@ -147,17 +148,28 @@ pvebackup_co_dump_pbs_cb( > return -1; > } > =20 > - pbs_res =3D proxmox_backup_co_write_data(backup_state.pbs, di->dev_i= d, buf, start, size, &local_err); > - qemu_co_mutex_unlock(&backup_state.dump_callback_mutex); > + uint64_t transferred =3D 0; > + uint64_t reused =3D 0; > + while (transferred < size) { > + uint64_t left =3D size - transferred; > + uint64_t to_transfer =3D left < di->block_size ? left : di->bloc= k_size; > =20 > - if (pbs_res < 0) { > - pvebackup_propagate_error(local_err); > - return pbs_res; > - } else { > - size_t reused =3D (pbs_res =3D=3D 0) ? size : 0; > - pvebackup_add_transfered_bytes(size, !buf ? size : 0, reused); > + pbs_res =3D proxmox_backup_co_write_data(backup_state.pbs, di->d= ev_id, > + buf ? buf + transferred : NULL, start + transferred, to_tran= sfer, &local_err); > + transferred +=3D to_transfer; > + > + if (pbs_res < 0) { > + pvebackup_propagate_error(local_err); > + qemu_co_mutex_unlock(&backup_state.dump_callback_mutex); > + return pbs_res; > + } > + > + reused +=3D pbs_res =3D=3D 0 ? to_transfer : 0; > } > =20 > + qemu_co_mutex_unlock(&backup_state.dump_callback_mutex); > + pvebackup_add_transfered_bytes(size, !buf ? size : 0, reused); > + > return size; > } > =20 > @@ -730,6 +742,8 @@ static void coroutine_fn pvebackup_co_prepare(void *o= paque) > PVEBackupDevInfo *di =3D (PVEBackupDevInfo *)l->data; > l =3D g_list_next(l); > =20 > + di->block_size =3D dump_cb_block_size; > + > const char *devname =3D bdrv_get_device_name(di->bs); > =20 > BdrvDirtyBitmap *bitmap =3D bdrv_find_dirty_bitmap(di->bs, P= BS_BITMAP_NAME); > --=20 > 2.20.1 >=20 >=20 >=20 > _______________________________________________ > pbs-devel mailing list > pbs-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel >=20 >=20 >=20 =