From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 1249E9E632 for ; Mon, 27 Nov 2023 18:27:55 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E8B01C0B9 for ; Mon, 27 Nov 2023 18:27:54 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 27 Nov 2023 18:27:54 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 034ED44CC9 for ; Mon, 27 Nov 2023 18:27:54 +0100 (CET) Date: Mon, 27 Nov 2023 18:27:53 +0100 (CET) From: =?UTF-8?Q?Fabian_Gr=C3=BCnbichler?= To: Wolfgang Bumiller Cc: pbs-devel@lists.proxmox.com Message-ID: <1541339416.519.1701106073073@webmail.proxmox.com> In-Reply-To: References: <20230629103213.1041236-1-f.gruenbichler@proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-Priority: 3 Importance: Normal X-Mailer: Open-Xchange Mailer v7.10.6-Rev55 X-Originating-Client: open-xchange-appsuite X-SPAM-LEVEL: Spam detection results: 0 AWL 0.064 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [pbs-devel] [PATCH proxmox-backup] mapped loop device: use read loop instead of read_exact X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Nov 2023 17:27:55 -0000 > Wolfgang Bumiller hat am 27.11.2023 14:22 CET ge= schrieben: > =20 > On Thu, Jun 29, 2023 at 12:32:13PM +0200, Fabian Gr=C3=BCnbichler wrote: > > since read_exact does not support short reads, which can easily happen = if the > > mapped image's EOF is not aligned with the request size. > >=20 > > Signed-off-by: Fabian Gr=C3=BCnbichler > > --- > >=20 > > Notes: > > reported on the forum: > > =20 > > https://forum.proxmox.com/threads/problem-backing-up-using-backup-c= lient.129347 > > =20 > > did a quick test reading from a mapped image full of random data, o= bserved > > no performance difference.. >=20 > Do you get one if we just drop the loop logic and *actually* just > `read()` once? IMO this is more in line with what a read syscall > *should* be doing. > Further, we use a `CachedChunkReader` under it which actually does a > read loop anyway, so AFAICT this *can't* make a difference. with a plain read (+ optional truncate of the reply buf) performance is sti= ll the same. but (and I am unfortunately not sure if this is a regression i= n the meantime, or was also broken back when I originally wrote this patch)= access via the loop device actually truncates the resulting data: - my test input image is 1701838801 bytes long (arbitrary misaligned size, = straight from /dev/urandom) - the fuse session correctly gets this passed in as size - a regular restore restores as many (correct) bytes - reading via the loop device with bs=3D1024 or bs=3D512 or bs=3D32 only re= turns 1701838336 bytes (465 are missing) -- the fuse requests quickly ramp up to 128k request size (no matter the bl= ock size used to read from the loop device) -- the last fuse read request is for 16384 bytes, but the read from PBS (co= rrectly!) only returns 16337 -- 16337 - 31*512 =3D 465 -- so it seems the short read result is lost somewhere? -- reading with O_DIRECT doesn't help (in fact, it tanks performance while = still reproducing the issue) anyhow, this requires further analysis and fixing before being applied in w= hichever fashion..