From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 285B48CFA for ; Mon, 31 Jul 2023 15:34:15 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 116353A85 for ; Mon, 31 Jul 2023 15:34:15 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 31 Jul 2023 15:34:13 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 59BA642D35 for ; Mon, 31 Jul 2023 15:34:13 +0200 (CEST) From: Max Carrara To: pbs-devel@lists.proxmox.com Date: Mon, 31 Jul 2023 15:34:04 +0200 Message-Id: <20230731133404.859756-2-m.carrara@proxmox.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20230731133404.859756-1-m.carrara@proxmox.com> References: <20230731133404.859756-1-m.carrara@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.009 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: [pbs-devel] [PATCH v2 pxar 2/2] decoder: aio: improve performance of async file reads X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jul 2023 13:34:15 -0000 In order to bring `aio::Decoder` on par with its `sync` counterpart as well as `sync::Accessor` and `aio::Accessor`, its input is now buffered. As the `tokio` docs mention themselves [0], it can be really inefficient to directly work with an (unbuffered) `AsyncRead` instance. The other aforementioned types already buffer their reads in one way or another, so wrapping the input reader in `tokio::io::BufReader` results in a substantial performance gain. [1] `tokio/io-util` is added as dependency in order to use `tokio::io::BufReader`. [0]: https://docs.rs/tokio/1.29.1/tokio/io/struct.BufReader.html [1]: Tested via examples/compare-read.rs on a large (13GB) pxar archive Before: > PXAR Read Performance Comparison > Running in mode: release > > First pass: > With aio::Decoder: Ok(()) (elapsed: 20.532270177s) > With sync::Decoder: Ok(()) (elapsed: 3.498566141s) > With aio::Accessor: Ok(()) (elapsed: 3.978160609s) > With sync::Accessor: Ok(()) (elapsed: 3.885640895s) > > Second pass: > With aio::Decoder: Ok(()) (elapsed: 18.648986266s) > With sync::Decoder: Ok(()) (elapsed: 3.617167922s) > With aio::Accessor: Ok(()) (elapsed: 4.083678211s) > With sync::Accessor: Ok(()) (elapsed: 4.103763507s) After: > PXAR Read Performance Comparison > Running in mode: release > > First pass: > With aio::Decoder: Ok(()) (elapsed: 9.546522171s) > With sync::Decoder: Ok(()) (elapsed: 3.535062119s) > With aio::Accessor: Ok(()) (elapsed: 3.926439101s) > With sync::Accessor: Ok(()) (elapsed: 3.905232916s) > > Second pass: > With aio::Decoder: Ok(()) (elapsed: 10.633561678s) > With sync::Decoder: Ok(()) (elapsed: 3.528989778s) > With aio::Accessor: Ok(()) (elapsed: 3.831093917s) > With sync::Accessor: Ok(()) (elapsed: 3.848684845s) Signed-off-by: Max Carrara --- Changes v1 --> v2: * Include addition of `tokio/io-util` as dependency * Use new examples/compare-read.rs instead of old custom tool to measure performance impact * Use default buffer size (8K) instead of 16K (I wasn't able to reproduce the performance gains, so ...) Cargo.toml | 2 +- src/decoder/aio.rs | 10 +++++++--- 2 files changed, 8 insertions(+), 4 deletions(-) diff --git a/Cargo.toml b/Cargo.toml index 8669e30..08c0973 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -63,7 +63,7 @@ libc = "0.2" [features] default = [ "tokio-io" ] -tokio-io = [ "tokio" ] +tokio-io = [ "tokio", "tokio/io-util" ] tokio-fs = [ "tokio-io", "tokio/fs" ] full = [ "tokio-fs"] diff --git a/src/decoder/aio.rs b/src/decoder/aio.rs index 200dd3d..7cb9c12 100644 --- a/src/decoder/aio.rs +++ b/src/decoder/aio.rs @@ -79,14 +79,18 @@ mod tok { use std::pin::Pin; use std::task::{Context, Poll}; - /// Read adapter for `futures::io::AsyncRead` + use tokio::io::AsyncRead; + + /// Read adapter for `tokio::io::AsyncRead` pub struct TokioReader { - inner: T, + inner: tokio::io::BufReader, } impl TokioReader { pub fn new(inner: T) -> Self { - Self { inner } + Self { + inner: tokio::io::BufReader::new(inner), + } } } -- 2.39.2