From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 5D8F56A605 for ; Thu, 25 Mar 2021 10:24:32 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4A4551AD9B for ; Thu, 25 Mar 2021 10:24:02 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 5D4021AD8E for ; Thu, 25 Mar 2021 10:24:01 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 1AD2D42D46 for ; Thu, 25 Mar 2021 10:24:01 +0100 (CET) To: Wolfgang Bumiller Cc: pbs-devel@lists.proxmox.com References: <20210324151827.26200-1-s.reiter@proxmox.com> <20210324151827.26200-2-s.reiter@proxmox.com> <20210325090836.r662otu4kyc4mokp@wobu-vie.proxmox.com> From: Stefan Reiter Message-ID: Date: Thu, 25 Mar 2021 10:23:59 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <20210325090836.r662otu4kyc4mokp@wobu-vie.proxmox.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.019 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_SHORT 0.001 Use of a URL Shortener for very short URL NICE_REPLY_A -0.001 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [aio.rs, docs.rs] Subject: Re: [pbs-devel] [PATCH v2 pxar 01/20] decoder/aio: add contents() and content_size() calls X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Mar 2021 09:24:32 -0000 On 3/25/21 10:08 AM, Wolfgang Bumiller wrote: > On Wed, Mar 24, 2021 at 04:18:08PM +0100, Stefan Reiter wrote: >> Returns a decoder::Contents without a wrapper type, since in this case >> we don't want to hide the SeqRead implementation (as done in >> decoder::sync). For conviencience also implement AsyncRead if "tokio-io" >> is enabled. >> >> Signed-off-by: Stefan Reiter >> --- >> >> proxmox-backup requires a dependency bump on this! >> >> v2: >> * make contents() call available without tokio-io feature >> * drop peek() implementation >> >> src/decoder/aio.rs | 36 +++++++++++++++++++++++++++++++++++- >> 1 file changed, 35 insertions(+), 1 deletion(-) >> >> diff --git a/src/decoder/aio.rs b/src/decoder/aio.rs >> index 82030b0..ee4d51f 100644 >> --- a/src/decoder/aio.rs >> +++ b/src/decoder/aio.rs >> @@ -5,7 +5,7 @@ use std::io; >> #[cfg(feature = "tokio-fs")] >> use std::path::Path; >> >> -use crate::decoder::{self, SeqRead}; >> +use crate::decoder::{self, Contents, SeqRead}; >> use crate::Entry; >> >> /// Asynchronous `pxar` decoder. >> @@ -56,6 +56,16 @@ impl Decoder { >> self.inner.next_do().await.transpose() >> } >> >> + /// Get a reader for the contents of the current entry, if the entry has contents. >> + pub fn contents(&mut self) -> Option> { >> + self.inner.content_reader() >> + } >> + >> + /// Get the size of the current contents, if the entry has contents. >> + pub fn content_size(&self) -> Option { >> + self.inner.content_size() >> + } >> + >> /// Include goodbye tables in iteration. >> pub fn enable_goodbye_entries(&mut self, on: bool) { >> self.inner.with_goodbye_tables = on; >> @@ -67,6 +77,7 @@ mod tok { >> use std::io; >> use std::pin::Pin; >> use std::task::{Context, Poll}; >> + use crate::decoder::{Contents, SeqRead}; >> >> /// Read adapter for `futures::io::AsyncRead` >> pub struct TokioReader { >> @@ -93,6 +104,29 @@ mod tok { >> } >> } >> } >> + >> + impl<'a, T: crate::decoder::SeqRead> tokio::io::AsyncRead for Contents<'a, T> { >> + fn poll_read( >> + self: Pin<&mut Self>, >> + cx: &mut Context<'_>, >> + buf: &mut tokio::io::ReadBuf<'_>, >> + ) -> Poll> { >> + unsafe { >> + // Safety: poll_seq_read will *probably* only write to the buffer, so we don't >> + // initialize it first, instead we treat is a &[u8] immediately and uphold the >> + // ReadBuf invariants in the conditional below. >> + let write_buf = >> + &mut *(buf.unfilled_mut() as *mut [std::mem::MaybeUninit] as *mut [u8]); >> + let result = self.poll_seq_read(cx, write_buf); >> + if let Poll::Ready(Ok(n)) = result { >> + // if we've written data, advance both initialized and filled bytes cursor >> + buf.assume_init(buf.filled().len() + n); >> + buf.advance(n); > > The differences here look fishy. The docs about `assume_init()` say it > marks the "first `n` *unfilled* bytes`, so IMO both calls should use > just `n` as they both start from the beginning of the unfilled range? > Hm, you're right: https://docs.rs/tokio/1.4.0/src/tokio/io/read_buf.rs.html#240 That doc is a bit confusing IMO... >> + } >> + result.map(|_| Ok(())) >> + } >> + } >> + } >> } >> >> #[cfg(feature = "tokio-io")] >> -- >> 2.20.1