From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id F2FEB69E52 for ; Mon, 10 Aug 2020 17:14:06 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E008C16CC6 for ; Mon, 10 Aug 2020 17:13:36 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 7B8BE16CB9 for ; Mon, 10 Aug 2020 17:13:35 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 3BB7A44568 for ; Mon, 10 Aug 2020 17:13:35 +0200 (CEST) To: Proxmox Backup Server development discussion , =?UTF-8?Q?Fabian_Gr=c3=bcnbichler?= References: <20200810112509.70129-1-f.gruenbichler@proxmox.com> <20200810112509.70129-2-f.gruenbichler@proxmox.com> From: Stefan Reiter Message-ID: Date: Mon, 10 Aug 2020 17:13:34 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <20200810112509.70129-2-f.gruenbichler@proxmox.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.664 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -1.455 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [PATCH proxmox-backup-qemu] adapt to chunk reader changes X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 10 Aug 2020 15:14:07 -0000 makes sense in general, one comment inline On 8/10/20 1:25 PM, Fabian Grünbichler wrote: > for verification of chunk crypt mode <=> index crypt mode match > > Signed-off-by: Fabian Grünbichler > --- > follows patch #4 in proxmox-backup > > src/restore.rs | 30 +++++++++++++----------------- > 1 file changed, 13 insertions(+), 17 deletions(-) > > diff --git a/src/restore.rs b/src/restore.rs > index e43d040..9d8c4a2 100644 > --- a/src/restore.rs > +++ b/src/restore.rs > @@ -1,5 +1,4 @@ > use std::sync::{Arc, Mutex}; > -use std::collections::HashMap; > use std::io::SeekFrom; > use std::convert::TryInto; > > @@ -27,7 +26,6 @@ pub(crate) struct RestoreTask { > runtime: Arc, > crypt_config: Option>, > client: OnceCell>, > - chunk_reader: OnceCell, > manifest: OnceCell>, > image_registry: Arc>>, > } > @@ -59,7 +57,6 @@ impl RestoreTask { > crypt_config, > client: OnceCell::new(), > manifest: OnceCell::new(), > - chunk_reader: OnceCell::new(), > image_registry: Arc::new(Mutex::new(Registry::::new())), > }) > } > @@ -94,15 +91,6 @@ impl RestoreTask { > true > ).await?; > > - let chunk_reader = RemoteChunkReader::new( > - client.clone(), > - self.crypt_config.clone(), > - HashMap::with_capacity(0), > - ); > - > - self.chunk_reader.set(chunk_reader) > - .map_err(|_| format_err!("already connected!"))?; > - > let (manifest, _) = client.download_manifest().await?; > > self.manifest.set(Arc::new(manifest)) > @@ -150,9 +138,12 @@ impl RestoreTask { > > let most_used = index.find_most_used_chunks(8); > > + let file_info = manifest.lookup_file_info(&archive_name)?; > + > let mut chunk_reader = RemoteChunkReader::new( > client.clone(), > self.crypt_config.clone(), > + file_info.chunk_crypt_mode(), > most_used, > ); > > @@ -219,11 +210,6 @@ impl RestoreTask { > None => bail!("not connected"), > }; > > - let chunk_reader = match self.chunk_reader.get() { > - Some(chunk_reader) => chunk_reader.clone(), > - None => bail!("not connected"), > - }; > - > let manifest = match self.manifest.get() { > Some(manifest) => manifest.clone(), > None => bail!("no manifest"), > @@ -231,6 +217,16 @@ impl RestoreTask { > > let index = client.download_fixed_index(&manifest, &archive_name).await?; > let archive_size = index.index_bytes(); > + let most_used = index.find_most_used_chunks(8); You change the cache hint from previously an empty HashMap to the 8 most-used chunks, is that intentional? I don't believe that really does anything for a fixed index, and the AsyncIndexReader already caches the currently accessed chunk. If anything, maybe add the zero digest here to keep that in cache, or even better maybe add that to the RemoteChunkReader? > + > + let file_info = manifest.lookup_file_info(&archive_name)?; > + > + let chunk_reader = RemoteChunkReader::new( > + client.clone(), > + self.crypt_config.clone(), > + file_info.chunk_crypt_mode(), > + most_used, > + ); > > let reader = AsyncIndexReader::new(index, chunk_reader); > >