From: Stefan Reiter <s.reiter@proxmox.com>
To: pve-devel@lists.proxmox.com, pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH v2 proxmox-backup-qemu 9/9] access: use CachedChunkReader
Date: Mon, 7 Jun 2021 17:35:32 +0200 [thread overview]
Message-ID: <20210607153532.2522267-10-s.reiter@proxmox.com> (raw)
In-Reply-To: <20210607153532.2522267-1-s.reiter@proxmox.com>
Use the new CachedChunkReader with the shared_cache implementation to
provide a concurrency-safe async way of accessing data. This provides
two benefits:
* uses a shared LRU cache, which is very helpful for random-access like
during a live-restore
* does away with the global Mutex in read_image_at, providing real
concurrency without lock contention
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
src/restore.rs | 30 ++++++++----------------------
1 file changed, 8 insertions(+), 22 deletions(-)
diff --git a/src/restore.rs b/src/restore.rs
index 0790d7f..33959d9 100644
--- a/src/restore.rs
+++ b/src/restore.rs
@@ -1,10 +1,8 @@
use std::sync::{Arc, Mutex};
-use std::io::SeekFrom;
use std::convert::TryInto;
use anyhow::{format_err, bail, Error};
use once_cell::sync::OnceCell;
-use tokio::io::{AsyncReadExt, AsyncSeekExt};
use tokio::runtime::Runtime;
use proxmox_backup::tools::runtime::get_runtime_with_builder;
@@ -14,9 +12,10 @@ use proxmox_backup::client::{HttpClient, HttpClientOptions, BackupReader, Remote
use super::BackupSetup;
use crate::registry::Registry;
use crate::capi_types::DataPointer;
+use crate::shared_cache::get_shared_chunk_cache;
struct ImageAccessInfo {
- reader: Arc<tokio::sync::Mutex<AsyncIndexReader<RemoteChunkReader, FixedIndexReader>>>,
+ reader: Arc<CachedChunkReader<FixedIndexReader, RemoteChunkReader>>,
_archive_name: String,
archive_size: u64,
}
@@ -229,12 +228,13 @@ impl RestoreTask {
most_used,
);
- let reader = AsyncIndexReader::new(index, chunk_reader);
+ let cache = get_shared_chunk_cache();
+ let reader = Arc::new(CachedChunkReader::new_with_cache(chunk_reader, index, cache));
let info = ImageAccessInfo {
archive_size,
_archive_name: archive_name, /// useful to debug
- reader: Arc::new(tokio::sync::Mutex::new(reader)),
+ reader,
};
(*self.image_registry.lock().unwrap()).register(info)
@@ -258,23 +258,9 @@ impl RestoreTask {
bail!("read index {} out of bounds {}", offset, image_size);
}
- let mut reader = reader.lock().await;
-
- let buf: &mut [u8] = unsafe { std::slice::from_raw_parts_mut(data.0 as *mut u8, size as usize)};
- let mut read = 0;
-
- while read < size {
- reader.seek(SeekFrom::Start(offset + read)).await?;
- let bytes = reader.read(&mut buf[read as usize..]).await?;
-
- if bytes == 0 {
- // EOF
- break;
- }
-
- read += bytes as u64;
- }
-
+ let buf: &mut [u8] =
+ unsafe { std::slice::from_raw_parts_mut(data.0 as *mut u8, size as usize) };
+ let read = reader.read_at(buf, offset).await?;
Ok(read.try_into()?)
}
}
--
2.30.2
next prev parent reply other threads:[~2021-06-07 15:35 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-07 15:35 [pbs-devel] [PATCH v2 0/9] Improve live-restore speed and replace AsyncIndexReader Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 1/9] tools/BroadcastFuture: add testcase for better understanding Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 2/9] tools: add AsyncLruCache as a wrapper around sync LruCache Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 3/9] backup: add CachedChunkReader utilizing AsyncLruCache Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 4/9] backup: add AsyncRead/Seek to CachedChunkReader Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 5/9] replace AsyncIndexReader with SeekableCachedChunkReader Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 6/9] backup: remove AsyncIndexReader Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 7/9] tools/lru_cache: make minimum capacity 1 Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup-qemu 8/9] add shared_cache module Stefan Reiter
2021-06-07 15:35 ` Stefan Reiter [this message]
2021-06-08 7:58 ` [pbs-devel] applied: [PATCH v2 0/9] Improve live-restore speed and replace AsyncIndexReader Wolfgang Bumiller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210607153532.2522267-10-s.reiter@proxmox.com \
--to=s.reiter@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox