From: Stefan Reiter <s.reiter@proxmox.com>
To: pve-devel@lists.proxmox.com, pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH v2 proxmox-backup-qemu 8/9] add shared_cache module
Date: Mon, 7 Jun 2021 17:35:31 +0200 [thread overview]
Message-ID: <20210607153532.2522267-9-s.reiter@proxmox.com> (raw)
In-Reply-To: <20210607153532.2522267-1-s.reiter@proxmox.com>
Provides a shared AsyncLruCache of 256MB (w/ 4MB chunks) that can be
used by multiple readers at the same time. It is dropped once no more
readers exist, so the memory gets freed if all QEMU block/pbs instances
disappear.
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
v2:
* adapt to dropping ChunkCache type alias from proxmox-backup
* I re-add it here locally only because otherwise the type of SHARED_CACHE
ends in 7 '>'s
src/lib.rs | 7 ++++++-
src/shared_cache.rs | 37 +++++++++++++++++++++++++++++++++++++
2 files changed, 43 insertions(+), 1 deletion(-)
create mode 100644 src/shared_cache.rs
diff --git a/src/lib.rs b/src/lib.rs
index 05d7b58..aa167f7 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -25,6 +25,7 @@ mod restore;
use restore::*;
mod tools;
+mod shared_cache;
pub const PROXMOX_BACKUP_DEFAULT_CHUNK_SIZE: u64 = 1024*1024*4;
@@ -804,7 +805,11 @@ pub extern "C" fn proxmox_restore_connect_async(
pub extern "C" fn proxmox_restore_disconnect(handle: *mut ProxmoxRestoreHandle) {
let restore_task = handle as * mut Arc<RestoreTask>;
- unsafe { Box::from_raw(restore_task) }; //drop(restore_task)
+ let restore_task = unsafe { Box::from_raw(restore_task) };
+ drop(restore_task);
+
+ // after dropping, cache may be unused (if no other handles open)
+ shared_cache::shared_chunk_cache_cleanup();
}
/// Restore an image (sync)
diff --git a/src/shared_cache.rs b/src/shared_cache.rs
new file mode 100644
index 0000000..32ac430
--- /dev/null
+++ b/src/shared_cache.rs
@@ -0,0 +1,37 @@
+use once_cell::sync::OnceCell;
+use proxmox_backup::tools::async_lru_cache::AsyncLruCache;
+use std::sync::{Arc, Mutex};
+
+type ChunkCache = AsyncLruCache<[u8; 32], Arc<Vec<u8>>>;
+
+const SHARED_CACHE_CAPACITY: usize = 64; // 256 MB
+static SHARED_CACHE: OnceCell<Mutex<Option<Arc<ChunkCache>>>> = OnceCell::new();
+
+pub fn get_shared_chunk_cache() -> Arc<ChunkCache> {
+ let mut guard = SHARED_CACHE
+ .get_or_init(|| Mutex::new(None))
+ .lock()
+ .unwrap();
+ match &*guard {
+ Some(cache) => Arc::clone(cache),
+ None => {
+ let cache = Arc::new(AsyncLruCache::new(SHARED_CACHE_CAPACITY));
+ *guard = Some(Arc::clone(&cache));
+ cache
+ }
+ }
+}
+
+pub fn shared_chunk_cache_cleanup() {
+ let mut guard = SHARED_CACHE
+ .get_or_init(|| Mutex::new(None))
+ .lock()
+ .unwrap();
+ if let Some(arc) = guard.as_ref() {
+ let refcount = Arc::strong_count(arc);
+ if refcount == 1 {
+ // no one else is using the cache anymore, drop it
+ let _drop = guard.take();
+ }
+ }
+}
--
2.30.2
next prev parent reply other threads:[~2021-06-07 15:35 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-07 15:35 [pbs-devel] [PATCH v2 0/9] Improve live-restore speed and replace AsyncIndexReader Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 1/9] tools/BroadcastFuture: add testcase for better understanding Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 2/9] tools: add AsyncLruCache as a wrapper around sync LruCache Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 3/9] backup: add CachedChunkReader utilizing AsyncLruCache Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 4/9] backup: add AsyncRead/Seek to CachedChunkReader Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 5/9] replace AsyncIndexReader with SeekableCachedChunkReader Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 6/9] backup: remove AsyncIndexReader Stefan Reiter
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup 7/9] tools/lru_cache: make minimum capacity 1 Stefan Reiter
2021-06-07 15:35 ` Stefan Reiter [this message]
2021-06-07 15:35 ` [pbs-devel] [PATCH v2 proxmox-backup-qemu 9/9] access: use CachedChunkReader Stefan Reiter
2021-06-08 7:58 ` [pbs-devel] applied: [PATCH v2 0/9] Improve live-restore speed and replace AsyncIndexReader Wolfgang Bumiller
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210607153532.2522267-9-s.reiter@proxmox.com \
--to=s.reiter@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox