[pbs-devel] [RFC proxmox-backup 2/2] image backup: use 4M input buffer

["Fabian Grünbichler" <f.gruenbichler@proxmox.com>]

with the default 8k input buffer size, the client will spend most of the time
polling instead of reading/chunking/uploading.

tested with 16G random data file from tmpfs to fresh datastore backed by tmpfs,
without encryption.

stock:

Time (mean ± σ):     36.064 s ±  0.655 s    [User: 21.079 s, System: 26.415 s]
  Range (min … max):   35.663 s … 36.819 s    3 runs

patched:

 Time (mean ± σ):     23.591 s ±  0.807 s    [User: 16.532 s, System: 18.629 s]
  Range (min … max):   22.663 s … 24.125 s    3 runs

Summary
  patched ran
    1.53 ± 0.06 times faster than stock

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
obviously, this slightly increases memory usage..

the effect here is less pronounced than for the example because the actual
reading and chunking part is not all the client has to do for a backup - digest
calculation and TLS crypto make up the bulk of the rest..

 proxmox-backup-client/src/main.rs | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/proxmox-backup-client/src/main.rs b/proxmox-backup-client/src/main.rs
index 6a7d09047..5edb2a824 100644
--- a/proxmox-backup-client/src/main.rs
+++ b/proxmox-backup-client/src/main.rs
@@ -286,8 +286,12 @@ async fn backup_image<P: AsRef<Path>>(
 
     let file = tokio::fs::File::open(path).await?;
 
-    let stream = tokio_util::codec::FramedRead::new(file, tokio_util::codec::BytesCodec::new())
-        .map_err(Error::from);
+    let stream = tokio_util::codec::FramedRead::with_capacity(
+        file,
+        tokio_util::codec::BytesCodec::new(),
+        4 * 1024 * 1024,
+    )
+    .map_err(Error::from);
 
     let stream = FixedChunkStream::new(stream, chunk_size.unwrap_or(4 * 1024 * 1024));
 
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel