all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Robert Obkircher <r.obkircher@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [PATCH v6 proxmox-backup 13/18] client: don't poll terminated source in ChunkStream
Date: Tue, 10 Feb 2026 16:06:29 +0100	[thread overview]
Message-ID: <20260210150642.469670-14-r.obkircher@proxmox.com> (raw)
In-Reply-To: <20260210150642.469670-1-r.obkircher@proxmox.com>

The ChunkStream may return a partial chunk after the input is done.
This is fine, but it must remember to not poll the input again when it
is polled one more time.

The DummyInput of the test case now verifies that it isn't polled
after it returned None.

Signed-off-by: Robert Obkircher <r.obkircher@proxmox.com>
---
 pbs-client/src/chunk_stream.rs | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/pbs-client/src/chunk_stream.rs b/pbs-client/src/chunk_stream.rs
index 94a45ac7..9763bbd7 100644
--- a/pbs-client/src/chunk_stream.rs
+++ b/pbs-client/src/chunk_stream.rs
@@ -34,6 +34,7 @@ impl InjectionData {
 /// Split input stream into dynamic sized chunks
 pub struct ChunkStream<S: Unpin> {
     input: S,
+    input_is_done: bool,
     chunker: Box<dyn Chunker + Send>,
     buffer: BytesMut,
     scan_pos: usize,
@@ -51,6 +52,7 @@ impl<S: Unpin> ChunkStream<S> {
         let chunk_size = chunk_size.unwrap_or(4 * 1024 * 1024);
         Self {
             input,
+            input_is_done: false,
             chunker: if let Some(suggested) = suggested_boundaries {
                 Box::new(PayloadChunker::new(chunk_size, suggested))
             } else {
@@ -162,12 +164,16 @@ where
                 }
             }
 
+            if this.input_is_done {
+                return Poll::Ready(None);
+            }
             match ready!(Pin::new(&mut this.input).try_poll_next(cx)) {
                 Some(Err(err)) => {
                     return Poll::Ready(Some(Err(err.into())));
                 }
                 None => {
                     this.scan_pos = 0;
+                    this.input_is_done = true;
                     if !this.buffer.is_empty() {
                         return Poll::Ready(Some(Ok(this.buffer.split())));
                     } else {
@@ -254,11 +260,12 @@ mod test {
 
     struct DummyInput {
         data: Vec<u8>,
+        done: bool,
     }
 
     impl DummyInput {
         fn new(data: Vec<u8>) -> Self {
-            Self { data }
+            Self { data, done: false }
         }
     }
 
@@ -268,7 +275,11 @@ mod test {
         fn poll_next(self: Pin<&mut Self>, _cx: &mut Context) -> Poll<Option<Self::Item>> {
             let this = self.get_mut();
             match this.data.len() {
-                0 => Poll::Ready(None),
+                0 => {
+                    assert!(!this.done);
+                    this.done = true;
+                    Poll::Ready(None)
+                }
                 size if size > 10 => Poll::Ready(Some(Ok(this.data.split_off(10)))),
                 _ => Poll::Ready(Some(Ok(std::mem::take(&mut this.data)))),
             }
-- 
2.47.3





  parent reply	other threads:[~2026-02-10 15:07 UTC|newest]

Thread overview: 24+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-02-10 15:06 [PATCH v6 proxmox-backup 00/18] fix: #3847 pipe from STDIN to proxmox-backup-client Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 01/18] datastore: remove Arc<ChunkStore> from FixedIndexWriter Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 02/18] datastore: remove Arc<ChunkStore> from DynamicIndexWriter Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 03/18] datastore: add TempDir that is automatically deleted on drop Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 04/18] datastore: use temporary directory for chunk store test Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 05/18] datastore: combine public FixedIndexWriter methods into add_chunk Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 06/18] datastore: use fixed size types for FixedIndexWriter Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 07/18] datastore: verify that chunk_size is a power of two Robert Obkircher
2026-02-17  9:13   ` Robert Obkircher
2026-02-17  9:40     ` Christian Ebner
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 08/18] datastore: support writing fidx files of unknown size Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 09/18] datastore: test FixedIndexWriter Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 10/18] api: backup: make fixed index file size optional Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 11/18] api: verify fixed index writer size on close Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 12/18] client: don't poll terminated source in FixedChunkStream Robert Obkircher
2026-02-17 10:01   ` Christian Ebner
2026-02-17 10:06     ` Christian Ebner
2026-02-10 15:06 ` Robert Obkircher [this message]
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 14/18] fix #3847: client: support fifo pipe inputs for image backups Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 15/18] client: Fail early if the same pipe is specified for multiple inputs Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 16/18] datastore: compute fidx file size with overflow checks Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 17/18] datastore: support writing fidx files on systems with larger page size Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 18/18] datastore: support incremental fidx uploads with different size Robert Obkircher
2026-02-17 12:42 ` [PATCH v6 proxmox-backup 00/18] fix: #3847 pipe from STDIN to proxmox-backup-client Christian Ebner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260210150642.469670-14-r.obkircher@proxmox.com \
    --to=r.obkircher@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal