From: Robert Obkircher <r.obkircher@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [PATCH v6 proxmox-backup 12/18] client: don't poll terminated source in FixedChunkStream
Date: Tue, 10 Feb 2026 16:06:28 +0100 [thread overview]
Message-ID: <20260210150642.469670-13-r.obkircher@proxmox.com> (raw)
In-Reply-To: <20260210150642.469670-1-r.obkircher@proxmox.com>
Fix incorrect chunking when the input file is a pts character device
(stdin in an interactive terminal). Previously, the stream could
produce multiple smaller chunks from interactive input, and a single
Ctrl+D was not enough to terminate it. This was because a None from
the source resulted in Some partial chunk, but the next time the input
was polled again and it suddenly had more input.
The documentation of Stream::poll_next clearly states that calling it
on a terminated stream may panic, block forever, or cause other kinds
of problems, so this should be fixed even it it appeared to work with
normal files.
Using the fuse method or adding a FusedStream trait bound might be
nicer that seemed like a larger change compared to the boolean.
Signed-off-by: Robert Obkircher <r.obkircher@proxmox.com>
---
pbs-client/src/chunk_stream.rs | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/pbs-client/src/chunk_stream.rs b/pbs-client/src/chunk_stream.rs
index e3f0980c..94a45ac7 100644
--- a/pbs-client/src/chunk_stream.rs
+++ b/pbs-client/src/chunk_stream.rs
@@ -187,6 +187,7 @@ pub struct FixedChunkStream<S: Unpin> {
input: S,
chunk_size: usize,
buffer: BytesMut,
+ done: bool,
}
impl<S: Unpin> FixedChunkStream<S> {
@@ -195,6 +196,7 @@ impl<S: Unpin> FixedChunkStream<S> {
input,
chunk_size,
buffer: BytesMut::new(),
+ done: false,
}
}
}
@@ -213,6 +215,9 @@ where
cx: &mut Context,
) -> Poll<Option<Result<BytesMut, S::Error>>> {
let this = self.get_mut();
+ if this.done {
+ return Poll::Ready(None);
+ }
loop {
if this.buffer.len() >= this.chunk_size {
return Poll::Ready(Some(Ok(this.buffer.split_to(this.chunk_size))));
@@ -223,6 +228,9 @@ where
return Poll::Ready(Some(Err(err)));
}
None => {
+ // Must not call input.try_poll_next again!
+ this.done = true;
+
// last chunk can have any size
if !this.buffer.is_empty() {
return Poll::Ready(Some(Ok(this.buffer.split())));
--
2.47.3
next prev parent reply other threads:[~2026-02-10 15:06 UTC|newest]
Thread overview: 24+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-02-10 15:06 [PATCH v6 proxmox-backup 00/18] fix: #3847 pipe from STDIN to proxmox-backup-client Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 01/18] datastore: remove Arc<ChunkStore> from FixedIndexWriter Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 02/18] datastore: remove Arc<ChunkStore> from DynamicIndexWriter Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 03/18] datastore: add TempDir that is automatically deleted on drop Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 04/18] datastore: use temporary directory for chunk store test Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 05/18] datastore: combine public FixedIndexWriter methods into add_chunk Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 06/18] datastore: use fixed size types for FixedIndexWriter Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 07/18] datastore: verify that chunk_size is a power of two Robert Obkircher
2026-02-17 9:13 ` Robert Obkircher
2026-02-17 9:40 ` Christian Ebner
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 08/18] datastore: support writing fidx files of unknown size Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 09/18] datastore: test FixedIndexWriter Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 10/18] api: backup: make fixed index file size optional Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 11/18] api: verify fixed index writer size on close Robert Obkircher
2026-02-10 15:06 ` Robert Obkircher [this message]
2026-02-17 10:01 ` [PATCH v6 proxmox-backup 12/18] client: don't poll terminated source in FixedChunkStream Christian Ebner
2026-02-17 10:06 ` Christian Ebner
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 13/18] client: don't poll terminated source in ChunkStream Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 14/18] fix #3847: client: support fifo pipe inputs for image backups Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 15/18] client: Fail early if the same pipe is specified for multiple inputs Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 16/18] datastore: compute fidx file size with overflow checks Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 17/18] datastore: support writing fidx files on systems with larger page size Robert Obkircher
2026-02-10 15:06 ` [PATCH v6 proxmox-backup 18/18] datastore: support incremental fidx uploads with different size Robert Obkircher
2026-02-17 12:42 ` [PATCH v6 proxmox-backup 00/18] fix: #3847 pipe from STDIN to proxmox-backup-client Christian Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260210150642.469670-13-r.obkircher@proxmox.com \
--to=r.obkircher@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.