From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 470221FF139 for ; Tue, 10 Feb 2026 16:06:47 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id F0F37671F; Tue, 10 Feb 2026 16:07:31 +0100 (CET) From: Robert Obkircher To: pbs-devel@lists.proxmox.com Subject: [PATCH v6 proxmox-backup 00/18] fix: #3847 pipe from STDIN to proxmox-backup-client Date: Tue, 10 Feb 2026 16:06:16 +0100 Message-ID: <20260210150642.469670-1-r.obkircher@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1770735933742 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.051 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: IYQWVZ6WDHAEHDFBG4S5Z6UAEG5BBJRU X-Message-ID-Hash: IYQWVZ6WDHAEHDFBG4S5Z6UAEG5BBJRU X-MailFrom: r.obkircher@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Add support for commands like: ssh host cmd | proxmox-backup-client backup data.img:/dev/stdin proxmox-backup-client backup a.img:<(mysqldump) b.img:<(pgdump) Changes since v5: Client: - add IndexType::to_prefix_and_size helper and fix format string nit - drop support for '-' but log helpful warnings - fail early for duplicate pipes - avoid calling poll_next on terminated streams Server: - mention why Arc can be removed from didx as well - (Test)TempDir: - add to parent mod in earlier commit - drop "Test" from the name - improved API so it could be moved to a crate (no unwrap in new, no Deref, no impl From<&TempDir> for PathBuf) - added tests - reorder add_chunk commit before grow_to_size - support reuse-csum with arbitrary sizes - made up my mind on u32 vs u64 and reordered commits - check that chunk_size is power of two Robert Obkircher (18): datastore: remove Arc from FixedIndexWriter datastore: remove Arc from DynamicIndexWriter datastore: add TempDir that is automatically deleted on drop datastore: use temporary directory for chunk store test datastore: combine public FixedIndexWriter methods into add_chunk datastore: use fixed size types for FixedIndexWriter datastore: verify that chunk_size is a power of two datastore: support writing fidx files of unknown size datastore: test FixedIndexWriter api: backup: make fixed index file size optional api: verify fixed index writer size on close client: don't poll terminated source in FixedChunkStream client: don't poll terminated source in ChunkStream fix #3847: client: support fifo pipe inputs for image backups client: Fail early if the same pipe is specified for multiple inputs datastore: compute fidx file size with overflow checks datastore: support writing fidx files on systems with larger page size datastore: support incremental fidx uploads with different size pbs-client/src/backup_writer.rs | 37 ++- pbs-client/src/chunk_stream.rs | 23 +- pbs-datastore/src/chunk_store.rs | 14 +- pbs-datastore/src/datastore.rs | 19 +- pbs-datastore/src/dynamic_index.rs | 5 +- pbs-datastore/src/fixed_index.rs | 513 +++++++++++++++++++++++++---- pbs-datastore/src/lib.rs | 3 + pbs-datastore/src/temp_dir.rs | 142 ++++++++ proxmox-backup-client/src/main.rs | 55 +++- src/api2/backup/environment.rs | 37 ++- src/api2/backup/mod.rs | 6 +- src/server/push.rs | 11 +- 12 files changed, 725 insertions(+), 140 deletions(-) create mode 100644 pbs-datastore/src/temp_dir.rs -- 2.47.3