From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 45D861FF141 for ; Fri, 30 Jan 2026 17:46:07 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id BAFD828D8; Fri, 30 Jan 2026 17:46:33 +0100 (CET) From: Robert Obkircher To: pbs-devel@lists.proxmox.com Subject: [PATCH v5 proxmox-backup 00/16] fix: #3847 pipe from STDIN to proxmox-backup-client Date: Fri, 30 Jan 2026 17:45:24 +0100 Message-ID: <20260130164552.281581-1-r.obkircher@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1769791520503 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.052 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [lib.rs,push.rs,datastore.rs,environment.rs,mod.rs,main.rs] Message-ID-Hash: 6QLGH3NSG3FFZWOVPAY65PQHU5TNO3OE X-Message-ID-Hash: 6QLGH3NSG3FFZWOVPAY65PQHU5TNO3OE X-MailFrom: r.obkircher@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Add support for commands like: ssh host cmd | proxmox-backup-client backup data.img:/dev/stdin proxmox-backup-client backup a.img:<(mysqldump) b.img:<(pgdump) Changes since v4: - reordered the ChunkStore removal commit and adjusted dynamic writer - moved TempTestDir into separate file, use it for test_chunk_store1 - improved commit messages and comments - changed u32 param to u64 - removed unnecessary casts reported by clippy - added two commits at the end which restrict chunk_size to u32 (I'd fixup commit 12/16 if we actually want to keep those changes) An open question is if we should maybe fsync the data before renaming. Or is that too much overhead for slightly better crash consistency? Robert Obkircher (16): datastore: remove Arc from FixedIndexWriter datastore: remove Arc from DynamicIndexWriter datastore: add TempTestDir that is automatically deleted on drop datastore: use temporary directory for chunk store test datastore: support writing fidx files of unknown size datastore: test FixedIndexWriter api: backup: make fixed index file size optional api: verify fixed index writer size on close fix #3847: client: support fifo pipe inputs for images client: treat minus sign as stdin datastore: combine public FixedIndexWriter methods into add_chunk. datastore: use u64 instead of usize for fidx writer content size datastore: compute fidx file size with overflow checks datastore: support writing fidx files on systems with larger page size datastore: FixedIndexWriter: switch public chunk_size to u32 datastore: FixedIndexWriter: switch internal chunk_size to u32 pbs-client/src/backup_writer.rs | 38 ++- pbs-datastore/src/chunk_store.rs | 7 +- pbs-datastore/src/datastore.rs | 19 +- pbs-datastore/src/dynamic_index.rs | 5 +- pbs-datastore/src/fixed_index.rs | 435 +++++++++++++++++++++++++---- pbs-datastore/src/lib.rs | 3 + pbs-datastore/src/temp_test_dir.rs | 33 +++ proxmox-backup-client/src/main.rs | 37 ++- src/api2/backup/environment.rs | 33 ++- src/api2/backup/mod.rs | 6 +- src/server/push.rs | 11 +- 11 files changed, 499 insertions(+), 128 deletions(-) create mode 100644 pbs-datastore/src/temp_test_dir.rs -- 2.47.3