From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 5855D1FF13E for ; Fri, 09 Jan 2026 18:37:06 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B5E7E1BF20; Fri, 9 Jan 2026 18:37:07 +0100 (CET) From: Robert Obkircher To: pbs-devel@lists.proxmox.com Date: Fri, 9 Jan 2026 18:35:35 +0100 Message-ID: <20260109173548.301653-1-r.obkircher@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1767980154397 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.050 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [push.rs, datastore.rs, mod.rs, main.rs, environment.rs] Subject: [pbs-devel] [PATCH v3 proxmox-backup 0/5] fix: #3847 pipe from STDIN to proxmox-backup-client X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" Add support for commands like: ssh host cmd | proxmox-backup-client backup data.img:/dev/stdin proxmox-backup-client backup a.img:<(mysqldump) b.img:<(pgdump) Changes since v2: client: - Renamed ChunkSize to IndexType as suggested. datastore: - Introduced explicit `index_capacity` field. - Previously I (ab)used `index_length` for the capacity and computed the actual index length from `size`. - This removes the assert mentioned in the review. - Renamed INITIAL_CHUNKS_IF_UNKNOWN to INITIAL_CAPACITY and increased it to 128 blocks which is enough for 512 MiB of content. - Only remap when capacity increases, not every time size increases. - Increase capacity to next power of 2 instead of by 1.5x. - This is less code and file systems probably don't do in place updates where 1.5x is theoretically better. - Ensure that add_digest and close fail after remap errors. - Keep seek+write_all instead of write_all_at to minimize changes. - Imroved error messages in clone_data_from. - Refuse creation of empty files because the original also did that. - Added tests for FixedIndexWriter: - Is it okay to to write to a directory in the cwd? - This was inspired by `chunk_store::test_chunk_store1` - Removed test script. Changes since v1: - use mremap+ftruncate instead of write_all_at - make the size API parameter optional instead of using 0 - use an enum to represent fixed/dynamic chunk size in UploadOptions - alias "-" to "/dev/stdin" - split changes into separate commits Robert Obkircher (5): fix #3847: datastore: support writing fidx files of unknown size fix #3847: datastore: test FixedIndexWriter fix #3847: api: backup: make fixed index file size optional fix #3847: client: support fifo pipe inputs for images fix #3847: client: treat minus sign as stdin pbs-client/src/backup_writer.rs | 38 ++-- pbs-datastore/src/datastore.rs | 2 +- pbs-datastore/src/fixed_index.rs | 304 +++++++++++++++++++++++++++++- proxmox-backup-client/src/main.rs | 37 ++-- src/api2/backup/environment.rs | 8 +- src/api2/backup/mod.rs | 4 +- src/server/push.rs | 11 +- 7 files changed, 366 insertions(+), 38 deletions(-) -- 2.47.3 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel