From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 0B31A1FF13F for ; Wed, 14 Jan 2026 14:13:36 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B1E9312636; Wed, 14 Jan 2026 14:13:32 +0100 (CET) Date: Wed, 14 Jan 2026 14:13:27 +0100 From: Fabian =?iso-8859-1?q?Gr=FCnbichler?= To: Proxmox Backup Server development discussion References: <20260109173548.301653-1-r.obkircher@proxmox.com> In-Reply-To: <20260109173548.301653-1-r.obkircher@proxmox.com> MIME-Version: 1.0 User-Agent: astroid/0.17.0 (https://github.com/astroidmail/astroid) Message-Id: <1768394944.5orvv5pwav.astroid@yuna.none> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1768396364301 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.047 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [mod.rs, environment.rs, push.rs, main.rs, datastore.rs, proxmox.com] Subject: Re: [pbs-devel] [PATCH v3 proxmox-backup 0/5] fix: #3847 pipe from STDIN to proxmox-backup-client X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" On January 9, 2026 6:35 pm, Robert Obkircher wrote: > Add support for commands like: > ssh host cmd | proxmox-backup-client backup data.img:/dev/stdin > proxmox-backup-client backup a.img:<(mysqldump) b.img:<(pgdump) this is already shaping up nicely, some comments on individual patches, and some feedback for the organization of the series: the actual fix for the referenced bug is the last two patches - the rest is just preparation work but has no effect yet until it is wired up in the client (AFAICT?). that means that the "fix .." prefix should only go into those two patches (or technically even just #4, since #5 is just syntactic sugar on top). please ensure each commit builds when applied in series. the first patch breaks the backup API and should include changes making it compatible (i.e., wrapping things in Some(..) as needed). the third patch would then replace those compat changes with making it optional in the API and handling that correctly. thanks! > > Changes since v2: > > client: > - Renamed ChunkSize to IndexType as suggested. > > datastore: > - Introduced explicit `index_capacity` field. > - Previously I (ab)used `index_length` for the capacity and > computed the actual index length from `size`. > - This removes the assert mentioned in the review. > - Renamed INITIAL_CHUNKS_IF_UNKNOWN to INITIAL_CAPACITY and increased > it to 128 blocks which is enough for 512 MiB of content. > - Only remap when capacity increases, not every time size increases. > - Increase capacity to next power of 2 instead of by 1.5x. > - This is less code and file systems probably don't do in place > updates where 1.5x is theoretically better. > - Ensure that add_digest and close fail after remap errors. > - Keep seek+write_all instead of write_all_at to minimize changes. > - Imroved error messages in clone_data_from. > - Refuse creation of empty files because the original also did that. > - Added tests for FixedIndexWriter: > - Is it okay to to write to a directory in the cwd? > - This was inspired by `chunk_store::test_chunk_store1` > - Removed test script. > > Changes since v1: > - use mremap+ftruncate instead of write_all_at > - make the size API parameter optional instead of using 0 > - use an enum to represent fixed/dynamic chunk size in UploadOptions > - alias "-" to "/dev/stdin" > - split changes into separate commits > > Robert Obkircher (5): > fix #3847: datastore: support writing fidx files of unknown size > fix #3847: datastore: test FixedIndexWriter > fix #3847: api: backup: make fixed index file size optional > fix #3847: client: support fifo pipe inputs for images > fix #3847: client: treat minus sign as stdin > > pbs-client/src/backup_writer.rs | 38 ++-- > pbs-datastore/src/datastore.rs | 2 +- > pbs-datastore/src/fixed_index.rs | 304 +++++++++++++++++++++++++++++- > proxmox-backup-client/src/main.rs | 37 ++-- > src/api2/backup/environment.rs | 8 +- > src/api2/backup/mod.rs | 4 +- > src/server/push.rs | 11 +- > 7 files changed, 366 insertions(+), 38 deletions(-) > > -- > 2.47.3 > > > > _______________________________________________ > pbs-devel mailing list > pbs-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel > > > _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel