From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 022331FF13B for ; Wed, 22 Apr 2026 19:33:08 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D244126896; Wed, 22 Apr 2026 19:33:07 +0200 (CEST) Message-ID: Date: Wed, 22 Apr 2026 19:32:33 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta Subject: Re: [PATCH proxmox-backup v8 03/10] tools: add bounded join set to run concurrent tasks bound by limit To: Christian Ebner , pbs-devel@lists.proxmox.com References: <20260422131820.769620-1-c.ebner@proxmox.com> <20260422131820.769620-4-c.ebner@proxmox.com> Content-Language: en-US From: Thomas Lamprecht In-Reply-To: <20260422131820.769620-4-c.ebner@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1776879065391 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.002 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [lib.rs] Message-ID-Hash: 5T26FREXQZWKO72ORUKVLWSXNHKDS6RM X-Message-ID-Hash: 5T26FREXQZWKO72ORUKVLWSXNHKDS6RM X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Am 22.04.26 um 15:17 schrieb Christian Ebner: > The BoundedJoinSet allows to run tasks concurrently via a JoinSet, > but constrains the number of concurrent tasks by an upper limit. > > In contrast to the ParallelHandler implementation, which is a purely > sync implementation and does not provide easy handling for returned > results, this allows to execute tasks in an async context with straight > forward handling of results, as required for e.g. pulling/pushing of > backup groups in parallel for sync jobs. Also, log context is easily > preserved, which is of importance for task logging. > > Co-developed-by: Fabian Grünbichler > Signed-off-by: Christian Ebner > --- > pbs-tools/src/bounded_join_set.rs | 81 +++++++++++++++++++++++++++++++ > pbs-tools/src/lib.rs | 1 + > 2 files changed, 82 insertions(+) > create mode 100644 pbs-tools/src/bounded_join_set.rs > > diff --git a/pbs-tools/src/bounded_join_set.rs b/pbs-tools/src/bounded_join_set.rs > new file mode 100644 > index 000000000..07500c66b > --- /dev/null > +++ b/pbs-tools/src/bounded_join_set.rs > @@ -0,0 +1,81 @@ > +//! JoinSet with an upper bound of concurrent tasks. > +//! > +//! Allows to run up to the configured number of tasks concurrently in an async > +//! context. similar to my reply to the buffered log module: Might be nicer if this lives in the proxmox rust workspace, and in fact Lukas factored out the parallel-handler recently into its own crate, depending on the semantic and dependency overlap this might also go into the newish proxmox-parallel-handler crate or its own nano crate. But fine for now to be here, no need for another revision.