From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id B9BC11FF13B for ; Wed, 22 Apr 2026 21:25:57 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 97D3327C8A; Wed, 22 Apr 2026 21:25:57 +0200 (CEST) From: Thomas Lamprecht To: pbs-devel@lists.proxmox.com, Christian Ebner Subject: applied: [PATCH proxmox{,-backup} v8 00/10] fix #4182: concurrent group pull/push support for sync jobs Date: Wed, 22 Apr 2026 21:23:36 +0200 Message-ID: <177688579422.1297506.10413901646965954565.b4-ty@b4> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260422131820.769620-1-c.ebner@proxmox.com> References: <20260422131820.769620-1-c.ebner@proxmox.com> MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1776885835737 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.123 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LOTSOFHASH 0.25 Emails with lots of hash-like gibberish SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: RDWDAGYOI5U7KZE3JMRGWFAOBYTOETPT X-Message-ID-Hash: RDWDAGYOI5U7KZE3JMRGWFAOBYTOETPT X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Wed, 22 Apr 2026 15:18:10 +0200, Christian Ebner wrote: > Syncing contents from/to a remote source via a sync job suffers from > low throughput on high latency networks because of limitations by the > HTTP/2 connection, as described in [0]. To improve, syncing multiple > groups in parallel by establishing multiple reader instances has been > suggested. > > This patch series implements the functionality by adding the sync job > configuration property `worker-threads`, allowing to define the > number of groups pull/push tokio tasks to be executed in parallel on > the runtime during each job. > > [...] Applied, with some minor follow-ups, thanks! [1/9] tools: implement buffered logger for concurrent log messages commit: e842d62d9cf169058b19197f08dac72aebfbedeb [2/9] tools: add bounded join set to run concurrent tasks bound by limit commit: 75cd11b19e2041f10957405fec1372082926226d [3/9] api: config/sync: add optional `worker-threads` property commit: 8711d4552961d8b70c9b592a79461e8dabad9f28 [4/9] fix #4182: server: sync: allow pulling backup groups in parallel commit: fbff37959849ff79780622d68ac86ee6c74cb7b0 [5/9] server: pull: prefix log messages and add error context commit: 44e682c7892424e64797fb0a958e63fb29e3d52f [6/9] server: sync: allow pushing groups concurrently commit: 3b4cf51e727e38a99eb0277b44eb7efd73422aab [7/9] server: push: prefix log messages and add additional logging commit: 3be2b6f6073b1a066ddb4736fb05bc118151d6c6 [8/9] sync: move in-progress snapshot filter to helper and use log line sender commit: 38caf283dc69eef15625304a234a21ce4c3ef97e [9/9] ui: expose group worker setting in sync job edit window commit: cf09063fc5ebd8134941c98eb6ebd69af84da393