From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 35BE81FF13A for ; Wed, 15 Apr 2026 13:56:15 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id CC97C10F4C; Wed, 15 Apr 2026 13:56:14 +0200 (CEST) Message-ID: <53ba7742-4349-49f8-ba1d-83b6d7bce16a@proxmox.com> Date: Wed, 15 Apr 2026 13:56:11 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH proxmox-backup 1/2] fix #6373: HTTP level reader heartbeat for proxy connection keepalive To: =?UTF-8?Q?Fabian_Gr=C3=BCnbichler?= , pbs-devel@lists.proxmox.com References: <20260129122700.448448-1-c.ebner@proxmox.com> <20260129122700.448448-3-c.ebner@proxmox.com> <1776240724.q4ii7heihr.astroid@yuna.none> <76864022-8da1-40d4-82e3-be681a6f1c2e@proxmox.com> <1776250781.4vefy5jjsj.astroid@yuna.none> <1776252750.j77nhquhlu.astroid@yuna.none> Content-Language: en-US, de-DE From: Christian Ebner In-Reply-To: <1776252750.j77nhquhlu.astroid@yuna.none> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1776254094383 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.069 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: 4WENGXHJXPHZOMIP4OQZSL3TETPQJS5L X-Message-ID-Hash: 4WENGXHJXPHZOMIP4OQZSL3TETPQJS5L X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 4/15/26 1:47 PM, Fabian Grünbichler wrote: > On April 15, 2026 1:22 pm, Christian Ebner wrote: >> On 4/15/26 12:59 PM, Fabian Grünbichler wrote: >>> On April 15, 2026 10:45 am, Christian Ebner wrote: >>>> On 4/15/26 10:32 AM, Fabian Grünbichler wrote: >>>>> On January 29, 2026 1:26 pm, Christian Ebner wrote: >>>>>> Backup readers can have long periods of idle connections, e.g. if a >>>>>> backup snapshot has been mounted and all relevant chunks are locally >>>>>> cached or a backup session with previous metadata archive not needing >>>>>> to fetch new contents while the backup is ongoing. >>>>>> >>>>>> Proxies like e.g. HAProxy might however close idle connections for >>>>>> better resource handling [0,1], even multiplexed HTTP/2 connections as >>>>>> are being used for the Proxmox Backup Sever backup/reader protocol. >>>>>> >>>>>> This mainly affects the backup reader, while the backup writer will >>>>>> do indexing and chunk uploads anyways. >>>>> >>>>> but if the storage is slow, there might not be chunk traffic for a few >>>>> seconds as well? >>>> So you suggest to implement the same for the backup writer as well? >>> >>> I am wondering whether it wouldn't make sense (though I guess 5s is >>> quite agressive anyway, and higher idle timeouts make it unlikely to >>> trigger in practice?) >> >> >> Yes, I think for the writer this is way less relevant. >> Only the super slow storage case where the chunk is uploaded quickly, >> but then the storage takes ages to write it. > > I was more worried about the *reading* being slow client-side, and thus > the chunk stream having big pauses between chunks, in particular when a > lot/most/all chunks are read, but not uploaded. > > e.g., if I ratelimit the input side to 10MB/s for an incremental backup, > there's long breaks between traffic from proxmox-backup-client to > proxmox-backup-proxy, roughly every 2m there's some sort of lower-level > keep-alive mechanism, and otherwise there is only traffic whenever the > client has read 128 re-used chunks (with 2.5 chunks/second that takes a > while in this example ;)) or some non-incremental data comes along and > forces a "flush". Yeah, so we can conclude it makes sense to have it for the writer as well. So if agreed, I would add this controlled by PBS_WRITER_HEARTBEAT_TIMEOUT then.