From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id A10721FF0AA for ; Tue, 17 Nov 2026 14:26:13 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 33A8821590; Tue, 17 Nov 2026 14:26:13 +0100 (CET) Message-ID: Date: Tue, 17 Nov 2026 14:26:07 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH proxmox-backup 4/9] datastore: add method to create zero-byte chunk markers To: Shan Shaji , pbs-devel@lists.proxmox.com References: <20261019160359.20851-1-s.shaji@proxmox.com> <20261019160359.20851-5-s.shaji@proxmox.com> <3f1efc7a-04ed-4dfa-9c8f-2d2eb9aeaa68@proxmox.com> Content-Language: en-US, de-DE From: Christian Ebner In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794921968575 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.374 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: SNAYNGTATJIW3BVMKEUHIZD42O6NTPAW X-Message-ID-Hash: SNAYNGTATJIW3BVMKEUHIZD42O6NTPAW X-MailFrom: c.ebner@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox Backup Server development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 11/17/26 2:14 PM, Shan Shaji wrote: > On Wed Nov 11, 2026 at 4:04 PM CET, Christian Ebner wrote: >> One comment inline. [..] >> comment: the object keys here do also potentially include bad chunks >> with their respective suffix, so the markers would be created for these >> as well, leading to no further good chunks being uploaded and corrupt >> backups. > > Thank you for catching this. I think I should either skip marker > creation for bad chunks or create the corresponding .N.bad marker files. > I haven't tested this yet. I'll check and get back. > >> Further I think there still is a pre-existing issue with the marker >> files for bad chunks potentially being missing, which would lead to >> removing these from the S3 backend even if no new, good chunk exists. >> > > Do you mean the garbage-collection code path in [1]? If I understand > correctly, phase 1 updates existing local .N.bad files while iterating > over the index files [2], but does not recreate missing ones. During > phase 2, if the corresponding local .N.bad file is missing and > clear_chunk_expected_mark() returns false, atime becomes None. This > causes the bad chunk object to be queued for deletion from S3, > even if no good replacement exists. > > - [1] https://git.proxmox.com/?p=proxmox-backup.git;a=blob;f=pbs-datastore/src/datastore.rs;h=e6766a3b93c6c350d8ac5150a6e23f86a7659af9;hb=HEAD#l2544 > - [2] https://git.proxmox.com/?p=proxmox-backup.git;a=blob;f=pbs-datastore/src/datastore.rs;h=e6766a3b93c6c350d8ac5150a6e23f86a7659af9;hb=HEAD#l2175 Yes, as mentioned off-list, this is incomplete and should be covered as well. bad files should not be cleaned up by GC unless there is a good chunk present, so if no good chunk marker does (yet) exist, and the bad one does neither, it would be best to create it and avoid cleanup. That way we could recover any missing bad chunk markers after a datastore recreation from a bucket I think.