From: Christian Ebner <c.ebner@proxmox.com>
To: Shan Shaji <s.shaji@proxmox.com>, pbs-devel@lists.proxmox.com
Subject: Re: [PATCH proxmox-backup 4/9] datastore: add method to create zero-byte chunk markers
Date: Tue, 17 Nov 2026 14:26:07 +0100 [thread overview]
Message-ID: <db5e4467-ccdd-4cf3-883a-d2e88afa1db1@proxmox.com> (raw)
In-Reply-To: <DMXIHLAR8L2E.144KCUPJHPZU6@proxmox.com>
On 11/17/26 2:14 PM, Shan Shaji wrote:
> On Wed Nov 11, 2026 at 4:04 PM CET, Christian Ebner wrote:
>> One comment inline.
[..]
>> comment: the object keys here do also potentially include bad chunks
>> with their respective suffix, so the markers would be created for these
>> as well, leading to no further good chunks being uploaded and corrupt
>> backups.
>
> Thank you for catching this. I think I should either skip marker
> creation for bad chunks or create the corresponding .N.bad marker files.
> I haven't tested this yet. I'll check and get back.
>
>> Further I think there still is a pre-existing issue with the marker
>> files for bad chunks potentially being missing, which would lead to
>> removing these from the S3 backend even if no new, good chunk exists.
>>
>
> Do you mean the garbage-collection code path in [1]? If I understand
> correctly, phase 1 updates existing local .N.bad files while iterating
> over the index files [2], but does not recreate missing ones. During
> phase 2, if the corresponding local .N.bad file is missing and
> clear_chunk_expected_mark() returns false, atime becomes None. This
> causes the bad chunk object to be queued for deletion from S3,
> even if no good replacement exists.
>
> - [1] https://git.proxmox.com/?p=proxmox-backup.git;a=blob;f=pbs-datastore/src/datastore.rs;h=e6766a3b93c6c350d8ac5150a6e23f86a7659af9;hb=HEAD#l2544
> - [2] https://git.proxmox.com/?p=proxmox-backup.git;a=blob;f=pbs-datastore/src/datastore.rs;h=e6766a3b93c6c350d8ac5150a6e23f86a7659af9;hb=HEAD#l2175
Yes, as mentioned off-list, this is incomplete and should be covered as
well. bad files should not be cleaned up by GC unless there is a good
chunk present, so if no good chunk marker does (yet) exist, and the bad
one does neither, it would be best to create it and avoid cleanup. That
way we could recover any missing bad chunk markers after a datastore
recreation from a bucket I think.
next prev parent reply other threads:[~2026-11-17 13:26 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-19 16:03 [PATCH proxmox{-backup,} 0/9] fix #7903: rebuild local S3 chunk markers Shan Shaji
2026-10-19 16:03 ` [PATCH proxmox-backup 1/9] refactor: remove local `LogThrottle` in favor of proxmox-log Shan Shaji
2026-10-19 16:03 ` [PATCH proxmox-backup 2/9] datastore: factor out s3 chunk objects pagination into a helper Shan Shaji
2026-10-19 16:03 ` [PATCH proxmox-backup 3/9] pbs-datastore: inherit proxmox-log crate from workspace Shan Shaji
2026-11-11 15:05 ` Christian Ebner
2026-10-19 16:03 ` [PATCH proxmox-backup 4/9] datastore: add method to create zero-byte chunk markers Shan Shaji
2026-11-11 15:04 ` Christian Ebner
2026-11-17 13:14 ` Shan Shaji
2026-11-17 13:26 ` Christian Ebner [this message]
2026-10-19 16:03 ` [PATCH proxmox-backup 5/9] fix #7908: datastore: api: add option " Shan Shaji
2026-10-19 16:03 ` [PATCH proxmox-backup 6/9] fix #7908: datastore: ui: " Shan Shaji
2026-10-19 16:03 ` [PATCH proxmox-backup 7/9] fix #7908: backup-manager: " Shan Shaji
2026-10-19 16:03 ` [PATCH proxmox-backup 8/9] docs: add explanation regarding ``rebuild-chunk-markers`` option Shan Shaji
2026-11-11 15:04 ` Christian Ebner
2026-11-16 14:38 ` Shan Shaji
2026-10-19 16:03 ` [PATCH proxmox 9/9] proxmox-log: add LogThrottle implementation Shan Shaji
2026-11-11 15:05 ` Christian Ebner
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=db5e4467-ccdd-4cf3-883a-d2e88afa1db1@proxmox.com \
--to=c.ebner@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
--cc=s.shaji@proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.