From: "Fabian Grünbichler" <f.gruenbichler@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup 0/3] reduce GC S3 locking
Date: Fri, 21 Nov 2025 10:05:58 +0100 [thread overview]
Message-ID: <20251121090605.262675-1-f.gruenbichler@proxmox.com> (raw)
this patch series tries to reduce the number of open locks held by GC,
in particular in case most objects returned by the S3 backend are
garbage that need deletion.
the first patch reduces the number of open locks by at least a factor of
10 in the worst case (from up to 1000 to up to 100).
the second patch just refactors some now common code.
the third patch tries to reduce the number of delete calls for regular
GC runs by batching deletes more efficiently.
Fabian Grünbichler (3):
GC: S3: reduce number of open FDs for to-be-deleted objects
GC: S3: factor out batch object deletion
GC: S3: phase2: delete last partial batch of objects at the very end
pbs-datastore/src/datastore.rs | 49 +++++++++++++++++++++++-----------
1 file changed, 33 insertions(+), 16 deletions(-)
--
2.47.3
_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
next reply other threads:[~2025-11-21 9:06 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-11-21 9:05 Fabian Grünbichler [this message]
2025-11-21 9:05 ` [pbs-devel] [PATCH proxmox-backup 1/3] GC: S3: reduce number of open FDs for to-be-deleted objects Fabian Grünbichler
2025-11-21 9:43 ` Christian Ebner
2025-11-21 9:06 ` [pbs-devel] [PATCH proxmox-backup 2/3] GC: S3: factor out batch object deletion Fabian Grünbichler
2025-11-21 9:06 ` [pbs-devel] [PATCH proxmox-backup 3/3] GC: S3: phase2: delete last partial batch of objects at the very end Fabian Grünbichler
2025-11-21 9:31 ` Christian Ebner
2025-11-21 9:46 ` Fabian Grünbichler
2025-11-21 9:53 ` Christian Ebner
2025-11-21 10:05 ` [pbs-devel] [PATCH proxmox-backup 0/3] reduce GC S3 locking Christian Ebner
2025-11-21 10:19 ` [pbs-devel] superseded: " Fabian Grünbichler
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20251121090605.262675-1-f.gruenbichler@proxmox.com \
--to=f.gruenbichler@proxmox.com \
--cc=pbs-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox