From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 5353C1FF16B for ; Thu, 14 Nov 2024 10:40:17 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 47BAE2D335; Thu, 14 Nov 2024 10:40:18 +0100 (CET) Date: Thu, 14 Nov 2024 10:39:45 +0100 From: Gabriel Goller To: Christian Ebner Message-ID: References: <20241113155545.354045-1-c.ebner@proxmox.com> <20241113155545.354045-2-c.ebner@proxmox.com> MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: <20241113155545.354045-2-c.ebner@proxmox.com> User-Agent: NeoMutt/20241002-35-39f9a6 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.040 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pbs-devel] [PATCH v2 proxmox-backup 2/2] docs: deduplicate background details for garbage collection X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox Backup Server development discussion Cc: pbs-devel@lists.proxmox.com Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" On 13.11.2024 16:55, Christian Ebner wrote: >diff --git a/docs/maintenance.rst b/docs/maintenance.rst >index e8a26d69c..bba3feff4 100644 >--- a/docs/maintenance.rst >+++ b/docs/maintenance.rst >@@ -197,6 +197,8 @@ It's recommended to setup a schedule to ensure that unused space is cleaned up > periodically. For most setups a weekly schedule provides a good interval to > start. > >+.. _gc_background: >+ > GC Background > ^^^^^^^^^^^^^ > >@@ -222,17 +224,31 @@ datastore or interfering with other backups. > The garbage collection (GC) process is performed per datastore and is split > into two phases: > >-- Phase one: Mark >- All index files are read, and the access time of the referred chunk files is >- updated. >- >-- Phase two: Sweep >- The task iterates over all chunks, checks their file access time, and if it >- is older than the cutoff time (i.e., the time when GC started, plus some >- headroom for safety and Linux file system behavior), the task knows that the >- chunk was neither referred to in any backup index nor part of any currently >- running backup that has no index to scan for. As such, the chunk can be >- safely deleted. >+- Phase one (Mark): >+ >+ All index files are read, and the access time (``atime``) of the referenced >+ chunk files is updated. >+ >+- Phase two (Sweep): >+ >+ The task iterates over all chunks and checks their file access time against a >+ cutoff time. The cutoff time is given by either the oldest backup writer >+ instance, if present, or 24 hours and 5 minutes after the start of garbage >+ collection. >+ >+ Garbage collection can consider chunk files with access time older than the s/can consider/considers/ It always considers chunks with atime older than cutoff to be dangling afaik. >+ cutoff time to be neither referenced by any backup snapshot's index, nor part >+ of any currently running backup job. Therefore, these chunks can safeley be >+ deleted. s/safeley/safely/ >+ >+ Chunks within the grace period will not be deleted and logged at the end of >+ the garbage collection task as *Pending removals*. >+ >+.. note:: The grace period for backup chunk removal is not arbitrary, but stems >+ from the fact that filesystems are typically mounted with the ``relatime`` >+ option by default. This results in better performance by only updating the >+ ``atime`` property if a file has been modified since the last access or the >+ last access has been at least 24 hours ago. > > Manually Starting GC > ^^^^^^^^^^^^^^^^^^^^ Otherwise this is great! _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel