From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <pbs-devel-bounces@lists.proxmox.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 498661FF15C for <inbox@lore.proxmox.com>; Wed, 19 Feb 2025 17:49:35 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 49BB22FCA9; Wed, 19 Feb 2025 17:49:31 +0100 (CET) From: Christian Ebner <c.ebner@proxmox.com> To: pbs-devel@lists.proxmox.com Date: Wed, 19 Feb 2025 17:48:38 +0100 Message-Id: <20250219164847.757184-1-c.ebner@proxmox.com> X-Mailer: git-send-email 2.39.5 MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.031 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [datastore.rs, proxmox.com] Subject: [pbs-devel] [PATCH proxmox proxmox-backup 0/9] fix #5892: check atime update is honored X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion <pbs-devel.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pbs-devel>, <mailto:pbs-devel-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pbs-devel/> List-Post: <mailto:pbs-devel@lists.proxmox.com> List-Help: <mailto:pbs-devel-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel>, <mailto:pbs-devel-request@lists.proxmox.com?subject=subscribe> Reply-To: Proxmox Backup Server development discussion <pbs-devel@lists.proxmox.com> Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pbs-devel-bounces@lists.proxmox.com Sender: "pbs-devel" <pbs-devel-bounces@lists.proxmox.com> These patches add a check to phase 1 of garbage collection in order to detect when the filesystem backing the chunk store does not honor atime updates. This avoids possible data loss for situations where garbage collection could otherwise delete chunks still referenced by a backup snaphost's index file. The check is performed by inserting a fixed size 4 MiB unencrypted and compressed chunk of all-zeros. The check is performed by setting the atime to 1 second in the past followed by setting it to now in order to test if atime updates are honored immediately. The test is performed on datastore creation for early detection and before garbage collection phase 1. The test is enabled by default, but an opt-out option can be set via the datastore tuning parameters for backwards compatibility. Further, add a datastore tuning parameter to reduce the wait period for chunk removal in phase 2 of garbage collection. Make this conditional on the atime update check being enabled and successful, to avoid possible data loss. Most notable changes sice version 1 (thanks Fabian and Thomas for comments and suggestions): - Optimize check by using the all zero chunk - Enable the check by default and fail GC job if not honored, but allow to opt-out - Add GC wait period tuning option Link to the issue in the bugtracker: https://bugzilla.proxmox.com/show_bug.cgi?id=5982 proxmox: Christian Ebner (2): pbs api types: add check garbage collection atime updates flag pbs api types: add option to set GC chunk cleanup wait period pbs-api-types/src/datastore.rs | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) proxmox-backup: Christian Ebner (7): datastore: move utimensat() related constants to module scope fix #5982: garbage collection: check atime updates are honored ui: expose atime update check flag in datastore tuning options docs: mention GC atime update check for tuning options datastore: conditionally use custom GC wait period if set ui: expose GC wait period in datastore tuning option docs: mention gc-wait-period option as datastore tuning parameter docs/storage.rst | 19 ++++++- pbs-datastore/src/chunk_store.rs | 96 +++++++++++++++++++++++++++++--- pbs-datastore/src/datastore.rs | 23 ++++++++ src/api2/config/datastore.rs | 1 + www/Utils.js | 9 +++ www/datastore/OptionView.js | 17 ++++++ 6 files changed, 153 insertions(+), 12 deletions(-) -- 2.39.5 _______________________________________________ pbs-devel mailing list pbs-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel