From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id E74C472A67 for ; Tue, 24 May 2022 10:14:59 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D4F8B1BB78 for ; Tue, 24 May 2022 10:14:29 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 441A01BB6E for ; Tue, 24 May 2022 10:14:29 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 1011C43423 for ; Tue, 24 May 2022 10:14:23 +0200 (CEST) Message-ID: <440e01fc-e931-bf2a-1508-0e26a719abb4@proxmox.com> Date: Tue, 24 May 2022 10:14:21 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:101.0) Gecko/20100101 Thunderbird/101.0 Content-Language: en-GB To: Proxmox Backup Server development discussion , =?UTF-8?Q?Fabian_Gr=c3=bcnbichler?= References: <20220520124228.3368960-1-d.csapak@proxmox.com> <20220520124228.3368960-4-d.csapak@proxmox.com> <1653289622.043mjuz08t.astroid@nora.none> From: Thomas Lamprecht In-Reply-To: <1653289622.043mjuz08t.astroid@nora.none> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 1.681 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -3.328 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: Re: [pbs-devel] [PATCH proxmox-backup 3/5] datastore: implement sync-level tuning for datastores X-BeenThere: pbs-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox Backup Server development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 May 2022 08:15:00 -0000 On 23/05/2022 09:13, Fabian Grünbichler wrote: > I am not sure whether that "in practice" statement really holds > - we only tested the exact failure case on a few filesystems, there > might be ones in use out there where a powerloss can also lead to a > truncated chunk, not only an empty/missing one. granted, both will be > detected on the next verification, but only the latter will be > automatically cleaned up by a subsequent backup task that uploads this > chunk.. I don't think partial written files can happen on journaled FS in that way, at least as long as the default-on write barriers are not disabled. XFS, ext4 and btrfs are fine in that regard, FWICT, didn't checked others, but I'd think that additionally NFS, CIFS and ZFS would be interesting. > - the FS underlying the datastore might be used for many datastores, or > even other, busy, non-datastore usage. not an ideal setup, but there > might be $reasons. in this case, syncfs might have a much bigger > negative effect (because of syncing out other, unrelated I/O) than > fsync. Yeah, edge case exists, in practice means for the general case, in which PBS is still its own appliance that one won't also co-host their high churn appache cassandra DB or whatever on it, so this still holds and in most cases it will be more efficient then too than two fsyncs per chunk (which internally often flushes more than just the two inodes too). If an admin still does it for $reasons they can still switch to fsync based level, I'd find it odd if the "in practice" of a doc comment would hinder them of ever trying, especially as such setups most of the time get created when the users won't care for best practice anyway. > - not sure what effect syncfs has if a datastore is really busy (as in, > has tons of basically no-op backups over a short period of time) What effects do you imagine? It just starts a writeback kernel worker that flushes all dirty inodes belonging to a super block from the time the syncfs was called in a lockless manner using the RCU (see sync.c and fs-writeback.c in the kernel fs/ tree), new IO isn't stopped and the inodes would have been synced over the next 30s (default) anyway.. > > I'd rather mark 'Filesystem' as a good compromise, and the 'File' on as > most consistent.