From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id ACD8D1FF168
for <inbox@lore.proxmox.com>; Mon, 14 Oct 2024 12:09:11 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id B7EC731EC8;
Mon, 14 Oct 2024 12:09:41 +0200 (CEST)
From: Dominik Csapak <d.csapak@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Mon, 14 Oct 2024 12:09:38 +0200
Message-Id: <20241014100938.1288020-1-d.csapak@proxmox.com>
X-Mailer: git-send-email 2.39.5
MIME-Version: 1.0
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.016 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to
Validity was blocked. See
https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more
information.
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: [pve-devel] [PATCH cluster] fix #5728: pmxcfs: allow bigger writes
than 4k for fuse
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
by default libfuse2 limits writes to 4k size, which means that on writes
bigger than that, we do a whole write cycle for each 4k block that comes
in. To avoid that, add the option 'big_writes' to allow writes bigger
than 4k at once (namely up to 128 KiB).
This means that if we update a file with more than 4KiB data, the
following pattern occurs:
* cfs_fuse_write is called with at offset 0 with 4096 size
* sqlite writes the partial file to disk since it's a transaction
* cfs_fuse_write is called with an offset 4096 and with 4096 size
* sqlite updates the data and writes again
* repeat until all data reached cfs_fuse_write
So when cfs_fuse_write accepts bigger chunks, we have less
cfs_fuse_write -> sqlite write cycles, leading to a reduced disk
activity.
Note that sqlite itself uses 4096 byte blocks to write to the file
system layer below.
Most files on pmxcfs are written with `file_set_contents`, which writes
the file into a tmp file and renames it, so we always have some write
overhead.
Previous to pve-common commit
ef0bcc9 (tools: file_set_contents: use syswrite instead of print)
it used `print` to write, which uses an internal 8k buffer, and after
the commit it uses `syswrite`, which writes the file unbuffered in one
go. (Fuse still splits writes at it's defined maximum)
The commit message of that patch includes benchmarks for various sizes
of writes on pmxcfs with this patch included. Results show that we can
reduce the amount of bytes written to disk for files larger than 4 KiB
by a significant amount (with both patches we can reduce the
amplification at 8KiB from ~15x to ~11x, and for 1024KiB from ~360x to
~15x)
When we change to libfuse3, we have to remove this option again, since
it got removed and is the default there.
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from rfc:
* improve commit message to contain more detail and reference
Filips commit that improves `file_set_contents`
* add a comment above the option to remove it with change to libfuse3
src/pmxcfs/pmxcfs.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c
index 1cf8ab8..f0d5102 100644
--- a/src/pmxcfs/pmxcfs.c
+++ b/src/pmxcfs/pmxcfs.c
@@ -945,7 +945,8 @@ int main(int argc, char *argv[])
mkdir(CFSDIR, 0755);
- char *fa[] = { "-f", "-odefault_permissions", "-oallow_other", NULL};
+ // TODO: remove big_writes with change to libfuse3
+ char *fa[] = { "-f", "-odefault_permissions", "-oallow_other", "-obig_writes", NULL};
struct fuse_args fuse_args = FUSE_ARGS_INIT(sizeof (fa)/sizeof(gpointer) - 1, fa);
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel