From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id 4609A1FF15D
for <inbox@lore.proxmox.com>; Thu, 19 Sep 2024 14:24:10 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 62A26150BC;
Thu, 19 Sep 2024 14:24:17 +0200 (CEST)
References: <20240919095202.1375181-1-d.csapak@proxmox.com>
In-Reply-To: <20240919095202.1375181-1-d.csapak@proxmox.com>
Date: Thu, 19 Sep 2024 14:23:55 +0200
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
MIME-Version: 1.0
Message-ID: <mailman.14.1726748656.332.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Esi Y via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Esi Y <esiy0676+proxmox@gmail.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: Re: [pve-devel] [RFC PATCH pve-cluster] fix #5728: pmxcfs: allow
bigger writes than 4k for fuse
Content-Type: multipart/mixed; boundary="===============0895427658109762656=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
--===============0895427658109762656==
Content-Type: message/rfc822
Content-Disposition: inline
Return-Path: <esiy0676@gmail.com>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 8FD71C04C4
for <pve-devel@lists.proxmox.com>; Thu, 19 Sep 2024 14:24:15 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 6F94E15090
for <pve-devel@lists.proxmox.com>; Thu, 19 Sep 2024 14:24:15 +0200 (CEST)
Received: from mail-qt1-x82f.google.com (mail-qt1-x82f.google.com [IPv6:2607:f8b0:4864:20::82f])
(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Thu, 19 Sep 2024 14:24:14 +0200 (CEST)
Received: by mail-qt1-x82f.google.com with SMTP id d75a77b69052e-457e153cbdcso6869011cf.2
for <pve-devel@lists.proxmox.com>; Thu, 19 Sep 2024 05:24:14 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1726748647; x=1727353447; darn=lists.proxmox.com;
h=content-transfer-encoding:to:subject:message-id:date:from
:in-reply-to:references:mime-version:from:to:cc:subject:date
:message-id:reply-to;
bh=MUFCJFJOR4KJTVry3bKjgiJMH2L2UMyJ4f0OGwDq+RI=;
b=F8szO2D4QFwa6r4jDjvcpw5StGOwW6rv2zhFn69iodkfj86ndVXbMrKDxs6QcrWnbB
jkR1yUJfoPgkANi1Jyz9yY5y5D/Pw60KUcGWdHszZuNNwTO1XQU4Qyu1tROgaV/iqUvQ
Q4+NmKWI38pzuqNTXoToJrrAm9uKiYjqVkmU/Jv00+1goZIKjZ3XWFRd6AZasnHZ0gxd
3Z+xEWUMupfU/y78Ex3UB00Z52wmR0sxGqMGsSiAwyIVZYtAdyiukyp0K1LI0VCR0jJT
3+NjbzcRdMr59boGJ5p7WDGRriXlHpiFYy2lwrtW0J4A0e2wbjPuUbfP7fNnr0IBrMYO
OlRA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1726748647; x=1727353447;
h=content-transfer-encoding:to:subject:message-id:date:from
:in-reply-to:references:mime-version:x-gm-message-state:from:to:cc
:subject:date:message-id:reply-to;
bh=MUFCJFJOR4KJTVry3bKjgiJMH2L2UMyJ4f0OGwDq+RI=;
b=kO9IOEth0K4riWIvzMJeWaQDTCYDcKp+6LkR88CUiaJGcoGgsNNdKwGO8dSvS/G+I0
0au0uTx1bmkGweF6PwweLtzHWbuICTWTBddY7rUnb9XzP00wAUDfn3/7iU2pj+r7CHdR
D2TohH6sTUVE5LcLLCCMgj7wMa5B6gRVZ26/9jxAo7kMTekbDteyk3pQF539PpJFzIB0
DIe9t838kB0ItgqYFXABdvz5QMr2gFqA4ipLm4FcaoMXXTN19NC5+teGtnZtkG0w2g8A
RvBIDAf9laNxYY/3b5Jxmau3xVcAhBqo9TdziYgfIGb7HkoZLP+ejdl4/82yNAKP6XHb
mtAA==
X-Gm-Message-State: AOJu0YwmV1xOHO4bXZNrr5g/SxU7Pd27/R9z0T3thKhe/Nok1vJt7RuP
UC4uQWqzS7SjkcXmERnUdnoeJ4hO8ZqrzBDNKf+DKCByZ4BpL0titiHANJDtSfKLmARd4kC1/IN
Nxa5BeRU2x6pBzQNiZ93FiB6NyYwTrYaC
X-Google-Smtp-Source: AGHT+IF2a2kpbZ8mr0mXWwEBRKyndZCQpgmTAJPtRjwFo+Sw75F1pNMAjhTiSiC4h+hvbfQqou7BboxTWKboyM7kox8=
X-Received: by 2002:a05:6214:4293:b0:6c5:1f05:dcec with SMTP id
6a1803df08f44-6c5736e12c7mr364874376d6.49.1726748646894; Thu, 19 Sep 2024
05:24:06 -0700 (PDT)
MIME-Version: 1.0
References: <20240919095202.1375181-1-d.csapak@proxmox.com>
In-Reply-To: <20240919095202.1375181-1-d.csapak@proxmox.com>
From: Esi Y <esiy0676+proxmox@gmail.com>
Date: Thu, 19 Sep 2024 14:23:55 +0200
Message-ID: <CABtLnHpSk+XWjzfvG=qOqGOet0JFda4_Lb_DkqR3ZgjsXvQHFA@mail.gmail.com>
Subject: Re: [pve-devel] [RFC PATCH pve-cluster] fix #5728: pmxcfs: allow
bigger writes than 4k for fuse
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-SPAM-LEVEL: Spam detection results: 0
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain
DMARC_PASS -0.1 DMARC pass policy
FREEMAIL_ENVFROM_END_DIGIT 0.25 Envelope-from freemail username ends in digit
FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider
RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
First of all, thanks for spotting this!
I believe you will still be limited to 128K, this could be tuned up to
max 1M [1] (current file limit size in pmxcfs, coincidentally), but
needs FUSE 3.6.0 [2]:
[1] https://github.com/torvalds/linux/commit/5da784cce4308ae10a79e3c8c41b13=
fb9568e4e0
[2] https://github.com/libfuse/libfuse/blob/master/ChangeLog.rst#libfuse-36=
0-2019-06-13
Obviously, this should be buffered within the pmxcfs logic instead, however=
.
--=20
Esi Y
On Thu, Sep 19, 2024 at 11:52=E2=80=AFAM Dominik Csapak <d.csapak@proxmox.c=
om> wrote:
>
> by default libfuse2 limits writes to 4k size, which means that on writes
> bigger than that, we do a whole write cycle for each 4k block that comes
> in. To avoid that, add the option 'big_writes' to allow writes bigger
> than 4k at once.
>
> This should improve pmxcfs performance for situations where we often
> write large files (e.g. big ha status) and maybe reduce writes to disk.
>
> If we'd change to libfuse3, this would be a non-issue, since that option
> got removed and is the default there.
>
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> sending as RFC, since I'm not sure if there are maybe any sideeffects i
> overlooked. I scanned the pmxcfs code for potential problems where we
> would e.g. assume a maximum size of 4096 for fuse writes, but could not
> find any. (also my C foo is not that great). I run it here currently,
> and it seems to work just fine atm.
>
> src/pmxcfs/pmxcfs.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c
> index 1cf8ab8..c5b8f04 100644
> --- a/src/pmxcfs/pmxcfs.c
> +++ b/src/pmxcfs/pmxcfs.c
> @@ -945,7 +945,7 @@ int main(int argc, char *argv[])
>
> mkdir(CFSDIR, 0755);
>
> - char *fa[] =3D { "-f", "-odefault_permissions", "-oallow_other", =
NULL};
> + char *fa[] =3D { "-f", "-odefault_permissions", "-oallow_other", =
"-obig_writes", NULL};
>
> struct fuse_args fuse_args =3D FUSE_ARGS_INIT(sizeof (fa)/sizeof(=
gpointer) - 1, fa);
>
> --
> 2.39.2
>
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
>
--===============0895427658109762656==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--===============0895427658109762656==--