From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 73BEC643B0 for ; Tue, 6 Oct 2020 13:22:09 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 64C712519D for ; Tue, 6 Oct 2020 13:22:09 +0200 (CEST) Received: from mail-out.auranext.com (mail2.auranext.com [195.134.160.3]) by firstgate.proxmox.com (Proxmox) with ESMTP id 73A9325187 for ; Tue, 6 Oct 2020 13:22:07 +0200 (CEST) Received: from mailhost.auranext.com (unknown [172.31.2.1]) by mail-out.auranext.com (Postfix) with ESMTP id A5B8E8E0070 for ; Tue, 6 Oct 2020 13:12:28 +0200 (CEST) From: Maxime AUGER To: "'pve-user@lists.proxmox.com'" Thread-Topic: bad ZFS performance with proxmox 6.2 Thread-Index: Adab0W6NGVLK6RaDRbKmhzviWowocQ== Date: Tue, 6 Oct 2020 11:12:13 +0000 Accept-Language: fr-FR, en-US Content-Language: fr-FR x-originating-ip: [172.31.4.93] MIME-Version: 1.0 Message-Id: <20201006111213.9D6441764081@mailhost.auranext.com> X-SPAM-LEVEL: Spam detection results: 1 HTML_MESSAGE 0.001 HTML included in message KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years KHOP_HELO_FCRDNS 0.398 Relay HELO differs from its IP's reverse DNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: [PVE-User] bad ZFS performance with proxmox 6.2 X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Oct 2020 11:22:09 -0000 Hello, We notice a significant difference in ZFS performances between proxmox 5.3-= 5 and proxmox 6.2. Last year when we tested proxmox5.3 on the HPE-DL360Gen9 hardware. This hardware is provided with 8 ssd disks (2x 200-GB OS dedicated mdadm RA= ID0 and 6x 1-TB ZFS pool for VMs storage) On ZFS pool we measured the peak value at 2.8GB/s (write) Actually on promox6 we measured the peak value at 1.5GB/s (write). One server, ITXPVE03 was running Initially on proxmox 5.3-5. peak performance 2.8GB/s Recently it has been reinstalled on proxmox 6.2 (from iso) peak performance 1.5GB/s to confirm this observation we have extended the checks to the 4 servers, i= dentical hardware and low level software, BIOS and firmwares versions) The measurements confirm the statement All tests are done in serveurs idle state conditions with zero ative worklo= ad, all VMs/Containers shutdown. ZFS configuration are identical, no compression, no deduplicatation root@CLIPVE03(PVE6.2):~# dd if=3D/dev/zero of=3D/zfsraid10/iotest bs=3D1M c= ount=3D1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.851028 s, 1.2 GB/s root@ITXPV03(PVE6.2):~# dd if=3D/dev/zero of=3D/zfsraid10/iotest bs=3D1M co= unt=3D1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.722055 s, 1.5 GB/s root@CLIB05PVE02(PVE5.3-5):~# dd if=3D/dev/zero of=3D/zfsraid10/iotest bs= =3D1M count=3D1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.397212 s, 2.6 GB/s root@CLIB05PVE01(PVE5.3-5):~# dd if=3D/dev/zero of=3D/zfsraid10/iotest bs= =3D1M count=3D1000 1000+0 records in 1000+0 records out 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.39394 s, 2.7 GB/s At the ZFS level we can notice a difference in the version of zfsutils-linu= x 0.7.X on PVE5.3-5 (0.7.12) 0.8.X on PVE6.2 (same measure on 0.8.3 and 0.8.4) has anyone experienced this problem ? Maxime AUGER Network Team Leader AURANEXT