From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 5752B600AB for ; Tue, 6 Oct 2020 16:59:55 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4740327D3D for ; Tue, 6 Oct 2020 16:59:55 +0200 (CEST) Received: from mout.web.de (mout.web.de [217.72.192.78]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-256) server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 71EA327D31 for ; Tue, 6 Oct 2020 16:59:53 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=web.de; s=dbaedf251592; t=1601996393; bh=n/4Lb08AO97uU2gHLCmDHFfmpzepzcow8ZzmSGksJBs=; h=X-UI-Sender-Class:Subject:To:References:From:Date:In-Reply-To; b=dGB8/WV19csGRIbApZn/+DpEhXMa1wvNKFoLAJdnlx3fOVrVosQ1f6uI0foF/BZSF deKi+hGxliQs/ZMcCQxqMPRrRNhOUIOEdBPTn1L/JYdHk0Zy+TCyisspEMk/hhnTY2 GMyVDK0T8tf0H2UBIFaoU0Q3CzAjgzD8UeU21Ooo= X-UI-Sender-Class: c548c8c5-30a9-4db5-a2e7-cb6cb037b8f9 Received: from Rolands-MacBook-Pro.cybercon.de ([5.147.253.138]) by smtp.web.de (mrweb103 [213.165.67.124]) with ESMTPSA (Nemesis) id 0MGA7n-1kCNwz40gh-00FDsD; Tue, 06 Oct 2020 16:54:43 +0200 To: Proxmox VE user list , Maxime AUGER References: <20201006111213.9D6441764081@mailhost.auranext.com> From: Roland Message-ID: <5b443799-6007-212a-2b09-3833ee6a5555@web.de> Date: Tue, 6 Oct 2020 16:54:45 +0200 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <20201006111213.9D6441764081@mailhost.auranext.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: quoted-printable X-Provags-ID: V03:K1:e57iYwRAgcIq+mfBdwWP3q/w0uAMxmgqU2LsYwYUguXNRE3pXVP LZ1X1QoQI2VIw+/EbVZHhwdteknb9Hfp+kZT3ZOKvuYQWBCA2yxTDd03gmJbPNkHgWdWv/h u4xjSURIkJob9tRdMxnW6GDZ2fA0cZ2iUk8WI+mytYYwxTVBL1xycTQiVCFCfCi+x3iKJvq YkMczQ0u6KdsKmJIomVCw== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:eOQohNQx81g=:chZZlajfldzy1dQBdMG1VO c0exGLx6V4INZM9DgBCHdM8OdiHyrgS+B4KWyqjvlsYCKAKJa0PuSt4wQXXbyUvTvB5d3rpsJ KbFVCJ/7rbU97rPypxnkpTKzylo4OZkAxtCJh/2aR+e5Px8lWQCmUKvF6TWrH1BurzNjiKLIZ FeC8qZB33xrmrrvQYCB5BEmfKXhppMb5QSG8T4wXBgWvbONa+KbG6ekByItIRE5bcWIbJrm2u jFG1YBo6pEHSppYDFG/wrqqvgNAIPff0smwwwSh6qEeKHcp6/RIayXtLvTytb5WxwEJn8Vxtf QN3zBufvcolcvBN+s0Ngyjp6D1ZBfIon6Spz27gvRj6Sd4oQA5IJg9fvJmDQDmBLsKMhqX1s8 Ie0jI22XPcnAGL7+5CgSePhlhrWUfomJnKCBT4gooAqok80ZDB3OZn1Phth+oK3w7eyFiOluB o7uWS3XUVsaXbwYw9AbMC8xnVqAYJ1IDb6eufcnhj/x85G2xzi5snNQcSSO/YdFYZQEOjdsd0 /X8MWfSIrSnGH8SioZMSIflp6GmY1IMMHVAYnYzAOY78zgk6LtOWiZlXPyclOX+cXgZLsxeyq +2YWX2Bf/KY2wWqOWnpGtwyUNTkUAulseYTS/f6hViHepIQmuwLWmPP03u7OM7nGyyS6Q4sQj wy6gkuuwEm3eJL/pkdomSxMk5b71fRg1qXKrEqRt7A/ZpxjFDF8CeDF0GMVQggZJwrnjsVRhX GzaQjVm+Xa1WDQxdD45pedqmfretotHjGcUlTn6kzqb/k2Ksz3/NgAohThsjsrbT6V/WxnJvB nPQzECzV1YNUkh5uw1BB/jS9r9czvxj1TqmsrOm+KMm1UntmlO65sSw2XZPmEAOpJctd+u4yF FRNQZHQ+vtdxfUdsCLRzJ/OMCtkk5r8tisdkzqDnTmCjOvpZbTbpEzsHEyJ6tugQ9UajiS51D 6e+Rgo8wzyPc1+ckZdEMnQFdngNlTTNpjLzhsm7+Ci59PyEaUkUCuuh1AjCjd39DG0uStUKa4 i6gF33L91BrsdEsVyHvNyJmlrBFwe1GIeArpCGAChhUeOh3IYojnjgN1vB9PedtYcWM5PKUeR Jlf6FOyfYz9LzY3KCC7h4e0gzKTo93UpTUd5MiRflZBw2nRadG3egM9UfAeegUwy3Nt/UtKm3 lIEinq95bVh+O56Bw9xpK96SERRqirKMsJ+cINVuFPr7CqeulXno8O7iwTDtNTRc3kQU9r1Fn c+8dDB5R8xjrm7bJU X-SPAM-LEVEL: Spam detection results: 0 AWL -0.256 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years NICE_REPLY_A -0.001 Looks like a legit reply (A) RCVD_IN_DNSWL_LOW -0.7 Sender listed at https://www.dnswl.org/, low trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [PVE-User] bad ZFS performance with proxmox 6.2 X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Oct 2020 14:59:55 -0000 hi, i remember having seen performance degradation reports on zfsonlinux issue tracker, for example: https://github.com/openzfs/zfs/issues/8836 to blame zfs for this, you should first make sure it is not related to driver issues or other kernel/disk related stuff, so i would recommend comparing raw disk write performance first, to see if that may also make a difference already. furthermore, i would recommend using more data when doing write performance comparison at such high troughput rate regards roland Am 06.10.20 um 13:12 schrieb Maxime AUGER: > Hello, > > We notice a significant difference in ZFS performances between proxmox 5= .3-5 and proxmox 6.2. > Last year when we tested proxmox5.3 on the HPE-DL360Gen9 hardware. > This hardware is provided with 8 ssd disks (2x 200-GB OS dedicated mdadm= RAID0 and 6x 1-TB ZFS pool for VMs storage) > On ZFS pool we measured the peak value at 2.8GB/s (write) > Actually on promox6 we measured the peak value at 1.5GB/s (write). > > One server, ITXPVE03 was running Initially on proxmox 5.3-5. > peak performance 2.8GB/s > Recently it has been reinstalled on proxmox 6.2 (from iso) > peak performance 1.5GB/s > > to confirm this observation we have extended the checks to the 4 servers= , identical hardware and low level software, BIOS and firmwares versions) > The measurements confirm the statement > All tests are done in serveurs idle state conditions with zero ative wor= kload, all VMs/Containers shutdown. > ZFS configuration are identical, no compression, no deduplicatation > > root@CLIPVE03(PVE6.2):~# dd if=3D/dev/zero of=3D/zfsraid10/iotest bs=3D1= M count=3D1000 > 1000+0 records in > 1000+0 records out > 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.851028 s, 1.2 GB/s > > root@ITXPV03(PVE6.2):~# dd if=3D/dev/zero of=3D/zfsraid10/iotest bs=3D1M= count=3D1000 > 1000+0 records in > 1000+0 records out > 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.722055 s, 1.5 GB/s > > root@CLIB05PVE02(PVE5.3-5):~# dd if=3D/dev/zero of=3D/zfsraid10/iotest b= s=3D1M count=3D1000 > 1000+0 records in > 1000+0 records out > 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.397212 s, 2.6 GB/s > > root@CLIB05PVE01(PVE5.3-5):~# dd if=3D/dev/zero of=3D/zfsraid10/iotest b= s=3D1M count=3D1000 > 1000+0 records in > 1000+0 records out > 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.39394 s, 2.7 GB/s > > > At the ZFS level we can notice a difference in the version of zfsutils-l= inux > 0.7.X on PVE5.3-5 (0.7.12) > 0.8.X on PVE6.2 (same measure on 0.8.3 and 0.8.4) > > has anyone experienced this problem ? > > Maxime AUGER > Network Team Leader > AURANEXT > _______________________________________________ > pve-user mailing list > pve-user@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user >