From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 6996C60243 for ; Tue, 6 Oct 2020 17:59:45 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 60A532879E for ; Tue, 6 Oct 2020 17:59:15 +0200 (CEST) Received: from mail-ed1-x544.google.com (mail-ed1-x544.google.com [IPv6:2a00:1450:4864:20::544]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 4768F28791 for ; Tue, 6 Oct 2020 17:59:14 +0200 (CEST) Received: by mail-ed1-x544.google.com with SMTP id p13so8187752edi.7 for ; Tue, 06 Oct 2020 08:59:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=reply-to:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=ehfF47yDiU/a/ryzhpo8P997tyQtjk0EnXvHOuiYLS4=; b=pkFHC59CB8Hptux2uTFZUIFFbddHIY5x3wcODUvWbjKBb9dIMaMul1sOgXEYrjmZjW agQ+tGuCUgYN0SpNrFqJctu4A8qoDxEGnuoYxMHwWj6+m/fhQzWnPNgSf5Ls13EaLn8r tsaQSnLf5PFWJPaYItAvSxytYvM13tOdZJFUL6l/rVFuY4f8PZxA8sbxOLyoO/AQmkOr o086FTMuYsRsd9NAWm+vq1xhHYfV/3smLs/5+VwzOySPGhQCzpwQtLIr6JgYW2QiylJj duFA2vTO4aBsrQQtHWYF107zHSJKe22b5bT5AaR2MpVoJvMo5aOSdx0xg8oJIo2CwgPE v80w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:reply-to:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=ehfF47yDiU/a/ryzhpo8P997tyQtjk0EnXvHOuiYLS4=; b=BalykKFf87TmtnvfOJyFh0LbL40t1bNvMSCU2s71MMsbEJYKm1k6uA2WYlu5+Vin8Z kc2mVL4bopFGrB87MUTr4ApHsAGYdxIbG+rtxYffD9z7nxpWzR2Cd9FGoggiRgxzbgKR aohTXD0YSn7TKppGrS3/UsgXyq8gHP+keEqC1CSIP92Yix0E5AcM16AHrtDIaWfF2fnC 3WW7ydZF20mXMR5XFoawrLXFXtjvfG9AVG87Zp6/G8ZZV502fJEeiMWkA7GIjZhB2cFC NQg89CpiKhdbyUma6LCLkPPf6u9qk7XV3FdB7m1AKHLpChQQ3Tzjqi6rGrcYiC8crBsF HW8w== X-Gm-Message-State: AOAM532jwjzK675x1AqDOiedxtTAV1aD0vyWxkds8Mx0A6pp0jQ6i0rX HIJ8jEzyxX+jZ/mVcvF8umxzXy+c/9s= X-Google-Smtp-Source: ABdhPJw8SFjU/k+FidXEsH6fwiR7m+9VkEm1KjZj15Q0apLt2Bvc27mUYnkBg0coD6o6OKDDb7Yivg== X-Received: by 2002:a05:6402:10c9:: with SMTP id p9mr6328101edu.156.1601999947190; Tue, 06 Oct 2020 08:59:07 -0700 (PDT) Received: from ?IPv6:2a02:8070:a38c:fc00:a7ce:8954:6dd1:654d? ([2a02:8070:a38c:fc00:a7ce:8954:6dd1:654d]) by smtp.googlemail.com with ESMTPSA id u13sm2558173ejn.82.2020.10.06.08.59.06 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Tue, 06 Oct 2020 08:59:06 -0700 (PDT) Reply-To: uwe.sauter.de@gmail.com To: pve-user@lists.proxmox.com References: <20201006111213.9D6441764081@mailhost.auranext.com> <5b443799-6007-212a-2b09-3833ee6a5555@web.de> From: Uwe Sauter Message-ID: <17cc6d8f-f2b8-eab8-d001-7f63e92a2e47@gmail.com> Date: Tue, 6 Oct 2020 17:59:05 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <5b443799-6007-212a-2b09-3833ee6a5555@web.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: de-DE Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 1 DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain FREEMAIL_FROM 0.001 Sender email is commonly abused enduser mail provider KAM_ASCII_DIVIDERS 0.8 Spam that uses ascii formatting tricks KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years NICE_REPLY_A -0.001 Looks like a legit reply (A) RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [PVE-User] bad ZFS performance with proxmox 6.2 X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Oct 2020 15:59:45 -0000 What kind of checksum/redundancy are you using? Some time ago the Linux kernel removed the symbols for various SIMD (AVX, etc.) functions for non-GNU modules which lead to performace degradation. While the ZFS team found ways to circumvent this it might be that the performance hasn't caught up to where it was. Regards, Uwe Am 06.10.20 um 16:54 schrieb Roland: > hi, > > i remember having seen performance degradation reports on zfsonlinux > issue tracker, for example: > > https://github.com/openzfs/zfs/issues/8836 > > > to blame zfs for this, you should first make sure it is not related to > driver issues or other kernel/disk related stuff, so i would recommend > comparing raw disk write performance first, to see if that may also make > a difference already. > > furthermore, i would recommend using more data when doing write > performance comparison at such high troughput rate > > regards > roland > > Am 06.10.20 um 13:12 schrieb Maxime AUGER: >> Hello, >> >> We notice a significant difference in ZFS performances between proxmox 5.3-5 and proxmox 6.2. >> Last year when we tested proxmox5.3 on the HPE-DL360Gen9 hardware. >> This hardware is provided with 8 ssd disks (2x 200-GB OS dedicated mdadm RAID0 and 6x 1-TB ZFS pool for VMs storage) >> On ZFS pool we measured the peak value at 2.8GB/s (write) >> Actually on promox6 we measured the peak value at 1.5GB/s (write). >> >> One server, ITXPVE03 was running Initially on proxmox 5.3-5. >> peak performance 2.8GB/s >> Recently it has been reinstalled on proxmox 6.2 (from iso) >> peak performance 1.5GB/s >> >> to confirm this observation we have extended the checks to the 4 servers, identical hardware and low level software, >> BIOS and firmwares versions) >> The measurements confirm the statement >> All tests are done in serveurs idle state conditions with zero ative workload, all VMs/Containers shutdown. >> ZFS configuration are identical, no compression, no deduplicatation >> >> root@CLIPVE03(PVE6.2):~# dd if=/dev/zero of=/zfsraid10/iotest bs=1M count=1000 >> 1000+0 records in >> 1000+0 records out >> 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.851028 s, 1.2 GB/s >> >> root@ITXPV03(PVE6.2):~# dd if=/dev/zero of=/zfsraid10/iotest bs=1M count=1000 >> 1000+0 records in >> 1000+0 records out >> 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.722055 s, 1.5 GB/s >> >> root@CLIB05PVE02(PVE5.3-5):~# dd if=/dev/zero of=/zfsraid10/iotest bs=1M count=1000 >> 1000+0 records in >> 1000+0 records out >> 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.397212 s, 2.6 GB/s >> >> root@CLIB05PVE01(PVE5.3-5):~# dd if=/dev/zero of=/zfsraid10/iotest bs=1M count=1000 >> 1000+0 records in >> 1000+0 records out >> 1048576000 bytes (1.0 GB, 1000 MiB) copied, 0.39394 s, 2.7 GB/s >> >> >> At the ZFS level we can notice a difference in the version of zfsutils-linux >> 0.7.X on PVE5.3-5 (0.7.12) >> 0.8.X on PVE6.2 (same measure on 0.8.3 and 0.8.4) >> >> has anyone experienced this problem ? >> >> Maxime AUGER >> Network Team Leader >> AURANEXT >> _______________________________________________ >> pve-user mailing list >> pve-user@lists.proxmox.com >> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user >> > > _______________________________________________ > pve-user mailing list > pve-user@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user >