From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id E06FE6C8DC; Tue, 30 Mar 2021 10:07:24 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id D1EF81C5D7; Tue, 30 Mar 2021 10:07:24 +0200 (CEST) Received: from mailhub.plus-plus.su (rev-72.180.alfatelecom.org [217.198.180.72]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id D6EED1C5CD; Tue, 30 Mar 2021 10:07:23 +0200 (CEST) Received: from rev-12.178.alfatelecom.org ([217.198.178.12] helo=[192.168.10.56]) by mailhub.plus-plus.su with esmtpsa (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1lR8tL-00054M-4M; Tue, 30 Mar 2021 10:34:29 +0300 To: Proxmox VE user list , Gregor Burck , pve-user@pve.proxmox.com References: <20210329152820.EGroupware._t1Sneg3e4BsuWUv_4EVNL7@heim.aeppelbroe.de> From: Mikhail Autocrypt: addr=m@plus-plus.su; prefer-encrypt=mutual; keydata= xsFNBF2A3rkBEADTNqMfrRQP6yLC4zmFVaqpoqQDJfXVv43SST+GXwm+2Y9VGD+rhB0NBJRc DyFuK+AaMC+y9MpCD/o/Yv5SIgqG0zOmzCPyUCaQe6e9q8toTa482/06ZCgEmIEZXvY/m7Mu cklszyL9LEukZT/6xwGILhhVwaR8NV1X6J1IhKz4/qCq6eGMK+oK9ZoHYAg6hWdkZnT03iQN 00lDCkNlyFcbqqj6JJaRaLw+3FDtUhVyiADmK4Zpt0RW7An/sMOFnYIpUxyjq9N+DOqWquJ8 EeZFL0/5WXqJmgV528tw4jFq4RyfseI1OvSLdPz6vEewWJye7wIvbDEoELWk7g95m8qM6ldl dbhwr8br4ZfrVHc3OGizx4eRXje855TJM2JU9lQgUkCWTnJtWOTAPpOiA6DeTwBUwxpTTXeM LeS9ndeql40HslaPbtxh2cBy/du3TgEeqms/fvQFg2oKssnQQqzSYUA30SAfSIfRvFvXlN0H S5fkQvGjHu61VZ2aiQD/wr4MwaXqnLspjgXLs0RNsfO5DMS5bbeUuXtfNIKJrFf/r3JlMhzS US0VbLo2UJSS2GnP7ni8/qCELU4ZrAT0upulqfwujkyPhy3g23d3rcdO4qzQUH82me+Xv+Fd T+duHt3olDG364WIGIOmaHpnJoFmmwP6U+zMcz2rByQRXY8cNwARAQABzRhNaWtoYWlsIDxt QHBsdXMtcGx1cy5zdT7CwasEEwEKAD4WIQR712E7rGMe7jGRLVgVWm7ZsXEv1wUCXYDeuQIb IwUJEswDAAULCQgHAgYVCgkICwIEFgIDAQIeAQIXgAAhCRAVWm7ZsXEv1xYhBHvXYTusYx7u MZEtWBVabtmxcS/XIYsP/iKWI26zqWfSEyyiz9f4hNyMVOSt7/T+Ai3OTxdsLFcbYfAPcnIw i/1PrdoAJ1+1dk3y746oJ2+gGHwqy2UdmbWIUq+TI/YwA7cEGMUZ61X1xBYhcKr5Owh9STvW LKQ7pr7y/07fz3I6/vWynmDtOlny+TjTEyEEZtIADE9WTUHbkR3GcYDOI0Ppl6x0z9ZqgERK jj6A+y8Cw6I2iDYUvZjqCzkEFnsjB9ymIh8ZhuWWHcaiaCZy971kM0C9FC399guKP7Rv92oe nA+WcmPPkUwUuBd8SUlr9gakmaczzQ1W/itLCGoWJdzedGm+RGsyTGPAh+5vOqN7lDUcjoIX 9QH0faOhadMND492ERgw13NNdvZqnk15CF0XWJlCB2byX5dp4zzKmU2xgYnX8bnyhSd5hQ9C 2Ur7oa1DZzITE7x/OeRxgw0Z9XYpGpiCeSXWg2U6a8gC4+hCskHbTfQ6vOTceRa286qMKcTE RGe8k2t0cicnAfWQrgZWWeaQw9m9RfStdeKYDtUlI2VS8hD7cXuDyBsrolNU2u0Bd1imRfKt H3a2oM64Usgnm/9g5MXnWQNMg+XKJCyR3LtxQc29Zc4e9AFfrXsH2BuS8U2jgNjB0REtMoiV yDUYDdxEN2EUmEAM+psXnw/I8cVvZ6lrXDytHNXYss9WTyZjufZMQitdzsFNBF2A3rkBEADJ UggCzlchvanVhXlSVgqQMO9IIBRr5FxV+FEeWiXSy9JScsDOuLPYqhLCQbPuSCZQhy+FhZsb sfwe4uI4t3KduLKdMzFlt5e07imPHBGJJPlRq4hGHyNGV08kJoVlHeOml0pd9HWEp9AZ2TOC hLqgug8qGSJ5pNUcxY9PHVPY5lroPc+whp06gpckagD+dMG+Cs/VVtJIPHiJVe17YRxZuhv/ Dj/zPks6/aA+8Bndwh5jAehig64vBHkhlb5n2lScVl5yMumZxPHeZ7FnzDl3h9RPsnXfYXkK zlobCjANe9/uO02jFIelW6Btd3OIB+LPZKFEQGT1RONofyiKZqPsGpp1PjZkbVjc2eKWfY84 Rp9qfRpTeyVqWBcZJYp+VFtsj2HQAs6HdcpwQzNobR5UeTTf+2+RrNJuTOmoSkkRqx7QKh0i NSMJdR8QlyhFirM8DI2TveDASQaQAlYUaEbilFbo0RZNUFb+3ZIDnxBNnQJmoKMsrF21NHof pckAgbRAJUI5ZSmcwxJadXp7xOXYV6WEWMB/RONy82KCfsYLyB9kZPheApBQ+RrF+izBgsGx 00M/GKN0wcW2JrwqwpIB6TEZYbyGUTd587TLKjkqhrFVG51cJx0OT3acD8NkFlWmMxvd4Wcd ujFeR6tWB3e3A282CAWwlu6kcMwFgF1zZQARAQABwsGTBBgBCgAmFiEEe9dhO6xjHu4xkS1Y FVpu2bFxL9cFAl2A3rkCGwwFCRLMAwAAIQkQFVpu2bFxL9cWIQR712E7rGMe7jGRLVgVWm7Z sXEv112uEACMC0GrrXp9z4tpg1CAsvjAAJZKoyq+CdUD/1AeTLkXhLuVdQRzQaMMybcm3gEe skcQ1z7/csSnO6WUnrAA4j3Yo6KDh538Tb+v58i2xYe5ozwl/zBztfIh/zgLiFiA6oIiObIy di3HfJ6oum4nnNoIgJ88DYuyFCQ4RKwYNCQ6dW4/z+WYZ25PpDX4J+KtiRM6Etf61VEUeSka U1nUU1CzXN0NxLVial65FF7eIR+PkmKgNDHz02VOY8hhtVijFg2QiOMDHWpuJMITfO8Liv1o lXh+1LAGBx55uGK3QvmYlbiqtLECJc82ByVbRDnmFI+5R2At8L4GqTgkL0yys6WAcLb08KlJ RpM0Q4R8NgI3t0q7sVSOIFFcni4sST9oTAge0DnNh4sEZRQbuyDGe2D+1zBZKtvau+mKs/zL YZObO9s/xvJ9xCQj0CYpNlOcVnc1fHrDjEiaJbqz1fJ2EwKV2s6k38pY18iFdAgeM4vYiLWL VsKrMnGCvX5n0eo8xCwpotPZl/pE5uU2PUk3Z+Ip4GbMnnh0lfPg0sbwyX8o22OxQiqAFAvi cucnz/deKywENNRbbiCFzzyWAHe3BvvKIfsc1Mql4XuBL7zxUnB+57tA8TBEcH0AgsoQDLGJ OBxrcFDf4o2Buk2c4Voasrli2oWfU+JvS0Qb2g1CA2WIhA== Message-ID: Date: Tue, 30 Mar 2021 10:34:05 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: <20210329152820.EGroupware._t1Sneg3e4BsuWUv_4EVNL7@heim.aeppelbroe.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 1 KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods KHOP_HELO_FCRDNS 0.4 Relay HELO differs from its IP's reverse DNS NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [proxmox.com] Subject: Re: [PVE-User] ZFZ questions - performance and caching X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Mar 2021 08:07:24 -0000 Hi Gregor, How do you plan to setup backup on this server? Are you going to use Proxmox Backup Server for this, or just plain vzdump files? If you're considering PBS, which you should, and you have at least a pair of NVME devices available, then I would set ZFS to use those NVME as special device. We have similar setup deployed recently - 12x3,5 HGST 14TB SAS disks + 2 x Intel 1.6TB NVME (using rear bays, Supermicro 2U platform). This server is purely used as PBS. ZFS is built on top of 11 x 14TB SAS drives in RAIDZ-3 config (I know not optimal for speed, but we want to see all space available under single pool) and NVME disks are used as ZFS special device (mirrored). Remaining 12th SAS disk is set to be hot-spare under ZFS pool. This gives us ~100TB of space and we are satisfied with RAIDZ-3 speeds we get - we're able to read and write at 1 Gigabyte/s, which is enough for our case as we're limited by 10gbit networking. Using NVME as SLOG/L2ARC device does not seem to be optimal in case when used with PBS. Also, if you're thinking about Supermicro then I would add SATADOM devices for the OS - this will allow you to use entire NVME disks for ZFS without partitioning it for OS/boot. regards, Mikhail. On 3/29/21 4:28 PM, Gregor Burck wrote: > Hi, > > for calculating an backup server wich should be in worst case > additionaly an virtualisation server we thougth about a setup with HDDs > and for ZFZ chaching NVMes. > > We plan all with enterprise products,... > > I guess following Setup: > > A 12 3,5 '' bay Server, 6 PVEe sockets, maybee with an 2,5'' Rear > entension kit for the OS > > Backup envirement: > > 4 (or6) HDDs, 6 (or 8) TB without chaching > > For virtualisation enviroment: > > 4 (or6) HDDs, 6 (or 8) TB with NVMe chaching > > > I think, for the backup enviroment there shouldn't be a chaching > neccessary, cause here is the network the limitation. > > The VE part should be a little performer, but how big should the NVMe > cache? As I read, it is for the ZFZ log. > And how much is the performance increasemnt? > > Bye > > Gregor > > > _______________________________________________ > pve-user mailing list > pve-user@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user >