From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 98A36601E7 for ; Fri, 4 Feb 2022 18:14:38 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 85B03B48 for ; Fri, 4 Feb 2022 18:14:08 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id B5B13B3A for ; Fri, 4 Feb 2022 18:14:06 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 795C7431C2 for ; Fri, 4 Feb 2022 18:14:06 +0100 (CET) Message-ID: <6154f7e9-6ed2-5956-792e-a8c52bd823e2@proxmox.com> Date: Fri, 4 Feb 2022 18:14:04 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:97.0) Gecko/20100101 Thunderbird/97.0 Content-Language: en-US To: Proxmox VE development discussion , Aaron Lauterer References: <20220128112241.3435277-1-a.lauterer@proxmox.com> From: Thomas Lamprecht In-Reply-To: <20220128112241.3435277-1-a.lauterer@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.059 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_SHORT 0.001 Use of a URL Shortener for very short URL SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [rbdplugin.pm, ceph.io] Subject: [pve-devel] applied: [PATCH v2 storage] fix #1816: rbd: add support for erasure coded ec pools X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Feb 2022 17:14:38 -0000 On 28.01.22 12:22, Aaron Lauterer wrote: > The first step is to allocate rbd images correctly. > > The metadata objects still need to be stored in a replicated pool, but > by providing the --data-pool parameter on image creation, we can place > the data objects on the erasure coded (EC) pool. > > Signed-off-by: Aaron Lauterer > --- > changes: add data-pool parameter in clone_image() if present > > > Right now this only this only affects disk image creation and cloning. > The EC pool needs to be created manually to test this. > > The Ceph blog about EC with RBD + CephFS gives a nice introduction and > the necessary steps to set up such a pool [0]. > > The steps needed are: > > - create EC profile (a 21 profile is only useful for testing purposes in > a 3 node cluster, not something that should be considered for > production use!) > # ceph osd erasure-code-profile set ec-21-profile k=2 m=1 crush-failure-domain=host > > - create a new pool with that profile > # ceph osd pool create ec21pool erasure ec-21-profile > > - allow overwrite > # ceph osd pool set ec21pool allow_ec_overwrites true > > - enable application rbd on the pool (the command in the blog seems to > have gotten the order of parameters a bit wrong here) > # ceph osd pool application enable ec21pool rbd > > - add storage configuration > # pvesm add rbd ectest --pool --data-pool ec21pool > > For the replicated pool, either create a new one without adding the PVE > storage config or use a namespace to separate it from the existing pool. > > To create a namespace: > # rbd namespace create / > > add the '--namespace' parameter in the pvesm add command. > > To check if the objects are stored correclty you can run rados: > > # rados -p ls > > This should only show metadata objects > > # rados -p ls > > This should then show only `rbd_data.xxx` objects. > If you configured a namespace, you also need to add the `--namespace` > parameter to the rados command. > > > [0] https://ceph.io/en/news/blog/2017/new-luminous-erasure-coding-rbd-cephfs/ > argh, above should have been in the commit message and I overlooked that.. > > PVE/Storage/RBDPlugin.pm | 23 +++++++++++++---------- > 1 file changed, 13 insertions(+), 10 deletions(-) > > applied, thanks! Made a few followups to improve readbillity, perl captures all in the last array, e.g.: my ($a, @foo) = ('a', 'all', @this, 'will', @be_in_foo)