From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id DDF7A63CE5 for ; Thu, 27 Jan 2022 12:27:08 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DBF672209F for ; Thu, 27 Jan 2022 12:27:08 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 2649622094 for ; Thu, 27 Jan 2022 12:27:08 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id EA52D46C79; Thu, 27 Jan 2022 12:27:07 +0100 (CET) Message-ID: <1ae1d772-c466-5694-cf77-4018aedddafc@proxmox.com> Date: Thu, 27 Jan 2022 12:27:06 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.1 Content-Language: en-US To: Proxmox VE development discussion , Alwin Antreich References: <20220126160734.2868618-1-a.lauterer@proxmox.com> <3dbb90bb8bfec2db7a08965c0301480f@antreich.com> From: Aaron Lauterer In-Reply-To: <3dbb90bb8bfec2db7a08965c0301480f@antreich.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.000 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [PATCH storage] rbd: add support for erasure coded ec pools X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2022 11:27:08 -0000 On 1/26/22 19:30, Alwin Antreich wrote: > Hello Aaron, > > nice to see EC pools are coming. ;) > > January 26, 2022 5:07 PM, "Aaron Lauterer" wrote: > >> The first step is to allocate rbd images correctly. >> >> The metadata objects still need to be stored in a replicated pool, but >> by providing the --data-pool parameter on image creation, we can place >> the data objects on the erasure coded (EC) pool. > > AFAICR, there is an undocumented location for a ceph.conf, > `/etc/pve/priv/ceph/.conf`. Thanks for the hint, as I wasn't aware of it. It will not be considered for PVE managed Ceph though, so not really an option here.[0] [0] https://git.proxmox.com/?p=pve-storage.git;a=blob;f=PVE/CephConfig.pm;h=c388f025b409c660913c082763765dda0fba2c6c;hb=HEAD#l192 > > The config should have the following content. > ``` > [client.admin_ec] > rbd default data pool = ceph_pool_ec > ```` > Then rbd will use the data pool. This should probably work for all storage operations. > > Newer ceph versions should also work with the config db option. > ``` > ceph config set client.xxx rbd_default_data_pool ceph_pool_e> ``` What these approaches do have in common, is that we spread the config over multiple places and cannot set different data pools for different storages. I rather keep the data pool stored in our storage.cfg and apply the parameter where needed. From what I can tell, I missed the image clone in this patch, where the data-pool also needs to be applied. But this way we have the settings for that storage in one place we control and are also able to have different EC pools for different storages. Not that I expect it to happen a lot in practice, but you never know. > > Cheers, > Alwin > > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel > >