From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 71E69B5DC for ; Fri, 29 Apr 2022 12:55:51 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 62EECA231 for ; Fri, 29 Apr 2022 12:55:21 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 974D4A227 for ; Fri, 29 Apr 2022 12:55:20 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 6965043026 for ; Fri, 29 Apr 2022 12:55:20 +0200 (CEST) From: Aaron Lauterer To: pve-devel@lists.proxmox.com Date: Fri, 29 Apr 2022 12:55:19 +0200 Message-Id: <20220429105519.1547112-1-a.lauterer@proxmox.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.004 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] [PATCH docs] pveceph: add section about erasure code pools X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 29 Apr 2022 10:55:51 -0000 Signed-off-by: Aaron Lauterer --- pveceph.adoc | 65 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 65 insertions(+) diff --git a/pveceph.adoc b/pveceph.adoc index 0984a52..dff44e0 100644 --- a/pveceph.adoc +++ b/pveceph.adoc @@ -547,6 +547,71 @@ operation footnote:[Ceph pool operation manual. +[[pve_ceph_ec_pools]] +Erasure Coded (EC) Pools +~~~~~~~~~~~~~~~~~~~~~~~~ + +Erasure coded (EC) pools can offer more usable space for the price of +performance. In replicated pools, multiple replicas of the data are stored +('size'). In erasure coded pool, data is split into 'k' data chunks with +additional 'm' coding chunks. The coding chunks can be used to recreate data +should data chunks be missing. The number of coding chunks, 'm', defines how +many OSDs can be lost without losing any data. The total amount of objects +stored is 'k + m'. + +The default 'min_size' of an EC pool depends on the 'm' parameter. If 'm = 1', +the 'min_size' of the EC pool will be 'k'. The 'min_size' will be 'k + 1' if +'m > 1'. The Ceph documentation recommends a conservative 'min_size' of 'k + 2' +footnote:[Ceph Erasure Coded Pool Recovery +{cephdocs-url}/rados/operations/erasure-code/#erasure-coded-pool-recovery]. + +If there are less than 'min_size' OSDs available, any IO to the pool will be +blocked until there are enough OSDs available again. + +NOTE: When planning an erasure coded pool, keep an eye on the 'min_size' as it +defines how many OSDs need to be available. Otherwise, IO will be blocked. + +For example, an EC pool with 'k = 2' and 'm = 1' will have 'size = 3', +'min_size = 2' and will stay operational if one OSD fails. If the pool is +configured with 'k = 2', 'm = 2', it will have a 'size = 4' and 'min_size = 3' +and stay operational if one OSD is lost. + +To create a new EC pool, run the following command: + +[source,bash] +---- +pceveph pool create --erasure-coding k=2,m=1 +---- + +Optional parameters are 'failure-domain' and 'device-class'. If you +need to change any EC profile settings used by the pool, you will have to +create a new pool with a new profile. + +This will create a new EC pool plus the needed replicated pool to store the RBD +omap and other metadata. In the end, there will be a '-data' and +'-metada' pool. The default behavior is to create a matching storage +configuration as well. If that behavior is not wanted, you can disable it by +providing the '--add_storages 0' parameter. When configuring the storage +configuration manually, keep in mind that the 'data-pool' parameter needs to be +set. Only then will the EC pool be used to store the data objects. For example: + +[source,bash] +---- +pvesm add rbd --pool --data-pool +---- + +If there is a need to further customize the EC profile, you can do so by +creating it with the Ceph tools directly footnote:[Ceph Erasure Code Profile +{cephdocs-url}/rados/operations/erasure-code/#erasure-code-profiles], and +specify the profile to use with the 'profile' parameter. + +For example: +[source,bash] +---- +pceveph pool create --erasure-coding profile= +---- + + Destroy Pools ~~~~~~~~~~~~~ -- 2.30.2