From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with UTF8SMTPS id DC4B470262 for ; Mon, 7 Jun 2021 09:14:54 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with UTF8SMTP id D2388D9B8 for ; Mon, 7 Jun 2021 09:14:24 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with UTF8SMTPS id E9F6FD9A1 for ; Mon, 7 Jun 2021 09:14:20 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with UTF8SMTP id B52A742BB5; Mon, 7 Jun 2021 09:14:14 +0200 (CEST) Message-ID: Date: Mon, 7 Jun 2021 09:14:13 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:89.0) Gecko/20100101 Thunderbird/89.0 Content-Language: en-US To: aderumier@odiso.com, Proxmox VE development discussion , pve-devel References: <2422013ee08dd4e3abb04ba5360a084beddf5183.camel@odiso.com> <1536e582aba69f723b2a306d9c4176da502f5588.camel@odiso.com> From: Dominik Csapak In-Reply-To: <1536e582aba69f723b2a306d9c4176da502f5588.camel@odiso.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.991 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] ceph create pool with min_size=1 not possible anymore with last gui wizard X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jun 2021 07:14:54 -0000 On 6/7/21 08:57, aderumier@odiso.com wrote: > Le vendredi 04 juin 2021 à 15:23 +0200, Dominik Csapak a écrit : >> On 6/4/21 04:47, aderumier@odiso.com  wrote: >>> Hi, >>> >> >> >> Hi, >> >>> I was doing a training week with students, >>> >>> and I see that the new ceph wizard to create pool don't allow to set >>> min_size=1 anymore. >>> >>> It's currently displaying a warning "min_size <= size/2 can lead to >>> data loss, incomplete PGs or unfound objects", >>> >>> that's ok  ,  but It's also blocking the validation button. >>> >> >> yes, in our experience, setting min_size to 1 is always a bad idea >> and most likely not what you want >> >> what is possible though is to either create the pool on the cli, >> or changing the min_size after creation to 1 (this is not blocked) >> > yes, Sute. I could be great to be able to change size/min_size from the > gui too. > > this should already possible in current versions, but as i said not for pool creation, only afterwards > >>> >>> >>> Some users with small cluster/budgets want to do only size=2, >>> >>> so with min_size=2, the cluster will go read only in case of any osd >>> down. >>> >>> It could be great to allow at least min_size=1 when size=2 is used. >>> >> >> "great" but very dangerous >> >>> >>> also, >>> Other setup like size=4, min_size=2, also display the warning, but >>> allow to validate the form. >>> >>> I'm not sure this warning is correct in this case , as since octopus, >>> min_size >>> is auto compute when pool is created, and a simple >>> >>> ceph osd pool create mypool 128 --size=4  , create a pool with >>> min_size=2 by default. >>> >>> >> >> the rationale behind this decision was (i think) because >> if you have exactly 50% min_size of size (e.g. 4/2) >> you can get inconsistent pgs, with no quorum as to >> which pg is correct? >> (though don't quote me on that) >> >> so i think its always better to have > 50% min_size of size >> > Well, afaik, they are no "quorum" on pg consistency for repair currently. > if a pg is corrupt, ceph is simply copy data from a pg copy where > checksum is ok. > and if no checksum is available, it take a random copy. (maybe it need a > manual pg_repair in this case). > But They are not something like "theses 2 copies have the more majority > (quorum) of checksum. > > (Maybe I'm wrong, but 1 or 2 year ago, Sage have confirmed this on the > ceph mailing) > > i thought more about 'inconsistent' pgs, maybe i am wrong but how does ceph cope with multiple 'valid' objects (all checksums are ok) but different content? (e.g, when during a write, theres a power cut?) i assumed that there a 'majority' must be established? although i did not find any document to support that, and in [0] it is only mentioned it will take the authoritative copy i'll discuss this with my colleages, and check more sources to maybe relax the '> 50%' rule a little for the warning thanks :) 0: https://docs.ceph.com/en/latest/rados/operations/pg-repair