From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 7480C609C3 for ; Thu, 26 Nov 2020 21:32:18 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 6B19115D04 for ; Thu, 26 Nov 2020 21:31:48 +0100 (CET) Received: from mail.merit.unu.edu (mail.merit.unu.edu [192.87.143.6]) by firstgate.proxmox.com (Proxmox) with ESMTP id E55CC15CFA for ; Thu, 26 Nov 2020 21:31:47 +0100 (CET) Received: from [10.242.6.10] (unknown [192.87.143.22]) by mail.merit.unu.edu (Postfix) with ESMTPSA id 7ADC4806EE751; Thu, 26 Nov 2020 21:31:41 +0100 (CET) To: Alejandro Bonilla Cc: Proxmox VE user list References: <5f9fa038-d250-65a8-d762-11ad1fd74955@merit.unu.edu> <7B860047-76E2-44B5-8F66-D04FFA216C07@suse.com> From: mj Message-ID: Date: Thu, 26 Nov 2020 21:31:40 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.3 MIME-Version: 1.0 In-Reply-To: <7B860047-76E2-44B5-8F66-D04FFA216C07@suse.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.175 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [PVE-User] confirmation on osd replacement X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 26 Nov 2020 20:32:18 -0000 Hi Alejandro, Thanks for your feedback, much appreciated! Enjoy your weekend! MJ On 11/26/20 4:39 PM, Alejandro Bonilla wrote: > > >> On Nov 26, 2020, at 2:54 AM, mj wrote: >> >> Hi, >> >> Yes, perhaps I should have given more details :-) >> >> On 11/25/20 3:03 PM, Alejandro Bonilla wrote: >> >>> Have a look at /etc/fstab for any disk path mounts - since I think Proxmox uses lvm mostly, you shouldn’t see a problem. >> I will, thanks! >> >>> What is the pool replication configuration or ec-profile? How many nodes in the cluster? >> We're 3/2 replication, no ec. It's a three-node (small) cluster, 8 filestore OSDs per node, with an SSD journal (wear evel 75%) > > If it’s 3 replicas, min 2, then you should be able to clear all drives from a system at once and replace them all to minimize the amount of times the cluster will end up rebalancing. > >> >> We will be using samsung PM833 bluestore of the same size >> (4TB spinners to 3.83GB PM833 SSDs) >> >>> Are you planning to remove all disks per server at once or disk by disk? >> I was planning to: >> - first add two SSDs to each server, and gradually increase their weight > > Two per server to ensure the disk replacement will work as expected is a good idea - I don’t think you’ll gain anything with a gradual re-weight. > >> - then, disk-by-disk, replace the 8 (old) spinners with the 6 remaining SSDs > > IF you have two other replicas, then a full system disk replacement should be no trouble - especially after two other SSDs were added and most data was shuffled around. > >> >>> Will all new drives equal or increase the disk capacity of the cluster? >> Approx equal yes. >> The aim in not to increase space. > > There are other reasons why I ask, specifically based on PG count and balancing of the cluster. > >> >> MJ >> >