From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 8CD8A7540B for ; Mon, 12 Jul 2021 09:02:33 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 834461165D for ; Mon, 12 Jul 2021 09:02:03 +0200 (CEST) Received: from mx1.xpecto.de (mx1.xpecto.de [212.102.161.249]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 95B3111653 for ; Mon, 12 Jul 2021 09:02:01 +0200 (CEST) Received: from localhost (mx1.xpecto.de [127.0.0.1]) by mx1.xpecto.de (Postfix) with ESMTP id 45C7E320003 for ; Mon, 12 Jul 2021 09:01:55 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=xpecto.com; s=20170902; t=1626073315; bh=EviveOh9lklaBpbTQgWPzUXkPPJRBGDuXCdC0sKl0lQ=; h=From:To:Subject:Date:References:In-Reply-To:From; b=wBarXzuSI8scJ3z+P8TSHHbQT+hLHtKQU1SCBs5G/L5LRQMMjN8VQ7DiCZkc+XyYs TCaYqdfG3dzkjnMDG/06dYhP5lXfkeStLZMXp6XUFh/gzV6NJm0y0/XpAjTmVG/rIo UD1rxdewzZIMdis1fV9KrcSNB2jwP5uraKnvNkJO9GNZBp05vyGLFbcDBtIGLBfdwV AIZ/s4w748bzpbsmqQeTWS1dCvQkBsUlNVO6iTHgzZ5WuDO7SbIrV5voajKk9VJYce OmO3DIVAF1GaWalBIXVFjhLbpQtQna6snPKVojyODcYvXPmgRD9lDgz2SVEEytPlsC 8+FZ7+HN9CyXA== X-Virus-Scanned: Debian amavisd-new at mx1.xpecto.de X-Spam-Flag: NO X-Spam-Score: -50.59 X-Spam-Status: No, score=-50.59 tagged_above=-999 required=2 tests=[BAYES_00=-1, KHOP_HELO_FCRDNS=0.399, LOCAL_RCVD=-50, SPF_HELO_NONE=0.001, T_SPF_PERMERROR=0.01] autolearn=ham autolearn_force=no Received: from mx1.xpecto.de ([127.0.0.1]) by localhost (mx1.xpecto.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id KkGgTi3Cf9Is for ; Mon, 12 Jul 2021 09:01:54 +0200 (CEST) Received: from postman.xpecto.de (postman.xpecto.local [10.208.30.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mx1.xpecto.de (Postfix) with ESMTPS for ; Mon, 12 Jul 2021 09:01:54 +0200 (CEST) Received: from postman.xpecto.local (10.208.30.11) by postman.xpecto.local (10.208.30.11) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256) id 15.1.2242.10; Mon, 12 Jul 2021 09:01:53 +0200 Received: from postman.xpecto.local ([fe80::f965:5e8e:bcd:74f3]) by postman.xpecto.local ([fe80::f965:5e8e:bcd:74f3%4]) with mapi id 15.01.2242.010; Mon, 12 Jul 2021 09:01:53 +0200 From: Christoph Weber To: "'pve-user@lists.proxmox.com'" Thread-Topic: pve-user Digest, Vol 160, Issue 13 Thread-Index: AQHXdXJeHLf1kHsFBkOb/DFEOwDcYqs+4tWQ Date: Mon, 12 Jul 2021 07:01:53 +0000 Message-ID: <1375e99a83d247728c6213d6788800f5@xpecto.com> References: In-Reply-To: Accept-Language: de-DE, en-US Content-Language: de-DE X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.168.2.17] Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-SPAM-LEVEL: Spam detection results: 0 AWL -0.060 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain KAM_NUMSUBJECT 0.5 Subject ends in numbers excluding current years SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record T_SPF_PERMERROR 0.01 SPF: test of record failed (permerror) Subject: Re: [PVE-User] pve-user Digest, Vol 160, Issue 13 X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Jul 2021 07:02:33 -0000 Thanks for your advice, Ralf and Eneko , @Ralph > please read the documentation about removing/replacing a Node carefully > - you may not reinstall it with the same IP and/or Name as this will cras= h your > whole cluster! This is my bigges fear ... Thanks for pointing me to the manual - there is in fact an detailed explana= tion which files to keep before reinstalling the node under "Re-installing = a cluster node".=20 This is excactly what I was looking for, with detailed step-by-step walktro= ugh. Though I'm not sure if I will give it a try - looks a bit risky to me= ss things up in one of the many manual steps. > For the boot disks we use 2 or 3 mirrored zfs disks to be sure... Sounds like a good idea in retrospect ;-)=20 Until now I just was under the impression that the system disk does not con= tain very important data and can easily be replaced ... =20 @Eneko Lacunza > Do you have Ceph OSD journal/DB/WALs on system disk? No, we have everything belonging to one OSDs on the corresponding SSD drive= . > Moving OSDs from node3 to node6 would trigger data movement, but I'd go Will it? I thought it might just relocate the ODSs location in the crush ma= p from node3 to node6 when I shut down the prodve3, remove the disks and re= insert them in node 6? At least that was my impression in a thread here on = the mailinglist a few weeks ago. > node3, "pvecm delnode" it, reinstall with new system disk and rejoin clus= ter. This seems to be the best solution.=20 Maybe we will just discontinue the node, as it is getting near its planned = lifetime ...