From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id B3B7893F27 for ; Mon, 9 Jan 2023 12:48:41 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 8D16E2CE12 for ; Mon, 9 Jan 2023 12:48:11 +0100 (CET) Received: from mail.iper.net (mail.iper.net [94.236.32.73]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-SHA384 (256/256 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 9 Jan 2023 12:48:10 +0100 (CET) Received: from servercsa.csaricerche.com (UnknownHost [82.134.223.122]) by mail.iper.net with SMTP (version=TLS\Tls12 cipher=Aes256 bits=256); Mon, 9 Jan 2023 12:47:51 +0100 Received: from [192.168.64.90] (psala-lx.csaricerche.com [192.168.64.90]) (Authenticated sender: psala) by servercsa.csaricerche.com (Postfix) with ESMTPSA id 1B2DA201369B1 for ; Mon, 9 Jan 2023 12:47:53 +0100 (CET) Message-ID: Date: Mon, 9 Jan 2023 12:47:52 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.6.0 Content-Language: it, en-US To: pve-user@lists.proxmox.com References: <7606f44a-279c-c797-8206-95ea281245b1@riminilug.it> From: Piviul In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Rspamd-Server: servercsa.csaricerche.com X-Spamd-Result: default: False [0.97 / 20.00]; BAYES_SPAM(1.07)[82.27%]; MIME_GOOD(-0.10)[text/plain]; RCVD_COUNT_ZERO(0.00)[0]; RCPT_COUNT_ONE(0.00)[1]; MIME_TRACE(0.00)[0:+]; FROM_EQ_ENVFROM(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_NONE(0.00)[] X-Rspamd-Queue-Id: 1B2DA201369B1 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.459 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_SOFTFAIL 0.665 SPF: sender does not match SPF record (softfail) Subject: Re: [PVE-User] ceph X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 09 Jan 2023 11:48:41 -0000 On 1/9/23 10:54, Eneko Lacunza via pve-user wrote: > Hi, > > You need to route traffic between LAN network and Ceph network, so > that this works. When you have all monitors using ceph network IPs, > undo the routing. the routing table on a CEPH/PVE node is: $ ip route default via 192.168.64.1 dev vmbr0 proto kernel onlink 192.168.64.0/20 dev vmbr0 proto kernel scope link src 192.168.70.30 192.168.254.0/24 dev vmbr2 proto kernel scope link src 192.168.254.1 192.168.255.0/24 dev vmbr1 proto kernel scope link src 192.168.255.1 vmbr2 is the CEPH network, vmbr1 is the PVE network and vmbr0 is the LAN network. So you suggest me to add first the 3 ceph monitors using the CEPH IPs network and then destroy the 3 monitors having LAN IPs? Similarly for the ceph managers? Piviul