From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id E46D66D850 for ; Tue, 28 Sep 2021 15:07:26 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id DB2E9F510 for ; Tue, 28 Sep 2021 15:06:56 +0200 (CEST) Received: from spider.fraudbuster.mobi (spider.fraudbuster.mobi [62.4.12.223]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 4D602F4FD for ; Tue, 28 Sep 2021 15:06:55 +0200 (CEST) Received: from sylvain-pc.fraudbuster.mobi (unknown [213.41.5.130]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by spider.fraudbuster.mobi (Postfix) with ESMTPSA id A8AE6233A6; Tue, 28 Sep 2021 14:57:10 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=fraudbuster.mobi; s=rsa-20200712; t=1632833830; bh=2J5rkTrLCHYdd5xl2bpoJcTeZhY5ymLzYb6zJ0xnIR4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=XVca8k2C1EGTtC+oHsvJYbYqZLO6fLMz8XrHBc9abEJHMQ+WcVe7NRVrTibwWhZLY kQqyiCazhukcdeRmiPGJjwExjzVtIfW4cJ2aMFtFC64nfwWMGniCTQHdsZNwRCWiUa dPbJkODw1ZnRpp8kLHukb0sJjr2iNr8BL+vt98sohNfKggNKGXKQApD19dz54bSYNl 6nXD+dnGFrDPPv6kQsZiZfVc9Mudka67Y1sB3iwLosFTJV6phdPlkS32nRqwP8U/dp psj3uLl4xD+qYJoovhltYg7gvvqIpCuYvflsWhdhq3+RZQDJh6/DG8DuinlKBfZO2l e29wQwuPnnwKQ== From: Sylvain Faivre To: pve-devel@lists.proxmox.com Cc: Sylvain Faivre Date: Tue, 28 Sep 2021 14:56:34 +0200 Message-Id: <20210928125634.169905-2-sylvain.faivre@fraudbuster.mobi> X-Mailer: git-send-email 2.30.2 In-Reply-To: <20210928125634.169905-1-sylvain.faivre@fraudbuster.mobi> References: <20210928125634.169905-1-sylvain.faivre@fraudbuster.mobi> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:8220, ipnet:213.41.0.0/17, country:GB]; IP_WHITELIST(0.00)[213.41.5.130] X-Rspamd-Pre-Result: action=no action; module=multimap; Matched map: IP_WHITELIST X-Rspamd-Queue-Id: A8AE6233A6 X-Rspamd-Server: spider X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [kronosnet.org, fraudbuster.mobi] X-Mailman-Approved-At: Tue, 28 Sep 2021 15:52:13 +0200 Subject: [pve-devel] [PATCH pve-docs 1/1] pvecm.adoc, pve-firewall.adoc: add info about ports used by corosync and others X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2021 13:07:26 -0000 --- pve-firewall.adoc | 9 ++++++--- pvecm.adoc | 6 +++--- 2 files changed, 9 insertions(+), 6 deletions(-) diff --git a/pve-firewall.adoc b/pve-firewall.adoc index f59c302..ca8acfe 100644 --- a/pve-firewall.adoc +++ b/pve-firewall.adoc @@ -426,7 +426,7 @@ following traffic is still allowed for all {pve} hosts in the cluster: * TCP traffic from management hosts to port 3128 for connections to the SPICE proxy * TCP traffic from management hosts to port 22 to allow ssh access -* UDP traffic in the cluster network to port 5404 and 5405 for corosync +* UDP traffic in the cluster network to ports 5405 and following ports for corosync. If you have setup redundant links, corosync port for each link is UDP/5405+linknumber [0-7] * UDP multicast traffic in the cluster network * ICMP traffic type 3 (Destination Unreachable), 4 (congestion control) or 11 (Time Exceeded) @@ -628,13 +628,16 @@ corresponding link local addresses. (See the Ports used by {pve} ------------------- -* Web interface: 8006 (TCP, HTTP/1.1 over TLS) +* Web interface: 8006 (TCP, HTTP/1.1 over TLS). Also needs to be open between nodes in a cluster, to allow operations in the web UI. * VNC Web console: 5900-5999 (TCP, WebSocket) * SPICE proxy: 3128 (TCP) * sshd (used for cluster actions): 22 (TCP) + +NOTE: You can run sshd on a non-standard port if you set this port in both the SSH client and server config, on all cluster nodes. + * rpcbind: 111 (UDP) * sendmail: 25 (TCP, outgoing) -* corosync cluster traffic: 5404, 5405 UDP +* corosync cluster traffic: 5405 (UDP) and following ports. If you have setup redundant links, corosync port for each link is UDP/5405+linknumber [0-7] * live migration (VM memory and local-disk data): 60000-60050 (TCP) ifdef::manvolnum[] diff --git a/pvecm.adoc b/pvecm.adoc index 0b1857e..07a8a66 100644 --- a/pvecm.adoc +++ b/pvecm.adoc @@ -58,8 +58,7 @@ Grouping nodes into a cluster has the following advantages: Requirements ------------ -* All nodes must be able to connect to each other via UDP ports 5404 and 5405 - for corosync to work. +* All nodes must be able to connect to each other via UDP ports 5405 and following ports for corosync to work. If you have setup redundant links, corosync port for each link is UDP/5405+linknumber [0-7]. * Date and time must be synchronized. @@ -524,7 +523,7 @@ be generated - no manual action is required. NOTE: Corosync used Multicast before version 3.0 (introduced in {pve} 6.0). Modern versions rely on https://kronosnet.org/[Kronosnet] for cluster -communication, which, for now, only supports regular UDP unicast. +communication, which, for now, only supports regular UDP unicast. More advanced information about Kronosnet can be found in http://people.redhat.com/ccaulfie/docs/KnetCorosync.pdf[KnetCorosync.pdf]. CAUTION: You can still enable Multicast or legacy unicast by setting your transport to `udp` or `udpu` in your xref:pvecm_edit_corosync_conf[corosync.conf], @@ -885,6 +884,7 @@ pvecm status If you see a healthy cluster state, it means that your new link is being used. +NOTE: If you experience communication problems, please check your firewall setup. With PVE 6.x+ and the introduction of Corosync3/Kronosnet, Corosync uses one port for each link, starting with port 5405. So the port number for each link is UDP/5405+linknumber [0-7]. Role of SSH in {pve} Clusters ----------------------------- -- 2.30.2