From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 8A1781FF0A8 for ; Sat, 03 Oct 2026 15:16:53 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 8650A21712; Sat, 03 Oct 2026 15:16:50 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dualfroz.com; s=dkim; t=1791033405; h=from:subject:date:message-id:to:mime-version: content-transfer-encoding; bh=n2XoVMOXtIVYChOJKhSMr293cqDF1+iH8LauZGyof+Q=; b=mEhQ8+8ACoDB81PiWagbBSf1eySpbagV7W5lZViRlnntvH3BirisECtpYk1BfjtePE4Mrj C2mw150GrcdIftLZqtXBZVAK0z28Uw+VeH3MxbCaV7gc1FaaX4C8srka13ykCBxb97DEZ6 OQH78tlBowMudcjA3ELbnmNq3tUk3UzSsoIOZg8Yqw0lnSvO7tu5bOUAw5ap8juoA+ldqO ZABAAmwmDdbTujIxUtvmwN4U2HwS/t87pxDLXO4MYs5iDe4wNYUJubGRqjxi2Mz1eUhFdl AU9osqD9OA3fO3GM4kDragPt9Ss86pTMPrA0Rdgy0oZHGl+izgL/7zW12RZPww== From: Michal Fox To: pve-devel@lists.proxmox.com Subject: [PATCH cluster] fix #5794: cluster join: check that the interfaces of the links are up Date: Sat, 3 Oct 2026 13:16:42 +0000 Message-ID: <20261003131642.7-1-me@dualfroz.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Last-TLS-Session-Version: TLSv1.3 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.261 Adjusted score from AWL reputation of From: address DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain DMARC_PASS -0.1 DMARC pass policy SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: EWYPM2XKFMM7REV6P24W4AA77RPSPV5B X-Message-ID-Hash: EWYPM2XKFMM7REV6P24W4AA77RPSPV5B X-MailFrom: me@dualfroz.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Before joining a cluster, the IPs of the corosync links are checked to be configured on the node. An IP on an interface without carrier, for example with an unplugged cable, still counts as configured, so the join is started even though corosync cannot reach the other nodes. The join then fails halfway, and the node stays without quorum until the link comes up, which looks like a split brain. Additionally check the operational state of the interface the IP is configured on and refuse to join if it is not up. Interfaces that report their state as unknown, like dummy interfaces, are treated as up. Like the other checks, this can be overridden with 'force'. Signed-off-by: Michal Fox --- Tested with a script that runs assert_joinable() in a container with a veth pair: with the peer down, the IP of link0 is on an interface in state LOWERLAYERDOWN, and the join is refused with "link0: cannot use IP '10.100.0.29', interface 'veth0' is not up (LOWERLAYERDOWN)". With the peer up, the check passes. Missing IPs and interfaces that are administratively down are reported as before. Without the fix, the join is not refused. The corosync parser and MAC prefix tests pass. src/PVE/Cluster/Setup.pm | 20 ++++++++++++++++++-- 1 file changed, 18 insertions(+), 2 deletions(-) diff --git a/src/PVE/Cluster/Setup.pm b/src/PVE/Cluster/Setup.pm index 53935dc..6bc8f18 100644 --- a/src/PVE/Cluster/Setup.pm +++ b/src/PVE/Cluster/Setup.pm @@ -6,6 +6,7 @@ use warnings; use Digest::HMAC_SHA1; use Digest::SHA; use IO::File; +use JSON; use MIME::Base64; use Net::IP; use UUID; @@ -661,8 +662,23 @@ sub assert_joinable { my $cidr = (Net::IP::ip_is_ipv6($ip)) ? "$ip/128" : "$ip/32"; my $configured_ips = PVE::Network::get_local_ip_from_cidr($cidr); - $error->("$logid: cannot use IP '$ip', not found on local node!\n") - if scalar(@$configured_ips) < 1; + if (scalar(@$configured_ips) < 1) { + $error->("$logid: cannot use IP '$ip', not found on local node!\n"); + return; + } + + # the IP is also listed if the interface has no carrier, but corosync cannot use it then + my $addresses = ''; + PVE::Tools::run_command( + ['/sbin/ip', '-json', 'address', 'show', 'to', $cidr, 'up'], + outfunc => sub { $addresses .= shift }, + ); + for my $iface (decode_json($addresses)->@*) { + my $name = $iface->{ifname} or next; + my $state = $iface->{operstate} // 'UNKNOWN'; + next if $state eq 'UP' || $state eq 'UNKNOWN'; + $error->("$logid: cannot use IP '$ip', interface '$name' is not up ($state)\n"); + } }; $check_ip->($local_addr, 'local node address'); -- 2.43.0