From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 174D91FF0E4 for ; Tue, 28 Jul 2026 13:37:35 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id AE81E206D8; Tue, 28 Jul 2026 13:37:34 +0200 (CEST) From: "Max R. Carrara" To: pve-devel@lists.proxmox.com Subject: [PATCH pve-manager v1] fix #5327: api: ceph: support multiple cluster networks on OSD creation Date: Tue, 28 Jul 2026 13:37:25 +0200 Message-ID: <20260728113729.2136104-1-m.carrara@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1785238612546 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.028 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_LOW -0.7 Sender listed at https://www.dnswl.org/, low trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: D6K7ESXE3EGLXEZAPGMXPRFGK3DCRLNV X-Message-ID-Hash: D6K7ESXE3EGLXEZAPGMXPRFGK3DCRLNV X-MailFrom: m.carrara@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Both the `cluster_network` and `public_network` fields in `ceph.conf` may contain a list of comma-separated CIDRs [0]. When a user has more than one CIDR specified in `cluster_network` -- or `public_network`, which is used as a fallback if `cluster_network` is empty -- our check here passes the entire string to our networking helpers without splitting it first. To fix this, split the `$osd_network` string that we obtain in a rather lenient manner and try to resolve the node's local IP for each CIDR. Additionally, `warn` if resolving a local IP fails, but do not error out as long as at least one IP can be resolved. OSD creation can therefore still succeed if part of the network is misconfigured. This can be useful in situations where the user has to e.g. work on or fix things on their Ceph cluster if their network happens to struggle at the same time, or they're dealing with other configuration issues. [0] https://docs.ceph.com/en/latest/rados/configuration/network-config-ref/#network-config-settings Fixes: https://bugzilla.proxmox.com/show_bug.cgi?id=5327 Reported-by: Alexander Zeidler Signed-off-by: Max R. Carrara --- PVE/API2/Ceph/OSD.pm | 43 ++++++++++++++++++++++++++++++++++++------- 1 file changed, 36 insertions(+), 7 deletions(-) diff --git a/PVE/API2/Ceph/OSD.pm b/PVE/API2/Ceph/OSD.pm index f6581564..c267adc8 100644 --- a/PVE/API2/Ceph/OSD.pm +++ b/PVE/API2/Ceph/OSD.pm @@ -344,14 +344,43 @@ __PACKAGE__->register_method({ $osd_network //= $ceph_conf->{global}->{public_network}; # fallback if ($osd_network) { # check only if something is configured - my $cluster_net_ips = PVE::Network::get_local_ip_from_cidr($osd_network); - if (scalar(@$cluster_net_ips) < 1) { - my $osd_net_obj = PVE::Network::IP_from_cidr($osd_network); - my $osd_base_cidr = $osd_net_obj->{ip} . "/" . $osd_net_obj->{prefixlen}; + # Multiple subnets may be specified for both cluster_network and public_network: + # https://docs.ceph.com/en/latest/rados/configuration/network-config-ref/#confval-cluster_network + # + # Split them in a more lenient manner and filter out empty strings + # for additional resilience + my $cidrs = [grep { $_ } split(/,+\s*/, $osd_network)]; - die - "No address from ceph cluster network (${osd_base_cidr}) found on node '$nodename'. " - . "Check your network config.\n"; + my $cluster_net_ips = []; + + for my $cidr ($cidrs->@*) { + next if !$cidr; + + my $ips = eval { PVE::Network::get_local_ip_from_cidr($cidr) }; + + if (my $err = $@) { + warn "Failed to resolve local IP from CIDR '$cidr' in Ceph configuration" + . " - $err\n"; + next; + } + + push($cluster_net_ips->@*, $ips->@*); + } + + if (scalar($cluster_net_ips->@*) < 1) { + my $osd_base_cidrs = []; + for my $cidr ($cidrs->@*) { + my $osd_net_obj = PVE::Network::IP_from_cidr($cidr); + push( + $osd_base_cidrs->@*, + $osd_net_obj->{ip} . "/" . $osd_net_obj->{prefixlen}, + ); + } + + my $base_cidrs_formatted = join(', ', $osd_base_cidrs->@*); + + die "No address from Ceph cluster network (${base_cidrs_formatted}) found" + . " on node '$nodename'. Check your network config.\n"; } } -- 2.47.3