* [PATCH v2 storage 01/13] multipath: add helper library and managed configuration
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 storage 02/13] api: disks: add read-only multipath status endpoint Thomas Lamprecht
` (11 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Multipath on PVE is configured by hand and per node today, with nothing
that keeps it consistent across a cluster. Add the foundation for
managing it cluster-wide instead.
The library reads the assembled maps and their health from multipathd.
The configuration is a SectionConfig kept in pmxcfs: one 'defaults'
section for the global multipathd knobs, plus one 'wwid' section per
allow-listed LUN holding its optional alias and any per-LUN knobs.
Parameters are kebab-case and rendered to multipathd's snake_case
keywords, validated through the section schema so a bad value cannot
reach the generated drop-in.
The managed baseline is deliberately conservative: it only assembles
explicitly allow-listed LUNs and keeps map names stable and WWID-based,
so a device is named the same on every node and an LVM PV on it stays
stable cluster-wide. Hardware-specific tuning lives in a separate,
admin-owned override rather than in the generated baseline, and the two
are written to distinct drop-ins, as multipath does not accept a
repeated 'defaults' section in one file.
Parsing and generation stay in a pure module with no dependency on
PVE::Cluster, so they remain unit-testable and usable on a node whose
pve-cluster does not yet observe the new file; registering it in pmxcfs
needs the matching pve-cluster change.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Changes in v2:
- prune only WWIDs that PVE itself added from /etc/multipath/wwids
- record ownership only for WWIDs PVE actually added, so a hand-made
entry that overlaps the cluster config is never adopted or pruned
- tear the drop-ins and the ownership record down when the cluster
config is emptied, instead of enforcing the defaults forever
- re-check for multipath-tools when it was absent, so a running
pvestatd notices a later install; report the missing package as an
apply error on nodes a multipath storage is enabled on
- double-quote generated values containing whitespace, multipath.conf
only accepts whitespace inside quoted strings
- harden the override guard against same-line brace tricks and
constrain path-selector values
- rename list_wwids to list_etc_multipath_wwids
- sync() returns its apply error instead of swallowing it
src/PVE/Makefile | 4 +
src/PVE/Multipath.pm | 395 +++++++++++++++++++++++++++++
src/PVE/Multipath/ClusterConfig.pm | 73 ++++++
src/PVE/Multipath/Config.pm | 380 +++++++++++++++++++++++++++
src/PVE/Multipath/Generator.pm | 190 ++++++++++++++
src/test/Makefile | 5 +-
src/test/run_multipath_tests.pl | 360 ++++++++++++++++++++++++++
7 files changed, 1406 insertions(+), 1 deletion(-)
create mode 100644 src/PVE/Multipath.pm
create mode 100644 src/PVE/Multipath/ClusterConfig.pm
create mode 100644 src/PVE/Multipath/Config.pm
create mode 100644 src/PVE/Multipath/Generator.pm
create mode 100755 src/test/run_multipath_tests.pl
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index 9e9f6aa..7ddd646 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -4,6 +4,10 @@
install:
install -D -m 0644 Storage.pm ${DESTDIR}${PERLDIR}/PVE/Storage.pm
install -D -m 0644 Diskmanage.pm ${DESTDIR}${PERLDIR}/PVE/Diskmanage.pm
+ install -D -m 0644 Multipath.pm ${DESTDIR}${PERLDIR}/PVE/Multipath.pm
+ install -D -m 0644 Multipath/Config.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Config.pm
+ install -D -m 0644 Multipath/ClusterConfig.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/ClusterConfig.pm
+ install -D -m 0644 Multipath/Generator.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Generator.pm
install -D -m 0644 CephConfig.pm ${DESTDIR}${PERLDIR}/PVE/CephConfig.pm
install -D -m 0644 GuestImport.pm ${DESTDIR}${PERLDIR}/PVE/GuestImport.pm
make -C Storage install
diff --git a/src/PVE/Multipath.pm b/src/PVE/Multipath.pm
new file mode 100644
index 0000000..64118b3
--- /dev/null
+++ b/src/PVE/Multipath.pm
@@ -0,0 +1,395 @@
+package PVE::Multipath;
+
+use strict;
+use warnings;
+
+use JSON qw(decode_json);
+
+use PVE::Tools qw(run_command file_read_firstline file_get_contents file_set_contents);
+
+use PVE::Multipath::Config;
+
+# Helper library around device-mapper multipath (multipathd). The single place that knows how to
+# talk to multipathd and turn its state into a normalized, stable structure for the rest of the
+# storage stack: health reporting, the API, and consumers that wait for a map before binding to it.
+#
+# Everything keys on the SCSI/NAA WWID, as reported by multipathd in the map's 'uuid' field. Map
+# names (mpathN / aliases) and the underlying 'sdX' paths are node-local and unstable; the WWID is
+# not.
+
+my $MULTIPATH = '/sbin/multipath';
+my $MULTIPATHD = '/sbin/multipathd';
+
+# health states for a single map, derived from its path states
+use constant {
+ HEALTH_OPTIMAL => 'optimal', # all paths active
+ HEALTH_DEGRADED => 'degraded', # some but not all paths active
+ HEALTH_FAILED => 'failed', # no active path left
+};
+
+my $supported;
+
+# Cached only while true: a long-lived daemon like pvestatd must notice a later multipath-tools
+# install without a restart, so a negative result is re-checked on every call.
+sub is_supported {
+ return $supported if $supported;
+ $supported = (-x $MULTIPATH && -x $MULTIPATHD) ? 1 : 0;
+ return $supported;
+}
+
+sub assert_supported {
+ die "no multipath support - please install 'multipath-tools'\n" if !is_supported();
+ return 1;
+}
+
+# Returns whether the multipathd daemon is reachable. Used for status output, so it never dies and
+# just reports 0 when multipath is unavailable or the daemon socket cannot be queried.
+sub is_running {
+ return 0 if !is_supported();
+
+ my $running = 0;
+ eval {
+ run_command(
+ [$MULTIPATHD, 'show', 'daemon'],
+ outfunc => sub {
+ my ($line) = @_;
+ # example: "pid 1234 idle" / "pid 1234 running"
+ $running = 1 if $line =~ m/^pid \d+ \S+/;
+ },
+ errfunc => sub { },
+ );
+ };
+ return $running;
+}
+
+# Runs `multipathd show <subcmd> json` and returns the raw output. Kept separate from parsing so
+# tests can feed recorded fixtures to the parse_*() functions without a running daemon.
+my sub query_multipathd_json {
+ my ($subcmd) = @_;
+
+ assert_supported();
+
+ my $output = '';
+ run_command(
+ [$MULTIPATHD, 'show', $subcmd, 'json'],
+ outfunc => sub { $output .= "$_[0]\n"; },
+ errfunc => sub { warn "$_[0]\n"; },
+ );
+
+ return $output;
+}
+
+my sub derive_health {
+ my ($paths_active, $paths_total) = @_;
+
+ return HEALTH_FAILED if !$paths_total || !$paths_active;
+ return HEALTH_OPTIMAL if $paths_active == $paths_total;
+ return HEALTH_DEGRADED;
+}
+
+my sub normalize_path {
+ my ($path) = @_;
+
+ my $res = {
+ dev => $path->{dev},
+ # 'active' or 'failed' - the state device-mapper sees
+ 'dm-state' => $path->{dm_st},
+ # 'running', 'faulty' or 'offline' - the state the kernel block layer sees
+ 'dev-state' => $path->{dev_st},
+ # path checker result, e.g. 'ready' / 'faulty' / 'ghost'
+ 'check-state' => $path->{chk_st},
+ };
+ $res->{priority} = int($path->{pri}) if defined($path->{pri});
+
+ # multipathd renders unset string fields as the literal '[undef]'
+ my $wwpn = $path->{target_wwpn};
+ my $hba = $path->{host_adapter};
+ undef $wwpn if !defined($wwpn) || $wwpn eq '[undef]' || $wwpn eq '';
+ undef $hba if !defined($hba) || $hba eq '[undef]' || $hba eq '';
+ $res->{'target-wwpn'} = $wwpn if defined($wwpn);
+ $res->{'host-adapter'} = $hba if defined($hba);
+ # a real target WWPN (0x...) means Fibre Channel; iSCSI/SAS transport is derived from sysfs by
+ # get_maps(). Do NOT treat the field's mere presence as FC, multipathd reports it as '[undef]'
+ # for iSCSI.
+ $res->{transport} = 'fc' if defined($wwpn) && $wwpn =~ /^0x[0-9a-f]+$/i;
+
+ return $res;
+}
+
+# Turns the output of `multipathd show maps json` into a normalized list of maps. Pure (no I/O) on
+# purpose: it derives everything it can from the JSON alone, so it can be unit-tested against
+# recorded fixtures. Live-only bits (byte size, transport) are added by get_maps() below.
+sub parse_maps_json {
+ my ($json) = @_;
+
+ my $data = eval { decode_json($json) };
+ die "could not parse multipathd maps JSON: $@\n" if $@;
+
+ my $maps = [];
+ for my $map (($data->{maps} // [])->@*) {
+ my $path_groups = [];
+ my ($paths_total, $paths_active) = (0, 0);
+
+ for my $group (($map->{path_groups} // [])->@*) {
+ my $paths = [];
+ for my $path (($group->{paths} // [])->@*) {
+ my $normalized = normalize_path($path);
+ $paths_total++;
+ $paths_active++ if ($normalized->{'dm-state'} // '') eq 'active';
+ push $paths->@*, $normalized;
+ }
+ push $path_groups->@*,
+ {
+ group => int($group->{group} // 0),
+ 'dm-state' => $group->{dm_st},
+ priority => int($group->{pri} // 0),
+ paths => $paths,
+ };
+ }
+
+ push $maps->@*, {
+ wwid => $map->{uuid},
+ name => $map->{name},
+ sysfs => $map->{sysfs}, # the 'dm-N' kernel name
+ 'dm-state' => $map->{dm_st},
+ 'paths-total' => $paths_total,
+ 'paths-active' => $paths_active,
+ health => derive_health($paths_active, $paths_total),
+ 'path-groups' => $path_groups,
+ };
+ }
+
+ return $maps;
+}
+
+# Best-effort byte size of a dm device from sysfs (Linux reports size in 512b sectors regardless of
+# the real block size).
+my sub dm_size_bytes {
+ my ($sysfs) = @_;
+
+ return undef if !$sysfs;
+ my $sectors = file_read_firstline("/sys/block/$sysfs/size");
+ return undef if !defined($sectors) || $sectors !~ m/^\d+$/;
+ return int($sectors) * 512;
+}
+
+my sub dir_has_entries {
+ my ($dir) = @_;
+
+ return 0 if !-d $dir;
+ opendir(my $dh, $dir) or return 0;
+ my @entries = grep { $_ ne '.' && $_ ne '..' } readdir($dh);
+ closedir($dh);
+ return scalar(@entries) ? 1 : 0;
+}
+
+# Best-effort transport of a single 'sdX' path from its sysfs topology; only iSCSI and SAS need it,
+# Fibre Channel is already set from the map JSON.
+my sub path_transport {
+ my ($dev) = @_;
+
+ return undef if !$dev;
+ my $link = readlink("/sys/block/$dev");
+ return undef if !$link;
+ return 'iscsi' if $link =~ m{/session\d+/};
+ return 'fc' if $link =~ m{/rport-\d+};
+ return 'sas' if $link =~ m{/end_device-};
+ return undef;
+}
+
+# Returns the normalized maps enriched with information that requires the local system (size, a
+# stable consumer path). Dies if multipath is not supported; callers that just want status should
+# guard with is_supported()/is_running().
+sub get_maps {
+ my $maps = parse_maps_json(query_multipathd_json('maps'));
+
+ for my $map ($maps->@*) {
+ $map->{size} = dm_size_bytes($map->{sysfs});
+ # WWID-stable path, present independently of the (node-local) map name
+ $map->{path} = "/dev/disk/by-id/dm-uuid-mpath-$map->{wwid}"
+ if defined($map->{wwid});
+ $map->{used} = dir_has_entries("/sys/block/$map->{sysfs}/holders")
+ if $map->{sysfs};
+
+ my %transports;
+ for my $group ($map->{'path-groups'}->@*) {
+ for my $path ($group->{paths}->@*) {
+ $path->{transport} //= path_transport($path->{dev});
+ $transports{ $path->{transport} } = 1 if defined($path->{transport});
+ }
+ }
+ # only expose a map-level transport when all paths agree on it
+ my @transports = keys %transports;
+ $map->{transport} = $transports[0] if scalar(@transports) == 1;
+ }
+
+ return $maps;
+}
+
+sub get_map_for_wwid {
+ my ($wwid) = @_;
+
+ for my $map (get_maps()->@*) {
+ return $map if defined($map->{wwid}) && $map->{wwid} eq $wwid;
+ }
+ return undef;
+}
+
+# Polls until a map for the given WWID exists, up to $timeout seconds. A consumer like the iSCSI
+# plugin uses this after a login or rescan to bind to the coalesced dm device rather than to a
+# transient single 'sdX' path.
+sub wait_for_map {
+ my ($wwid, $timeout) = @_;
+
+ $timeout //= 10;
+
+ my $deadline = time() + $timeout;
+ while (1) {
+ my $map = eval { get_map_for_wwid($wwid) };
+ return $map if $map;
+ return undef if time() >= $deadline;
+ sleep(1);
+ }
+}
+
+my $WWIDS_FILE = '/etc/multipath/wwids';
+
+# Node-local record of the WWIDs Proxmox VE added to $WWIDS_FILE, so the generator prunes only its
+# own entries and never ones from a hand-made or boot-from-SAN setup that share the file. Not
+# consulted by multipathd itself.
+my $MANAGED_WWIDS_FILE = '/etc/multipath/wwids.pve';
+
+# The WWIDs in /etc/multipath/wwids, the on-disk allow-list that multipathd assembles from with
+# 'find_multipaths strict'. Distinct from the cluster config's desired set (Config::wwid_list).
+sub list_etc_multipath_wwids {
+ return [] if !-e $WWIDS_FILE;
+ return PVE::Multipath::Config::parse_wwids(file_get_contents($WWIDS_FILE));
+}
+
+sub add_wwid {
+ my ($wwid) = @_;
+
+ assert_supported();
+ run_command([$MULTIPATH, '-a', $wwid]);
+}
+
+sub remove_wwid {
+ my ($wwid) = @_;
+
+ assert_supported();
+ run_command([$MULTIPATH, '-w', $wwid]);
+}
+
+# The WWIDs Proxmox VE added to the allow-list, from the node-local record (empty if none yet).
+sub managed_wwids {
+ return [] if !-e $MANAGED_WWIDS_FILE;
+ return PVE::Multipath::Config::parse_wwids(file_get_contents($MANAGED_WWIDS_FILE));
+}
+
+sub set_managed_wwids {
+ my ($wwids) = @_;
+
+ # an absent record means Proxmox VE owns nothing, the same state as before first use
+ if (!scalar($wwids->@*)) {
+ if (-e $MANAGED_WWIDS_FILE) {
+ unlink($MANAGED_WWIDS_FILE)
+ or die "could not remove '$MANAGED_WWIDS_FILE': $!\n";
+ }
+ return;
+ }
+ file_set_contents($MANAGED_WWIDS_FILE, PVE::Multipath::Config::format_wwids($wwids));
+}
+
+# Plan the allow-list changes for the generator: add the desired WWIDs that are not active yet, and
+# remove only WWIDs that Proxmox VE added before ($managed) and no longer wants, so entries from a
+# hand-made or boot-from-SAN setup are never touched. The arguments are sets (hashref of wwid => 1).
+sub plan_wwid_changes {
+ my ($desired, $current, $managed) = @_;
+
+ my @to_add = sort grep { !$current->{$_} } keys %$desired;
+ my @to_remove = sort grep { !$desired->{$_} && $current->{$_} } keys %$managed;
+ return (\@to_add, \@to_remove);
+}
+
+# Pure: the WWIDs Proxmox VE owns after a reconcile pass, for the node-local record that limits
+# future pruning. Ownership covers only entries PVE put into the allow-list itself: what it already
+# owned and still wants, what it just added ($added), and entries whose removal failed ($keep, so
+# the prune is retried). A desired WWID that was already in the allow-list from someone else stays
+# foreign, so dropping it from the cluster config later never prunes the hand-made entry. All
+# arguments are sets (hashref of wwid => 1).
+sub owned_wwid_record {
+ my ($desired, $managed, $added, $keep) = @_;
+
+ my %owned = map { $_ => 1 } (grep { $managed->{$_} || $added->{$_} } keys %$desired),
+ keys %$keep;
+ return [sort keys %owned];
+}
+
+# Pure: derive from the parsed cluster-wide storage configuration which nodes are expected to
+# carry multipath maps, and which storage consumes which LUN. A LUN consumed by an LVM storage
+# through a multipath base volume is expected wherever that chain is enabled, the intersection of
+# both node restrictions (an unrestricted storage counts as all nodes); any other allow-listed LUN
+# is expected wherever some multipath storage is enabled. Returns:
+# consumers { wwid => storeid } of the consuming LVM storage
+# nodes { node => 1 } union over all enabled multipath storages
+# 'wwid-nodes' { wwid => { node => 1 } } for consumed LUNs
+sub storage_expectations {
+ my ($storage_cfg, $all_nodes) = @_;
+
+ my $res = { consumers => {}, nodes => {}, 'wwid-nodes' => {} };
+ my $ids = $storage_cfg->{ids} // {};
+
+ my $storage_nodes = sub {
+ my ($scfg) = @_;
+ return { $scfg->{nodes}->%* } if $scfg->{nodes} && %{ $scfg->{nodes} };
+ return { map { $_ => 1 } $all_nodes->@* };
+ };
+
+ for my $storeid (keys %$ids) {
+ my $scfg = $ids->{$storeid};
+ next if ($scfg->{type} // '') ne 'multipath' || $scfg->{disable};
+ my $nodes = $storage_nodes->($scfg);
+ $res->{nodes}->{$_} = 1 for keys %$nodes;
+ }
+
+ for my $storeid (sort keys %$ids) {
+ my $scfg = $ids->{$storeid};
+ next if ($scfg->{type} // '') ne 'lvm' || !defined($scfg->{base});
+
+ # minimal 'storage:volname' split; the volname of a multipath base volume is the WWID
+ my ($baseid, $wwid) = $scfg->{base} =~ m/^([a-zA-Z][a-zA-Z0-9\-_.]*):(\S+)$/;
+ next if !defined($wwid);
+ my $basecfg = $ids->{$baseid};
+ next if !$basecfg || ($basecfg->{type} // '') ne 'multipath';
+
+ $res->{consumers}->{$wwid} = $storeid;
+ next if $scfg->{disable} || $basecfg->{disable};
+
+ my $base_nodes = $storage_nodes->($basecfg);
+ my $lvm_nodes = $storage_nodes->($scfg);
+ $res->{'wwid-nodes'}->{$wwid} =
+ { map { $_ => 1 } grep { $base_nodes->{$_} } keys %$lvm_nodes };
+ }
+
+ return $res;
+}
+
+# Live wrapper around storage_expectations(), reading the cluster-wide storage configuration and
+# node list. Never dies; degrades to empty expectations when either is unavailable.
+sub cluster_storage_expectations {
+ require PVE::Storage;
+ require PVE::Cluster;
+
+ my $cfg = eval { PVE::Storage::config() };
+ return { consumers => {}, nodes => {}, 'wwid-nodes' => {} } if !$cfg;
+ my $all_nodes = eval { PVE::Cluster::get_nodelist() } // [];
+ return storage_expectations($cfg, $all_nodes);
+}
+
+# Re-read the configuration and rebuild maps accordingly, after a config or allow-list change.
+sub reconfigure {
+ assert_supported();
+ run_command([$MULTIPATHD, 'reconfigure']);
+}
+
+1;
diff --git a/src/PVE/Multipath/ClusterConfig.pm b/src/PVE/Multipath/ClusterConfig.pm
new file mode 100644
index 0000000..aa8400e
--- /dev/null
+++ b/src/PVE/Multipath/ClusterConfig.pm
@@ -0,0 +1,73 @@
+package PVE::Multipath::ClusterConfig;
+
+use strict;
+use warnings;
+
+use Digest::SHA ();
+
+use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_write_file cfs_lock_file);
+
+use PVE::Multipath::Config;
+
+# Cluster-wide multipath configuration, replicated by pmxcfs. The structured allow-list, aliases and
+# knobs live in multipath.cfg (a SectionConfig); the free-form hardware override text lives in a
+# separate plain file so it stays hand-editable and diffable.
+my $FILENAME = 'multipath.cfg';
+my $OVERRIDES_FILENAME = 'multipath-overrides.conf';
+
+cfs_register_file(
+ $FILENAME,
+ sub { PVE::Multipath::Config->parse_config(@_); },
+ sub { PVE::Multipath::Config->write_config(@_); },
+);
+
+cfs_register_file(
+ $OVERRIDES_FILENAME,
+ \&PVE::Multipath::Config::parse_overrides,
+ \&PVE::Multipath::Config::write_overrides,
+);
+
+sub read_config {
+ return cfs_read_file($FILENAME);
+}
+
+sub write_config {
+ my ($cfg) = @_;
+ cfs_write_file($FILENAME, $cfg);
+}
+
+sub read_overrides {
+ return cfs_read_file($OVERRIDES_FILENAME);
+}
+
+sub write_overrides {
+ my ($text) = @_;
+ cfs_write_file($OVERRIDES_FILENAME, $text);
+}
+
+# Digest over the raw override text, to detect concurrent modifications; the SectionConfig digest
+# of multipath.cfg does not cover this separate file.
+sub overrides_digest {
+ my ($text) = @_;
+ return Digest::SHA::sha1_hex($text // '');
+}
+
+sub lock_config {
+ my ($code, $errmsg) = @_;
+
+ cfs_lock_file($FILENAME, undef, $code);
+ if (my $err = $@) {
+ $errmsg ? die "$errmsg: $err" : die $err;
+ }
+}
+
+sub lock_overrides {
+ my ($code, $errmsg) = @_;
+
+ cfs_lock_file($OVERRIDES_FILENAME, undef, $code);
+ if (my $err = $@) {
+ $errmsg ? die "$errmsg: $err" : die $err;
+ }
+}
+
+1;
diff --git a/src/PVE/Multipath/Config.pm b/src/PVE/Multipath/Config.pm
new file mode 100644
index 0000000..9db59df
--- /dev/null
+++ b/src/PVE/Multipath/Config.pm
@@ -0,0 +1,380 @@
+package PVE::Multipath::Config;
+
+use strict;
+use warnings;
+
+use PVE::SectionConfig;
+
+use base qw(PVE::SectionConfig);
+
+# Parser and writer for the cluster-wide source of truth in pmxcfs (/etc/pve/multipath.cfg). It is a
+# SectionConfig: a single 'defaults' section for global multipathd knobs, plus one 'wwid' section
+# per allow-listed LUN holding its optional alias and per-LUN knobs. Free-form hardware overrides
+# (device {} entries) live in a separate plain file, see PVE::Multipath::ClusterConfig. Kept pure so
+# it stays unit-testable without PVE::Cluster.
+
+# Conservative, cluster-friendly defaults, applied when the 'defaults' section omits them:
+# - find_multipaths strict -> only explicitly allow-listed LUNs get assembled, so boot/root and
+# unrelated disks stay untouched.
+# - user_friendly_names no -> the map name is the WWID, identical on every node, so an LVM PV on
+# /dev/disk/by-id/dm-uuid-mpath-<wwid> is stable cluster-wide without a node-local bindings file.
+my $MANAGED_DEFAULTS = {
+ 'find-multipaths' => 'strict',
+ 'user-friendly-names' => 'no',
+ 'polling-interval' => 5,
+};
+
+sub managed_defaults { return { $MANAGED_DEFAULTS->%* }; }
+
+# Knobs valid both globally (defaults) and per-LUN (a multipaths{} entry), per multipath.conf(5).
+my $shared_knobs = {
+ 'no-path-retry' => {
+ type => 'string',
+ pattern => '(?:queue|fail|\d+)',
+ typetext => 'queue|fail|<count>',
+ description =>
+ "How to react when all paths are down: keep queuing, fail at once, or retry"
+ . " for the given number of polling intervals.",
+ optional => 1,
+ },
+ 'path-grouping-policy' => {
+ type => 'string',
+ enum => [qw(failover multibus group_by_serial group_by_prio group_by_node_name)],
+ description => "How paths are grouped into priority groups.",
+ optional => 1,
+ },
+ failback => {
+ type => 'string',
+ pattern => '(?:manual|immediate|followover|\d+)',
+ typetext => 'manual|immediate|followover|<seconds>',
+ description => "When to fail back to a restored higher-priority path group.",
+ optional => 1,
+ },
+ 'path-selector' => {
+ type => 'string',
+ pattern =>
+ '(?:round-robin|queue-length|service-time|historical-service-time|io-affinity) \d+',
+ typetext => '<selector> <version>',
+ description => "Path selector algorithm used within a priority group, for example"
+ . " 'service-time 0'.",
+ optional => 1,
+ },
+};
+
+# Knobs that only make sense globally.
+my $defaults_only_knobs = {
+ 'find-multipaths' => {
+ type => 'string',
+ enum => [qw(yes no strict greedy smart)],
+ default => 'strict',
+ description => "Which devices multipathd assembles into a map. 'strict' only takes"
+ . " explicitly allow-listed WWIDs.",
+ optional => 1,
+ },
+ 'user-friendly-names' => {
+ type => 'string',
+ enum => [qw(yes no)],
+ default => 'no',
+ description => "Whether to use node-local mpathN names. Keep 'no' for stable WWID-based"
+ . " names across the cluster.",
+ optional => 1,
+ },
+ 'polling-interval' => {
+ type => 'integer',
+ minimum => 1,
+ default => 5,
+ description => "Interval between path checks, in seconds.",
+ optional => 1,
+ },
+};
+
+# Knobs that only make sense per-LUN.
+my $wwid_only_knobs = {
+ alias => {
+ type => 'string',
+ pattern => '[a-zA-Z0-9][a-zA-Z0-9._-]*',
+ maxLength => 64,
+ description =>
+ "Human-readable map name for this WWID; multipathd uses it as the map name.",
+ optional => 1,
+ },
+ 'rr-min-io-rq' => {
+ type => 'integer',
+ minimum => 1,
+ description =>
+ "Number of I/O requests to route to a path before switching, request-based.",
+ optional => 1,
+ },
+ 'rr-weight' => {
+ type => 'string',
+ enum => [qw(priorities uniform)],
+ description => "Whether to weight paths by priority when balancing I/O.",
+ optional => 1,
+ },
+};
+
+my $defaultData = {
+ propertyList => {
+ type => { description => "Section type ('defaults' or 'wwid')." },
+ id => {
+ type => 'string',
+ description =>
+ "Section ID: the literal 'defaults', or a LUN WWID for 'wwid' sections.",
+ pattern => '[a-zA-Z0-9._:-]+',
+ maxLength => 128,
+ },
+ $shared_knobs->%*,
+ $defaults_only_knobs->%*,
+ $wwid_only_knobs->%*,
+ },
+};
+
+sub private { return $defaultData; }
+
+package PVE::Multipath::Config::Defaults;
+
+use base qw(PVE::Multipath::Config);
+
+sub type { return 'defaults'; }
+
+sub options {
+ return {
+ 'find-multipaths' => { optional => 1 },
+ 'user-friendly-names' => { optional => 1 },
+ 'polling-interval' => { optional => 1 },
+ 'no-path-retry' => { optional => 1 },
+ 'path-grouping-policy' => { optional => 1 },
+ failback => { optional => 1 },
+ 'path-selector' => { optional => 1 },
+ };
+}
+
+__PACKAGE__->register();
+
+package PVE::Multipath::Config::Wwid;
+
+use base qw(PVE::Multipath::Config);
+
+sub type { return 'wwid'; }
+
+sub options {
+ return {
+ alias => { optional => 1 },
+ 'no-path-retry' => { optional => 1 },
+ 'path-grouping-policy' => { optional => 1 },
+ failback => { optional => 1 },
+ 'path-selector' => { optional => 1 },
+ 'rr-min-io-rq' => { optional => 1 },
+ 'rr-weight' => { optional => 1 },
+ };
+}
+
+__PACKAGE__->register();
+
+package PVE::Multipath::Config;
+
+__PACKAGE__->init();
+
+# multipathd subsections accept only these top-level keywords; the admin override file is checked
+# against them. 'multipaths' is generated from the wwid sections, so an admin block would collide.
+my $OVERRIDE_KEYWORDS =
+ { devices => 1, overrides => 1, defaults => 1, blacklist => 1, blacklist_exceptions => 1 };
+
+# Validate the free-form override text before it can break multipathd's parser cluster-wide. This is
+# a guard, not a full parse: balanced braces, only known top-level sections, and no 'multipaths'
+# block (that is generated from the wwid sections and a duplicate is fatal to multipathd).
+sub check_overrides {
+ my ($text) = @_;
+
+ return if !defined($text) || $text !~ /\S/;
+
+ # walk every block opener and brace in order, so a section keyword after a closing brace on
+ # the same line is still checked at its real depth
+ my $depth = 0;
+ for my $line (split(/\n/, $text)) {
+ next if $line =~ /^\s*#/;
+ while ($line =~ /(\w+)\s*\{|([{}])/g) {
+ if (defined($1)) {
+ my $kw = $1;
+ die "multipath overrides: 'multipaths' is managed via aliases, do not set it"
+ . " here\n"
+ if $kw eq 'multipaths';
+ die "multipath overrides: unknown top-level section '$kw'\n"
+ if $depth == 0 && !$OVERRIDE_KEYWORDS->{$kw};
+ $depth++;
+ } elsif ($2 eq '{') {
+ $depth++;
+ } else {
+ die "multipath overrides: unbalanced braces\n" if $depth == 0;
+ $depth--;
+ }
+ }
+ }
+ die "multipath overrides: unbalanced braces\n" if $depth != 0;
+
+ return;
+}
+
+# Read/write the separate, admin-owned override file (/etc/pve/multipath-overrides.conf). Stored and
+# rendered verbatim, so it stays hand-editable and diffable.
+sub parse_overrides {
+ my ($filename, $raw) = @_;
+ return $raw // '';
+}
+
+sub write_overrides {
+ my ($filename, $text) = @_;
+ $text //= '';
+ $text =~ s/\s+$//;
+ return length($text) ? "$text\n" : '';
+}
+
+my $MANAGED_HEADER =
+ "# This file is managed by Proxmox VE - do not edit by hand.\n"
+ . "# Hardware-/node-specific overrides belong in the override config.\n";
+
+# A multipath.conf value must be a single word or one double-quoted string; only the quoted form
+# can carry whitespace, see multipath.conf(5). The schemas admit no double quotes in values, so no
+# escaping is needed.
+my sub render_value {
+ my ($value) = @_;
+ return $value =~ /\s/ ? "\"$value\"" : $value;
+}
+
+# Renders a named section from a key => value hash, keys sorted for a stable, diffable result. The
+# config and API use kebab-case parameters, multipathd keywords are snake_case, so map '-' to '_'.
+my sub render_section {
+ my ($name, $kv) = @_;
+
+ my $out = "$name {\n";
+ for my $key (sort keys $kv->%*) {
+ (my $keyword = $key) =~ tr/-/_/;
+ $out .= "\t$keyword " . render_value($kv->{$key}) . "\n";
+ }
+ $out .= "}\n";
+ return $out;
+}
+
+# Builds the Proxmox-managed baseline drop-in (header + defaults section) from the effective global
+# knobs. Admin overrides are not merged in here: they go into a separate conf.d file, as multipath
+# rejects two 'defaults' blocks in one file (duplicate keyword) and drops the second.
+sub generate_managed_conf {
+ my ($defaults) = @_;
+ $defaults //= managed_defaults();
+
+ return $MANAGED_HEADER . "\n" . render_section('defaults', $defaults);
+}
+
+# The WWID allow-list file (/etc/multipath/wwids) holds one '/<wwid>/' per line; parse it and back.
+sub parse_wwids {
+ my ($text) = @_;
+
+ my $wwids = [];
+ for my $line (split(/\n/, $text // '')) {
+ next if $line =~ /^\s*#/;
+ next if $line =~ /^\s*$/;
+ if ($line =~ m{^/(.+)/\s*$}) {
+ push $wwids->@*, $1;
+ }
+ }
+ return $wwids;
+}
+
+sub format_wwids {
+ my ($wwids) = @_;
+
+ my $out = "# Multipath wwids, managed by Proxmox VE\n";
+ $out .= "/$_/\n" for sort $wwids->@*;
+ return $out;
+}
+
+# Builds a 'multipaths {}' block from the per-WWID sections (alias plus any per-LUN knobs); returns
+# the empty string when no WWID has an alias or a knob set.
+sub build_multipaths_block {
+ my ($wwid_opts) = @_;
+
+ my @entries = grep { %{ $wwid_opts->{$_} } } sort keys %$wwid_opts;
+ return '' if !@entries;
+
+ my $out = "multipaths {\n";
+ for my $wwid (@entries) {
+ my $opts = $wwid_opts->{$wwid};
+ $out .= "\tmultipath {\n";
+ $out .= "\t\twwid $wwid\n";
+ for my $key (sort keys %$opts) {
+ (my $keyword = $key) =~ tr/-/_/;
+ $out .= "\t\t$keyword " . render_value($opts->{$key}) . "\n";
+ }
+ $out .= "\t}\n";
+ }
+ $out .= "}\n";
+ return $out;
+}
+
+# The knob property definitions as an API parameter schema. Strip the schema 'default' so an update
+# that omits a knob leaves it unchanged instead of resetting it to the managed default.
+my sub api_schema {
+ my ($props) = @_;
+
+ my $res = {};
+ for my $key (keys %$props) {
+ $res->{$key} = { $props->{$key}->%* };
+ delete $res->{$key}->{default};
+ }
+ return $res;
+}
+
+# Settable global knobs (the 'defaults' section) as an API parameter schema.
+sub defaults_api_schema {
+ return api_schema({ $shared_knobs->%*, $defaults_only_knobs->%* });
+}
+
+# Settable per-WWID knobs (including the alias) as an API parameter schema.
+sub wwid_api_schema {
+ return api_schema({ $shared_knobs->%*, $wwid_only_knobs->%* });
+}
+
+# Effective global knobs: the 'defaults' section merged onto the conservative managed defaults.
+sub effective_defaults {
+ my ($cfg) = @_;
+
+ my $defaults = managed_defaults();
+ if (my $section = $cfg->{ids}->{defaults}) {
+ $defaults->{$_} = $section->{$_} for grep { $_ ne 'type' } keys %$section;
+ }
+ return $defaults;
+}
+
+# The allow-listed WWIDs, that is the ids of the 'wwid' sections.
+sub wwid_list {
+ my ($cfg) = @_;
+ return [sort grep { ($cfg->{ids}->{$_}->{type} // '') eq 'wwid' } keys $cfg->{ids}->%*];
+}
+
+# { wwid => alias } for the WWIDs that have one.
+sub aliases {
+ my ($cfg) = @_;
+
+ my $res = {};
+ for my $wwid (keys $cfg->{ids}->%*) {
+ my $section = $cfg->{ids}->{$wwid};
+ next if ($section->{type} // '') ne 'wwid';
+ $res->{$wwid} = $section->{alias} if defined($section->{alias});
+ }
+ return $res;
+}
+
+# { wwid => { alias?, knob => value, ... } }, the per-WWID input to build_multipaths_block().
+sub wwid_opts {
+ my ($cfg) = @_;
+
+ my $res = {};
+ for my $wwid (keys $cfg->{ids}->%*) {
+ my $section = $cfg->{ids}->{$wwid};
+ next if ($section->{type} // '') ne 'wwid';
+ $res->{$wwid} = { map { $_ => $section->{$_} } grep { $_ ne 'type' } keys %$section };
+ }
+ return $res;
+}
+
+1;
diff --git a/src/PVE/Multipath/Generator.pm b/src/PVE/Multipath/Generator.pm
new file mode 100644
index 0000000..b840e9f
--- /dev/null
+++ b/src/PVE/Multipath/Generator.pm
@@ -0,0 +1,190 @@
+package PVE::Multipath::Generator;
+
+use strict;
+use warnings;
+
+use File::Path qw(make_path);
+
+use PVE::Tools qw(file_get_contents file_set_contents);
+
+use PVE::Multipath;
+use PVE::Multipath::Config;
+use PVE::Multipath::ClusterConfig;
+
+# Renders the effective node-local multipath configuration from the cluster-wide source of truth
+# (/etc/pve/multipath.cfg) and reloads multipathd when something changed.
+#
+# The rendered files live on the local filesystem, so they survive reboots and are available to
+# multipathd at boot even before pmxcfs is up; the last successful render is the boot-time fallback.
+
+# Proxmox-owned drop-ins; the admin's /etc/multipath.conf keeps its default 'config_dir
+# /etc/multipath/conf.d'. The managed baseline, the admin overrides, and the generated aliases each
+# get their own file so multipath merges them across files instead of hitting a duplicate section
+# keyword: two 'defaults' blocks in one file are rejected outright, and our 'multipaths' alias block
+# would clash with a 'multipaths' section in the overrides. The overrides file sorts after the
+# baseline, so an admin's defaults override it; the aliases file is a separate 'multipaths' block,
+# so its order does not matter.
+my $DEFAULTS_DROPIN = '/etc/multipath/conf.d/pve-defaults.conf';
+my $OVERRIDES_DROPIN = '/etc/multipath/conf.d/pve-overrides.conf';
+my $ALIASES_DROPIN = '/etc/multipath/conf.d/pve-aliases.conf';
+
+my sub write_if_changed {
+ my ($path, $content) = @_;
+
+ my $old = -e $path ? eval { file_get_contents($path) } : undef;
+ return 0 if defined($old) && $old eq $content;
+
+ my $dir = $path =~ s!/[^/]+$!!r;
+ make_path($dir) if !-d $dir;
+ file_set_contents($path, $content);
+ return 1;
+}
+
+my sub remove_if_present {
+ my ($path) = @_;
+
+ return 0 if !-e $path;
+ unlink($path) or die "could not remove '$path': $!\n";
+ return 1;
+}
+
+# Whether the cluster-wide configuration has anything for the nodes to render; when it does not,
+# the generator tears its local files down instead of enforcing the managed defaults forever, so
+# the node falls back to a hand-managed or pristine multipath setup.
+my sub config_in_use {
+ my ($cfg, $overrides) = @_;
+
+ return
+ scalar(PVE::Multipath::Config::wwid_list($cfg)->@*)
+ || (defined($overrides) && length($overrides))
+ || $cfg->{ids}->{defaults};
+}
+
+sub regenerate {
+ my ($cfg, $overrides) = @_;
+ $cfg //= PVE::Multipath::ClusterConfig::read_config();
+ $overrides //= PVE::Multipath::ClusterConfig::read_overrides();
+
+ my $changed = 0;
+
+ if (config_in_use($cfg, $overrides)) {
+ my $defaults = PVE::Multipath::Config::effective_defaults($cfg);
+ $changed = 1
+ if write_if_changed(
+ $DEFAULTS_DROPIN,
+ PVE::Multipath::Config::generate_managed_conf($defaults),
+ );
+ } else {
+ $changed = 1 if remove_if_present($DEFAULTS_DROPIN);
+ }
+
+ if (defined($overrides) && length($overrides)) {
+ my $content =
+ "# Managed by Proxmox VE - edit overrides in /etc/pve/multipath-overrides.conf.\n\n"
+ . "$overrides\n";
+ $changed = 1 if write_if_changed($OVERRIDES_DROPIN, $content);
+ } else {
+ $changed = 1 if remove_if_present($OVERRIDES_DROPIN);
+ }
+
+ my $block =
+ PVE::Multipath::Config::build_multipaths_block(PVE::Multipath::Config::wwid_opts($cfg));
+ if (length($block)) {
+ my $content =
+ "# Managed by Proxmox VE - edit aliases and per-LUN options in /etc/pve/multipath.cfg.\n\n"
+ . $block;
+ $changed = 1 if write_if_changed($ALIASES_DROPIN, $content);
+ } else {
+ $changed = 1 if remove_if_present($ALIASES_DROPIN);
+ }
+
+ # Bring the WWID allow-list (/etc/multipath/wwids) in line with the cluster config through
+ # multipath's own add/remove, so its on-disk format stays intact. Prune only WWIDs that Proxmox
+ # VE added itself (tracked node-locally), never ones from a hand-made or boot-from-SAN setup that
+ # share the file. Isolate each op: one failing WWID must not abort the whole pass, or it would
+ # stall every other WWID on every run; a failed op leaves the file unchanged and is retried next
+ # pass.
+ my %desired = map { $_ => 1 } PVE::Multipath::Config::wwid_list($cfg)->@*;
+ my %current = map { $_ => 1 } PVE::Multipath::list_etc_multipath_wwids()->@*;
+ my %managed = map { $_ => 1 } PVE::Multipath::managed_wwids()->@*;
+ my ($to_add, $to_remove) = PVE::Multipath::plan_wwid_changes(\%desired, \%current, \%managed);
+
+ my @errors;
+ my %added; # WWIDs this pass actually put into the allow-list, the only new ones PVE owns
+ for my $wwid ($to_add->@*) {
+ eval { PVE::Multipath::add_wwid($wwid); };
+ if (my $err = $@) {
+ push @errors, "adding WWID '$wwid' failed - $err";
+ } else {
+ $changed = 1;
+ $added{$wwid} = 1;
+ }
+ }
+ my %kept; # WWIDs whose prune failed: keep owning them so the removal is retried
+ for my $wwid ($to_remove->@*) {
+ eval { PVE::Multipath::remove_wwid($wwid); };
+ if (my $err = $@) {
+ push @errors, "removing WWID '$wwid' failed - $err";
+ $kept{$wwid} = 1;
+ } else {
+ $changed = 1;
+ }
+ }
+
+ # remember what Proxmox VE now owns so the next pass prunes only its own entries; write only when
+ # the set actually changes, to avoid needless churn
+ my $owned = PVE::Multipath::owned_wwid_record(\%desired, \%managed, \%added, \%kept);
+ if (join("\0", $owned->@*) ne join("\0", sort keys %managed)) {
+ eval { PVE::Multipath::set_managed_wwids($owned); };
+ push @errors, "recording managed WWIDs failed - $@" if $@;
+ }
+
+ # reload the daemon for whatever did converge, even if some ops failed
+ if ($changed && PVE::Multipath::is_running()) {
+ eval { PVE::Multipath::reconfigure(); };
+ push @errors, "reconfigure failed - $@" if $@;
+ }
+
+ die join('', @errors) if @errors;
+
+ return $changed;
+}
+
+# Safe periodic entry point for a status loop like pvestatd: a no-op when multipath is not in use on
+# this node, and never throws. Returns undef on success or a no-op, or an error message the caller
+# can surface (for example by broadcasting it) so a drifted node does not fail silently.
+sub sync {
+ my $cfg = eval { PVE::Multipath::ClusterConfig::read_config() };
+ return "reading cluster config failed - $@" if $@;
+
+ my $overrides = eval { PVE::Multipath::ClusterConfig::read_overrides() };
+ my $in_use = config_in_use($cfg, $overrides);
+
+ if (!PVE::Multipath::is_supported()) {
+ # a node a multipath storage is enabled on cannot apply an in-use config without the
+ # tools, so report that instead of silently showing up as 'missing' in the health matrix;
+ # unconcerned nodes (quorum or compute-only) stay silent
+ return if !$in_use;
+ require PVE::INotify;
+ my $node = PVE::INotify::nodename();
+ my $expectations = PVE::Multipath::cluster_storage_expectations();
+ return if !$expectations->{nodes}->{$node};
+ return "multipath-tools is not installed\n";
+ }
+
+ # stay out of the way unless the feature is in use or its local files still need a teardown
+ # (a leftover drop-in or managed-WWIDs record means a prior pass failed partway; retry it)
+ return
+ if !$in_use
+ && !-e $DEFAULTS_DROPIN
+ && !-e $OVERRIDES_DROPIN
+ && !-e $ALIASES_DROPIN
+ && !scalar(PVE::Multipath::managed_wwids()->@*);
+
+ eval { regenerate($cfg, $overrides) };
+ return "$@" if $@;
+
+ return;
+}
+
+1;
diff --git a/src/test/Makefile b/src/test/Makefile
index ee025bc..51c7360 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_zfspoolplugin test_lvmplugin test_disklist test_bwlimit test_plugin test_ovf test_volume_access
+test: test_zfspoolplugin test_lvmplugin test_disklist test_bwlimit test_plugin test_ovf test_volume_access test_multipath
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -22,3 +22,6 @@ test_ovf: run_ovf_tests.pl
test_volume_access: run_volume_access_tests.pl
./run_volume_access_tests.pl
+
+test_multipath: run_multipath_tests.pl
+ ./run_multipath_tests.pl
diff --git a/src/test/run_multipath_tests.pl b/src/test/run_multipath_tests.pl
new file mode 100755
index 0000000..09b6061
--- /dev/null
+++ b/src/test/run_multipath_tests.pl
@@ -0,0 +1,360 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use JSON;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Multipath;
+use PVE::Multipath::Config;
+
+# A recorded `multipathd show maps json` reply with three maps exercising each
+# health state: an all-active map, a partially-failed map and an all-failed map.
+my $maps_json = <<'EOF';
+{
+ "major_version": 0,
+ "minor_version": 1,
+ "maps": [
+ {
+ "name": "mpatha",
+ "uuid": "3600140500a1b2c3d4e5f6a7b8c9d0e1f",
+ "sysfs": "dm-0",
+ "dm_st": "active",
+ "paths": 2,
+ "path_groups": [
+ {
+ "group": 1,
+ "dm_st": "active",
+ "pri": 50,
+ "paths": [
+ { "dev": "sdb", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 50, "target_wwpn": "0x500a098000000001" },
+ { "dev": "sdc", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 50, "target_wwpn": "0x500a098000000002" }
+ ]
+ }
+ ]
+ },
+ {
+ "name": "mpathb",
+ "uuid": "360014050aabbccddeeff00112233445566",
+ "sysfs": "dm-1",
+ "dm_st": "active",
+ "paths": 2,
+ "path_groups": [
+ {
+ "group": 1,
+ "dm_st": "active",
+ "pri": 10,
+ "paths": [
+ { "dev": "sdd", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 10 }
+ ]
+ },
+ {
+ "group": 2,
+ "dm_st": "enabled",
+ "pri": 0,
+ "paths": [
+ { "dev": "sde", "dm_st": "failed", "dev_st": "faulty", "chk_st": "faulty", "pri": 0 }
+ ]
+ }
+ ]
+ },
+ {
+ "name": "mpathc",
+ "uuid": "36001405ffffffffffffffffffffffffff",
+ "sysfs": "dm-2",
+ "dm_st": "active",
+ "paths": 1,
+ "path_groups": [
+ {
+ "group": 1,
+ "dm_st": "enabled",
+ "pri": 0,
+ "paths": [
+ { "dev": "sdf", "dm_st": "failed", "dev_st": "faulty", "chk_st": "faulty", "pri": 0, "target_wwpn": "[undef]", "host_adapter": "[undef]" }
+ ]
+ }
+ ]
+ }
+ ]
+}
+EOF
+
+my $maps = PVE::Multipath::parse_maps_json($maps_json);
+
+is(scalar($maps->@*), 3, 'parsed all three maps');
+
+my ($a, $b, $c) = $maps->@*;
+
+# fully healthy map
+is($a->{wwid}, '3600140500a1b2c3d4e5f6a7b8c9d0e1f', 'map a wwid taken from uuid');
+is($a->{name}, 'mpatha', 'map a name');
+is($a->{sysfs}, 'dm-0', 'map a sysfs name');
+is($a->{'paths-total'}, 2, 'map a counts both paths');
+is($a->{'paths-active'}, 2, 'map a has two active paths');
+is($a->{health}, 'optimal', 'map a is optimal');
+is(scalar($a->{'path-groups'}->@*), 1, 'map a has one path group');
+is(
+ $a->{'path-groups'}->[0]->{paths}->[0]->{'target-wwpn'},
+ '0x500a098000000001',
+ 'FC target wwpn is preserved',
+);
+is(
+ $a->{'path-groups'}->[0]->{paths}->[0]->{transport},
+ 'fc',
+ 'transport derived as fc from a target wwpn',
+);
+
+# one failed path out of two
+is($b->{'paths-total'}, 2, 'map b counts both paths across groups');
+is($b->{'paths-active'}, 1, 'map b has one active path');
+is($b->{health}, 'degraded', 'map b is degraded');
+
+# no active path left
+is($c->{'paths-total'}, 1, 'map c counts its single path');
+is($c->{'paths-active'}, 0, 'map c has no active path');
+is($c->{health}, 'failed', 'map c is failed');
+ok(
+ !defined($c->{'path-groups'}->[0]->{paths}->[0]->{'target-wwpn'}),
+ "multipathd '[undef]' target_wwpn is cleaned away (not stored)",
+);
+ok(
+ !defined($c->{'path-groups'}->[0]->{paths}->[0]->{transport}),
+ "'[undef]' target_wwpn does not imply fc transport",
+);
+
+# empty / no maps must parse to an empty list, not die
+my $empty = PVE::Multipath::parse_maps_json('{ "major_version": 0, "maps": [] }');
+is_deeply($empty, [], 'no maps parses to empty list');
+
+# malformed input must die with a clear error
+eval { PVE::Multipath::parse_maps_json('not json') };
+ok($@ =~ m/could not parse multipathd maps JSON/, 'invalid JSON raises a clear error');
+
+# --- config generation / WWID allow-list ---
+my $conf = PVE::Multipath::Config::generate_managed_conf();
+like($conf, qr/managed by Proxmox VE/, 'managed conf carries the managed header');
+like($conf, qr/user_friendly_names no/, 'baseline sets user_friendly_names no');
+like($conf, qr/find_multipaths strict/, 'baseline opts in explicitly via find_multipaths strict');
+is(
+ scalar(() = $conf =~ /^defaults \{/mg),
+ 1,
+ 'baseline has exactly one defaults block (a second would be a duplicate-keyword error)',
+);
+
+# multipath.conf only allows whitespace inside double-quoted values
+like(
+ PVE::Multipath::Config::generate_managed_conf({
+ PVE::Multipath::Config::managed_defaults()->%*, 'path-selector' => 'service-time 0',
+ }),
+ qr/^\tpath_selector "service-time 0"$/m,
+ 'a value containing whitespace renders double-quoted, single words stay bare',
+);
+
+my $wwids = PVE::Multipath::Config::parse_wwids("# Multipath wwids\n/3600abc/\n/3600def/\n");
+is_deeply($wwids, ['3600abc', '3600def'], 'parse_wwids extracts the wwids');
+like(
+ PVE::Multipath::Config::format_wwids(['3600def', '3600abc']),
+ qr{/3600abc/\n/3600def/},
+ 'format_wwids sorts and slash-wraps',
+);
+
+# --- cluster config (pmxcfs source of truth): SectionConfig parse/write ---
+my $raw =
+ "defaults: defaults\n\tfind-multipaths strict\n\tno-path-retry queue\n\n"
+ . "wwid: 3600def\n\talias san-b-lun0\n\n"
+ . "wwid: 3600abc\n\talias san-a-lun0\n\tno-path-retry 18\n";
+my $cc = PVE::Multipath::Config->parse_config('multipath.cfg', $raw);
+is_deeply(
+ PVE::Multipath::Config::wwid_list($cc),
+ ['3600abc', '3600def'],
+ 'wwid sections become the allow-list (sorted)',
+);
+is_deeply(
+ PVE::Multipath::Config::aliases($cc),
+ { '3600abc' => 'san-a-lun0', '3600def' => 'san-b-lun0' },
+ 'aliases read from the wwid sections',
+);
+is(
+ PVE::Multipath::Config::effective_defaults($cc)->{'no-path-retry'},
+ 'queue',
+ 'defaults section knob is read',
+);
+is(
+ PVE::Multipath::Config::effective_defaults($cc)->{'user-friendly-names'},
+ 'no',
+ 'an unset defaults knob falls back to the managed default',
+);
+
+my $written = PVE::Multipath::Config->write_config('multipath.cfg', $cc);
+my $cc2 = PVE::Multipath::Config->parse_config('multipath.cfg', $written);
+is_deeply(
+ PVE::Multipath::Config::wwid_list($cc2),
+ ['3600abc', '3600def'],
+ 'wwids survive the SectionConfig round-trip',
+);
+is_deeply(
+ PVE::Multipath::Config::aliases($cc2),
+ PVE::Multipath::Config::aliases($cc),
+ 'aliases survive the round-trip',
+);
+is($cc2->{ids}->{'3600abc'}->{'no-path-retry'}, 18, 'a per-WWID knob survives the round-trip');
+
+is_deeply(
+ PVE::Multipath::Config::wwid_list(PVE::Multipath::Config->parse_config('multipath.cfg', '')),
+ [],
+ 'an empty cluster config has no WWIDs',
+);
+
+# --- multipaths{} block (alias plus per-WWID knobs) ---
+my $block = PVE::Multipath::Config::build_multipaths_block({
+ '3600def' => { alias => 'san-b-lun0' },
+ '3600abc' => { alias => 'san-a-lun0', 'no-path-retry' => 18 },
+ '3600nul' => {},
+});
+like($block, qr/^multipaths \{/m, 'block opens with multipaths {');
+is(
+ scalar(() = $block =~ /^\tmultipath \{/mg),
+ 2,
+ 'one multipath{} per WWID that has an alias or a knob (the empty WWID is skipped)',
+);
+like(
+ $block,
+ qr/wwid 3600abc.*?alias san-a-lun0.*?no_path_retry 18/s,
+ 'block carries the alias and the per-WWID knob',
+);
+my $abc_pos = index($block, 'wwid 3600abc');
+my $def_pos = index($block, 'wwid 3600def');
+ok($abc_pos < $def_pos, 'block emits entries in WWID-sorted order');
+is(PVE::Multipath::Config::build_multipaths_block({}), '', 'no WWIDs render to the empty string');
+like(
+ PVE::Multipath::Config::build_multipaths_block({
+ '3600abc' => { 'path-selector' => 'round-robin 0' },
+ }),
+ qr/^\t\tpath_selector "round-robin 0"$/m,
+ 'a per-WWID value containing whitespace renders double-quoted',
+);
+
+# --- WWID allow-list reconcile plan: only prune what PVE itself added ---
+# 'f' is a foreign, hand-added WWID present in the local allow-list but never managed by PVE.
+{
+ my ($add, $remove) = PVE::Multipath::plan_wwid_changes(
+ { a => 1, b => 1 }, # desired (cluster config)
+ { b => 1, f => 1 }, # current allow-list
+ { b => 1 }, # WWIDs PVE added before
+ );
+ is_deeply($add, ['a'], 'adds desired WWIDs that are not active yet');
+ is_deeply($remove, [], 'a foreign WWID PVE never added is left untouched');
+
+ ($add, $remove) = PVE::Multipath::plan_wwid_changes(
+ { a => 1 }, # desired
+ { a => 1, b => 1, f => 1 }, # current allow-list
+ { a => 1, b => 1 }, # PVE added a and b before
+ );
+ is_deeply($add, [], 'nothing to add when the desired WWID is already active');
+ is_deeply(
+ $remove,
+ ['b'],
+ 'prunes a PVE-managed WWID dropped from the config, but never the foreign one',
+ );
+}
+
+# --- ownership record: never adopt a pre-existing hand-made allow-list entry ---
+{
+ # 'h' was already in the allow-list when it appeared in the cluster config, so PVE never added
+ # it and it must stay foreign even while desired; dropping it from the config later must not
+ # prune the hand-made entry (think a boot-from-SAN LUN added just for health monitoring)
+ my $owned = PVE::Multipath::owned_wwid_record(
+ { a => 1, h => 1 }, # desired
+ { a => 1 }, # managed before
+ {}, # nothing added this pass (both were already in the allow-list)
+ {},
+ );
+ is_deeply($owned, ['a'], 'a desired but hand-added WWID is not adopted into the record');
+
+ $owned = PVE::Multipath::owned_wwid_record(
+ { a => 1, n => 1 }, # desired
+ { a => 1 }, # managed before
+ { n => 1 }, # newly added by this pass
+ {},
+ );
+ is_deeply($owned, ['a', 'n'], 'a WWID PVE just added becomes owned');
+
+ $owned = PVE::Multipath::owned_wwid_record(
+ { a => 1 }, # desired
+ { a => 1, gone => 1 }, # managed; 'gone' was dropped from the config
+ {},
+ { gone => 1 }, # but its prune failed
+ );
+ is_deeply($owned, ['a', 'gone'], 'a failed prune keeps ownership so it is retried');
+
+ $owned = PVE::Multipath::owned_wwid_record(
+ { a => 1, f => 1 }, # desired
+ { a => 1 }, # managed before
+ {}, # adding 'f' failed, so it never reached the allow-list
+ {},
+ );
+ is_deeply($owned, ['a'], 'a failed add does not claim ownership');
+
+ is_deeply(PVE::Multipath::owned_wwid_record({}, {}, {}, {}), [], 'empty in, empty out');
+}
+
+# --- storage-derived expectations (consumers and per-LUN expected node sets) ---
+{
+ my $storage_cfg = {
+ ids => {
+ mp => { type => 'multipath' },
+ mpb => { type => 'multipath', nodes => { n1 => 1, n2 => 1 } },
+ mpoff => { type => 'multipath', disable => 1, nodes => { n9 => 1 } },
+ lvm1 => { type => 'lvm', base => 'mp:3600abc', nodes => { n1 => 1 } },
+ lvm2 => { type => 'lvm', base => 'mpb:3600def' },
+ lvmplain => { type => 'lvm' },
+ other => { type => 'nfs' },
+ },
+ };
+ my $exp = PVE::Multipath::storage_expectations($storage_cfg, ['n1', 'n2', 'n3']);
+ is_deeply(
+ $exp->{consumers},
+ { '3600abc' => 'lvm1', '3600def' => 'lvm2' },
+ 'LVM storages over a multipath base are found as consumers',
+ );
+ is_deeply(
+ $exp->{nodes},
+ { n1 => 1, n2 => 1, n3 => 1 },
+ 'expected union covers all nodes for an unrestricted storage and skips disabled ones',
+ );
+ is_deeply(
+ $exp->{'wwid-nodes'}->{'3600abc'},
+ { n1 => 1 },
+ 'a consumed LUN is expected only on the restricted consumer nodes',
+ );
+ is_deeply(
+ $exp->{'wwid-nodes'}->{'3600def'},
+ { n1 => 1, n2 => 1 },
+ 'the base storage restriction caps an unrestricted consumer',
+ );
+}
+
+# --- override guard ---
+eval { PVE::Multipath::Config::check_overrides("devices {\n\tdevice {\n\t\tvendor X\n\t}\n}\n") };
+is($@, '', 'a well-formed devices{} block passes the guard');
+eval { PVE::Multipath::Config::check_overrides("multipaths {\n}\n") };
+like($@, qr/managed via aliases/, 'a multipaths{} block is rejected, it is generated');
+eval { PVE::Multipath::Config::check_overrides("devices {\n") };
+like($@, qr/unbalanced braces/, 'unbalanced braces are rejected');
+eval { PVE::Multipath::Config::check_overrides("frobnicate {\n}\n") };
+like($@, qr/unknown top-level section/, 'an unknown top-level section is rejected');
+eval { PVE::Multipath::Config::check_overrides("blacklist { } multipaths {\n}\n") };
+like($@, qr/managed via aliases/, 'a multipaths{} block cannot hide behind a same-line close');
+eval { PVE::Multipath::Config::check_overrides("blacklist { } frobnicate {\n}\n") };
+like($@, qr/unknown top-level section/, 'an unknown section cannot hide behind a same-line close');
+eval { PVE::Multipath::Config::check_overrides("}\ndevices {\n") };
+like($@, qr/unbalanced braces/, 'a closing brace before any open is rejected');
+is(
+ PVE::Multipath::Config::write_overrides('x', "text \n\n"),
+ "text\n",
+ 'the overrides writer trims trailing whitespace',
+);
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 storage 02/13] api: disks: add read-only multipath status endpoint
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 storage 01/13] multipath: add helper library and managed configuration Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 storage 03/13] api: multipath: add cluster-wide configuration endpoints Thomas Lamprecht
` (10 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Expose multipath map state under /nodes/{node}/disks/multipath so the
web UI and operators can see map health, per-path transport and which
LVM volume group consumes a map, rather than parsing 'multipath -ll' by
hand.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
No changes since v1.
src/PVE/API2/Disks.pm | 7 ++
src/PVE/API2/Disks/Makefile | 1 +
src/PVE/API2/Disks/Multipath.pm | 206 ++++++++++++++++++++++++++++++++
3 files changed, 214 insertions(+)
create mode 100644 src/PVE/API2/Disks/Multipath.pm
diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index e707a9e..984d890 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -14,6 +14,7 @@ use PVE::Tools qw(run_command);
use PVE::API2::Disks::Directory;
use PVE::API2::Disks::LVM;
use PVE::API2::Disks::LVMThin;
+use PVE::API2::Disks::Multipath;
use PVE::API2::Disks::ZFS;
use PVE::RESTHandler;
@@ -34,6 +35,11 @@ __PACKAGE__->register_method({
path => 'directory',
});
+__PACKAGE__->register_method({
+ subclass => "PVE::API2::Disks::Multipath",
+ path => 'multipath',
+});
+
__PACKAGE__->register_method({
subclass => "PVE::API2::Disks::ZFS",
path => 'zfs',
@@ -70,6 +76,7 @@ __PACKAGE__->register_method({
{ name => 'lvm' },
{ name => 'lvmthin' },
{ name => 'directory' },
+ { name => 'multipath' },
{ name => 'wipedisk' },
{ name => 'zfs' },
];
diff --git a/src/PVE/API2/Disks/Makefile b/src/PVE/API2/Disks/Makefile
index 9152aed..f6f8449 100644
--- a/src/PVE/API2/Disks/Makefile
+++ b/src/PVE/API2/Disks/Makefile
@@ -1,6 +1,7 @@
SOURCES= LVM.pm\
LVMThin.pm\
+ Multipath.pm\
ZFS.pm\
Directory.pm
diff --git a/src/PVE/API2/Disks/Multipath.pm b/src/PVE/API2/Disks/Multipath.pm
new file mode 100644
index 0000000..5cf2d17
--- /dev/null
+++ b/src/PVE/API2/Disks/Multipath.pm
@@ -0,0 +1,206 @@
+package PVE::API2::Disks::Multipath;
+
+use strict;
+use warnings;
+
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Multipath;
+use PVE::Storage::LVMPlugin;
+
+use PVE::RESTHandler;
+
+use base qw(PVE::RESTHandler);
+
+my $path_groups_schema = {
+ type => 'array',
+ description => 'The path groups of the map, in priority order.',
+ items => {
+ type => 'object',
+ additionalProperties => 1,
+ properties => {
+ group => { type => 'integer' },
+ 'dm-state' => { type => 'string', optional => 1 },
+ priority => { type => 'integer' },
+ paths => {
+ type => 'array',
+ items => {
+ type => 'object',
+ additionalProperties => 1,
+ properties => {
+ dev => {
+ type => 'string',
+ description => 'The underlying block device of the path.',
+ },
+ 'dm-state' => {
+ type => 'string',
+ description => "Path state as seen by device-mapper "
+ . "('active' or 'failed').",
+ optional => 1,
+ },
+ 'dev-state' => {
+ type => 'string',
+ description => "Path state as seen by the kernel block layer.",
+ optional => 1,
+ },
+ 'check-state' => {
+ type => 'string',
+ description => 'Result of the path checker.',
+ optional => 1,
+ },
+ priority => { type => 'integer', optional => 1 },
+ transport => {
+ type => 'string',
+ description => 'Transport of this path (iscsi, fc, sas).',
+ optional => 1,
+ },
+ },
+ },
+ },
+ },
+ },
+};
+
+# Annotates each map with the LVM volume group sitting on it, if any; the map's PV shows up under
+# its /dev/mapper/<name> path in the VG -> PV listing.
+my sub annotate_lvm_usage {
+ my ($maps) = @_;
+
+ return if !scalar(@$maps);
+
+ my $pv_to_vg = {};
+ eval {
+ my $vgs = PVE::Storage::LVMPlugin::lvm_vgs(1);
+ for my $vgname (keys %$vgs) {
+ for my $pv ($vgs->{$vgname}->{pvs}->@*) {
+ $pv_to_vg->{ $pv->{name} } = $vgname;
+ }
+ }
+ };
+ warn $@ if $@;
+
+ for my $map (@$maps) {
+ next if !defined($map->{name});
+ my $vg = $pv_to_vg->{"/dev/mapper/$map->{name}"};
+ $map->{'used-by'} = "LVM VG '$vg'" if defined($vg);
+ }
+}
+
+__PACKAGE__->register_method({
+ name => 'index',
+ path => '',
+ method => 'GET',
+ proxyto => 'node',
+ protected => 1,
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ description => "List and report the health of device-mapper multipath maps.",
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ },
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ supported => {
+ type => 'boolean',
+ description => "Whether multipath-tools is installed on the node.",
+ },
+ running => {
+ type => 'boolean',
+ description => "Whether the multipathd daemon is reachable.",
+ },
+ maps => {
+ type => 'array',
+ items => {
+ type => 'object',
+ additionalProperties => 1,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID, the stable identity of the LUN.',
+ optional => 1,
+ },
+ name => {
+ type => 'string',
+ description => 'The (node-local) multipath map name.',
+ optional => 1,
+ },
+ path => {
+ type => 'string',
+ description => 'Stable WWID-based device path of the map.',
+ optional => 1,
+ },
+ sysfs => {
+ type => 'string',
+ description => "The 'dm-N' kernel device name.",
+ optional => 1,
+ },
+ size => {
+ type => 'integer',
+ description => 'Size of the map in bytes.',
+ optional => 1,
+ },
+ health => {
+ type => 'string',
+ description => "Aggregated map health: 'optimal' (all paths "
+ . "active), 'degraded' (some paths failed) or 'failed' "
+ . "(no active path).",
+ enum => ['optimal', 'degraded', 'failed'],
+ },
+ 'dm-state' => { type => 'string', optional => 1 },
+ 'paths-total' => {
+ type => 'integer',
+ description => 'Total number of paths.',
+ },
+ 'paths-active' => {
+ type => 'integer',
+ description => 'Number of currently active paths.',
+ },
+ transport => {
+ type => 'string',
+ description =>
+ "Transport shared by all of the map's paths, if uniform.",
+ optional => 1,
+ },
+ used => {
+ type => 'boolean',
+ description => 'Whether something sits on the map, such as an LVM'
+ . ' physical volume.',
+ optional => 1,
+ },
+ 'used-by' => {
+ type => 'string',
+ description => 'What consumes the map, if known, such as an LVM'
+ . ' volume group.',
+ optional => 1,
+ },
+ 'path-groups' => $path_groups_schema,
+ },
+ },
+ },
+ },
+ },
+ code => sub {
+ my ($param) = @_;
+
+ my $supported = PVE::Multipath::is_supported();
+ my $running = $supported ? PVE::Multipath::is_running() : 0;
+
+ my $maps = [];
+ if ($running) {
+ $maps = PVE::Multipath::get_maps();
+ annotate_lvm_usage($maps);
+ }
+
+ return {
+ supported => $supported ? 1 : 0,
+ running => $running ? 1 : 0,
+ maps => $maps,
+ };
+ },
+});
+
+1;
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 storage 03/13] api: multipath: add cluster-wide configuration endpoints
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 storage 01/13] multipath: add helper library and managed configuration Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 storage 02/13] api: disks: add read-only multipath status endpoint Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 storage 04/13] multipath: add storage plugin for multipath LUNs Thomas Lamprecht
` (9 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Add a CRUD API over the cluster-wide multipath.cfg, mounted by
pve-manager at /cluster/multipath: read the effective configuration,
manage the WWID allow-list, set the global defaults and the per-LUN
knobs and aliases, and replace the verbatim override sections.
Updates take the settable knobs straight from the section schema, with
the schema 'default' stripped so omitting a knob leaves it unchanged
rather than resetting it; 'delete' is guarded to the settable
properties so it cannot strip a section's type and corrupt the file.
Applying the result on each node is handled separately by pvestatd via
the generator.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Changes in v2:
- guard the config writes with a digest against concurrent
modifications
- move the free-form overrides to their own GET/PUT endpoint with a
digest and cfs lock of their own (the edit window could never save)
- require the alias parameter on the alias POST instead of clearing
the alias when it is omitted
- add a wwid index GET, the per-LUN options were write-only
- drop the defaults section again when its last knob is deleted, an
empty leftover would keep the generator engaged forever
- reserve the 'defaults' id; de-duplicate the write paths
src/PVE/API2/Makefile | 1 +
src/PVE/API2/Multipath.pm | 516 ++++++++++++++++++++++++++++++++++++++
2 files changed, 517 insertions(+)
create mode 100644 src/PVE/API2/Multipath.pm
diff --git a/src/PVE/API2/Makefile b/src/PVE/API2/Makefile
index fe316c5..002a3bd 100644
--- a/src/PVE/API2/Makefile
+++ b/src/PVE/API2/Makefile
@@ -3,5 +3,6 @@
.PHONY: install
install:
install -D -m 0644 Disks.pm ${DESTDIR}${PERLDIR}/PVE/API2/Disks.pm
+ install -D -m 0644 Multipath.pm ${DESTDIR}${PERLDIR}/PVE/API2/Multipath.pm
make -C Storage install
make -C Disks install
diff --git a/src/PVE/API2/Multipath.pm b/src/PVE/API2/Multipath.pm
new file mode 100644
index 0000000..cb138f6
--- /dev/null
+++ b/src/PVE/API2/Multipath.pm
@@ -0,0 +1,516 @@
+package PVE::API2::Multipath;
+
+use strict;
+use warnings;
+
+use PVE::Exception qw(raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Tools qw(extract_param);
+
+use PVE::Multipath;
+use PVE::Multipath::Config;
+use PVE::Multipath::ClusterConfig;
+
+use PVE::RESTHandler;
+
+use base qw(PVE::RESTHandler);
+
+# WWID as exported by /dev/disk/by-id/dm-uuid-mpath-<wwid>; keep validation lenient, multipathd is
+# the authority on what is valid.
+my $WWID_RE = qr/^[a-zA-Z0-9._:-]+\z/;
+
+# multipathd resolves an alias to a map name, so two WWIDs sharing one alias makes it drop a map
+# (the loser is order-dependent and only logged at level 1). Reject a collision up front.
+my sub assert_alias_free {
+ my ($cfg, $wwid, $alias) = @_;
+
+ for my $other (keys $cfg->{ids}->%*) {
+ next if $other eq $wwid;
+ my $section = $cfg->{ids}->{$other};
+ next if ($section->{type} // '') ne 'wwid';
+ die "alias '$alias' is already assigned to WWID '$other'\n"
+ if defined($section->{alias}) && $section->{alias} eq $alias;
+ }
+}
+
+# Apply a section update: set the given properties, then unset those named in $delete. Guard $delete
+# to the settable options and never set and delete the same key, so a stray 'delete=type' cannot
+# strip the section type and corrupt the file (as PVE::API2::Storage::Config guards it too).
+my sub apply_section_update {
+ my ($section, $param, $delete, $settable) = @_;
+
+ my @delete = split(/,/, $delete // '');
+ for my $key (@delete) {
+ raise_param_exc({ delete => "'$key' is not a settable property" }) if !$settable->{$key};
+ raise_param_exc({ $key => "cannot set and delete a property at the same time" })
+ if defined($param->{$key});
+ }
+ $section->{$_} = $param->{$_} for keys %$param;
+ delete $section->{$_} for @delete;
+}
+
+# Read the cluster config inside the lock and reject the write if it changed since the caller read
+# the digest, so a concurrent edit is not silently overwritten.
+my sub read_locked_config {
+ my ($digest) = @_;
+
+ my $cfg = PVE::Multipath::ClusterConfig::read_config();
+ PVE::Tools::assert_if_modified($cfg->{digest}, $digest);
+ return $cfg;
+}
+
+# Read the locked config (asserting the digest), run $code to mutate it, then write it back; wraps
+# the read-modify-write envelope so each endpoint carries only its own mutation.
+my sub edit_locked_config {
+ my ($digest, $errmsg, $code) = @_;
+
+ PVE::Multipath::ClusterConfig::lock_config(
+ sub {
+ my $cfg = read_locked_config($digest);
+ $code->($cfg);
+ PVE::Multipath::ClusterConfig::write_config($cfg);
+ },
+ $errmsg,
+ );
+}
+
+# Validate a WWID from the API and reject the reserved 'defaults' id (the global section), so a WWID
+# endpoint can never read, overwrite, or delete it.
+my sub assert_valid_wwid {
+ my ($wwid) = @_;
+
+ raise_param_exc({ wwid => "does not look like a valid WWID" })
+ if !defined($wwid) || $wwid !~ $WWID_RE || $wwid eq 'defaults';
+}
+
+__PACKAGE__->register_method({
+ name => 'read',
+ path => '',
+ method => 'GET',
+ description => "Read the cluster-wide multipath configuration.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ defaults => {
+ type => 'object',
+ description => 'Effective global multipathd knobs.',
+ additionalProperties => 1,
+ },
+ wwids => {
+ type => 'array',
+ description => 'The multipath WWID allow-list.',
+ items => { type => 'string' },
+ },
+ aliases => {
+ type => 'object',
+ description => 'Per-WWID human-readable aliases.',
+ additionalProperties => { type => 'string' },
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ code => sub {
+ my $cfg = PVE::Multipath::ClusterConfig::read_config();
+ return {
+ defaults => PVE::Multipath::Config::effective_defaults($cfg),
+ wwids => PVE::Multipath::Config::wwid_list($cfg),
+ aliases => PVE::Multipath::Config::aliases($cfg),
+ digest => $cfg->{digest},
+ };
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'read_overrides',
+ path => 'overrides',
+ method => 'GET',
+ description => "Read the verbatim multipath.conf override sections.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ overrides => {
+ type => 'string',
+ description => 'Verbatim override sections.',
+ optional => 1,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ code => sub {
+ my $overrides = PVE::Multipath::ClusterConfig::read_overrides();
+ return {
+ digest => PVE::Multipath::ClusterConfig::overrides_digest($overrides),
+ defined($overrides) && length($overrides) ? (overrides => $overrides) : (),
+ };
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'set_overrides',
+ path => 'overrides',
+ method => 'PUT',
+ protected => 1,
+ description => "Set the verbatim multipath.conf override sections, such as "
+ . "hardware-specific 'device {}' or 'overrides {}' blocks.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ overrides => {
+ type => 'string',
+ description => 'Verbatim override sections. Omit to clear them.',
+ maxLength => 128 * 1024,
+ optional => 1,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $overrides = $param->{overrides};
+ my $digest = extract_param($param, 'digest');
+ eval { PVE::Multipath::Config::check_overrides($overrides) };
+ raise_param_exc({ overrides => $@ }) if $@;
+
+ PVE::Multipath::ClusterConfig::lock_overrides(
+ sub {
+ my $current = PVE::Multipath::ClusterConfig::read_overrides();
+ PVE::Tools::assert_if_modified(
+ PVE::Multipath::ClusterConfig::overrides_digest($current), $digest,
+ );
+ PVE::Multipath::ClusterConfig::write_overrides($overrides // '');
+ },
+ "updating multipath overrides failed",
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'set_defaults',
+ path => 'defaults',
+ method => 'PUT',
+ protected => 1,
+ description => "Set the global multipathd defaults applied on every node.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ PVE::Multipath::Config::defaults_api_schema()->%*,
+ delete => {
+ type => 'string',
+ description => 'A list of settings to reset to their default, comma-separated.',
+ optional => 1,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $delete = extract_param($param, 'delete');
+ my $digest = extract_param($param, 'digest');
+ my $settable = PVE::Multipath::Config::defaults_api_schema();
+
+ edit_locked_config(
+ $digest,
+ "updating multipath defaults failed",
+ sub {
+ my ($cfg) = @_;
+ my $section = $cfg->{ids}->{defaults} //= { type => 'defaults' };
+ apply_section_update($section, $param, $delete, $settable);
+ # drop the section when its last knob was deleted: the reserved id has no remove
+ # endpoint, and an empty section would keep the generator engaged forever
+ delete $cfg->{ids}->{defaults} if !grep { $_ ne 'type' } keys %$section;
+ },
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'wwid_index',
+ path => 'wwid',
+ method => 'GET',
+ description => "List the allow-listed WWIDs with their alias and per-LUN multipathd knobs.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => {
+ wwid => { type => 'string', description => 'The LUN WWID.' },
+ PVE::Multipath::Config::wwid_api_schema()->%*,
+ },
+ },
+ links => [{ rel => 'child', href => '{wwid}' }],
+ },
+ code => sub {
+ my $cfg = PVE::Multipath::ClusterConfig::read_config();
+ my $opts = PVE::Multipath::Config::wwid_opts($cfg);
+ return [map { { wwid => $_, $opts->{$_}->%* } } sort keys %$opts];
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'add_wwid',
+ path => 'wwid',
+ method => 'POST',
+ protected => 1,
+ description => "Add a WWID to the multipath allow-list.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID of the LUN to manage via multipath.',
+ maxLength => 128,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $wwid = $param->{wwid};
+ my $digest = extract_param($param, 'digest');
+ assert_valid_wwid($wwid);
+
+ edit_locked_config(
+ $digest,
+ "adding WWID '$wwid' failed",
+ sub {
+ my ($cfg) = @_;
+ $cfg->{ids}->{$wwid} //= { type => 'wwid' };
+ },
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'update_wwid',
+ path => 'wwid/{wwid}',
+ method => 'PUT',
+ protected => 1,
+ description => "Set the alias and per-LUN multipathd knobs for an allow-listed WWID.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID to update.',
+ maxLength => 128,
+ },
+ PVE::Multipath::Config::wwid_api_schema()->%*,
+ delete => {
+ type => 'string',
+ description => 'A list of settings to unset, comma-separated.',
+ optional => 1,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $wwid = extract_param($param, 'wwid');
+ my $delete = extract_param($param, 'delete');
+ my $digest = extract_param($param, 'digest');
+ my $settable = PVE::Multipath::Config::wwid_api_schema();
+ assert_valid_wwid($wwid);
+
+ edit_locked_config(
+ $digest,
+ "updating WWID '$wwid' failed",
+ sub {
+ my ($cfg) = @_;
+ my $section = $cfg->{ids}->{$wwid};
+ die "WWID '$wwid' is not on the allow-list, add it first\n" if !$section;
+
+ assert_alias_free($cfg, $wwid, $param->{alias}) if defined($param->{alias});
+ apply_section_update($section, $param, $delete, $settable);
+ },
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'remove_wwid',
+ path => 'wwid/{wwid}',
+ method => 'DELETE',
+ protected => 1,
+ description => "Remove a WWID from the multipath allow-list.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID to remove.',
+ maxLength => 128,
+ },
+ force => {
+ type => 'boolean',
+ description => 'Remove even while a storage still consumes the LUN.',
+ optional => 1,
+ default => 0,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $wwid = $param->{wwid};
+ my $digest = extract_param($param, 'digest');
+ assert_valid_wwid($wwid);
+
+ # dropping a consumed WWID flushes its map on the next sync, so the backing storage loses
+ # the device; refuse unless explicitly forced
+ if (!$param->{force}) {
+ my $expectations = PVE::Multipath::cluster_storage_expectations();
+ my $storeid = $expectations->{consumers}->{$wwid};
+ die "WWID '$wwid' is still used by storage '$storeid', remove that"
+ . " storage first or pass 'force' to override\n"
+ if defined($storeid);
+ }
+
+ edit_locked_config(
+ $digest,
+ "removing WWID '$wwid' failed",
+ sub {
+ my ($cfg) = @_;
+ delete $cfg->{ids}->{$wwid};
+ },
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'set_alias',
+ path => 'alias',
+ method => 'POST',
+ protected => 1,
+ description => "Set or replace the human-readable alias for an allow-listed WWID."
+ . " The alias becomes the dm-multipath map name on every node.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID to label.',
+ maxLength => 128,
+ },
+ alias => { PVE::Multipath::Config::wwid_api_schema()->{alias}->%*, optional => 0 },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $wwid = $param->{wwid};
+ my $alias = $param->{alias};
+ my $digest = extract_param($param, 'digest');
+ assert_valid_wwid($wwid);
+
+ edit_locked_config(
+ $digest,
+ "setting alias for WWID '$wwid' failed",
+ sub {
+ my ($cfg) = @_;
+ die "WWID '$wwid' is not on the allow-list, add it first\n"
+ if !$cfg->{ids}->{$wwid};
+ assert_alias_free($cfg, $wwid, $alias);
+ $cfg->{ids}->{$wwid}->{alias} = $alias;
+ },
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'remove_alias',
+ path => 'alias/{wwid}',
+ method => 'DELETE',
+ protected => 1,
+ description => "Remove the alias for a multipath WWID.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID whose alias to remove.',
+ maxLength => 128,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $wwid = $param->{wwid};
+ my $digest = extract_param($param, 'digest');
+ assert_valid_wwid($wwid);
+
+ edit_locked_config(
+ $digest,
+ "removing alias for WWID '$wwid' failed",
+ sub {
+ my ($cfg) = @_;
+ delete $cfg->{ids}->{$wwid}->{alias} if $cfg->{ids}->{$wwid};
+ },
+ );
+ return undef;
+ },
+});
+
+1;
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 storage 04/13] multipath: add storage plugin for multipath LUNs
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (2 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 storage 03/13] api: multipath: add cluster-wide configuration endpoints Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 storage 05/13] lvm: allow a multipath storage as the base device Thomas Lamprecht
` (8 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Expose the dm-multipath maps present on a node as a first-class storage
of raw volumes, each keyed by its WWID and reachable at the WWID-stable
path /dev/disk/by-id/dm-uuid-mpath-<wwid>. The maps are assembled by
multipathd from the cluster-wide allow-list, so the storage carries no
configuration of its own and only reflects what is already there.
This is meant primarily as the base of a shared LVM storage, giving a
volume group on a multipath LUN full path redundancy without any manual
device wiring.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Changes in v2:
- honor the requested content types in list_volumes
- the type is inherently shared, drop the user-settable option
src/PVE/Storage.pm | 2 +
src/PVE/Storage/Makefile | 3 +-
src/PVE/Storage/MultipathPlugin.pm | 187 +++++++++++++++++++++++++++++
src/PVE/Storage/Plugin.pm | 2 +-
4 files changed, 192 insertions(+), 2 deletions(-)
create mode 100644 src/PVE/Storage/MultipathPlugin.pm
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 64ea9da..4daeecf 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -39,6 +39,7 @@ use PVE::Storage::ZFSPlugin;
use PVE::Storage::PBSPlugin;
use PVE::Storage::BTRFSPlugin;
use PVE::Storage::ESXiPlugin;
+use PVE::Storage::MultipathPlugin;
# Storage API version. Increment it on changes in storage API interface.
use constant APIVER => 15;
@@ -64,6 +65,7 @@ PVE::Storage::ZFSPlugin->register();
PVE::Storage::PBSPlugin->register();
PVE::Storage::BTRFSPlugin->register();
PVE::Storage::ESXiPlugin->register();
+PVE::Storage::MultipathPlugin->register();
# load third-party plugins
if (-d '/usr/share/perl5/PVE/Storage/Custom') {
diff --git a/src/PVE/Storage/Makefile b/src/PVE/Storage/Makefile
index a67dc25..d76ec25 100644
--- a/src/PVE/Storage/Makefile
+++ b/src/PVE/Storage/Makefile
@@ -14,7 +14,8 @@ SOURCES= \
PBSPlugin.pm \
BTRFSPlugin.pm \
LvmThinPlugin.pm \
- ESXiPlugin.pm
+ ESXiPlugin.pm \
+ MultipathPlugin.pm
.PHONY: install
install:
diff --git a/src/PVE/Storage/MultipathPlugin.pm b/src/PVE/Storage/MultipathPlugin.pm
new file mode 100644
index 0000000..950e5e8
--- /dev/null
+++ b/src/PVE/Storage/MultipathPlugin.pm
@@ -0,0 +1,187 @@
+package PVE::Storage::MultipathPlugin;
+
+use strict;
+use warnings;
+
+use PVE::Multipath;
+use PVE::Storage::Plugin;
+
+use base qw(PVE::Storage::Plugin);
+
+# A block-LUN provider over device-mapper multipath: it exposes the maps present on the node as raw
+# volumes keyed by their WWID, reachable at the WWID-stable /dev/disk/by-id/dm-uuid-mpath-<wwid>
+# path. The maps are assembled by multipathd from the cluster-wide allow-list, so this storage has
+# no configuration of its own and only reflects what is there. Its main use is as the 'base' of a
+# shared LVM storage.
+
+sub type {
+ return 'multipath';
+}
+
+sub plugindata {
+ return {
+ content => [{ images => 1, none => 1 }, { images => 1 }],
+ format => [{ raw => 1 }, 'raw'],
+ select_existing => 1,
+ 'sensitive-properties' => {},
+ };
+}
+
+sub properties {
+ return {};
+}
+
+sub options {
+ return {
+ nodes => { optional => 1 },
+ disable => { optional => 1 },
+ content => { optional => 1 },
+ bwlimit => { optional => 1 },
+ };
+}
+
+# the volume name is the LUN's WWID
+sub parse_volname {
+ my ($class, $volname) = @_;
+
+ if ($volname =~ m!^([a-zA-Z0-9._:-]+)$!) {
+ return ('images', $1, undef, undef, undef, undef, 'raw');
+ }
+
+ die "unable to parse multipath volume name '$volname'\n";
+}
+
+sub filesystem_path {
+ my ($class, $scfg, $volname, $snapname) = @_;
+
+ die "snapshot is not possible on multipath storage\n" if defined($snapname);
+
+ my ($vtype, $wwid, $vmid) = $class->parse_volname($volname);
+
+ my $path = "/dev/disk/by-id/dm-uuid-mpath-$wwid";
+
+ return wantarray ? ($path, $vmid, $vtype) : $path;
+}
+
+sub create_base {
+ die "can't create base images in multipath storage\n";
+}
+
+sub clone_image {
+ die "can't clone images in multipath storage\n";
+}
+
+sub alloc_image {
+ die "can't allocate space in multipath storage\n";
+}
+
+sub free_image {
+ die "can't free space in multipath storage\n";
+}
+
+sub list_images {
+ my ($class, $storeid, $scfg, $vmid, $vollist, $cache) = @_;
+
+ my $res = [];
+
+ # multipath LUNs have no owning guest
+ return $res if defined($vmid) && !$vollist;
+
+ $cache->{multipath_maps} //= eval { PVE::Multipath::get_maps() } // [];
+
+ for my $map ($cache->{multipath_maps}->@*) {
+ next if !defined($map->{wwid});
+
+ my $volid = "$storeid:$map->{wwid}";
+ if ($vollist) {
+ next if !grep { $_ eq $volid } @$vollist;
+ }
+
+ push @$res,
+ {
+ volid => $volid,
+ format => 'raw',
+ size => $map->{size} // 0,
+ vmid => 0,
+ };
+ }
+
+ return $res;
+}
+
+sub list_volumes {
+ my ($class, $storeid, $scfg, $vmid, $content_types) = @_;
+
+ return [] if !grep { $_ eq 'images' } $content_types->@*;
+
+ my $res = $class->list_images($storeid, $scfg, $vmid);
+ $_->{content} = 'images' for $res->@*;
+
+ return $res;
+}
+
+sub status {
+ my ($class, $storeid, $scfg, $cache) = @_;
+
+ my $active = PVE::Multipath::is_running() ? 1 : 0;
+
+ return (0, 0, 0, $active);
+}
+
+sub activate_storage {
+ my ($class, $storeid, $scfg, $cache) = @_;
+
+ PVE::Multipath::assert_supported();
+ # nothing to log into: multipathd assembles the maps from the allow-list
+}
+
+sub deactivate_storage {
+ my ($class, $storeid, $scfg, $cache) = @_;
+
+ # the maps are shared infrastructure, do not tear them down
+}
+
+sub check_connection {
+ my ($class, $storeid, $scfg) = @_;
+
+ return PVE::Multipath::is_running() ? 1 : 0;
+}
+
+sub activate_volume {
+ my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+
+ my ($vtype, $wwid) = $class->parse_volname($volname);
+
+ my $map = PVE::Multipath::wait_for_map($wwid);
+ die "multipath map for WWID '$wwid' is not present - is it in the multipath"
+ . " allow-list and are its paths up?\n"
+ if !$map;
+}
+
+sub deactivate_volume {
+ my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+
+ # do not drop the map, it is shared infrastructure
+}
+
+sub volume_resize {
+ my ($class, $scfg, $storeid, $volname, $size, $running) = @_;
+
+ die "volume resize is not possible on multipath storage\n";
+}
+
+sub volume_has_feature {
+ my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running) = @_;
+
+ return { copy => { current => 1 } }->{$feature}->{current};
+}
+
+sub volume_export_formats {
+ return ();
+}
+
+sub volume_import_formats {
+ return ();
+}
+
+1;
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 4f69f9b..423b445 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -35,7 +35,7 @@ our @COMMON_TAR_FLAGS = qw(
);
our @SHARED_STORAGE = (
- 'iscsi', 'nfs', 'cifs', 'rbd', 'cephfs', 'iscsidirect', 'zfs', 'drbd', 'pbs',
+ 'iscsi', 'nfs', 'cifs', 'rbd', 'cephfs', 'iscsidirect', 'zfs', 'drbd', 'pbs', 'multipath',
);
our $QCOW2_PREALLOCATION = {
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 storage 05/13] lvm: allow a multipath storage as the base device
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (3 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 storage 04/13] multipath: add storage plugin for multipath LUNs Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 storage 06/13] multipath: broadcast per-node map health to the cluster KV store Thomas Lamprecht
` (7 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
on_add_hook only accepted an iscsi base when creating the volume group.
Accept the new multipath type as well, so a shared VG can sit on a
multipath LUN and inherit its path redundancy.
The map device is assembled on demand by multipathd, so activate the
base volume to ensure the map is present before pvcreate runs on it.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
No changes since v1.
src/PVE/Storage/LVMPlugin.pm | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index a313ecc..433c942 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -485,13 +485,14 @@ sub on_add_hook {
my $basecfg = PVE::Storage::storage_config($cfg, $baseid, 1);
die "base storage ID '$baseid' does not exist\n" if !$basecfg;
- # we only support iscsi for now
- die "unsupported base type '$basecfg->{type}'"
- if $basecfg->{type} ne 'iscsi';
+ die "unsupported base type '$basecfg->{type}'\n"
+ if $basecfg->{type} ne 'iscsi' && $basecfg->{type} ne 'multipath';
my $path = PVE::Storage::path($cfg, $base);
PVE::Storage::activate_storage($cfg, $baseid);
+ # the multipath map must be assembled before we can pvcreate on it
+ PVE::Storage::activate_volumes($cfg, [$base]) if $basecfg->{type} eq 'multipath';
lvm_create_volume_group($path, $scfg->{vgname}, $scfg->{shared});
}
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 storage 06/13] multipath: broadcast per-node map health to the cluster KV store
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (4 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 storage 05/13] lvm: allow a multipath storage as the base device Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 storage 07/13] api: multipath: add cluster-wide health status endpoint Thomas Lamprecht
` (6 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Map health is inherently per-node: each node has its own paths to the
same LUN, so whether a LUN has full path redundancy can only be told
per node. To make a cluster-wide view possible, reduce the local maps
to a small per-WWID summary and push it under the cluster KV key
'multipath' via pmxcfs.
A present value also signals that the node is actively multipathing:
clear the key when no maps are assembled, so the status aggregation can
combine just the active nodes without extra bookkeeping. The summary
stays well under the 32 KiB KV limit for typical setups (about 130
bytes per map); the full per-path detail stays behind the per-node
disks/multipath API.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Changes in v2:
- timestamp the broadcast payload and skip rebroadcasting unchanged
state, refreshing every 60s instead of every status cycle
src/PVE/Multipath.pm | 86 +++++++++++++++++++++++++++++++++
src/test/run_multipath_tests.pl | 50 +++++++++++++++++++
2 files changed, 136 insertions(+)
diff --git a/src/PVE/Multipath.pm b/src/PVE/Multipath.pm
index 64118b3..ec87b08 100644
--- a/src/PVE/Multipath.pm
+++ b/src/PVE/Multipath.pm
@@ -392,4 +392,90 @@ sub reconfigure {
run_command([$MULTIPATHD, 'reconfigure']);
}
+# Pure: reduce the rich get_maps() output to the compact per-WWID dict broadcast under the cluster
+# KV key 'multipath'. Holds only what the cluster-wide health matrix needs; the full per-path detail
+# stays available behind the per-node disks/multipath API.
+sub summarize_maps_for_broadcast {
+ my ($maps) = @_;
+
+ my $out = {};
+ for my $map ($maps->@*) {
+ next if !defined($map->{wwid});
+ $out->{ $map->{wwid} } = {
+ state => $map->{health},
+ 'paths-active' => $map->{'paths-active'} // 0,
+ 'paths-total' => $map->{'paths-total'} // 0,
+ defined($map->{transport}) ? (transport => $map->{transport}) : (),
+ defined($map->{size}) ? (size => $map->{size}) : (),
+ };
+ }
+ return $out;
+}
+
+# Rebroadcast protocol for the node KV values: an unchanged value is refreshed every
+# $KV_REFRESH_SECONDS instead of on every status cycle (each KV write is a cluster-wide corosync
+# message), and consumers treat a payload timestamp older than $KV_STALE_SECONDS as coming from a
+# reporter that stopped updating, so a stalled status daemon's last snapshot does not read as
+# current health. Relies on the cluster-wide clock sync corosync needs anyway.
+our $KV_REFRESH_SECONDS = 60;
+our $KV_STALE_SECONDS = 3 * $KV_REFRESH_SECONDS;
+
+# Stamp $data (a hashref, or undef to clear) with the broadcast time and push it into the cluster
+# KV store under $key. Skips the write while the content is unchanged and fresh, or when the key is
+# already cleared; a write that dies (an oversized value, for example) is not recorded and is
+# retried on the next call. Transport hiccups get absorbed by pve-cluster itself, a lost refresh
+# repeats within $KV_REFRESH_SECONDS and consumers demote older values either way.
+my $last_kv = {};
+
+my sub update_node_kv {
+ my ($key, $data) = @_;
+
+ require PVE::Cluster;
+
+ my $canonical = JSON->new->canonical;
+ my $content = defined($data) ? $canonical->encode($data) : undef;
+ my $now = time();
+ if (my $last = $last_kv->{$key}) {
+ if (defined($content) && defined($last->{content})) {
+ return if $content eq $last->{content} && $now - $last->{time} < $KV_REFRESH_SECONDS;
+ } elsif (!defined($content) && !defined($last->{content})) {
+ return;
+ }
+ }
+
+ my $value = defined($content) ? $canonical->encode({ $data->%*, time => $now }) : undef;
+ eval { PVE::Cluster::broadcast_node_kv($key, $value) };
+ if (my $err = $@) {
+ warn "multipath: broadcasting '$key' failed - $err";
+ return;
+ }
+ $last_kv->{$key} = { content => $content, time => $now };
+}
+
+# Push a compact per-WWID health snapshot into the cluster KV store under the key 'multipath'. A
+# present value also means "this node is actively multipathing", so clear the key when no maps are
+# assembled and the status aggregation then only combines the active nodes. Never throws, so it is
+# safe to call from a status loop where multipath is not the primary concern.
+sub broadcast_health {
+ if (!is_running()) {
+ update_node_kv('multipath', undef);
+ return;
+ }
+
+ my $maps = eval { get_maps() };
+ if (my $err = $@) {
+ # keep the last value: it ages past $KV_STALE_SECONDS and consumers demote it, which beats
+ # clearing (that would read as "no maps assembled" and flag the LUNs as missing)
+ warn "multipath: collecting maps for broadcast failed - $err";
+ return;
+ }
+
+ my $summary = summarize_maps_for_broadcast($maps);
+ if (!%$summary) {
+ update_node_kv('multipath', undef);
+ return;
+ }
+ update_node_kv('multipath', { maps => $summary });
+}
+
1;
diff --git a/src/test/run_multipath_tests.pl b/src/test/run_multipath_tests.pl
index 09b6061..a4ad57d 100755
--- a/src/test/run_multipath_tests.pl
+++ b/src/test/run_multipath_tests.pl
@@ -357,4 +357,54 @@ is(
'the overrides writer trims trailing whitespace',
);
+# --- broadcast summary (per-WWID condensation of get_maps for the cluster KV) ---
+my $summary = PVE::Multipath::summarize_maps_for_broadcast($maps);
+is_deeply(
+ [sort keys %$summary],
+ [sort map { $_->{wwid} } $maps->@*],
+ 'every map with a WWID appears in the summary',
+);
+is($summary->{ $a->{wwid} }->{state}, 'optimal', 'optimal map summarized as optimal');
+is($summary->{ $a->{wwid} }->{'paths-active'}, 2, 'optimal map active path count carried');
+is($summary->{ $a->{wwid} }->{'paths-total'}, 2, 'optimal map total path count carried');
+is($summary->{ $b->{wwid} }->{state}, 'degraded', 'degraded map summarized as degraded');
+is($summary->{ $c->{wwid} }->{state}, 'failed', 'failed map summarized as failed');
+ok(
+ !exists $summary->{ $a->{wwid} }->{transport},
+ 'transport omitted when not derived (get_maps fills it live)',
+);
+
+is_deeply(
+ PVE::Multipath::summarize_maps_for_broadcast([]),
+ {},
+ 'empty maps list summarizes to empty hash (caller clears the KV)',
+);
+
+# transport/size propagate when the caller (get_maps) has set them
+my $enriched = [{
+ wwid => '3600x',
+ health => 'optimal',
+ 'paths-active' => 2,
+ 'paths-total' => 2,
+ transport => 'iscsi',
+ size => 34359738368,
+}];
+my $enr = PVE::Multipath::summarize_maps_for_broadcast($enriched);
+is($enr->{'3600x'}->{transport}, 'iscsi', 'transport carried into the summary');
+is($enr->{'3600x'}->{size}, 34359738368, 'size carried into the summary');
+
+# size budget: well under the 32 KiB pmxcfs KV limit even for many maps
+my $many = [
+ map { {
+ wwid => sprintf('3600140500000000000000000000%04x', $_),
+ health => 'optimal',
+ 'paths-active' => 4,
+ 'paths-total' => 4,
+ transport => 'iscsi',
+ size => 1099511627776,
+ } } 0 .. 99
+];
+my $big = JSON::encode_json(PVE::Multipath::summarize_maps_for_broadcast($many));
+ok(length($big) < 32 * 1024, "100-map summary (" . length($big) . " B) fits the KV size limit");
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 storage 07/13] api: multipath: add cluster-wide health status endpoint
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (5 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 storage 06/13] multipath: broadcast per-node map health to the cluster KV store Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 cluster 08/13] pmxcfs: track cluster-wide multipath configuration Thomas Lamprecht
` (5 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
A per-node view cannot tell whether a LUN is healthy across the whole
cluster. Add an endpoint that collects the per-node broadcasts and
combines them into a per-WWID by per-node matrix, rolled up to one
cluster-state per LUN.
The broadcasts are cross-checked against live membership, so a stale
value from an offline node reads as 'unknown' rather than as healthy.
The roll-up is taken over the nodes that are actively multipathing, so
a LUN that is optimal on three nodes but degraded on a fourth shows up
as degraded instead of hiding behind the healthy majority. A node where
a multipath storage is enabled but that broadcasts nothing is surfaced
as missing rather than vanishing from the matrix. Consuming storages
are labeled from the cluster storage config.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Changes in v2:
- status returns { luns, nodes } with per-node config-apply errors
- broadcast and surface a node's apply failure (liveness-checked)
- mark a LUN 'missing' only on nodes expected to carry it, derived
per LUN from the consuming storage chain's node restrictions
- demote broadcasts older than three refresh intervals to 'unknown'
- tolerate malformed values from a peer's broadcast instead of dying
- clamp unrecognized peer states; drop the unneeded protected flag
src/PVE/API2/Multipath.pm | 135 ++++++++++++++++++++++++
src/PVE/Multipath.pm | 132 ++++++++++++++++++++++++
src/test/run_multipath_tests.pl | 176 ++++++++++++++++++++++++++++++++
3 files changed, 443 insertions(+)
diff --git a/src/PVE/API2/Multipath.pm b/src/PVE/API2/Multipath.pm
index cb138f6..d43217a 100644
--- a/src/PVE/API2/Multipath.pm
+++ b/src/PVE/API2/Multipath.pm
@@ -3,6 +3,9 @@ package PVE::API2::Multipath;
use strict;
use warnings;
+use JSON qw(decode_json);
+
+use PVE::Cluster;
use PVE::Exception qw(raise_param_exc);
use PVE::JSONSchema qw(get_standard_option);
use PVE::Tools qw(extract_param);
@@ -127,6 +130,138 @@ __PACKAGE__->register_method({
},
});
+__PACKAGE__->register_method({
+ name => 'status',
+ path => 'status',
+ method => 'GET',
+ description => "Cluster-wide multipath health: a per-WWID by per-node matrix"
+ . " rolled up over the nodes that are actively multipathing.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ luns => {
+ type => 'array',
+ description => "Per-WWID health: a per-node map-state matrix rolled up to one"
+ . " cluster-state per LUN.",
+ items => {
+ type => 'object',
+ additionalProperties => 1,
+ properties => {
+ wwid => { type => 'string', description => 'The LUN WWID.' },
+ alias => {
+ type => 'string',
+ description => 'The configured alias, if any.',
+ optional => 1,
+ },
+ 'used-by' => {
+ type => 'string',
+ description => 'The storage consuming this LUN, if any.',
+ optional => 1,
+ },
+ size => {
+ type => 'integer',
+ description => 'LUN size in bytes, as reported by a node.',
+ optional => 1,
+ },
+ 'cluster-state' => {
+ type => 'string',
+ description => "Worst map state across the actively multipathing"
+ . " nodes: 'optimal', 'degraded' (some paths down on a node),"
+ . " 'missing' (an active node has not assembled it), 'failed'"
+ . " (no active path), or 'unknown' (no active node reports it).",
+ enum => ['optimal', 'degraded', 'missing', 'failed', 'unknown'],
+ },
+ nodes => {
+ type => 'object',
+ description => 'Per-node map state, keyed by node name.',
+ additionalProperties => 1,
+ },
+ },
+ },
+ },
+ nodes => {
+ type => 'object',
+ description =>
+ "Per-node config-apply status, keyed by node name; only nodes that"
+ . " failed to apply the configuration appear.",
+ additionalProperties => 1,
+ },
+ },
+ },
+ code => sub {
+ my $cfg = PVE::Multipath::ClusterConfig::read_config();
+
+ my $now = time();
+ my $raw_kv = PVE::Cluster::get_node_kv('multipath');
+ my $node_kv = {};
+ my $stale = {};
+ for my $node (keys %$raw_kv) {
+ my $decoded = eval { decode_json($raw_kv->{$node}) };
+ next if !$decoded || ref($decoded) ne 'HASH' || ref($decoded->{maps}) ne 'HASH';
+ $node_kv->{$node} = $decoded->{maps};
+ # a reporter that stopped refreshing its broadcast gets demoted below: its last
+ # snapshot must not read as current health
+ $stale->{$node} = 1
+ if !defined($decoded->{time})
+ || $now - $decoded->{time} > $PVE::Multipath::KV_STALE_SECONDS;
+ }
+
+ my $expectations = PVE::Multipath::cluster_storage_expectations();
+ my $allow_wwids = PVE::Multipath::Config::wwid_list($cfg);
+
+ # per-LUN expected node set: the consuming storage chain when known, else wherever a
+ # multipath storage is enabled
+ my $expected =
+ { map { $_ => $expectations->{'wwid-nodes'}->{$_} // $expectations->{nodes} }
+ $allow_wwids->@* };
+
+ # resolve liveness for every node we might place in the matrix: those that broadcast and
+ # those a multipath storage expects (and that may be silent)
+ my $members = PVE::Cluster::get_members() // {};
+ my $online = {};
+ for my $node (keys %$node_kv, keys $expectations->{nodes}->%*) {
+ # standalone clusters carry no member info; treat the reporter as live
+ $online->{$node} =
+ (!%$members || ($members->{$node} && $members->{$node}->{online}))
+ && !$stale->{$node} ? 1 : 0;
+ }
+
+ my $luns = PVE::Multipath::aggregate_cluster_status(
+ $allow_wwids,
+ PVE::Multipath::Config::aliases($cfg),
+ $expectations->{consumers},
+ $node_kv,
+ $online,
+ $expected,
+ );
+
+ # surface nodes that could not apply the cluster config, so local drift is visible instead
+ # of silently diverging from the cluster-wide configuration
+ my $apply_kv = PVE::Cluster::get_node_kv('multipath-apply');
+ my $nodes = {};
+ for my $node (keys %$apply_kv) {
+ # an offline node cannot clear its own KV, so skip its stale apply error (same liveness
+ # rule as the health roll-up); a standalone cluster has no member info, so keep it
+ next if %$members && !($members->{$node} && $members->{$node}->{online});
+ my $decoded = eval { decode_json($apply_kv->{$node}) };
+ next if !$decoded || ref($decoded) ne 'HASH' || !$decoded->{error};
+ $nodes->{$node} = {
+ 'apply-error' => $decoded->{error},
+ 'apply-time' => $decoded->{time},
+ };
+ }
+
+ return { luns => $luns, nodes => $nodes };
+ },
+});
+
__PACKAGE__->register_method({
name => 'read_overrides',
path => 'overrides',
diff --git a/src/PVE/Multipath.pm b/src/PVE/Multipath.pm
index ec87b08..67165c6 100644
--- a/src/PVE/Multipath.pm
+++ b/src/PVE/Multipath.pm
@@ -478,4 +478,136 @@ sub broadcast_health {
update_node_kv('multipath', { maps => $summary });
}
+# Broadcast whether this node could apply the cluster-wide multipath config, so the datacenter view
+# can flag a node whose local multipathd state drifted from the configured one instead of failing
+# silently. Pass the error to publish it, or nothing to clear it after a successful apply.
+sub broadcast_apply_status {
+ my ($error) = @_;
+ update_node_kv('multipath-apply', $error ? { error => "$error" } : undef);
+}
+
+# Severity ordering for rolling per-node states up into a cluster state; a higher number is worse.
+# 'unknown' is a stale or offline node and never drives the roll-up, so it sits below 'optimal'.
+my $STATE_RANK = {
+ unknown => -1,
+ optimal => 0,
+ degraded => 1,
+ missing => 2,
+ failed => 3,
+};
+
+# Clamp a state string from another node's broadcast to the known set, so a newer or buggy peer
+# cannot inject an unrankable state into the matrix.
+my sub known_state {
+ my ($state) = @_;
+ return defined($state) && exists($STATE_RANK->{$state}) ? $state : 'unknown';
+}
+
+# Pure: fold the per-node health summaries (already JSON-decoded) into a per-WWID cluster matrix.
+# Inputs:
+# $allow_wwids arrayref, the cluster WWID allow-list
+# $aliases { wwid => name }
+# $used_by { wwid => storage-id } of consuming LVM storages
+# $node_kv { node => { wwid => summary } } as broadcast by broadcast_health()
+# $online { node => bool }; a node absent here counts as offline
+# $expected { wwid => { node => 1 } } nodes that are supposed to carry each allow-listed
+# LUN, derived from storage_expectations()
+#
+# The cluster-state rolls up over the nodes that should carry each LUN: an expected node without
+# the map is 'missing', whether it broadcasts other maps or nothing at all (it lost every path and
+# cleared its broadcast). A node outside the LUN's expected set is never marked missing, so a SAN
+# zoned to only some nodes or a purely hand-managed map does not drag unrelated nodes red; a real
+# state such a node reports still counts. An allow-listed LUN without expectation info (no
+# multipath storage configured) falls back to expecting every actively multipathing node, the only
+# signal left then. Offline nodes show as 'unknown' and never drive the roll-up. WWIDs outside the
+# allow-list list only the nodes that actually report them.
+sub aggregate_cluster_status {
+ my ($allow_wwids, $aliases, $used_by, $node_kv, $online, $expected) = @_;
+
+ $allow_wwids //= [];
+ $aliases //= {};
+ $used_by //= {};
+ $node_kv //= {};
+ $online //= {};
+ $expected //= {};
+
+ my %allow = map { $_ => 1 } $allow_wwids->@*;
+
+ # report the allow-list plus any WWID a node actually sees
+ my %wwids = %allow;
+ for my $node (keys %$node_kv) {
+ $wwids{$_} = 1 for keys $node_kv->{$node}->%*;
+ }
+
+ my $active_nodes = { map { $_ => 1 } grep { $online->{$_} } keys %$node_kv };
+
+ my $res = [];
+ for my $wwid (sort keys %wwids) {
+ my $nodes = {};
+ my $worst = 'optimal';
+ my $have_active = 0;
+ my $size;
+
+ my $rank = sub {
+ my ($state) = @_;
+ $worst = $state if $STATE_RANK->{$state} > $STATE_RANK->{$worst};
+ };
+
+ my $exp = $allow{$wwid} ? $expected->{$wwid} : undef;
+ $exp = $active_nodes if $allow{$wwid} && !($exp && %$exp);
+
+ for my $node (sort keys %$node_kv) {
+ my $entry = $node_kv->{$node}->{$wwid};
+
+ if (!$online->{$node}) {
+ $nodes->{$node} = { state => 'unknown' } if $entry;
+ next;
+ }
+
+ if ($entry) {
+ my $state = known_state($entry->{state});
+ $have_active = 1 if $state ne 'unknown';
+ $nodes->{$node} = {
+ state => $state,
+ 'paths-active' => $entry->{'paths-active'},
+ 'paths-total' => $entry->{'paths-total'},
+ defined($entry->{transport}) ? (transport => $entry->{transport}) : (),
+ };
+ $size //= $entry->{size};
+ $rank->($state);
+ } elsif ($exp && $exp->{$node}) {
+ # expected to carry this LUN but has not assembled it
+ $have_active = 1;
+ $nodes->{$node} = { state => 'missing' };
+ $rank->('missing');
+ }
+ }
+
+ # Expected nodes with no broadcast at all are missing the map (online) or unreachable
+ # (offline); fold them in so a node that lost every path surfaces instead of vanishing.
+ for my $node (sort keys %{ $exp // {} }) {
+ next if exists $nodes->{$node};
+ if ($online->{$node}) {
+ $have_active = 1;
+ $nodes->{$node} = { state => 'missing' };
+ $rank->('missing');
+ } else {
+ $nodes->{$node} = { state => 'unknown' };
+ }
+ }
+
+ push $res->@*,
+ {
+ wwid => $wwid,
+ defined($aliases->{$wwid}) ? (alias => $aliases->{$wwid}) : (),
+ defined($used_by->{$wwid}) ? ('used-by' => $used_by->{$wwid}) : (),
+ defined($size) ? (size => $size) : (),
+ 'cluster-state' => $have_active ? $worst : 'unknown',
+ nodes => $nodes,
+ };
+ }
+
+ return $res;
+}
+
1;
diff --git a/src/test/run_multipath_tests.pl b/src/test/run_multipath_tests.pl
index a4ad57d..793c4b4 100755
--- a/src/test/run_multipath_tests.pl
+++ b/src/test/run_multipath_tests.pl
@@ -407,4 +407,180 @@ my $many = [
my $big = JSON::encode_json(PVE::Multipath::summarize_maps_for_broadcast($many));
ok(length($big) < 32 * 1024, "100-map summary (" . length($big) . " B) fits the KV size limit");
+# --- cluster status aggregation ---
+my $node_kv = {
+ nodeA => {
+ wA =>
+ { state => 'optimal', 'paths-active' => 2, 'paths-total' => 2, transport => 'iscsi' },
+ wB => {
+ state => 'optimal',
+ 'paths-active' => 2,
+ 'paths-total' => 2,
+ transport => 'iscsi',
+ size => 42,
+ },
+ },
+ nodeB => {
+ wA =>
+ { state => 'degraded', 'paths-active' => 1, 'paths-total' => 2, transport => 'iscsi' },
+ # nodeB is active but does not see wB
+ },
+ nodeC => {
+ # stale broadcast from an offline node
+ wA => { state => 'optimal', 'paths-active' => 2, 'paths-total' => 2 },
+ },
+};
+my $agg = PVE::Multipath::aggregate_cluster_status(
+ ['wA', 'wB', 'wZ'], # allow-list incl. an unseen WWID
+ { wA => 'lun-a' }, # alias
+ { wB => 'mptank' }, # used-by
+ $node_kv,
+ { nodeA => 1, nodeB => 1, nodeC => 0 }, # nodeC offline
+);
+my %by_wwid = map { $_->{wwid} => $_ } $agg->@*;
+
+is_deeply([sort keys %by_wwid], ['wA', 'wB', 'wZ'], 'matrix covers allow-list and seen WWIDs');
+
+is($by_wwid{wA}->{alias}, 'lun-a', 'alias surfaced on the WWID row');
+is($by_wwid{wA}->{'cluster-state'}, 'degraded', 'degraded on one active node rolls up to degraded');
+is($by_wwid{wA}->{nodes}->{nodeA}->{state}, 'optimal', 'per-node optimal cell kept');
+is($by_wwid{wA}->{nodes}->{nodeB}->{state}, 'degraded', 'per-node degraded cell kept');
+is($by_wwid{wA}->{nodes}->{nodeC}->{state}, 'unknown',
+ 'offline node with stale data shows unknown');
+
+is($by_wwid{wB}->{'used-by'}, 'mptank', 'consuming storage surfaced as used-by');
+is($by_wwid{wB}->{size}, 42, 'LUN size surfaced from a reporting node');
+is(
+ $by_wwid{wB}->{'cluster-state'},
+ 'missing',
+ 'active node not assembling the LUN rolls up to missing',
+);
+is(
+ $by_wwid{wB}->{nodes}->{nodeB}->{state},
+ 'missing',
+ 'missing marked on the active node lacking it',
+);
+
+is(
+ $by_wwid{wZ}->{'cluster-state'},
+ 'missing',
+ 'allow-listed WWID no active node assembled is missing everywhere',
+);
+is($by_wwid{wZ}->{nodes}->{nodeA}->{state}, 'missing', 'active node missing the allow-listed WWID');
+ok(!exists $by_wwid{wZ}->{nodes}->{nodeC}, 'offline node contributes no cell for an unseen WWID');
+
+# a WWID only an offline node ever reported, with no online active node, is unknown
+my $agg_off = PVE::Multipath::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ { dead => { wA => { state => 'optimal', 'paths-active' => 2, 'paths-total' => 2 } } },
+ { dead => 0 },
+);
+is(
+ $agg_off->[0]->{'cluster-state'},
+ 'unknown',
+ 'no online active node leaves the cluster-state unknown',
+);
+is($agg_off->[0]->{nodes}->{dead}->{state}, 'unknown', 'stale offline node shown as unknown');
+
+# failure outranks degraded in the roll-up
+my $agg2 = PVE::Multipath::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ {
+ n1 => { wA => { state => 'degraded', 'paths-active' => 1, 'paths-total' => 2 } },
+ n2 => { wA => { state => 'failed', 'paths-active' => 0, 'paths-total' => 2 } },
+ },
+ { n1 => 1, n2 => 1 },
+);
+is($agg2->[0]->{'cluster-state'}, 'failed', 'failed outranks degraded in the cluster roll-up');
+
+# --- expected-node set: a node that lost all paths (silent) must not vanish ---
+# nodeS is expected (a multipath storage is enabled there) and online, but
+# broadcasts nothing - e.g. every path to the SAN is down so it cleared its KV.
+my $exp_kv = {
+ nodeA => { wA => { state => 'optimal', 'paths-active' => 2, 'paths-total' => 2 } },
+};
+my $online = { nodeA => 1, nodeS => 1, nodeOff => 0 };
+my $expected = { wA => { nodeA => 1, nodeS => 1, nodeOff => 1 } };
+my $eagg = PVE::Multipath::aggregate_cluster_status(
+ ['wA'], {}, {}, $exp_kv, $online, $expected,
+);
+my $row = $eagg->[0];
+is($row->{nodes}->{nodeA}->{state}, 'optimal', 'reporting node keeps its real state');
+is(
+ $row->{nodes}->{nodeS}->{state},
+ 'missing',
+ 'expected online but silent node shows missing instead of vanishing',
+);
+is($row->{nodes}->{nodeOff}->{state}, 'unknown', 'expected offline node shows unknown');
+is($row->{'cluster-state'}, 'missing', 'a silent expected node drags the cluster-state to missing');
+
+# without $expected the silent node would have been invisible (regression guard
+# for the old behavior, proving the new param is what surfaces it)
+my $noexp = PVE::Multipath::aggregate_cluster_status(['wA'], {}, {}, $exp_kv, $online);
+ok(
+ !exists $noexp->[0]->{nodes}->{nodeS},
+ 'without the expected set the silent node is absent (the gap the param closes)',
+);
+is($noexp->[0]->{'cluster-state'}, 'optimal', 'and the cluster-state would falsely read optimal');
+
+# expected augmentation applies only to allow-listed WWIDs, not to a LUN that a
+# node merely happens to report off-list
+my $offlist = PVE::Multipath::aggregate_cluster_status(
+ [],
+ {},
+ {},
+ { nodeA => { wX => { state => 'optimal', 'paths-active' => 1, 'paths-total' => 1 } } },
+ { nodeA => 1, nodeS => 1 },
+ {},
+);
+ok(
+ !exists $offlist->[0]->{nodes}->{nodeS},
+ 'non-allow-listed WWID does not synthesize missing cells for expected nodes',
+);
+
+# a broadcasting node outside a LUN's expected set is never marked missing (a SAN zoned to only
+# some nodes), and a hand-made off-list map lists only the nodes that report it
+my $zoned = PVE::Multipath::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ {
+ n1 => { wA => { state => 'optimal', 'paths-active' => 2, 'paths-total' => 2 } },
+ n2 => { wHand => { state => 'degraded', 'paths-active' => 1, 'paths-total' => 2 } },
+ },
+ { n1 => 1, n2 => 1 },
+ { wA => { n1 => 1 } },
+);
+my %zoned_rows = map { $_->{wwid} => $_ } $zoned->@*;
+ok(
+ !exists $zoned_rows{wA}->{nodes}->{n2},
+ 'a multipathing node outside the expected set is not marked missing',
+);
+is($zoned_rows{wA}->{'cluster-state'}, 'optimal', 'the zoned LUN stays green on its own nodes');
+ok(
+ !exists $zoned_rows{wHand}->{nodes}->{n1},
+ 'an off-list map does not drag other multipathing nodes into its row',
+);
+is(
+ $zoned_rows{wHand}->{'cluster-state'},
+ 'degraded',
+ 'an off-list map still surfaces its real state',
+);
+
+# an unrecognized state from a (newer or buggy) peer clamps to unknown and does not fake a report
+my $clamped = PVE::Multipath::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ { n1 => { wA => { state => 'frobnicated', 'paths-active' => 1, 'paths-total' => 2 } } },
+ { n1 => 1 },
+ { wA => { n1 => 1 } },
+);
+is($clamped->[0]->{nodes}->{n1}->{state}, 'unknown', 'unrecognized peer state clamps to unknown');
+is($clamped->[0]->{'cluster-state'}, 'unknown', 'a clamped state does not count as active');
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 cluster 08/13] pmxcfs: track cluster-wide multipath configuration
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (6 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 storage 07/13] api: multipath: add cluster-wide health status endpoint Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 manager 09/13] pvestatd: apply the cluster-wide multipath config on each node Thomas Lamprecht
` (4 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
pve-storage registers two cluster-wide files via cfs_register_file: the
managed /etc/pve/multipath.cfg (the WWID allow-list plus the global and
per-LUN multipathd knobs) and /etc/pve/multipath-overrides.conf for the
free-form, admin-owned hardware override sections. Both need to be
observed and tracked for change notification, so they are replicated
across the cluster and consumers are notified on change.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
No changes since v1.
src/PVE/Cluster.pm | 2 ++
src/pmxcfs/status.c | 2 ++
2 files changed, 4 insertions(+)
diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index 034b78c..0a53549 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -50,6 +50,8 @@ my $observed = {
'vzdump.conf' => 1,
'jobs.cfg' => 1,
'storage.cfg' => 1,
+ 'multipath.cfg' => 1,
+ 'multipath-overrides.conf' => 1,
'datacenter.cfg' => 1,
'replication.cfg' => 1,
'corosync.conf' => 1,
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index 12a6c46..70dcc72 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -80,6 +80,8 @@ static memdb_change_t memdb_change_array[] = {
{.path = "corosync.conf"},
{.path = "corosync.conf.new"},
{.path = "storage.cfg"},
+ {.path = "multipath.cfg"},
+ {.path = "multipath-overrides.conf"},
{.path = "user.cfg"},
{.path = "domains.cfg"},
{.path = "notifications.cfg"},
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 manager 09/13] pvestatd: apply the cluster-wide multipath config on each node
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (7 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 cluster 08/13] pmxcfs: track cluster-wide multipath configuration Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 manager 10/13] api: cluster: mount the multipath configuration endpoint Thomas Lamprecht
` (3 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Write out the cluster-wide multipath.cfg on each node once per status
loop: build the local multipathd drop-ins and reload multipathd when
they change. The work is a cheap no-op unless multipath is in use, and
the logic is kept out of pvestatd itself, so this is just a thin
trigger.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Changes in v2:
- broadcast whether this node could apply the config
- use PVE::Multipath explicitly instead of relying on the generator
pulling it in
PVE/Service/pvestatd.pm | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/PVE/Service/pvestatd.pm b/PVE/Service/pvestatd.pm
index 34f9da71f..1bdd1db65 100755
--- a/PVE/Service/pvestatd.pm
+++ b/PVE/Service/pvestatd.pm
@@ -20,6 +20,8 @@ use PVE::RS::SDN::Fabrics;
use PVE::NodeConfig;
use PVE::Cluster qw(cfs_read_file);
use PVE::Storage;
+use PVE::Multipath;
+use PVE::Multipath::Generator;
use PVE::QemuServer;
use PVE::QemuServer::Monitor;
use PVE::LXC;
@@ -848,6 +850,16 @@ sub update_status {
$err = $@;
syslog('err', "storage status update error: $err") if $err;
+ # apply the cluster-wide multipath config on this node; a cheap no-op unless multipath is in
+ # use, and the logic is kept out of pvestatd so this stays a thin trigger. Broadcast a failed
+ # apply so a node drifting from the configured state shows up in the datacenter view.
+ my $apply_err = eval { PVE::Multipath::Generator::sync(); };
+ $apply_err = $@ if $@;
+ syslog('err', "multipath config sync error: $apply_err") if $apply_err;
+ eval { PVE::Multipath::broadcast_apply_status($apply_err); };
+ $err = $@;
+ syslog('err', "multipath apply-status broadcast error: $err") if $err;
+
eval { remove_stale_lxc_consoles(); };
$err = $@;
syslog('err', "lxc console cleanup error: $err") if $err;
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 manager 10/13] api: cluster: mount the multipath configuration endpoint
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (8 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 manager 09/13] pvestatd: apply the cluster-wide multipath config on each node Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 manager 11/13] pvestatd: broadcast multipath map health to the cluster Thomas Lamprecht
` (2 subsequent siblings)
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Expose pve-storage's cluster-wide multipath configuration API at
/cluster/multipath, next to the other cluster-level config endpoints.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
No changes since v1.
PVE/API2/Cluster.pm | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index 4e5efbfd9..576840181 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -35,6 +35,7 @@ use PVE::API2::Cluster::Qemu;
use PVE::API2::ClusterConfig;
use PVE::API2::Firewall::Cluster;
use PVE::API2::HAConfig;
+use PVE::API2::Multipath;
use PVE::API2::ReplicationConfig;
my $have_sdn;
@@ -100,6 +101,11 @@ __PACKAGE__->register_method({
path => 'ceph',
});
+__PACKAGE__->register_method({
+ subclass => "PVE::API2::Multipath",
+ path => 'multipath',
+});
+
__PACKAGE__->register_method({
subclass => "PVE::API2::Cluster::Jobs",
path => 'jobs',
@@ -169,6 +175,7 @@ __PACKAGE__->register_method({
{ name => 'log' },
{ name => 'mapping' },
{ name => 'metrics' },
+ { name => 'multipath' },
{ name => 'notifications' },
{ name => 'nextid' },
{ name => 'options' },
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 manager 11/13] pvestatd: broadcast multipath map health to the cluster
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (9 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 manager 10/13] api: cluster: mount the multipath configuration endpoint Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 manager 12/13] ui: dc: add multipath health matrix and config editor Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 manager 13/13] ui: node: show multipath maps and their paths under Disks Thomas Lamprecht
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Publish this node's per-WWID map health into the cluster KV store on
each status loop, so the datacenter-wide health matrix can combine it
without querying every node live. The helper clears the key when this
node is not multipathing, keeping the matrix scoped to the active
nodes. Guarded and self-contained like the nearby config sync.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Changes in v2:
- the explicit PVE::Multipath use moved to the pvestatd apply patch
PVE/Service/pvestatd.pm | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/PVE/Service/pvestatd.pm b/PVE/Service/pvestatd.pm
index 1bdd1db65..0ac56eb7d 100755
--- a/PVE/Service/pvestatd.pm
+++ b/PVE/Service/pvestatd.pm
@@ -860,6 +860,12 @@ sub update_status {
$err = $@;
syslog('err', "multipath apply-status broadcast error: $err") if $err;
+ # publish this node's multipath map health into the cluster KV store so the datacenter-wide
+ # health matrix can roll it up; clears itself when not in use.
+ eval { PVE::Multipath::broadcast_health(); };
+ $err = $@;
+ syslog('err', "multipath health broadcast error: $err") if $err;
+
eval { remove_stale_lxc_consoles(); };
$err = $@;
syslog('err', "lxc console cleanup error: $err") if $err;
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 manager 12/13] ui: dc: add multipath health matrix and config editor
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (10 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 manager 11/13] pvestatd: broadcast multipath map health to the cluster Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
2026-07-03 12:46 ` [PATCH v2 manager 13/13] ui: node: show multipath maps and their paths under Disks Thomas Lamprecht
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Give cluster operators a single place to answer the question that no
per-node view can: for each multipath LUN, is every node that should see
it seeing it with full path redundancy? The grid loads the cluster
status and lays out a WWID by node matrix, coloring each cell by the
per-node map state and the combined cluster state, so a LUN that is
degraded on just one node stands out at a glance. The same panel manages
the cluster config: the WWID allow-list, per-WWID aliases, and the
verbatim hardware override sections. Node columns are built from the
loaded data, since a node only appears once it is actively multipathing.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Changes in v2:
- update the grid in place via a DiffStore instead of reloading
- never load the diff store from its empty memory proxy: the stateful
grid re-applies its state on reconfigure, and the resulting load
wiped the first synced batch of rows right after opening the panel
- give the base columns stable state ids, reconfigure invalidated
their saved widths and order on every panel open
- pass the force flag for removals in the query string, a request
body is rejected for DELETE
- warn about nodes that failed to apply the config
- edit the overrides through their own endpoint; the v1 save never
worked, its auto-sent digest was rejected by the PUT schema
- refresh after edits via apiCallDone (taskDone never fired) and keep
the selection-bound actions current on in-place row updates
- HTML-encode all values from other nodes in the renderers
- read the new { luns, nodes } shape; assorted review fixes
www/manager6/Makefile | 1 +
www/manager6/Utils.js | 25 ++
www/manager6/dc/Config.js | 6 +
www/manager6/dc/Multipath.js | 444 +++++++++++++++++++++++++++++++++++
4 files changed, 476 insertions(+)
create mode 100644 www/manager6/dc/Multipath.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index d4dd3f351..1a3b845c8 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -183,6 +183,7 @@ JSSRC= \
dc/Guests.js \
dc/Health.js \
dc/Log.js \
+ dc/Multipath.js \
dc/NodeView.js \
dc/OptionView.js \
dc/PermissionView.js \
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index 040b5ae01..e5854cb73 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -274,6 +274,31 @@ Ext.define('PVE.Utils', {
return '<i class="fa fa-' + iconCls + '"></i> ' + value;
},
+ // Maps a multipath map state (optimal/degraded/missing/failed/unknown) to an icon; shared
+ // by the datacenter health matrix and the node view.
+ render_multipath_health: function (value) {
+ if (typeof value === 'undefined') {
+ return '';
+ }
+ let iconCls = 'question-circle';
+ switch (value) {
+ case 'optimal':
+ iconCls = 'check-circle good';
+ break;
+ case 'degraded':
+ iconCls = 'exclamation-circle warning';
+ break;
+ case 'missing':
+ case 'failed':
+ iconCls = 'times-circle critical';
+ break;
+ case 'unknown':
+ default:
+ iconCls = 'question-circle faded';
+ }
+ return '<i class="fa fa-' + iconCls + '"></i> ' + Ext.htmlEncode(value);
+ },
+
validateZfsBlocksize: function (value) {
if (!value) {
return true;
diff --git a/www/manager6/dc/Config.js b/www/manager6/dc/Config.js
index e17066368..dfba75e8a 100644
--- a/www/manager6/dc/Config.js
+++ b/www/manager6/dc/Config.js
@@ -120,6 +120,12 @@ Ext.define('PVE.dc.Config', {
iconCls: 'fa fa-gear',
itemId: 'options',
},
+ {
+ xtype: 'pveDcMultipath',
+ title: gettext('Multipath'),
+ iconCls: 'fa fa-road',
+ itemId: 'multipath',
+ },
);
}
diff --git a/www/manager6/dc/Multipath.js b/www/manager6/dc/Multipath.js
new file mode 100644
index 000000000..fac353d5b
--- /dev/null
+++ b/www/manager6/dc/Multipath.js
@@ -0,0 +1,444 @@
+Ext.define('PVE-dc-multipath-status', {
+ extend: 'Ext.data.Model',
+ fields: ['wwid', 'alias', 'used-by', 'size', 'cluster-state', 'nodes'],
+ idProperty: 'wwid',
+});
+
+// Edit window to add a WWID to the cluster-wide multipath allow-list.
+Ext.define('PVE.dc.MultipathWwidEdit', {
+ extend: 'Proxmox.window.Edit',
+ xtype: 'pveDcMultipathWwidEdit',
+
+ subject: gettext('Multipath LUN'),
+ isCreate: true,
+ method: 'POST',
+ url: '/cluster/multipath/wwid',
+
+ items: [
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'wwid',
+ fieldLabel: 'WWID',
+ allowBlank: false,
+ emptyText: '3600140500a1b2c3d4e5f6a7b8c9d0e1f',
+ },
+ ],
+});
+
+// Edit window to set or replace a WWID's alias.
+Ext.define('PVE.dc.MultipathAliasEdit', {
+ extend: 'Proxmox.window.Edit',
+ xtype: 'pveDcMultipathAliasEdit',
+
+ subject: gettext('Multipath Alias'),
+ method: 'POST',
+ url: '/cluster/multipath/alias',
+
+ initComponent: function () {
+ let me = this;
+
+ if (!me.wwid) {
+ throw 'no wwid specified';
+ }
+ me.isCreate = !me.alias;
+
+ Ext.apply(me, {
+ items: [
+ {
+ xtype: 'displayfield',
+ name: 'wwid',
+ fieldLabel: 'WWID',
+ value: me.wwid,
+ submitValue: true,
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'alias',
+ fieldLabel: gettext('Alias'),
+ allowBlank: false,
+ value: me.alias,
+ emptyText: 'san-a-lun0',
+ },
+ ],
+ });
+
+ me.callParent();
+ },
+});
+
+// Edit window for the verbatim multipath.conf override sections (hardware- and SAN-specific
+// 'device {}' / 'overrides {}' blocks). Loads the current text and writes the overrides back.
+Ext.define('PVE.dc.MultipathOverridesEdit', {
+ extend: 'Proxmox.window.Edit',
+ xtype: 'pveDcMultipathOverridesEdit',
+
+ subject: gettext('Multipath Overrides'),
+ width: 600,
+ autoLoad: true,
+ method: 'PUT',
+ url: '/cluster/multipath/overrides',
+
+ items: [
+ {
+ xtype: 'textarea',
+ name: 'overrides',
+ fieldLabel: gettext('Overrides'),
+ height: 320,
+ fieldStyle: 'font-family: monospace;',
+ emptyText: 'device {\n\tvendor "..."\n\tproduct "..."\n\tpath_grouping_policy ...\n}',
+ },
+ ],
+});
+
+Ext.define('PVE.dc.MultipathView', {
+ extend: 'Ext.grid.Panel',
+ xtype: 'pveDcMultipath',
+
+ stateful: true,
+ stateId: 'grid-dc-multipath',
+
+ emptyText: gettext('No multipath LUNs configured or reported.'),
+
+ viewModel: {
+ data: {
+ wwid: '',
+ alias: '',
+ },
+ },
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ reload: function () {
+ this.getView().rstore.load();
+ },
+
+ selectedRecord: function () {
+ let sel = this.getView().getSelection();
+ return sel.length ? sel[0] : undefined;
+ },
+
+ addWwid: function () {
+ let me = this;
+ Ext.create('PVE.dc.MultipathWwidEdit', {
+ apiCallDone: (success) => success && me.reload(),
+ autoShow: true,
+ });
+ },
+
+ setAlias: function () {
+ let me = this;
+ let rec = me.selectedRecord();
+ if (!rec) {
+ return;
+ }
+ Ext.create('PVE.dc.MultipathAliasEdit', {
+ wwid: rec.data.wwid,
+ alias: rec.data.alias,
+ apiCallDone: (success) => success && me.reload(),
+ autoShow: true,
+ });
+ },
+
+ removeAlias: function () {
+ let me = this;
+ let rec = me.selectedRecord();
+ if (!rec || !rec.data.alias) {
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: `/cluster/multipath/alias/${encodeURIComponent(rec.data.wwid)}`,
+ method: 'DELETE',
+ waitMsgTarget: me.getView(),
+ failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
+ success: () => me.reload(),
+ });
+ },
+
+ removeWwid: function () {
+ let me = this;
+ let rec = me.selectedRecord();
+ if (!rec) {
+ return;
+ }
+ let wwid = rec.data.wwid;
+ let usedBy = rec.data['used-by'];
+ let msg = usedBy
+ ? Ext.String.format(
+ gettext(
+ "WWID '{0}' is still used by storage '{1}'. Removing it drops the multipath map and the storage loses its device. Remove anyway?",
+ ),
+ wwid,
+ usedBy,
+ )
+ : Ext.String.format(
+ gettext("Remove WWID '{0}' from the multipath allow-list?"),
+ wwid,
+ );
+ Ext.Msg.confirm(gettext('Confirm'), msg, function (btn) {
+ if (btn !== 'yes') {
+ return;
+ }
+ // the flag must go into the query string, a request body is rejected for DELETE
+ let force = usedBy ? '?force=1' : '';
+ Proxmox.Utils.API2Request({
+ url: `/cluster/multipath/wwid/${encodeURIComponent(wwid)}${force}`,
+ method: 'DELETE',
+ waitMsgTarget: me.getView(),
+ failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
+ success: () => me.reload(),
+ });
+ });
+ },
+
+ editOverrides: function () {
+ let me = this;
+ Ext.create('PVE.dc.MultipathOverridesEdit', {
+ apiCallDone: (success) => success && me.reload(),
+ autoShow: true,
+ });
+ },
+
+ // Rebuild the per-node columns from the loaded matrix; nodes only appear once they
+ // actively multipath, so the column set is data-driven.
+ onStoreLoad: function (store, records, success) {
+ let me = this;
+ let view = me.getView();
+
+ if (!success) {
+ return;
+ }
+
+ // surface nodes that could not apply the cluster config (sidecar to the per-WWID rows)
+ let nodeStatus = view.applyNodeStatus || {};
+ let failed = Object.keys(nodeStatus).filter((n) => nodeStatus[n]['apply-error']);
+ let warn = view.down('#applyWarning');
+ if (warn) {
+ if (failed.length) {
+ let lines = failed
+ .map(
+ (n) =>
+ `${Ext.htmlEncode(n)}: ${Ext.htmlEncode(nodeStatus[n]['apply-error'])}`,
+ )
+ .join('<br>');
+ warn.setHtml(
+ '<i class="fa fa-exclamation-triangle warning"></i> ' +
+ gettext('Some nodes could not apply the multipath configuration:') +
+ '<br>' +
+ lines,
+ );
+ }
+ warn.setHidden(!failed.length);
+ }
+
+ let nodes = {};
+ (records || []).forEach((rec) => {
+ Ext.Object.each(rec.data.nodes || {}, (node) => {
+ nodes[node] = true;
+ });
+ });
+ let nodeList = Object.keys(nodes).sort();
+
+ // the DiffStore updates selected rows in place, so refresh the selection-derived
+ // view-model values (alias may have changed without a selectionchange event)
+ me.syncSelection();
+
+ if (Ext.Array.equals(nodeList, view.shownNodes || [])) {
+ return;
+ }
+ view.shownNodes = nodeList;
+
+ let nodeColumns = nodeList.map((node) => ({
+ text: node,
+ stateId: `node-${node}`,
+ align: 'center',
+ width: 120,
+ sortable: false,
+ menuDisabled: true,
+ renderer: function (val, meta, rec) {
+ let st = (rec.data.nodes || {})[node];
+ if (!st) {
+ return '<span style="opacity:0.5">-</span>';
+ }
+ let out = PVE.Utils.render_multipath_health(st.state);
+ if (st['paths-total'] !== undefined) {
+ let paths = `${st['paths-active']}/${st['paths-total']}`;
+ out += ` (${Ext.htmlEncode(paths)})`;
+ }
+ return out;
+ },
+ }));
+
+ view.reconfigure(undefined, view.baseColumns.concat(nodeColumns));
+ },
+
+ syncSelection: function () {
+ let vm = this.getViewModel();
+ let rec = this.selectedRecord();
+ vm.set('wwid', rec ? rec.data.wwid : '');
+ vm.set('alias', rec && rec.data.alias ? rec.data.alias : '');
+ },
+
+ control: {
+ '#': {
+ selectionchange: 'syncSelection',
+ },
+ },
+ },
+
+ // explicit stateIds: the auto-generated ones change on every reconfigure() for the dynamic
+ // node columns, which would keep invalidating the saved widths and order of these
+ baseColumns: [
+ {
+ text: 'WWID',
+ dataIndex: 'wwid',
+ stateId: 'wwid',
+ flex: 2,
+ renderer: Ext.htmlEncode,
+ },
+ {
+ text: gettext('Alias'),
+ dataIndex: 'alias',
+ stateId: 'alias',
+ width: 140,
+ renderer: (v) => Ext.htmlEncode(v || ''),
+ },
+ {
+ text: gettext('Used By'),
+ dataIndex: 'used-by',
+ stateId: 'used-by',
+ width: 120,
+ renderer: (v) => Ext.htmlEncode(v || ''),
+ },
+ {
+ text: gettext('Size'),
+ dataIndex: 'size',
+ stateId: 'size',
+ width: 90,
+ align: 'right',
+ renderer: (v) => (v ? Proxmox.Utils.format_size(v) : ''),
+ },
+ {
+ text: gettext('Cluster Health'),
+ dataIndex: 'cluster-state',
+ stateId: 'cluster-state',
+ width: 130,
+ renderer: PVE.Utils.render_multipath_health,
+ },
+ ],
+
+ initComponent: function () {
+ let me = this;
+
+ let caps = Ext.state.Manager.get('GuiCap');
+ let canModify = !!(caps && caps.dc && caps.dc['Sys.Modify']);
+
+ me.columns = me.baseColumns;
+
+ // health is live status: poll while the panel is shown so a path going down surfaces
+ // without a manual reload. Wrap the polling store in a DiffStore so only changed rows
+ // update instead of the whole grid rebuilding and flickering every interval.
+ let rstore = Ext.create('Proxmox.data.UpdateStore', {
+ storeid: 'dc-multipath-status',
+ interval: 3000,
+ model: 'PVE-dc-multipath-status',
+ proxy: {
+ type: 'proxmox',
+ url: '/api2/json/cluster/multipath/status',
+ reader: {
+ // rows come from data.luns; stash the sibling per-node apply-status sidecar so
+ // onStoreLoad can flag nodes that failed to apply the config
+ rootProperty: 'data.luns',
+ transform: (raw) => {
+ me.applyNodeStatus = (raw && raw.data && raw.data.nodes) || {};
+ return raw;
+ },
+ },
+ },
+ });
+ me.rstore = rstore;
+
+ me.store = Ext.create('Proxmox.data.DiffStore', {
+ rstore: rstore,
+ sorters: [{ property: 'wwid' }],
+ });
+ // the stateful grid re-applies its saved state on every reconfigure, and that issues a
+ // load on this store, replacing the records synced from the rstore with the empty
+ // memory-proxy content; data only ever comes from the rstore, so drop proxy loads
+ me.store.load = Ext.emptyFn;
+
+ // shown when a node fails to apply the cluster config, populated from the status sidecar
+ me.dockedItems = [
+ {
+ xtype: 'component',
+ itemId: 'applyWarning',
+ dock: 'top',
+ hidden: true,
+ padding: '5 10',
+ // themed hint surface (light/dark aware), matching other inline PVE warnings
+ userCls: 'pmx-hint',
+ html: '',
+ },
+ ];
+
+ // a row selection only gates the modify actions when the user may modify
+ let selectionGate = canModify ? { bind: { disabled: '{!wwid}' } } : { disabled: true };
+
+ me.tbar = [
+ {
+ text: gettext('Reload'),
+ iconCls: 'fa fa-refresh',
+ handler: 'reload',
+ },
+ {
+ text: gettext('Add WWID'),
+ iconCls: 'fa fa-plus-circle',
+ disabled: !canModify,
+ handler: 'addWwid',
+ },
+ Ext.apply(
+ {
+ text: gettext('Set Alias'),
+ iconCls: 'fa fa-tag',
+ handler: 'setAlias',
+ },
+ selectionGate,
+ ),
+ Ext.apply(
+ {
+ text: gettext('Remove'),
+ iconCls: 'fa fa-trash-o',
+ menu: [
+ {
+ text: gettext('Remove WWID'),
+ iconCls: 'fa fa-fw fa-trash-o',
+ handler: 'removeWwid',
+ },
+ {
+ text: gettext('Remove Alias'),
+ iconCls: 'fa fa-fw fa-eraser',
+ bind: { disabled: '{!alias}' },
+ handler: 'removeAlias',
+ },
+ ],
+ },
+ selectionGate,
+ ),
+ '->',
+ {
+ text: gettext('Edit Overrides'),
+ iconCls: 'fa fa-pencil',
+ disabled: !canModify,
+ handler: 'editOverrides',
+ },
+ ];
+
+ me.callParent();
+
+ Proxmox.Utils.monStoreErrors(me, rstore, true);
+ me.mon(rstore, 'load', me.getController().onStoreLoad, me.getController());
+ me.on('activate', () => rstore.startUpdate());
+ me.on('deactivate', () => rstore.stopUpdate());
+ me.on('destroy', () => rstore.stopUpdate());
+ },
+});
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread* [PATCH v2 manager 13/13] ui: node: show multipath maps and their paths under Disks
2026-07-03 12:46 [PATCH v2 storage,cluster,manager 0/13] multipath: cluster-wide config, storage and health overview Thomas Lamprecht
` (11 preceding siblings ...)
2026-07-03 12:46 ` [PATCH v2 manager 12/13] ui: dc: add multipath health matrix and config editor Thomas Lamprecht
@ 2026-07-03 12:46 ` Thomas Lamprecht
12 siblings, 0 replies; 14+ messages in thread
From: Thomas Lamprecht @ 2026-07-03 12:46 UTC (permalink / raw)
To: pve-devel
Add a read-only per-node view of the device-mapper multipath maps and
their individual paths, the detail behind the datacenter health matrix.
Each map expands to its paths with their device, state and transport, so
when the matrix flags a node as degraded an operator can see exactly
which path is down without dropping to the shell.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Changes in v2:
- capitalize the 'Used By' column header
- show the correct reason for an empty grid (not installed / daemon down)
- HTML-encode the name, WWID, and transport renderers
www/manager6/Makefile | 1 +
www/manager6/node/Config.js | 7 ++
www/manager6/node/Multipath.js | 171 +++++++++++++++++++++++++++++++++
3 files changed, 179 insertions(+)
create mode 100644 www/manager6/node/Multipath.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 1a3b845c8..9b5125a66 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -245,6 +245,7 @@ JSSRC= \
node/Directory.js \
node/LVM.js \
node/LVMThin.js \
+ node/Multipath.js \
node/StatusView.js \
node/Subscription.js \
node/Summary.js \
diff --git a/www/manager6/node/Config.js b/www/manager6/node/Config.js
index 217ee284c..bb1c30fd4 100644
--- a/www/manager6/node/Config.js
+++ b/www/manager6/node/Config.js
@@ -357,6 +357,13 @@ Ext.define('PVE.node.Config', {
groups: ['storage'],
xtype: 'pveZFSList',
},
+ {
+ xtype: 'pveMultipathStatus',
+ title: gettext('Multipath'),
+ itemId: 'multipath',
+ iconCls: 'fa fa-road',
+ groups: ['storage'],
+ },
{
xtype: 'pveNodeCephStatus',
title: 'Ceph',
diff --git a/www/manager6/node/Multipath.js b/www/manager6/node/Multipath.js
new file mode 100644
index 000000000..f21a91483
--- /dev/null
+++ b/www/manager6/node/Multipath.js
@@ -0,0 +1,171 @@
+Ext.define('PVE-node-multipath-map', {
+ extend: 'Ext.data.Model',
+ idProperty: 'wwid',
+ fields: [
+ 'wwid',
+ 'name',
+ 'health',
+ 'transport',
+ 'used-by',
+ 'path-groups',
+ { name: 'paths-active', type: 'number' },
+ { name: 'paths-total', type: 'number' },
+ { name: 'size', type: 'number' },
+ {
+ // pre-rendered path detail for the row expander; keeps the hyphenated map keys out
+ // of the XTemplate and recomputes when the paths change
+ name: 'pathsHtml',
+ calculate: function (data) {
+ let html = '';
+ (data['path-groups'] || []).forEach((pg) => {
+ (pg.paths || []).forEach((p) => {
+ let state = p['dm-state'] || '';
+ let icon =
+ state === 'active' ? 'check-circle good' : 'times-circle critical';
+ let extra = [p['dev-state'], p.transport].filter((v) => v).join(', ');
+ html +=
+ '<div style="padding:1px 0;">' +
+ `<i class="fa fa-fw fa-${icon}"></i> ` +
+ `${Ext.htmlEncode(p.dev || '')} - ${Ext.htmlEncode(state)}` +
+ (extra ? ` (${Ext.htmlEncode(extra)})` : '') +
+ '</div>';
+ });
+ });
+ return html;
+ },
+ },
+ ],
+});
+
+// Read-only per-node view of the device-mapper multipath maps and their paths, the detail behind
+// the datacenter-wide health matrix. The map rows update in place via a DiffStore, so live
+// path-state changes do not flicker or collapse the expanded rows.
+Ext.define('PVE.node.MultipathStatus', {
+ extend: 'Ext.grid.Panel',
+ xtype: 'pveMultipathStatus',
+
+ stateful: true,
+ stateId: 'grid-node-multipath',
+
+ emptyText: gettext('No multipath maps on this node.'),
+
+ plugins: [
+ {
+ ptype: 'rowexpander',
+ rowBodyTpl: ['<div style="padding:2px 0 6px 36px;">{pathsHtml}</div>'],
+ },
+ ],
+
+ columns: [
+ {
+ text: gettext('Name'),
+ dataIndex: 'name',
+ flex: 1,
+ renderer: (v, meta, rec) => Ext.htmlEncode(v || rec.data.wwid),
+ },
+ {
+ text: 'WWID',
+ dataIndex: 'wwid',
+ flex: 1,
+ renderer: Ext.htmlEncode,
+ },
+ {
+ text: gettext('Health'),
+ dataIndex: 'health',
+ width: 130,
+ renderer: PVE.Utils.render_multipath_health,
+ },
+ {
+ text: gettext('Paths'),
+ width: 90,
+ align: 'right',
+ renderer: (v, meta, rec) => `${rec.data['paths-active']}/${rec.data['paths-total']}`,
+ },
+ {
+ text: gettext('Transport'),
+ dataIndex: 'transport',
+ width: 90,
+ renderer: (v) => Ext.htmlEncode(v || ''),
+ },
+ {
+ text: gettext('Size'),
+ dataIndex: 'size',
+ width: 100,
+ align: 'right',
+ renderer: (v) => (v ? Proxmox.Utils.format_size(v) : ''),
+ },
+ {
+ text: gettext('Used By'),
+ dataIndex: 'used-by',
+ flex: 1,
+ renderer: (v) => Ext.htmlEncode(v || ''),
+ },
+ ],
+
+ initComponent: function () {
+ let me = this;
+
+ me.nodename = me.pveSelNode.data.node;
+ if (!me.nodename) {
+ throw 'no node name specified';
+ }
+
+ let rstore = Ext.create('Proxmox.data.UpdateStore', {
+ interval: 3000,
+ storeid: `node-multipath-${me.nodename}`,
+ model: 'PVE-node-multipath-map',
+ proxy: {
+ type: 'proxmox',
+ url: `/api2/json/nodes/${me.nodename}/disks/multipath`,
+ reader: {
+ // maps come from data.maps; stash the supported/running envelope so the load
+ // handler can explain an empty grid (not installed / daemon down / no maps)
+ rootProperty: 'data.maps',
+ transform: (raw) => {
+ me.mpInfo = (raw && raw.data) || {};
+ return raw;
+ },
+ },
+ },
+ });
+
+ me.store = Ext.create('Proxmox.data.DiffStore', {
+ rstore: rstore,
+ sorters: [{ property: 'wwid' }],
+ });
+
+ me.tbar = [
+ {
+ text: gettext('Reload'),
+ iconCls: 'fa fa-refresh',
+ handler: () => rstore.load(),
+ },
+ ];
+
+ me.callParent();
+
+ Proxmox.Utils.monStoreErrors(me, rstore, true);
+
+ // the empty grid means different things (no maps, daemon down, not installed); pull the
+ // reason from the envelope to guide the operator
+ me.mon(rstore, 'load', function (store, records, success) {
+ if (!success) {
+ return;
+ }
+ let info = me.mpInfo || {};
+ let text;
+ if (!info.supported) {
+ text = gettext('multipath-tools is not installed on this node.');
+ } else if (!info.running) {
+ text = gettext('The multipathd daemon is not running.');
+ } else {
+ text = gettext('No multipath maps on this node.');
+ }
+ me.setEmptyText(text);
+ });
+
+ me.on('activate', () => rstore.startUpdate());
+ me.on('deactivate', () => rstore.stopUpdate());
+ me.on('destroy', () => rstore.stopUpdate());
+ },
+});
--
2.47.3
^ permalink raw reply related [flat|nested] 14+ messages in thread