From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id C57DB1FF0E3 for ; Wed, 18 Nov 2026 21:32:28 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id C76E621BCD; Wed, 18 Nov 2026 21:28:09 +0100 (CET) From: Thomas Lamprecht To: pve-devel@lists.proxmox.com Subject: [PATCH v7 storage 13/43] multipath: publish per-node status Date: Wed, 18 Nov 2026 21:25:56 +0100 Message-ID: <20261118202646.3017286-14-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261118202646.3017286-1-t.lamprecht@proxmox.com> References: <20261118202646.3017286-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1795033611821 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.164 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: M62YXMSYHRTH35KUH7Q6RGGVDRHU3CSY X-Message-ID-Hash: M62YXMSYHRTH35KUH7Q6RGGVDRHU3CSY X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Path health differs between nodes. Add the function for the status daemon to call on each node. It applies the multipath configuration and publishes the state of the maps of the node in the cluster key-value store. Besides the maps, a report has the iSCSI initiator name, which an admin needs for the access rules on the array, and the error of a failed configuration run. If multipath is not usable, the report has the reason instead of the maps. A node can have hundreds of maps that something else set up, for example a storage plugin with one LUN per guest disk. But a value of the key-value store is limited to 32 KiB. So a map only gets a row if the node manages it, if it is the base volume of an LVM storage or if it holds the volume group of a storage. The other maps are only counted. A LUN that the node requests for a storage, but that has no map, gets a row as missing, unless the node only added its WWID in this pass and the reload has yet to create the map. A report that is still too large drops the list of maps and reports an error instead. An unchanged report is sent again once a minute, so that readers can tell a stale one. The volume groups come from the hint file of LVM, where a volume group on a partition of a map counts for the map. LVM leaves the file empty while it sees duplicate PVs. A node that leaves maps out then says so in its report, as it cannot tell whether one of them holds the volume group of a storage. The file can also be outdated. After the LVM hints are turned off, it stays until the node reboots or someone removes it. And it lacks a physical volume that another node added to a shared volume group until LVM scans the devices of the node again. Signed-off-by: Thomas Lamprecht --- src/PVE/Makefile | 1 + src/PVE/Multipath/Status.pm | 112 ++++++++++++++ src/PVE/Storage/ISCSIPlugin.pm | 16 ++ src/PVE/Storage/LVMPlugin.pm | 24 ++- src/test/run_multipath_api_tests.pl | 223 ++++++++++++++++++++++++++++ src/test/run_san_library_tests.pl | 55 +++++++ 6 files changed, 430 insertions(+), 1 deletion(-) create mode 100644 src/PVE/Multipath/Status.pm diff --git a/src/PVE/Makefile b/src/PVE/Makefile index 87d5c41f..13fecbf8 100644 --- a/src/PVE/Makefile +++ b/src/PVE/Makefile @@ -9,6 +9,7 @@ install: install -D -m 0644 Multipath/Config.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Config.pm install -D -m 0644 Multipath/Generator.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Generator.pm install -D -m 0644 Multipath/Identity.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Identity.pm + install -D -m 0644 Multipath/Status.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Status.pm install -D -m 0644 CephConfig.pm ${DESTDIR}${PERLDIR}/PVE/CephConfig.pm install -D -m 0644 GuestImport.pm ${DESTDIR}${PERLDIR}/PVE/GuestImport.pm make -C Storage install diff --git a/src/PVE/Multipath/Status.pm b/src/PVE/Multipath/Status.pm new file mode 100644 index 00000000..2dcd430b --- /dev/null +++ b/src/PVE/Multipath/Status.pm @@ -0,0 +1,112 @@ +package PVE::Multipath::Status; + +use v5.36; + +use JSON qw(decode_json); + +use PVE::Cluster; +use PVE::INotify; + +use PVE::Multipath; +use PVE::Multipath::Generator; + +my $json = JSON->new->utf8->canonical; +my $KEY = 'multipath'; + +sub publish($report) { + my $node = PVE::INotify::nodename(); + my $now = time(); + my $data = { %$report, time => $now }; + my $encoded = $json->encode($data); + if (length($encoded) >= 32 * 1024) { + $data = { %$data, error => 'too many LUNs to report' }; + delete $data->{luns}; + $encoded = $json->encode($data); + } + my $raw = PVE::Cluster::get_node_kv($KEY, $node); + my $previous = eval { decode_json($raw->{$node} // '') }; + if (ref($previous) eq 'HASH') { + my $age = $now - ($previous->{time} // 0); + $previous->{time} = $now; + return if $age >= 0 && $age < 60 && $json->encode($previous) eq $encoded; + } + PVE::Cluster::broadcast_node_kv($KEY, $encoded); +} + +sub update_node_status() { + my $apply = PVE::Multipath::Generator::sync(); + my $report = { luns => [] }; + $report->{'apply-error'} = $apply->{error} if defined($apply->{error}); + require PVE::Storage::ISCSIPlugin; + my $initiator = eval { PVE::Storage::ISCSIPlugin::read_initiator_info() }; + $report->{'iscsi-initiator-name'} = $initiator->{name} + if $initiator && defined($initiator->{name}); + eval { + die "multipath-tools is not installed\n" if !PVE::Multipath::is_supported(); + die "multipathd is not running or does not respond\n" if !PVE::Multipath::is_running(); + require PVE::Storage::LVMPlugin; + my $pv_vgs = PVE::Storage::LVMPlugin::cached_pv_volume_groups(); + # A node can have many maps that something else set up, for example a storage plugin with + # one LUN per guest disk. Only the maps that the node manages, that are the base volume of a + # storage or hold its volume group get a row, the others are counted. + require PVE::Storage; + my $wanted = $apply->{wwids}; + my $storages = eval { PVE::Storage::config()->{ids} }; + my (%storage_vgs, %base); + for my $scfg (values %{ $storages // {} }) { + $storage_vgs{ $scfg->{vgname} } = 1 if defined($scfg->{vgname}); + $base{$1} = 1 + if ($scfg->{base} // '') =~ m/(?:^|\.)(?:dm-uuid-(?:part\d+-)?mpath|scsi)-(\S+)$/; + } + # without the wanted WWIDs or the storages nothing tells the maps apart + my $all = !$wanted || !$storages; + my %mapped; + for my $map (PVE::Multipath::get_maps()->@*) { + my $wwid = $map->{wwid} // next; + $mapped{$wwid} = 1; + my $vg = defined($map->{sysfs}) ? ($pv_vgs // {})->{"/dev/$map->{sysfs}"} : undef; + if (!$all && !$wanted->{$wwid} && !$base{$wwid} && !$storage_vgs{ $vg // '' }) { + $report->{'other-maps'}++; + next; + } + my $info = ($wanted // {})->{$wwid} // {}; + my $lun = { + wwid => $wwid, + 'paths-total' => $map->{'paths-total'}, + 'paths-active' => $map->{'paths-active'}, + state => PVE::Multipath::derive_health( + $map->{'paths-active'}, + $map->{'paths-total'}, + $info->{'paths-expected'}, + ), + }; + $lun->{size} = $map->{size} if defined($map->{size}); + $lun->{name} = $map->{name} if defined($map->{name}) && $map->{name} ne $wwid; + $lun->{storages} = $info->{storages} if $info->{storages}; + $lun->{vgname} = $vg if defined($vg); + push $report->{luns}->@*, $lun; + } + # The cluster status has no other source for a LUN of a storage that no node assembled. A + # WWID added in this pass gets its map from the reload that follows. + my %added = map { $_ => 1 } ($apply->{added} // [])->@*; + for my $wwid (sort keys %{ $wanted // {} }) { + my $info = $wanted->{$wwid}; + next if $mapped{$wwid} || $added{$wwid} || !$info->{storages}; + my $lun = { wwid => $wwid, state => 'missing', storages => $info->{storages} }; + $lun->{'paths-expected'} = $info->{'paths-expected'} + if defined($info->{'paths-expected'}); + push $report->{luns}->@*, $lun; + } + # the volume groups come from the hint file of LVM, which it does not always have + $report->{'vgs-unknown'} = 1 if !$pv_vgs && $report->{'other-maps'}; + }; + if (my $error = $@) { + chomp $error; + $report->{error} = $error; + delete $report->{luns}; + } + eval { publish($report) }; + warn "multipath: cannot publish node status - $@" if $@; +} + +1; diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm index 08c766df..5cc0ad7e 100644 --- a/src/PVE/Storage/ISCSIPlugin.pm +++ b/src/PVE/Storage/ISCSIPlugin.pm @@ -21,8 +21,24 @@ use base qw(PVE::Storage::Plugin); # iscsi helper function my $ISCSIADM = '/usr/bin/iscsiadm'; +our $INITIATOR_NAME_FILE = '/etc/iscsi/initiatorname.iscsi'; our $SYSFS = '/sys'; +sub read_initiator_info { + my $content = eval { PVE::Tools::file_get_contents($INITIATOR_NAME_FILE, 64 * 1024) }; + return undef if !defined($content); + + my $res = {}; + for my $line (split(/\n/, $content)) { + if ($line =~ m/^\s*InitiatorName\s*=\s*(\S+)\s*$/) { + $res->{name} = $1; + } elsif ($line =~ m/^\s*InitiatorAlias\s*=\s*(.+?)\s*$/) { + $res->{alias} = $1; + } + } + return defined($res->{name}) ? $res : undef; +} + my $found_iscsi_adm_exe; my sub assert_iscsi_support { my ($noerr) = @_; diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm index 966a70d3..bc15f2b3 100644 --- a/src/PVE/Storage/LVMPlugin.pm +++ b/src/PVE/Storage/LVMPlugin.pm @@ -19,7 +19,7 @@ use PVE::INotify; use PVE::JSONSchema qw(get_standard_option); use PVE::Multipath; use PVE::RESTEnvironment qw(log_warn); -use PVE::Tools qw(run_command file_read_firstline trim); +use PVE::Tools qw(run_command file_get_contents file_read_firstline trim); use PVE::Storage::Common; use PVE::Storage::Plugin; @@ -140,6 +140,28 @@ sub lvm_pv_info { return $pvinfo; } +sub cached_pv_volume_groups { + my $data = eval { file_get_contents('/run/lvm/hints') } // ''; + # LVM leaves the file without content when devices change and while it sees duplicate PVs + return undef if $data !~ m/^hints_version: 2\./m; + my $res = {}; + while ($data =~ m/^scan:(\/dev\/\S+) pvid:\S+ devn:\d+:\d+ vg:([a-zA-Z0-9+_.-]+)$/mg) { + next if $2 eq '-'; + my ($device, $vg) = (abs_path($1) // $1, $2); + $res->{$device} = $vg; + # a volume group on a partition of a multipath map counts for the map as well + my $sysdir = '/sys/class/block/' . basename($device); + my $map = PVE::Multipath::parse_dm_uuid(file_read_firstline("$sysdir/dm/uuid")); + next if !$map || !$map->{partition}; + PVE::Tools::dir_glob_foreach( + "$sysdir/slaves", + 'dm-\d+', + sub { $res->{"/dev/$_[0]"} //= $vg }, + ); + } + return $res; +} + sub clear_first_sector { my ($dev) = shift; diff --git a/src/test/run_multipath_api_tests.pl b/src/test/run_multipath_api_tests.pl index 0ee83dfb..37588f99 100755 --- a/src/test/run_multipath_api_tests.pl +++ b/src/test/run_multipath_api_tests.pl @@ -9,9 +9,11 @@ BEGIN { } use lib ('.', '..'); use File::Temp qw(tempdir); +use JSON qw(encode_json decode_json); use Test::More; use Test::MockModule; use PVE::Multipath::Generator; +use PVE::Storage; subtest 'add-only generation and failed reload throttle' => sub { my $dir = tempdir('multipath-test-XXXX', DIR => '.', CLEANUP => 1); @@ -139,4 +141,225 @@ subtest 'storage LUNs are the identified paths of the storage target' => sub { ); }; +subtest 'an unconfigured node does not write drop-ins or reload' => sub { + my $dir = tempdir('multipath-empty-XXXX', DIR => '.', CLEANUP => 1); + local $PVE::Multipath::Generator::DEFAULTS_DROPIN = "$dir/defaults"; + local $PVE::Multipath::Generator::OVERRIDES_DROPIN = "$dir/overrides"; + local $PVE::Multipath::Generator::ALIASES_DROPIN = "$dir/aliases"; + local $PVE::Multipath::Generator::RECONFIGURE_PENDING = "$dir/pending"; + my $config = Test::MockModule->new('PVE::Multipath::Config'); + $config->mock(read_config => sub { { ids => {} } }); + $config->mock(read_overrides => sub { '' }); + my $storage = Test::MockModule->new('PVE::Storage'); + my $identity = Test::MockModule->new('PVE::Multipath::Identity'); + $identity->mock(collect => sub { die "unexpected identity collection\n" }); + my $mp = Test::MockModule->new('PVE::Multipath'); + $mp->mock(is_supported => sub { 1 }); + $mp->mock(is_running => sub { 1 }); + $mp->mock(get_maps => sub { [] }); + $mp->mock(list_etc_multipath_wwids => sub { [] }); + my $generator = Test::MockModule->new('PVE::Multipath::Generator'); + my @changes; + + for my $function (qw(file_set_contents make_path)) { + $generator->mock($function => sub { push @changes, $function }); + } + for my $function (qw(add_wwid reconfigure)) { + $mp->mock($function => sub { push @changes, $function }); + } + my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin'); + $iscsi->mock(read_initiator_info => sub { {} }); + my $status = Test::MockModule->new('PVE::Multipath::Status'); + my $report; + $status->mock(publish => sub { $report = $_[0] }); + for my $ids ({}, { plain => { type => 'iscsi', target => 'iqn.test' } }) { + $storage->mock(config => sub { { ids => $ids } }); + PVE::Multipath::Status::update_node_status(); + ok(!exists($report->{error}), 'running daemon reports successfully'); + ok(!exists($report->{'apply-error'}), 'unconfigured application succeeds'); + is_deeply(\@changes, [], 'no write, mkdir, allow-list addition or reload'); + is_deeply([glob("$dir/*")], [], 'no generated files or pending marker'); + } +}; + +subtest 'node reports use read-back, refresh and a bounded error report' => sub { + my $cluster = Test::MockModule->new('PVE::Cluster'); + my $node = Test::MockModule->new('PVE::INotify'); + $node->mock(nodename => sub { 'n1' }); + my (%raw, @sent); + $cluster->mock(get_node_kv => sub { \%raw }); + $cluster->mock( + broadcast_node_kv => sub($key, $data) { + is($key, 'multipath', 'one node key'); + push @sent, decode_json($data); + $raw{n1} = $data; + }, + ); + my $report = { + luns => [{ + wwid => '3600a', + name => 'data', + state => 'optimal', + 'paths-total' => 2, + 'paths-active' => 2, + }], + }; + PVE::Multipath::Status::publish($report); + $NOW += 59; + PVE::Multipath::Status::publish($report); + is(scalar(@sent), 1, 'same report younger than 60 seconds is skipped'); + $NOW++; + PVE::Multipath::Status::publish($report); + is(scalar(@sent), 2, 'report refreshed at 60 seconds'); + %raw = (); + PVE::Multipath::Status::publish($report); + is(scalar(@sent), 3, 'lost pmxcfs value is republished without local send state'); + PVE::Multipath::Status::publish( + { luns => [('x' x 33000)], 'iscsi-initiator-name' => 'iqn.2026-11.test:n1' }); + is($sent[-1]->{error}, 'too many LUNs to report', 'oversized report is an error'); + ok(!exists($sent[-1]->{luns}), 'not an empty or partial LUN report'); + is( + $sent[-1]->{'iscsi-initiator-name'}, + 'iqn.2026-11.test:n1', + 'which keeps the node identity', + ); +}; + +subtest 'only managed maps and those under a storage get a row' => sub { + my $gen = Test::MockModule->new('PVE::Multipath::Generator'); + my $status = Test::MockModule->new('PVE::Multipath::Status'); + my $mp = Test::MockModule->new('PVE::Multipath'); + my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin'); + my $storage = Test::MockModule->new('PVE::Storage'); + my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin'); + my $report; + $status->mock(publish => sub { $report = $_[0] }); + $iscsi->mock(read_initiator_info => sub { undef }); + $mp->mock(is_supported => sub { 1 }, is_running => sub { 1 }); + $mp->mock( + get_maps => sub { + return [ + map { { + wwid => "3600$_", + sysfs => "dm-$_", + 'paths-active' => 2, + 'paths-total' => 2, + } } 1 .. 5 + ]; + }, + ); + $gen->mock(sync => sub { { wwids => { '36001' => {} } } }); + $lvm->mock( + cached_pv_volume_groups => sub { { '/dev/dm-2' => 'vgsan', '/dev/dm-3' => 'vgother' } }, + ); + my $ids = { + san => { type => 'lvm', vgname => 'vgsan' }, + based => + { type => 'lvm', vgname => 'vgbase', base => 'conn:0.0.0.dm-uuid-mpath-36004' }, + }; + $storage->mock(config => sub { { ids => $ids } }); + PVE::Multipath::Status::update_node_status(); + is_deeply( + [map { $_->{wwid} } $report->{luns}->@*], + ['36001', '36002', '36004'], + 'a wanted map, one with the volume group of a storage and a base volume', + ); + is($report->{'other-maps'}, 2, 'the other maps are counted'); + ok(!exists($report->{'vgs-unknown'}), 'volume groups known from the hints of LVM'); + { + local $ids->{legacy} = + { type => 'lvm', vgname => 'vglegacy', base => 'conn:0.0.1.scsi-36005' }; + PVE::Multipath::Status::update_node_status(); + ok( + grep({ $_->{wwid} eq '36005' } $report->{luns}->@*), + 'and a base volume by its SCSI name', + ); + } + $lvm->mock(cached_pv_volume_groups => sub { undef }); + PVE::Multipath::Status::update_node_status(); + is_deeply( + [map { $_->{wwid} } $report->{luns}->@*], + ['36001', '36004'], + 'without hints only the maps that the configuration names', + ); + is($report->{'other-maps'}, 3, 'the map of the volume group counts as other'); + ok($report->{'vgs-unknown'}, 'and the report says that volume groups are unknown'); + $storage->mock(config => sub { die "no configuration\n" }); + PVE::Multipath::Status::update_node_status(); + is(scalar($report->{luns}->@*), 5, 'all maps without the storage configuration'); + ok(!exists($report->{'other-maps'}), 'and no count'); + $storage->mock(config => sub { { ids => $ids } }); + $mp->mock(get_maps => sub { [] }); + PVE::Multipath::Status::update_node_status(); + ok(!exists($report->{'vgs-unknown'}), + 'unknown volume groups only matter for left out maps'); +}; + +subtest 'a LUN of a storage without a map is missing' => sub { + my $gen = Test::MockModule->new('PVE::Multipath::Generator'); + my $status = Test::MockModule->new('PVE::Multipath::Status'); + my $mp = Test::MockModule->new('PVE::Multipath'); + my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin'); + my $storage = Test::MockModule->new('PVE::Storage'); + my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin'); + my %raw; + $status->mock(publish => sub { $raw{n1} = encode_json({ $_[0]->%*, time => $NOW }) }); + $iscsi->mock(read_initiator_info => sub { undef }); + $mp->mock(is_supported => sub { 1 }, is_running => sub { 1 }, get_maps => sub { [] }); + $lvm->mock(cached_pv_volume_groups => sub { {} }); + $gen->mock( + sync => sub { + return { + wwids => { + '3600a' => { storages => ['san'], 'paths-expected' => 2 }, + '3600b' => {}, + '3600c' => { storages => ['san'] }, + }, + added => ['3600c'], + }; + }, + ); + $storage->mock( + config => sub { { ids => { san => { type => 'iscsi', 'managed-multipath' => 1 } } } }); + PVE::Multipath::Status::update_node_status(); + is_deeply( + decode_json($raw{n1})->{luns}, + [{ wwid => '3600a', state => 'missing', storages => ['san'], 'paths-expected' => 2 }], + 'the LUN of the storage, not one of an entry, nor one added in this pass', + ); +}; + +subtest 'one entry point applies configuration and reports unavailable reasons' => sub { + my $gen = Test::MockModule->new('PVE::Multipath::Generator'); + my $status = Test::MockModule->new('PVE::Multipath::Status'); + my $mp = Test::MockModule->new('PVE::Multipath'); + my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin'); + my ($report, @order); + $gen->mock(sync => sub { push @order, 'apply'; { error => 'reload failed' } }); + $status->mock(publish => sub { push @order, 'publish'; $report = $_[0] }); + $iscsi->mock(read_initiator_info => sub { { name => 'iqn.2025-01.example:node' } }); + $mp->mock(is_supported => sub { 0 }); + PVE::Multipath::Status::update_node_status(); + is($report->{error}, 'multipath-tools is not installed', 'missing tools reason only'); + is_deeply(\@order, [qw(apply publish)], 'single call owns ordering'); + is($report->{'apply-error'}, 'reload failed', 'apply failure is separate'); + $mp->mock(is_supported => sub { 1 }); + $mp->mock(is_running => sub { 0 }); + PVE::Multipath::Status::update_node_status(); + is( + $report->{error}, + 'multipathd is not running or does not respond', + 'stopped daemon reason only', + ); + $mp->mock(is_running => sub { 1 }); + $mp->mock(get_maps => sub { die "bad JSON\n" }); + PVE::Multipath::Status::update_node_status(); + is($report->{error}, 'bad JSON', 'query error without unavailable prefix'); + $mp->mock(get_maps => + sub { [{ wwid => '3600a', name => 'data', 'paths-active' => 1, 'paths-total' => 2 }] }); + PVE::Multipath::Status::update_node_status(); + is($report->{luns}->[0]->{state}, 'degraded', 'all maps while the wanted ones are unknown'); + ok(!exists($report->{supported}), 'no separate capability state'); +}; + done_testing(); diff --git a/src/test/run_san_library_tests.pl b/src/test/run_san_library_tests.pl index 2ec2a651..f7ae68d1 100755 --- a/src/test/run_san_library_tests.pl +++ b/src/test/run_san_library_tests.pl @@ -50,4 +50,59 @@ subtest 'omitted optional parameters retain explicit undef behavior' => sub { } }; +subtest 'LVM hints are optional cached display metadata' => sub { + require PVE::Storage::LVMPlugin; + my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin'); + my $contents; + $lvm->mock( + file_get_contents => sub { + is($_[0], '/run/lvm/hints', 'only the existing LVM cache is read'); + die "No such file\n" if !defined($contents); + return $contents; + }, + ); + $lvm->mock(abs_path => sub { $_[0] eq '/dev/disk/by-id/test' ? '/dev/dm-0' : $_[0] }); + $lvm->mock(run_command => sub { die "unexpected device probe\n" }); + $lvm->redefine( + file_read_firstline => sub { $_[0] =~ m{/dm-3/dm/uuid$} ? 'part1-mpath-36001' : undef }, + ); + my $tools = Test::MockModule->new('PVE::Tools'); + $tools->redefine( + dir_glob_foreach => sub { $_[2]->('dm-2') if $_[0] eq '/sys/class/block/dm-3/slaves' }); + my $head = "# Created by vgs pid 1\nhints_version: 2.1\n"; + for my $case ( + [undef, undef, 'missing file'], + ['', undef, 'empty file'], + ["# Created empty by vgs pid 1\n", undef, 'file that LVM left without content'], + ["scan:/dev/sda pvid:abc devn:8:0 vg:data\n", undef, 'no version line'], + [ + "hints_version: 3.0\nscan:/dev/sda pvid:abc devn:8:0 vg:data\n", + undef, + 'other format version', + ], + [$head, {}, 'no PVs'], + ["${head}scan=/dev/sda pvid=abc devn=8:0 vg=data\n", {}, 'different format'], + ["${head}scan:/dev/sda pvid:abc devn:8:0 vg:-\n", {}, 'orphan PV'], + ["${head}scan:/dev/sda pvid:abc devn:8:0 vg:[unknown]\n", {}, 'unknown VG'], + [ + "${head}scan:/dev/sda pvid:abc devn:8:0 vg:data\n", + { '/dev/sda' => 'data' }, + 'cached VG', + ], + [ + "${head}scan:/dev/disk/by-id/test pvid:abc devn:253:0 vg:data\n", + { '/dev/dm-0' => 'data' }, + 'resolved device name', + ], + [ + "${head}scan:/dev/dm-3 pvid:abc devn:253:3 vg:data\n", + { '/dev/dm-3' => 'data', '/dev/dm-2' => 'data' }, + 'partition of a multipath map', + ], + ) { + $contents = $case->[0]; + is_deeply(PVE::Storage::LVMPlugin::cached_pv_volume_groups(), $case->[1], $case->[2]); + } +}; + done_testing();