* [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup
@ 2026-11-05 23:30 Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 proxmox 01/76] add proxmox-multipath crate Thomas Lamprecht
` (75 more replies)
0 siblings, 76 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
This v3 combines the cluster-wide multipath series [0][1] with
Dietmar's RFC for guided remote storage setup and SAN visibility [2].
What it adds:
- cluster-wide dm-multipath management through /etc/pve/multipath.cfg:
iSCSI storages with the new 'multipath' option supply their LUNs;
explicit entries cover individual LUNs, for example Fibre Channel.
pvestatd applies the configuration on each node without removing
allow-list entries added by hand or for boot from SAN. Array-specific
settings go in /etc/pve/multipath-overrides.conf.
- SAN views: Datacenter > SAN shows node status and initiator names
for target access rules. Its Multipath view groups path health per
LUN, including expected paths, report age, and unresolved identities.
Node > SAN lists LUNs, paths, usage, and iSCSI targets and sessions,
with actions to enable multipath or wipe a LUN as root@pam.
- guided setup for NFS, SMB/CIFS, iSCSI, Fibre Channel, and ZFS over
iSCSI. For shared LVM over iSCSI, it creates or reuses the connection
storage, waits for LUN discovery, and creates a volume group on the
selected LUN, using its map when multipath is enabled. Partial setups
retain their settings and retry only missing steps; unanswered
requests are checked before retrying.
- guards for destructive actions: SAN wipes and volume-group creation
check cluster storage references, local users, and device identities.
Before a wipe, the web UI also asks the other nodes for users of the
LUN. Wipes keep the checked device references and recheck identities
between writes. Inconclusive checks refuse the operation. Session
logout is refused if it would remove the last usable path of a LUN in
use. Cluster locks do not cover hosts outside the cluster.
Patches 04-11 are independent fixes and could go in first, with their
documentation and pvestatd parts in patches 43 and 46: status-only
activation and bounded LVM queries, shared VGs missed by LVM hints,
per-target rescans, LUNs listed per iSCSI session, validation before
volume-group creation, and a target check that refuses composite
devices. The smaller fixes for SCSI padding, iSCSI discovery errors,
iSER and offload sessions, LVM base volumes and the iSCSI edit dialog
are already applied.
Changes since v2:
- combined with Dietmar's RFC; multipath is an iSCSI storage option
rather than a separate per-LUN storage type
- Node > SAN offers Enable Multipath; the datacenter view uses one
row per LUN with expandable node details, not a column per node
(Dominik)
- addressed Dominik's API and code remarks: add_wwid accepts alias and
settings, alias changes use the update endpoint, config helpers are
renamed, group_by_tpg is available, and the overrides example is
fixed
- preserved existing allow-list entries, addressing Maximiliano's
migration concern without a warning on first use
- added expected path counts, stalled-node detection, withdrawn LUN
status, unresolved identities, and duplicate-PV hints for clones
- added LVM and wipe guards, bounded status commands, the node iSCSI
API, and pvenode commands
- added an admin guide chapter, including removal, growth, and clone
procedures tested on real hardware
- moved the parsing of multipathd map status and the health derivation
to a new proxmox-multipath crate, used through pve-rs bindings
Testing used a five-node nested cluster attached to a NetApp AFF A20
through two independent paths over iSCSI and NVMe/TCP, plus host-side
Linux LIO and nvmet targets. Scenarios covered path, LIF, and
whole-array outages with guests running; LUN unmapping with queued I/O;
revoked node access; online growth; snapshot clones; reboot; and adding
30 LUNs at once. Wizard and wipe tests covered lost responses, stuck
tasks, denied logins, and LVM metadata written by other nodes. The
screens were also reviewed from four user perspectives. The bounded
status commands and the wizard's LUN identity check also have focused
regression tests. Fibre Channel and offload iSCSI initiators, which can
run several sessions on one SCSI host, were not tested on hardware.
Open points:
- path-redundancy loss is visible in the web UI and API but sends no
notification (can be more easily done once Lukas' notification
matcher rework has landed)
- an optimal LUN is marked incomplete while its state on an expected
node is unknown, but there is no count of the current reports per LUN
- LUN removal is documented, not guided - can be added later.
- if thin_check takes longer than the shared status-command budget,
that status pass reports no LVM or LVM-thin status. Later passes
retry the activation; starting a guest activates the pool without
this budget
- pvcreate and vgcreate receive a checked, identity-bearing by-id name,
but resolve it themselves, leaving a short race after the final check
- integration with the per-node target and portal mapping work [3][4]
Apply in patch order. The pve-rs bindings need the proxmox-multipath
crate packaged first. pve-storage needs pve-cluster with the multipath
configuration files registered and libpve-rs-perl with the bindings.
pve-manager needs the matching pve-storage APIs and status-command
helper, plus the pve-docs and pve-doc-generator changes.
Feature-specific dependency bumps are deferred until the providing
package versions are known and must be added before shipping. This
includes a Breaks of libpve-storage-perl on older pve-manager, whose
Ceph OSD creation would accept the iSCSI LUNs that the disk list now
includes (patch 45).
[0]: https://lore.proxmox.com/pve-devel/20260626121000.2095591-1-t.lamprecht@proxmox.com/
[1]: https://lore.proxmox.com/pve-devel/20260703124707.1172980-2-t.lamprecht@proxmox.com/
[2]: https://lore.proxmox.com/pve-devel/20260731102156.3947857-1-dietmar@proxmox.com/
[3]: https://lore.proxmox.com/pve-devel/20260430173220.441001-1-m.limbeck@proxmox.com/
[4]: https://lore.proxmox.com/pve-devel/20260804090819.2136483-1-dietmar@proxmox.com/
proxmox:
Thomas Lamprecht (1):
add proxmox-multipath crate
Cargo.toml | 2 +
proxmox-multipath/Cargo.toml | 17 ++
proxmox-multipath/debian/changelog | 7 +
proxmox-multipath/debian/control | 38 +++
proxmox-multipath/debian/copyright | 18 ++
proxmox-multipath/debian/debcargo.toml | 7 +
proxmox-multipath/src/lib.rs | 531 +++++++++++++++++++++++++++++++++
7 files changed, 620 insertions(+)
proxmox-perl-rs:
Thomas Lamprecht (1):
pve-rs: add PVE::RS::Multipath bindings for multipathd status parsing
pve-rs/Cargo.toml | 1 +
pve-rs/Makefile | 2 ++
pve-rs/debian/control | 1 +
pve-rs/src/bindings/mod.rs | 3 ++
pve-rs/src/bindings/multipath.rs | 40 ++++++++++++++++++++++
pve-rs/test/Makefile | 1 +
pve-rs/test/multipath.pl | 74 ++++++++++++++++++++++++++++++++++++++++
7 files changed, 122 insertions(+)
pve-cluster:
Thomas Lamprecht (1):
cfs: observe the multipath configuration files
src/PVE/Cluster.pm | 2 ++
src/pmxcfs/status.c | 2 ++
2 files changed, 4 insertions(+)
pve-storage:
Dietmar Maurer (12):
diskmanage: collect disk transport type from lsblk
diskmanage: add helper to list multipath devices
diskmanage: qualify NVMe over fabrics transport
disks: list: add include-remote parameter
diskmanage: include iSCSI session devices in disk enumeration
diskmanage: link multipath member disks to their map device
iscsi: factor out session device map from device list
api: scan: add a SAN LUN inventory
pvesm: expose the SAN LUN inventory
diskmanage: collect native NVMe multipath path state
api: scan: san-luns: report path state
api: add node-level iSCSI status and maintenance
Thomas Lamprecht (27):
storage: distinguish status-only activation in plugins
lvm: find a shared volume group created by another node
storage: bound LVM status commands
iscsi: rate-limit session rescans per target
iscsi: list the LUNs of each session, not of its SCSI host
lvm: refuse to create a volume group on a device that holds data
lvm: refuse a base volume of a target that serves disk images
iscsi: refuse virtual devices other than multipath maps as volumes
disks: lvm: allow volume groups on SAN LUNs and multipath maps
multipath: add a library around multipathd
diskmanage: collect local block device users
api: iscsi: add session logout with last-path protection
multipath: add a cluster-wide configuration
iscsi: add an option to manage storage LUNs with multipath
lvm: activate a missing base volume before using the volume group
multipath: apply the configuration on every node
iscsi: share the initiator identity reader
multipath: publish per-node status
api: add endpoints for the cluster-wide multipath configuration
api: multipath: add the cluster-wide health status
multipath: collect identities of non-iSCSI SCSI paths
api: scan: san-luns: report the multipath state of each LUN
api: scan: flag duplicate physical volumes on distinct LUNs
api: disks: recheck local users and retain wipe device references
iscsi: name plain LUNs after their identity link
api: disks: guard SAN wipes with stable identities and storage checks
api: scan: refresh SAN usage with bounded probes
ApiChangeLog | 24 +
debian/control | 2 +
src/PVE/API2/Disks.pm | 76 +-
src/PVE/API2/Disks/LVM.pm | 74 +-
src/PVE/API2/ISCSI.pm | 423 +++++++
src/PVE/API2/Makefile | 2 +
src/PVE/API2/Multipath.pm | 689 +++++++++++
src/PVE/API2/Storage/Scan.pm | 247 ++++
src/PVE/CLI/pvesm.pm | 13 +
src/PVE/Diskmanage.pm | 1007 +++++++++++++++-
src/PVE/Makefile | 6 +
src/PVE/Multipath.pm | 375 ++++++
src/PVE/Multipath/ClusterConfig.pm | 64 +
src/PVE/Multipath/Config.pm | 451 +++++++
src/PVE/Multipath/Generator.pm | 314 +++++
src/PVE/Multipath/Identity.pm | 402 +++++++
src/PVE/Multipath/Status.pm | 486 ++++++++
src/PVE/Storage.pm | 272 ++++-
src/PVE/Storage/ISCSIPlugin.pm | 695 +++++++++--
src/PVE/Storage/LVMPlugin.pm | 323 ++++-
src/PVE/Storage/LvmThinPlugin.pm | 39 +-
src/PVE/Storage/Makefile | 4 +-
src/PVE/Storage/Plugin.pm | 2 +-
src/PVE/Storage/SANScan.pm | 429 +++++++
src/PVE/Storage/StatusCommand.pm | 241 ++++
src/test/Makefile | 61 +-
src/test/disk_api_test.pm | 194 +++
src/test/disk_tests/iscsi/disklist | 1 +
src/test/disk_tests/iscsi/disklist_expected.json | 19 +
src/test/disk_tests/iscsi/lsblk | 5 +
src/test/disk_tests/iscsi/sda/device/model | 1 +
src/test/disk_tests/iscsi/sda/device/vendor | 1 +
src/test/disk_tests/iscsi/sda/queue/rotational | 1 +
src/test/disk_tests/iscsi/sda/size | 1 +
src/test/disk_tests/iscsi/sda_udevadm | 21 +
src/test/disk_tests/multipath/disklist | 11 +
.../disk_tests/multipath/disklist_expected.json | 111 ++
src/test/disk_tests/multipath/dm-0/dm/name | 1 +
src/test/disk_tests/multipath/dm-0/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-0/size | 1 +
src/test/disk_tests/multipath/dm-0_slaves | 4 +
src/test/disk_tests/multipath/dm-1/dm/name | 1 +
src/test/disk_tests/multipath/dm-1/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-1/size | 1 +
src/test/disk_tests/multipath/dm-1_slaves | 4 +
src/test/disk_tests/multipath/dm-2/dm/name | 1 +
src/test/disk_tests/multipath/dm-2/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-2/size | 1 +
src/test/disk_tests/multipath/dm-2_slaves | 1 +
src/test/disk_tests/multipath/dm-3/dm/name | 1 +
src/test/disk_tests/multipath/dm-3/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-3/size | 1 +
src/test/disk_tests/multipath/dm-3_slaves | 1 +
src/test/disk_tests/multipath/dm-4/dm/name | 1 +
src/test/disk_tests/multipath/dm-4/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-4/size | 1 +
src/test/disk_tests/multipath/dm-4_slaves | 1 +
src/test/disk_tests/multipath/lsblk | 77 ++
src/test/disk_tests/multipath/lvs | 0
src/test/disk_tests/multipath/mounts | 1 +
.../disk_tests/multipath/multipath_expected.json | 59 +
src/test/disk_tests/multipath/multipathd | 124 ++
.../multipath/multipathd_status_expected.json | 42 +
.../nvme-subsystem/nvme-subsys0/nvme3/address | 1 +
.../nvme-subsys0/nvme3/nvme2c3n1/ana_state | 1 +
.../nvme-subsystem/nvme-subsys0/nvme3/state | 1 +
.../nvme-subsystem/nvme-subsys0/nvme4/address | 1 +
.../nvme-subsys0/nvme4/nvme2c4n1/ana_state | 1 +
.../nvme-subsystem/nvme-subsys0/nvme4/state | 1 +
.../nvme-subsystem/nvme-subsys1/nvme0/address | 1 +
.../nvme-subsystem/nvme-subsys1/nvme0/state | 1 +
src/test/disk_tests/multipath/nvme8n1/device/model | 1 +
.../disk_tests/multipath/nvme8n1/device/transport | 1 +
.../disk_tests/multipath/nvme8n1/queue/rotational | 1 +
src/test/disk_tests/multipath/nvme8n1/size | 1 +
src/test/disk_tests/multipath/nvme8n1_udevadm | 14 +
src/test/disk_tests/multipath/nvme9n1/device/model | 1 +
.../disk_tests/multipath/nvme9n1/device/transport | 1 +
.../disk_tests/multipath/nvme9n1/device/vendor | 1 +
.../disk_tests/multipath/nvme9n1/queue/rotational | 1 +
src/test/disk_tests/multipath/nvme9n1/size | 1 +
src/test/disk_tests/multipath/nvme9n1_udevadm | 12 +
.../multipath/nvme_path_status_expected.json | 19 +
src/test/disk_tests/multipath/partlist | 0
src/test/disk_tests/multipath/pvs | 1 +
src/test/disk_tests/multipath/sda/device/model | 1 +
src/test/disk_tests/multipath/sda/device/vendor | 1 +
src/test/disk_tests/multipath/sda/holders/holder | 1 +
src/test/disk_tests/multipath/sda/queue/rotational | 1 +
src/test/disk_tests/multipath/sda/size | 1 +
src/test/disk_tests/multipath/sda_udevadm | 18 +
src/test/disk_tests/multipath/sdb/device/model | 1 +
src/test/disk_tests/multipath/sdb/device/vendor | 1 +
src/test/disk_tests/multipath/sdb/holders/holder | 1 +
src/test/disk_tests/multipath/sdb/queue/rotational | 1 +
src/test/disk_tests/multipath/sdb/size | 1 +
src/test/disk_tests/multipath/sdb_udevadm | 18 +
src/test/disk_tests/multipath/sdc/device/model | 1 +
src/test/disk_tests/multipath/sdc/device/vendor | 1 +
src/test/disk_tests/multipath/sdc/holders/holder | 1 +
src/test/disk_tests/multipath/sdc/queue/rotational | 1 +
src/test/disk_tests/multipath/sdc/size | 1 +
src/test/disk_tests/multipath/sdc_udevadm | 18 +
src/test/disk_tests/multipath/sdd/device/model | 1 +
src/test/disk_tests/multipath/sdd/device/vendor | 1 +
src/test/disk_tests/multipath/sdd/holders/holder | 1 +
src/test/disk_tests/multipath/sdd/queue/rotational | 1 +
src/test/disk_tests/multipath/sdd/size | 1 +
src/test/disk_tests/multipath/sdd_udevadm | 18 +
src/test/disk_tests/multipath/zpool | 1 +
src/test/disk_tests/sas/disklist_expected.json | 1 +
src/test/disk_tests/sas/lsblk | 5 +
src/test/disklist_test.pm | 117 +-
src/test/lvm-fixtures/README.md | 20 +
.../lvm-fixtures/lab1-nomda-orphan-derived.colon | 2 +
src/test/lvm-fixtures/lab1-nomda-orphan.txt | 23 +
src/test/run_block_device_users_tests.pl | 172 +++
src/test/run_disk_api_tests.pl | 11 +
src/test/run_iscsi_activation_tests.pl | 171 +++
src/test/run_iscsi_device_list_tests.pl | 78 ++
src/test/run_iscsi_logout_tests.pl | 271 +++++
src/test/run_iscsi_rescan_tests.pl | 109 ++
src/test/run_iscsi_session_tests.pl | 15 +-
src/test/run_iscsi_volume_tests.pl | 81 +-
src/test/run_lun_identity_tests.pl | 1262 ++++++++++++++++++++
src/test/run_multipath_api_tests.pl | 693 +++++++++++
src/test/run_multipath_tests.pl | 1240 +++++++++++++++++++
src/test/run_san_api_tests.pl | 180 +++
src/test/run_san_library_tests.pl | 77 ++
src/test/run_san_lun_scan_tests.pl | 945 +++++++++++++++
src/test/run_san_lvm_create_tests.pl | 611 ++++++++++
src/test/run_san_taint_tests.pl | 80 ++
src/test/run_san_wipe_tests.pl | 673 +++++++++++
src/test/run_status_command_contention_tests.pl | 520 ++++++++
src/test/run_status_command_tests.pl | 189 +++
src/test/run_test_lvm_base_activation.pl | 130 ++
src/test/run_test_lvm_base_conflicts.pl | 62 +
src/test/run_test_lvm_base_volume.pl | 8 +-
src/test/run_test_lvm_create.pl | 243 ++++
src/test/run_test_lvm_hints.pl | 110 ++
src/test/run_test_lvm_status.pl | 220 ++++
src/test/run_wipe_partition_tests.pl | 387 ++++++
src/test/run_wipe_readiness_tests.pl | 138 +++
src/test/run_wipe_reference_tests.pl | 243 ++++
src/test/run_wipedisk_tests.pl | 110 ++
145 files changed, 16554 insertions(+), 215 deletions(-)
pve-docs:
Thomas Lamprecht (2):
storage: lvm: note the rescan after extending a shared volume group
storage: document the cluster-wide multipath management
pve-storage-iscsi.adoc | 13 +-
pve-storage-lvm.adoc | 9 +
pve-storage-multipath.adoc | 527 +++++++++++++++++++++++++++++++++++++++++++++
pvesm.adoc | 13 +-
4 files changed, 554 insertions(+), 8 deletions(-)
pve-manager:
Dietmar Maurer (10):
ui: storage: allow changing the node for NFS/CIFS scans
ui: storage: add guided remote storage wizard with NFS support
ui: storage wizard: add SMB/CIFS support
ui: storage wizard: add ZFS over iSCSI support
ui: dc: storage: add remote storage wizard entry to the add menu
ui: node: add SAN LUNs panel
ui: san luns: show multipath path state
api: nodes: expose iSCSI status and maintenance
pvenode: add iscsi commands
ui: node: san: add iSCSI target and session management
Thomas Lamprecht (22):
api: ceph: osd: keep iSCSI LUNs out of OSD creation
pvestatd: track pending storage status commands across re-exec
ui: storage: factor out the scan picker error mask
ui: storage: nfs: show scan errors and empty results
ui: storage wizard: preserve settings after partial setup
ui: storage wizard: add Fibre Channel shared LVM setup
ui: storage wizard: reconcile unanswered storage creation
pvestatd: apply the multipath configuration and report the node status
api: cluster: add the multipath endpoints
ui: dc: add a cluster-wide multipath view
ui: storage: iscsi: add the multipath option
ui: san luns: show multipath state and manage LUN entries
ui: storage wizard: add iSCSI support
ui: storage wizard: iscsi: show cluster initiator names
ui: storage wizard: iscsi: enable multipath for shared LVM
ui: node: san: wipe a LUN that no storage uses
ui: storage wizard: iscsi: reuse existing target connections
ui: storage wizard: fibre channel: enable multipath for a LUN
ui: storage wizard: let root@pam wipe a LUN with data
ui: storage wizard: do not offer LUNs in use for a new volume group
ui: san: show the local users of a LUN in its usage tooltip
ui: san: check the other nodes for users before wiping a LUN
PVE/API2/Ceph/OSD.pm | 3 +
PVE/API2/Cluster.pm | 7 +
PVE/API2/Nodes.pm | 7 +
PVE/CLI/pvenode.pm | 48 ++
PVE/Service/pvestatd.pm | 34 +
debian/control | 1 +
test/Makefile | 10 +
test/SANUtils_test.js | 204 +++++
test/StorageWizard_test.js | 128 +++
www/manager6/Makefile | 21 +
www/manager6/SANUtils.js | 579 +++++++++++++
www/manager6/Utils.js | 47 ++
www/manager6/dc/Config.js | 28 +-
www/manager6/dc/Multipath.js | 535 ++++++++++++
www/manager6/dc/MultipathWwidEdit.js | 233 ++++++
www/manager6/dc/SAN.js | 326 ++++++++
www/manager6/dc/StorageView.js | 13 +-
www/manager6/node/Config.js | 24 +
www/manager6/node/ISCSITargets.js | 615 ++++++++++++++
www/manager6/node/SANLuns.js | 771 ++++++++++++++++++
www/manager6/panel/SANNotice.js | 26 +
www/manager6/storage/CIFSEdit.js | 29 +-
www/manager6/storage/IScsiEdit.js | 38 +
www/manager6/storage/NFSEdit.js | 13 +
www/manager6/storage/wizard/CIFS.js | 118 +++
www/manager6/storage/wizard/CommonSettings.js | 75 ++
www/manager6/storage/wizard/FCRecovery.js | 242 ++++++
www/manager6/storage/wizard/FibreChannel.js | 1083 +++++++++++++++++++++++++
www/manager6/storage/wizard/IScsi.js | 707 ++++++++++++++++
www/manager6/storage/wizard/LunPicker.js | 101 +++
www/manager6/storage/wizard/LunSelect.js | 485 +++++++++++
www/manager6/storage/wizard/NFS.js | 89 ++
www/manager6/storage/wizard/NodeIdentities.js | 123 +++
www/manager6/storage/wizard/SanLvm.js | 179 ++++
www/manager6/storage/wizard/WipeStatus.js | 34 +
www/manager6/storage/wizard/Wizard.js | 970 ++++++++++++++++++++++
www/manager6/storage/wizard/ZFSOverISCSI.js | 180 ++++
www/manager6/window/SANLunWipe.js | 331 ++++++++
38 files changed, 8432 insertions(+), 25 deletions(-)
^ permalink raw reply [flat|nested] 77+ messages in thread
* [PATCH v3 proxmox 01/76] add proxmox-multipath crate
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 proxmox-perl-rs 02/76] pve-rs: add PVE::RS::Multipath bindings for multipathd status parsing Thomas Lamprecht
` (74 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
SAN scans, node status reports and iSCSI storage activation need the
same view of the multipath maps and path states that multipathd
reports. Provide one typed model keyed on the LUN WWID, with the maps,
their member paths and a health verdict. It does not query the daemon
itself, so recorded replies can test it.
The model only carries what the daemon output can tell: byte size and
the transport of non-FC paths need sysfs and stay with the caller.
State strings keep the multipathd vocabulary verbatim. An ALUA standby
path with a ghost checker state counts as usable as long as
device-mapper keeps it active. The health derivation takes an optional
expected path count, as a path the transport already removed no longer
shows in the totals and would turn a degraded two-path map into a
seemingly optimal single-path one.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Cargo.toml | 2 +
proxmox-multipath/Cargo.toml | 17 +
proxmox-multipath/debian/changelog | 7 +
proxmox-multipath/debian/control | 38 ++
proxmox-multipath/debian/copyright | 18 +
proxmox-multipath/debian/debcargo.toml | 7 +
proxmox-multipath/src/lib.rs | 531 +++++++++++++++++++++++++
7 files changed, 620 insertions(+)
create mode 100644 proxmox-multipath/Cargo.toml
create mode 100644 proxmox-multipath/debian/changelog
create mode 100644 proxmox-multipath/debian/control
create mode 100644 proxmox-multipath/debian/copyright
create mode 100644 proxmox-multipath/debian/debcargo.toml
create mode 100644 proxmox-multipath/src/lib.rs
diff --git a/Cargo.toml b/Cargo.toml
index 79b0fcbd..5c204a48 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -33,6 +33,7 @@ members = [
"proxmox-login",
"proxmox-metrics",
"proxmox-match-expression",
+ "proxmox-multipath",
"proxmox-network-api",
"proxmox-network-types",
"proxmox-node-status",
@@ -182,6 +183,7 @@ proxmox-lang = { version = "1.5", path = "proxmox-lang" }
proxmox-log = { version = "1.0.0", path = "proxmox-log" }
proxmox-login = { version = "1.0.0", path = "proxmox-login" }
proxmox-match-expression = { version = "0.1", path = "proxmox-match-expression" }
+proxmox-multipath = { version = "0.1.0", path = "proxmox-multipath" }
proxmox-network-types = { version = "1.2.0", path = "proxmox-network-types" }
proxmox-parallel-handler = { version = "1.0.0", path = "proxmox-parallel-handler" }
proxmox-pgp = { version = "1.0.0", path = "proxmox-pgp" }
diff --git a/proxmox-multipath/Cargo.toml b/proxmox-multipath/Cargo.toml
new file mode 100644
index 00000000..dc1a1e45
--- /dev/null
+++ b/proxmox-multipath/Cargo.toml
@@ -0,0 +1,17 @@
+[package]
+name = "proxmox-multipath"
+description = "device-mapper multipath maps and multipathd status parsing"
+version = "0.1.0"
+
+authors.workspace = true
+edition.workspace = true
+exclude.workspace = true
+homepage.workspace = true
+license.workspace = true
+repository.workspace = true
+rust-version.workspace = true
+
+[dependencies]
+anyhow.workspace = true
+serde = { workspace = true, features = ["derive"] }
+serde_json.workspace = true
diff --git a/proxmox-multipath/debian/changelog b/proxmox-multipath/debian/changelog
new file mode 100644
index 00000000..ff80b64e
--- /dev/null
+++ b/proxmox-multipath/debian/changelog
@@ -0,0 +1,7 @@
+rust-proxmox-multipath (0.1.0-1) trixie; urgency=medium
+
+ * initial release: normalized model of device-mapper multipath maps, a
+ parser for the JSON output of multipathd, and health derivation from the
+ path states
+
+ -- Proxmox Support Team <support@proxmox.com> Thu, 03 Sep 2026 21:30:00 +0200
diff --git a/proxmox-multipath/debian/control b/proxmox-multipath/debian/control
new file mode 100644
index 00000000..a2c1f488
--- /dev/null
+++ b/proxmox-multipath/debian/control
@@ -0,0 +1,38 @@
+Source: rust-proxmox-multipath
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.88) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-anyhow-1+default-dev <!nocheck>,
+ librust-serde-1+default-dev <!nocheck>,
+ librust-serde-1+derive-dev <!nocheck>,
+ librust-serde-json-1+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.2
+Vcs-Git: git://git.proxmox.com/git/proxmox.git
+Vcs-Browser: https://git.proxmox.com/?p=proxmox.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: proxmox-multipath
+
+Package: librust-proxmox-multipath-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-anyhow-1+default-dev,
+ librust-serde-1+default-dev,
+ librust-serde-1+derive-dev,
+ librust-serde-json-1+default-dev
+Provides:
+ librust-proxmox-multipath+default-dev (= ${binary:Version}),
+ librust-proxmox-multipath-0-dev (= ${binary:Version}),
+ librust-proxmox-multipath-0+default-dev (= ${binary:Version}),
+ librust-proxmox-multipath-0.1-dev (= ${binary:Version}),
+ librust-proxmox-multipath-0.1+default-dev (= ${binary:Version}),
+ librust-proxmox-multipath-0.1.0-dev (= ${binary:Version}),
+ librust-proxmox-multipath-0.1.0+default-dev (= ${binary:Version})
+Description: Device-mapper multipath maps and multipathd status parsing - Rust source code
+ Source code for Debianized Rust crate "proxmox-multipath"
diff --git a/proxmox-multipath/debian/copyright b/proxmox-multipath/debian/copyright
new file mode 100644
index 00000000..01138fa0
--- /dev/null
+++ b/proxmox-multipath/debian/copyright
@@ -0,0 +1,18 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+
+Files:
+ *
+Copyright: 2026 Proxmox Server Solutions GmbH <support@proxmox.com>
+License: AGPL-3.0-or-later
+ This program is free software: you can redistribute it and/or modify it under
+ the terms of the GNU Affero General Public License as published by the Free
+ Software Foundation, either version 3 of the License, or (at your option) any
+ later version.
+ .
+ This program is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+ details.
+ .
+ You should have received a copy of the GNU Affero General Public License along
+ with this program. If not, see <https://www.gnu.org/licenses/>.
diff --git a/proxmox-multipath/debian/debcargo.toml b/proxmox-multipath/debian/debcargo.toml
new file mode 100644
index 00000000..b7864cdb
--- /dev/null
+++ b/proxmox-multipath/debian/debcargo.toml
@@ -0,0 +1,7 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+
+[source]
+vcs_git = "git://git.proxmox.com/git/proxmox.git"
+vcs_browser = "https://git.proxmox.com/?p=proxmox.git"
diff --git a/proxmox-multipath/src/lib.rs b/proxmox-multipath/src/lib.rs
new file mode 100644
index 00000000..7be46b17
--- /dev/null
+++ b/proxmox-multipath/src/lib.rs
@@ -0,0 +1,531 @@
+//! Normalized model of device-mapper multipath maps and a parser for the JSON status output of
+//! `multipathd`.
+//!
+//! Everything keys on the SCSI/NAA WWID, which `multipathd` reports as the map's `uuid`. Map names
+//! (`mpathN` or configured aliases) and the `sdX` member paths are node-local and unstable; the
+//! WWID is not, so it is what a cluster-wide health view or a consumer waiting for a map binds to.
+//!
+//! The crate is pure: it turns text into typed values and derives health from path states, so it
+//! can be tested against recorded `multipathd show maps json` replies. Querying the daemon and
+//! adding live-only information (byte size, transport from sysfs) is left to the caller.
+
+#![deny(missing_docs)]
+
+use std::fmt;
+use std::str::FromStr;
+
+use anyhow::{Error, bail, format_err};
+use serde::{Deserialize, Deserializer, Serialize};
+
+/// Health of a map, derived from the states of its member paths.
+#[non_exhaustive]
+#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
+#[serde(rename_all = "lowercase")]
+pub enum Health {
+ /// Every path is active and no expected path is missing.
+ Optimal,
+ /// The map is usable, but some paths are down or missing.
+ Degraded,
+ /// No active path is left.
+ Failed,
+}
+
+impl Health {
+ /// The lowercase name of the state, as used in configuration and APIs.
+ pub fn as_str(self) -> &'static str {
+ match self {
+ Health::Optimal => "optimal",
+ Health::Degraded => "degraded",
+ Health::Failed => "failed",
+ }
+ }
+}
+
+impl fmt::Display for Health {
+ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+ f.write_str(self.as_str())
+ }
+}
+
+impl FromStr for Health {
+ type Err = Error;
+
+ fn from_str(s: &str) -> Result<Self, Self::Err> {
+ Ok(match s {
+ "optimal" => Health::Optimal,
+ "degraded" => Health::Degraded,
+ "failed" => Health::Failed,
+ other => bail!("unknown multipath health state '{other}'"),
+ })
+ }
+}
+
+/// Derive the health of a map from its path counts.
+///
+/// A path `multipathd` still lists but marks as failed lowers `active` below `total`. A path the
+/// transport already removed (a Fibre Channel remote port past its `dev_loss_tmo`, for example)
+/// is gone from `total` too, so the current counts alone cannot tell a two-path map that lost a
+/// path from a single-path map. Pass the number of paths the map should have as `expected`
+/// whenever the transport configuration knows it, and such a map stays degraded.
+pub fn derive_health(active: u64, total: u64, expected: Option<u64>) -> Health {
+ if total == 0 || active == 0 {
+ return Health::Failed;
+ }
+ if active < total {
+ return Health::Degraded;
+ }
+ if expected.is_some_and(|expected| total < expected) {
+ return Health::Degraded;
+ }
+ Health::Optimal
+}
+
+/// Transport a path is attached through.
+#[non_exhaustive]
+#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)]
+#[serde(rename_all = "lowercase")]
+pub enum Transport {
+ /// Fibre Channel.
+ Fc,
+ /// iSCSI (any of the open-iscsi transports).
+ Iscsi,
+ /// Serial Attached SCSI.
+ Sas,
+}
+
+/// One member path of a map.
+///
+/// The `*_state` fields carry the vocabulary of `multipathd` verbatim: the device-mapper state is
+/// `active`, `failed` or `undef`, the sysfs device state `running`, `offline` or `unknown`, and
+/// the checker state `ready`, `faulty`, `ghost` for an ALUA standby port, and so on. Fields that
+/// `multipathd` leaves empty or prints as `[undef]` are `None`.
+#[non_exhaustive]
+#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+pub struct Path {
+ /// Kernel block device name, for example `sdb`.
+ pub dev: String,
+ /// The state device-mapper sees.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub dm_state: Option<String>,
+ /// The state the kernel block layer sees.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub dev_state: Option<String>,
+ /// The result of the last path checker run.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub check_state: Option<String>,
+ /// Path priority.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub priority: Option<i64>,
+ /// WWPN of the host (initiator) port, Fibre Channel only.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub host_wwpn: Option<String>,
+ /// WWPN of the target port, Fibre Channel only.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub target_wwpn: Option<String>,
+ /// Host adapter the path goes through: its PCI or CCW bus ID, or the local IP address for
+ /// iSCSI.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub host_adapter: Option<String>,
+ /// The transport, as far as the daemon output alone tells it: a target WWPN means Fibre
+ /// Channel. Other transports need the sysfs topology of the path device.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub transport: Option<Transport>,
+}
+
+impl Path {
+ /// Whether device-mapper has this path active, also in a priority group not currently in use.
+ pub fn is_active(&self) -> bool {
+ self.dm_state.as_deref() == Some("active")
+ }
+}
+
+/// A priority group of a map.
+#[non_exhaustive]
+#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+pub struct PathGroup {
+ /// The group number, starting at 1.
+ pub group: u64,
+ /// The device-mapper state of the group, for example `active` or `enabled`.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub dm_state: Option<String>,
+ /// The group priority.
+ pub priority: i64,
+ /// The member paths.
+ pub paths: Vec<Path>,
+}
+
+/// A multipath map, that is one LUN with all its paths.
+#[non_exhaustive]
+#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
+#[serde(rename_all = "kebab-case")]
+pub struct Map {
+ /// The WWID of the LUN, the cluster-wide stable identity of the map.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub wwid: Option<String>,
+ /// The node-local map name (`mpathN` or the configured alias).
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub name: Option<String>,
+ /// The kernel device name of the map, for example `dm-3`.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub sysfs: Option<String>,
+ /// The device-mapper state of the map.
+ #[serde(default, skip_serializing_if = "Option::is_none")]
+ pub dm_state: Option<String>,
+ /// Path failures counted by `multipathd`; resetting the map statistics clears the count.
+ pub path_faults: u64,
+ /// Number of member paths over all groups.
+ pub paths_total: u64,
+ /// Number of member paths device-mapper has active, over all groups.
+ pub paths_active: u64,
+ /// Health derived from the path states, without an expected path count.
+ pub health: Health,
+ /// The priority groups with their paths.
+ pub path_groups: Vec<PathGroup>,
+}
+
+impl Map {
+ /// Iterate over the paths of all groups.
+ pub fn paths(&self) -> impl Iterator<Item = &Path> {
+ self.path_groups.iter().flat_map(|group| group.paths.iter())
+ }
+}
+
+/// `multipathd` renders unset string fields as the literal `[undef]` and sometimes as the empty
+/// string; neither is a value.
+fn clean(value: Option<String>) -> Option<String> {
+ value.filter(|value| !value.is_empty() && value != "[undef]")
+}
+
+/// `multipathd` prints `[unknown]` for a Fibre Channel port name it cannot read from sysfs.
+fn clean_wwpn(value: Option<String>) -> Option<String> {
+ clean(value).filter(|value| value != "[unknown]")
+}
+
+/// `multipathd` prints numbers as JSON integers; integer strings are accepted as well.
+#[derive(Deserialize)]
+#[serde(untagged)]
+enum NumberOrString {
+ Number(i64),
+ String(String),
+}
+
+fn deserialize_number<'de, D>(deserializer: D) -> Result<Option<i64>, D::Error>
+where
+ D: Deserializer<'de>,
+{
+ let value = Option::<NumberOrString>::deserialize(deserializer)?;
+ Ok(match value {
+ None => None,
+ Some(NumberOrString::Number(number)) => Some(number),
+ Some(NumberOrString::String(string)) => match string.trim() {
+ "" | "[undef]" => None,
+ trimmed => Some(trimmed.parse().map_err(serde::de::Error::custom)?),
+ },
+ })
+}
+
+#[derive(Deserialize)]
+struct RawPath {
+ dev: Option<String>,
+ dm_st: Option<String>,
+ dev_st: Option<String>,
+ chk_st: Option<String>,
+ #[serde(default, deserialize_with = "deserialize_number")]
+ pri: Option<i64>,
+ host_wwpn: Option<String>,
+ target_wwpn: Option<String>,
+ host_adapter: Option<String>,
+}
+
+#[derive(Deserialize)]
+struct RawGroup {
+ #[serde(default, deserialize_with = "deserialize_number")]
+ group: Option<i64>,
+ dm_st: Option<String>,
+ #[serde(default, deserialize_with = "deserialize_number")]
+ pri: Option<i64>,
+ #[serde(default)]
+ paths: Vec<RawPath>,
+}
+
+#[derive(Deserialize)]
+struct RawMap {
+ name: Option<String>,
+ uuid: Option<String>,
+ sysfs: Option<String>,
+ dm_st: Option<String>,
+ #[serde(default, deserialize_with = "deserialize_number")]
+ path_faults: Option<i64>,
+ #[serde(default)]
+ path_groups: Vec<RawGroup>,
+}
+
+#[derive(Deserialize)]
+struct RawOutput {
+ #[serde(default)]
+ maps: Vec<RawMap>,
+}
+
+fn is_wwpn(value: &str) -> bool {
+ value
+ .strip_prefix("0x")
+ .is_some_and(|hex| !hex.is_empty() && hex.chars().all(|c| c.is_ascii_hexdigit()))
+}
+
+impl From<RawPath> for Path {
+ fn from(raw: RawPath) -> Self {
+ let target_wwpn = clean_wwpn(raw.target_wwpn);
+ // only a real WWPN means Fibre Channel: the field is present but unset for iSCSI
+ let transport = target_wwpn
+ .as_deref()
+ .is_some_and(is_wwpn)
+ .then_some(Transport::Fc);
+ Path {
+ dev: raw.dev.unwrap_or_default(),
+ dm_state: clean(raw.dm_st),
+ dev_state: clean(raw.dev_st),
+ check_state: clean(raw.chk_st),
+ priority: raw.pri,
+ host_wwpn: clean_wwpn(raw.host_wwpn),
+ target_wwpn,
+ host_adapter: clean(raw.host_adapter),
+ transport,
+ }
+ }
+}
+
+impl From<RawMap> for Map {
+ fn from(raw: RawMap) -> Self {
+ let path_groups: Vec<PathGroup> = raw
+ .path_groups
+ .into_iter()
+ .map(|group| PathGroup {
+ group: group.group.unwrap_or(0).max(0) as u64,
+ dm_state: clean(group.dm_st),
+ priority: group.pri.unwrap_or(0),
+ paths: group.paths.into_iter().map(Path::from).collect(),
+ })
+ .collect();
+ let paths_total = path_groups
+ .iter()
+ .map(|group| group.paths.len() as u64)
+ .sum();
+ let paths_active = path_groups
+ .iter()
+ .flat_map(|group| group.paths.iter())
+ .filter(|path| path.is_active())
+ .count() as u64;
+ Map {
+ wwid: clean(raw.uuid),
+ name: clean(raw.name),
+ sysfs: clean(raw.sysfs),
+ dm_state: clean(raw.dm_st),
+ path_faults: raw.path_faults.unwrap_or(0).max(0) as u64,
+ paths_total,
+ paths_active,
+ health: derive_health(paths_active, paths_total, None),
+ path_groups,
+ }
+ }
+}
+
+/// Parse the output of `multipathd show maps json` into normalized maps, in the daemon's order.
+pub fn parse_maps_json(json: &str) -> Result<Vec<Map>, Error> {
+ let output: RawOutput = serde_json::from_str(json)
+ .map_err(|err| format_err!("could not parse multipathd maps JSON: {err}"))?;
+ Ok(output.maps.into_iter().map(Map::from).collect())
+}
+
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ // three maps, one per health state, in the output format of multipathd
+ const THREE_STATES: &str = r#"{
+ "major_version": 0,
+ "minor_version": 1,
+ "maps": [
+ {
+ "name": "mpatha",
+ "uuid": "3600140500a1b2c3d4e5f6a7b8c9d0e1f",
+ "sysfs": "dm-0",
+ "dm_st": "active",
+ "paths": 2,
+ "path_faults": 0,
+ "path_groups": [
+ {
+ "group": 1,
+ "dm_st": "active",
+ "pri": 50,
+ "paths": [
+ { "dev": "sdb", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 50, "host_wwpn": "0x21000024ff7d6a1c", "target_wwpn": "0x500a098000000001" },
+ { "dev": "sdc", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 50, "host_wwpn": "0x21000024ff7d6a1d", "target_wwpn": "0x500a098000000002" }
+ ]
+ }
+ ]
+ },
+ {
+ "name": "mpathb",
+ "uuid": "360014050aabbccddeeff00112233445566",
+ "sysfs": "dm-1",
+ "dm_st": "active",
+ "paths": 2,
+ "path_faults": 3,
+ "path_groups": [
+ {
+ "group": 1,
+ "dm_st": "active",
+ "pri": 10,
+ "paths": [
+ { "dev": "sdd", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 10 }
+ ]
+ },
+ {
+ "group": 2,
+ "dm_st": "enabled",
+ "pri": 0,
+ "paths": [
+ { "dev": "sde", "dm_st": "failed", "dev_st": "offline", "chk_st": "faulty", "pri": 0 }
+ ]
+ }
+ ]
+ },
+ {
+ "name": "mpathc",
+ "uuid": "36001405ffffffffffffffffffffffffff",
+ "sysfs": "dm-2",
+ "dm_st": "active",
+ "paths": 1,
+ "path_groups": [
+ {
+ "group": 1,
+ "dm_st": "enabled",
+ "pri": 0,
+ "paths": [
+ { "dev": "sdf", "dm_st": "failed", "dev_st": "offline", "chk_st": "faulty", "pri": 0, "target_wwpn": "[undef]", "host_adapter": "[undef]" }
+ ]
+ }
+ ]
+ }
+ ]
+ }"#;
+
+ #[test]
+ fn parses_every_health_state() {
+ let maps = parse_maps_json(THREE_STATES).unwrap();
+ assert_eq!(maps.len(), 3);
+
+ let a = &maps[0];
+ assert_eq!(a.wwid.as_deref(), Some("3600140500a1b2c3d4e5f6a7b8c9d0e1f"));
+ assert_eq!(a.name.as_deref(), Some("mpatha"));
+ assert_eq!(a.sysfs.as_deref(), Some("dm-0"));
+ assert_eq!((a.paths_active, a.paths_total), (2, 2));
+ assert_eq!(a.health, Health::Optimal);
+ assert_eq!(a.path_faults, 0);
+ assert_eq!(a.path_groups.len(), 1);
+ let path = &a.path_groups[0].paths[0];
+ assert_eq!(path.target_wwpn.as_deref(), Some("0x500a098000000001"));
+ assert_eq!(path.host_wwpn.as_deref(), Some("0x21000024ff7d6a1c"));
+ assert_eq!(path.transport, Some(Transport::Fc));
+ assert_eq!(path.priority, Some(50));
+
+ let b = &maps[1];
+ assert_eq!((b.paths_active, b.paths_total), (1, 2));
+ assert_eq!(b.health, Health::Degraded);
+ assert_eq!(b.path_faults, 3);
+ assert_eq!(b.path_groups[1].dm_state.as_deref(), Some("enabled"));
+
+ let c = &maps[2];
+ assert_eq!((c.paths_active, c.paths_total), (0, 1));
+ assert_eq!(c.health, Health::Failed);
+ let path = &c.path_groups[0].paths[0];
+ assert_eq!(path.target_wwpn, None, "'[undef]' is cleaned away");
+ assert_eq!(path.host_adapter, None);
+ assert_eq!(
+ path.transport, None,
+ "'[undef]' target_wwpn does not imply fc"
+ );
+ }
+
+ #[test]
+ fn alua_standby_paths_count_as_active() {
+ // an ALUA standby port: device-mapper keeps the path active, the checker reports ghost
+ let json = r#"{ "maps": [ { "uuid": "3600a", "path_groups": [
+ { "group": 1, "pri": 50, "paths": [ { "dev": "sda", "dm_st": "active", "chk_st": "ready" } ] },
+ { "group": 2, "pri": 10, "paths": [ { "dev": "sdb", "dm_st": "active", "chk_st": "ghost" } ] }
+ ] } ] }"#;
+ let maps = parse_maps_json(json).unwrap();
+ assert_eq!((maps[0].paths_active, maps[0].paths_total), (2, 2));
+ assert_eq!(maps[0].health, Health::Optimal);
+ }
+
+ #[test]
+ fn unreadable_fc_port_names_are_unset() {
+ let json = r#"{ "maps": [ { "uuid": "3600a", "path_groups": [ { "group": 1, "pri": 50, "paths": [
+ { "dev": "sda", "dm_st": "active", "host_wwpn": "[unknown]", "target_wwpn": "[unknown]" }
+ ] } ] } ] }"#;
+ let maps = parse_maps_json(json).unwrap();
+ let path = &maps[0].path_groups[0].paths[0];
+ assert_eq!(path.host_wwpn, None);
+ assert_eq!(path.target_wwpn, None);
+ assert_eq!(path.transport, None);
+ }
+
+ #[test]
+ fn numbers_may_come_as_strings() {
+ let json = r#"{ "maps": [ { "uuid": "3600a", "path_faults": "7", "path_groups": [
+ { "group": "1", "pri": "50", "paths": [ { "dev": "sda", "dm_st": "active", "pri": "50" } ] }
+ ] } ] }"#;
+ let maps = parse_maps_json(json).unwrap();
+ assert_eq!(maps[0].path_faults, 7);
+ assert_eq!(maps[0].path_groups[0].group, 1);
+ assert_eq!(maps[0].path_groups[0].priority, 50);
+ assert_eq!(maps[0].path_groups[0].paths[0].priority, Some(50));
+ }
+
+ #[test]
+ fn empty_and_invalid_input() {
+ assert!(
+ parse_maps_json(r#"{ "major_version": 0, "maps": [] }"#)
+ .unwrap()
+ .is_empty()
+ );
+ assert!(parse_maps_json("{}").unwrap().is_empty());
+ let err = parse_maps_json("not json").unwrap_err();
+ assert!(
+ err.to_string()
+ .starts_with("could not parse multipathd maps JSON")
+ );
+ }
+
+ #[test]
+ fn serializes_with_kebab_case_keys() {
+ let maps = parse_maps_json(THREE_STATES).unwrap();
+ let value = serde_json::to_value(&maps[2]).unwrap();
+ assert_eq!(value["paths-active"], 0);
+ assert_eq!(value["health"], "failed");
+ let path = &value["path-groups"][0]["paths"][0];
+ assert_eq!(path["dm-state"], "failed");
+ assert_eq!(path["check-state"], "faulty");
+ assert!(
+ path.get("target-wwpn").is_none(),
+ "unset fields are omitted"
+ );
+ }
+
+ #[test]
+ fn health_derivation() {
+ assert_eq!(derive_health(0, 0, None), Health::Failed);
+ assert_eq!(derive_health(0, 2, None), Health::Failed);
+ assert_eq!(derive_health(1, 2, None), Health::Degraded);
+ assert_eq!(derive_health(2, 2, None), Health::Optimal);
+ // a path the transport removed no longer counts in total, only the expectation shows it
+ assert_eq!(derive_health(1, 1, Some(2)), Health::Degraded);
+ assert_eq!(derive_health(2, 2, Some(2)), Health::Optimal);
+ assert_eq!(derive_health(3, 3, Some(2)), Health::Optimal);
+ assert_eq!("degraded".parse::<Health>().unwrap(), Health::Degraded);
+ assert!("bogus".parse::<Health>().is_err());
+ }
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 proxmox-perl-rs 02/76] pve-rs: add PVE::RS::Multipath bindings for multipathd status parsing
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 proxmox 01/76] add proxmox-multipath crate Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 cluster 03/76] cfs: observe the multipath configuration files Thomas Lamprecht
` (73 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
Expose the multipathd map parser and the health derivation of the
proxmox-multipath crate to Perl. The managed dm-multipath support in
pve-storage then only runs the daemon command and passes its output
on, instead of keeping a parser of its own. The parser takes the raw
command output, so map aliases with non-ASCII characters keep their
names without a decoding step on the Perl side.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
pve-rs/Cargo.toml | 1 +
pve-rs/Makefile | 2 +
pve-rs/debian/control | 1 +
pve-rs/src/bindings/mod.rs | 3 ++
pve-rs/src/bindings/multipath.rs | 40 +++++++++++++++++
pve-rs/test/Makefile | 1 +
pve-rs/test/multipath.pl | 74 ++++++++++++++++++++++++++++++++
7 files changed, 122 insertions(+)
create mode 100644 pve-rs/src/bindings/multipath.rs
create mode 100755 pve-rs/test/multipath.pl
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index ecedb1e..b89f91a 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -38,6 +38,7 @@ proxmox-frr = { version = "0.6.0" }
proxmox-http = { version = "1.0.2", features = ["client-sync", "client-trait"] }
proxmox-http-error = "1"
proxmox-log = "1"
+proxmox-multipath = "0.1"
proxmox-network-types = "1.1.2"
proxmox-notify = { version = "2", features = ["pve-context", "legacy-matchers"] }
proxmox-oci = "0.2.2"
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index 60f68c7..458ea68 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -27,6 +27,7 @@ PERLMOD_GENPACKAGE := /usr/lib/perlmod/genpackage.pl \
PERLMOD_PACKAGES := \
PVE::RS::Firewall::SDN \
+ PVE::RS::Multipath \
PVE::RS::Notify \
PVE::RS::NVML \
PVE::RS::OCI \
@@ -59,6 +60,7 @@ PROXMOX_DOC_CRATES := \
proxmox-http \
proxmox-http-error \
proxmox-log \
+ proxmox-multipath \
proxmox-notify \
proxmox-openid \
proxmox-resource-scheduling \
diff --git a/pve-rs/debian/control b/pve-rs/debian/control
index ad8611e..33a9bcb 100644
--- a/pve-rs/debian/control
+++ b/pve-rs/debian/control
@@ -26,6 +26,7 @@ Build-Depends: debhelper-compat (= 13),
librust-proxmox-http-1+default-dev (>= 1.0.2-~~),
librust-proxmox-http-error-1+default-dev,
librust-proxmox-log-1+default-dev,
+ librust-proxmox-multipath-0.1+default-dev,
librust-proxmox-network-types-1+default-dev (>= 1.1.2-~~),
librust-proxmox-notify-2+default-dev,
librust-proxmox-notify-2+legacy-matchers-dev,
diff --git a/pve-rs/src/bindings/mod.rs b/pve-rs/src/bindings/mod.rs
index 143a62b..c229327 100644
--- a/pve-rs/src/bindings/mod.rs
+++ b/pve-rs/src/bindings/mod.rs
@@ -1,5 +1,8 @@
//! This contains all the perl bindings.
+mod multipath;
+pub use multipath::pve_rs_multipath;
+
mod nvml;
pub use nvml::pve_rs_nvml;
diff --git a/pve-rs/src/bindings/multipath.rs b/pve-rs/src/bindings/multipath.rs
new file mode 100644
index 0000000..447c4aa
--- /dev/null
+++ b/pve-rs/src/bindings/multipath.rs
@@ -0,0 +1,40 @@
+//! Device-mapper multipath status parsing and health derivation.
+
+#[perlmod::package(name = "PVE::RS::Multipath", lib = "pve_rs")]
+pub mod pve_rs_multipath {
+ //! The `PVE::RS::Multipath` package.
+ //!
+ //! Pure helpers around device-mapper multipath state. The Perl side queries the daemon and
+ //! passes its output here.
+ //!
+ //! See [`proxmox_multipath`].
+
+ use anyhow::Error;
+
+ use proxmox_multipath::{Health, Map};
+
+ /// Parse the output of `multipathd show maps json` into normalized maps, each with its LUN
+ /// WWID, path groups, member paths and derived health.
+ ///
+ /// Takes the raw command output. `multipathd` prints map aliases as configured, so bytes that
+ /// are not valid UTF-8 are replaced rather than failing the status of every map.
+ ///
+ /// See [`proxmox_multipath::parse_maps_json`].
+ #[export]
+ pub fn parse_maps_json(json: &[u8]) -> Result<Vec<Map>, Error> {
+ proxmox_multipath::parse_maps_json(&String::from_utf8_lossy(json))
+ }
+
+ /// Derive the health of a map from its active and total path counts, and the number of paths
+ /// it is expected to have when the transport configuration knows that.
+ ///
+ /// See [`proxmox_multipath::derive_health`].
+ #[export]
+ pub fn derive_health(
+ paths_active: u64,
+ paths_total: u64,
+ paths_expected: Option<u64>,
+ ) -> Health {
+ proxmox_multipath::derive_health(paths_active, paths_total, paths_expected)
+ }
+}
diff --git a/pve-rs/test/Makefile b/pve-rs/test/Makefile
index 29dc955..1ed3210 100644
--- a/pve-rs/test/Makefile
+++ b/pve-rs/test/Makefile
@@ -2,6 +2,7 @@
test: Proxmox/Lib/PVE.pm
@echo "-- running pve-rs tests --"
perl -I. -I.. -I../.. ./resource_scheduling.pl
+ perl -I. -I.. -I../.. ./multipath.pl
# The test stub, we don't know where to look for the library from in here!
Proxmox/Lib/PVE.pm:
diff --git a/pve-rs/test/multipath.pl b/pve-rs/test/multipath.pl
new file mode 100755
index 0000000..2f30008
--- /dev/null
+++ b/pve-rs/test/multipath.pl
@@ -0,0 +1,74 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Test::More;
+
+use PVE::RS::Multipath;
+
+# a recorded `multipathd show maps json` reply: one healthy map, one with a failed path
+my $json = <<'JSON';
+{
+ "major_version": 0,
+ "minor_version": 1,
+ "maps": [
+ {
+ "name": "mpatha", "uuid": "3600140500a1b2c3d4e5f6a7b8c9d0e1f", "sysfs": "dm-0",
+ "dm_st": "active", "path_faults": 0,
+ "path_groups": [ { "group": 1, "dm_st": "active", "pri": 50, "paths": [
+ { "dev": "sdb", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 50,
+ "host_wwpn": "0x21000024ff7d6a1c", "target_wwpn": "0x500a098000000001" },
+ { "dev": "sdc", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 50,
+ "host_wwpn": "0x21000024ff7d6a1d", "target_wwpn": "0x500a098000000002" }
+ ] } ]
+ },
+ {
+ "name": "mpathb", "uuid": "360014050aabbccddeeff00112233445566", "sysfs": "dm-1",
+ "dm_st": "active", "path_faults": "3",
+ "path_groups": [
+ { "group": 1, "dm_st": "active", "pri": 10, "paths": [
+ { "dev": "sdd", "dm_st": "active", "chk_st": "ready", "pri": 10, "target_wwpn": "[undef]" } ] },
+ { "group": 2, "dm_st": "enabled", "pri": 0, "paths": [
+ { "dev": "sde", "dm_st": "failed", "chk_st": "faulty", "pri": 0 } ] }
+ ]
+ }
+ ]
+}
+JSON
+
+my $maps = PVE::RS::Multipath::parse_maps_json($json);
+is(scalar($maps->@*), 2, 'both maps parsed');
+
+my ($a, $b) = $maps->@*;
+is($a->{wwid}, '3600140500a1b2c3d4e5f6a7b8c9d0e1f', 'map keyed on the daemon uuid');
+is($a->{health}, 'optimal', 'all paths active is optimal');
+is($a->{'paths-active'}, 2, 'active path count');
+is($a->{'paths-total'}, 2, 'total path count');
+is($a->{'path-groups'}->[0]->{paths}->[0]->{transport}, 'fc', 'a target WWPN means FC');
+is($a->{'path-groups'}->[0]->{paths}->[0]->{'host-wwpn'}, '0x21000024ff7d6a1c', 'host WWPN kept');
+
+is($b->{health}, 'degraded', 'a failed path makes the map degraded');
+is($b->{'path-faults'}, 3, 'a string number is accepted');
+ok(!exists $b->{'path-groups'}->[0]->{paths}->[0]->{'target-wwpn'}, "'[undef]' is dropped");
+ok(!exists $b->{'path-groups'}->[0]->{paths}->[0]->{transport}, "'[undef]' does not imply FC");
+is($b->{'path-groups'}->[1]->{'dm-state'}, 'enabled', 'group state kept');
+
+is(PVE::RS::Multipath::derive_health(2, 2, undef), 'optimal', 'health without expectation');
+is(PVE::RS::Multipath::derive_health(1, 1, 2), 'degraded', 'a removed path counts against it');
+is(PVE::RS::Multipath::derive_health(0, 2, undef), 'failed', 'no active path is failed');
+
+# raw command output with a UTF-8 alias and one with an invalid byte, then the same reply decoded
+my $raw = qq({ "maps": [ { "name": "lun-\xc3\xa9", "uuid": "3600a", "path_groups": [] },
+ { "name": "bad-\xff", "uuid": "3600b", "path_groups": [] } ] });
+my $raw_maps = PVE::RS::Multipath::parse_maps_json($raw);
+is($raw_maps->[0]->{name}, "lun-\x{e9}", 'a UTF-8 alias in raw output is decoded');
+is($raw_maps->[1]->{name}, "bad-\x{fffd}", 'an invalid byte does not fail the reply');
+my $decoded = $raw =~ s/\xff//r;
+utf8::decode($decoded);
+is(PVE::RS::Multipath::parse_maps_json($decoded)->[0]->{name}, "lun-\x{e9}", 'decoded input');
+
+eval { PVE::RS::Multipath::parse_maps_json('not json') };
+like($@, qr/could not parse multipathd maps JSON/, 'invalid input dies with a clear error');
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 cluster 03/76] cfs: observe the multipath configuration files
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 proxmox 01/76] add proxmox-multipath crate Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 proxmox-perl-rs 02/76] pve-rs: add PVE::RS::Multipath bindings for multipathd status parsing Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 04/76] storage: distinguish status-only activation in plugins Thomas Lamprecht
` (72 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
pve-storage needs to register and lock /etc/pve/multipath.cfg and
/etc/pve/multipath-overrides.conf through PVE::Cluster. Add them to the
observed files so pmxcfs tracks their versions and readers can cache
the parsed configuration until it changes.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Cluster.pm | 2 ++
src/pmxcfs/status.c | 2 ++
2 files changed, 4 insertions(+)
diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index 1661a26..5fe51ea 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -50,6 +50,8 @@ my $observed = {
'vzdump.conf' => 1,
'jobs.cfg' => 1,
'storage.cfg' => 1,
+ 'multipath.cfg' => 1,
+ 'multipath-overrides.conf' => 1,
'datacenter.cfg' => 1,
'replication.cfg' => 1,
'corosync.conf' => 1,
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index e1ab438..e746c17 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -81,6 +81,8 @@ static memdb_change_t memdb_change_array[] = {
{.path = "corosync.conf"},
{.path = "corosync.conf.new"},
{.path = "storage.cfg"},
+ {.path = "multipath.cfg"},
+ {.path = "multipath-overrides.conf"},
{.path = "user.cfg"},
{.path = "domains.cfg"},
{.path = "notifications.cfg"},
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 04/76] storage: distinguish status-only activation in plugins
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (2 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 cluster 03/76] cfs: observe the multipath configuration files Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 05/76] lvm: find a shared volume group created by another node Thomas Lamprecht
` (71 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
pvestatd, the storage index API, and 'pvesm status' activate enabled
storages to collect status. Plugins cannot distinguish this from
activation for actual use, such as starting a guest. Waiting for a
missing device can therefore stall every status pass, while skipping
the wait can make guest startup fail immediately after boot.
Mark status-only activation in the cache and pass it to
check_connection() too, so plugins can choose whether to wait. Plugins
that ignore the new information are unaffected. Bump APIVER and APIAGE
for the plugin API extension.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
ApiChangeLog | 24 ++++++++++++++++++++++++
src/PVE/Storage.pm | 9 +++++----
src/PVE/Storage/Plugin.pm | 2 +-
3 files changed, 30 insertions(+), 5 deletions(-)
diff --git a/ApiChangeLog b/ApiChangeLog
index e9fbcabe..83dda939 100644
--- a/ApiChangeLog
+++ b/ApiChangeLog
@@ -6,6 +6,30 @@ without breaking anything unaware of it.)
Future changes should be documented in here.
+## Version 17:
+
+* Pass the activation cache to the `check_connection()` plugin method
+
+ Old signature of `check_connection()`:
+
+ sub($plugin, $storeid, $scfg)
+
+ New signature of `check_connection()`:
+
+ sub($plugin, $storeid, $scfg, $cache)
+
+ It is the cache of the activation that calls it, the same one the following
+ `activate_storage()` call gets. Plugins that do not need it can ignore the additional parameter.
+
+* Add the `status_only` flag to the activation cache
+
+ `PVE::Storage::storage_info()`, which collects the status of the storages for pvestatd, the
+ storage index API and `pvesm status`, sets `$cache->{status_only}` in the cache it activates the
+ storages with. Plugins can check it in `check_connection()` and `activate_storage()` to skip
+ waiting for what only a use of the storage needs, for example a device that is still appearing,
+ so a storage that cannot get ready does not stall the status updates of the node. Plugins that
+ do not check it are not affected.
+
## Version 16:
* Add `volname_for_format()`, `get_parsed_format()`, `volname_with_format()`, `is_valid_format()` and
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index d660cfc4..891d1885 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -41,11 +41,11 @@ use PVE::Storage::BTRFSPlugin;
use PVE::Storage::ESXiPlugin;
# Storage API version. Increment it on changes in storage API interface.
-use constant APIVER => 16;
+use constant APIVER => 17;
# Age is the number of versions we're backward compatible with.
# This is like having 'current=APIVER' and age='APIAGE' in libtool,
# see https://www.gnu.org/software/libtool/manual/html_node/Libtool-versioning.html
-use constant APIAGE => 7;
+use constant APIAGE => 8;
our $KNOWN_EXPORT_FORMATS = ['raw+size', 'tar+size', 'qcow2+size', 'vmdk+size', 'zfs', 'btrfs'];
@@ -1356,7 +1356,7 @@ sub activate_storage {
activate_storage($cfg, $baseid, $cache);
}
- if (!eval { $plugin->check_connection($storeid, $scfg) }) {
+ if (!eval { $plugin->check_connection($storeid, $scfg, $cache) }) {
die "connection check for storage '$storeid' failed - $@\n" if $@;
die "storage '$storeid' is not online\n";
}
@@ -1485,7 +1485,8 @@ sub storage_info {
push @$slist, $storeid;
}
- my $cache = {};
+ # Status-only activation must not wait for resources needed only when using a storage.
+ my $cache = { status_only => 1 };
foreach my $storeid (keys %$ids) {
my $scfg = $ids->{$storeid};
diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 3d07f749..bec8ae7a 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -2169,7 +2169,7 @@ sub deactivate_volume {
}
sub check_connection {
- my ($class, $storeid, $scfg) = @_;
+ my ($class, $storeid, $scfg, $cache) = @_;
# do nothing by default
return 1;
}
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 05/76] lvm: find a shared volume group created by another node
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (3 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 04/76] storage: distinguish status-only activation in plugins Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 06/76] storage: bound LVM status commands Thomas Lamprecht
` (70 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
LVM hints restrict scans to devices that previously held PVs. A shared
volume group created by another node on an already attached device can
remain invisible until the hints are invalidated, leaving the storage
inactive and blocking guest startup and migration to that node. The
fallback vgscan finds the group but updates neither the hints nor the
cached volume-group list.
Retry the listing without hints. If it finds additional groups, rebuild
the hints with 'pvscan --cache', update the cached groups, and
invalidate the logical-volume list used by thin pool activation. Rebuild
only when needed: pvscan also resets LVM autoactivation records and must
not run on every status pass merely because a group remains absent.
This was observed on a five-node cluster with shared groups created on
iSCSI multipath LUNs and NVMe over Fabrics namespaces already attached
to the other nodes.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Storage/LVMPlugin.pm | 43 ++++++++-----
src/test/Makefile | 5 +-
src/test/run_test_lvm_hints.pl | 109 +++++++++++++++++++++++++++++++++
3 files changed, 140 insertions(+), 17 deletions(-)
create mode 100755 src/test/run_test_lvm_hints.pl
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index a6a5eb70..3ed5bc78 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -161,7 +161,7 @@ sub lvm_destroy_volume_group {
}
sub lvm_vgs {
- my ($includepvs) = @_;
+ my ($includepvs, $nohints) = @_;
my $cmd = [
'/sbin/vgs',
@@ -172,8 +172,9 @@ sub lvm_vgs {
'b',
'--unbuffered',
'--nosuffix',
- '--options',
];
+ push @$cmd, '--nohints' if $nohints;
+ push @$cmd, '--options';
my $cols = [qw(vg_name vg_size vg_free lv_count)];
@@ -1420,25 +1421,35 @@ sub volume_snapshot_info {
return $info;
}
+# LVM hints can miss PVs written by another node onto previously empty devices. Probe without
+# hints once per activation cache; rebuild only when new VGs appear, since rebuilding also resets
+# LVM autoactivation records.
+my sub find_unhinted_vgs {
+ my ($cache) = @_;
+
+ $cache->{vgscaned} = 1;
+
+ my $vgs = lvm_vgs(0, 1);
+ my @unhinted = grep { !$cache->{vgs}->{$_} } sort keys %$vgs;
+ return if !@unhinted;
+
+ eval {
+ run_command(['/sbin/pvscan', '--cache'], outfunc => sub { });
+ };
+ warn "rebuilding the LVM device hints failed - $@" if $@;
+
+ # add, never replace: an incomplete listing must not hide volume groups found before
+ $cache->{vgs}->{$_} = $vgs->{$_} for @unhinted;
+ # the cached logical volume list lacks the volumes of these volume groups
+ delete $cache->{lvs};
+}
+
sub activate_storage {
my ($class, $storeid, $scfg, $cache) = @_;
$cache->{vgs} = lvm_vgs() if !$cache->{vgs};
- # In LVM2, vgscans take place automatically;
- # this is just to be sure
- if (
- $cache->{vgs}
- && !$cache->{vgscaned}
- && !$cache->{vgs}->{ $scfg->{vgname} }
- ) {
- $cache->{vgscaned} = 1;
- my $cmd = ['/sbin/vgscan', '--ignorelockingfailure', '--mknodes'];
- eval {
- run_command($cmd, outfunc => sub { });
- };
- warn $@ if $@;
- }
+ find_unhinted_vgs($cache) if !$cache->{vgscaned} && !$cache->{vgs}->{ $scfg->{vgname} };
# we do not acticate any volumes here ('vgchange -aly')
# instead, volumes are activate individually later
diff --git a/src/test/Makefile b/src/test/Makefile
index 1fe50885..02d833e8 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_lock_retry test_lvm_base_volume test_disklist test_bwlimit test_plugin test_ovf test_volume_access
+test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_disklist test_bwlimit test_plugin test_ovf test_volume_access
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -15,6 +15,9 @@ test_lvm_lock_retry: run_test_lvm_lock_retry.pl
test_lvm_base_volume: run_test_lvm_base_volume.pl
./run_test_lvm_base_volume.pl
+test_lvm_hints: run_test_lvm_hints.pl
+ ./run_test_lvm_hints.pl
+
test_disklist: run_disk_tests.pl
./run_disk_tests.pl
diff --git a/src/test/run_test_lvm_hints.pl b/src/test/run_test_lvm_hints.pl
new file mode 100755
index 00000000..fc2ffff7
--- /dev/null
+++ b/src/test/run_test_lvm_hints.pl
@@ -0,0 +1,109 @@
+#!/usr/bin/perl
+
+# Unit test for the activation of an LVM storage whose volume group is missing from the volume
+# groups LVM lists through its device hints, as after another node created it on a device this node
+# already had: a listing without the hints finds it, and only then the hints are rebuilt. vgs and
+# pvscan are mocked, so the test needs neither root nor a real LVM setup.
+
+use strict;
+use warnings;
+
+use lib '..';
+use Test::More;
+use Test::MockModule;
+
+use PVE::Storage::LVMPlugin;
+use PVE::Storage::LvmThinPlugin;
+
+my $CLASS = 'PVE::Storage::LVMPlugin';
+
+# the volume groups vgs lists with and without the hints, and the commands run
+my ($hinted, $all, $vgs_fails, $pvscan_fails, @calls);
+
+my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
+$lvm->mock(
+ run_command => sub {
+ my ($cmd, %param) = @_;
+ if ($cmd->[0] eq '/sbin/vgs') {
+ my $nohints = grep { $_ eq '--nohints' } @$cmd;
+ push @calls, $nohints ? 'vgs --nohints' : 'vgs';
+ die "vgs failed\n" if $nohints && $vgs_fails;
+ $param{outfunc}->("$_:1024:512:0") for ($nohints ? $all : $hinted)->@*;
+ } elsif ($cmd->[0] eq '/sbin/pvscan') {
+ push @calls, 'pvscan --cache';
+ die "pvscan failed\n" if $pvscan_fails;
+ } else {
+ push @calls, join(' ', @$cmd);
+ }
+ },
+);
+
+my sub activate {
+ my ($vgname, $cache) = @_;
+
+ @calls = ();
+ my @warnings;
+ local $SIG{__WARN__} = sub { push @warnings, $_[0] };
+ $CLASS->activate_storage('lvm', { vgname => $vgname }, $cache);
+ return (join(', ', @calls), \@warnings);
+}
+
+($hinted, $all) = (['vg1'], ['vg1']);
+my ($calls) = activate('vg1', {});
+is($calls, 'vgs', 'listed volume group: no listing without the hints');
+
+($hinted, $all) = (['vg1'], ['vg1', 'vg2']);
+my $cache = { lvs => { vg1 => {} } };
+($calls) = activate('vg2', $cache);
+is($calls, 'vgs, vgs --nohints, pvscan --cache', 'unhinted volume group: hints rebuilt');
+ok($cache->{vgs}->{vg2} && $cache->{vgs}->{vg1}, 'and added to the volume groups of the cache');
+ok(!exists($cache->{lvs}), 'and the logical volume list of the cache is dropped');
+is_deeply(
+ [$CLASS->status('lvm', { vgname => 'vg2' }, $cache)],
+ [1024, 512, 512, 1],
+ 'the status of the same cache sees the volume group',
+);
+
+($calls) = activate('vg3', $cache);
+is($calls, '', 'once per cache: no second listing for the next missing volume group');
+
+($hinted, $all) = (['vg1'], ['vg1']);
+($calls) = activate('vg2', {});
+is($calls, 'vgs, vgs --nohints', 'volume group missing everywhere: hints left alone');
+
+($hinted, $all, $vgs_fails) = (['vg1'], [], 1);
+$cache = {};
+my $warnings;
+($calls, $warnings) = activate('vg2', $cache);
+is($calls, 'vgs, vgs --nohints', 'failed listing without the hints: hints left alone');
+ok($cache->{vgs}->{vg1}, 'and the volume groups found before stay');
+like($warnings->[0] // '', qr/vgs failed/, 'and it warns');
+$vgs_fails = 0;
+
+($hinted, $all, $pvscan_fails) = (['vg1'], ['vg1', 'vg2'], 1);
+$cache = {};
+($calls, $warnings) = activate('vg2', $cache);
+ok($cache->{vgs}->{vg2}, 'failed rebuild of the hints: the volume group still counts');
+like(
+ $warnings->[0] // '',
+ qr/^rebuilding the LVM device hints failed - pvscan failed/,
+ 'and it warns about the rebuild',
+);
+$pvscan_fails = 0;
+
+# a thin pool storage activates through the LVM activation and reads the logical volume list
+my $listed;
+$lvm->mock(
+ lvm_list_volumes => sub {
+ $listed++;
+ return { vg2 => { data => { lv_type => 't', lv_state => 'a' } } };
+ },
+);
+($hinted, $all) = (['vg1'], ['vg1', 'vg2']);
+$cache = { lvs => { vg1 => {} } };
+PVE::Storage::LvmThinPlugin->activate_storage(
+ 'thin', { vgname => 'vg2', thinpool => 'data' }, $cache,
+);
+ok($listed, 'thin pool: the logical volumes are listed again after the volume group was found');
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 06/76] storage: bound LVM status commands
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (4 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 05/76] lvm: find a shared volume group created by another node Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 07/76] iscsi: rate-limit session rescans per target Thomas Lamprecht
` (69 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
LVM can wait indefinitely for an unavailable device, stalling the
storage status pass. Bound its commands with a shared time budget
and retain an inherited lock so a stuck command prevents more probes
from accumulating. Killing a command does not guarantee that it
exits, so cleanup must not wait indefinitely either.
Give concurrent callers the same short, bounded lock wait, including
when a previous command remains stuck. Count that wait against the
budget and reject incomplete results. Retain pending children for
later reaping and hand their records over across daemon re-exec.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
This should likely become a generic helper in pve-common. Keep it here
to avoid adding another repository to this already cross-repository
series.
src/PVE/Storage.pm | 13 +-
src/PVE/Storage/LVMPlugin.pm | 56 +-
src/PVE/Storage/LvmThinPlugin.pm | 39 +-
src/PVE/Storage/Makefile | 3 +-
src/PVE/Storage/StatusCommand.pm | 241 ++++++++
src/test/Makefile | 12 +-
src/test/run_san_library_tests.pl | 35 ++
.../run_status_command_contention_tests.pl | 520 ++++++++++++++++++
src/test/run_status_command_tests.pl | 189 +++++++
src/test/run_test_lvm_hints.pl | 7 +-
src/test/run_test_lvm_status.pl | 220 ++++++++
11 files changed, 1299 insertions(+), 36 deletions(-)
create mode 100644 src/PVE/Storage/StatusCommand.pm
create mode 100755 src/test/run_san_library_tests.pl
create mode 100755 src/test/run_status_command_contention_tests.pl
create mode 100755 src/test/run_status_command_tests.pl
create mode 100755 src/test/run_test_lvm_status.pl
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 891d1885..2fc91a82 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -1487,6 +1487,7 @@ sub storage_info {
# Status-only activation must not wait for resources needed only when using a storage.
my $cache = { status_only => 1 };
+ my $warned = {};
foreach my $storeid (keys %$ids) {
my $scfg = $ids->{$storeid};
@@ -1518,12 +1519,14 @@ sub storage_info {
eval { activate_storage($cfg, $storeid, $cache); };
if (my $err = $@) {
- warn $err;
+ warn $err if !$warned->{$err}++;
next;
}
my ($total, $avail, $used, $active) = eval { $plugin->status($storeid, $scfg, $cache); };
- warn $@ if $@;
+ if (my $err = $@) {
+ warn $err if !$warned->{$err}++;
+ }
next if !$active;
$info->{$storeid}->{total} = int($total);
$info->{$storeid}->{avail} = int($avail);
@@ -1531,6 +1534,12 @@ sub storage_info {
$info->{$storeid}->{active} = $active;
}
+ # A later command failure invalidates earlier results from the same shared inventory.
+ for my $storeid (keys(($cache->{'withdrawn-status'} // {})->%*)) {
+ next if !$info->{$storeid};
+ $info->{$storeid}->{$_} = 0 for qw(total avail used active);
+ }
+
return $info;
}
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index 3ed5bc78..eb9806b2 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -20,6 +20,7 @@ use PVE::Tools qw(run_command file_read_firstline trim);
use PVE::Storage::Common;
use PVE::Storage::Plugin;
+use PVE::Storage::StatusCommand;
use base qw(PVE::Storage::Plugin);
@@ -160,8 +161,9 @@ sub lvm_destroy_volume_group {
);
}
+# Status-only callers must reject partial listings.
sub lvm_vgs {
- my ($includepvs, $nohints) = @_;
+ my ($includepvs, $nohints, $cache) = @_;
my $cmd = [
'/sbin/vgs',
@@ -186,7 +188,8 @@ sub lvm_vgs {
my $vgs = {};
eval {
- run_command(
+ PVE::Storage::StatusCommand::run_for_cache(
+ $cache,
$cmd,
outfunc => sub {
my $line = shift;
@@ -214,6 +217,7 @@ sub lvm_vgs {
);
};
my $err = $@;
+ die $err if $err && $cache && $cache->{status_only};
# just warn (vgs return error code 5 if clvmd does not run)
# but output is still OK (list without clustered VGs)
@@ -223,7 +227,7 @@ sub lvm_vgs {
}
sub lvm_list_volumes {
- my ($vgname) = @_;
+ my ($vgname, $cache) = @_;
my $option_list =
'vg_name,lv_name,lv_size,lv_attr,pool_lv,data_percent,metadata_percent,snap_percent,uuid,tags,metadata_size,time';
@@ -246,7 +250,8 @@ sub lvm_list_volumes {
push @$cmd, $vgname if $vgname;
my $lvs = {};
- run_command(
+ PVE::Storage::StatusCommand::run_for_cache(
+ $cache,
$cmd,
outfunc => sub {
my $line = shift;
@@ -1353,15 +1358,17 @@ sub list_images {
sub status {
my ($class, $storeid, $scfg, $cache) = @_;
- $cache->{vgs} = lvm_vgs() if !$cache->{vgs};
+ return PVE::Storage::StatusCommand::collect_shared_status(
+ $cache,
+ $storeid,
+ 'LVM',
+ sub {
+ $cache->{vgs} = lvm_vgs(undef, undef, $cache) if !$cache->{vgs};
- my $vgname = $scfg->{vgname};
-
- if (my $info = $cache->{vgs}->{$vgname}) {
- return ($info->{size}, $info->{free}, $info->{size} - $info->{free}, 1);
- }
-
- return undef;
+ my $info = $cache->{vgs}->{ $scfg->{vgname} } or return undef;
+ return ($info->{size}, $info->{free}, $info->{size} - $info->{free}, 1);
+ },
+ );
}
sub volume_snapshot_info {
@@ -1429,14 +1436,21 @@ my sub find_unhinted_vgs {
$cache->{vgscaned} = 1;
- my $vgs = lvm_vgs(0, 1);
+ my $vgs = lvm_vgs(0, 1, $cache);
my @unhinted = grep { !$cache->{vgs}->{$_} } sort keys %$vgs;
return if !@unhinted;
eval {
- run_command(['/sbin/pvscan', '--cache'], outfunc => sub { });
+ PVE::Storage::StatusCommand::run_for_cache(
+ $cache,
+ ['/sbin/pvscan', '--cache'],
+ outfunc => sub { },
+ );
};
- warn "rebuilding the LVM device hints failed - $@" if $@;
+ if (my $err = $@) {
+ die $err if $cache->{status_only};
+ warn "rebuilding the LVM device hints failed - $err";
+ }
# add, never replace: an incomplete listing must not hide volume groups found before
$cache->{vgs}->{$_} = $vgs->{$_} for @unhinted;
@@ -1447,9 +1461,17 @@ my sub find_unhinted_vgs {
sub activate_storage {
my ($class, $storeid, $scfg, $cache) = @_;
- $cache->{vgs} = lvm_vgs() if !$cache->{vgs};
+ PVE::Storage::StatusCommand::collect_shared_status(
+ $cache,
+ $storeid,
+ 'LVM',
+ sub {
+ $cache->{vgs} = lvm_vgs(undef, undef, $cache) if !$cache->{vgs};
- find_unhinted_vgs($cache) if !$cache->{vgscaned} && !$cache->{vgs}->{ $scfg->{vgname} };
+ find_unhinted_vgs($cache)
+ if !$cache->{vgscaned} && !$cache->{vgs}->{ $scfg->{vgname} };
+ },
+ );
# we do not acticate any volumes here ('vgchange -aly')
# instead, volumes are activate individually later
diff --git a/src/PVE/Storage/LvmThinPlugin.pm b/src/PVE/Storage/LvmThinPlugin.pm
index 6b953693..01b3164d 100644
--- a/src/PVE/Storage/LvmThinPlugin.pm
+++ b/src/PVE/Storage/LvmThinPlugin.pm
@@ -10,6 +10,7 @@ use PVE::Tools qw(run_command trim);
use PVE::Storage::Plugin;
use PVE::Storage::LVMPlugin;
+use PVE::Storage::StatusCommand;
# see: man lvmthin
# lvcreate -n ThinDataLV -L LargeSize VG
@@ -229,32 +230,41 @@ sub list_thinpools {
sub status {
my ($class, $storeid, $scfg, $cache) = @_;
- my $lvs = $cache->{lvs} ||= PVE::Storage::LVMPlugin::lvm_list_volumes();
+ return PVE::Storage::StatusCommand::collect_shared_status(
+ $cache,
+ $storeid,
+ 'LVM',
+ sub {
+ my $lvs = $cache->{lvs} ||=
+ PVE::Storage::LVMPlugin::lvm_list_volumes(undef, $cache);
- return if !$lvs->{ $scfg->{vgname} };
+ return if !$lvs->{ $scfg->{vgname} };
- my $info = $lvs->{ $scfg->{vgname} }->{ $scfg->{thinpool} };
+ my $info = $lvs->{ $scfg->{vgname} }->{ $scfg->{thinpool} };
- return if !$info || $info->{lv_type} ne 't' || !$info->{lv_size};
+ return if !$info || $info->{lv_type} ne 't' || !$info->{lv_size};
- return (
- $info->{lv_size},
- $info->{lv_size} - $info->{used},
- $info->{used},
- $info->{lv_state} eq 'a' ? 1 : 0,
+ return (
+ $info->{lv_size},
+ $info->{lv_size} - $info->{used},
+ $info->{used},
+ $info->{lv_state} eq 'a' ? 1 : 0,
+ );
+ },
);
}
my $activate_lv = sub {
my ($vg, $lv, $cache) = @_;
- my $lvs = $cache->{lvs} ||= PVE::Storage::LVMPlugin::lvm_list_volumes();
+ my $lvs = $cache->{lvs} ||= PVE::Storage::LVMPlugin::lvm_list_volumes(undef, $cache);
die "no such logical volume $vg/$lv\n" if !$lvs->{$vg} || !$lvs->{$vg}->{$lv};
return if $lvs->{$vg}->{$lv}->{lv_state} eq 'a';
- run_command(
+ PVE::Storage::StatusCommand::run_for_cache(
+ $cache,
['lvchange', '-ay', '-K', "$vg/$lv"],
errmsg => "activating LV '$vg/$lv' failed",
);
@@ -269,7 +279,12 @@ sub activate_storage {
$class->SUPER::activate_storage($storeid, $scfg, $cache);
- $activate_lv->($scfg->{vgname}, $scfg->{thinpool}, $cache);
+ PVE::Storage::StatusCommand::collect_shared_status(
+ $cache,
+ $storeid,
+ 'LVM',
+ sub { $activate_lv->($scfg->{vgname}, $scfg->{thinpool}, $cache) },
+ );
}
sub activate_volume {
diff --git a/src/PVE/Storage/Makefile b/src/PVE/Storage/Makefile
index a67dc25f..47813553 100644
--- a/src/PVE/Storage/Makefile
+++ b/src/PVE/Storage/Makefile
@@ -14,7 +14,8 @@ SOURCES= \
PBSPlugin.pm \
BTRFSPlugin.pm \
LvmThinPlugin.pm \
- ESXiPlugin.pm
+ ESXiPlugin.pm \
+ StatusCommand.pm
.PHONY: install
install:
diff --git a/src/PVE/Storage/StatusCommand.pm b/src/PVE/Storage/StatusCommand.pm
new file mode 100644
index 00000000..b0b48d59
--- /dev/null
+++ b/src/PVE/Storage/StatusCommand.pm
@@ -0,0 +1,241 @@
+package PVE::Storage::StatusCommand;
+
+# LVM can remain blocked in I/O after SIGKILL. Child stdin retains a node-wide lock so another
+# observer does not start a competing command while it stays held. Every observer waits briefly for
+# the lock, including when a previous command did not stop. The shared command budget allows 0.1
+# seconds for cleanup, excluding scheduling overhead; it does not bound the entire status pass.
+
+use v5.36;
+
+use Fcntl qw(:DEFAULT :flock);
+use File::Basename qw(basename);
+use IO::Select;
+use POSIX qw(:sys_wait_h);
+use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC);
+
+use PVE::Tools;
+
+our $LOCK_FILE = '/run/pve-storage-status-command.lock';
+our $KILL_SIGNAL = 'KILL';
+
+# the time all status commands of one pass, sharing an activation cache, may take together
+our $BUDGET = 15;
+
+our $OUTPUT_LIMIT = 16 * 1024 * 1024;
+
+our $CLOCK = sub () { return clock_gettime(CLOCK_MONOTONIC); };
+
+# A forked child must not take over its parent's pending commands: { pid => name }.
+my $pending = {};
+my $pending_owner = $$;
+
+sub reap_pending() {
+ if ($pending_owner != $$) {
+ ($pending, $pending_owner) = ({}, $$);
+ return;
+ }
+ for my $pid (keys %$pending) {
+ my $res = waitpid($pid, WNOHANG);
+ next if $res == 0 || ($res < 0 && $!{EINTR});
+ delete $pending->{$pid};
+ }
+}
+
+# Re-exec preserves the PID and child relationships; pending children must survive the handover.
+sub export_pending() {
+ reap_pending();
+ return "$$:" . join(',', map { "$_=$pending->{$_}" } sort keys %$pending);
+}
+
+# Restore the pending commands that export_pending() of this same process saved before its re-exec.
+sub import_pending($value) {
+ my ($owner, $list) = ($value // '') =~ m/^(\d+):((?:\d+=[\w.-]+,?)*)$/ or return;
+ return if $owner != $$;
+ ($pending, $pending_owner) = ({}, $$);
+ for my $entry (split(/,/, $list)) {
+ my ($pid, $name) = $entry =~ m/^(\d+)=([\w.-]+)$/;
+ $pending->{$pid} = $name;
+ }
+ reap_pending();
+}
+
+my sub exec_child($cmd, $lock, $out, $err) {
+ eval {
+ $SIG{$_} = 'DEFAULT' for keys %SIG;
+ POSIX::sigprocmask(POSIX::SIG_SETMASK(), POSIX::SigSet->new());
+ POSIX::dup2(fileno($lock), 0) // die;
+ POSIX::dup2(fileno($out), 1) // die;
+ POSIX::dup2(fileno($err), 2) // die;
+ for my $fd (map { m!/(\d+)$! ? $1 : () } glob('/proc/self/fd/*')) {
+ POSIX::close($fd) if $fd > 2;
+ }
+ # the output is parsed, and LVM must not write its reports to other descriptors
+ $ENV{LC_ALL} = 'C';
+ delete $ENV{$_} for grep { m/^LVM_\w+_FD$/ } keys %ENV;
+ exec({ $cmd->[0] } $cmd->@*);
+ };
+ POSIX::_exit(127);
+}
+
+my sub collect($pid, $name, $deadline, $out, $err, %param) {
+ my $func = { fileno($out) => $param{outfunc}, fileno($err) => $param{errfunc} };
+ my $buffer = { fileno($out) => '', fileno($err) => '' };
+ my $select = IO::Select->new($out, $err);
+ my ($length, $last_error) = (0, '');
+
+ my $line = sub($fd, $text) {
+ $last_error = $text if $fd == fileno($err) && $text =~ m/\S/;
+ $func->{$fd}->($text) if $func->{$fd};
+ };
+
+ while (1) {
+ if (!$select->count()) {
+ my $res = waitpid($pid, WNOHANG);
+ return ($?, $last_error) if $res == $pid;
+ die "status command '$name' was reaped elsewhere\n" if $res < 0;
+ }
+ my $left = $deadline - $CLOCK->();
+ die "status command '$name' did not finish within the shared $BUDGET-second limit\n"
+ if $left <= 0;
+ if (!$select->count()) {
+ # the output is complete, but the process did not exit yet
+ select(undef, undef, undef, $left < 0.01 ? $left : 0.01);
+ next;
+ }
+ for my $fh ($select->can_read($left)) {
+ my $fd = fileno($fh);
+ my $count = sysread($fh, my $data, 65536);
+ if (!$count) {
+ $select->remove($fh);
+ $line->($fd, $buffer->{$fd}) if length($buffer->{$fd});
+ next;
+ }
+ $length += $count;
+ die "status command '$name' wrote more than $OUTPUT_LIMIT bytes\n"
+ if $length > $OUTPUT_LIMIT;
+ $buffer->{$fd} .= $data;
+ $line->($fd, $1) while $buffer->{$fd} =~ s/^([^\n]*)\n//;
+ }
+ }
+}
+
+my sub run_bounded($cmd, $deadline, %param) {
+ my $name = basename($cmd->[0]);
+ reap_pending();
+ die "status command '$name' not started - shared time limit of $BUDGET seconds reached\n"
+ if $CLOCK->() >= $deadline;
+
+ sysopen(my $lock, $LOCK_FILE, O_RDWR | O_CREAT, 0600)
+ or die "opening '$LOCK_FILE' failed - $!\n";
+ if (!flock($lock, LOCK_EX | LOCK_NB)) {
+ die "locking '$LOCK_FILE' failed - $!\n" if !$!{EWOULDBLOCK} && !$!{EAGAIN} && !$!{EINTR};
+
+ my $wait_until = $CLOCK->() + 2;
+ my $budget_limited = $deadline - 0.5 < $wait_until;
+ $wait_until = $deadline - 0.5 if $budget_limited;
+ my $locked = eval {
+ # Use scalar context: list context reports timeout separately instead of throwing it.
+ my $left = $wait_until - $CLOCK->();
+ die "got timeout\n" if $left <= 0;
+ scalar PVE::Tools::run_with_timeout(
+ $left,
+ sub () {
+ while (!flock($lock, LOCK_EX)) {
+ die "locking '$LOCK_FILE' failed - $!\n" if !$!{EINTR};
+ die "got timeout\n" if $CLOCK->() >= $wait_until;
+ }
+ return 1;
+ },
+ );
+ };
+ if (!$locked) {
+ my $err = $@;
+ die $err if $err ne "got timeout\n";
+ die "status command '$name' not started - not enough time left after waiting for"
+ . " another status command (shared limit: $BUDGET seconds)\n"
+ if $budget_limited;
+ die "another status command is still running\n";
+ }
+ }
+
+ pipe(my $out_r, my $out_w) or die "pipe failed - $!\n";
+ pipe(my $err_r, my $err_w) or die "pipe failed - $!\n";
+ die
+ "status command '$name' not started - not enough time left (shared limit: $BUDGET seconds)\n"
+ if $deadline - $CLOCK->() < 0.5;
+ my $pid = fork() // die "fork failed - $!\n";
+ exec_child($cmd, $lock, $out_w, $err_w) if $pid == 0;
+
+ # the child holds the lock from now on, until it exits; no unlock, that would end it for both
+ close($lock);
+ close($out_w);
+ close($err_w);
+
+ my ($status, $last_error) = eval { collect($pid, $name, $deadline, $out_r, $err_r, %param) };
+ if (my $err = $@) {
+ # Use one cleanup deadline across interrupted polls; debit it from the shared budget and
+ # never block in waitpid(), even after killing the child.
+ my $cleanup_until = $CLOCK->() + 0.1;
+ kill($KILL_SIGNAL, $pid);
+ while (1) {
+ my $res = waitpid($pid, WNOHANG);
+ last if $res != 0 && !($res < 0 && $!{EINTR});
+ my $left = $cleanup_until - $CLOCK->();
+ if ($left <= 0) {
+ $pending->{$pid} = $name;
+ last;
+ }
+ select(undef, undef, undef, $left < 0.01 ? $left : 0.01);
+ }
+ die $err;
+ }
+
+ my $detail = $last_error ne '' ? " - $last_error" : '';
+ die "status command '$name' failed: exit code " . ($status >> 8) . "$detail\n" if $status >> 8;
+ die "status command '$name' failed: killed by signal " . ($status & 127) . "\n"
+ if $status & 127;
+ return 0;
+}
+
+# Command, timeout or contention failures poison subsequent commands sharing this cache.
+sub run($cache, $cmd, %param) {
+ die $cache->{'status-command-error'} if defined($cache->{'status-command-error'});
+
+ my $start = $CLOCK->();
+ my $budget = $cache->{'status-command-budget'} // $BUDGET;
+ my $res = eval { run_bounded($cmd, $start + $budget, %param) };
+ my $err = $@;
+ $cache->{'status-command-budget'} = $budget - ($CLOCK->() - $start);
+ if ($err) {
+ $cache->{'status-command-error'} = $err;
+ die $err;
+ }
+ return $res;
+}
+
+sub run_for_cache($cache, $cmd, %param) {
+ return run($cache, $cmd, %param) if $cache && $cache->{status_only};
+ return PVE::Tools::run_command($cmd, %param);
+}
+
+# A command failure invalidates all results sharing this inventory, including earlier successes;
+# storage_info() withdraws them. Name the shared kind in errors so the status loop can deduplicate.
+sub collect_shared_status($cache, $storeid, $kind, $code) {
+ return $code->() if !$cache || !$cache->{status_only};
+
+ $cache->{'shared-status'}->{$kind}->{$storeid} = 1;
+ my $want = wantarray;
+ my @res = eval {
+ die $cache->{'status-command-error'} if defined($cache->{'status-command-error'});
+ $want ? $code->() : scalar($code->());
+ };
+ my $err = $@;
+ if (defined(my $failed = $cache->{'status-command-error'})) {
+ $cache->{'withdrawn-status'}->{$_} = 1 for keys $cache->{'shared-status'}->{$kind}->%*;
+ die "$kind status unavailable - $failed";
+ }
+ die $err if $err;
+ return $want ? @res : $res[0];
+}
+
+1;
diff --git a/src/test/Makefile b/src/test/Makefile
index 02d833e8..0b9b95b2 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_disklist test_bwlimit test_plugin test_ovf test_volume_access
+test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_bwlimit test_plugin test_ovf test_volume_access test_san_libraries
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -18,6 +18,13 @@ test_lvm_base_volume: run_test_lvm_base_volume.pl
test_lvm_hints: run_test_lvm_hints.pl
./run_test_lvm_hints.pl
+test_lvm_status: run_test_lvm_status.pl
+ ./run_test_lvm_status.pl
+
+test_status_command: run_status_command_tests.pl run_status_command_contention_tests.pl
+ ./run_status_command_tests.pl
+ ./run_status_command_contention_tests.pl
+
test_disklist: run_disk_tests.pl
./run_disk_tests.pl
@@ -37,3 +44,6 @@ test_iscsi: run_iscsi_discovery_tests.pl run_iscsi_session_tests.pl run_iscsi_vo
./run_iscsi_discovery_tests.pl
./run_iscsi_session_tests.pl
./run_iscsi_volume_tests.pl
+
+test_san_libraries: run_san_library_tests.pl
+ ./run_san_library_tests.pl
diff --git a/src/test/run_san_library_tests.pl b/src/test/run_san_library_tests.pl
new file mode 100755
index 00000000..6d509979
--- /dev/null
+++ b/src/test/run_san_library_tests.pl
@@ -0,0 +1,35 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Cluster;
+use PVE::Storage::StatusCommand;
+
+subtest 'shared status retains callback context and named command parameters' => sub {
+ for my $cache (undef, { status_only => 1 }) {
+ my $code = sub { return wantarray ? ('a', 'b') : 'scalar' };
+ my @result =
+ PVE::Storage::StatusCommand::collect_shared_status($cache, 'san', 'lvm', $code);
+ is_deeply(\@result, ['a', 'b'], 'list context reaches the callback');
+ my $result =
+ PVE::Storage::StatusCommand::collect_shared_status($cache, 'san', 'lvm', $code);
+ is($result, 'scalar', 'scalar context reaches the callback');
+ }
+ my $tools = Test::MockModule->new('PVE::Tools');
+ my @args;
+ $tools->mock(run_command => sub { @args = @_; return 7 });
+ is(
+ PVE::Storage::StatusCommand::run_for_cache(undef, ['probe'], timeout => 3),
+ 7,
+ 'command result is retained',
+ );
+ is_deeply(\@args, [['probe'], timeout => 3], 'named parameters are forwarded');
+};
+
+done_testing();
diff --git a/src/test/run_status_command_contention_tests.pl b/src/test/run_status_command_contention_tests.pl
new file mode 100755
index 00000000..3c535d33
--- /dev/null
+++ b/src/test/run_status_command_contention_tests.pl
@@ -0,0 +1,520 @@
+#!/usr/bin/perl
+
+# Real-clock, cross-process admission tests. Each lock holder announces readiness before another
+# observer starts, and every stand-in has a release channel and a safety alarm. Exec modes exercise
+# the daemon's opaque handover while its pending direct child is alive and after that child exits.
+
+use strict;
+use warnings;
+
+use lib '..';
+use Fcntl qw(:DEFAULT :flock);
+use File::Temp qw(tempdir);
+use JSON::PP qw(encode_json decode_json);
+use POSIX ();
+use Test::More;
+use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC sleep);
+
+use PVE::Tools;
+use PVE::Storage::StatusCommand;
+
+my $mode = shift(@ARGV) // '';
+my $dir = $mode ? shift(@ARGV) : tempdir(CLEANUP => 1);
+$PVE::Storage::StatusCommand::LOCK_FILE = "$dir/lock";
+my $timer = \&PVE::Tools::run_with_timeout;
+my $busy = "another status command is still running\n";
+my (@children, @commands);
+my $sequence = 0;
+
+my sub now { return clock_gettime(CLOCK_MONOTONIC); }
+
+my sub put {
+ my ($path, $value) = @_;
+ open(my $fh, '>', $path) or die "open $path: $!\n";
+ print {$fh} $value or die "write $path: $!\n";
+ close($fh) or die "close $path: $!\n";
+}
+
+my sub get {
+ my ($path) = @_;
+ open(my $fh, '<', $path) or die "open $path: $!\n";
+ local $/;
+ return <$fh>;
+}
+
+my sub await {
+ my ($code, $what) = @_;
+ my $deadline = now() + 5;
+ until ($code->()) {
+ die "timed out waiting for $what\n" if now() >= $deadline;
+ sleep(0.005);
+ }
+}
+
+my sub release {
+ my ($fifo) = @_;
+ await(
+ sub {
+ if (sysopen(my $fh, $fifo, O_WRONLY | O_NONBLOCK)) {
+ close($fh);
+ return 1;
+ }
+ die "open $fifo: $!\n" if !$!{ENXIO};
+ return 0;
+ },
+ "FIFO reader $fifo",
+ );
+}
+
+my sub spawn {
+ my ($code) = @_;
+ my $path = "$dir/job-" . ++$sequence;
+ my $pid = fork() // die "fork: $!\n";
+ if (!$pid) {
+ @children = @commands = ();
+ # Signal trains can overlap result publication after a case restores its local handler.
+ $SIG{USR1} = 'IGNORE';
+ alarm(12);
+ my $result = eval { $code->($path) };
+ $result = { fatal => $@ } if $@;
+ put($path, encode_json($result));
+ POSIX::_exit(0);
+ }
+ push @children, $pid;
+ return { pid => $pid, path => $path };
+}
+
+my sub finish {
+ my ($job) = @_;
+ waitpid($job->{pid}, 0) == $job->{pid} or die "waitpid: $!\n";
+ die "test child failed: $?\n" if $?;
+ @children = grep { $_ != $job->{pid} } @children;
+ my $result = decode_json(get($job->{path}));
+ die $result->{fatal} if $result->{fatal};
+ return $result;
+}
+
+my sub observe {
+ my ($budget, $cmd, %param) = @_;
+ my $cache = { 'status-command-budget' => $budget };
+ # Compare debiting against the same clock samples, not scheduler time outside run().
+ my ($first, $last);
+ my $clock = $PVE::Storage::StatusCommand::CLOCK;
+ local $PVE::Storage::StatusCommand::CLOCK = sub {
+ $last = $clock->();
+ $first //= $last;
+ return $last;
+ };
+ my $start = now();
+ eval { PVE::Storage::StatusCommand::run($cache, $cmd, %param) };
+ my $err = $@;
+ my $elapsed = now() - $start;
+ eval { PVE::Storage::StatusCommand::run($cache, ['/bin/true']) } if $err;
+ return {
+ error => $err,
+ elapsed => $elapsed,
+ charged => $last - $first,
+ remaining => $cache->{'status-command-budget'},
+ cached => $err ? $@ : '',
+ };
+}
+
+my sub contender {
+ my ($budget, $tag, $delay_after_lock) = @_;
+ return spawn(
+ sub {
+ no warnings 'redefine';
+ local *PVE::Tools::run_with_timeout = sub {
+ put("$dir/$tag-waiting", '');
+ my $res = scalar $timer->(@_);
+ sleep($delay_after_lock) if $delay_after_lock;
+ return $res;
+ };
+ return observe($budget, ['/bin/sh', '-c', "echo run >> '$dir/launched'"]);
+ },
+ );
+}
+
+my sub holder {
+ my $fifo = "$dir/holder-" . ++$sequence;
+ POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!\n";
+ my $job = spawn(
+ sub {
+ sysopen(my $lock, $PVE::Storage::StatusCommand::LOCK_FILE, O_RDWR | O_CREAT, 0600)
+ or die "open lock: $!\n";
+ flock($lock, LOCK_EX) or die "flock: $!\n";
+ put("$fifo-ready", '');
+ open(my $fh, '<', $fifo) or die "open FIFO: $!\n";
+ close($fh);
+ return {};
+ },
+ );
+ await(sub { -e "$fifo-ready" }, 'holder readiness');
+ return ($job, $fifo);
+}
+
+my $standin = <<'PERL';
+use strict;
+use warnings;
+$SIG{USR1} = 'IGNORE';
+alarm(10);
+open(my $pidfile, '>', $ARGV[1]) or die $!;
+print {$pidfile} $$;
+close($pidfile);
+$| = 1;
+print "ready\n";
+print 'x' x 4096 if $ARGV[2];
+open(my $fh, '<', $ARGV[0]) or die $!;
+close($fh);
+PERL
+
+my sub abandon {
+ my ($kind, $callback) = @_;
+ my $fifo = "$dir/command-" . ++$sequence;
+ POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!\n";
+ local $PVE::Storage::StatusCommand::KILL_SIGNAL = 'USR1';
+ local $PVE::Storage::StatusCommand::OUTPUT_LIMIT = $kind eq 'output' ? 1024 : 16384;
+ # Start the short timeout only once output proves that the helper installed its signal
+ # handler and wrote its PID. Startup scheduling must not turn it into a killed child.
+ my $advance = 0;
+ my $started = now();
+ local $PVE::Storage::StatusCommand::CLOCK = sub { now() + $advance };
+ my $result = observe(
+ 10,
+ [$^X, '-e', $standin, $fifo, "$fifo-pid", $kind eq 'output'],
+ outfunc => sub {
+ $advance = 9.45 - (now() - $started) if $kind eq 'timeout';
+ $callback->() if $callback;
+ die "callback refused output\n" if $kind eq 'callback';
+ },
+ );
+ await(sub { -s "$fifo-pid" }, 'command PID');
+ my $pid = get("$fifo-pid");
+ push @commands, $pid;
+ return ($result, $pid, $fifo);
+}
+
+my sub reap {
+ my ($pid) = @_;
+ await(
+ sub {
+ PVE::Storage::StatusCommand::reap_pending();
+ return !-e "/proc/$pid/stat";
+ },
+ 'abandoned child reap',
+ );
+ @commands = grep { $_ != $pid } @commands;
+}
+
+# Both handovers export while the child is alive. The second waits for a zombie before exec so
+# import must reap it, rather than merely restore a live child's pending entry.
+if ($mode eq '--handover-start') {
+ my (undef, $pid, $fifo) = abandon('callback');
+ my $state = {
+ pending => PVE::Storage::StatusCommand::export_pending(),
+ pid => $pid,
+ fifo => $fifo,
+ };
+ put("$dir/handover-state", encode_json($state));
+ put("$dir/handover-ready", '');
+ await(sub { -e "$dir/exec-live" }, 'live exec request');
+ exec($^X, $0, '--handover-live', $dir) or die "exec: $!\n";
+} elsif ($mode eq '--handover-live') {
+ my $state = decode_json(get("$dir/handover-state"));
+ PVE::Storage::StatusCommand::import_pending($state->{pending});
+ die 'pending child lost during exec'
+ if PVE::Storage::StatusCommand::export_pending() ne $state->{pending};
+ put("$dir/handover-live", '');
+ await(sub { -e "$dir/exec-exited" }, 'exited exec request');
+ $state->{pending} = PVE::Storage::StatusCommand::export_pending();
+ release($state->{fifo});
+ await(
+ sub { get("/proc/$state->{pid}/stat") =~ m/^\d+ \(.*\) Z /s },
+ 'child exit during handover',
+ );
+ put("$dir/handover-state", encode_json($state));
+ exec($^X, $0, '--handover-exited', $dir) or die "exec: $!\n";
+} elsif ($mode eq '--handover-exited') {
+ my $state = decode_json(get("$dir/handover-state"));
+ PVE::Storage::StatusCommand::import_pending($state->{pending});
+ die 'exited child not reaped' if -e "/proc/$state->{pid}/stat";
+ die 'pending not empty' if PVE::Storage::StatusCommand::export_pending() ne "$$:";
+ put("$dir/handover-done", '');
+ POSIX::_exit(0);
+}
+
+END {
+ kill('KILL', $_) for @children, @commands;
+ waitpid($_, 0) for @children, @commands;
+}
+
+alarm(45);
+
+# Independent status and scan observers share admission, not their 15/10 second budgets or cache.
+{
+ my $fifo = "$dir/healthy";
+ POSIX::mkfifo($fifo, 0600) or die "mkfifo: $!\n";
+ my $first = spawn(
+ sub {
+ return observe(
+ 15,
+ [
+ $^X,
+ '-e',
+ '$| = 1; print "ready\n"; open(my $f, "<", $ARGV[0]) or die',
+ $fifo,
+ ],
+ outfunc => sub { put("$dir/healthy-ready", '') },
+ );
+ },
+ );
+ await(sub { -e "$dir/healthy-ready" }, 'first observer');
+ my $second = contender(10, 'healthy');
+ await(sub { -e "$dir/healthy-waiting" }, 'second observer');
+ sleep(0.12);
+ release($fifo);
+ my $a = finish($first);
+ my $b = finish($second);
+ is($a->{error}, '', 'healthy: status observer succeeds');
+ is($b->{error}, '', 'healthy: simultaneous scan observer waits and succeeds');
+ cmp_ok($b->{elapsed}, '>=', 0.1, 'healthy: real lock wait occurred');
+ cmp_ok($b->{elapsed}, '<', 1.5, 'healthy: released within wait cap');
+ cmp_ok(abs(10 - $b->{remaining} - $b->{charged}), '<', 0.00001, 'healthy: wait debited');
+ cmp_ok($a->{remaining}, '>', $b->{remaining} + 4, 'healthy: budgets remain independent');
+ is(get("$dir/launched"), "run\n", 'healthy: second observer launched exactly one command');
+ unlink("$dir/launched") or die $!;
+}
+
+{
+ my ($h, $fifo) = holder();
+ my $result = observe(10, ['/bin/sh', '-c', "touch '$dir/launched'"]);
+ is($result->{error}, $busy, 'holder: two-second cap returns busy');
+ cmp_ok($result->{elapsed}, '>=', 1.9, 'holder: did not fail at once');
+ cmp_ok($result->{elapsed}, '<', 4, 'holder: bounded wait');
+ cmp_ok(abs(10 - $result->{remaining} - $result->{charged}), '<', 0.00001, 'cap: wait debited');
+ is($result->{cached}, $busy, 'cap: first failure cached');
+ ok(!-e "$dir/launched", 'cap: timed out without forking a command');
+ release($fifo);
+ finish($h);
+}
+
+{
+ my ($h, $fifo) = holder();
+ my $waiter = spawn(
+ sub {
+ my $interrupts = 0;
+ local $SIG{USR1} = sub { $interrupts++ };
+ no warnings 'redefine';
+ local *PVE::Tools::run_with_timeout = sub {
+ put("$dir/eintr-ready", '');
+ return scalar $timer->(@_);
+ };
+ my $result = observe(1.25, ['/bin/sh', '-c', "touch '$dir/launched'"]);
+ $result->{interrupts} = $interrupts;
+ return $result;
+ },
+ );
+ await(sub { -e "$dir/eintr-ready" }, 'interruptible wait');
+ # A renewed 0.75-second deadline would follow the signal train for four seconds. Stop
+ # sending as soon as the observer finishes, so the correct path does not pay that cost.
+ my $until = now() + 4;
+ while (now() < $until && !-e $waiter->{path}) {
+ kill('USR1', $waiter->{pid});
+ sleep(0.01);
+ }
+ my $result = finish($waiter);
+ like(
+ $result->{error},
+ qr/not started - not enough time left after waiting for another status command \(shared limit: 15 seconds\)/,
+ 'short wait: own reserve is limiting',
+ );
+ cmp_ok($result->{interrupts}, '>=', 2, 'EINTR: repeated signals reached blocked observer');
+ cmp_ok($result->{elapsed}, '>=', 0.7, 'EINTR: waited for fixed deadline');
+ cmp_ok($result->{elapsed}, '<', 1.75, 'EINTR: did not renew deadline');
+ ok(!-e "$dir/launched", 'sub-second timeout: no command launched');
+ release($fifo);
+ finish($h);
+}
+
+for my $contended (0, 1) {
+ my ($h, $fifo) = $contended ? holder() : ();
+ my $timers = 0;
+ no warnings 'redefine';
+ local *PVE::Tools::run_with_timeout = sub {
+ $timers++;
+ return scalar $timer->(@_);
+ };
+ my $result = observe(0.49, ['/bin/sh', '-c', "touch '$dir/launched'"]);
+ like(
+ $result->{error},
+ $contended
+ ? qr/not started - not enough time left after waiting for another status command \(shared limit: 15 seconds\)/
+ : qr/not started - not enough time left \(shared limit: 15 seconds\)/,
+ "floor: rejects admission, contended=$contended",
+ );
+ is($timers, 0, 'floor: no nonpositive timer installed');
+ cmp_ok($result->{elapsed}, '<', 1, 'floor: rejects without waiting');
+ ok(!-e "$dir/launched", 'floor: no command launched');
+ if ($h) {
+ release($fifo);
+ finish($h);
+ }
+}
+
+{
+ my ($h, $fifo) = holder();
+ my $waiter = contender(1.5, 'late', 1.1);
+ await(sub { -e "$dir/late-waiting" }, 'late waiter');
+ release($fifo);
+ finish($h);
+ my $result = finish($waiter);
+ like(
+ $result->{error},
+ qr/not started - not enough time left \(shared limit: 15 seconds\)/,
+ 'late lock: pre-fork floor rechecked',
+ );
+ ok(!-e "$dir/launched", 'late lock: no command launched after admission delay');
+}
+
+{
+ my ($result, $pid, $fifo) = abandon('timeout');
+ like(
+ $result->{error},
+ qr/did not finish within the shared 15-second limit/,
+ 'left behind: direct child abandoned',
+ );
+ cmp_ok($result->{elapsed}, '>=', 0.64, 'cleanup: polls for the single grace');
+ cmp_ok(
+ $result->{elapsed},
+ '<',
+ 1.8,
+ 'cleanup: single grace plus scheduling, not the helper safety alarm',
+ );
+ cmp_ok(
+ abs(10 - $result->{remaining} - $result->{charged}),
+ '<',
+ 0.00001,
+ 'cleanup: grace debited',
+ );
+ is(
+ PVE::Storage::StatusCommand::export_pending(),
+ "$$:$pid=perl",
+ 'left behind: direct child pending for reaping',
+ );
+ my $reader = spawn(
+ sub {
+ my @results = map {
+ observe($_, ['/bin/sh', '-c', "touch '$dir/launched'"])
+ } (15, 10);
+ return {
+ results => \@results,
+ pending => PVE::Storage::StatusCommand::export_pending(),
+ owner => $$,
+ };
+ },
+ );
+ my $reads = finish($reader);
+ for my $read ($reads->{results}->@*) {
+ is($read->{error}, $busy, 'left behind: later process fails busy');
+ cmp_ok($read->{elapsed}, '>=', 1.9, 'left behind: each observation waits');
+ cmp_ok($read->{elapsed}, '<', 4, 'left behind: each wait is bounded');
+ }
+ is($reads->{pending}, "$reads->{owner}:", 'fork: child does not inherit pending ownership');
+ is(
+ PVE::Storage::StatusCommand::export_pending(),
+ "$$:$pid=perl",
+ 'fork: parent retains its pending child',
+ );
+ ok(!-e "$dir/launched", 'left behind: repeated calls launched zero commands');
+ release($fifo);
+ reap($pid);
+ is(PVE::Storage::StatusCommand::export_pending(), "$$:", 'reap: pending child removed');
+ is(observe(10, ['/bin/true'])->{error}, '', 'reap: next observation succeeds');
+}
+
+for my $kind ('callback', 'output') {
+ my ($result, $pid, $fifo) = abandon($kind);
+ like(
+ $result->{error},
+ $kind eq 'callback' ? qr/callback refused/ : qr/wrote more than/,
+ "early $kind: original failure retained",
+ );
+ is(
+ PVE::Storage::StatusCommand::export_pending(),
+ "$$:$pid=perl",
+ "early $kind: surviving direct child pending",
+ );
+ my $reader = contender(10, "early-$kind");
+ my $read = finish($reader);
+ is($read->{error}, $busy, "early $kind: later process fails busy");
+ cmp_ok($read->{elapsed}, '>=', 1.9, "early $kind: later observer waits");
+ cmp_ok($read->{elapsed}, '<', 4, "early $kind: bounded before original deadline");
+ ok(!-e "$dir/launched", "early $kind: no command launched");
+ release($fifo);
+ reap($pid);
+}
+
+{
+ my $signaler;
+ my $interrupts = 0;
+ local $SIG{USR1} = sub { $interrupts++ };
+ my ($result, $pid, $fifo) = abandon(
+ 'callback',
+ sub {
+ my $owner = $$;
+ $signaler = spawn(
+ sub {
+ my $until = now() + 4;
+ put("$dir/cleanup-signaler-ready", '');
+ while (now() < $until && !-e "$dir/cleanup-done") {
+ kill('USR1', $owner);
+ sleep(0.005);
+ }
+ return {};
+ },
+ );
+ await(sub { -e "$dir/cleanup-signaler-ready" }, 'cleanup signaler');
+ },
+ );
+ put("$dir/cleanup-done", '');
+ cmp_ok($interrupts, '>=', 2, 'cleanup EINTR: repeated signals delivered');
+ cmp_ok($result->{elapsed}, '>=', 0.09, 'cleanup EINTR: grace observed');
+ cmp_ok($result->{elapsed}, '<', 1.5, 'cleanup EINTR: interrupted sleeps do not renew grace');
+ cmp_ok(
+ abs(10 - $result->{remaining} - $result->{charged}),
+ '<',
+ 0.00001,
+ 'cleanup EINTR: entire grace debited',
+ );
+ finish($signaler);
+ release($fifo);
+ reap($pid);
+}
+
+{
+ my $pid = fork() // die "fork: $!\n";
+ if (!$pid) {
+ exec($^X, $0, '--handover-start', $dir) or die "exec: $!\n";
+ }
+ push @children, $pid;
+ await(sub { -e "$dir/handover-ready" }, 'handover owner readiness');
+ my $before = finish(contender(10, 'before-exec'));
+ is($before->{error}, $busy, 'exec: surviving child holds lock before exec');
+ cmp_ok($before->{elapsed}, '>=', 1.9, 'exec: observer waits before exec');
+ cmp_ok($before->{elapsed}, '<', 4, 'exec: wait before exec is bounded');
+ put("$dir/exec-live", '');
+ await(sub { -e "$dir/handover-live" }, 'live import after real exec');
+ my $after = finish(contender(10, 'after-exec'));
+ is($after->{error}, $busy, 'exec: surviving child holds lock after exec');
+ cmp_ok($after->{elapsed}, '>=', 1.9, 'exec: observer waits after exec');
+ cmp_ok($after->{elapsed}, '<', 4, 'exec: wait after exec is bounded');
+ put("$dir/exec-exited", '');
+ waitpid($pid, 0) == $pid or die "waitpid: $!\n";
+ is($?, 0, 'exec: owner verifies reaping after exit during second handover');
+ @children = grep { $_ != $pid } @children;
+ ok(-e "$dir/handover-done", 'exec: real exec/import/reap completed');
+ ok(!-e "$dir/launched", 'exec: later readers launched zero commands');
+}
+
+alarm(0);
+done_testing();
diff --git a/src/test/run_status_command_tests.pl b/src/test/run_status_command_tests.pl
new file mode 100755
index 00000000..65b4497a
--- /dev/null
+++ b/src/test/run_status_command_tests.pl
@@ -0,0 +1,189 @@
+#!/usr/bin/perl
+
+# Test the bounded status commands: the shared wait budget of a pass, failure handling, the lock
+# that a command left running keeps through its standard input, and the handover of pending command
+# records across a re-exec. The tests run real commands with an isolated lock path. The budget clock
+# can advance on output; separate contention tests exercise real-clock admission and abandonment.
+
+use strict;
+use warnings;
+
+use lib '..';
+use Fcntl qw(:DEFAULT :flock);
+use File::Temp qw(tempdir);
+use POSIX ();
+use Test::More;
+use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC);
+
+use PVE::Tools;
+use PVE::Storage::StatusCommand;
+
+my $dir = tempdir(CLEANUP => 1);
+$PVE::Storage::StatusCommand::LOCK_FILE = "$dir/lock";
+$PVE::Storage::StatusCommand::BUDGET = 1;
+
+# the real clock, plus what the test adds
+my $skipped = 0;
+$PVE::Storage::StatusCommand::CLOCK = sub { clock_gettime(CLOCK_MONOTONIC) + $skipped };
+
+# Wait until no command left behind holds the lock any more and every one was reaped, as a killed
+# command can take a moment to exit, so each case starts from a clean state.
+my sub settle {
+ for (1 .. 100) {
+ PVE::Storage::StatusCommand::reap_pending();
+ sysopen(my $fh, $PVE::Storage::StatusCommand::LOCK_FILE, O_RDWR | O_CREAT) or die "$!\n";
+ my $free = flock($fh, LOCK_EX | LOCK_NB);
+ close($fh);
+ return if $free && PVE::Storage::StatusCommand::export_pending() eq "$$:";
+ select(undef, undef, undef, 0.1);
+ }
+ die "commands left behind did not end within 10 seconds\n";
+}
+
+# Let a stand-in waiting on the FIFO $path end, without hanging if none is waiting (yet).
+my sub release {
+ my ($path) = @_;
+ for (1 .. 100) {
+ if (sysopen(my $fh, $path, O_WRONLY | O_NONBLOCK)) {
+ close($fh);
+ return;
+ }
+ die "opening '$path' failed - $!\n" if !$!{ENXIO};
+ select(undef, undef, undef, 0.1);
+ }
+ die "nothing waits on '$path'\n";
+}
+
+my sub run_lines {
+ my ($cache, @cmd) = @_;
+ my $lines = [];
+ PVE::Storage::StatusCommand::run(
+ $cache, \@cmd,
+ outfunc => sub { push $lines->@*, $_[0] },
+ errfunc => sub { },
+ );
+ return $lines;
+}
+
+is_deeply(
+ run_lines({}, '/bin/sh', '-c', 'echo one; echo two; printf three'),
+ ['one', 'two', 'three'],
+ 'output: every line, also an unterminated last one',
+);
+
+eval { run_lines({}, '/bin/sh', '-c', 'echo "vgs: no luck" >&2; exit 5') };
+is($@, "status command 'sh' failed: exit code 5 - vgs: no luck\n", 'failure: exit code and error');
+
+eval { run_lines({}, "$dir/missing") };
+like($@, qr/^status command 'missing' failed: exit code 127/, 'failure: a command that is missing');
+
+{
+ # each command takes 0.6 of the 1 second budget, by a clock that only moves when a command
+ # prints its line
+ local $PVE::Storage::StatusCommand::BUDGET = 1;
+ my $now = 0;
+ local $PVE::Storage::StatusCommand::CLOCK = sub { $now };
+ my $cache = {};
+ my $slow = sub {
+ PVE::Storage::StatusCommand::run(
+ $cache, ['/bin/echo', 'line'],
+ outfunc => sub { $now += 0.6 },
+ errfunc => sub { },
+ );
+ };
+ eval { $slow->() };
+ is($@, '', 'budget: a command within it succeeds');
+ eval { $slow->() };
+ like(
+ $@,
+ qr/not started - not enough time left \(shared limit: 1 seconds\)/,
+ 'budget: shared by the commands of a pass',
+ );
+ my $err = $@;
+ eval { run_lines($cache, '/bin/true') };
+ is($@, $err, 'budget: and every later command of the pass fails with the first error');
+}
+settle();
+
+my $start = clock_gettime(CLOCK_MONOTONIC);
+eval { run_lines({}, '/bin/sh', '-c', 'exec >/dev/null 2>&1; exec sleep 30') };
+like(
+ $@,
+ qr/did not finish within the shared 1-second limit/,
+ 'end of output: still waits for the exit, within the budget',
+);
+ok(clock_gettime(CLOCK_MONOTONIC) - $start < 10, 'end of output: and gives up at the deadline');
+settle();
+
+# a child of the command keeps the standard input, and with it the lock, after the command itself
+# was killed, like a command that cannot be killed while its I/O is stuck; it ends once the test
+# opens the FIFO for writing and closes it again
+my $release = "$dir/release";
+POSIX::mkfifo($release, 0600) or die "mkfifo failed - $!\n";
+my $holder = 'my $child = fork() // die; if ($child) { sleep 60 } else { $| = 1; print "ready\n";'
+ . ' open(my $fh, "<", $ARGV[0]) or die; <$fh> }';
+{
+ # the budget runs out once the child is in place, the real one only bounds a broken stand-in
+ local $PVE::Storage::StatusCommand::BUDGET = 30;
+ eval {
+ PVE::Storage::StatusCommand::run(
+ {}, [$^X, '-e', $holder, $release],
+ outfunc => sub { $skipped += 60 if $_[0] eq 'ready' },
+ errfunc => sub { },
+ );
+ };
+ like(
+ $@,
+ qr/did not finish within the shared 30-second limit/,
+ 'left behind: the command is abandoned',
+ );
+}
+$start = clock_gettime(CLOCK_MONOTONIC);
+my $wait;
+{
+ # Check the reserve at timer installation, without relying on a sub-second scheduler margin.
+ my $timer = \&PVE::Tools::run_with_timeout;
+ no warnings 'redefine';
+ local *PVE::Tools::run_with_timeout = sub {
+ $wait = $_[0];
+ return scalar $timer->(@_);
+ };
+ eval { run_lines({}, '/bin/true') };
+}
+like(
+ $@,
+ qr/not started - not enough time left after waiting for another status command \(shared limit: 1 seconds\)/,
+ 'descendant: inherited stdin lock uses the bounded wait',
+);
+my $elapsed = clock_gettime(CLOCK_MONOTONIC) - $start;
+ok(defined($wait) && $wait > 0 && $wait <= 0.5, 'descendant: reserves half a second');
+ok($elapsed >= 0.45 && $elapsed < 2, 'descendant: bounded wait without launching');
+release($release);
+settle();
+is_deeply(run_lines({}, '/bin/echo', 'ok'), ['ok'], 'left behind: works again once it ended');
+
+# a command stuck in I/O stays a child of this process; a child of the test stands in, until the
+# test closes the pipe it waits on
+pipe(my $hold_r, my $hold_w) or die "pipe failed - $!\n";
+my $child = fork() // die;
+if (!$child) {
+ close($hold_w);
+ sysread($hold_r, my $byte, 1);
+ POSIX::_exit(0);
+}
+close($hold_r);
+PVE::Storage::StatusCommand::import_pending("1:$child=vgs");
+is(PVE::Storage::StatusCommand::export_pending(), "$$:", 'hand-over: not from another process');
+PVE::Storage::StatusCommand::import_pending("$$:$child=vgs");
+is(PVE::Storage::StatusCommand::export_pending(), "$$:$child=vgs", 'hand-over: from this process');
+close($hold_w);
+settle();
+is(PVE::Storage::StatusCommand::export_pending(), "$$:", 'hand-over: reaped once it exited');
+
+{
+ local $PVE::Storage::StatusCommand::OUTPUT_LIMIT = 1000;
+ eval { run_lines({}, '/bin/sh', '-c', 'yes | head -c 5000') };
+ like($@, qr/wrote more than 1000 bytes/, 'output: bounded');
+}
+
+done_testing();
diff --git a/src/test/run_test_lvm_hints.pl b/src/test/run_test_lvm_hints.pl
index fc2ffff7..4eb2e224 100755
--- a/src/test/run_test_lvm_hints.pl
+++ b/src/test/run_test_lvm_hints.pl
@@ -21,9 +21,10 @@ my $CLASS = 'PVE::Storage::LVMPlugin';
my ($hinted, $all, $vgs_fails, $pvscan_fails, @calls);
my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
-$lvm->mock(
- run_command => sub {
- my ($cmd, %param) = @_;
+my $status_command = Test::MockModule->new('PVE::Storage::StatusCommand');
+$status_command->mock(
+ run_for_cache => sub {
+ my ($cache, $cmd, %param) = @_;
if ($cmd->[0] eq '/sbin/vgs') {
my $nohints = grep { $_ eq '--nohints' } @$cmd;
push @calls, $nohints ? 'vgs --nohints' : 'vgs';
diff --git a/src/test/run_test_lvm_status.pl b/src/test/run_test_lvm_status.pl
new file mode 100755
index 00000000..9f335f63
--- /dev/null
+++ b/src/test/run_test_lvm_status.pl
@@ -0,0 +1,220 @@
+#!/usr/bin/perl
+
+# Test status collection for LVM and LVM-thin storages with the bounded status commands. If any
+# command fails, all LVM results of that pass are withdrawn, including earlier ones, and the failure
+# is reported once. No further LVM commands run during that pass; storages of other types are
+# unaffected. The tests use the real status pass with mocked commands, except for one case that runs
+# a real command through the real runner.
+
+use strict;
+use warnings;
+
+use lib '..';
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use PVE::Storage;
+use PVE::Storage::ISCSIPlugin;
+use PVE::Storage::StatusCommand;
+
+# A hash that keeps the order of its keys, so the status pass goes through the storages in the
+# order a case needs.
+package OrderedHash {
+ sub TIEHASH { return bless { keys => [], values => {} }, $_[0]; }
+
+ sub STORE {
+ my ($self, $key, $value) = @_;
+ push $self->{keys}->@*, $key if !exists($self->{values}->{$key});
+ $self->{values}->{$key} = $value;
+ }
+ sub FETCH { return $_[0]->{values}->{ $_[1] }; }
+ sub EXISTS { return exists($_[0]->{values}->{ $_[1] }); }
+
+ sub DELETE {
+ my ($self, $key) = @_;
+ $self->{keys} = [grep { $_ ne $key } $self->{keys}->@*];
+ return delete($self->{values}->{$key});
+ }
+
+ sub CLEAR { $_[0]->{keys} = []; $_[0]->{values} = {}; }
+
+ sub FIRSTKEY {
+ $_[0]->{next} = 0;
+ return $_[0]->NEXTKEY();
+ }
+ sub NEXTKEY { return $_[0]->{keys}->[$_[0]->{next}++]; }
+ sub SCALAR { return scalar($_[0]->{keys}->@*); }
+}
+
+my $dir = tempdir(CLEANUP => 1);
+$PVE::Storage::StatusCommand::LOCK_FILE = "$dir/lock";
+
+my $cfg = {
+ ids => {
+ lvm1 => { type => 'lvm', vgname => 'vg1', content => { images => 1 } },
+ lvm2 => { type => 'lvm', vgname => 'vg2', content => { images => 1 } },
+ thin =>
+ { type => 'lvmthin', vgname => 'vg1', thinpool => 'pool', content => { images => 1 } },
+ local => { type => 'dir', path => $dir, content => { iso => 1 } },
+ },
+};
+
+# command outcomes of the pass by name ('vgs', 'vgs --nohints', 'pvscan', 'lvs', 'lvchange'): a
+# callback that dies, the lines to print, or { real => $cmd } to run $cmd through the real runner
+# instead
+my ($script, @calls);
+my $real_run = \&PVE::Storage::StatusCommand::run;
+my $status_command = Test::MockModule->new('PVE::Storage::StatusCommand');
+$status_command->mock(
+ run => sub {
+ my ($cache, $cmd, %param) = @_;
+ die $cache->{'status-command-error'} if defined($cache->{'status-command-error'});
+ my $name = $cmd->[0] =~ s|.*/||r;
+ $name .= ' --nohints' if grep { $_ eq '--nohints' } $cmd->@*;
+ push @calls, $name;
+ my $action = $script->{$name} // [];
+ return $real_run->($cache, $action->{real}, %param) if ref($action) eq 'HASH';
+ if (ref($action) eq 'CODE') {
+ eval { $action->() };
+ if (my $err = $@) {
+ $cache->{'status-command-error'} = $err;
+ die $err;
+ }
+ return 0;
+ }
+ $param{outfunc}->($_) for $action->@*;
+ return 0;
+ },
+);
+
+my sub pass {
+ my ($commands, $config) = @_;
+ ($script, @calls) = ($commands);
+ my @warnings;
+ local $SIG{__WARN__} = sub { push @warnings, $_[0] };
+ my $info = PVE::Storage::storage_info($config // $cfg);
+ return ($info, \@warnings);
+}
+
+# storage_info() goes through the storages in the key order of the configuration: the LVM
+# storages, then the thin pool, then an iSCSI storage
+my sub lvm_before_thin {
+ tie(my %ids, 'OrderedHash');
+ $ids{$_} = $cfg->{ids}->{$_} for qw(lvm1 lvm2 thin local);
+ $ids{san} = { type => 'iscsi', portal => '192.0.2.10', target => 'iqn.2003-01.test:t1' };
+ return { ids => \%ids };
+}
+
+my @activated;
+my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+$iscsi->mock(
+ check_connection => sub { return 1 },
+ activate_storage => sub { push @activated, $_[1]; return 1 },
+ status => sub { return (1024, 512, 512, 1) },
+);
+
+my $lvs = ['vg1:pool:1048576:twi-a-tz--::10.00:5.00::uuid::4096:0'];
+my ($info, $warnings) = pass({ vgs => ['vg1:1024:512:1', 'vg2:2048:1024:0'], lvs => $lvs });
+is_deeply(
+ [map { $info->{$_}->{active} } qw(lvm1 lvm2 thin local)],
+ [1, 1, 1, 1],
+ 'all listings answer: every storage active',
+);
+is(scalar($warnings->@*), 0, 'and nothing is warned');
+
+($info, $warnings) = pass(
+ {
+ vgs => ['vg1:1024:512:1', 'vg2:2048:1024:0'],
+ lvs =>
+ sub { die "status command 'lvs' did not finish within the shared 15-second limit\n" },
+ },
+ lvm_before_thin(),
+);
+is_deeply([grep { m/^(vgs|lvs)$/ } @calls], ['vgs', 'lvs'], 'the LVM status is collected first');
+is_deeply(
+ [map { $info->{$_}->{active} } qw(lvm1 lvm2 thin)],
+ [0, 0, 0],
+ 'a later listing fails: every LVM result of the pass is withdrawn, also those before',
+);
+is_deeply(\@activated, ['san'], 'a transport storage after the failure is still activated');
+is($info->{san}->{active}, 1, 'and active');
+is($info->{lvm1}->{total}, 0, 'with its sizes');
+is($info->{local}->{active}, 1, 'a storage of another kind is not affected');
+is_deeply(
+ $warnings,
+ [
+ "LVM status unavailable - status command 'lvs' did not finish within the shared 15-second limit\n"
+ ],
+ 'the failure is warned once for all storages',
+);
+ok(!grep({ m/lvchange|pvscan/ } @calls), 'no LVM command after the failure');
+
+($info) = pass({ vgs => sub { die "another status command is still running\n" } });
+is_deeply(
+ [map { $info->{$_}->{active} } qw(lvm1 lvm2 thin local)],
+ [0, 0, 0, 1],
+ 'the first listing fails: no LVM storage active',
+);
+is(scalar(grep { m/^vgs/ } @calls), 1, 'and only that one listing ran');
+
+($info, $warnings) = pass({
+ vgs => ['vg1:1024:512:1', 'vg2:2048:1024:0'],
+ lvs => ['vg1:pool:1048576:twi---tz--::10.00:5.00::uuid::4096:0'],
+ lvchange => sub { die "activating LV 'vg1/pool' failed: exit code 5\n" },
+});
+ok(grep({ $_ eq 'lvchange' } @calls), 'an inactive thin pool is activated');
+is_deeply(
+ [map { $info->{$_}->{active} } qw(lvm1 lvm2 thin local)],
+ [0, 0, 0, 1],
+ 'its activation fails: every LVM result withdrawn',
+);
+is_deeply(
+ $warnings,
+ ["LVM status unavailable - activating LV 'vg1/pool' failed: exit code 5\n"],
+ 'and warned once',
+);
+
+($info, $warnings) = pass({
+ vgs =>
+ { real => ['/bin/sh', '-c', 'echo vg1:1024:512:1; echo "vgs: device lost" >&2; exit 5'] },
+ lvs => $lvs,
+});
+is_deeply(
+ [map { $info->{$_}->{active} } qw(lvm1 lvm2 thin local)],
+ [0, 0, 0, 1],
+ 'a real listing prints part of its output, then fails: no partial result is used',
+);
+is_deeply(
+ $warnings,
+ ["LVM status unavailable - status command 'sh' failed: exit code 5 - vgs: device lost\n"],
+ 'and its error is warned once',
+);
+
+# the volume group of lvm2 is missing from the hinted list
+($info) = pass({ vgs => ['vg1:1024:512:1'], lvs => $lvs });
+is_deeply(
+ [map { $info->{$_}->{active} } qw(lvm1 lvm2 thin)],
+ [1, 0, 1],
+ 'a volume group missing everywhere: only that storage inactive',
+);
+($info) = pass({
+ vgs => ['vg1:1024:512:1'],
+ 'vgs --nohints' => ['vg1:1024:512:1', 'vg2:2048:1024:0'],
+ pvscan =>
+ sub { die "status command 'pvscan' did not finish within the shared 15-second limit\n" },
+ lvs => $lvs,
+});
+is_deeply(
+ [map { $info->{$_}->{active} } qw(lvm1 lvm2 thin)],
+ [0, 0, 0],
+ 'the rebuild of the hints after a successful probe fails: every LVM result withdrawn',
+);
+
+my $plain;
+my $tools = Test::MockModule->new('PVE::Tools');
+$tools->mock(run_command => sub { $plain++; return 0 });
+PVE::Storage::LVMPlugin::lvm_vgs(undef, undef, {});
+ok($plain, 'without a status pass the listing runs as before');
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 07/76] iscsi: rate-limit session rescans per target
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (5 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 06/76] storage: bound LVM status commands Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 08/76] iscsi: list the LUNs of each session, not of its SCSI host Thomas Lamprecht
` (68 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
The node-wide rescan limit lets frequent activation of one storage
starve other targets, leaving new LUNs and capacity changes unseen.
Limit rescans to once every ten seconds per target instead. Protect the
persistent timestamp with a file lock, released before rescanning.
Storage entries for the same target share the limit; unmanaged sessions
are not rescanned. Bookkeeping failures warn and skip the rescan without
making established sessions unusable.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
Whether pvestatd (the main caller, but not the only one) should rescan
as often as every 10s is another question.
src/PVE/Storage/ISCSIPlugin.pm | 54 +++++++++-----
src/test/Makefile | 3 +-
src/test/run_iscsi_rescan_tests.pl | 109 +++++++++++++++++++++++++++++
3 files changed, 146 insertions(+), 20 deletions(-)
create mode 100755 src/test/run_iscsi_rescan_tests.pl
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index 3bba27a6..20605859 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -3,7 +3,8 @@ package PVE::Storage::ISCSIPlugin;
use strict;
use warnings;
-use File::stat;
+use Digest::SHA qw(sha256_hex);
+use Fcntl qw(LOCK_EX LOCK_NB);
use IO::Dir;
use IO::File;
@@ -212,27 +213,42 @@ sub iscsi_logout {
run_command([$ISCSIADM, '--mode', 'node', '--targetname', $target, '--logout']);
}
-my $rescan_filename = "/var/run/pve-iscsi-rescan.lock";
+our $ISCSI_RESCAN_DIR = '/run';
+
+sub claim_iscsi_rescan {
+ my ($target) = @_;
+
+ my $filename = "$ISCSI_RESCAN_DIR/pve-iscsi-rescan-" . sha256_hex($target);
+ my $claimed = eval {
+ my $fh = IO::File->new($filename, '+>>') or die "cannot open '$filename' - $!\n";
+ flock($fh, LOCK_EX | LOCK_NB) or die "cannot lock '$filename' - $!\n";
+ $fh->seek(0, 0) or die "cannot seek '$filename' - $!\n";
+ my $last = $fh->getline();
+ die "cannot read '$filename' - $!\n" if $fh->error();
+ die "invalid timestamp in '$filename'\n" if defined($last) && $last !~ m/^\d+\n$/;
+ my $now = time();
+ my $elapsed = defined($last) ? $now - $last : undef;
+ my $claim = !defined($elapsed) || $elapsed < 0 || $elapsed >= 10;
+
+ if ($claim) {
+ $fh->truncate(0) or die "cannot update '$filename' - $!\n";
+ $fh->print("$now\n") && $fh->flush() or die "cannot update '$filename' - $!\n";
+ }
+ $fh->close() or die "cannot close '$filename' - $!\n";
+ $claim;
+ };
+ if (my $err = $@) {
+ warn "could not rate-limit iSCSI rescan for target '$target' - $err";
+ return 0;
+ }
+ return $claimed;
+}
sub iscsi_session_rescan {
- my $session_list = shift;
+ my ($target, $session_list) = @_;
assert_iscsi_support();
-
- my $rstat = stat($rescan_filename);
-
- if (!$rstat) {
- if (my $fh = IO::File->new($rescan_filename, "a")) {
- utime undef, undef, $fh;
- close($fh);
- }
- } else {
- my $atime = $rstat->atime;
- my $tdiff = time() - $atime;
- # avoid frequent rescans
- return if !($tdiff < 0 || $tdiff > 10);
- utime undef, undef, $rescan_filename;
- }
+ return if !claim_iscsi_rescan($target);
foreach my $session (@$session_list) {
my $cmd = [$ISCSIADM, '--mode', 'session', '--sid', $session->{session_id}, '--rescan'];
@@ -517,7 +533,7 @@ sub activate_storage {
warn $@ if $@;
} else {
# make sure we get all devices
- iscsi_session_rescan($sessions);
+ iscsi_session_rescan($scfg->{target}, $sessions);
}
}
diff --git a/src/test/Makefile b/src/test/Makefile
index 0b9b95b2..404a1524 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -40,8 +40,9 @@ test_ovf: run_ovf_tests.pl
test_volume_access: run_volume_access_tests.pl
./run_volume_access_tests.pl
-test_iscsi: run_iscsi_discovery_tests.pl run_iscsi_session_tests.pl run_iscsi_volume_tests.pl
+test_iscsi: run_iscsi_discovery_tests.pl run_iscsi_rescan_tests.pl run_iscsi_session_tests.pl run_iscsi_volume_tests.pl
./run_iscsi_discovery_tests.pl
+ ./run_iscsi_rescan_tests.pl
./run_iscsi_session_tests.pl
./run_iscsi_volume_tests.pl
diff --git a/src/test/run_iscsi_rescan_tests.pl b/src/test/run_iscsi_rescan_tests.pl
new file mode 100755
index 00000000..22d70c9c
--- /dev/null
+++ b/src/test/run_iscsi_rescan_tests.pl
@@ -0,0 +1,109 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Digest::SHA qw(sha256_hex);
+use Errno qw(EACCES ENOSPC);
+use Fcntl qw(LOCK_EX LOCK_NB);
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Storage::ISCSIPlugin;
+
+my $dir = tempdir(CLEANUP => 1);
+local $PVE::Storage::ISCSIPlugin::ISCSI_RESCAN_DIR = $dir;
+my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+my @rescans;
+my $target;
+$plugin->mock(
+ iscsi_session =>
+ sub { [{ session_id => 1, portal => 'portal' }, { session_id => 2, portal => 'portal' }] },
+ iscsi_portals => sub { ['portal'] },
+ run_command => sub {
+ my ($cmd) = @_;
+ push @rescans, $cmd->[-2];
+ is($cmd->[-1], '--rescan', 'only session rescans run');
+ my $filename = "$dir/pve-iscsi-rescan-" . sha256_hex($target);
+ open(my $fh, '>>', $filename) or die $!;
+ ok(flock($fh, LOCK_EX | LOCK_NB), 'timestamp lock is released before rescan');
+ },
+);
+
+my sub activate {
+ my ($storeid, $name) = @_;
+ $target = $name;
+ PVE::Storage::ISCSIPlugin->activate_storage($storeid, { target => $target }, {});
+}
+
+activate('first', 'iqn.first');
+is_deeply(\@rescans, [1, 2], 'all sessions of first target rescanned');
+activate('second', 'iqn.second/../unsafe');
+is_deeply(\@rescans, [1, 2, 1, 2], 'second target has its own window and a safe filename');
+activate('duplicate', 'iqn.first');
+is(scalar(@rescans), 4, 'duplicate storage shares the target allowance');
+activate('second', 'iqn.second/../unsafe');
+is(scalar(@rescans), 4, 'independent activation cache does not bypass persisted claim');
+my $filename = "$dir/pve-iscsi-rescan-" . sha256_hex('iqn.first');
+my $inode = (stat($filename))[1];
+open(my $stamp, '>', $filename) or die $!;
+print $stamp (time() - 11) . "\n";
+close($stamp) or die $!;
+activate('duplicate', 'iqn.first');
+is_deeply(\@rescans, [1, 2, 1, 2, 1, 2], 'next window rescans every session again');
+activate('second', 'iqn.second/../unsafe');
+is(scalar(@rescans), 6, 'other target remains rate limited');
+
+is((stat($filename))[1], $inode, 'timestamp updates do not replace the locked inode');
+my @files = glob("$dir/*");
+is(scalar(@files), 2, 'exactly one timestamp and lock file per target');
+
+my $io = Test::MockModule->new('IO::File');
+my $new = IO::File->can('new');
+my $print = IO::File->can('print');
+my $close = IO::File->can('close');
+my $failure = '';
+$io->mock(
+ new => sub {
+ if ($failure eq 'open') { $! = EACCES; return undef; }
+ $new->(@_);
+ },
+ print => sub {
+ if ($failure eq 'update') { $! = ENOSPC; return 0; }
+ $print->(@_);
+ },
+ close => sub {
+ my $res = $close->(@_);
+ if ($failure eq 'close') { $! = ENOSPC; return 0; }
+ return $res;
+ },
+);
+for my $case (qw(open update close lock)) {
+ my $name = "iqn.failure-$case";
+ my $file = "$dir/pve-iscsi-rescan-" . sha256_hex($name);
+ my $lock;
+ if ($case eq 'lock') {
+ open($lock, '>>', $file) or die $!;
+ flock($lock, LOCK_EX | LOCK_NB) or die $!;
+ }
+ $failure = $case;
+ my $warnings = '';
+ local $SIG{__WARN__} = sub { $warnings .= $_[0] };
+ my $before = scalar(@rescans);
+ eval { activate('failure', $name) };
+ is($@, '', "$case failure does not fail activation");
+ like(
+ $warnings,
+ qr/^could not rate-limit iSCSI rescan for target '\Q$name\E' - /,
+ "$case failure identifies best-effort bookkeeping",
+ );
+ is(scalar(@rescans), $before, "$case failure skips the rescan");
+ close($lock) if $lock;
+}
+$failure = '';
+activate('recovered', 'iqn.failure-update');
+is(scalar(@rescans), 8, 'empty timestamp after failed update permits a subsequent first claim');
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 08/76] iscsi: list the LUNs of each session, not of its SCSI host
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (6 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 07/76] iscsi: rate-limit session rescans per target Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 09/76] lvm: refuse to create a volume group on a device that holds data Thomas Lamprecht
` (67 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
The LUN list of a target is built from its sessions, but each session
picked up every LUN of its SCSI host. Software iSCSI creates one host
per session, so this did not matter there. Offload initiators can run
sessions to several targets on one host; each of their targets then
listed the LUNs of the other targets as well.
Take the LUNs below the session's own SCSI target instead, which is
where the iSCSI transport class of the kernel places them.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Storage/ISCSIPlugin.pm | 53 +++++++++++------
src/test/Makefile | 3 +-
src/test/run_iscsi_device_list_tests.pl | 78 +++++++++++++++++++++++++
3 files changed, 114 insertions(+), 20 deletions(-)
create mode 100755 src/test/run_iscsi_device_list_tests.pl
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index 20605859..536b2e94 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -18,6 +18,7 @@ use base qw(PVE::Storage::Plugin);
# iscsi helper function
my $ISCSIADM = '/usr/bin/iscsiadm';
+our $SYSFS = '/sys';
my $found_iscsi_adm_exe;
my sub assert_iscsi_support {
@@ -283,11 +284,33 @@ sub load_stable_scsi_paths {
return $stable_paths;
}
+# Offload initiators can run several sessions on one SCSI host, so the LUNs of a session are the
+# ones below its own target, not all LUNs of its host.
+my sub foreach_session_lun {
+ my ($sessiondir, $code) = @_;
+
+ dir_glob_foreach(
+ $sessiondir,
+ 'target\d+:\d+:\d+',
+ sub {
+ my ($targetdir) = @_;
+ dir_glob_foreach(
+ "$sessiondir/$targetdir",
+ '\d+:(\d+):(\d+):(\d+)',
+ sub {
+ my ($hctl, $channel, $id, $lun) = @_;
+ $code->("$sessiondir/$targetdir/$hctl", $channel, $id, $lun);
+ },
+ );
+ },
+ );
+}
+
sub iscsi_device_list {
my $res = {};
- my $dirname = '/sys/class/iscsi_session';
+ my $dirname = "$SYSFS/class/iscsi_session";
my $stable_paths = load_stable_scsi_paths();
@@ -300,39 +323,33 @@ sub iscsi_device_list {
my $target = file_read_firstline("$dirname/$ent/targetname");
return if !$target;
- my (undef, $host) = dir_glob_regex("$dirname/$ent/device", 'target(\d+):.*');
- return if !defined($host);
-
- dir_glob_foreach(
- "/sys/bus/scsi/devices",
- "$host:" . '(\d+):(\d+):(\d+)',
+ foreach_session_lun(
+ "$dirname/$ent/device",
sub {
- my ($tmp, $channel, $id, $lun) = @_;
+ my ($devdir, $channel, $id, $lun) = @_;
- my $type = file_read_firstline("/sys/bus/scsi/devices/$tmp/type");
+ my $type = file_read_firstline("$devdir/type");
return if !defined($type) || $type ne '0'; # list disks only
my $bdev;
- if (-d "/sys/bus/scsi/devices/$tmp/block") { # newer kernels
- (undef, $bdev) =
- dir_glob_regex("/sys/bus/scsi/devices/$tmp/block/", '([A-Za-z]\S*)');
+ if (-d "$devdir/block") { # newer kernels
+ (undef, $bdev) = dir_glob_regex("$devdir/block/", '([A-Za-z]\S*)');
} else {
- (undef, $bdev) =
- dir_glob_regex("/sys/bus/scsi/devices/$tmp", 'block:(\S+)');
+ (undef, $bdev) = dir_glob_regex($devdir, 'block:(\S+)');
}
return if !$bdev;
#check multipath
- if (-d "/sys/block/$bdev/holders") {
+ if (-d "$SYSFS/block/$bdev/holders") {
my $multipathdev =
- dir_glob_regex("/sys/block/$bdev/holders", '[A-Za-z]\S*');
+ dir_glob_regex("$SYSFS/block/$bdev/holders", '[A-Za-z]\S*');
$bdev = $multipathdev if $multipathdev;
}
my $blockdev = $stable_paths->{$bdev};
return if !$blockdev;
- my $size = file_read_firstline("/sys/block/$bdev/size");
+ my $size = file_read_firstline("$SYSFS/block/$bdev/size");
return if !$size;
my $volid = "$channel.$id.$lun.$blockdev";
@@ -345,8 +362,6 @@ sub iscsi_device_list {
'id' => int($id),
'lun' => int($lun),
};
-
- #print "TEST: $target $session $host,$bus,$tg,$lun $blockdev\n";
},
);
diff --git a/src/test/Makefile b/src/test/Makefile
index 404a1524..24abdebc 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -40,9 +40,10 @@ test_ovf: run_ovf_tests.pl
test_volume_access: run_volume_access_tests.pl
./run_volume_access_tests.pl
-test_iscsi: run_iscsi_discovery_tests.pl run_iscsi_rescan_tests.pl run_iscsi_session_tests.pl run_iscsi_volume_tests.pl
+test_iscsi: run_iscsi_discovery_tests.pl run_iscsi_rescan_tests.pl run_iscsi_device_list_tests.pl run_iscsi_session_tests.pl run_iscsi_volume_tests.pl
./run_iscsi_discovery_tests.pl
./run_iscsi_rescan_tests.pl
+ ./run_iscsi_device_list_tests.pl
./run_iscsi_session_tests.pl
./run_iscsi_volume_tests.pl
diff --git a/src/test/run_iscsi_device_list_tests.pl b/src/test/run_iscsi_device_list_tests.pl
new file mode 100755
index 00000000..6df3a416
--- /dev/null
+++ b/src/test/run_iscsi_device_list_tests.pl
@@ -0,0 +1,78 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use File::Path qw(make_path);
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Storage::ISCSIPlugin;
+
+my $root = tempdir(CLEANUP => 1);
+local $PVE::Storage::ISCSIPlugin::SYSFS = "$root/sys";
+
+my sub put {
+ my ($path, $value) = @_;
+ my ($dir) = "$root/sys/$path" =~ m{^(.*)/[^/]+$};
+ make_path($dir);
+ open(my $fh, '>', "$root/sys/$path") or die $!;
+ print $fh "$value\n";
+ close($fh) or die $!;
+}
+
+# The sysfs layout of the kernel's iSCSI transport class: the SCSI target and its LUNs are children
+# of the session device, and the SCSI bus lists the LUNs by H:C:T:L.
+my sub session {
+ my ($sid, $host, $target, @luns) = @_;
+ my $sessiondir = "devices/platform/host$host/session$sid";
+ make_path("$root/sys/$sessiondir");
+ put("class/iscsi_session/session$sid/targetname", $target);
+ symlink("$root/sys/$sessiondir", "$root/sys/class/iscsi_session/session$sid/device")
+ or die $!;
+ for my $lun (@luns) {
+ my ($hctl, $dev) = @$lun;
+ my ($tdir) = $hctl =~ m/^(\d+:\d+:\d+):\d+$/;
+ my $devdir = "$sessiondir/target$tdir/$hctl";
+ put("$devdir/type", '0');
+ make_path("$root/sys/$devdir/block/$dev");
+ make_path("$root/sys/bus/scsi/devices");
+ symlink("$root/sys/$devdir", "$root/sys/bus/scsi/devices/$hctl") or die $!;
+ put("block/$dev/size", '2048');
+ make_path("$root/sys/block/$dev/holders");
+ }
+}
+
+# An offload initiator with two sessions on one SCSI host, a software initiator session on a host
+# of its own, and a session without LUNs.
+session(1, 7, 'iqn.target-a', ['7:0:0:1', 'sda']);
+session(2, 7, 'iqn.target-b', ['7:0:1:1', 'sdb'], ['7:0:1:2', 'sdc']);
+session(3, 8, 'iqn.target-c', ['8:0:0:0', 'sdd']);
+session(4, 9, 'iqn.target-d');
+
+my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+$plugin->mock(
+ load_stable_scsi_paths => sub {
+ return { map { $_ => "scsi-3$_" } qw(sda sdb sdc sdd) };
+ },
+);
+
+my $list = PVE::Storage::ISCSIPlugin::iscsi_device_list();
+is_deeply(
+ { map { $_ => [sort keys $list->{$_}->%*] } keys %$list },
+ {
+ 'iqn.target-a' => ['0.0.1.scsi-3sda'],
+ 'iqn.target-b' => ['0.1.1.scsi-3sdb', '0.1.2.scsi-3sdc'],
+ 'iqn.target-c' => ['0.0.0.scsi-3sdd'],
+ },
+ 'each target lists the LUNs of its own session, also on a shared SCSI host',
+);
+is_deeply(
+ $list->{'iqn.target-b'}->{'0.1.2.scsi-3sdc'},
+ { format => 'raw', size => 2048 * 512, vmid => 0, channel => 0, id => 1, lun => 2 },
+ 'LUN address and size',
+);
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 09/76] lvm: refuse to create a volume group on a device that holds data
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (7 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 08/76] iscsi: list the LUNs of each session, not of its SCSI host Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 10/76] lvm: refuse a base volume of a target that serves disk images Thomas Lamprecht
` (66 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
Base-volume initialization must not destroy existing file systems,
partition tables, or volume groups. Probe immediately before writing
with bounded commands, refusing creation if a probe fails.
Read PV membership without LVM hints and require explicit evidence that
a PV is orphaned. Missing group metadata, including the unknown-name
sentinel, is not enough. A PV already in the requested group counts as
created; a confirmed orphan left by a failed attempt can be initialized
again.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Diskmanage.pm | 16 ++
src/PVE/Storage/LVMPlugin.pm | 106 +++++++-
src/test/Makefile | 5 +-
src/test/lvm-fixtures/README.md | 20 ++
.../lab1-nomda-orphan-derived.colon | 2 +
src/test/lvm-fixtures/lab1-nomda-orphan.txt | 23 ++
src/test/run_test_lvm_create.pl | 243 ++++++++++++++++++
7 files changed, 407 insertions(+), 8 deletions(-)
create mode 100644 src/test/lvm-fixtures/README.md
create mode 100644 src/test/lvm-fixtures/lab1-nomda-orphan-derived.colon
create mode 100644 src/test/lvm-fixtures/lab1-nomda-orphan.txt
create mode 100755 src/test/run_test_lvm_create.pl
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index d5bd1473..4185d2f7 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -73,6 +73,22 @@ sub init_disk {
return 1;
}
+sub parse_wipefs_signatures {
+ my ($json) = @_;
+
+ my $probe = decode_json($json);
+ die "Malformed wipefs output\n"
+ if ref($probe) ne 'HASH' || ref($probe->{signatures}) ne 'ARRAY';
+ for my $signature ($probe->{signatures}->@*) {
+ die "Malformed wipefs signature\n"
+ if ref($signature) ne 'HASH'
+ || !defined($signature->{type})
+ || ref($signature->{type})
+ || $signature->{type} !~ m/^[a-zA-Z0-9_.+-]+$/;
+ }
+ return $probe->{signatures};
+}
+
sub disk_is_used {
my ($disk) = @_;
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index eb9806b2..26908d7e 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -11,6 +11,7 @@ use JSON;
use List::Util qw(max);
use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC);
+use PVE::Diskmanage;
use PVE::Exception qw(raise_param_exc);
use PVE::Format qw(render_bytes render_duration);
use PVE::INotify;
@@ -55,25 +56,36 @@ my sub fork_cleanup_worker {
}
sub lvm_pv_info {
- my ($device) = @_;
+ my ($device, $require_pv) = @_;
die "no device specified" if !$device;
my $has_label = 0;
+ my $file_result;
- my $cmd = ['/usr/bin/file', '-L', '-s', $device];
+ my $cmd = ['/usr/bin/file', '-E', '-L', '-s', $device];
run_command(
$cmd,
+ timeout => 10,
outfunc => sub {
my $line = shift;
+ die "unable to probe '$device' - malformed file output\n"
+ if defined($file_result) || $line !~ m/^\Q$device\E:\s+\S/;
+ $file_result = $line;
$has_label = 1 if $line =~ m/LVM2/;
},
);
+ die "unable to probe '$device' - empty file output\n" if !defined($file_result);
- return undef if !$has_label;
+ return undef if !$has_label && !$require_pv;
$cmd = [
'/sbin/pvs',
+ '--nohints',
+ '--readonly',
+ '--devices',
+ $device,
+ '--binary',
'--separator',
':',
'--noheadings',
@@ -82,19 +94,30 @@ sub lvm_pv_info {
'--unbuffered',
'--nosuffix',
'--options',
- 'pv_name,pv_size,vg_name,pv_uuid',
+ 'pv_name,pv_size,vg_name,pv_uuid,pv_in_use',
$device,
];
my $pvinfo;
run_command(
$cmd,
+ timeout => 10,
outfunc => sub {
my $line = shift;
$line = trim($line);
- my ($pvname, $size, $vgname, $uuid) = split(':', $line);
+ my ($pvname, $size, $vgname, $uuid, $in_use, @extra) = split(':', $line, -1);
+ # LVM names a device-mapper device by its /dev/mapper link, not the dm node
+ my $listed = defined($pvname) ? abs_path($pvname) // $pvname : '';
+ die "unable to probe '$device' - malformed pvs output\n"
+ if @extra
+ || !defined($in_use)
+ || $in_use !~ m/^(?:0|1|unknown|)$/
+ || $listed ne (abs_path($device) // $device)
+ || $size !~ m/^\d+(?:\.\d+)?$/
+ || ($vgname ne '[unknown]' && $vgname !~ m/^[a-zA-Z0-9+_.-]*$/)
+ || $uuid !~ m/^[a-zA-Z0-9]{6}(?:-[a-zA-Z0-9]{4}){5}-[a-zA-Z0-9]{6}$/;
die "found multiple pvs entries for device '$device'\n"
if $pvinfo;
@@ -104,10 +127,12 @@ sub lvm_pv_info {
size => int($size),
vgname => $vgname,
uuid => $uuid,
+ in_use => $in_use,
};
},
);
+ die "unable to probe '$device' - no PV found\n" if !$pvinfo;
return $pvinfo;
}
@@ -124,13 +149,80 @@ sub clear_first_sector {
sub lvm_create_volume_group {
my ($device, $vgname, $shared) = @_;
- my $res = lvm_pv_info($device);
+ my $requested_device = $device;
+ $device = PVE::Diskmanage::verify_blockdev_path($device);
+ my $refuse_data = sub {
+ my (@types) = @_;
+ die "device '$device' contains data ("
+ . join(', ', sort @types)
+ . ") - wipe it first if it is no longer needed, for example with 'pvesh set /nodes/"
+ . PVE::INotify::nodename()
+ . "/disks/wipedisk --disk $requested_device'\n";
+ };
- if ($res->{vgname}) {
+ my $sysdir = '/sys/class/block/' . basename($device);
+ my $uuid = file_read_firstline("$sysdir/dm/uuid");
+ my $mounted = PVE::Diskmanage::is_mounted($device);
+ die "unable to create volume group - '$mounted' is mounted\n" if $mounted;
+ if (my $held = PVE::Diskmanage::has_holder($device)) {
+ # kpartx maps the partitions of a device-mapper device as its holders. Such holders indicate
+ # a partition table, not a user.
+ my @holders;
+ PVE::Tools::dir_glob_foreach("$sysdir/holders", '[^.].*', sub { push @holders, $_[0] });
+ my $partitions = defined($uuid) && @holders && !grep {
+ (file_read_firstline("/sys/class/block/$_/dm/uuid") // '') !~ m/^part\d+-\Q$uuid\E$/
+ } @holders;
+ $refuse_data->('partitions') if $partitions;
+ die "unable to create volume group - another block device uses '$held'\n";
+ }
+
+ my $size = file_read_firstline("$sysdir/size");
+ die "unable to probe '$device' - device is unavailable\n"
+ if !defined($size) || $size !~ m/^\d+$/ || !$size;
+ die "unable to probe '$device' - missing device mapper identity\n"
+ if basename($device) =~ m/^dm-\d+$/ && !defined($uuid);
+ if (defined($uuid) && $uuid =~ m/^mpath-/) {
+ # A queueing map without a running path can stall even a read-only probe.
+ my $running = 0;
+ PVE::Tools::dir_glob_foreach(
+ "$sysdir/slaves",
+ '[^.].*',
+ sub {
+ my $state = file_read_firstline("$sysdir/slaves/$_[0]/device/state");
+ $running = 1 if defined($state) && $state eq 'running';
+ },
+ );
+ die "unable to probe '$device' - multipath map has no running path\n" if !$running;
+ }
+
+ my $output = '';
+ run_command(
+ ['wipefs', '--no-act', '--json', $device],
+ timeout => 10,
+ outfunc => sub { $output .= "$_[0]\n" },
+ );
+ my $probe = eval { PVE::Diskmanage::parse_wipefs_signatures($output) };
+ if (my $err = $@) {
+ my $kind = $err eq "Malformed wipefs signature\n" ? 'signature' : 'output';
+ die "unable to probe '$device' - malformed wipefs $kind\n";
+ }
+ my %signatures = map { $_->{type} => 1 } $probe->@*;
+ my $res = lvm_pv_info($device, $signatures{LVM2_member});
+
+ # A PV without metadata copies can belong to a VG whose other devices are not visible. The
+ # sentinel is not a VG name or orphan proof, even if the use flag contradicts it.
+ die "device '$device' belongs to a volume group whose metadata is not visible on this node\n"
+ if $res
+ && ($res->{vgname} eq '[unknown]' || (!$res->{vgname} && $res->{in_use} ne '0'));
+
+ if ($res && $res->{vgname}) {
return if $res->{vgname} eq $vgname; # already created
die "device '$device' is already used by volume group '$res->{vgname}'\n";
}
+ delete $signatures{LVM2_member} if $res && !$res->{vgname};
+ $refuse_data->(keys %signatures) if %signatures;
+
clear_first_sector($device); # else pvcreate fails
# we use --metadatasize 250k, which reseults in "pe_start = 512"
diff --git a/src/test/Makefile b/src/test/Makefile
index 24abdebc..1cc8f00e 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_bwlimit test_plugin test_ovf test_volume_access test_san_libraries
+test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_bwlimit test_plugin test_ovf test_volume_access test_san_libraries
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -9,6 +9,9 @@ test_lvmplugin: run_test_lvmplugin.pl run_test_lvm_throughput.pl
./run_test_lvmplugin.pl
./run_test_lvm_throughput.pl
+test_lvm_create: run_test_lvm_create.pl
+ ./run_test_lvm_create.pl
+
test_lvm_lock_retry: run_test_lvm_lock_retry.pl
./run_test_lvm_lock_retry.pl
diff --git a/src/test/lvm-fixtures/README.md b/src/test/lvm-fixtures/README.md
new file mode 100644
index 00000000..d024eacf
--- /dev/null
+++ b/src/test/lvm-fixtures/README.md
@@ -0,0 +1,20 @@
+# LVM missing-metadata fixtures
+
+`lab1-nomda-orphan.txt` is the unchanged lvm2 2.03.31 lab capture from
+`netapp-lab-20ee/drills/lvm-fixtures/lab1-nomda-orphan.txt`. It includes a genuine orphan and a PV
+with no metadata copies whose metadata-bearing peer is not visible on the observing node.
+
+The wipe and scan tests extract the JSON between `== devices B,C` and `== full nohints` without
+rewriting it. Both log warnings remain in the command output supplied to the parsers. The capture
+also contains a full-report row summary and hinted output; those sections are preserved for
+provenance, not passed to the JSON parsers.
+
+`lab1-nomda-orphan-derived.colon` is a derived create-probe fixture, not captured command output.
+It preserves the device names, VG names, UUIDs, and binary use flags from the captured JSON. The
+`pv_size` column is a synthetic 1024 KiB because the captured command did not request a size.
+Each line represents a separate device-restricted create probe, not a combined command response.
+
+These fixtures establish the `[unknown]` VG spelling with use flag `1` and an empty VG with use
+flag `0`. Contradictory flags, disappeared aliases, duplicate UUIDs, and malformed fields use
+synthetic tests. Neither additional VG sentinels nor a `[unknown]` device name are established by
+this capture.
diff --git a/src/test/lvm-fixtures/lab1-nomda-orphan-derived.colon b/src/test/lvm-fixtures/lab1-nomda-orphan-derived.colon
new file mode 100644
index 00000000..44fb154e
--- /dev/null
+++ b/src/test/lvm-fixtures/lab1-nomda-orphan-derived.colon
@@ -0,0 +1,2 @@
+/dev/mapper/360014056dbd1054071a4de3bd8460ae6:1024::YNV8yf-k7Ty-yCya-Jt9U-9Q5P-OMgB-EviHBt:0
+/dev/mapper/36001405c5a6706aeb1b40c89ba3363eb:1024:[unknown]:EqLIyc-SBrd-vjIE-TB66-ZRRA-JXvu-yHq011:1
diff --git a/src/test/lvm-fixtures/lab1-nomda-orphan.txt b/src/test/lvm-fixtures/lab1-nomda-orphan.txt
new file mode 100644
index 00000000..de1cd259
--- /dev/null
+++ b/src/test/lvm-fixtures/lab1-nomda-orphan.txt
@@ -0,0 +1,23 @@
+== devices B,C
+ {
+ "report": [
+ {
+ "pv": [
+ {"pv_name":"/dev/mapper/360014056dbd1054071a4de3bd8460ae6", "vg_name":"", "pv_uuid":"YNV8yf-k7Ty-yCya-Jt9U-9Q5P-OMgB-EviHBt", "pv_in_use":"0"},
+ {"pv_name":"/dev/mapper/36001405c5a6706aeb1b40c89ba3363eb", "vg_name":"[unknown]", "pv_uuid":"EqLIyc-SBrd-vjIE-TB66-ZRRA-JXvu-yHq011", "pv_in_use":"1"}
+ ]
+ }
+ ]
+ ,
+ "log": [
+ {"log_seq_num":"1", "log_type":"warn", "log_context":"processing", "log_object_type":"orphan", "log_object_name":"lvm2", "log_object_id":"", "log_object_group":"", "log_object_group_id":"", "log_message":"WARNING: PV /dev/mapper/36001405c5a6706aeb1b40c89ba3363eb is marked in use but no VG was found using it.", "log_errno":"0", "log_ret_code":"0"},
+ {"log_seq_num":"2", "log_type":"warn", "log_context":"processing", "log_object_type":"orphan", "log_object_name":"lvm2", "log_object_id":"", "log_object_group":"", "log_object_group_id":"", "log_message":"WARNING: PV /dev/mapper/36001405c5a6706aeb1b40c89ba3363eb might need repairing.", "log_errno":"0", "log_ret_code":"0"}
+ ]
+ }
+== full nohints (B, C rows)
+[{'pv_name': '/dev/mapper/360014056dbd1054071a4de3bd8460ae6', 'vg_name': '', 'pv_uuid': 'YNV8yf-k7Ty-yCya-Jt9U-9Q5P-OMgB-EviHBt', 'pv_in_use': '0'}, {'pv_name': '/dev/mapper/36001405c5a6706aeb1b40c89ba3363eb', 'vg_name': '[unknown]', 'pv_uuid': 'EqLIyc-SBrd-vjIE-TB66-ZRRA-JXvu-yHq011', 'pv_in_use': '1'}]
+
+== hinted B
+ WARNING: PV /dev/mapper/36001405c5a6706aeb1b40c89ba3363eb is marked in use but no VG was found using it.
+ WARNING: PV /dev/mapper/36001405c5a6706aeb1b40c89ba3363eb might need repairing.
+ /dev/mapper/36001405c5a6706aeb1b40c89ba3363eb [unknown] used
diff --git a/src/test/run_test_lvm_create.pl b/src/test/run_test_lvm_create.pl
new file mode 100755
index 00000000..2982cda4
--- /dev/null
+++ b/src/test/run_test_lvm_create.pl
@@ -0,0 +1,243 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use FindBin qw($Bin);
+use JSON;
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Storage::LVMPlugin;
+
+my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
+my $disk = Test::MockModule->new('PVE::Diskmanage');
+my $tools = Test::MockModule->new('PVE::Tools');
+my $inotify = Test::MockModule->new('PVE::INotify');
+$inotify->mock(nodename => sub { 'testnode' });
+my ($mounted, $held, $map, $running, $size, $signatures, $label, $pv, $fail, $bad_json);
+my ($holder_uuid, $pv_alias);
+my @writes;
+my @probes;
+$disk->mock(
+ verify_blockdev_path => sub { '/dev/sda' },
+ is_mounted => sub { $mounted },
+ has_holder => sub { $held },
+);
+$tools->mock(dir_glob_foreach => sub { $_[2]->($_[0] =~ m{/holders$} ? 'dm-5' : 'sdb') });
+$lvm->mock(
+ file_read_firstline => sub {
+ return $size if $_[0] =~ m{/size$};
+ return $holder_uuid if $_[0] =~ m{/dm-5/dm/uuid$};
+ return $map ? 'mpath-3600abc' : undef if $_[0] =~ m{/uuid$};
+ return $running ? 'running' : 'offline' if $_[0] =~ m{/state$};
+ die "unexpected read $_[0]";
+ },
+ clear_first_sector => sub { push @writes, 'clear' },
+ # pvs lists a multipath map by its /dev/mapper link
+ abs_path => sub { $_[0] eq $pv_alias ? '/dev/sda' : $_[0] },
+ run_command => sub {
+ my ($cmd, %opt) = @_;
+ my $tool = $cmd->[0];
+ if ($tool =~ m{(?:file|pvs|wipefs)$}) {
+ push @probes, $tool;
+ is($opt{timeout}, 10, "$tool has a timeout");
+ die "$tool failed or timed out\n" if ($fail // '') eq $tool;
+ if ($tool eq 'wipefs') {
+ $opt{outfunc}->($bad_json // encode_json({ signatures => $signatures }));
+ } elsif ($tool eq '/usr/bin/file') {
+ $opt{outfunc}->($label);
+ } else {
+ is_deeply(
+ [@$cmd[1 .. 6]],
+ [
+ '--nohints',
+ '--readonly',
+ '--devices',
+ '/dev/sda',
+ '--binary',
+ '--separator',
+ ],
+ 'PV lookup reads the candidate without hints or activation',
+ );
+ ok(
+ grep($_ eq 'pv_name,pv_size,vg_name,pv_uuid,pv_in_use', @$cmd),
+ 'PV use requested',
+ );
+ $opt{outfunc}->($pv) if defined($pv);
+ }
+ } else {
+ push @writes, $tool;
+ }
+ },
+);
+
+my sub reset_case {
+ ($mounted, $held, $map, $running, $size) = (undef, undef, 0, 1, 1024);
+ $signatures = [];
+ $label = '/dev/sda: data';
+ $pv = undef;
+ $pv_alias = '/dev/mapper/3600abc';
+ $fail = undef;
+ $bad_json = undef;
+ @writes = ();
+ @probes = ();
+}
+
+my sub create {
+ eval { PVE::Storage::LVMPlugin::lvm_create_volume_group('/dev/disk/by-id/test', 'test', 1) };
+ return $@;
+}
+
+reset_case();
+is(create(), '', 'blank device accepted');
+is_deeply(\@writes, ['clear', '/sbin/pvcreate', '/sbin/vgcreate'], 'writes only after probes');
+
+for my $case (
+ ['mount', sub { $mounted = '/dev/sda1' }, qr/is mounted/],
+ ['holder', sub { $held = '/dev/sda' }, qr{another block device uses '/dev/sda'}],
+ ['unavailable', sub { $size = 0 }, qr/unavailable/],
+ ['lost paths', sub { $map = 1; $running = 0 }, qr/no running path/],
+) {
+ reset_case();
+ $case->[1]->();
+ like(create(), $case->[2], "$case->[0] refused");
+ is_deeply(\@probes, [], "$case->[0]: no device reads");
+ is_deeply(\@writes, [], "$case->[0]: no writes");
+}
+
+reset_case();
+$map = 1;
+is(create(), '', 'map with a running slave accepted');
+
+reset_case();
+$signatures = [{ type => 'gpt' }, { type => 'ext4' }];
+is(
+ create(),
+ "device '/dev/sda' contains data (ext4, gpt) - wipe it first if it is no longer needed,"
+ . " for example with 'pvesh set /nodes/testnode/disks/wipedisk --disk /dev/disk/by-id/test'\n",
+ 'refusal names signatures, local node, and the caller device reference',
+);
+is_deeply(\@writes, [], 'signatures never overwritten');
+
+my $uuid = 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg';
+for my $vg ('test', 'other', 'unknown', '') {
+ reset_case();
+ $signatures = [{ type => 'LVM2_member' }];
+ $label = '/dev/sda: LVM2 PV';
+ $pv = "/dev/sda:1024:$vg:$uuid:" . ($vg ? '1' : '0');
+ if ($vg && $vg ne 'test') {
+ like(create(), qr/already used by volume group/, 'foreign VG refused');
+ } else {
+ is(create(), '', $vg ? 'same VG accepted' : 'orphan PV accepted');
+ }
+ is(scalar(@writes), $vg ? 0 : 3, 'only the orphan PV is initialized');
+}
+
+for my $vg ('', '[unknown]') {
+ for my $used ('1', '', 'unknown', $vg ? ('0') : ()) {
+ reset_case();
+ $signatures = [{ type => 'LVM2_member' }];
+ $pv = "/dev/mapper/3600abc:1024:$vg:$uuid:$used";
+ like(
+ create(),
+ qr/belongs to a volume group whose metadata is not visible/,
+ "VG '$vg' with use flag '$used' refused before named-VG handling",
+ );
+ is_deeply(\@writes, [], 'uncertain orphan never initialized');
+ }
+}
+reset_case();
+$signatures = [{ type => 'LVM2_member' }];
+$pv = "/dev/sda:1024:[unknown]:$uuid:1";
+eval { PVE::Storage::LVMPlugin::lvm_create_volume_group('/dev/sda', '[unknown]', 1) };
+like($@, qr/metadata is not visible/, 'sentinel cannot match the requested VG as already created');
+is_deeply(\@writes, [], 'sentinel match causes no writes');
+
+subtest 'derived colon-format no-metadata PV and orphan probes' => sub {
+ my $fixture = PVE::Tools::file_get_contents(
+ "$Bin/lvm-fixtures/lab1-nomda-orphan-derived.colon",
+ );
+ my @lines = split('\n', $fixture);
+ is(scalar(@lines), 2, 'derived fixture contains one probe for each captured PV');
+ for my $line (@lines) {
+ reset_case();
+ $signatures = [{ type => 'LVM2_member' }];
+ $label = '/dev/sda: LVM2 PV';
+ $pv = $line;
+ my @fields = split(':', $line, -1);
+ $pv_alias = $fields[0];
+ if ($fields[2] eq '[unknown]') {
+ like(
+ create(),
+ qr/belongs to a volume group whose metadata is not visible/,
+ 'derived no-metadata probe receives the specific refusal',
+ );
+ is_deeply(\@writes, [], 'derived no-metadata probe causes no writes');
+ } else {
+ is(create(), '', 'derived genuine orphan probe permits creation');
+ is_deeply(
+ \@writes,
+ ['clear', '/sbin/pvcreate', '/sbin/vgcreate'],
+ 'only the derived genuine orphan reaches the mocked writes',
+ );
+ }
+ }
+};
+
+# kpartx maps the partitions of a map as its holders, which indicates a partition table
+reset_case();
+($map, $held, $holder_uuid) = (1, '/dev/sda', 'part1-mpath-3600abc');
+like(
+ create(),
+ qr{contains data \(partitions\) - wipe it first .* --disk /dev/disk/by-id/test'},
+ 'partition maps of the map refused as data',
+);
+is_deeply(\@writes, [], 'partition maps: no writes');
+($map, $held, $holder_uuid) = (1, '/dev/sda', 'LVM-otherlv');
+like(create(), qr{another block device uses '/dev/sda'}, 'another holder refused as a user');
+
+reset_case();
+$signatures = [{ type => 'LVM2_member' }];
+$label = '/dev/sda: LVM2 PV';
+$pv = "/dev/mapper/3600abc:1024:test:$uuid:1";
+is(create(), '', 'same VG on a map listed by its mapper name accepted');
+is(scalar(@writes), 0, 'same VG on a map: no writes');
+
+for my $case (
+ ['missing PV', sub { $pv = undef }, qr/no PV found/],
+ ['wrong PV', sub { $pv = "/dev/sdb:1024::$uuid:0" }, qr/malformed pvs/],
+ ['bad PV size', sub { $pv = "/dev/sda:nan::$uuid:0" }, qr/malformed pvs/],
+ ['missing PV use', sub { $pv = "/dev/sda:1024::$uuid" }, qr/malformed pvs/],
+ ['invalid PV use', sub { $pv = "/dev/sda:1024::$uuid:maybe" }, qr/malformed pvs/],
+ ['sentinel missing use', sub { $pv = "/dev/sda:1024:[unknown]:$uuid" }, qr/malformed pvs/],
+ [
+ 'sentinel invalid use',
+ sub { $pv = "/dev/sda:1024:[unknown]:$uuid:maybe" },
+ qr/malformed pvs/,
+ ],
+ ['invalid VG sentinel', sub { $pv = "/dev/sda:1024:[missing]:$uuid:0" }, qr/malformed pvs/],
+ ['missing device', sub { $pv = "[unknown]:1024:[unknown]:$uuid:1" }, qr/malformed pvs/],
+ [
+ 'extra signature',
+ sub { push @$signatures, { type => 'gpt' } },
+ qr/contains data \(gpt\)/,
+ ],
+ ['bad file', sub { $label = 'garbage' }, qr/malformed file/],
+ ['bad JSON', sub { $bad_json = '{}' }, qr/malformed wipefs/],
+ ['bad signature', sub { $signatures = [{}] }, qr/malformed wipefs/],
+ map {
+ my $tool = $_;
+ ["failed $tool", sub { $fail = $tool }, qr/failed or timed out/]
+ } ('wipefs', '/usr/bin/file', '/sbin/pvs'),
+) {
+ reset_case();
+ $signatures = [{ type => 'LVM2_member' }];
+ $pv = "/dev/sda:1024::$uuid:0";
+ $case->[1]->();
+ like(create(), $case->[2], "$case->[0] refused");
+ is_deeply(\@writes, [], "$case->[0]: no writes");
+}
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 10/76] lvm: refuse a base volume of a target that serves disk images
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (8 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 09/76] lvm: refuse to create a volume group on a device that holds data Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 11/76] iscsi: refuse virtual devices other than multipath maps as volumes Thomas Lamprecht
` (65 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
A LUN can be a guest disk without any signature. Refuse LVM
initialization if another storage uses its target for disk images: iSCSI
with content images, iSCSI direct, or ZFS over iSCSI. Check all storages
using the target, regardless of their state or node restriction.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Storage/LVMPlugin.pm | 11 +++++
src/test/Makefile | 3 +-
src/test/run_test_lvm_base_conflicts.pl | 62 +++++++++++++++++++++++++
3 files changed, 75 insertions(+), 1 deletion(-)
create mode 100755 src/test/run_test_lvm_base_conflicts.pl
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index 26908d7e..d718d8f8 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -1041,6 +1041,17 @@ sub on_add_hook {
my $basecfg = PVE::Storage::storage_config($cfg, $baseid, 1);
die "base storage ID '$baseid' does not exist\n" if !$basecfg;
+ if (my $target = $basecfg->{target}) {
+ for my $sid (sort keys $cfg->{ids}->%*) {
+ my $other = $cfg->{ids}->{$sid};
+ next if ($other->{target} // '') ne $target;
+ die "storage '$sid' serves disk images from this target\n"
+ if $other->{content}->{images}
+ || $other->{type} eq 'iscsidirect'
+ || $other->{type} eq 'zfs';
+ }
+ }
+
# we only support iscsi for now
die "unsupported base type '$basecfg->{type}'"
if $basecfg->{type} ne 'iscsi';
diff --git a/src/test/Makefile b/src/test/Makefile
index 1cc8f00e..415d62bf 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -9,8 +9,9 @@ test_lvmplugin: run_test_lvmplugin.pl run_test_lvm_throughput.pl
./run_test_lvmplugin.pl
./run_test_lvm_throughput.pl
-test_lvm_create: run_test_lvm_create.pl
+test_lvm_create: run_test_lvm_create.pl run_test_lvm_base_conflicts.pl
./run_test_lvm_create.pl
+ ./run_test_lvm_base_conflicts.pl
test_lvm_lock_retry: run_test_lvm_lock_retry.pl
./run_test_lvm_lock_retry.pl
diff --git a/src/test/run_test_lvm_base_conflicts.pl b/src/test/run_test_lvm_base_conflicts.pl
new file mode 100755
index 00000000..ccb77e00
--- /dev/null
+++ b/src/test/run_test_lvm_base_conflicts.pl
@@ -0,0 +1,62 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Storage;
+use PVE::Storage::LVMPlugin;
+
+my $storage = Test::MockModule->new('PVE::Storage');
+my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
+my ($cfg, $created);
+$storage->mock(
+ config => sub { $cfg },
+ path => sub { '/dev/sda' },
+ activate_storage => sub { },
+ activate_volumes => sub { },
+);
+$lvm->mock(
+ lvm_create_volume_group => sub { $created++ },
+ assert_discard_supported => sub { },
+);
+
+for my $case (
+ ['iscsi', {}, 0],
+ ['iscsi', { images => 1 }, 1],
+ ['iscsidirect', {}, 1],
+ ['zfs', {}, 1],
+ ['vendor', { images => 1 }, 1],
+) {
+ my ($type, $content, $conflict) = @$case;
+ $created = 0;
+ $cfg = {
+ ids => {
+ base => { type => 'iscsi', target => 'iqn.test', content => { none => 1 } },
+ other => {
+ type => $type,
+ target => 'iqn.test',
+ content => $content,
+ disable => 1,
+ nodes => { elsewhere => 1 },
+ },
+ },
+ };
+ eval {
+ PVE::Storage::LVMPlugin->on_add_hook(
+ 'test',
+ { base => 'base:0.0.1.scsi-test', vgname => 'test' },
+ );
+ };
+ if ($conflict) {
+ like($@, qr/storage 'other' serves disk images from this target/, "$type conflict refused");
+ } else {
+ is($@, '', 'connection-only storage permits initialization');
+ }
+ is($created, $conflict ? 0 : 1, 'no writes on conflicting target');
+}
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 11/76] iscsi: refuse virtual devices other than multipath maps as volumes
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (9 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 10/76] lvm: refuse a base volume of a target that serves disk images Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 12/76] diskmanage: collect disk transport type from lsblk Thomas Lamprecht
` (64 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
The target check of an iSCSI volume accepts a virtual device if any
disk below it belongs to the target. The rule is meant for multipath
maps, whose paths all lead to one LUN, possibly through different
target names. A linear map, a logical volume or a software RAID can
combine a disk of the target with disks outside of it and passes the
same check. Used as the base volume of an LVM storage, such a device
is initialized as a physical volume, which also writes to the disks
outside the target.
Accept a virtual device only if it is a multipath map or a partition
of one; any of its paths still proves membership. The volume list of
an iSCSI storage only offers SCSI disks and multipath maps, so only
hand-written volume names are affected.
Fixes: c06b7c8b ("iscsi: verify volume disks are part of target")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Storage/ISCSIPlugin.pm | 11 +++-
src/test/run_iscsi_volume_tests.pl | 81 ++++++++++++++++++++++++------
2 files changed, 76 insertions(+), 16 deletions(-)
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index 536b2e94..81f06979 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -617,6 +617,14 @@ $resolve_virtual_devices = sub {
my $resolved = [];
if ($dev =~ m!^/devices/virtual/block/!) {
+ # All paths of a multipath map lead to one LUN, possibly through other target names. Other
+ # virtual devices, such as linear maps or logical volumes, can include disks outside the
+ # target, so one disk of the target does not make them part of it.
+ my $uuid = file_read_firstline("/sys/$dev/dm/uuid") // '';
+ my ($name) = $dev =~ m!([^/]+)$!;
+ die "'$name' is neither a multipath map nor a partition of one\n"
+ if $uuid !~ m/^(?:part\d+-)?mpath-\S+$/;
+
dir_glob_foreach(
"/sys/$dev/slaves",
'([^.].+)',
@@ -654,7 +662,8 @@ sub activate_volume {
die "resolved realpath '$real_path' is not a block device\n" if !-b $real_path;
my $device_path = $udev_query_path->($real_path);
- my $resolved_paths = $resolve_virtual_devices->($device_path);
+ my $resolved_paths = eval { $resolve_virtual_devices->($device_path) };
+ die "volume '$volname' not part of target '$scfg->{target}' - $@" if $@;
my $found = $check_devices_part_of_target->($resolved_paths, $scfg->{target});
die "volume '$volname' not part of target '$scfg->{target}'\n" if !$found;
diff --git a/src/test/run_iscsi_volume_tests.pl b/src/test/run_iscsi_volume_tests.pl
index eb8a86cd..cc94fc3a 100755
--- a/src/test/run_iscsi_volume_tests.pl
+++ b/src/test/run_iscsi_volume_tests.pl
@@ -13,39 +13,94 @@ use PVE::Storage::ISCSIPlugin;
my ($blockdev) = grep { -b $_ } map { s!^/sys/block/!/dev/!r } glob('/sys/block/*');
plan skip_all => 'block device required for volume activation' if !$blockdev;
-my $device_path;
+my $pci = '/devices/pci0000:00/0000:00:03.0/0000:05:00.0';
+
+# What udev and sysfs report for the virtual devices of the cases below and the disks they use.
+my $udev_paths = {
+ '/dev/sda' => "$pci/ata1/host0/target0:0:0/0:0:0:0/block/sda",
+ '/dev/sdb' => '/devices/platform/host3/session1/target3:0:0/3:0:0:1/block/sdb',
+ '/dev/sdc' => '/devices/platform/host4/session2/target4:0:0/4:0:0:1/block/sdc',
+ '/dev/dm-3' => '/devices/virtual/block/dm-3',
+};
+my $dm_uuids = {
+ 'dm-3' => 'mpath-3600a098038323536',
+ 'dm-4' => 'part1-mpath-3600a098038323536',
+ 'dm-5' => '',
+ 'dm-6' => 'LVM-0hRrw7Rb3cd2WVzQ0ZjOhyAZB9KDGUj5x3wNPtbWZLQfx3m5lhIsm0dw7Js5Gs7v',
+ 'dm-7' => 'mpath-3600a098038323537',
+};
+my $slaves = {
+ 'dm-3' => ['sdb', 'sdc'],
+ 'dm-4' => ['dm-3'],
+ 'dm-5' => ['sda', 'sdb'],
+ 'dm-6' => ['dm-3'],
+ 'dm-7' => ['sdc'],
+ 'md0' => ['sda', 'sdb'],
+};
+
+my ($device_path, $queried);
my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
$plugin->mock(
filesystem_path => sub { $blockdev },
run_command => sub {
my ($cmd, %options) = @_;
- $options{outfunc}->($device_path);
+ my $path = $queried++ ? $udev_paths->{ $cmd->[-1] } : $device_path;
+ die "no device path for '$cmd->[-1]'\n" if !defined($path);
+ $options{outfunc}->($path);
+ },
+ dir_glob_foreach => sub {
+ my ($dir, $regex, $func) = @_;
+ my ($name) = $dir =~ m!/([^/]+)/slaves$!;
+ $func->($_) for ($slaves->{$name} // [])->@*;
},
file_read_firstline => sub {
my ($file) = @_;
+ return $dm_uuids->{$1} if $file =~ m!/([^/]+)/dm/uuid$!;
return $file =~ m!/session1/targetname$! ? 'iqn.test' : 'iqn.other';
},
);
-my $pci = '/devices/pci0000:00/0000:00:03.0/0000:05:00.0';
+my $rejected = "volume '0.0.0.scsi-test' not part of target 'iqn.test'";
+my $not_multipath =
+ sub { "$rejected - '$_[0]' is neither a multipath map nor a partition of one\n" };
+
for my $case (
- ['software iSCSI', '/devices/platform/host3/session1/target3:0:0/3:0:0:1/block/sdb', 1],
- ['iSER or offload', "$pci/host7/session1/target7:0:0/7:0:0:1/block/sdc", 1],
+ ['software iSCSI', '/devices/platform/host3/session1/target3:0:0/3:0:0:1/block/sdb', ''],
+ ['iSER or offload', "$pci/host7/session1/target7:0:0/7:0:0:1/block/sdc", ''],
[
'session of another target',
'/devices/platform/host3/session2/target3:0:0/3:0:0:1/block/sdb',
- 0,
+ "$rejected\n",
],
[
'offload session of another target',
"$pci/host7/session2/target7:0:0/7:0:0:1/block/sdc",
- 0,
+ "$rejected\n",
+ ],
+ ['local SATA disk', "$pci/ata1/host0/target0:0:0/0:0:0:0/block/sda", "$rejected\n"],
+ ['local NVMe disk', "$pci/nvme/nvme0/nvme0n1", "$rejected\n"],
+ ['multipath map with paths through two target names', '/devices/virtual/block/dm-3', ''],
+ ['partition of a multipath map', '/devices/virtual/block/dm-4', ''],
+ ['multipath map of another target', '/devices/virtual/block/dm-7', "$rejected\n"],
+ [
+ 'linear map with a disk of the target',
+ '/devices/virtual/block/dm-5',
+ $not_multipath->('dm-5'),
+ ],
+ [
+ 'logical volume on a multipath map',
+ '/devices/virtual/block/dm-6',
+ $not_multipath->('dm-6'),
+ ],
+ [
+ 'software RAID with a disk of the target',
+ '/devices/virtual/block/md0',
+ $not_multipath->('md0'),
],
- ['local SATA disk', "$pci/ata1/host0/target0:0:0/0:0:0:0/block/sda", 0],
- ['local NVMe disk', "$pci/nvme/nvme0/nvme0n1", 0],
) {
- my ($name, $path, $accepted) = $case->@*;
+ my ($name, $path, $error) = $case->@*;
$device_path = $path;
+ $queried = 0;
eval {
PVE::Storage::ISCSIPlugin->activate_volume(
@@ -54,11 +109,7 @@ for my $case (
'0.0.0.scsi-test',
);
};
- is(
- $@,
- $accepted ? '' : "volume '0.0.0.scsi-test' not part of target 'iqn.test'\n",
- "$name: " . ($accepted ? 'accepted' : 'rejected'),
- );
+ is($@, $error, "$name: " . ($error eq '' ? 'accepted' : 'rejected'));
}
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 12/76] diskmanage: collect disk transport type from lsblk
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (10 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 11/76] iscsi: refuse virtual devices other than multipath maps as volumes Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 13/76] diskmanage: add helper to list multipath devices Thomas Lamprecht
` (63 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Expose the disk transport (sata, sas, fc, nvme, usb) so the storage
wizard can distinguish SAN transports when listing candidates for
shared LVM.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: clarify the transport description]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Disks.pm | 6 ++++++
src/PVE/Diskmanage.pm | 6 +++++-
src/test/disk_tests/sas/disklist_expected.json | 1 +
src/test/disk_tests/sas/lsblk | 5 +++++
4 files changed, 17 insertions(+), 1 deletion(-)
create mode 100644 src/test/disk_tests/sas/lsblk
diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index e707a9e9..f448f8bd 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -135,6 +135,12 @@ __PACKAGE__->register_method({
serial => { type => 'string', optional => 1 },
wwn => { type => 'string', optional => 1 },
health => { type => 'string', optional => 1 },
+ transport => {
+ type => 'string',
+ description =>
+ "The device transport, for example 'sata', 'sas', 'fc', 'nvme', or 'usb'.",
+ optional => 1,
+ },
parent => {
type => 'string',
description => 'For partitions only. The device path of '
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index 4185d2f7..92913d34 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -190,7 +190,7 @@ sub get_smart_data {
}
sub get_lsblk_info {
- my $cmd = [$LSBLK, '--json', '-o', 'path,parttype,fstype'];
+ my $cmd = [$LSBLK, '--json', '-o', 'path,parttype,fstype,tran'];
my $output = "";
eval {
run_command($cmd, outfunc => sub { $output .= "$_[0]\n"; });
@@ -207,6 +207,7 @@ sub get_lsblk_info {
$_->{path} => {
parttype => $_->{parttype},
fstype => $_->{fstype},
+ tran => $_->{tran},
}
} @{$list}
};
@@ -643,6 +644,9 @@ sub get_disks {
devpath => $devpath,
wearout => $wearout,
};
+
+ my $transport = $lsblk_info->{$devpath}->{tran};
+ $disklist->{$dev}->{transport} = $transport if defined($transport);
$disklist->{$dev}->{mounted} = 1 if exists $mounted->{$devpath};
my $by_id_link = $data->{by_id_link};
diff --git a/src/test/disk_tests/sas/disklist_expected.json b/src/test/disk_tests/sas/disklist_expected.json
index a4a1c4c1..26d0d173 100644
--- a/src/test/disk_tests/sas/disklist_expected.json
+++ b/src/test/disk_tests/sas/disklist_expected.json
@@ -13,6 +13,7 @@
"size" : 5120000,
"serial" : "SER2",
"wearout" : "N/A",
+ "transport" : "sas",
"by_id_link" : "/dev/disk/by-id/scsi-00000000000000000"
}
}
diff --git a/src/test/disk_tests/sas/lsblk b/src/test/disk_tests/sas/lsblk
new file mode 100644
index 00000000..cae98190
--- /dev/null
+++ b/src/test/disk_tests/sas/lsblk
@@ -0,0 +1,5 @@
+{
+ "blockdevices": [
+ {"path":"/dev/sda", "parttype":null, "fstype":null, "tran":"sas"}
+ ]
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 13/76] diskmanage: add helper to list multipath devices
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (11 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 12/76] diskmanage: collect disk transport type from lsblk Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 14/76] diskmanage: qualify NVMe over fabrics transport Thomas Lamprecht
` (62 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
get_disks excludes multipath maps because its callers manage local
disks. The storage wizard needs them for SAN LUNs, so add a separate
helper. Combine vendor, model, and transport from member paths and reuse
usage detection to distinguish unused LUNs, LVM PVs, and other uses.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: drop redundant comments]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Diskmanage.pm | 76 ++++++++++++++++++-
src/test/disk_tests/multipath/disklist | 9 +++
.../multipath/disklist_expected.json | 74 ++++++++++++++++++
src/test/disk_tests/multipath/dm-0/dm/name | 1 +
src/test/disk_tests/multipath/dm-0/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-0/size | 1 +
src/test/disk_tests/multipath/dm-0_slaves | 4 +
src/test/disk_tests/multipath/dm-1/dm/name | 1 +
src/test/disk_tests/multipath/dm-1/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-1/size | 1 +
src/test/disk_tests/multipath/dm-1_slaves | 4 +
src/test/disk_tests/multipath/dm-2/dm/name | 1 +
src/test/disk_tests/multipath/dm-2/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-2/size | 1 +
src/test/disk_tests/multipath/dm-2_slaves | 1 +
src/test/disk_tests/multipath/dm-3/dm/name | 1 +
src/test/disk_tests/multipath/dm-3/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-3/size | 1 +
src/test/disk_tests/multipath/dm-3_slaves | 1 +
src/test/disk_tests/multipath/dm-4/dm/name | 1 +
src/test/disk_tests/multipath/dm-4/dm/uuid | 1 +
src/test/disk_tests/multipath/dm-4/size | 1 +
src/test/disk_tests/multipath/dm-4_slaves | 1 +
src/test/disk_tests/multipath/lsblk | 63 +++++++++++++++
src/test/disk_tests/multipath/lvs | 0
src/test/disk_tests/multipath/mounts | 1 +
.../multipath/multipath_expected.json | 59 ++++++++++++++
src/test/disk_tests/multipath/partlist | 0
src/test/disk_tests/multipath/pvs | 1 +
.../disk_tests/multipath/sda/device/model | 1 +
.../disk_tests/multipath/sda/device/vendor | 1 +
.../disk_tests/multipath/sda/holders/holder | 1 +
.../disk_tests/multipath/sda/queue/rotational | 1 +
src/test/disk_tests/multipath/sda/size | 1 +
src/test/disk_tests/multipath/sda_udevadm | 18 +++++
.../disk_tests/multipath/sdb/device/model | 1 +
.../disk_tests/multipath/sdb/device/vendor | 1 +
.../disk_tests/multipath/sdb/holders/holder | 1 +
.../disk_tests/multipath/sdb/queue/rotational | 1 +
src/test/disk_tests/multipath/sdb/size | 1 +
src/test/disk_tests/multipath/sdb_udevadm | 18 +++++
.../disk_tests/multipath/sdc/device/model | 1 +
.../disk_tests/multipath/sdc/device/vendor | 1 +
.../disk_tests/multipath/sdc/holders/holder | 1 +
.../disk_tests/multipath/sdc/queue/rotational | 1 +
src/test/disk_tests/multipath/sdc/size | 1 +
src/test/disk_tests/multipath/sdc_udevadm | 18 +++++
.../disk_tests/multipath/sdd/device/model | 1 +
.../disk_tests/multipath/sdd/device/vendor | 1 +
.../disk_tests/multipath/sdd/holders/holder | 1 +
.../disk_tests/multipath/sdd/queue/rotational | 1 +
src/test/disk_tests/multipath/sdd/size | 1 +
src/test/disk_tests/multipath/sdd_udevadm | 18 +++++
src/test/disk_tests/multipath/zpool | 1 +
src/test/disklist_test.pm | 49 +++++++++++-
55 files changed, 449 insertions(+), 2 deletions(-)
create mode 100644 src/test/disk_tests/multipath/disklist
create mode 100644 src/test/disk_tests/multipath/disklist_expected.json
create mode 100644 src/test/disk_tests/multipath/dm-0/dm/name
create mode 100644 src/test/disk_tests/multipath/dm-0/dm/uuid
create mode 100644 src/test/disk_tests/multipath/dm-0/size
create mode 100644 src/test/disk_tests/multipath/dm-0_slaves
create mode 100644 src/test/disk_tests/multipath/dm-1/dm/name
create mode 100644 src/test/disk_tests/multipath/dm-1/dm/uuid
create mode 100644 src/test/disk_tests/multipath/dm-1/size
create mode 100644 src/test/disk_tests/multipath/dm-1_slaves
create mode 100644 src/test/disk_tests/multipath/dm-2/dm/name
create mode 100644 src/test/disk_tests/multipath/dm-2/dm/uuid
create mode 100644 src/test/disk_tests/multipath/dm-2/size
create mode 100644 src/test/disk_tests/multipath/dm-2_slaves
create mode 100644 src/test/disk_tests/multipath/dm-3/dm/name
create mode 100644 src/test/disk_tests/multipath/dm-3/dm/uuid
create mode 100644 src/test/disk_tests/multipath/dm-3/size
create mode 100644 src/test/disk_tests/multipath/dm-3_slaves
create mode 100644 src/test/disk_tests/multipath/dm-4/dm/name
create mode 100644 src/test/disk_tests/multipath/dm-4/dm/uuid
create mode 100644 src/test/disk_tests/multipath/dm-4/size
create mode 100644 src/test/disk_tests/multipath/dm-4_slaves
create mode 100644 src/test/disk_tests/multipath/lsblk
create mode 100644 src/test/disk_tests/multipath/lvs
create mode 100644 src/test/disk_tests/multipath/mounts
create mode 100644 src/test/disk_tests/multipath/multipath_expected.json
create mode 100644 src/test/disk_tests/multipath/partlist
create mode 100644 src/test/disk_tests/multipath/pvs
create mode 100644 src/test/disk_tests/multipath/sda/device/model
create mode 100644 src/test/disk_tests/multipath/sda/device/vendor
create mode 100644 src/test/disk_tests/multipath/sda/holders/holder
create mode 100644 src/test/disk_tests/multipath/sda/queue/rotational
create mode 100644 src/test/disk_tests/multipath/sda/size
create mode 100644 src/test/disk_tests/multipath/sda_udevadm
create mode 100644 src/test/disk_tests/multipath/sdb/device/model
create mode 100644 src/test/disk_tests/multipath/sdb/device/vendor
create mode 100644 src/test/disk_tests/multipath/sdb/holders/holder
create mode 100644 src/test/disk_tests/multipath/sdb/queue/rotational
create mode 100644 src/test/disk_tests/multipath/sdb/size
create mode 100644 src/test/disk_tests/multipath/sdb_udevadm
create mode 100644 src/test/disk_tests/multipath/sdc/device/model
create mode 100644 src/test/disk_tests/multipath/sdc/device/vendor
create mode 100644 src/test/disk_tests/multipath/sdc/holders/holder
create mode 100644 src/test/disk_tests/multipath/sdc/queue/rotational
create mode 100644 src/test/disk_tests/multipath/sdc/size
create mode 100644 src/test/disk_tests/multipath/sdc_udevadm
create mode 100644 src/test/disk_tests/multipath/sdd/device/model
create mode 100644 src/test/disk_tests/multipath/sdd/device/vendor
create mode 100644 src/test/disk_tests/multipath/sdd/holders/holder
create mode 100644 src/test/disk_tests/multipath/sdd/queue/rotational
create mode 100644 src/test/disk_tests/multipath/sdd/size
create mode 100644 src/test/disk_tests/multipath/sdd_udevadm
create mode 100644 src/test/disk_tests/multipath/zpool
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index 92913d34..e950b6dd 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -190,7 +190,7 @@ sub get_smart_data {
}
sub get_lsblk_info {
- my $cmd = [$LSBLK, '--json', '-o', 'path,parttype,fstype,tran'];
+ my $cmd = [$LSBLK, '--json', '-o', 'path,parttype,fstype,tran,pttype'];
my $output = "";
eval {
run_command($cmd, outfunc => sub { $output .= "$_[0]\n"; });
@@ -208,6 +208,7 @@ sub get_lsblk_info {
parttype => $_->{parttype},
fstype => $_->{fstype},
tran => $_->{tran},
+ pttype => $_->{pttype},
}
} @{$list}
};
@@ -789,6 +790,79 @@ sub get_disks {
return $disklist;
}
+# Keep multipath maps separate from get_disks(), whose callers manage local disks.
+sub get_multipath_disks {
+ my $mounted = mounted_blockdevs();
+ my $lsblk_info = get_lsblk_info();
+ my $zfshash = get_zfs_devices($lsblk_info);
+ my $lvmhash = get_lvm_devices($lsblk_info);
+
+ my $disklist = {};
+
+ dir_glob_foreach(
+ '/sys/block',
+ 'dm-\d+',
+ sub {
+ my ($dev) = @_;
+ my $sysdir = "/sys/block/$dev";
+
+ my $uuid = file_read_firstline("$sysdir/dm/uuid") // return;
+ return if $uuid !~ m/^mpath-(\S+)$/;
+ my $wwid = $1;
+
+ my $name = file_read_firstline("$sysdir/dm/name") // return;
+ my $devpath = "/dev/mapper/$name";
+ my $resolved_devpath = abs_path($devpath);
+ my @devpaths = ($devpath);
+ push @devpaths, $resolved_devpath
+ if defined($resolved_devpath) && $resolved_devpath ne $devpath;
+
+ my $size = get_sysdir_size($sysdir) or return;
+
+ my $slaves = [];
+ dir_glob_foreach("$sysdir/slaves", '[^.].*', sub { push @$slaves, $_[0]; });
+ @$slaves = sort @$slaves;
+
+ my ($vendor, $model, $transport) = ('unknown', 'unknown', undef);
+ if (my $first = $slaves->[0]) {
+ if (my $sysdata = get_sysdir_info("/sys/block/$first")) {
+ $vendor = $sysdata->{vendor};
+ $model = $sysdata->{model};
+ }
+ $transport = $lsblk_info->{"/dev/$first"}->{tran};
+ }
+
+ my $used;
+ $used = 'LVM' if grep { $lvmhash->{$_} } @devpaths;
+ $used //= 'ZFS' if grep { $zfshash->{$_} } @devpaths;
+ my $info = {};
+ for my $path (@devpaths) {
+ $info = $lsblk_info->{$path} // next;
+ last;
+ }
+ $used //= "$info->{fstype}" if defined($info->{fstype});
+ $used //= 'mounted' if grep { $mounted->{$_} } @devpaths;
+ $used //= 'partitions'
+ if defined($info->{pttype}) && $info->{pttype} ne '';
+ $used //= 'Device Mapper' if !dir_is_empty("$sysdir/holders");
+
+ $disklist->{$dev} = {
+ devpath => $devpath,
+ size => $size,
+ vendor => $vendor,
+ model => $model,
+ wwn => $wwid,
+ paths => scalar($slaves->@*),
+ slaves => [map { "/dev/$_" } $slaves->@*],
+ };
+ $disklist->{$dev}->{transport} = $transport if defined($transport);
+ $disklist->{$dev}->{used} = $used if $used;
+ },
+ );
+
+ return $disklist;
+}
+
sub get_partnum {
my ($part_path) = @_;
diff --git a/src/test/disk_tests/multipath/disklist b/src/test/disk_tests/multipath/disklist
new file mode 100644
index 00000000..92935894
--- /dev/null
+++ b/src/test/disk_tests/multipath/disklist
@@ -0,0 +1,9 @@
+sda
+sdb
+sdc
+sdd
+dm-0
+dm-1
+dm-2
+dm-3
+dm-4
diff --git a/src/test/disk_tests/multipath/disklist_expected.json b/src/test/disk_tests/multipath/disklist_expected.json
new file mode 100644
index 00000000..a8e74fa1
--- /dev/null
+++ b/src/test/disk_tests/multipath/disklist_expected.json
@@ -0,0 +1,74 @@
+{
+ "sda": {
+ "devpath": "/dev/sda",
+ "size": 10737418240,
+ "vendor": "SANVEND",
+ "model": "SANMODEL",
+ "serial": "SSERa",
+ "wwn": "0x600140500000000a",
+ "gpt": 0,
+ "rpm": -1,
+ "type": "unknown",
+ "health": "UNKNOWN",
+ "wearout": "N/A",
+ "osdid": -1,
+ "osdid-list": null,
+ "by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000a",
+ "transport": "fc",
+ "used": "Device Mapper"
+ },
+ "sdb": {
+ "devpath": "/dev/sdb",
+ "size": 10737418240,
+ "vendor": "SANVEND",
+ "model": "SANMODEL",
+ "serial": "SSERb",
+ "wwn": "0x600140500000000b",
+ "gpt": 0,
+ "rpm": -1,
+ "type": "unknown",
+ "health": "UNKNOWN",
+ "wearout": "N/A",
+ "osdid": -1,
+ "osdid-list": null,
+ "by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000b",
+ "transport": "fc",
+ "used": "Device Mapper"
+ },
+ "sdc": {
+ "devpath": "/dev/sdc",
+ "size": 10737418240,
+ "vendor": "SANVEND",
+ "model": "SANMODEL",
+ "serial": "SSERc",
+ "wwn": "0x600140500000000c",
+ "gpt": 0,
+ "rpm": -1,
+ "type": "unknown",
+ "health": "UNKNOWN",
+ "wearout": "N/A",
+ "osdid": -1,
+ "osdid-list": null,
+ "by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000c",
+ "transport": "fc",
+ "used": "Device Mapper"
+ },
+ "sdd": {
+ "devpath": "/dev/sdd",
+ "size": 10737418240,
+ "vendor": "SANVEND",
+ "model": "SANMODEL",
+ "serial": "SSERd",
+ "wwn": "0x600140500000000d",
+ "gpt": 0,
+ "rpm": -1,
+ "type": "unknown",
+ "health": "UNKNOWN",
+ "wearout": "N/A",
+ "osdid": -1,
+ "osdid-list": null,
+ "by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000d",
+ "transport": "fc",
+ "used": "Device Mapper"
+ }
+}
diff --git a/src/test/disk_tests/multipath/dm-0/dm/name b/src/test/disk_tests/multipath/dm-0/dm/name
new file mode 100644
index 00000000..0ed057a0
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-0/dm/name
@@ -0,0 +1 @@
+mpatha
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/dm-0/dm/uuid b/src/test/disk_tests/multipath/dm-0/dm/uuid
new file mode 100644
index 00000000..094955aa
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-0/dm/uuid
@@ -0,0 +1 @@
+mpath-3600140500000000000000000000000001
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/dm-0/size b/src/test/disk_tests/multipath/dm-0/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-0/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/dm-0_slaves b/src/test/disk_tests/multipath/dm-0_slaves
new file mode 100644
index 00000000..16373159
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-0_slaves
@@ -0,0 +1,4 @@
+.
+..
+sda
+sdb
diff --git a/src/test/disk_tests/multipath/dm-1/dm/name b/src/test/disk_tests/multipath/dm-1/dm/name
new file mode 100644
index 00000000..f70ac3d8
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-1/dm/name
@@ -0,0 +1 @@
+mpathb
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/dm-1/dm/uuid b/src/test/disk_tests/multipath/dm-1/dm/uuid
new file mode 100644
index 00000000..7e8567b8
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-1/dm/uuid
@@ -0,0 +1 @@
+mpath-3600140500000000000000000000000002
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/dm-1/size b/src/test/disk_tests/multipath/dm-1/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-1/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/dm-1_slaves b/src/test/disk_tests/multipath/dm-1_slaves
new file mode 100644
index 00000000..cde08b36
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-1_slaves
@@ -0,0 +1,4 @@
+.
+..
+sdc
+sdd
diff --git a/src/test/disk_tests/multipath/dm-2/dm/name b/src/test/disk_tests/multipath/dm-2/dm/name
new file mode 100644
index 00000000..4770de43
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-2/dm/name
@@ -0,0 +1 @@
+mpathc
diff --git a/src/test/disk_tests/multipath/dm-2/dm/uuid b/src/test/disk_tests/multipath/dm-2/dm/uuid
new file mode 100644
index 00000000..2d607eae
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-2/dm/uuid
@@ -0,0 +1 @@
+mpath-3600140500000000000000000000000003
diff --git a/src/test/disk_tests/multipath/dm-2/size b/src/test/disk_tests/multipath/dm-2/size
new file mode 100644
index 00000000..14faeece
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-2/size
@@ -0,0 +1 @@
+20971520
diff --git a/src/test/disk_tests/multipath/dm-2_slaves b/src/test/disk_tests/multipath/dm-2_slaves
new file mode 100644
index 00000000..8b137891
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-2_slaves
@@ -0,0 +1 @@
+
diff --git a/src/test/disk_tests/multipath/dm-3/dm/name b/src/test/disk_tests/multipath/dm-3/dm/name
new file mode 100644
index 00000000..a98ef69c
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-3/dm/name
@@ -0,0 +1 @@
+mpathd
diff --git a/src/test/disk_tests/multipath/dm-3/dm/uuid b/src/test/disk_tests/multipath/dm-3/dm/uuid
new file mode 100644
index 00000000..75f25b81
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-3/dm/uuid
@@ -0,0 +1 @@
+mpath-3600140500000000000000000000000004
diff --git a/src/test/disk_tests/multipath/dm-3/size b/src/test/disk_tests/multipath/dm-3/size
new file mode 100644
index 00000000..14faeece
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-3/size
@@ -0,0 +1 @@
+20971520
diff --git a/src/test/disk_tests/multipath/dm-3_slaves b/src/test/disk_tests/multipath/dm-3_slaves
new file mode 100644
index 00000000..8b137891
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-3_slaves
@@ -0,0 +1 @@
+
diff --git a/src/test/disk_tests/multipath/dm-4/dm/name b/src/test/disk_tests/multipath/dm-4/dm/name
new file mode 100644
index 00000000..e084e8f6
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-4/dm/name
@@ -0,0 +1 @@
+mpathe
diff --git a/src/test/disk_tests/multipath/dm-4/dm/uuid b/src/test/disk_tests/multipath/dm-4/dm/uuid
new file mode 100644
index 00000000..0ca78754
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-4/dm/uuid
@@ -0,0 +1 @@
+mpath-3600140500000000000000000000000005
diff --git a/src/test/disk_tests/multipath/dm-4/size b/src/test/disk_tests/multipath/dm-4/size
new file mode 100644
index 00000000..14faeece
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-4/size
@@ -0,0 +1 @@
+20971520
diff --git a/src/test/disk_tests/multipath/dm-4_slaves b/src/test/disk_tests/multipath/dm-4_slaves
new file mode 100644
index 00000000..8b137891
--- /dev/null
+++ b/src/test/disk_tests/multipath/dm-4_slaves
@@ -0,0 +1 @@
+
diff --git a/src/test/disk_tests/multipath/lsblk b/src/test/disk_tests/multipath/lsblk
new file mode 100644
index 00000000..91f74574
--- /dev/null
+++ b/src/test/disk_tests/multipath/lsblk
@@ -0,0 +1,63 @@
+{
+ "blockdevices": [
+ {
+ "path": "/dev/sda",
+ "parttype": null,
+ "fstype": null,
+ "tran": "fc"
+ },
+ {
+ "path": "/dev/sdb",
+ "parttype": null,
+ "fstype": null,
+ "tran": "fc"
+ },
+ {
+ "path": "/dev/sdc",
+ "parttype": null,
+ "fstype": null,
+ "tran": "fc"
+ },
+ {
+ "path": "/dev/sdd",
+ "parttype": null,
+ "fstype": null,
+ "tran": "fc"
+ },
+ {
+ "path": "/dev/mapper/mpatha",
+ "parttype": null,
+ "fstype": null,
+ "tran": null,
+ "pttype": null
+ },
+ {
+ "path": "/dev/mapper/mpathb",
+ "parttype": null,
+ "fstype": null,
+ "tran": null,
+ "pttype": null
+ },
+ {
+ "path": "/dev/mapper/mpathc",
+ "parttype": null,
+ "fstype": null,
+ "tran": null,
+ "pttype": "gpt"
+ },
+ {
+ "path": "/dev/dm-3",
+ "parttype": null,
+ "fstype": null,
+ "tran": null,
+ "pttype": null
+ },
+ {
+ "path": "/dev/dm-4",
+ "parttype": null,
+ "fstype": null,
+ "tran": null,
+ "pttype": null
+ }
+ ]
+}
diff --git a/src/test/disk_tests/multipath/lvs b/src/test/disk_tests/multipath/lvs
new file mode 100644
index 00000000..e69de29b
diff --git a/src/test/disk_tests/multipath/mounts b/src/test/disk_tests/multipath/mounts
new file mode 100644
index 00000000..06ada278
--- /dev/null
+++ b/src/test/disk_tests/multipath/mounts
@@ -0,0 +1 @@
+/dev/mapper/mpathe /mnt
diff --git a/src/test/disk_tests/multipath/multipath_expected.json b/src/test/disk_tests/multipath/multipath_expected.json
new file mode 100644
index 00000000..62e6e1c5
--- /dev/null
+++ b/src/test/disk_tests/multipath/multipath_expected.json
@@ -0,0 +1,59 @@
+{
+ "dm-0": {
+ "devpath": "/dev/mapper/mpatha",
+ "size": 10737418240,
+ "vendor": "SANVEND",
+ "model": "SANMODEL",
+ "wwn": "3600140500000000000000000000000001",
+ "paths": 2,
+ "slaves": [
+ "/dev/sda",
+ "/dev/sdb"
+ ],
+ "transport": "fc"
+ },
+ "dm-1": {
+ "devpath": "/dev/mapper/mpathb",
+ "size": 10737418240,
+ "vendor": "SANVEND",
+ "model": "SANMODEL",
+ "wwn": "3600140500000000000000000000000002",
+ "paths": 2,
+ "slaves": [
+ "/dev/sdc",
+ "/dev/sdd"
+ ],
+ "transport": "fc",
+ "used": "LVM"
+ },
+ "dm-2": {
+ "devpath": "/dev/mapper/mpathc",
+ "size": 10737418240,
+ "vendor": "unknown",
+ "model": "unknown",
+ "wwn": "3600140500000000000000000000000003",
+ "paths": 0,
+ "slaves": [],
+ "used": "partitions"
+ },
+ "dm-3": {
+ "devpath": "/dev/mapper/mpathd",
+ "size": 10737418240,
+ "vendor": "unknown",
+ "model": "unknown",
+ "wwn": "3600140500000000000000000000000004",
+ "paths": 0,
+ "slaves": [],
+ "used": "ZFS"
+ },
+ "dm-4": {
+ "devpath": "/dev/mapper/mpathe",
+ "size": 10737418240,
+ "vendor": "unknown",
+ "model": "unknown",
+ "wwn": "3600140500000000000000000000000005",
+ "paths": 0,
+ "slaves": [],
+ "used": "mounted"
+ }
+}
diff --git a/src/test/disk_tests/multipath/partlist b/src/test/disk_tests/multipath/partlist
new file mode 100644
index 00000000..e69de29b
diff --git a/src/test/disk_tests/multipath/pvs b/src/test/disk_tests/multipath/pvs
new file mode 100644
index 00000000..9c2f9b93
--- /dev/null
+++ b/src/test/disk_tests/multipath/pvs
@@ -0,0 +1 @@
+ /dev/dm-1
diff --git a/src/test/disk_tests/multipath/sda/device/model b/src/test/disk_tests/multipath/sda/device/model
new file mode 100644
index 00000000..2f50f3f3
--- /dev/null
+++ b/src/test/disk_tests/multipath/sda/device/model
@@ -0,0 +1 @@
+SANMODEL
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sda/device/vendor b/src/test/disk_tests/multipath/sda/device/vendor
new file mode 100644
index 00000000..ead8944c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sda/device/vendor
@@ -0,0 +1 @@
+SANVEND
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sda/holders/holder b/src/test/disk_tests/multipath/sda/holders/holder
new file mode 100644
index 00000000..9255a92c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sda/holders/holder
@@ -0,0 +1 @@
+dm-holder
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sda/queue/rotational b/src/test/disk_tests/multipath/sda/queue/rotational
new file mode 100644
index 00000000..56a6051c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sda/queue/rotational
@@ -0,0 +1 @@
+1
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sda/size b/src/test/disk_tests/multipath/sda/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/sda/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sda_udevadm b/src/test/disk_tests/multipath/sda_udevadm
new file mode 100644
index 00000000..7bd58b4e
--- /dev/null
+++ b/src/test/disk_tests/multipath/sda_udevadm
@@ -0,0 +1,18 @@
+P: /devices/pci0000:00/0000:00:03.0/0000:02:00.0/host4/rport-4:0-0/target4:0:0/4:0:0:0/block/sda
+N: sda
+S: disk/by-id/scsi-36001405000000000000000000000000a
+E: DEVLINKS=/dev/disk/by-id/scsi-36001405000000000000000000000000a
+E: DEVNAME=/dev/sda
+E: DEVPATH=/devices/pci0000:00/0000:00:03.0/0000:02:00.0/host4/rport-4:0-0/target4:0:0/4:0:0:0/block/sda
+E: DEVTYPE=disk
+E: ID_BUS=scsi
+E: ID_MODEL=SANMODEL
+E: ID_SCSI=1
+E: ID_SERIAL=SANSERIALa
+E: ID_SERIAL_SHORT=SSERa
+E: ID_TYPE=disk
+E: ID_VENDOR=SANVEND
+E: ID_WWN=0x600140500000000a
+E: MAJOR=8
+E: MINOR=0
+E: SUBSYSTEM=block
diff --git a/src/test/disk_tests/multipath/sdb/device/model b/src/test/disk_tests/multipath/sdb/device/model
new file mode 100644
index 00000000..2f50f3f3
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdb/device/model
@@ -0,0 +1 @@
+SANMODEL
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdb/device/vendor b/src/test/disk_tests/multipath/sdb/device/vendor
new file mode 100644
index 00000000..ead8944c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdb/device/vendor
@@ -0,0 +1 @@
+SANVEND
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdb/holders/holder b/src/test/disk_tests/multipath/sdb/holders/holder
new file mode 100644
index 00000000..9255a92c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdb/holders/holder
@@ -0,0 +1 @@
+dm-holder
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdb/queue/rotational b/src/test/disk_tests/multipath/sdb/queue/rotational
new file mode 100644
index 00000000..56a6051c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdb/queue/rotational
@@ -0,0 +1 @@
+1
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdb/size b/src/test/disk_tests/multipath/sdb/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdb/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdb_udevadm b/src/test/disk_tests/multipath/sdb_udevadm
new file mode 100644
index 00000000..9a04c63f
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdb_udevadm
@@ -0,0 +1,18 @@
+P: /devices/pci0000:00/0000:00:03.0/0000:02:00.0/host4/rport-4:0-0/target4:0:0/4:0:0:0/block/sdb
+N: sdb
+S: disk/by-id/scsi-36001405000000000000000000000000b
+E: DEVLINKS=/dev/disk/by-id/scsi-36001405000000000000000000000000b
+E: DEVNAME=/dev/sdb
+E: DEVPATH=/devices/pci0000:00/0000:00:03.0/0000:02:00.0/host4/rport-4:0-0/target4:0:0/4:0:0:0/block/sdb
+E: DEVTYPE=disk
+E: ID_BUS=scsi
+E: ID_MODEL=SANMODEL
+E: ID_SCSI=1
+E: ID_SERIAL=SANSERIALb
+E: ID_SERIAL_SHORT=SSERb
+E: ID_TYPE=disk
+E: ID_VENDOR=SANVEND
+E: ID_WWN=0x600140500000000b
+E: MAJOR=8
+E: MINOR=0
+E: SUBSYSTEM=block
diff --git a/src/test/disk_tests/multipath/sdc/device/model b/src/test/disk_tests/multipath/sdc/device/model
new file mode 100644
index 00000000..2f50f3f3
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdc/device/model
@@ -0,0 +1 @@
+SANMODEL
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdc/device/vendor b/src/test/disk_tests/multipath/sdc/device/vendor
new file mode 100644
index 00000000..ead8944c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdc/device/vendor
@@ -0,0 +1 @@
+SANVEND
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdc/holders/holder b/src/test/disk_tests/multipath/sdc/holders/holder
new file mode 100644
index 00000000..9255a92c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdc/holders/holder
@@ -0,0 +1 @@
+dm-holder
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdc/queue/rotational b/src/test/disk_tests/multipath/sdc/queue/rotational
new file mode 100644
index 00000000..56a6051c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdc/queue/rotational
@@ -0,0 +1 @@
+1
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdc/size b/src/test/disk_tests/multipath/sdc/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdc/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdc_udevadm b/src/test/disk_tests/multipath/sdc_udevadm
new file mode 100644
index 00000000..acff253e
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdc_udevadm
@@ -0,0 +1,18 @@
+P: /devices/pci0000:00/0000:00:03.0/0000:02:00.0/host4/rport-4:0-0/target4:0:0/4:0:0:0/block/sdc
+N: sdc
+S: disk/by-id/scsi-36001405000000000000000000000000c
+E: DEVLINKS=/dev/disk/by-id/scsi-36001405000000000000000000000000c
+E: DEVNAME=/dev/sdc
+E: DEVPATH=/devices/pci0000:00/0000:00:03.0/0000:02:00.0/host4/rport-4:0-0/target4:0:0/4:0:0:0/block/sdc
+E: DEVTYPE=disk
+E: ID_BUS=scsi
+E: ID_MODEL=SANMODEL
+E: ID_SCSI=1
+E: ID_SERIAL=SANSERIALc
+E: ID_SERIAL_SHORT=SSERc
+E: ID_TYPE=disk
+E: ID_VENDOR=SANVEND
+E: ID_WWN=0x600140500000000c
+E: MAJOR=8
+E: MINOR=0
+E: SUBSYSTEM=block
diff --git a/src/test/disk_tests/multipath/sdd/device/model b/src/test/disk_tests/multipath/sdd/device/model
new file mode 100644
index 00000000..2f50f3f3
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdd/device/model
@@ -0,0 +1 @@
+SANMODEL
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdd/device/vendor b/src/test/disk_tests/multipath/sdd/device/vendor
new file mode 100644
index 00000000..ead8944c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdd/device/vendor
@@ -0,0 +1 @@
+SANVEND
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdd/holders/holder b/src/test/disk_tests/multipath/sdd/holders/holder
new file mode 100644
index 00000000..9255a92c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdd/holders/holder
@@ -0,0 +1 @@
+dm-holder
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdd/queue/rotational b/src/test/disk_tests/multipath/sdd/queue/rotational
new file mode 100644
index 00000000..56a6051c
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdd/queue/rotational
@@ -0,0 +1 @@
+1
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdd/size b/src/test/disk_tests/multipath/sdd/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdd/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/sdd_udevadm b/src/test/disk_tests/multipath/sdd_udevadm
new file mode 100644
index 00000000..813c5f1d
--- /dev/null
+++ b/src/test/disk_tests/multipath/sdd_udevadm
@@ -0,0 +1,18 @@
+P: /devices/pci0000:00/0000:00:03.0/0000:02:00.0/host4/rport-4:0-0/target4:0:0/4:0:0:0/block/sdd
+N: sdd
+S: disk/by-id/scsi-36001405000000000000000000000000d
+E: DEVLINKS=/dev/disk/by-id/scsi-36001405000000000000000000000000d
+E: DEVNAME=/dev/sdd
+E: DEVPATH=/devices/pci0000:00/0000:00:03.0/0000:02:00.0/host4/rport-4:0-0/target4:0:0/4:0:0:0/block/sdd
+E: DEVTYPE=disk
+E: ID_BUS=scsi
+E: ID_MODEL=SANMODEL
+E: ID_SCSI=1
+E: ID_SERIAL=SANSERIALd
+E: ID_SERIAL_SHORT=SSERd
+E: ID_TYPE=disk
+E: ID_VENDOR=SANVEND
+E: ID_WWN=0x600140500000000d
+E: MAJOR=8
+E: MINOR=0
+E: SUBSYSTEM=block
diff --git a/src/test/disk_tests/multipath/zpool b/src/test/disk_tests/multipath/zpool
new file mode 100644
index 00000000..e723f7e3
--- /dev/null
+++ b/src/test/disk_tests/multipath/zpool
@@ -0,0 +1 @@
+ /dev/dm-3 -
diff --git a/src/test/disklist_test.pm b/src/test/disklist_test.pm
index a0dc01f6..9e35430d 100644
--- a/src/test/disklist_test.pm
+++ b/src/test/disklist_test.pm
@@ -18,6 +18,14 @@ my $testcount = 0; # testcount for TAP::Harness
my $diskmanage_module; # mockmodule for PVE::Diskmanage
my $print = 0;
+my $multipath_aliases = {
+ '/dev/mapper/mpatha' => '/dev/dm-0',
+ '/dev/mapper/mpathb' => '/dev/dm-1',
+ '/dev/mapper/mpathc' => '/dev/dm-2',
+ '/dev/mapper/mpathd' => '/dev/dm-3',
+ '/dev/mapper/mpathe' => '/dev/dm-4',
+};
+
sub mocked_run_command {
my ($cmd, %param) = @_;
@@ -101,6 +109,16 @@ sub mocked_is_iscsi {
return 0;
}
+sub mocked_file_read_firstline {
+ my ($path) = @_;
+
+ my $originalsub = $diskmanage_module->original('file_read_firstline');
+
+ $path =~ s|^/sys/block|disk_tests/$testcasedir|;
+
+ return &$originalsub($path);
+}
+
sub mocked_dir_glob_foreach {
my ($dir, $regex, $sub) = @_;
@@ -109,6 +127,8 @@ sub mocked_dir_glob_foreach {
# read lines in from file
if ($dir =~ m{^/sys/block$}) {
@$lines = split(/\n/, read_test_file('disklist'));
+ } elsif ($dir =~ m{^/sys/block/([^/]+)/slaves$}) {
+ @$lines = split(/\n/, read_test_file("${1}_slaves"));
} elsif ($dir =~ m{^/sys/block/([^/]+)}) {
@$lines = split(/\n/, read_test_file('partlist'));
}
@@ -213,6 +233,15 @@ sub test_disk_list {
$testcount++;
is_deeply($disks, $expected_disk_list, 'disk list should be the same');
+ if (-f "disk_tests/$testcasedir/multipath_expected.json") {
+ my $multipath = PVE::Diskmanage::get_multipath_disks();
+ my $expected_multipath = decode_json(read_test_file('multipath_expected.json'));
+
+ print Dumper($multipath) if $print;
+ $testcount++;
+ is_deeply($multipath, $expected_multipath, 'multipath list should be the same');
+ }
+
done_testing($testcount);
};
}
@@ -239,8 +268,15 @@ $diskmanage_module->mock('assert_blockdev' => sub { return 1; });
print("\tMocked assert_blockdev\n");
$diskmanage_module->mock(
'dir_is_empty' => sub {
- # all partitions have a holder dir
my $val = shift;
+ # let test cases provide the dir (e.g. holders) with content
+ my $mapped = $val;
+ $mapped =~ s|^/sys/block|disk_tests/$testcasedir|;
+ if (-d $mapped) {
+ my $originalsub = $diskmanage_module->original('dir_is_empty');
+ return &$originalsub($mapped);
+ }
+ # all partitions have a holder dir
if ($val =~ m|^/sys/block/.+/.+/|) {
return 0;
}
@@ -248,6 +284,17 @@ $diskmanage_module->mock(
},
);
print("\tMocked dir_is_empty\n");
+$diskmanage_module->mock('file_read_firstline' => \&mocked_file_read_firstline);
+print("\tMocked file_read_firstline\n");
+$diskmanage_module->mock(
+ 'abs_path' => sub {
+ my ($path) = @_;
+ return $multipath_aliases->{$path}
+ if defined($multipath_aliases->{$path});
+ return &{ $diskmanage_module->original('abs_path') }($path);
+ },
+);
+print("\tMocked abs_path\n");
$diskmanage_module->mock('check_bin' => sub { return 1; });
print("\tMocked check_bin\n");
my $tools_module = Test::MockModule->new('PVE::ProcFSTools', no_auto => 1);
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 14/76] diskmanage: qualify NVMe over fabrics transport
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (12 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 13/76] diskmanage: add helper to list multipath devices Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 15/76] disks: list: add include-remote parameter Thomas Lamprecht
` (61 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
lsblk reports 'nvme' for fabrics-attached namespaces, preventing clients
from distinguishing them from local PCIe drives. Read the transport from
the controller, or a subsystem controller for native multipath, and
report nvme-fc, nvme-tcp, nvme-rdma, or nvme-loop instead.
This lets SAN scans and local storage dialogs select suitable devices
without hiding fabrics-attached namespaces server-side.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Diskmanage.pm | 27 +++++++++++++++
src/test/disk_tests/multipath/disklist | 2 ++
.../multipath/disklist_expected.json | 33 +++++++++++++++++++
src/test/disk_tests/multipath/lsblk | 14 ++++++++
.../disk_tests/multipath/nvme8n1/device/model | 1 +
.../multipath/nvme8n1/device/transport | 1 +
.../multipath/nvme8n1/queue/rotational | 1 +
src/test/disk_tests/multipath/nvme8n1/size | 1 +
src/test/disk_tests/multipath/nvme8n1_udevadm | 14 ++++++++
.../disk_tests/multipath/nvme9n1/device/model | 1 +
.../multipath/nvme9n1/device/transport | 1 +
.../multipath/nvme9n1/device/vendor | 1 +
.../multipath/nvme9n1/queue/rotational | 1 +
src/test/disk_tests/multipath/nvme9n1/size | 1 +
src/test/disk_tests/multipath/nvme9n1_udevadm | 12 +++++++
15 files changed, 111 insertions(+)
create mode 100644 src/test/disk_tests/multipath/nvme8n1/device/model
create mode 100644 src/test/disk_tests/multipath/nvme8n1/device/transport
create mode 100644 src/test/disk_tests/multipath/nvme8n1/queue/rotational
create mode 100644 src/test/disk_tests/multipath/nvme8n1/size
create mode 100644 src/test/disk_tests/multipath/nvme8n1_udevadm
create mode 100644 src/test/disk_tests/multipath/nvme9n1/device/model
create mode 100644 src/test/disk_tests/multipath/nvme9n1/device/transport
create mode 100644 src/test/disk_tests/multipath/nvme9n1/device/vendor
create mode 100644 src/test/disk_tests/multipath/nvme9n1/queue/rotational
create mode 100644 src/test/disk_tests/multipath/nvme9n1/size
create mode 100644 src/test/disk_tests/multipath/nvme9n1_udevadm
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index e950b6dd..bc81e854 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -503,6 +503,26 @@ sub is_iscsi {
return 0;
}
+# Returns the NVMe transport (pcie, fc, tcp, rdma, loop) of a namespace, either directly from
+# its controller, or from any controller of its subsystem for native NVMe multipath setups.
+sub get_nvme_transport {
+ my ($sysdir) = @_;
+
+ my $transport = file_read_firstline("$sysdir/device/transport");
+ return $transport if defined($transport);
+
+ dir_glob_foreach(
+ "$sysdir/device",
+ 'nvme\d+',
+ sub {
+ my ($controller) = @_;
+ $transport //= file_read_firstline("$sysdir/device/$controller/transport");
+ },
+ );
+
+ return $transport;
+}
+
my sub is_ssdlike {
my ($type) = @_;
return $type eq 'ssd' || $type eq 'nvme';
@@ -595,6 +615,9 @@ sub get_disks {
# we do not want iscsi devices
return if is_iscsi($sysdir);
+ my $nvme_transport;
+ $nvme_transport = get_nvme_transport($sysdir) if $dev =~ m/^nvme/;
+
my $sysdata = get_sysdir_info($sysdir);
return if !defined($sysdata);
@@ -646,7 +669,11 @@ sub get_disks {
wearout => $wearout,
};
+ # lsblk reports plain 'nvme' for fabrics attached namespaces
my $transport = $lsblk_info->{$devpath}->{tran};
+ if (defined($nvme_transport) && $nvme_transport ne 'pcie') {
+ $transport = "nvme-$nvme_transport";
+ }
$disklist->{$dev}->{transport} = $transport if defined($transport);
$disklist->{$dev}->{mounted} = 1 if exists $mounted->{$devpath};
diff --git a/src/test/disk_tests/multipath/disklist b/src/test/disk_tests/multipath/disklist
index 92935894..f759ba9e 100644
--- a/src/test/disk_tests/multipath/disklist
+++ b/src/test/disk_tests/multipath/disklist
@@ -7,3 +7,5 @@ dm-1
dm-2
dm-3
dm-4
+nvme8n1
+nvme9n1
diff --git a/src/test/disk_tests/multipath/disklist_expected.json b/src/test/disk_tests/multipath/disklist_expected.json
index a8e74fa1..ec442969 100644
--- a/src/test/disk_tests/multipath/disklist_expected.json
+++ b/src/test/disk_tests/multipath/disklist_expected.json
@@ -70,5 +70,38 @@
"by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000d",
"transport": "fc",
"used": "Device Mapper"
+ },
+ "nvme8n1": {
+ "devpath": "/dev/nvme8n1",
+ "size": 10737418240,
+ "vendor": "unknown",
+ "model": "FCMODEL",
+ "serial": "FCNVME1",
+ "wwn": "uuid.20000000-0000-0000-0000-000000000001",
+ "gpt": 0,
+ "rpm": 0,
+ "type": "nvme",
+ "health": "UNKNOWN",
+ "wearout": "N/A",
+ "osdid": -1,
+ "osdid-list": null,
+ "by_id_link": "/dev/disk/by-id/nvme-FCMODEL_FCNVME1",
+ "transport": "nvme-fc"
+ },
+ "nvme9n1": {
+ "devpath": "/dev/nvme9n1",
+ "size": 10737418240,
+ "vendor": "FABVEND",
+ "model": "FABRICS CTRL",
+ "serial": "FAB1",
+ "wwn": "uuid.10000000-0000-0000-0000-000000000001",
+ "gpt": 0,
+ "rpm": 0,
+ "type": "nvme",
+ "health": "UNKNOWN",
+ "wearout": "N/A",
+ "osdid": -1,
+ "osdid-list": null,
+ "transport": "nvme-tcp"
}
}
diff --git a/src/test/disk_tests/multipath/lsblk b/src/test/disk_tests/multipath/lsblk
index 91f74574..dfc1ae1a 100644
--- a/src/test/disk_tests/multipath/lsblk
+++ b/src/test/disk_tests/multipath/lsblk
@@ -58,6 +58,20 @@
"fstype": null,
"tran": null,
"pttype": null
+ },
+ {
+ "path": "/dev/nvme8n1",
+ "parttype": null,
+ "fstype": null,
+ "tran": "nvme",
+ "pttype": null
+ },
+ {
+ "path": "/dev/nvme9n1",
+ "parttype": null,
+ "fstype": null,
+ "tran": "nvme",
+ "pttype": null
}
]
}
diff --git a/src/test/disk_tests/multipath/nvme8n1/device/model b/src/test/disk_tests/multipath/nvme8n1/device/model
new file mode 100644
index 00000000..1aac689e
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1/device/model
@@ -0,0 +1 @@
+FCMODEL
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme8n1/device/transport b/src/test/disk_tests/multipath/nvme8n1/device/transport
new file mode 100644
index 00000000..8b517d01
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1/device/transport
@@ -0,0 +1 @@
+fc
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme8n1/queue/rotational b/src/test/disk_tests/multipath/nvme8n1/queue/rotational
new file mode 100644
index 00000000..c2270834
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1/queue/rotational
@@ -0,0 +1 @@
+0
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme8n1/size b/src/test/disk_tests/multipath/nvme8n1/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme8n1_udevadm b/src/test/disk_tests/multipath/nvme8n1_udevadm
new file mode 100644
index 00000000..d0cd5ea9
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1_udevadm
@@ -0,0 +1,14 @@
+P: /devices/virtual/nvme-subsystem/nvme-subsys8/nvme8n1
+N: nvme8n1
+S: disk/by-id/nvme-FCMODEL_FCNVME1
+E: DEVLINKS=/dev/disk/by-id/nvme-FCMODEL_FCNVME1
+E: DEVNAME=/dev/nvme8n1
+E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys8/nvme8n1
+E: DEVTYPE=disk
+E: ID_MODEL=FCMODEL
+E: ID_SERIAL=FCNVME_1
+E: ID_SERIAL_SHORT=FCNVME1
+E: ID_WWN=uuid.20000000-0000-0000-0000-000000000001
+E: MAJOR=259
+E: MINOR=1
+E: SUBSYSTEM=block
diff --git a/src/test/disk_tests/multipath/nvme9n1/device/model b/src/test/disk_tests/multipath/nvme9n1/device/model
new file mode 100644
index 00000000..aa0287c7
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/device/model
@@ -0,0 +1 @@
+FABMODEL
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1/device/transport b/src/test/disk_tests/multipath/nvme9n1/device/transport
new file mode 100644
index 00000000..f5483dbf
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/device/transport
@@ -0,0 +1 @@
+tcp
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1/device/vendor b/src/test/disk_tests/multipath/nvme9n1/device/vendor
new file mode 100644
index 00000000..cdc6ddfa
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/device/vendor
@@ -0,0 +1 @@
+FABVEND
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1/queue/rotational b/src/test/disk_tests/multipath/nvme9n1/queue/rotational
new file mode 100644
index 00000000..c2270834
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/queue/rotational
@@ -0,0 +1 @@
+0
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1/size b/src/test/disk_tests/multipath/nvme9n1/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1_udevadm b/src/test/disk_tests/multipath/nvme9n1_udevadm
new file mode 100644
index 00000000..b6d13ccb
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1_udevadm
@@ -0,0 +1,12 @@
+P: /devices/virtual/nvme-subsystem/nvme-subsys9/nvme9n1
+N: nvme9n1
+E: DEVNAME=/dev/nvme9n1
+E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys9/nvme9n1
+E: DEVTYPE=disk
+E: ID_MODEL=FABRICS CTRL
+E: ID_SERIAL=FABRICS_1
+E: ID_SERIAL_SHORT=FAB1
+E: ID_WWN=uuid.10000000-0000-0000-0000-000000000001
+E: MAJOR=259
+E: MINOR=0
+E: SUBSYSTEM=block
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 15/76] disks: list: add include-remote parameter
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (13 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 14/76] diskmanage: qualify NVMe over fabrics transport Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 16/76] diskmanage: include iSCSI session devices in disk enumeration Thomas Lamprecht
` (60 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Generic disk selectors are primarily used for node-local storage.
Offering devices attached through an identifiable remote transport
there makes accidental reuse too easy, regardless of whether the
administrator assigned the LUN exclusively or shared it.
Require an explicit opt-in for FC, FCoE, iSCSI, and remote NVMe
attachments. SAS remains visible because its transport alone does not
identify a remote attachment, while NVMe loopback is local.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: drop redundant comments]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Disks.pm | 20 +++++-
src/PVE/Diskmanage.pm | 7 ++
src/test/Makefile | 5 +-
src/test/disk_api_test.pm | 113 +++++++++++++++++++++++++++++++++
src/test/run_disk_api_tests.pl | 11 ++++
5 files changed, 154 insertions(+), 2 deletions(-)
create mode 100644 src/test/disk_api_test.pm
create mode 100755 src/test/run_disk_api_tests.pl
diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index f448f8bd..ada836b6 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -82,7 +82,7 @@ __PACKAGE__->register_method({
name => 'list',
path => 'list',
method => 'GET',
- description => "List local disks.",
+ description => "List disks attached to the node.",
protected => 1,
proxyto => 'node',
permissions => {
@@ -98,6 +98,14 @@ __PACKAGE__->register_method({
optional => 1,
default => 0,
},
+ 'include-remote' => {
+ description => "Also include devices attached through a remote storage"
+ . " transport (for example FC, iSCSI, or NVMe over TCP), which are"
+ . " excluded by default.",
+ type => 'boolean',
+ optional => 1,
+ default => 0,
+ },
skipsmart => {
description => "Skip smart checks.",
type => 'boolean',
@@ -155,16 +163,26 @@ __PACKAGE__->register_method({
my $skipsmart = $param->{skipsmart} // 0;
my $include_partitions = $param->{'include-partitions'} // 0;
+ my $include_remote = $param->{'include-remote'} // 0;
my $disks = PVE::Diskmanage::get_disks(
undef, $skipsmart, $include_partitions,
);
+ my $by_devpath = {};
+ $by_devpath->{ $disks->{$_}->{devpath} } = $disks->{$_} for keys %$disks;
+
my $type = $param->{type} // '';
my $result = [];
foreach my $disk (sort keys %$disks) {
my $entry = $disks->{$disk};
+ if (!$include_remote) {
+ my $device = $entry;
+ $device = $by_devpath->{ $entry->{parent} } // $entry
+ if defined($entry->{parent});
+ next if PVE::Diskmanage::is_remote_transport($device->{transport});
+ }
if ($type eq 'journal_disks') {
next if $entry->{osdid} >= 0;
if (my $usage = $entry->{used}) {
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index bc81e854..6ea9f2e3 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -523,6 +523,13 @@ sub get_nvme_transport {
return $transport;
}
+# SAS does not distinguish local from remote attachments, so it is not classified as remote.
+sub is_remote_transport {
+ my ($transport) = @_;
+
+ return ($transport // '') =~ m/^(?:fc|fcoe|iscsi|nvme-(?:fc|tcp|rdma))$/;
+}
+
my sub is_ssdlike {
my ($type) = @_;
return $type eq 'ssd' || $type eq 'nvme';
diff --git a/src/test/Makefile b/src/test/Makefile
index 415d62bf..ea11d399 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_bwlimit test_plugin test_ovf test_volume_access test_san_libraries
+test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_san_libraries
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -32,6 +32,9 @@ test_status_command: run_status_command_tests.pl run_status_command_contention_t
test_disklist: run_disk_tests.pl
./run_disk_tests.pl
+test_disk_api: run_disk_api_tests.pl
+ ./run_disk_api_tests.pl
+
test_bwlimit: run_bwlimit_tests.pl
./run_bwlimit_tests.pl
diff --git a/src/test/disk_api_test.pm b/src/test/disk_api_test.pm
new file mode 100644
index 00000000..c7735c0f
--- /dev/null
+++ b/src/test/disk_api_test.pm
@@ -0,0 +1,113 @@
+package PVE::API2::Disks::Test;
+
+use strict;
+use warnings;
+
+use lib qw(..);
+
+use PVE::API2::Disks;
+use PVE::Diskmanage;
+
+use Test::MockModule;
+use Test::More;
+
+my @transport_cases = (
+ ['none', undef, 0],
+ ['unknown', 'unknown', 0],
+ ['ata', 'ata', 0],
+ ['sata', 'sata', 0],
+ ['sas', 'sas', 0],
+ ['spi', 'spi', 0],
+ ['sbp', 'sbp', 0],
+ ['usb', 'usb', 0],
+ ['fc', 'fc', 1],
+ ['fcoe', 'fcoe', 1],
+ ['iscsi', 'iscsi', 1],
+ ['nvme', 'nvme', 0],
+ ['virtio', 'virtio', 0],
+ ['nvme-fc', 'nvme-fc', 1],
+ ['nvme-tcp', 'nvme-tcp', 1],
+ ['nvme-rdma', 'nvme-rdma', 1],
+ ['nvme-loop', 'nvme-loop', 0],
+);
+
+my $disks = {};
+my (@all, @local);
+for my $case (@transport_cases) {
+ my ($name, $transport, $remote) = $case->@*;
+ my $devpath = "/dev/test-$name";
+ my $entry = {
+ devpath => $devpath,
+ gpt => 0,
+ osdid => -1,
+ };
+ $entry->{transport} = $transport if defined($transport);
+ $disks->{$name} = $entry;
+ push @all, $devpath;
+ push @local, $devpath if !$remote;
+}
+
+my $diskmanage_module = Test::MockModule->new('PVE::Diskmanage', no_auto => 1);
+$diskmanage_module->mock('get_disks' => sub { return $disks });
+
+my $list_handler = PVE::API2::Disks->map_method_by_name('list')->{code};
+my $list_devpaths = sub {
+ my ($param) = @_;
+ my $result = $list_handler->({ node => 'test-node', skipsmart => 1, $param->%* });
+ return [sort map { $_->{devpath} } $result->@*];
+};
+
+is_deeply(
+ $list_devpaths->({}), [sort @local], 'the default disk list excludes remote transports',
+);
+
+is_deeply(
+ $list_devpaths->({ type => 'unused' }),
+ [sort @local],
+ 'unused disk selection excludes remote transports',
+);
+
+is_deeply(
+ $list_devpaths->({ 'include-remote' => 1 }),
+ [sort @all],
+ 'include-remote returns all transports',
+);
+
+$diskmanage_module->mock(
+ 'get_disks' => sub {
+ return {
+ 'fc-disk' => {
+ devpath => '/dev/fc-disk',
+ gpt => 0,
+ osdid => -1,
+ transport => 'fc',
+ },
+ 'fc-partition' => {
+ devpath => '/dev/fc-disk1',
+ gpt => 0,
+ osdid => -1,
+ parent => '/dev/fc-disk',
+ },
+ 'loop-disk' => {
+ devpath => '/dev/loop-disk',
+ gpt => 0,
+ osdid => -1,
+ transport => 'nvme-loop',
+ },
+ 'loop-partition' => {
+ devpath => '/dev/loop-disk1',
+ gpt => 0,
+ osdid => -1,
+ parent => '/dev/loop-disk',
+ },
+ };
+ },
+);
+
+is_deeply(
+ $list_devpaths->({ 'include-partitions' => 1 }),
+ ['/dev/loop-disk', '/dev/loop-disk1'],
+ 'partitions inherit the visibility of their parent disk',
+);
+
+done_testing();
diff --git a/src/test/run_disk_api_tests.pl b/src/test/run_disk_api_tests.pl
new file mode 100755
index 00000000..75fc98c6
--- /dev/null
+++ b/src/test/run_disk_api_tests.pl
@@ -0,0 +1,11 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use TAP::Harness;
+
+my $harness = TAP::Harness->new({ verbosity => -2 });
+my $res = $harness->runtests("disk_api_test.pm");
+
+exit -1 if !$res || $res->{failed} || $res->{parse_errors};
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 16/76] diskmanage: include iSCSI session devices in disk enumeration
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (14 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 15/76] disks: list: add include-remote parameter Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 17/76] diskmanage: link multipath member disks to their map device Thomas Lamprecht
` (59 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Disk enumeration excluded iSCSI devices, making them unavailable to
SAN inventory clients. Include them with transport 'iscsi' so clients
can filter them as needed. The disk list still hides them unless
include-remote is set.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: keep iSCSI LUNs and their partitions out of local disk operations
(GPT init, LVM, LVM-thin, ZFS, directory), as other nodes may use them;
skip SMART queries, which can stall the disk list during recovery;
drop the volume group creation note, covered by the explicit SAN
opt-in; clarify the transport description]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Disks.pm | 3 +-
src/PVE/Diskmanage.pm | 45 ++++++++++++-------
src/test/disk_tests/iscsi/disklist | 1 +
.../disk_tests/iscsi/disklist_expected.json | 19 ++++++++
src/test/disk_tests/iscsi/lsblk | 5 +++
src/test/disk_tests/iscsi/sda/device/model | 1 +
src/test/disk_tests/iscsi/sda/device/vendor | 1 +
.../disk_tests/iscsi/sda/queue/rotational | 1 +
src/test/disk_tests/iscsi/sda/size | 1 +
src/test/disk_tests/iscsi/sda_udevadm | 21 +++++++++
src/test/disklist_test.pm | 15 ++++---
11 files changed, 90 insertions(+), 23 deletions(-)
create mode 100644 src/test/disk_tests/iscsi/disklist
create mode 100644 src/test/disk_tests/iscsi/disklist_expected.json
create mode 100644 src/test/disk_tests/iscsi/lsblk
create mode 100644 src/test/disk_tests/iscsi/sda/device/model
create mode 100644 src/test/disk_tests/iscsi/sda/device/vendor
create mode 100644 src/test/disk_tests/iscsi/sda/queue/rotational
create mode 100644 src/test/disk_tests/iscsi/sda/size
create mode 100644 src/test/disk_tests/iscsi/sda_udevadm
diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index ada836b6..fe383319 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -146,7 +146,8 @@ __PACKAGE__->register_method({
transport => {
type => 'string',
description =>
- "The device transport, for example 'sata', 'sas', 'fc', 'nvme', or 'usb'.",
+ "The device transport, for example 'sata', 'sas', 'usb', 'fc',"
+ . " 'iscsi', 'nvme', 'nvme-fc', or 'nvme-tcp'.",
optional => 1,
},
parent => {
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index 6ea9f2e3..be7d9828 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -89,6 +89,28 @@ sub parse_wipefs_signatures {
return $probe->{signatures};
}
+sub is_iscsi_device {
+ my ($dev) = @_;
+
+ my $link = readlink("/sys/block/$dev");
+ return defined($link) && $link =~ m{/session\d+/} ? 1 : 0;
+}
+
+# Partitions inherit their containing disk's classification, as they have no transport of their own.
+sub assert_local_device {
+ my ($disk, $disklist) = @_;
+
+ my $dev = strip_dev($disk);
+ $disklist //= get_disks($dev, 1, 1);
+ my $entry = $disklist->{$dev} // die "'$disk' is not a valid local disk\n";
+
+ my $parent = is_partition("/dev/$dev") ? strip_dev(get_blockdev("/dev/$dev")) : $dev;
+ my $transport = $entry->{transport} // ($disklist->{$parent} // {})->{transport} // '';
+ die "'$disk' is attached over iSCSI and not a local disk\n"
+ if $transport eq 'iscsi' || is_iscsi_device($parent);
+ return;
+}
+
sub disk_is_used {
my ($disk) = @_;
@@ -98,9 +120,10 @@ sub disk_is_used {
my $disklist = get_disks($dev, 1, 1);
die "'$disk' is not a valid local disk\n" if !defined($disklist->{$dev});
- return 1 if $disklist->{$dev}->{used};
- return 0;
+ assert_local_device($disk, $disklist);
+
+ return $disklist->{$dev}->{used} ? 1 : 0;
}
sub get_smart_data {
@@ -493,16 +516,6 @@ sub dir_is_empty {
return 1;
}
-sub is_iscsi {
- my ($sysdir) = @_;
-
- if (-l $sysdir && readlink($sysdir) =~ m|host[^/]*/session[^/]*|) {
- return 1;
- }
-
- return 0;
-}
-
# Returns the NVMe transport (pcie, fc, tcp, rdma, loop) of a namespace, either directly from
# its controller, or from any controller of its subsystem for native NVMe multipath setups.
sub get_nvme_transport {
@@ -619,9 +632,6 @@ sub get_disks {
my $sysdir = "/sys/block/$dev";
- # we do not want iscsi devices
- return if is_iscsi($sysdir);
-
my $nvme_transport;
$nvme_transport = get_nvme_transport($sysdir) if $dev =~ m/^nvme/;
@@ -644,7 +654,10 @@ sub get_disks {
}
my ($health, $wearout) = ('UNKNOWN', 'N/A');
- if (!$nosmart) {
+ # SMART of an iSCSI LUN goes to the target, which can stall while its session recovers
+ my $tran = $lsblk_info->{$devpath}->{tran};
+ my $iscsi = defined($tran) ? $tran eq 'iscsi' : is_iscsi_device($dev);
+ if (!$nosmart && !$iscsi) {
eval {
my $smartdata = get_smart_data($devpath, !is_ssdlike($type));
$health = $smartdata->{health} if $smartdata->{health};
diff --git a/src/test/disk_tests/iscsi/disklist b/src/test/disk_tests/iscsi/disklist
new file mode 100644
index 00000000..9191c61b
--- /dev/null
+++ b/src/test/disk_tests/iscsi/disklist
@@ -0,0 +1 @@
+sda
diff --git a/src/test/disk_tests/iscsi/disklist_expected.json b/src/test/disk_tests/iscsi/disklist_expected.json
new file mode 100644
index 00000000..ca334106
--- /dev/null
+++ b/src/test/disk_tests/iscsi/disklist_expected.json
@@ -0,0 +1,19 @@
+{
+ "sda": {
+ "devpath": "/dev/sda",
+ "size": 10737418240,
+ "vendor": "LIO-ORG",
+ "model": "vdisk1",
+ "serial": "ISER1",
+ "wwn": "0x60014051111111111",
+ "gpt": 0,
+ "rpm": -1,
+ "type": "unknown",
+ "health": "UNKNOWN",
+ "wearout": "N/A",
+ "osdid": -1,
+ "osdid-list": null,
+ "by_id_link": "/dev/disk/by-id/scsi-360014051111111111",
+ "transport": "iscsi"
+ }
+}
diff --git a/src/test/disk_tests/iscsi/lsblk b/src/test/disk_tests/iscsi/lsblk
new file mode 100644
index 00000000..526d27c5
--- /dev/null
+++ b/src/test/disk_tests/iscsi/lsblk
@@ -0,0 +1,5 @@
+{
+ "blockdevices": [
+ {"path":"/dev/sda", "parttype":null, "fstype":null, "tran":"iscsi"}
+ ]
+}
diff --git a/src/test/disk_tests/iscsi/sda/device/model b/src/test/disk_tests/iscsi/sda/device/model
new file mode 100644
index 00000000..b14d6fe7
--- /dev/null
+++ b/src/test/disk_tests/iscsi/sda/device/model
@@ -0,0 +1 @@
+vdisk1
diff --git a/src/test/disk_tests/iscsi/sda/device/vendor b/src/test/disk_tests/iscsi/sda/device/vendor
new file mode 100644
index 00000000..968831cd
--- /dev/null
+++ b/src/test/disk_tests/iscsi/sda/device/vendor
@@ -0,0 +1 @@
+LIO-ORG
diff --git a/src/test/disk_tests/iscsi/sda/queue/rotational b/src/test/disk_tests/iscsi/sda/queue/rotational
new file mode 100644
index 00000000..d00491fd
--- /dev/null
+++ b/src/test/disk_tests/iscsi/sda/queue/rotational
@@ -0,0 +1 @@
+1
diff --git a/src/test/disk_tests/iscsi/sda/size b/src/test/disk_tests/iscsi/sda/size
new file mode 100644
index 00000000..14faeece
--- /dev/null
+++ b/src/test/disk_tests/iscsi/sda/size
@@ -0,0 +1 @@
+20971520
diff --git a/src/test/disk_tests/iscsi/sda_udevadm b/src/test/disk_tests/iscsi/sda_udevadm
new file mode 100644
index 00000000..ca9f7815
--- /dev/null
+++ b/src/test/disk_tests/iscsi/sda_udevadm
@@ -0,0 +1,21 @@
+P: /devices/platform/host5/session1/target5:0:0/5:0:0:0/block/sda
+N: sda
+S: disk/by-id/scsi-360014051111111111
+S: disk/by-id/wwn-0x60014051111111111
+S: disk/by-path/ip-192.168.1.10:3260-iscsi-iqn.2003-01.org.linux-iscsi.storage:target1-lun-0
+E: DEVLINKS=/dev/disk/by-id/scsi-360014051111111111 /dev/disk/by-id/wwn-0x60014051111111111 /dev/disk/by-path/ip-192.168.1.10:3260-iscsi-iqn.2003-01.org.linux-iscsi.storage:target1-lun-0
+E: DEVNAME=/dev/sda
+E: DEVPATH=/devices/platform/host5/session1/target5:0:0/5:0:0:0/block/sda
+E: DEVTYPE=disk
+E: ID_BUS=scsi
+E: ID_MODEL=vdisk1
+E: ID_PATH=ip-192.168.1.10:3260-iscsi-iqn.2003-01.org.linux-iscsi.storage:target1-lun-0
+E: ID_SCSI=1
+E: ID_SERIAL=36001405111111111
+E: ID_SERIAL_SHORT=ISER1
+E: ID_TYPE=disk
+E: ID_VENDOR=LIO-ORG
+E: ID_WWN=0x60014051111111111
+E: MAJOR=8
+E: MINOR=0
+E: SUBSYSTEM=block
diff --git a/src/test/disklist_test.pm b/src/test/disklist_test.pm
index 9e35430d..dd4e4d73 100644
--- a/src/test/disklist_test.pm
+++ b/src/test/disklist_test.pm
@@ -105,10 +105,6 @@ sub mocked_get_sysdir_size {
return &$originalsub($param);
}
-sub mocked_is_iscsi {
- return 0;
-}
-
sub mocked_file_read_firstline {
my ($path) = @_;
@@ -242,6 +238,12 @@ sub test_disk_list {
is_deeply($multipath, $expected_multipath, 'multipath list should be the same');
}
+ if ($testcasedir eq 'iscsi') {
+ eval { PVE::Diskmanage::assert_disk_unused('/dev/sda'); };
+ $testcount++;
+ like($@, qr/attached over iSCSI/, 'local disk operations reject an iSCSI LUN');
+ }
+
done_testing($testcount);
};
}
@@ -262,8 +264,6 @@ $diskmanage_module->mock('get_sysdir_info' => \&mocked_get_sysdir_info);
print("\tMocked get_sysdir_info\n");
$diskmanage_module->mock('get_sysdir_size' => \&mocked_get_sysdir_size);
print("\tMocked get_sysdir_size\n");
-$diskmanage_module->mock('is_iscsi' => \&mocked_is_iscsi);
-print("\tMocked is_iscsi\n");
$diskmanage_module->mock('assert_blockdev' => sub { return 1; });
print("\tMocked assert_blockdev\n");
$diskmanage_module->mock(
@@ -297,6 +297,9 @@ $diskmanage_module->mock(
print("\tMocked abs_path\n");
$diskmanage_module->mock('check_bin' => sub { return 1; });
print("\tMocked check_bin\n");
+# the test cases name iSCSI LUNs through the lsblk transport, the sysfs of the host must not count
+$diskmanage_module->mock('is_iscsi_device' => sub { return 0; });
+print("\tMocked is_iscsi_device\n");
my $tools_module = Test::MockModule->new('PVE::ProcFSTools', no_auto => 1);
$tools_module->mock('parse_proc_mounts' => \&mocked_parse_proc_mounts);
print("\tMocked parse_proc_mounts\n");
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 17/76] diskmanage: link multipath member disks to their map device
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (15 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 16/76] diskmanage: include iSCSI session devices in disk enumeration Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 18/76] iscsi: factor out session device map from device list Thomas Lamprecht
` (58 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
The generic 'Device Mapper' usage neither identifies a multipath map
nor links its member paths. Report the map's device path in the
multipath property so clients can group members or hide them in favor
of the map.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: drop redundant comments; clarify the multipath member description]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Disks.pm | 7 +++++
src/PVE/Diskmanage.pm | 31 +++++++++++++++++++
.../multipath/disklist_expected.json | 12 ++++---
3 files changed, 46 insertions(+), 4 deletions(-)
diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index fe383319..dd3524b5 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -150,6 +150,13 @@ __PACKAGE__->register_method({
. " 'iscsi', 'nvme', 'nvme-fc', or 'nvme-tcp'.",
optional => 1,
},
+ multipath => {
+ type => 'string',
+ description =>
+ 'The device path of the multipath map this device belongs to.'
+ . ' Only set for multipath member devices.',
+ optional => 1,
+ },
parent => {
type => 'string',
description => 'For partitions only. The device path of '
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index be7d9828..807ef0ca 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -574,12 +574,40 @@ sub mounted_paths {
return $mounted;
}
+my sub get_multipath_slave_map {
+ my $slave_map = {};
+
+ dir_glob_foreach(
+ '/sys/block',
+ 'dm-\d+',
+ sub {
+ my ($dev) = @_;
+ my $sysdir = "/sys/block/$dev";
+
+ my $uuid = file_read_firstline("$sysdir/dm/uuid") // return;
+ return if $uuid !~ m/^mpath-/;
+
+ my $name = file_read_firstline("$sysdir/dm/name") // return;
+
+ dir_glob_foreach(
+ "$sysdir/slaves",
+ '[^.].*',
+ sub { $slave_map->{ $_[0] } = "/dev/mapper/$name"; },
+ );
+ },
+ );
+
+ return $slave_map;
+}
+
sub get_disks {
my ($disks, $nosmart, $include_partitions) = @_;
my $disklist = {};
my $mounted = mounted_blockdevs();
+ my $mpath_slave_map = get_multipath_slave_map();
+
my $lsblk_info = get_lsblk_info();
my $journalhash = get_ceph_journals($lsblk_info);
@@ -697,6 +725,9 @@ sub get_disks {
$disklist->{$dev}->{transport} = $transport if defined($transport);
$disklist->{$dev}->{mounted} = 1 if exists $mounted->{$devpath};
+ my $mpath = $mpath_slave_map->{$dev};
+ $disklist->{$dev}->{multipath} = $mpath if defined($mpath);
+
my $by_id_link = $data->{by_id_link};
$disklist->{$dev}->{by_id_link} = $by_id_link if defined($by_id_link);
diff --git a/src/test/disk_tests/multipath/disklist_expected.json b/src/test/disk_tests/multipath/disklist_expected.json
index ec442969..001193da 100644
--- a/src/test/disk_tests/multipath/disklist_expected.json
+++ b/src/test/disk_tests/multipath/disklist_expected.json
@@ -15,7 +15,8 @@
"osdid-list": null,
"by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000a",
"transport": "fc",
- "used": "Device Mapper"
+ "used": "Device Mapper",
+ "multipath": "/dev/mapper/mpatha"
},
"sdb": {
"devpath": "/dev/sdb",
@@ -33,7 +34,8 @@
"osdid-list": null,
"by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000b",
"transport": "fc",
- "used": "Device Mapper"
+ "used": "Device Mapper",
+ "multipath": "/dev/mapper/mpatha"
},
"sdc": {
"devpath": "/dev/sdc",
@@ -51,7 +53,8 @@
"osdid-list": null,
"by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000c",
"transport": "fc",
- "used": "Device Mapper"
+ "used": "Device Mapper",
+ "multipath": "/dev/mapper/mpathb"
},
"sdd": {
"devpath": "/dev/sdd",
@@ -69,7 +72,8 @@
"osdid-list": null,
"by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000d",
"transport": "fc",
- "used": "Device Mapper"
+ "used": "Device Mapper",
+ "multipath": "/dev/mapper/mpathb"
},
"nvme8n1": {
"devpath": "/dev/nvme8n1",
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 18/76] iscsi: factor out session device map from device list
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (16 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 17/76] diskmanage: link multipath member disks to their map device Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 19/76] api: scan: add a SAN LUN inventory Thomas Lamprecht
` (57 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Checking whether a disk belongs to configured iSCSI storage needs the
session-to-device information collected by device listing. Expose it
separately as a map from kernel device names to targets and addresses.
This does not change behavior.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: drop redundant comments; keep the per-session LUN walk and read
block devices below the sysfs root of the listing]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Storage/ISCSIPlugin.pm | 70 ++++++++++++++++++++++------------
1 file changed, 45 insertions(+), 25 deletions(-)
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index 81f06979..22f6c44f 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -306,14 +306,12 @@ my sub foreach_session_lun {
);
}
-sub iscsi_device_list {
+sub iscsi_session_device_map {
my $res = {};
my $dirname = "$SYSFS/class/iscsi_session";
- my $stable_paths = load_stable_scsi_paths();
-
dir_glob_foreach(
$dirname,
'session(\d+)',
@@ -339,28 +337,11 @@ sub iscsi_device_list {
}
return if !$bdev;
- #check multipath
- if (-d "$SYSFS/block/$bdev/holders") {
- my $multipathdev =
- dir_glob_regex("$SYSFS/block/$bdev/holders", '[A-Za-z]\S*');
- $bdev = $multipathdev if $multipathdev;
- }
-
- my $blockdev = $stable_paths->{$bdev};
- return if !$blockdev;
-
- my $size = file_read_firstline("$SYSFS/block/$bdev/size");
- return if !$size;
-
- my $volid = "$channel.$id.$lun.$blockdev";
-
- $res->{$target}->{$volid} = {
- 'format' => 'raw',
- 'size' => int($size * 512),
- 'vmid' => 0, # not assigned to any vm
- 'channel' => int($channel),
- 'id' => int($id),
- 'lun' => int($lun),
+ $res->{$bdev} = {
+ target => $target,
+ channel => int($channel),
+ id => int($id),
+ lun => int($lun),
};
},
);
@@ -371,6 +352,45 @@ sub iscsi_device_list {
return $res;
}
+sub iscsi_device_list {
+
+ my $res = {};
+
+ my $stable_paths = load_stable_scsi_paths();
+
+ my $device_map = iscsi_session_device_map();
+
+ for my $bdev (sort keys %$device_map) {
+ my $info = $device_map->{$bdev};
+ my ($channel, $id, $lun) = $info->@{qw(channel id lun)};
+
+ #check multipath
+ if (-d "$SYSFS/block/$bdev/holders") {
+ my $multipathdev = dir_glob_regex("$SYSFS/block/$bdev/holders", '[A-Za-z]\S*');
+ $bdev = $multipathdev if $multipathdev;
+ }
+
+ my $blockdev = $stable_paths->{$bdev};
+ next if !$blockdev;
+
+ my $size = file_read_firstline("$SYSFS/block/$bdev/size");
+ next if !$size;
+
+ my $volid = "$channel.$id.$lun.$blockdev";
+
+ $res->{ $info->{target} }->{$volid} = {
+ 'format' => 'raw',
+ 'size' => int($size * 512),
+ 'vmid' => 0, # not assigned to any vm
+ 'channel' => $channel,
+ 'id' => $id,
+ 'lun' => $lun,
+ };
+ }
+
+ return $res;
+}
+
# Configuration
sub type {
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 19/76] api: scan: add a SAN LUN inventory
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (17 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 18/76] iscsi: factor out session device map from device list Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 20/76] pvesm: expose the " Thomas Lamprecht
` (56 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
The storage wizard needs SAN devices suitable for shared LVM. The node
disk list omits multipath maps and cannot alone associate devices with
volume groups and configured storages.
Return one entry per whole disk or multipath map, folding member paths
into the map. Include transport, usage, volume-group names, and storage
IDs so clients can identify candidates and exclude LUNs already used
by configured storages. Also report storages using the LUN's target,
preferring those serving raw VM disks, so clients can warn before
initializing such a LUN.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: build the inventory in the new PVE::Storage::SANScan module; leave
out the placeholder 'unknown' that the disk scan fills in for a vendor,
model, serial or WWN it could not read; attribute a LUN to any storage
of its target, preferring one that serves disk images from it; name the
path count paths-total; also allow reads with Sys.Audit on the node;
clarify the transport description]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Storage/Scan.pm | 89 +++++++++++++++++++++++
src/PVE/Storage/Makefile | 3 +-
src/PVE/Storage/SANScan.pm | 129 ++++++++++++++++++++++++++++++++++
src/test/Makefile | 5 +-
src/test/run_san_api_tests.pl | 65 +++++++++++++++++
5 files changed, 289 insertions(+), 2 deletions(-)
create mode 100644 src/PVE/Storage/SANScan.pm
create mode 100755 src/test/run_san_api_tests.pl
diff --git a/src/PVE/API2/Storage/Scan.pm b/src/PVE/API2/Storage/Scan.pm
index a58bdd80..58570f46 100644
--- a/src/PVE/API2/Storage/Scan.pm
+++ b/src/PVE/API2/Storage/Scan.pm
@@ -9,9 +9,11 @@ use PVE::JSONSchema qw(get_standard_option);
use PVE::RESTHandler;
use PVE::SafeSyslog;
+use PVE::Storage::ISCSIPlugin;
use PVE::Storage::LVMPlugin;
use PVE::Storage::LvmThinPlugin;
use PVE::Storage::PBSPlugin;
+use PVE::Storage::SANScan;
use PVE::Storage;
use base qw(PVE::RESTHandler);
@@ -49,6 +51,7 @@ __PACKAGE__->register_method({
{ method => 'lvm' },
{ method => 'nfs' },
{ method => 'pbs' },
+ { method => 'san-luns' },
{ method => 'zfs' },
];
@@ -373,6 +376,92 @@ __PACKAGE__->register_method({
},
});
+__PACKAGE__->register_method({
+ name => 'sanlunscan',
+ path => 'san-luns',
+ method => 'GET',
+ description => "List block devices (disks and multipath devices) together with their"
+ . " usage, for example as candidates for a SAN backed LVM volume group.",
+ protected => 1,
+ proxyto => "node",
+ permissions => {
+ check => [
+ 'or',
+ ['perm', '/nodes/{node}', ['Sys.Audit']],
+ ['perm', '/storage', ['Datastore.Allocate']],
+ ],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ },
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => "object",
+ properties => {
+ devpath => {
+ description => "The device path.",
+ type => 'string',
+ },
+ size => {
+ description => "The device size in bytes.",
+ type => 'integer',
+ renderer => 'bytes',
+ optional => 1,
+ },
+ transport => {
+ description =>
+ "The device transport, for example 'sata', 'sas', 'usb', 'fc',"
+ . " 'iscsi', 'nvme', 'nvme-fc', or 'nvme-tcp'.",
+ type => 'string',
+ optional => 1,
+ },
+ vendor => { type => 'string', optional => 1 },
+ model => { type => 'string', optional => 1 },
+ serial => { type => 'string', optional => 1 },
+ wwn => { type => 'string', optional => 1 },
+ 'paths-total' => {
+ description => "The number of paths of a multipath mapped device.",
+ type => 'integer',
+ optional => 1,
+ },
+ usage => {
+ description => "How the device is currently used. 'lvm' means it is an LVM"
+ . " physical volume, see the 'vgname' property.",
+ type => 'string',
+ enum => ['unused', 'lvm', 'other'],
+ },
+ vgname => {
+ description => "The name of the LVM volume group the device belongs to.",
+ type => 'string',
+ optional => 1,
+ },
+ 'used-by-storage' => {
+ description => "The ID of an existing storage definition that uses this"
+ . " device, either through the LVM volume group on it or because it"
+ . " is a LUN of the storage's iSCSI target.",
+ type => 'string',
+ optional => 1,
+ },
+ detail => {
+ description =>
+ "Details about other usage, for example the file system type.",
+ type => 'string',
+ optional => 1,
+ },
+ },
+ },
+ },
+ code => sub {
+ my ($param) = @_;
+
+ return PVE::Storage::SANScan::scan(PVE::Storage::config(), $param->{node});
+ },
+});
+
__PACKAGE__->register_method({
name => 'zfsscan',
path => 'zfs',
diff --git a/src/PVE/Storage/Makefile b/src/PVE/Storage/Makefile
index 47813553..f6906e5e 100644
--- a/src/PVE/Storage/Makefile
+++ b/src/PVE/Storage/Makefile
@@ -15,7 +15,8 @@ SOURCES= \
BTRFSPlugin.pm \
LvmThinPlugin.pm \
ESXiPlugin.pm \
- StatusCommand.pm
+ StatusCommand.pm \
+ SANScan.pm
.PHONY: install
install:
diff --git a/src/PVE/Storage/SANScan.pm b/src/PVE/Storage/SANScan.pm
new file mode 100644
index 00000000..6e968f65
--- /dev/null
+++ b/src/PVE/Storage/SANScan.pm
@@ -0,0 +1,129 @@
+package PVE::Storage::SANScan;
+
+use v5.36;
+
+use PVE::Diskmanage;
+use PVE::Storage;
+use PVE::Storage::ISCSIPlugin;
+use PVE::Storage::LVMPlugin;
+
+my sub collect_inventory($cfg, $node) {
+ my $disks = PVE::Diskmanage::get_disks(undef, 1, 0);
+ my $multipath = PVE::Diskmanage::get_multipath_disks();
+
+ # map PV device to VG name, PVs may sit on a partition of a listed device
+ my $vgs = PVE::Storage::LVMPlugin::lvm_vgs(1);
+ my $pv2vg = {};
+ for my $vgname (sort keys %$vgs) {
+ for my $pv (@{ $vgs->{$vgname}->{pvs} // [] }) {
+ $pv2vg->{ $pv->{name} } = $vgname;
+ }
+ }
+ my $find_vg = sub($devpath) {
+ return $pv2vg->{$devpath} if defined($pv2vg->{$devpath});
+ for my $pv (sort keys %$pv2vg) {
+ return $pv2vg->{$pv} if $pv =~ m/^\Q$devpath\Ep?\d+$/;
+ }
+ return undef;
+ };
+
+ my $vg_used_by = {};
+ my $iscsi_target_used_by = {};
+ my $serves_images = sub($sid) {
+ my $scfg = $cfg->{ids}->{$sid};
+ return
+ $scfg->{content}->{images}
+ || $scfg->{type} eq 'iscsidirect'
+ || $scfg->{type} eq 'zfs';
+ };
+ my $attribute = sub($owners, $target, $sid) {
+ my $owner = $owners->{$target};
+ $owners->{$target} = $sid
+ if !defined($owner) || ($serves_images->($sid) && !$serves_images->($owner));
+ };
+ for my $sid (sort keys %{ $cfg->{ids} }) {
+ my $scfg = $cfg->{ids}->{$sid};
+ if (defined(my $vgname = $scfg->{vgname})) {
+ $vg_used_by->{$vgname} //= $sid;
+ }
+ if (defined(my $target = $scfg->{target})) {
+ $attribute->($iscsi_target_used_by, $target, $sid);
+ }
+ }
+
+ # map kernel device name to the storage using its iSCSI target
+ my $iscsi_used_by = {};
+ if (scalar(keys %$iscsi_target_used_by)) {
+ my $device_map = PVE::Storage::ISCSIPlugin::iscsi_session_device_map();
+ for my $bdev (sort keys %$device_map) {
+ my $sid = $iscsi_target_used_by->{ $device_map->{$bdev}->{target} } // next;
+ $iscsi_used_by->{$bdev} = $sid;
+ }
+ }
+ return {
+ disks => $disks,
+ multipath => $multipath,
+ vg_used_by => $vg_used_by,
+ iscsi_target_used_by => $iscsi_target_used_by,
+ find_vg => $find_vg,
+ iscsi_used_by => $iscsi_used_by,
+ };
+}
+
+my sub build_lun_rows($cfg, $inventory) {
+ my ($disks, $multipath, $vg_used_by, $find_vg, $iscsi_used_by) =
+ $inventory->@{qw(disks multipath vg_used_by find_vg iscsi_used_by)};
+
+ my $records = [];
+ my $add_entry = sub($disk) {
+ my $entry = { devpath => $disk->{devpath}, size => $disk->{size} };
+ # the disk scan fills in 'unknown' for what it could not read
+ $entry->{'paths-total'} = $disk->{paths} if defined($disk->{paths});
+ for my $key (qw(transport vendor model serial wwn)) {
+ my $value = $disk->{$key};
+ $entry->{$key} = $value if defined($value) && $value ne 'unknown';
+ }
+
+ if (!defined($disk->{used})) {
+ $entry->{usage} = 'unused';
+ } elsif ($disk->{used} eq 'LVM') {
+ $entry->{usage} = 'lvm';
+ if (defined(my $vgname = $find_vg->($disk->{devpath}))) {
+ $entry->{vgname} = $vgname;
+ if (defined(my $sid = $vg_used_by->{$vgname})) {
+ $entry->{'used-by-storage'} = $sid;
+ }
+ }
+ } else {
+ $entry->{usage} = 'other';
+ $entry->{detail} = $disk->{used};
+ }
+ if (!defined($entry->{'used-by-storage'})) {
+ my @members =
+ map { $_ =~ s|^/dev/||r } ($disk->{slaves} // [$disk->{devpath}])->@*;
+ for my $member (@members) {
+ if (defined(my $sid = $iscsi_used_by->{$member})) {
+ $entry->{'used-by-storage'} = $sid;
+ last;
+ }
+ }
+ }
+ push $records->@*, { row => $entry, disk => $disk };
+ };
+ for my $dev (sort keys %$disks) {
+ my $disk = $disks->{$dev};
+ next if defined($disk->{multipath});
+ $add_entry->($disk);
+ }
+ $add_entry->($multipath->{$_}) for sort keys %$multipath;
+ return $records;
+}
+
+sub scan($cfg, $node) {
+ my $inventory = collect_inventory($cfg, $node);
+ my $records = build_lun_rows($cfg, $inventory);
+
+ return [map { $_->{row} } $records->@*];
+}
+
+1;
diff --git a/src/test/Makefile b/src/test/Makefile
index ea11d399..26b6b12e 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_san_libraries
+test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_san_api test_san_libraries
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -47,6 +47,9 @@ test_ovf: run_ovf_tests.pl
test_volume_access: run_volume_access_tests.pl
./run_volume_access_tests.pl
+test_san_api: run_san_api_tests.pl
+ ./run_san_api_tests.pl
+
test_iscsi: run_iscsi_discovery_tests.pl run_iscsi_rescan_tests.pl run_iscsi_device_list_tests.pl run_iscsi_session_tests.pl run_iscsi_volume_tests.pl
./run_iscsi_discovery_tests.pl
./run_iscsi_rescan_tests.pl
diff --git a/src/test/run_san_api_tests.pl b/src/test/run_san_api_tests.pl
new file mode 100755
index 00000000..1490e3c2
--- /dev/null
+++ b/src/test/run_san_api_tests.pl
@@ -0,0 +1,65 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::API2::Storage::Scan;
+
+subtest 'node SAN read permissions' => sub {
+ for my $case (
+ ['PVE::API2::Storage::Scan', 'sanlunscan'],
+ ) {
+ my ($class, @methods) = @$case;
+ for my $name (@methods) {
+ is_deeply(
+ $class->map_method_by_name($name)->{permissions}->{check},
+ [
+ 'or',
+ ['perm', '/nodes/{node}', ['Sys.Audit']],
+ ['perm', '/storage', ['Datastore.Allocate']],
+ ],
+ "$class $name read gate",
+ );
+ }
+ }
+};
+
+subtest 'SAN LUN path counts' => sub {
+ my $disk = Test::MockModule->new('PVE::Diskmanage');
+ my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
+ my $storage = Test::MockModule->new('PVE::Storage');
+ my $disks =
+ { map { ("/dev/$_" => { devpath => "/dev/$_", size => 1024, multipath => 'm0' }) }
+ qw(sdb sdc) };
+ $disk->mock(
+ get_disks => sub { $disks },
+ get_multipath_disks => sub {
+ return {
+ '/dev/mapper/m0' => {
+ devpath => '/dev/mapper/m0',
+ size => 1024,
+ slaves => ['/dev/sdb', '/dev/sdc'],
+ paths => 2,
+ },
+ };
+ },
+ );
+ $lvm->mock(lvm_vgs => sub { {} });
+ $storage->mock(config => sub { { ids => {} } });
+ my $declared =
+ PVE::API2::Storage::Scan->map_method_by_name('sanlunscan')->{returns}->{items}
+ ->{properties};
+ my $rows = PVE::API2::Storage::Scan->sanlunscan({ node => 'test' });
+ my $by_dev = { map { ($_->{devpath} => $_) } @$rows };
+ is($by_dev->{'/dev/mapper/m0'}->{'paths-total'}, 2, 'map path count');
+ for my $row (@$rows) {
+ my @undeclared = grep { !$declared->{$_} } sort keys %$row;
+ is_deeply(\@undeclared, [], "$row->{devpath}: every field declared");
+ }
+};
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 20/76] pvesm: expose the SAN LUN inventory
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (18 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 19/76] api: scan: add a SAN LUN inventory Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 21/76] disks: lvm: allow volume groups on SAN LUNs and multipath maps Thomas Lamprecht
` (55 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Make SAN device identities, usage and storage attribution available
from the command line so administrators can inspect candidates before
setting up shared LVM.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: print a concise column set]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/CLI/pvesm.pm | 12 ++++++++++++
src/test/run_san_api_tests.pl | 22 ++++++++++++++++++++++
2 files changed, 34 insertions(+)
diff --git a/src/PVE/CLI/pvesm.pm b/src/PVE/CLI/pvesm.pm
index 1eb83331..501b7b61 100755
--- a/src/PVE/CLI/pvesm.pm
+++ b/src/PVE/CLI/pvesm.pm
@@ -736,6 +736,18 @@ our $cmddef = {
$print_api_result,
$PVE::RESTHandler::standard_output_options,
],
+ 'san-luns' => [
+ "PVE::API2::Storage::Scan",
+ 'sanlunscan',
+ [],
+ { node => $nodename },
+ sub {
+ my ($data, $schema, $options) = @_;
+ my $columns = [qw(devpath size transport usage vgname used-by-storage)];
+ PVE::CLIFormatter::print_api_result($data, $schema, $columns, $options);
+ },
+ $PVE::RESTHandler::standard_output_options,
+ ],
zfs => [
"PVE::API2::Storage::Scan",
'zfsscan',
diff --git a/src/test/run_san_api_tests.pl b/src/test/run_san_api_tests.pl
index 1490e3c2..de35217f 100755
--- a/src/test/run_san_api_tests.pl
+++ b/src/test/run_san_api_tests.pl
@@ -28,6 +28,28 @@ subtest 'node SAN read permissions' => sub {
}
};
+subtest 'SAN CLI columns leave machine output complete' => sub {
+ require PVE::CLI::pvesm;
+ my $formatter = Test::MockModule->new('PVE::CLIFormatter');
+ my $data = [{ devpath => '/dev/sda', serial => 'S1' }];
+ my $schema = PVE::API2::Storage::Scan->map_method_by_name('sanlunscan')->{returns};
+ $formatter->mock(
+ print_api_result => sub {
+ is($_[0], $data, 'complete records forwarded');
+ is($_[1], $schema, 'complete schema forwarded');
+ is_deeply(
+ $_[2],
+ [qw(devpath size transport usage vgname used-by-storage)],
+ 'concise text columns',
+ );
+ is($_[3]->{'output-format'}, 'json', 'format option unchanged');
+ },
+ );
+ $PVE::CLI::pvesm::cmddef->{scan}->{'san-luns'}->[4]
+ ->($data, $schema, { 'output-format' => 'json' });
+ is($schema->{items}->{properties}->{size}->{renderer}, 'bytes', 'size renderer');
+};
+
subtest 'SAN LUN path counts' => sub {
my $disk = Test::MockModule->new('PVE::Diskmanage');
my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 21/76] disks: lvm: allow volume groups on SAN LUNs and multipath maps
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (19 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 20/76] pvesm: expose the " Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 22/76] multipath: add a library around multipathd Thomas Lamprecht
` (54 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
The node disk API only accepts local disks for volume-group creation,
rejecting multipath maps and iSCSI LUNs. Allow this operation to opt in
to iSCSI, Fibre Channel, NVMe over Fabrics, and multipath devices so it
can set up shared LVM. Other local disk operations remain unchanged.
Check and initialize SAN LUNs under the cluster storage configuration
lock, also held by storage-add when initializing a base volume. This
prevents two nodes from both finding a LUN unused and initializing it.
Resolve the submitted device reference again under the locks. Refuse a
local device that has become a SAN device while the task waited, since
it would otherwise bypass the cluster lock. Local devices still need
no quorum.
Co-authored-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Disks/LVM.pm | 62 ++++++++++++++++-----
src/PVE/Diskmanage.pm | 53 ++++++++++++++++--
src/PVE/Storage/LVMPlugin.pm | 30 ++++++++--
src/test/disk_api_test.pm | 105 +++++++++++++++++++++++++++++++++++
src/test/disklist_test.pm | 23 ++++++++
5 files changed, 250 insertions(+), 23 deletions(-)
diff --git a/src/PVE/API2/Disks/LVM.pm b/src/PVE/API2/Disks/LVM.pm
index 9f8f9516..feab0a3b 100644
--- a/src/PVE/API2/Disks/LVM.pm
+++ b/src/PVE/API2/Disks/LVM.pm
@@ -3,6 +3,7 @@ package PVE::API2::Disks::LVM;
use strict;
use warnings;
+use PVE::Storage;
use PVE::Storage::LVMPlugin;
use PVE::Diskmanage;
use PVE::JSONSchema qw(get_standard_option);
@@ -131,7 +132,8 @@ __PACKAGE__->register_method({
name => get_standard_option('pve-storage-id'),
device => {
type => 'string',
- description => 'The block device you want to create the volume group on',
+ description => 'The block device you want to create the volume group on.'
+ . ' Multipath mapped devices are supported.',
},
add_storage => {
description => "Configure storage using the Volume Group",
@@ -149,11 +151,17 @@ __PACKAGE__->register_method({
my $user = $rpcenv->get_user();
my $name = $param->{name};
- my $dev = $param->{device};
+ my $reference = $param->{device};
my $node = $param->{node};
+ my $dev = PVE::Diskmanage::verify_blockdev_path($reference);
+ my $is_san = sub {
+ my ($device) = @_;
+ return PVE::Diskmanage::is_multipath_device($device)
+ || PVE::Diskmanage::is_san_device($device);
+ };
+ my $san = $is_san->($dev);
- $dev = PVE::Diskmanage::verify_blockdev_path($dev);
- PVE::Diskmanage::assert_disk_unused($dev);
+ PVE::Diskmanage::assert_disk_unused($dev, 1);
my $storage_params = {
type => 'lvm',
@@ -176,19 +184,45 @@ __PACKAGE__->register_method({
my $worker = sub {
PVE::Diskmanage::locked_disk_action(sub {
- PVE::Diskmanage::assert_disk_unused($dev);
- die "volume group with name '${name}' already exists on node '${node}'\n"
- if PVE::Storage::LVMPlugin::lvm_vgs()->{$name};
+ my $create = sub {
+ if ($san) {
+ $dev = PVE::Diskmanage::verify_blockdev_path($reference);
+ }
+ die "cannot use '$dev' - device topology changed\n"
+ if !$san && $is_san->($dev);
+ PVE::Diskmanage::assert_disk_unused($dev, 1);
+ die "volume group with name '${name}' already exists on node '${node}'\n"
+ if PVE::Storage::LVMPlugin::lvm_vgs()->{$name};
- if (PVE::Diskmanage::is_partition($dev)) {
- eval { PVE::Diskmanage::change_parttype($dev, '8E00'); };
- warn $@ if $@;
+ if (!$san && PVE::Diskmanage::is_partition($dev)) {
+ eval { PVE::Diskmanage::change_parttype($dev, '8E00'); };
+ warn $@ if $@;
+ }
+
+ my $guard = {
+ reference => $san ? $reference : $dev,
+ san => $san,
+ };
+ PVE::Storage::LVMPlugin::lvm_create_volume_group(
+ $guard->{reference},
+ $name,
+ undef,
+ $guard,
+ );
+
+ PVE::Diskmanage::udevadm_trigger($dev);
+ };
+
+ # Serialize with the storage API so two nodes cannot initialize the same LUN.
+ if ($san) {
+ PVE::Storage::lock_storage_config(
+ $create,
+ "creating volume group '$name' failed",
+ );
+ } else {
+ $create->();
}
- PVE::Storage::LVMPlugin::lvm_create_volume_group($dev, $name);
-
- PVE::Diskmanage::udevadm_trigger($dev);
-
if ($param->{add_storage}) {
PVE::API2::Storage::Config->create_or_update(
$name, $node, $storage_params, $verify_params,
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index 807ef0ca..caad0fee 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -112,16 +112,23 @@ sub assert_local_device {
}
sub disk_is_used {
- my ($disk) = @_;
+ my ($disk, $allow_san) = @_;
my $dev = $disk;
$dev =~ s|^/dev/||;
my $disklist = get_disks($dev, 1, 1);
- die "'$disk' is not a valid local disk\n" if !defined($disklist->{$dev});
+ if (!defined($disklist->{$dev})) {
+ if ($allow_san) {
+ my $multipath = lookup_multipath_map($disk);
+ return defined($multipath->{used}) ? 1 : 0 if defined($multipath);
+ }
- assert_local_device($disk, $disklist);
+ die "'$disk' is not a valid local disk\n";
+ }
+
+ assert_local_device($disk, $disklist) if !$allow_san;
return $disklist->{$dev}->{used} ? 1 : 0;
}
@@ -941,6 +948,42 @@ sub get_multipath_disks {
return $disklist;
}
+sub is_san_device {
+ my ($devpath) = @_;
+
+ my $dev = strip_dev(abs_path($devpath) // $devpath);
+ $dev = strip_dev(get_blockdev("/dev/$dev")) if is_partition("/dev/$dev");
+ if ($dev =~ m/^nvme\d+n\d+$/) {
+ my $transport = get_nvme_transport("/sys/block/$dev") // '';
+ return $transport =~ m/^(?:tcp|rdma|fc)$/ ? 1 : 0;
+ }
+ my $link = readlink("/sys/block/$dev") // return 0;
+ return $link =~ m{/session\d+/|/rport-\d+} ? 1 : 0;
+}
+
+# An unreadable dm UUID counts as multipath so callers fail closed.
+sub is_multipath_device {
+ my ($devpath) = @_;
+
+ my $path = abs_path($devpath) // $devpath;
+ my $dev = strip_dev($path);
+ return 0 if $dev !~ m/^dm-\d+$/;
+
+ my $uuid = file_read_firstline("/sys/block/$dev/dm/uuid");
+ return 1 if !defined($uuid);
+ return $uuid =~ m/^(?:part\d+-)?mpath-/ ? 1 : 0;
+}
+
+sub lookup_multipath_map {
+ my ($devpath) = @_;
+
+ my $path = abs_path($devpath) // $devpath;
+ my $dev = strip_dev($path);
+ return undef if $dev !~ m/^dm-\d+$/;
+
+ return get_multipath_disks()->{$dev};
+}
+
sub get_partnum {
my ($part_path) = @_;
@@ -995,8 +1038,8 @@ sub locked_disk_action {
}
sub assert_disk_unused {
- my ($dev) = @_;
- die "device '$dev' is already in use\n" if disk_is_used($dev);
+ my ($dev, $allow_san) = @_;
+ die "device '$dev' is already in use\n" if disk_is_used($dev, $allow_san);
return;
}
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index d718d8f8..916f1b8f 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -147,10 +147,26 @@ sub clear_first_sector {
}
sub lvm_create_volume_group {
- my ($device, $vgname, $shared) = @_;
+ my ($device, $vgname, $shared, $guard) = @_;
+ $guard = {%$guard} if $guard;
- my $requested_device = $device;
- $device = PVE::Diskmanage::verify_blockdev_path($device);
+ my $requested_device = $guard ? $guard->{reference} : $device;
+ $device = PVE::Diskmanage::verify_blockdev_path($requested_device);
+ my $san = $guard && $guard->{san};
+ if ($guard) {
+ if (!$san) {
+ $guard->{verify} = sub {
+ die "cannot use '$device' - device topology changed\n"
+ if PVE::Diskmanage::is_multipath_device($device)
+ || PVE::Diskmanage::is_san_device($device);
+ };
+ $guard->{verify}->();
+ }
+ }
+ my $check_instance = sub { $guard->{verify}->() if $guard && $guard->{verify} };
+ my $check_safe = sub {
+ $check_instance->();
+ };
my $refuse_data = sub {
my (@types) = @_;
die "device '$device' contains data ("
@@ -208,6 +224,7 @@ sub lvm_create_volume_group {
}
my %signatures = map { $_->{type} => 1 } $probe->@*;
my $res = lvm_pv_info($device, $signatures{LVM2_member});
+ $check_instance->();
# A PV without metadata copies can belong to a VG whose other devices are not visible. The
# sentinel is not a VG name or orphan proof, even if the use flag contradicts it.
@@ -223,17 +240,20 @@ sub lvm_create_volume_group {
delete $signatures{LVM2_member} if $res && !$res->{vgname};
$refuse_data->(keys %signatures) if %signatures;
- clear_first_sector($device); # else pvcreate fails
+ $check_safe->();
+ clear_first_sector($device);
# we use --metadatasize 250k, which reseults in "pe_start = 512"
# so pe_start is aligned on a 128k boundary (advantage for SSDs)
my $cmd = ['/sbin/pvcreate', '--metadatasize', '250k', $device];
+ $check_safe->();
run_command($cmd, errmsg => "pvcreate '$device' error");
$cmd = ['/sbin/vgcreate', $vgname, $device];
# push @$cmd, '-c', 'y' if $shared; # we do not use this yet
+ $check_safe->();
run_command(
$cmd,
errmsg => "vgcreate $vgname $device error",
@@ -1067,6 +1087,8 @@ sub on_add_hook {
assert_device_discard_supported($path)
if discard_on_remove_requested($scfg->{'on-volume-remove'});
+ # The storage API runs this hook under the cluster-wide storage configuration lock, which
+ # the disk API also takes for shared devices, so two nodes cannot initialize the same LUN.
lvm_create_volume_group($path, $scfg->{vgname}, $scfg->{shared});
}
diff --git a/src/test/disk_api_test.pm b/src/test/disk_api_test.pm
index c7735c0f..893b4e04 100644
--- a/src/test/disk_api_test.pm
+++ b/src/test/disk_api_test.pm
@@ -6,7 +6,10 @@ use warnings;
use lib qw(..);
use PVE::API2::Disks;
+use PVE::API2::Disks::LVM;
use PVE::Diskmanage;
+use PVE::Storage;
+use PVE::Storage::LVMPlugin;
use Test::MockModule;
use Test::More;
@@ -110,4 +113,106 @@ is_deeply(
'partitions inherit the visibility of their parent disk',
);
+subtest 'a volume group on a map is created under the storage configuration lock' => sub {
+ my $storage = Test::MockModule->new('PVE::Storage');
+ my $disk = $diskmanage_module;
+ my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
+ my $rpc = Test::MockModule->new('PVE::RPCEnvironment');
+ my (@stack, @created);
+ my $lock = sub {
+ my ($name, $code) = @_;
+ push @stack, $name;
+ my $res = eval { $code->() };
+ my $err = $@;
+ pop @stack;
+ die $err if $err;
+ return $res;
+ };
+ $storage->mock(lock_storage_config => sub { $lock->('storage.cfg', $_[0]) });
+ $disk->mock(locked_disk_action => sub { $lock->('disk', $_[0]) });
+ $disk->mock(verify_blockdev_path => sub { $_[0] });
+ $disk->mock(assert_disk_unused => sub { });
+ $disk->mock(is_partition => sub { 0 });
+ $disk->mock(udevadm_trigger => sub { });
+ $lvm->mock(lvm_vgs => sub { {} });
+ $lvm->mock(lvm_create_volume_group => sub { push @created, [$_[0], join('>', @stack)] });
+ {
+
+ package MultipathTestRPC;
+ sub get_user { 'root@pam' }
+ sub check { 1 }
+ sub fork_worker { $_[4]->(); return 'test-upid' }
+ }
+ $rpc->mock(get => sub { bless {}, 'MultipathTestRPC' });
+
+ my ($is_map, $is_san) = (1, 0);
+ $disk->mock(is_multipath_device => sub { $is_map });
+ $disk->mock(is_san_device => sub { $is_san });
+ PVE::API2::Disks::LVM->create({ node => 'n1', name => 'vg1', device => '/dev/dm-3' });
+ ($is_map, $is_san) = (0, 1);
+ PVE::API2::Disks::LVM->create({ node => 'n1', name => 'vg2', device => '/dev/sdc' });
+ ($is_map, $is_san) = (0, 0);
+ PVE::API2::Disks::LVM->create({ node => 'n1', name => 'vg3', device => '/dev/sdb' });
+ is_deeply(
+ \@created,
+ [
+ ['/dev/dm-3', 'disk>storage.cfg'],
+ ['/dev/sdc', 'disk>storage.cfg'],
+ ['/dev/sdb', 'disk'],
+ ],
+ 'a map or SAN LUN is initialized under the cluster-wide lock, a local disk is not',
+ );
+ @created = ();
+ $disk->mock(assert_disk_unused => sub { die "device is already in use\n" });
+ eval {
+ PVE::API2::Disks::LVM->create({
+ node => 'n1',
+ name => 'vg',
+ device => '/dev/sdc',
+ });
+ };
+ like($@, qr/already in use/, 'disk API rejects a used inventory entry');
+ is_deeply(\@created, [], 'used inventory prevents initialization');
+ $disk->mock(assert_disk_unused => sub { });
+ $lvm->mock(lvm_vgs => sub { { vg => {} } });
+ eval {
+ PVE::API2::Disks::LVM->create({
+ node => 'n1',
+ name => 'vg',
+ device => '/dev/sdc',
+ });
+ };
+ like(
+ $@,
+ qr/volume group with name 'vg' already exists/,
+ 'stale inventory does not permit an existing VG',
+ );
+ is_deeply(\@created, [], 'existing VG prevents initialization');
+ $disk->unmock($_)
+ for qw(
+ locked_disk_action verify_blockdev_path assert_disk_unused is_partition
+ udevadm_trigger is_multipath_device is_san_device
+ );
+};
+
+subtest 'only multipath maps count as shared devices' => sub {
+ my $disk = $diskmanage_module;
+ my $uuid;
+ $disk->mock(abs_path => sub { $_[0] =~ m!^/dev/(?:mapper|disk)/! ? '/dev/dm-900' : $_[0] });
+ $disk->mock(file_read_firstline => sub { $uuid });
+ $uuid = 'mpath-3600abc';
+ ok(PVE::Diskmanage::is_multipath_device('/dev/disk/by-id/dm-name-lun'), 'a map alias');
+ $uuid = 'part1-mpath-3600abc';
+ ok(PVE::Diskmanage::is_multipath_device('/dev/mapper/lunp1'), 'a partition of a map');
+ $uuid = 'LVM-abcdef';
+ ok(!PVE::Diskmanage::is_multipath_device('/dev/mapper/vg-lv'), 'not a logical volume');
+ $uuid = undef;
+ ok(
+ PVE::Diskmanage::is_multipath_device('/dev/mapper/x'),
+ 'an unreadable identity is shared',
+ );
+ ok(!PVE::Diskmanage::is_multipath_device('/dev/sdb'), 'a plain disk is not');
+ $disk->unmock($_) for qw(abs_path file_read_firstline);
+};
+
done_testing();
diff --git a/src/test/disklist_test.pm b/src/test/disklist_test.pm
index dd4e4d73..21823daa 100644
--- a/src/test/disklist_test.pm
+++ b/src/test/disklist_test.pm
@@ -236,12 +236,35 @@ sub test_disk_list {
print Dumper($multipath) if $print;
$testcount++;
is_deeply($multipath, $expected_multipath, 'multipath list should be the same');
+
+ eval { PVE::Diskmanage::assert_disk_unused('/dev/dm-0'); };
+ $testcount++;
+ like($@, qr/not a valid local disk/,
+ 'multipath devices require an explicit opt-in');
+
+ eval { PVE::Diskmanage::assert_disk_unused('/dev/mapper/mpatha', 1); };
+ $testcount++;
+ is($@, '', 'unused multipath alias should pass');
+
+ for my $dev (qw(dm-1 dm-2 dm-3 dm-4)) {
+ eval { PVE::Diskmanage::assert_disk_unused("/dev/$dev", 1); };
+ $testcount++;
+ like($@, qr/already in use/, "used multipath device $dev should fail");
+ }
+
+ eval { PVE::Diskmanage::assert_disk_unused('/dev/dm-99', 1); };
+ $testcount++;
+ like($@, qr/not a valid local disk/, 'unknown device mapper device should fail');
}
if ($testcasedir eq 'iscsi') {
eval { PVE::Diskmanage::assert_disk_unused('/dev/sda'); };
$testcount++;
like($@, qr/attached over iSCSI/, 'local disk operations reject an iSCSI LUN');
+
+ eval { PVE::Diskmanage::assert_disk_unused('/dev/sda', 1); };
+ $testcount++;
+ is($@, '', 'the SAN opt-in accepts an unused iSCSI LUN');
}
done_testing($testcount);
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 22/76] multipath: add a library around multipathd
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (20 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 21/76] disks: lvm: allow volume groups on SAN LUNs and multipath maps Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 23/76] diskmanage: collect native NVMe multipath path state Thomas Lamprecht
` (53 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
SAN scans and status reporting need map health, while iSCSI activation
must wait for a map before using its LUN. Share bounded multipathd
queries and Rust-based reply parsing between these callers.
Key maps by LUN WWID, which is stable across nodes and reboots unlike
map names and path devices, and include path groups and health.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
debian/control | 2 +
src/PVE/Makefile | 1 +
src/PVE/Multipath.pm | 159 ++++++++++++++++++++++++++++++++
src/test/Makefile | 5 +-
src/test/run_multipath_tests.pl | 133 ++++++++++++++++++++++++++
5 files changed, 299 insertions(+), 1 deletion(-)
create mode 100644 src/PVE/Multipath.pm
create mode 100755 src/test/run_multipath_tests.pl
diff --git a/debian/control b/debian/control
index 1bf02bd6..0f1f3850 100644
--- a/debian/control
+++ b/debian/control
@@ -8,6 +8,7 @@ Build-Depends: debhelper-compat (= 13),
libpve-access-control,
libpve-cluster-perl (>= 8.0.6),
libpve-common-perl (>= 9.2.3),
+ libpve-rs-perl,
librados2-perl,
libtest-mockmodule-perl,
libxml-libxml-perl,
@@ -39,6 +40,7 @@ Depends: bzip2,
libpve-apiclient-perl (>= 3.1-1),
libpve-cluster-perl (>= 8.0.6),
libpve-common-perl (>= 9.2.3),
+ libpve-rs-perl,
librados2-perl,
libxml-libxml-perl,
lvm2,
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index 9e9f6aac..91f22282 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -4,6 +4,7 @@
install:
install -D -m 0644 Storage.pm ${DESTDIR}${PERLDIR}/PVE/Storage.pm
install -D -m 0644 Diskmanage.pm ${DESTDIR}${PERLDIR}/PVE/Diskmanage.pm
+ install -D -m 0644 Multipath.pm ${DESTDIR}${PERLDIR}/PVE/Multipath.pm
install -D -m 0644 CephConfig.pm ${DESTDIR}${PERLDIR}/PVE/CephConfig.pm
install -D -m 0644 GuestImport.pm ${DESTDIR}${PERLDIR}/PVE/GuestImport.pm
make -C Storage install
diff --git a/src/PVE/Multipath.pm b/src/PVE/Multipath.pm
new file mode 100644
index 00000000..69589ac3
--- /dev/null
+++ b/src/PVE/Multipath.pm
@@ -0,0 +1,159 @@
+package PVE::Multipath;
+
+use v5.36;
+
+use PVE::Tools qw(run_command file_read_firstline);
+
+use PVE::RS::Multipath;
+
+my $MULTIPATH = '/sbin/multipath';
+my $MULTIPATHD = '/sbin/multipathd';
+
+# Reconfigure re-evaluates every path and needs more time than queries or allow-list edits.
+my $QUERY_TIMEOUT = 10;
+my $TOOL_TIMEOUT = 30;
+my $RECONFIGURE_TIMEOUT = 120;
+
+my $supported;
+
+# Cached only while true: a long-lived daemon like pvestatd must notice a later multipath-tools
+# install without a restart, so a negative result is re-checked on every call.
+sub is_supported() {
+ return $supported if $supported;
+ $supported = (-x $MULTIPATH && -x $MULTIPATHD) ? 1 : 0;
+ return $supported;
+}
+
+sub assert_supported() {
+ die "multipath-tools is not installed\n" if !is_supported();
+ return 1;
+}
+
+sub is_running() {
+ return 0 if !is_supported();
+
+ my $running = 0;
+ eval {
+ run_command(
+ [$MULTIPATHD, 'show', 'daemon'],
+ timeout => $QUERY_TIMEOUT,
+ outfunc => sub($line) {
+ $running = 1 if $line =~ m/^pid \d+ \S+/;
+ },
+ errfunc => sub($line) { },
+ );
+ };
+ return $running;
+}
+
+my sub query_maps_json() {
+ assert_supported();
+
+ my $output = '';
+ run_command(
+ [$MULTIPATHD, 'show', 'maps', 'json'],
+ timeout => $QUERY_TIMEOUT,
+ outfunc => sub($line) { $output .= "$line\n"; },
+ errfunc => sub($line) { warn "$line\n"; },
+ );
+
+ return $output;
+}
+
+sub parse_maps_json($json) {
+ return PVE::RS::Multipath::parse_maps_json($json);
+}
+
+# Expected paths include missing paths that the daemon no longer counts.
+sub derive_health($paths_active, $paths_total, $paths_expected = undef) {
+ return PVE::RS::Multipath::derive_health($paths_active, $paths_total, $paths_expected);
+}
+
+# sysfs size uses 512-byte sectors regardless of the device's logical block size.
+my sub dm_size_bytes($sysfs) {
+ return undef if !$sysfs;
+ my $sectors = file_read_firstline("/sys/block/$sysfs/size");
+ return undef if !defined($sectors) || $sectors !~ m/^\d+$/;
+ return int($sectors) * 512;
+}
+
+my sub path_transport($dev) {
+ return undef if !$dev;
+ my $link = readlink("/sys/block/$dev");
+ return undef if !$link;
+ return 'iscsi' if $link =~ m{/session\d+/};
+ return 'fc' if $link =~ m{/rport-\d+};
+ return 'sas' if $link =~ m{/end_device-};
+ return undef;
+}
+
+sub get_maps() {
+ my $maps = parse_maps_json(query_maps_json());
+
+ for my $map ($maps->@*) {
+ $map->{size} = dm_size_bytes($map->{sysfs});
+ $map->{path} = "/dev/disk/by-id/dm-uuid-mpath-$map->{wwid}"
+ if defined($map->{wwid});
+
+ my %transports;
+ for my $group ($map->{'path-groups'}->@*) {
+ for my $path ($group->{paths}->@*) {
+ $path->{transport} //= path_transport($path->{dev});
+ $transports{ $path->{transport} } = 1 if defined($path->{transport});
+ }
+ }
+ my @transports = keys %transports;
+ $map->{transport} = $transports[0] if scalar(@transports) == 1;
+ }
+
+ return $maps;
+}
+
+sub get_map_for_wwid($wwid) {
+ for my $map (get_maps()->@*) {
+ return $map if defined($map->{wwid}) && $map->{wwid} eq $wwid;
+ }
+ return undef;
+}
+
+sub path_status_by_wwid() {
+ my $res = {};
+ for my $map (get_maps()->@*) {
+ my $wwid = $map->{wwid} // next;
+
+ my $paths = [];
+ for my $path (map { $_->{paths}->@* } $map->{'path-groups'}->@*) {
+ my $entry = { device => "/dev/$path->{dev}", state => $path->{'dm-state'} // 'undef' };
+ $entry->{'checker-state'} = $path->{'check-state'} if defined($path->{'check-state'});
+ for my $key (qw(host-wwpn target-wwpn)) {
+ $entry->{$key} = $path->{$key} if defined($path->{$key});
+ }
+ push $paths->@*, $entry;
+ }
+ $res->{$wwid} = {
+ active => $map->{'paths-active'},
+ faults => $map->{'path-faults'},
+ paths => $paths,
+ };
+ }
+ return $res;
+}
+
+# An in-flight query can outlast the polling timeout.
+sub wait_for_map($wwid, $timeout = undef) {
+ # an assembled map can be used whether multipathd answers right now or not
+ my $link = "/dev/disk/by-id/dm-uuid-mpath-$wwid";
+ return { wwid => $wwid, path => $link } if -b $link;
+
+ $timeout //= 10;
+
+ my $deadline = time() + $timeout;
+ while (1) {
+ my $map = eval { get_map_for_wwid($wwid) };
+ return $map if $map;
+ return undef if time() >= $deadline;
+ sleep(1);
+ }
+}
+
+1;
diff --git a/src/test/Makefile b/src/test/Makefile
index 26b6b12e..c287ebdd 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_san_api test_san_libraries
+test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_san_api test_san_libraries
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -47,6 +47,9 @@ test_ovf: run_ovf_tests.pl
test_volume_access: run_volume_access_tests.pl
./run_volume_access_tests.pl
+test_multipath: run_multipath_tests.pl
+ ./run_multipath_tests.pl
+
test_san_api: run_san_api_tests.pl
./run_san_api_tests.pl
diff --git a/src/test/run_multipath_tests.pl b/src/test/run_multipath_tests.pl
new file mode 100755
index 00000000..484e9348
--- /dev/null
+++ b/src/test/run_multipath_tests.pl
@@ -0,0 +1,133 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Multipath;
+
+# A recorded `multipathd show maps json` reply with three maps exercising each
+# health state: an all-active map, a partially-failed map and an all-failed map.
+my $maps_json = <<'EOF';
+{
+ "major_version": 0,
+ "minor_version": 1,
+ "maps": [
+ {
+ "name": "mpatha",
+ "uuid": "3600140500a1b2c3d4e5f6a7b8c9d0e1f",
+ "sysfs": "dm-0",
+ "dm_st": "active",
+ "paths": 2,
+ "path_groups": [
+ {
+ "group": 1,
+ "dm_st": "active",
+ "pri": 50,
+ "paths": [
+ { "dev": "sdb", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 50, "target_wwpn": "0x500a098000000001" },
+ { "dev": "sdc", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 50, "target_wwpn": "0x500a098000000002" }
+ ]
+ }
+ ]
+ },
+ {
+ "name": "mpathb",
+ "uuid": "360014050aabbccddeeff00112233445566",
+ "sysfs": "dm-1",
+ "dm_st": "active",
+ "paths": 2,
+ "path_groups": [
+ {
+ "group": 1,
+ "dm_st": "active",
+ "pri": 10,
+ "paths": [
+ { "dev": "sdd", "dm_st": "active", "dev_st": "running", "chk_st": "ready", "pri": 10 }
+ ]
+ },
+ {
+ "group": 2,
+ "dm_st": "enabled",
+ "pri": 0,
+ "paths": [
+ { "dev": "sde", "dm_st": "failed", "dev_st": "faulty", "chk_st": "faulty", "pri": 0 }
+ ]
+ }
+ ]
+ },
+ {
+ "name": "mpathc",
+ "uuid": "36001405ffffffffffffffffffffffffff",
+ "sysfs": "dm-2",
+ "dm_st": "active",
+ "paths": 1,
+ "path_groups": [
+ {
+ "group": 1,
+ "dm_st": "enabled",
+ "pri": 0,
+ "paths": [
+ { "dev": "sdf", "dm_st": "failed", "dev_st": "faulty", "chk_st": "faulty", "pri": 0, "target_wwpn": "[undef]", "host_adapter": "[undef]" }
+ ]
+ }
+ ]
+ }
+ ]
+}
+EOF
+
+my $maps = PVE::Multipath::parse_maps_json($maps_json);
+
+is(scalar($maps->@*), 3, 'parsed all three maps');
+
+my ($a, $b, $c) = $maps->@*;
+
+# fully healthy map
+is($a->{wwid}, '3600140500a1b2c3d4e5f6a7b8c9d0e1f', 'map a wwid taken from uuid');
+is($a->{name}, 'mpatha', 'map a name');
+is($a->{sysfs}, 'dm-0', 'map a sysfs name');
+is($a->{'paths-total'}, 2, 'map a counts both paths');
+is($a->{'paths-active'}, 2, 'map a has two active paths');
+is($a->{health}, 'optimal', 'map a is optimal');
+is(scalar($a->{'path-groups'}->@*), 1, 'map a has one path group');
+is(
+ $a->{'path-groups'}->[0]->{paths}->[0]->{'target-wwpn'},
+ '0x500a098000000001',
+ 'FC target wwpn is preserved',
+);
+is(
+ $a->{'path-groups'}->[0]->{paths}->[0]->{transport},
+ 'fc',
+ 'transport derived as fc from a target wwpn',
+);
+
+# one failed path out of two
+is($b->{'paths-total'}, 2, 'map b counts both paths across groups');
+is($b->{'paths-active'}, 1, 'map b has one active path');
+is($b->{health}, 'degraded', 'map b is degraded');
+
+# no active path left
+is($c->{'paths-total'}, 1, 'map c counts its single path');
+is($c->{'paths-active'}, 0, 'map c has no active path');
+is($c->{health}, 'failed', 'map c is failed');
+ok(
+ !defined($c->{'path-groups'}->[0]->{paths}->[0]->{'target-wwpn'}),
+ "multipathd '[undef]' target_wwpn is cleaned away (not stored)",
+);
+ok(
+ !defined($c->{'path-groups'}->[0]->{paths}->[0]->{transport}),
+ "'[undef]' target_wwpn does not imply fc transport",
+);
+
+# empty / no maps must parse to an empty list, not die
+my $empty = PVE::Multipath::parse_maps_json('{ "major_version": 0, "maps": [] }');
+is_deeply($empty, [], 'no maps parses to empty list');
+
+# malformed input must die with a clear error
+eval { PVE::Multipath::parse_maps_json('not json') };
+ok($@ =~ m/could not parse multipathd maps JSON/, 'invalid JSON raises a clear error');
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 23/76] diskmanage: collect native NVMe multipath path state
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (21 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 22/76] multipath: add a library around multipathd Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 24/76] api: scan: san-luns: report " Thomas Lamprecht
` (52 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
NVMe over Fabrics usually uses native multipath, whose paths are
unknown to multipathd. Collect controller state and address, plus
per-path namespace ANA state, from sysfs and group them by head
namespace. This allows uniform health reporting for both multipath
stacks. Local single-controller devices have no per-path entries and
are omitted.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: drop redundant comments]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Diskmanage.pm | 60 +++++++++++++++++++
.../nvme-subsystem/nvme-subsys0/nvme3/address | 1 +
.../nvme-subsys0/nvme3/nvme2c3n1/ana_state | 1 +
.../nvme-subsystem/nvme-subsys0/nvme3/state | 1 +
.../nvme-subsystem/nvme-subsys0/nvme4/address | 1 +
.../nvme-subsys0/nvme4/nvme2c4n1/ana_state | 1 +
.../nvme-subsystem/nvme-subsys0/nvme4/state | 1 +
.../nvme-subsystem/nvme-subsys1/nvme0/address | 1 +
.../nvme-subsystem/nvme-subsys1/nvme0/state | 1 +
.../multipath/nvme_path_status_expected.json | 19 ++++++
src/test/disklist_test.pm | 16 +++++
11 files changed, 103 insertions(+)
create mode 100644 src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/address
create mode 100644 src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/nvme2c3n1/ana_state
create mode 100644 src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/state
create mode 100644 src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/address
create mode 100644 src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/nvme2c4n1/ana_state
create mode 100644 src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/state
create mode 100644 src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys1/nvme0/address
create mode 100644 src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys1/nvme0/state
create mode 100644 src/test/disk_tests/multipath/nvme_path_status_expected.json
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index caad0fee..a7eb2b21 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -974,6 +974,66 @@ sub is_multipath_device {
return $uuid =~ m/^(?:part\d+-)?mpath-/ ? 1 : 0;
}
+# Native multipath creates per-path namespace children nvmeXcYnZ below controllers. Local
+# single-controller devices lack these children and are omitted.
+sub get_nvme_path_status {
+ my $res = {};
+
+ dir_glob_foreach(
+ '/sys/class/nvme-subsystem',
+ 'nvme-subsys\d+',
+ sub {
+ my ($subsys) = @_;
+ my $subsysdir = "/sys/class/nvme-subsystem/$subsys";
+
+ dir_glob_foreach(
+ $subsysdir,
+ 'nvme\d+',
+ sub {
+ my ($ctrl) = @_;
+ my $ctrldir = "$subsysdir/$ctrl";
+ my $state = file_read_firstline("$ctrldir/state") // return;
+ my $address = file_read_firstline("$ctrldir/address");
+
+ dir_glob_foreach(
+ $ctrldir,
+ 'nvme\d+c\d+n\d+',
+ sub {
+ my ($pathdev) = @_;
+ return if $pathdev !~ m/^nvme(\d+)c\d+n(\d+)$/;
+ my $head = "nvme$1n$2";
+
+ my $entry = { device => "/dev/$ctrl", state => $state };
+ $entry->{address} = $address
+ if defined($address) && $address ne '';
+ my $ana = file_read_firstline("$ctrldir/$pathdev/ana_state");
+ $entry->{'ana-state'} = $ana if defined($ana);
+
+ # 'change' is transient; paths in 'inaccessible' or 'persistent-loss'
+ # cannot serve IO even on a live controller
+ my $usable = $state eq 'live'
+ && (!defined($ana)
+ || $ana eq 'optimized'
+ || $ana eq 'non-optimized');
+
+ my $status = $res->{$head} //= { active => 0, paths => [] };
+ $status->{active}++ if $usable;
+ push @{ $status->{paths} }, $entry;
+ },
+ );
+ },
+ );
+ },
+ );
+
+ for my $status (values %$res) {
+ @{ $status->{paths} } =
+ sort { $a->{device} cmp $b->{device} } @{ $status->{paths} };
+ }
+
+ return $res;
+}
+
sub lookup_multipath_map {
my ($devpath) = @_;
diff --git a/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/address b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/address
new file mode 100644
index 00000000..803fab3e
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/address
@@ -0,0 +1 @@
+traddr=192.168.3.209,trsvcid=4420,src_addr=192.168.2.171
diff --git a/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/nvme2c3n1/ana_state b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/nvme2c3n1/ana_state
new file mode 100644
index 00000000..e48c58f5
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/nvme2c3n1/ana_state
@@ -0,0 +1 @@
+optimized
diff --git a/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/state b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/state
new file mode 100644
index 00000000..e23fe645
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme3/state
@@ -0,0 +1 @@
+live
diff --git a/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/address b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/address
new file mode 100644
index 00000000..36618744
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/address
@@ -0,0 +1 @@
+traddr=192.168.3.109,trsvcid=4420,src_addr=192.168.2.171
diff --git a/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/nvme2c4n1/ana_state b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/nvme2c4n1/ana_state
new file mode 100644
index 00000000..436c48ee
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/nvme2c4n1/ana_state
@@ -0,0 +1 @@
+inaccessible
diff --git a/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/state b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/state
new file mode 100644
index 00000000..e23fe645
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys0/nvme4/state
@@ -0,0 +1 @@
+live
diff --git a/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys1/nvme0/address b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys1/nvme0/address
new file mode 100644
index 00000000..f99fcb14
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys1/nvme0/address
@@ -0,0 +1 @@
+0000:02:00.0
diff --git a/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys1/nvme0/state b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys1/nvme0/state
new file mode 100644
index 00000000..e23fe645
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme-subsystem/nvme-subsys1/nvme0/state
@@ -0,0 +1 @@
+live
diff --git a/src/test/disk_tests/multipath/nvme_path_status_expected.json b/src/test/disk_tests/multipath/nvme_path_status_expected.json
new file mode 100644
index 00000000..62611b3f
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme_path_status_expected.json
@@ -0,0 +1,19 @@
+{
+ "nvme2n1": {
+ "active": 1,
+ "paths": [
+ {
+ "device": "/dev/nvme3",
+ "state": "live",
+ "address": "traddr=192.168.3.209,trsvcid=4420,src_addr=192.168.2.171",
+ "ana-state": "optimized"
+ },
+ {
+ "device": "/dev/nvme4",
+ "state": "live",
+ "address": "traddr=192.168.3.109,trsvcid=4420,src_addr=192.168.2.171",
+ "ana-state": "inaccessible"
+ }
+ ]
+ }
+}
diff --git a/src/test/disklist_test.pm b/src/test/disklist_test.pm
index 21823daa..c6fb620c 100644
--- a/src/test/disklist_test.pm
+++ b/src/test/disklist_test.pm
@@ -111,6 +111,7 @@ sub mocked_file_read_firstline {
my $originalsub = $diskmanage_module->original('file_read_firstline');
$path =~ s|^/sys/block|disk_tests/$testcasedir|;
+ $path =~ s|^/sys/class/nvme-subsystem|disk_tests/$testcasedir/nvme-subsystem|;
return &$originalsub($path);
}
@@ -120,6 +121,12 @@ sub mocked_dir_glob_foreach {
my $lines = [];
+ # nvme subsystem trees are provided as directories, glob them like production code
+ if ($dir =~ s|^/sys/class/nvme-subsystem|disk_tests/$testcasedir/nvme-subsystem|) {
+ my $originalsub = $diskmanage_module->original('dir_glob_foreach');
+ return &$originalsub($dir, $regex, $sub);
+ }
+
# read lines in from file
if ($dir =~ m{^/sys/block$}) {
@$lines = split(/\n/, read_test_file('disklist'));
@@ -267,6 +274,15 @@ sub test_disk_list {
is($@, '', 'the SAN opt-in accepts an unused iSCSI LUN');
}
+ if (-f "disk_tests/$testcasedir/nvme_path_status_expected.json") {
+ my $status = PVE::Diskmanage::get_nvme_path_status();
+ my $expected_status = decode_json(read_test_file('nvme_path_status_expected.json'));
+
+ print Dumper($status) if $print;
+ $testcount++;
+ is_deeply($status, $expected_status, 'nvme path status should be the same');
+ }
+
done_testing($testcount);
};
}
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 24/76] api: scan: san-luns: report path state
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (22 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 23/76] diskmanage: collect native NVMe multipath path state Thomas Lamprecht
@ 2026-11-05 23:30 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 25/76] diskmanage: collect local block device users Thomas Lamprecht
` (51 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:30 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
A failed path stays in the device-mapper table and sysfs slave list
until the transport removes it, so counting slaves cannot detect a
degraded map. Report active path counts, fault counters, and per-path
states from multipathd instead. Device-mapper active paths count,
including ALUA standby paths; failed and undefined paths do not.
For native NVMe multipath, report controller and ANA states from sysfs.
Queries are best-effort: failures leave the optional fields absent.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: use the shared parser of PVE::Multipath instead of the helper and
parser of the RFC commit 'diskmanage: add helper querying multipath
path state', which is dropped; a path whose state multipathd reports as
undefined no longer counts as active; name the active path count
paths-active, like the total]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Storage/Scan.pm | 72 +++++++++-
src/PVE/CLI/pvesm.pm | 3 +-
src/PVE/Storage/SANScan.pm | 32 ++++-
src/test/disk_tests/multipath/multipathd | 124 ++++++++++++++++++
.../multipath/multipathd_status_expected.json | 42 ++++++
src/test/disklist_test.pm | 18 +++
src/test/run_san_api_tests.pl | 12 +-
7 files changed, 297 insertions(+), 6 deletions(-)
create mode 100644 src/test/disk_tests/multipath/multipathd
create mode 100644 src/test/disk_tests/multipath/multipathd_status_expected.json
diff --git a/src/PVE/API2/Storage/Scan.pm b/src/PVE/API2/Storage/Scan.pm
index 58570f46..73632a1f 100644
--- a/src/PVE/API2/Storage/Scan.pm
+++ b/src/PVE/API2/Storage/Scan.pm
@@ -381,7 +381,9 @@ __PACKAGE__->register_method({
path => 'san-luns',
method => 'GET',
description => "List block devices (disks and multipath devices) together with their"
- . " usage, for example as candidates for a SAN backed LVM volume group.",
+ . " usage, for example as candidates for a SAN backed LVM volume group. For multipath"
+ . " mapped devices and NVMe namespaces handled by native NVMe multipath, the state"
+ . " of the member paths is included.",
protected => 1,
proxyto => "node",
permissions => {
@@ -424,10 +426,76 @@ __PACKAGE__->register_method({
serial => { type => 'string', optional => 1 },
wwn => { type => 'string', optional => 1 },
'paths-total' => {
- description => "The number of paths of a multipath mapped device.",
+ description => "The number of paths of a multipath mapped device or of"
+ . " an NVMe namespace attached through multiple controllers.",
type => 'integer',
optional => 1,
},
+ 'paths-active' => {
+ description => "The number of paths that are usable, that is, neither"
+ . " reported as failed by multipathd nor on a dead controller or"
+ . " inaccessible through ANA.",
+ type => 'integer',
+ optional => 1,
+ },
+ 'path-faults' => {
+ description => "The number of path failures of a multipath mapped"
+ . " device since its creation, as reported by multipathd.",
+ type => 'integer',
+ optional => 1,
+ },
+ 'path-list' => {
+ description => "The member paths of the device with their state.",
+ type => 'array',
+ optional => 1,
+ items => {
+ type => 'object',
+ properties => {
+ device => {
+ description => "The path block device, or the NVMe"
+ . " controller device the path uses.",
+ type => 'string',
+ },
+ state => {
+ description => "The device mapper state of the path (for"
+ . " example active or failed) or the state of the NVMe"
+ . " controller (for example live or connecting).",
+ type => 'string',
+ },
+ 'checker-state' => {
+ description => "The state reported by the multipathd path"
+ . " checker, for example ready, ghost or faulty.",
+ type => 'string',
+ optional => 1,
+ },
+ 'ana-state' => {
+ description => "The Asymmetric Namespace Access state of"
+ . " the namespace on this path, for example optimized"
+ . " or inaccessible.",
+ type => 'string',
+ optional => 1,
+ },
+ 'host-wwpn' => {
+ description => "The WWPN of the host (initiator) port of"
+ . " Fibre Channel attached paths.",
+ type => 'string',
+ optional => 1,
+ },
+ 'target-wwpn' => {
+ description => "The WWPN of the target port of Fibre"
+ . " Channel attached paths.",
+ type => 'string',
+ optional => 1,
+ },
+ address => {
+ description => "The address of the NVMe controller the"
+ . " path uses.",
+ type => 'string',
+ optional => 1,
+ },
+ },
+ },
+ },
usage => {
description => "How the device is currently used. 'lvm' means it is an LVM"
. " physical volume, see the 'vgname' property.",
diff --git a/src/PVE/CLI/pvesm.pm b/src/PVE/CLI/pvesm.pm
index 501b7b61..67419da2 100755
--- a/src/PVE/CLI/pvesm.pm
+++ b/src/PVE/CLI/pvesm.pm
@@ -743,7 +743,8 @@ our $cmddef = {
{ node => $nodename },
sub {
my ($data, $schema, $options) = @_;
- my $columns = [qw(devpath size transport usage vgname used-by-storage)];
+ my $columns =
+ [qw(devpath size transport paths-active usage vgname used-by-storage)];
PVE::CLIFormatter::print_api_result($data, $schema, $columns, $options);
},
$PVE::RESTHandler::standard_output_options,
diff --git a/src/PVE/Storage/SANScan.pm b/src/PVE/Storage/SANScan.pm
index 6e968f65..ee311cb9 100644
--- a/src/PVE/Storage/SANScan.pm
+++ b/src/PVE/Storage/SANScan.pm
@@ -3,6 +3,7 @@ package PVE::Storage::SANScan;
use v5.36;
use PVE::Diskmanage;
+use PVE::Multipath;
use PVE::Storage;
use PVE::Storage::ISCSIPlugin;
use PVE::Storage::LVMPlugin;
@@ -11,6 +12,15 @@ my sub collect_inventory($cfg, $node) {
my $disks = PVE::Diskmanage::get_disks(undef, 1, 0);
my $multipath = PVE::Diskmanage::get_multipath_disks();
+ # Path state is auxiliary information; a multipathd failure must not break the scan.
+ my $mp_status = {};
+ if (scalar(keys %$multipath)) {
+ $mp_status = eval { PVE::Multipath::path_status_by_wwid() } // {};
+ warn $@ if $@;
+ }
+ my $nvme_status = eval { PVE::Diskmanage::get_nvme_path_status() } // {};
+ warn $@ if $@;
+
# map PV device to VG name, PVs may sit on a partition of a listed device
my $vgs = PVE::Storage::LVMPlugin::lvm_vgs(1);
my $pv2vg = {};
@@ -63,6 +73,8 @@ my sub collect_inventory($cfg, $node) {
return {
disks => $disks,
multipath => $multipath,
+ mp_status => $mp_status,
+ nvme_status => $nvme_status,
vg_used_by => $vg_used_by,
iscsi_target_used_by => $iscsi_target_used_by,
find_vg => $find_vg,
@@ -71,8 +83,9 @@ my sub collect_inventory($cfg, $node) {
}
my sub build_lun_rows($cfg, $inventory) {
- my ($disks, $multipath, $vg_used_by, $find_vg, $iscsi_used_by) =
- $inventory->@{qw(disks multipath vg_used_by find_vg iscsi_used_by)};
+ my ($disks, $multipath, $mp_status, $nvme_status, $vg_used_by, $find_vg, $iscsi_used_by) =
+ $inventory->@{ qw(disks multipath mp_status nvme_status vg_used_by find_vg
+ iscsi_used_by) };
my $records = [];
my $add_entry = sub($disk) {
@@ -84,6 +97,21 @@ my sub build_lun_rows($cfg, $inventory) {
$entry->{$key} = $value if defined($value) && $value ne 'unknown';
}
+ # aggregated multipath entries carry their member list and their WWID as wwn
+ if (defined($disk->{slaves})) {
+ if (my $status = $mp_status->{ $disk->{wwn} // '' }) {
+ $entry->{'paths-active'} = $status->{active};
+ $entry->{'path-faults'} = $status->{faults};
+ $entry->{'path-list'} = $status->{paths};
+ }
+ } elsif ($disk->{devpath} =~ m|^/dev/(nvme\d+n\d+)$|) {
+ if (my $status = $nvme_status->{$1}) {
+ $entry->{'paths-total'} = scalar($status->{paths}->@*);
+ $entry->{'paths-active'} = $status->{active};
+ $entry->{'path-list'} = $status->{paths};
+ }
+ }
+
if (!defined($disk->{used})) {
$entry->{usage} = 'unused';
} elsif ($disk->{used} eq 'LVM') {
diff --git a/src/test/disk_tests/multipath/multipathd b/src/test/disk_tests/multipath/multipathd
new file mode 100644
index 00000000..40243f91
--- /dev/null
+++ b/src/test/disk_tests/multipath/multipathd
@@ -0,0 +1,124 @@
+{
+ "major_version": 0,
+ "minor_version": 1,
+ "maps": [{
+ "name" : "mpatha",
+ "uuid" : "3600140500000000000000000000000001",
+ "sysfs" : "dm-0",
+ "failback" : "immediate",
+ "queueing" : "5 chk",
+ "paths" : 2,
+ "write_prot" : "rw",
+ "dm_st" : "active",
+ "features" : "1 queue_if_no_path",
+ "hwhandler" : "1 alua",
+ "action" : "",
+ "path_faults" : 0,
+ "vend" : "SANVEND ",
+ "prod" : "SANMODEL",
+ "rev" : "4.1",
+ "switch_grp" : 0,
+ "map_loads" : 1,
+ "total_q_time" : 0,
+ "q_timeouts" : 0,
+ "path_groups": [{
+ "selector" : "service-time 0",
+ "pri" : 50,
+ "dm_st" : "active",
+ "marginal_st" : "normal",
+ "group" : 1,
+ "paths": [{
+ "dev" : "sda",
+ "dev_t" : "8:0",
+ "dm_st" : "active",
+ "dev_st" : "running",
+ "chk_st" : "ready",
+ "checker" : "tur",
+ "pri" : 50,
+ "host_wwnn" : "0x20000024ff7d6a1c",
+ "target_wwnn" : "0x2000d039ea28bf00",
+ "host_wwpn" : "0x21000024ff7d6a1c",
+ "target_wwpn" : "0x2100d039ea28bf00",
+ "host_adapter" : "0000:41:00.0",
+ "lun_hex" : "0x0001000000000000",
+ "marginal_st" : "normal"
+ },
+ {
+ "dev" : "sdb",
+ "dev_t" : "8:16",
+ "dm_st" : "active",
+ "dev_st" : "running",
+ "chk_st" : "ready",
+ "checker" : "tur",
+ "pri" : 50,
+ "host_wwnn" : "0x20000024ff7d6a1d",
+ "target_wwnn" : "0x2000d039ea28bf00",
+ "host_wwpn" : "0x21000024ff7d6a1d",
+ "target_wwpn" : "0x2200d039ea28bf00",
+ "host_adapter" : "0000:41:00.1",
+ "lun_hex" : "0x0001000000000000",
+ "marginal_st" : "normal"
+ }]
+ }]
+ },
+ {
+ "name" : "mpathb",
+ "uuid" : "3600140500000000000000000000000002",
+ "sysfs" : "dm-1",
+ "failback" : "immediate",
+ "queueing" : "5 chk",
+ "paths" : 2,
+ "write_prot" : "rw",
+ "dm_st" : "active",
+ "features" : "1 queue_if_no_path",
+ "hwhandler" : "1 alua",
+ "action" : "",
+ "path_faults" : 3,
+ "vend" : "SANVEND ",
+ "prod" : "SANMODEL",
+ "rev" : "4.1",
+ "switch_grp" : 0,
+ "map_loads" : 1,
+ "total_q_time" : 0,
+ "q_timeouts" : 0,
+ "path_groups": [{
+ "selector" : "service-time 0",
+ "pri" : 50,
+ "dm_st" : "active",
+ "marginal_st" : "normal",
+ "group" : 1,
+ "paths": [{
+ "dev" : "sdc",
+ "dev_t" : "8:32",
+ "dm_st" : "active",
+ "dev_st" : "running",
+ "chk_st" : "ready",
+ "checker" : "tur",
+ "pri" : 50,
+ "host_wwnn" : "0x20000024ff7d6a1c",
+ "target_wwnn" : "0x2000d039ea28bf00",
+ "host_wwpn" : "0x21000024ff7d6a1c",
+ "target_wwpn" : "0x2100d039ea28bf00",
+ "host_adapter" : "0000:41:00.0",
+ "lun_hex" : "0x0002000000000000",
+ "marginal_st" : "normal"
+ },
+ {
+ "dev" : "sdd",
+ "dev_t" : "8:48",
+ "dm_st" : "failed",
+ "dev_st" : "running",
+ "chk_st" : "faulty",
+ "checker" : "tur",
+ "pri" : 0,
+ "host_wwnn" : "0x20000024ff7d6a1d",
+ "target_wwnn" : "0x2000d039ea28bf00",
+ "host_wwpn" : "0x21000024ff7d6a1d",
+ "target_wwpn" : "0x2200d039ea28bf00",
+ "host_adapter" : "0000:41:00.1",
+ "lun_hex" : "0x0002000000000000",
+ "marginal_st" : "normal"
+ }]
+ }]
+ }]
+}
diff --git a/src/test/disk_tests/multipath/multipathd_status_expected.json b/src/test/disk_tests/multipath/multipathd_status_expected.json
new file mode 100644
index 00000000..d57e620d
--- /dev/null
+++ b/src/test/disk_tests/multipath/multipathd_status_expected.json
@@ -0,0 +1,42 @@
+{
+ "3600140500000000000000000000000001": {
+ "active": 2,
+ "faults": 0,
+ "paths": [
+ {
+ "device": "/dev/sda",
+ "state": "active",
+ "checker-state": "ready",
+ "host-wwpn": "0x21000024ff7d6a1c",
+ "target-wwpn": "0x2100d039ea28bf00"
+ },
+ {
+ "device": "/dev/sdb",
+ "state": "active",
+ "checker-state": "ready",
+ "host-wwpn": "0x21000024ff7d6a1d",
+ "target-wwpn": "0x2200d039ea28bf00"
+ }
+ ]
+ },
+ "3600140500000000000000000000000002": {
+ "active": 1,
+ "faults": 3,
+ "paths": [
+ {
+ "device": "/dev/sdc",
+ "state": "active",
+ "checker-state": "ready",
+ "host-wwpn": "0x21000024ff7d6a1c",
+ "target-wwpn": "0x2100d039ea28bf00"
+ },
+ {
+ "device": "/dev/sdd",
+ "state": "failed",
+ "checker-state": "faulty",
+ "host-wwpn": "0x21000024ff7d6a1d",
+ "target-wwpn": "0x2200d039ea28bf00"
+ }
+ ]
+ }
+}
diff --git a/src/test/disklist_test.pm b/src/test/disklist_test.pm
index c6fb620c..ae072fb2 100644
--- a/src/test/disklist_test.pm
+++ b/src/test/disklist_test.pm
@@ -6,6 +6,7 @@ use warnings;
use lib qw(..);
use PVE::Diskmanage;
+use PVE::Multipath;
use PVE::Tools;
use Test::MockModule;
@@ -16,6 +17,7 @@ use Data::Dumper;
my $testcasedir; # current case directory
my $testcount = 0; # testcount for TAP::Harness
my $diskmanage_module; # mockmodule for PVE::Diskmanage
+my $multipath_module; # mockmodule for PVE::Multipath
my $print = 0;
my $multipath_aliases = {
@@ -68,6 +70,8 @@ sub mocked_run_command {
$content = '{}';
}
@$outputlines = split(/\n/, $content);
+ } elsif ($cmd->[0] =~ m/multipathd/i) {
+ @$outputlines = split(/\n/, read_test_file('multipathd'));
} else {
die "unexpected run_command call: '@$cmd', aborting\n";
}
@@ -274,6 +278,16 @@ sub test_disk_list {
is($@, '', 'the SAN opt-in accepts an unused iSCSI LUN');
}
+ if (-f "disk_tests/$testcasedir/multipathd_status_expected.json") {
+ my $status = PVE::Multipath::path_status_by_wwid();
+ my $expected_status =
+ decode_json(read_test_file('multipathd_status_expected.json'));
+
+ print Dumper($status) if $print;
+ $testcount++;
+ is_deeply($status, $expected_status, 'multipath status should be the same');
+ }
+
if (-f "disk_tests/$testcasedir/nvme_path_status_expected.json") {
my $status = PVE::Diskmanage::get_nvme_path_status();
my $expected_status = decode_json(read_test_file('nvme_path_status_expected.json'));
@@ -305,6 +319,10 @@ $diskmanage_module->mock('get_sysdir_size' => \&mocked_get_sysdir_size);
print("\tMocked get_sysdir_size\n");
$diskmanage_module->mock('assert_blockdev' => sub { return 1; });
print("\tMocked assert_blockdev\n");
+$multipath_module = Test::MockModule->new('PVE::Multipath', no_auto => 1);
+$multipath_module->mock('run_command' => \&mocked_run_command);
+$multipath_module->mock('is_supported' => sub { return 1; });
+print("\tMocked PVE::Multipath run_command and is_supported\n");
$diskmanage_module->mock(
'dir_is_empty' => sub {
my $val = shift;
diff --git a/src/test/run_san_api_tests.pl b/src/test/run_san_api_tests.pl
index de35217f..ac5a8be3 100755
--- a/src/test/run_san_api_tests.pl
+++ b/src/test/run_san_api_tests.pl
@@ -39,7 +39,7 @@ subtest 'SAN CLI columns leave machine output complete' => sub {
is($_[1], $schema, 'complete schema forwarded');
is_deeply(
$_[2],
- [qw(devpath size transport usage vgname used-by-storage)],
+ [qw(devpath size transport paths-active usage vgname used-by-storage)],
'concise text columns',
);
is($_[3]->{'output-format'}, 'json', 'format option unchanged');
@@ -72,12 +72,22 @@ subtest 'SAN LUN path counts' => sub {
);
$lvm->mock(lvm_vgs => sub { {} });
$storage->mock(config => sub { { ids => {} } });
+ # native NVMe multipath: two controllers, one of them usable
+ $disks->{'/dev/nvme0n1'} = { devpath => '/dev/nvme0n1', size => 1024 };
+ my $paths = [{ device => '/dev/nvme0' }, { device => '/dev/nvme1' }];
+ $disk->mock(
+ get_nvme_path_status => sub { { nvme0n1 => { active => 1, paths => $paths } } });
+ my $multipath = Test::MockModule->new('PVE::Multipath');
+ $multipath->mock(path_status_by_wwid => sub { {} });
my $declared =
PVE::API2::Storage::Scan->map_method_by_name('sanlunscan')->{returns}->{items}
->{properties};
my $rows = PVE::API2::Storage::Scan->sanlunscan({ node => 'test' });
my $by_dev = { map { ($_->{devpath} => $_) } @$rows };
is($by_dev->{'/dev/mapper/m0'}->{'paths-total'}, 2, 'map path count');
+ # native NVMe multipath
+ is($by_dev->{'/dev/nvme0n1'}->{'paths-total'}, 2, 'namespace path count');
+ is($by_dev->{'/dev/nvme0n1'}->{'paths-active'}, 1, 'namespace usable paths');
for my $row (@$rows) {
my @undeclared = grep { !$declared->{$_} } sort keys %$row;
is_deeply(\@undeclared, [], "$row->{devpath}: every field declared");
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 25/76] diskmanage: collect local block device users
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (23 preceding siblings ...)
2026-11-05 23:30 ` [PATCH v3 storage 24/76] api: scan: san-luns: report " Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 26/76] api: add node-level iSCSI status and maintenance Thomas Lamprecht
` (50 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Destructive operations must reject devices with local users, but each
caller checked only some forms of use. Share a check covering holders
recursively, partitions, mounts, swap, loop and btrfs devices, LIO and
nvmet exports, and processes with the device open. Failed probes are
errors, not evidence that a device is unused.
Run this check before each SAN volume-group initialization write through
the disk API and storage-add base-volume path. A retry for the same
volume group remains write-free. LVM-thin keeps its existing policy.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Diskmanage.pm | 147 +++++++++++++++
src/PVE/Storage/LVMPlugin.pm | 17 +-
src/test/Makefile | 6 +-
src/test/disk_api_test.pm | 26 ---
src/test/run_block_device_users_tests.pl | 134 +++++++++++++
src/test/run_san_lvm_create_tests.pl | 230 +++++++++++++++++++++++
6 files changed, 531 insertions(+), 29 deletions(-)
create mode 100755 src/test/run_block_device_users_tests.pl
create mode 100755 src/test/run_san_lvm_create_tests.pl
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index a7eb2b21..655cee37 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -6,6 +6,7 @@ use warnings;
use PVE::ProcFSTools;
use Data::Dumper;
use Cwd qw(abs_path);
+use Errno qw(ENOENT ESRCH);
use Fcntl ':mode';
use File::Basename;
use File::stat;
@@ -59,6 +60,152 @@ sub assert_blockdev {
return 1;
}
+# Device numbers also identify aliases and open descriptors after a device node was renamed.
+sub block_device_number {
+ my ($path) = @_;
+
+ my @st = CORE::stat($path);
+ die "unable to stat '$path' - $!\n" if !@st;
+ return undef if !S_ISBLK($st[2]);
+ return PVE::Tools::dev_t_major($st[6]) . ':' . PVE::Tools::dev_t_minor($st[6]);
+}
+
+sub block_device_users {
+ my ($devices, $options) = @_;
+ $options //= {};
+ my $sys = $options->{sysfs} // '/sys';
+ my $proc = $options->{procfs} // '/proc';
+
+ my $entries = sub {
+ my ($dir, $optional) = @_;
+ if (!opendir(my $dh, $dir)) {
+ return [] if $optional && ($! == ENOENT || $! == ESRCH);
+ die "unable to list '$dir' - $!\n";
+ } else {
+ my @names = sort grep { $_ ne '.' && $_ ne '..' } readdir($dh);
+ closedir($dh) or die "unable to close '$dir' - $!\n";
+ return \@names;
+ }
+ };
+ my $read = sub {
+ my ($file) = @_;
+ my $value = file_read_firstline($file);
+ die "unable to read '$file' - missing value\n" if !defined($value);
+ return $value;
+ };
+ my ($numbers, $seen, $users) = ({}, {}, []);
+ my $visit;
+ $visit = sub {
+ my ($name) = @_;
+ die "invalid block device name '$name'\n" if $name !~ m/^[a-zA-Z0-9_.!-]+$/;
+ return if $seen->{$name}++;
+ my $dir = "$sys/class/block/$name";
+ my $dev = $read->("$dir/dev");
+ die "invalid device number for '$name'\n" if $dev !~ m/^\d+:\d+$/;
+ $numbers->{$dev} = $name;
+ for my $part ($entries->($dir)->@*) {
+ next if !-e "$dir/$part/partition";
+ my $number = $read->("$dir/$part/partition");
+ die "invalid partition number for '$part'\n" if $number !~ m/^[1-9]\d*$/;
+ $visit->($part);
+ }
+ for my $holder ($entries->("$dir/holders")->@*) {
+ push $users->@*, "$name held by $holder";
+ $visit->($holder);
+ }
+ };
+ $visit->($_) for sort $devices->@*;
+ undef $visit;
+ return $users if !%$numbers;
+
+ my $mountinfo = file_get_contents("$proc/self/mountinfo");
+ die "unable to check mounts - empty mountinfo\n" if !$mountinfo;
+ for my $line (split(/\n/, $mountinfo)) {
+ my (undef, undef, $dev, undef, $target) = split(/\s+/, $line);
+ die "unable to check mounts - malformed mountinfo\n"
+ if !defined($target) || $dev !~ m/^\d+:\d+$/ || $line !~ m/ - /;
+ push $users->@*, "$numbers->{$dev} mounted at $target" if defined($numbers->{$dev});
+ }
+
+ my $by_path = sub {
+ my ($path, $what) = @_;
+ return if $path eq '';
+ $path =~ s/\\([0-7]{3})/chr(oct($1))/ge;
+ my $dev = block_device_number($path);
+ push $users->@*, "$numbers->{$dev} $what"
+ if defined($dev) && defined($numbers->{$dev});
+ };
+ my $swaps = file_get_contents("$proc/swaps");
+ die "unable to check swap - malformed swaps\n" if $swaps !~ s/^Filename\s+[^\n]*\n//;
+ for my $line (split(/\n/, $swaps)) {
+ my ($path) = $line =~ m/^(\/\S+)\s+/;
+ die "unable to check swap - malformed entry\n" if !defined($path);
+ $by_path->($path, 'used as swap');
+ }
+
+ for my $loop (grep { /^loop\d+$/ } $entries->("$sys/class/block")->@*) {
+ next if !-d "$sys/class/block/$loop/loop";
+ $by_path->($read->("$sys/class/block/$loop/loop/backing_file"), "backs $loop");
+ }
+ for my $fsid ($entries->("$sys/fs/btrfs", 1)->@*) {
+ next if $fsid !~ m/^[0-9a-f-]{36}$/;
+ my $dir = "$sys/fs/btrfs/$fsid/devices";
+ for my $device ($entries->($dir)->@*) {
+ my $dev = $read->("$dir/$device/dev");
+ die "invalid btrfs device number\n" if $dev !~ m/^\d+:\d+$/;
+ push $users->@*, "$numbers->{$dev} is a device of btrfs file system $fsid"
+ if defined($numbers->{$dev});
+ }
+ }
+ for my $target (
+ ['nvmet', "$sys/kernel/config/nvmet/subsystems"],
+ ['target_core_mod', "$sys/kernel/config/target/core"],
+ ) {
+ my ($module, $dir) = $target->@*;
+ my $optional = !-d "$sys/module/$module";
+ for my $name ($entries->($dir, $optional)->@*) {
+ if ($module eq 'nvmet') {
+ for my $ns ($entries->("$dir/$name/namespaces")->@*) {
+ $by_path->(
+ $read->("$dir/$name/namespaces/$ns/device_path"),
+ "exported by NVMe target subsystem '$name'",
+ );
+ }
+ } elsif ($name =~ m/^(?:iblock|pscsi)_\d+$/) {
+ for my $store ($entries->("$dir/$name")->@*) {
+ next if !-d "$dir/$name/$store";
+ $by_path->(
+ $read->("$dir/$name/$store/udev_path"),
+ "exported by LIO backstore '$store'",
+ );
+ }
+ }
+ }
+ }
+
+ for my $pid (grep { /^\d+$/ && $_ != $$ } $entries->($proc)->@*) {
+ for my $fd ($entries->("$proc/$pid/fd", 1)->@*) {
+ my $file = "$proc/$pid/fd/$fd";
+ my $target = readlink($file);
+ if (!defined($target)) {
+ next if $! == ENOENT || $! == ESRCH;
+ die "unable to read '$file' - $!\n";
+ }
+ # A process can open a device node outside /dev, including in another mount namespace.
+ next if $target !~ m|^/|;
+ my $dev = eval { block_device_number($file) };
+ if (my $err = $@) {
+ next if $! == ENOENT || $! == ESRCH;
+ die $err;
+ }
+ next if !defined($dev) || !defined($numbers->{$dev});
+ my $comm = file_read_firstline("$proc/$pid/comm") // '?';
+ push $users->@*, "$comm (PID $pid) has $numbers->{$dev} open";
+ }
+ }
+ return $users;
+}
+
sub init_disk {
my ($disk, $uuid) = @_;
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index 916f1b8f..723ed555 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -166,6 +166,13 @@ sub lvm_create_volume_group {
my $check_instance = sub { $guard->{verify}->() if $guard && $guard->{verify} };
my $check_safe = sub {
$check_instance->();
+ if ($san) {
+ my $users = PVE::Diskmanage::block_device_users(
+ [basename($device)], { 'partition-maps' => 1 },
+ );
+ die "cannot use '$device' - " . join(', ', $users->@*) . "\n" if $users->@*;
+ $check_instance->();
+ }
};
my $refuse_data = sub {
my (@types) = @_;
@@ -1089,7 +1096,15 @@ sub on_add_hook {
# The storage API runs this hook under the cluster-wide storage configuration lock, which
# the disk API also takes for shared devices, so two nodes cannot initialize the same LUN.
- lvm_create_volume_group($path, $scfg->{vgname}, $scfg->{shared});
+ lvm_create_volume_group(
+ $path,
+ $scfg->{vgname},
+ $scfg->{shared},
+ {
+ reference => $path,
+ san => 1,
+ },
+ );
}
assert_discard_supported($scfg->{vgname}, $scfg->{'on-volume-remove'}, $scfg->{nodes});
diff --git a/src/test/Makefile b/src/test/Makefile
index c287ebdd..9962bdfd 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -9,9 +9,10 @@ test_lvmplugin: run_test_lvmplugin.pl run_test_lvm_throughput.pl
./run_test_lvmplugin.pl
./run_test_lvm_throughput.pl
-test_lvm_create: run_test_lvm_create.pl run_test_lvm_base_conflicts.pl
+test_lvm_create: run_test_lvm_create.pl run_test_lvm_base_conflicts.pl run_san_lvm_create_tests.pl
./run_test_lvm_create.pl
./run_test_lvm_base_conflicts.pl
+ ./run_san_lvm_create_tests.pl
test_lvm_lock_retry: run_test_lvm_lock_retry.pl
./run_test_lvm_lock_retry.pl
@@ -29,8 +30,9 @@ test_status_command: run_status_command_tests.pl run_status_command_contention_t
./run_status_command_tests.pl
./run_status_command_contention_tests.pl
-test_disklist: run_disk_tests.pl
+test_disklist: run_disk_tests.pl run_block_device_users_tests.pl
./run_disk_tests.pl
+ ./run_block_device_users_tests.pl
test_disk_api: run_disk_api_tests.pl
./run_disk_api_tests.pl
diff --git a/src/test/disk_api_test.pm b/src/test/disk_api_test.pm
index 893b4e04..d1fc6465 100644
--- a/src/test/disk_api_test.pm
+++ b/src/test/disk_api_test.pm
@@ -162,32 +162,6 @@ subtest 'a volume group on a map is created under the storage configuration lock
],
'a map or SAN LUN is initialized under the cluster-wide lock, a local disk is not',
);
- @created = ();
- $disk->mock(assert_disk_unused => sub { die "device is already in use\n" });
- eval {
- PVE::API2::Disks::LVM->create({
- node => 'n1',
- name => 'vg',
- device => '/dev/sdc',
- });
- };
- like($@, qr/already in use/, 'disk API rejects a used inventory entry');
- is_deeply(\@created, [], 'used inventory prevents initialization');
- $disk->mock(assert_disk_unused => sub { });
- $lvm->mock(lvm_vgs => sub { { vg => {} } });
- eval {
- PVE::API2::Disks::LVM->create({
- node => 'n1',
- name => 'vg',
- device => '/dev/sdc',
- });
- };
- like(
- $@,
- qr/volume group with name 'vg' already exists/,
- 'stale inventory does not permit an existing VG',
- );
- is_deeply(\@created, [], 'existing VG prevents initialization');
$disk->unmock($_)
for qw(
locked_disk_action verify_blockdev_path assert_disk_unused is_partition
diff --git a/src/test/run_block_device_users_tests.pl b/src/test/run_block_device_users_tests.pl
new file mode 100755
index 00000000..5a27b503
--- /dev/null
+++ b/src/test/run_block_device_users_tests.pl
@@ -0,0 +1,134 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use File::Path qw(make_path remove_tree);
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Diskmanage;
+
+my $root = tempdir(CLEANUP => 1);
+my $options = { sysfs => "$root/sys", procfs => "$root/proc" };
+my $disk = Test::MockModule->new('PVE::Diskmanage');
+my %numbers = ('/dev/alias' => '8:1');
+$disk->mock(
+ block_device_number => sub {
+ my ($path) = @_;
+ die "stat failed\n" if !exists($numbers{$path});
+ return $numbers{$path};
+ },
+);
+
+my sub write_file {
+ my ($path, $value) = @_;
+ my ($dir) = "$root$path" =~ m{^(.*)/[^/]+$};
+ make_path($dir);
+ open(my $fh, '>', "$root$path") or die $!;
+ print $fh $value;
+ close($fh) or die $!;
+}
+
+my sub device {
+ my ($name, $number, $parent) = @_;
+ write_file("/sys/class/block/$name/dev", "$number\n");
+ make_path("$root/sys/class/block/$name/holders");
+ if ($parent) {
+ write_file("/sys/class/block/$name/partition", "1\n");
+ symlink("../$name", "$root/sys/class/block/$parent/$name") or die $!;
+ }
+}
+
+my sub users {
+ return PVE::Diskmanage::block_device_users(['sda'], $options);
+}
+
+write_file('/proc/self/mountinfo', "1 0 0:1 / / rw - rootfs rootfs rw\n");
+write_file('/proc/swaps', "Filename\tType\tSize\tUsed\tPriority\n");
+device('sda', '8:0');
+device('sda1', '8:1', 'sda');
+is_deeply(users(), [], 'unused disk and partition');
+write_file('/proc/self/mountinfo', "1 0 8:1 / /mnt rw - ext4 /dev/alias rw\n");
+is_deeply(users(), ['sda1 mounted at /mnt'], 'mounts matched by number');
+write_file('/proc/self/mountinfo', "1 0 0:1 / / rw - rootfs rootfs rw\n");
+write_file('/proc/swaps', "Filename\tType\tSize\tUsed\tPriority\n/dev/alias partition 1 0 -2\n");
+is_deeply(users(), ['sda1 used as swap'], 'swap alias');
+write_file('/proc/swaps', "Filename\tType\tSize\tUsed\tPriority\n");
+
+device('dm-0', '253:0');
+device('dm-1', '253:1', 'dm-0');
+symlink('../../dm-0', "$root/sys/class/block/sda/holders/dm-0") or die $!;
+is_deeply(users(), ['sda held by dm-0'], 'recursive holder');
+write_file('/proc/self/mountinfo', "1 0 253:1 / /mapped rw - ext4 /dev/mapper/part rw\n");
+is_deeply(users(), ['sda held by dm-0', 'dm-1 mounted at /mapped'], 'map partition by attribute');
+write_file('/proc/self/mountinfo', "1 0 0:1 / / rw - rootfs rootfs rw\n");
+
+make_path("$root/proc/999999/fd");
+symlink('/dev/alias', "$root/proc/999999/fd/3") or die $!;
+$numbers{"$root/proc/999999/fd/3"} = '8:1';
+write_file('/proc/999999/comm', "qemu\n");
+is_deeply(
+ PVE::Diskmanage::block_device_users(
+ ['sda'], $options,
+ ),
+ ['sda held by dm-0', 'qemu (PID 999999) has sda1 open'],
+ 'holders and direct users of a member partition are both listed',
+);
+unlink("$root/proc/999999/fd/3") or die $!;
+symlink('/srv/raw-disk', "$root/proc/999999/fd/3") or die $!;
+is_deeply(
+ PVE::Diskmanage::block_device_users(
+ ['sda'], $options,
+ ),
+ ['sda held by dm-0', 'qemu (PID 999999) has sda1 open'],
+ 'device node outside /dev is not silently skipped',
+);
+unlink("$root/proc/999999/fd/3") or die $!;
+
+write_file('/sys/class/block/loop0/loop/backing_file', "/dev/alias\n");
+write_file('/sys/kernel/config/nvmet/subsystems/test/namespaces/1/device_path', "/dev/alias\n");
+write_file('/sys/kernel/config/target/core/iblock_0/test/udev_path', "/dev/alias\n");
+my $fsid = '00000000-0000-0000-0000-000000000001';
+write_file("/sys/fs/btrfs/$fsid/devices/sda1/dev", "8:1\n");
+is_deeply(
+ users(),
+ [
+ 'sda held by dm-0',
+ 'sda1 backs loop0',
+ "sda1 is a device of btrfs file system $fsid",
+ "sda1 exported by NVMe target subsystem 'test'",
+ "sda1 exported by LIO backstore 'test'",
+ ],
+ 'kernel users without process descriptors',
+);
+
+unlink("$root/sys/class/block/sda1/dev") or die $!;
+eval { users() };
+like($@, qr/missing value/, 'missing partition identity fails closed');
+write_file('/sys/class/block/sda1/dev', "8:1\n");
+remove_tree("$root/sys/class/block/sda1/holders");
+eval { users() };
+like($@, qr/unable to list/, 'missing holders directory fails closed');
+make_path("$root/sys/class/block/sda1/holders");
+write_file('/proc/self/mountinfo', 'garbage');
+eval { users() };
+like($@, qr/malformed mountinfo/, 'bad mountinfo fails closed');
+write_file('/proc/self/mountinfo', "1 0 0:1 / / rw - rootfs rootfs rw\n");
+write_file('/proc/swaps', 'garbage');
+eval { users() };
+like($@, qr/malformed swaps/, 'bad swaps fails closed');
+write_file('/proc/swaps', "Filename\tType\tSize\tUsed\tPriority\n");
+$numbers{'/dev/alias'} = undef;
+delete $numbers{'/dev/alias'};
+eval { users() };
+like($@, qr/stat failed/, 'unreadable backing device fails closed');
+$numbers{'/dev/alias'} = '8:1';
+remove_tree("$root/sys/kernel/config/nvmet");
+make_path("$root/sys/module/nvmet");
+eval { users() };
+like($@, qr/unable to list.*nvmet/, 'loaded target without readable configfs fails closed');
+
+done_testing();
diff --git a/src/test/run_san_lvm_create_tests.pl b/src/test/run_san_lvm_create_tests.pl
new file mode 100755
index 00000000..200fd7cb
--- /dev/null
+++ b/src/test/run_san_lvm_create_tests.pl
@@ -0,0 +1,230 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use File::Basename qw(basename);
+use File::Path qw(make_path);
+use File::Temp qw(tempdir);
+use JSON qw(encode_json);
+use Storable qw(dclone);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::API2::Disks::LVM;
+use PVE::API2::Storage::Config;
+
+my $disk = Test::MockModule->new('PVE::Diskmanage');
+my $storage = Test::MockModule->new('PVE::Storage');
+my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
+my $rpc = Test::MockModule->new('PVE::RPCEnvironment');
+my $inotify = Test::MockModule->new('PVE::INotify');
+my $tools = Test::MockModule->new('PVE::Tools');
+my $glob = \&PVE::Tools::dir_glob_foreach;
+my $users = \&PVE::Diskmanage::block_device_users;
+my ($root, $resolved, $worker, $cfg, $local, $pv, $during_probe, $inventory_used);
+my (@locks, @writes);
+my $uuid = 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg';
+my $canonical = '/dev/mapper/lun';
+my sub put {
+ my ($path, $value) = @_;
+ my ($dir) = "$root/$path" =~ m{^(.*)/[^/]+$};
+ make_path($dir);
+ open(my $fh, '>', "$root/$path") or die $!;
+ print $fh "$value\n";
+ close($fh) or die $!;
+}
+my sub locked {
+ my ($name, $code) = @_;
+ push @locks, $name;
+ my $res = eval { $code->() };
+ my $err = $@;
+ pop @locks;
+ die $err if $err;
+ return $res;
+}
+my sub written {
+ my ($tool, $path) = @_;
+ push @writes, [$tool, $path];
+}
+{
+
+ package SanLVMTestRPC;
+ sub get_user { 'root@pam' }
+ sub check { 1 }
+ sub fork_worker { $worker = $_[4]; 'test-upid' }
+}
+$rpc->mock(get => sub { bless {}, 'SanLVMTestRPC' });
+$inotify->mock(nodename => sub { 'n1' });
+$disk->mock(
+ verify_blockdev_path => sub { $_[0] =~ m{^/dev/mapper/} ? $resolved : $_[0] },
+ is_san_device => sub { !$local },
+ is_multipath_device => sub { !$local && $_[0] =~ /dm-/ },
+ is_partition => sub { 0 },
+ block_device_users => sub {
+ $users->($_[0], { %{ $_[1] // {} }, sysfs => "$root/sys", procfs => "$root/proc" });
+ },
+ block_device_number => sub {
+ my $path = readlink($_[0]) // $_[0];
+ return PVE::Tools::file_read_firstline(
+ "$root/sys/class/block/" . basename($path) . '/dev');
+ },
+ is_mounted => sub { undef },
+ has_holder => sub { undef },
+ assert_disk_unused => sub { die "device '$_[0]' is already in use\n" if $inventory_used },
+ locked_disk_action => sub { locked('disk', $_[0]) },
+ udevadm_trigger => sub { },
+);
+$storage->mock(
+ lock_storage_config => sub {
+ die "no quorum\n" if $local;
+ ok(
+ !@locks || join('>', @locks) eq 'disk',
+ 'storage lock never precedes a nested disk lock',
+ );
+ locked('storage', $_[0]);
+ },
+ config =>
+ sub { is($locks[-1], 'storage', 'configuration read under storage lock'); dclone($cfg) },
+ write_config => sub { },
+ path => sub { $canonical },
+ activate_storage => sub { },
+ activate_volumes => sub { },
+);
+$tools->mock(
+ dir_glob_foreach => sub {
+ my $path = $_[0];
+ $path =~ s{^/sys}{$root/sys};
+ $glob->($path, $_[1], $_[2]);
+ },
+);
+$lvm->mock(
+ file_read_firstline => sub {
+ my $path = $_[0];
+ $path =~ s{^/sys}{$root/sys};
+ PVE::Tools::file_read_firstline($path);
+ },
+ abs_path => sub { $_[0] },
+ clear_first_sector => sub { written('clear', $_[0]) },
+ lvm_vgs => sub { $pv eq 'same' ? { vg => {} } : {} },
+ assert_discard_supported => sub { },
+ run_command => sub {
+ my ($cmd, %opt) = @_;
+ if ($cmd->[0] eq 'wipefs') {
+ $opt{outfunc}
+ ->(encode_json({ signatures => $pv eq 'blank' ? [] : [{ type => 'LVM2_member' }] })
+ );
+ } elsif ($cmd->[0] eq '/usr/bin/file') {
+ $opt{outfunc}->("$cmd->[-1]: " . ($pv eq 'blank' ? 'data' : 'LVM2 PV'));
+ $during_probe->() if $during_probe;
+ } elsif ($cmd->[0] eq '/sbin/pvs') {
+ $opt{outfunc}->("$resolved:8192:"
+ . ($pv eq 'same' ? 'vg' : '')
+ . ":$uuid:"
+ . ($pv eq 'same' ? '1' : '0'));
+ } else {
+ written($cmd->[0], $cmd->[-1]);
+ $pv = 'orphan' if $cmd->[0] eq '/sbin/pvcreate';
+ }
+ },
+);
+my sub reset_case {
+ $root = tempdir(CLEANUP => 1);
+ for my $name ('dm-900', 'vdz') {
+ put("sys/class/block/$name/dev", $name eq 'vdz' ? '252:0' : '253:900');
+ put("sys/class/block/$name/size", '8192');
+ make_path("$root/sys/class/block/$name/holders");
+ }
+ put('sys/class/block/dm-900/dm/uuid', 'mpath-test');
+ put('sys/class/block/dm-900/slaves/sda/device/state', 'running');
+ put('proc/self/mountinfo', '1 0 0:1 / / rw - rootfs rootfs rw');
+ put('proc/swaps', "Filename\tType\tSize\tUsed\tPriority");
+ ($resolved, $local, $pv) = ('/dev/dm-900', 0, 'blank');
+ ($during_probe, $worker) = (undef, undef);
+ $cfg =
+ { ids => { base => { type => 'iscsi', target => 'iqn.test', content => { none => 1 } } } };
+ @writes = ();
+ @locks = ();
+ $inventory_used = 0;
+}
+my sub queue {
+ PVE::API2::Disks::LVM->create({ node => 'n1', name => 'vg', device => $_[0] // $canonical });
+}
+my sub run_route {
+ my ($route) = @_;
+ eval {
+ if ($route eq 'disk') {
+ queue();
+ $worker->();
+ } else {
+ PVE::API2::Storage::Config->create({
+ type => 'lvm',
+ storage => 'new',
+ vgname => 'vg',
+ base => 'base:0.0.1.lun',
+ });
+ }
+ };
+ return $@;
+}
+for my $route ('disk', 'base') {
+ reset_case();
+ make_path("$root/proc/999999/fd");
+ symlink('/dev/dm-900', "$root/proc/999999/fd/3") or die $!;
+ put('proc/999999/comm', 'qemu');
+ like(
+ run_route($route),
+ qr/cannot use.*qemu .* has dm-900 open/,
+ "$route raw map opener refused",
+ );
+ is_deeply(\@writes, [], 'raw user receives no destructive command');
+ reset_case();
+ unlink("$root/proc/swaps") or die $!;
+ like(run_route($route), qr{can't open '.*/proc/swaps'}, "$route inconclusive users refuse");
+ is_deeply(\@writes, [], 'inconclusive users receive no destructive command');
+
+ for my $kind ('blank', 'orphan', 'same') {
+ reset_case();
+ $pv = $kind;
+ if ($kind eq 'same' && $route eq 'disk') {
+ $inventory_used = 1;
+ like(
+ run_route($route),
+ qr/device .* is already in use/,
+ 'disk API rejects the used inventory entry for an existing VG',
+ );
+ is_deeply(\@writes, [], 'used disk inventory prevents every write');
+ $inventory_used = 0;
+ like(
+ run_route($route),
+ qr/volume group with name 'vg' already exists/,
+ 'disk API rejects an existing VG even with a stale unused hint',
+ );
+ is_deeply(\@writes, [], 'existing VG prevents every disk API write');
+ } else {
+ is(run_route($route), '', "$route $kind succeeds");
+ is(
+ scalar(@writes),
+ $kind eq 'same' ? 0 : 3,
+ 'only the base-route same-VG retry succeeds without writes',
+ );
+ }
+ }
+}
+reset_case();
+$local = 1;
+$resolved = '/dev/vdz';
+queue('/dev/vdz');
+eval { $worker->() };
+is($@, '', 'local initialization works without quorum');
+is(scalar(@writes), 3, 'local behavior keeps its writes');
+reset_case();
+$local = 1;
+queue('/dev/vdz');
+$local = 0;
+eval { $worker->() };
+like($@, qr/cannot use.*device topology changed/, 'queued local-to-SAN transition refuses');
+is_deeply(\@writes, [], 'transition does not bypass storage lock');
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 26/76] api: add node-level iSCSI status and maintenance
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (24 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 25/76] diskmanage: collect local block device users Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 27/76] api: iscsi: add session logout with last-path protection Thomas Lamprecht
` (49 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
iSCSI storage activation logs in automatically, but inspecting sessions
and removing leftover node records requires iscsiadm on the shell.
Expose status and maintenance operations at /nodes/{node}/iscsi:
initiator identity, targets and portals with session state, transport
and storage ownership, negotiated parameters, session rescans, and
node-record removal per target or portal. Merge node records with active
sessions so disconnected targets remain visible. The session list now
records each session's transport for this status.
Normalize portal addresses before merging: iscsiadm uses bracketed or
bare IPv6 and IPv4-mapped addresses in different listings. For record
removal, rewrite an IPv4-mapped address's tail as hex groups, the only
form iscsiadm accepts with a port.
Allow reads with Sys.Audit on the node or Datastore.Allocate on /storage
so node auditors and storage administrators can use the SAN views.
Rescan and record removal require Sys.Modify.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: report the initiator name and alias as top-level fields, leave out
the session state 'logging-out' that is never reported, and also allow
reads with Sys.Audit on the node; move the iscsiadm helpers into
ISCSIPlugin; omit logout until its guarded implementation and adapt the
message; clarify support and node-record removal descriptions; record
the session transport]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/ISCSI.pm | 414 ++++++++++++++++++++++++++++
src/PVE/API2/Makefile | 1 +
src/PVE/Storage/ISCSIPlugin.pm | 156 ++++++++++-
src/test/run_iscsi_session_tests.pl | 15 +-
src/test/run_san_api_tests.pl | 88 ++++++
5 files changed, 665 insertions(+), 9 deletions(-)
create mode 100644 src/PVE/API2/ISCSI.pm
diff --git a/src/PVE/API2/ISCSI.pm b/src/PVE/API2/ISCSI.pm
new file mode 100644
index 00000000..28c5a6e4
--- /dev/null
+++ b/src/PVE/API2/ISCSI.pm
@@ -0,0 +1,414 @@
+package PVE::API2::ISCSI;
+
+use strict;
+use warnings;
+
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::RESTHandler;
+use PVE::Storage;
+use PVE::Storage::ISCSIPlugin;
+use PVE::Tools qw(run_command file_get_contents file_read_firstline);
+
+use base qw(PVE::RESTHandler);
+
+my $ISCSIADM = '/usr/bin/iscsiadm';
+my $INITIATOR_NAME_FILE = '/etc/iscsi/initiatorname.iscsi';
+
+my $session_id_schema = {
+ description => "Kernel iSCSI session id.",
+ type => 'integer',
+ minimum => 0,
+};
+
+my $target_schema = {
+ description => "iSCSI target name (IQN or EUI).",
+ type => 'string',
+ minLength => 1,
+ maxLength => 255,
+};
+
+my $status_return = {
+ type => 'object',
+ properties => {
+ supported => {
+ description => "Whether open-iscsi is installed on this node.",
+ type => 'boolean',
+ },
+ 'initiator-name' => {
+ description => "The initiator IQN (InitiatorName), if available.",
+ type => 'string',
+ optional => 1,
+ },
+ 'initiator-alias' => {
+ description => "The initiator alias (InitiatorAlias), if set.",
+ type => 'string',
+ optional => 1,
+ },
+ },
+};
+
+my $target_return = {
+ type => 'object',
+ properties => {
+ target => {
+ description => "The target IQN.",
+ type => 'string',
+ },
+ portal => {
+ description => "The portal ('host:port') this entry refers to.",
+ type => 'string',
+ },
+ 'session-id' => {
+ description => "The kernel session id; absent when there is no active session"
+ . " on this portal.",
+ type => 'integer',
+ optional => 1,
+ },
+ transport => {
+ description => "The transport the session runs over (for example 'tcp' or"
+ . " 'iser'); only set for active sessions.",
+ type => 'string',
+ optional => 1,
+ },
+ health => {
+ description => "The session health derived from the kernel session state:"
+ . " 'recovering' means the connection is down and iscsid is retrying the"
+ . " login, 'not-connected' means only a node record exists, without an"
+ . " active session.",
+ type => 'string',
+ enum => ['logged-in', 'recovering', 'not-connected', 'unknown'],
+ },
+ 'kernel-state' => {
+ description => "The raw kernel session state read from sysfs (for example"
+ . " 'LOGGED_IN'), if available.",
+ type => 'string',
+ optional => 1,
+ },
+ 'used-by-storage' => {
+ description => "Storage identifier of a configured iSCSI-type storage using"
+ . " this target.",
+ type => 'string',
+ optional => 1,
+ },
+ },
+};
+
+my $session_params_return = {
+ type => 'object',
+ properties => {
+ 'header-digest' => {
+ description => "The negotiated header digest.",
+ type => 'string',
+ optional => 1,
+ },
+ 'data-digest' => {
+ description => "The negotiated data digest.",
+ type => 'string',
+ optional => 1,
+ },
+ 'max-recv-data-segment-length' => {
+ description => "Max data segment length (bytes) the initiator accepts in a PDU.",
+ type => 'integer',
+ optional => 1,
+ },
+ 'max-xmit-data-segment-length' => {
+ description => "Max data segment length (bytes) the target accepts in a PDU.",
+ type => 'integer',
+ optional => 1,
+ },
+ 'first-burst-length' => {
+ description => "Max unsolicited data (bytes) sent with a command (first burst).",
+ type => 'integer',
+ optional => 1,
+ },
+ 'max-burst-length' => {
+ description => "Max data (bytes) in one solicited burst.",
+ type => 'integer',
+ optional => 1,
+ },
+ 'max-outstanding-r2t' => {
+ description => "Max number of outstanding Ready-To-Transfer PDUs.",
+ type => 'integer',
+ optional => 1,
+ },
+ 'immediate-data' => {
+ description => "Whether unsolicited immediate data is allowed ('Yes'/'No').",
+ type => 'string',
+ optional => 1,
+ },
+ 'initial-r2t' => {
+ description =>
+ "Whether the target requires an R2T before any data is sent ('Yes'/'No').",
+ type => 'string',
+ optional => 1,
+ },
+ 'recovery-timeout' => {
+ description =>
+ "Seconds the session waits for path recovery before failing its commands.",
+ type => 'integer',
+ optional => 1,
+ },
+ 'abort-timeout' => {
+ description => "Seconds to wait for a SCSI abort task to complete.",
+ type => 'integer',
+ optional => 1,
+ },
+ 'lu-reset-timeout' => {
+ description => "Seconds to wait for a LUN reset to complete.",
+ type => 'integer',
+ optional => 1,
+ },
+ 'tgt-reset-timeout' => {
+ description => "Seconds to wait for a target reset to complete.",
+ type => 'integer',
+ optional => 1,
+ },
+ },
+};
+
+my sub read_initiator_info {
+ my $content = eval { file_get_contents($INITIATOR_NAME_FILE) };
+ return undef if !defined($content);
+
+ my $res = {};
+ for my $line (split(/\n/, $content)) {
+ if ($line =~ m/^\s*InitiatorName\s*=\s*(\S+)\s*$/) {
+ $res->{name} = $1;
+ } elsif ($line =~ m/^\s*InitiatorAlias\s*=\s*(.+?)\s*$/) {
+ $res->{alias} = $1;
+ }
+ }
+ return defined($res->{name}) ? $res : undef;
+}
+
+my sub session_health {
+ my ($kernel_state) = @_;
+
+ return 'unknown' if !defined($kernel_state);
+ return 'logged-in' if $kernel_state eq 'LOGGED_IN';
+ return 'recovering' if $kernel_state eq 'FAILED';
+ return 'not-connected' if $kernel_state eq 'FREE';
+ return 'unknown';
+}
+
+my sub list_targets {
+ my $session_map = PVE::Storage::ISCSIPlugin::iscsi_session_list();
+ my $node_list = PVE::Storage::ISCSIPlugin::iscsi_node_list();
+
+ my $target_storage = {};
+ my $cfg = PVE::Storage::config();
+ for my $storeid (sort keys %{ $cfg->{ids} }) {
+ my $scfg = $cfg->{ids}->{$storeid};
+ next if $scfg->{type} ne 'iscsi' || !defined($scfg->{target});
+ $target_storage->{ $scfg->{target} } //= $storeid;
+ }
+
+ my $res = [];
+ my $have_session = {};
+ for my $target (sort keys %$session_map) {
+ for my $session (@{ $session_map->{$target} }) {
+ my $sid = int($session->{session_id});
+ my $state = file_read_firstline("/sys/class/iscsi_session/session${sid}/state");
+ my $portal = PVE::Storage::ISCSIPlugin::iscsi_normalize_portal($session->{portal});
+ my $entry = {
+ target => $target,
+ 'session-id' => $sid,
+ portal => $portal,
+ transport => $session->{transport},
+ health => session_health($state),
+ };
+ $entry->{'kernel-state'} = $state if defined($state);
+ $entry->{'used-by-storage'} = $target_storage->{$target}
+ if defined($target_storage->{$target});
+ push @$res, $entry;
+ $have_session->{"$target\0$portal"} = 1;
+ }
+ }
+
+ for my $node (@$node_list) {
+ my $portal = PVE::Storage::ISCSIPlugin::iscsi_normalize_portal($node->{portal});
+ next if $have_session->{"$node->{target}\0$portal"};
+ my $entry = {
+ target => $node->{target},
+ portal => $portal,
+ health => 'not-connected',
+ };
+ $entry->{'used-by-storage'} = $target_storage->{ $node->{target} }
+ if defined($target_storage->{ $node->{target} });
+ push @$res, $entry;
+ }
+
+ return [sort { $a->{target} cmp $b->{target} || $a->{portal} cmp $b->{portal} } @$res];
+}
+
+__PACKAGE__->register_method({
+ name => 'index',
+ path => '',
+ method => 'GET',
+ proxyto => 'node',
+ permissions => { user => 'all' },
+ description => "Node iSCSI initiator index.",
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ },
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => "object",
+ properties => {},
+ },
+ links => [{ rel => 'child', href => "{name}" }],
+ },
+ code => sub {
+ return [
+ { name => 'rescan' },
+ { name => 'session-params' },
+ { name => 'status' },
+ { name => 'target' },
+ ];
+ },
+});
+
+my $read_permissions = {
+ check => [
+ 'or',
+ ['perm', '/nodes/{node}', ['Sys.Audit']],
+ ['perm', '/storage', ['Datastore.Allocate']],
+ ],
+};
+__PACKAGE__->register_method({
+ name => 'status',
+ path => 'status',
+ method => 'GET',
+ protected => 1,
+ proxyto => 'node',
+ description => "Report iSCSI support and, when open-iscsi is installed, the"
+ . " initiator identity.",
+ permissions => $read_permissions,
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ },
+ },
+ returns => $status_return,
+ code => sub {
+ my $res = { supported => PVE::Storage::ISCSIPlugin::iscsi_supported() };
+ if ($res->{supported}) {
+ my $initiator = read_initiator_info();
+ if ($initiator) {
+ $res->{'initiator-name'} = $initiator->{name} if defined($initiator->{name});
+ $res->{'initiator-alias'} = $initiator->{alias} if defined($initiator->{alias});
+ }
+ }
+ return $res;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'targets',
+ path => 'target',
+ method => 'GET',
+ protected => 1,
+ proxyto => 'node',
+ description => "List the iSCSI targets known to the initiator, together with the state"
+ . " of their sessions.",
+ permissions => $read_permissions,
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ },
+ },
+ returns => {
+ type => 'array',
+ items => $target_return,
+ },
+ code => sub {
+ return list_targets();
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'target_delete',
+ path => 'target',
+ method => 'DELETE',
+ protected => 1,
+ proxyto => 'node',
+ description => "Remove a target's node records from the initiator database, for one portal"
+ . " or all portals. Log out of the affected sessions first.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ target => $target_schema,
+ portal => {
+ description => "Only remove the node record of this portal.",
+ type => 'string',
+ minLength => 2,
+ maxLength => 256,
+ optional => 1,
+ },
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+ return PVE::Storage::ISCSIPlugin::iscsi_delete_records($param->{target},
+ $param->{portal});
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'session_params',
+ path => 'session-params',
+ method => 'GET',
+ protected => 1,
+ proxyto => 'node',
+ description => "Read the parameters an active iSCSI session negotiated at login.",
+ permissions => $read_permissions,
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ 'session-id' => $session_id_schema,
+ },
+ },
+ returns => $session_params_return,
+ code => sub {
+ my ($param) = @_;
+ return PVE::Storage::ISCSIPlugin::iscsi_session_parameters($param->{'session-id'});
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'rescan',
+ path => 'rescan',
+ method => 'POST',
+ protected => 1,
+ proxyto => 'node',
+ description => "Rescan an iSCSI session to pick up newly attached LUNs.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ 'session-id' => $session_id_schema,
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+ return PVE::Storage::ISCSIPlugin::iscsi_rescan_session($param->{'session-id'});
+ },
+});
+
+1;
diff --git a/src/PVE/API2/Makefile b/src/PVE/API2/Makefile
index fe316c52..10e18f95 100644
--- a/src/PVE/API2/Makefile
+++ b/src/PVE/API2/Makefile
@@ -3,5 +3,6 @@
.PHONY: install
install:
install -D -m 0644 Disks.pm ${DESTDIR}${PERLDIR}/PVE/API2/Disks.pm
+ install -D -m 0644 ISCSI.pm ${DESTDIR}${PERLDIR}/PVE/API2/ISCSI.pm
make -C Storage install
make -C Disks install
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index 22f6c44f..e7d1ff3f 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -50,10 +50,15 @@ sub iscsi_session_list {
outfunc => sub {
my $line = shift;
# example: tcp: [1] 192.168.122.252:3260,1 iqn.2003-01.org.linux-iscsi.proxmox-nfs.x8664:sn.00567885ba8f (non-flash)
- if ($line =~ m/^\S+:\s+\[(\S+)\]\s+(\S+:\d+)\,\S+\s+(\S+)\s+\S+?\s*$/) {
- my ($session_id, $portal, $target) = ($1, $2, $3);
+ if ($line =~ m/^(\S+):\s+\[(\S+)\]\s+(\S+:\d+)\,\S+\s+(\S+)\s+\S+?\s*$/) {
+ my ($transport, $session_id, $portal, $target) = ($1, $2, $3, $4);
# there can be several sessions per target (multipath)
- push @{ $res->{$target} }, { session_id => $session_id, portal => $portal };
+ push @{ $res->{$target} },
+ {
+ session_id => $session_id,
+ portal => $portal,
+ transport => $transport,
+ };
}
},
);
@@ -65,6 +70,151 @@ sub iscsi_session_list {
return $res;
}
+sub iscsi_node_list {
+ assert_iscsi_support();
+
+ my $res = [];
+ eval {
+ run_command(
+ [$ISCSIADM, '--mode', 'node'],
+ errmsg => 'iscsi node record scan failed',
+ outfunc => sub {
+ my $line = shift;
+ if ($line =~ $ISCSI_TARGET_RE) {
+ push @$res, { portal => $1, target => $2 };
+ }
+ },
+ );
+ };
+ if (my $err = $@) {
+ die $err if $err !~ m/: No records found/i;
+ }
+
+ return $res;
+}
+
+sub iscsi_supported {
+ return -x $ISCSIADM ? 1 : 0;
+}
+
+# Session and node listings spell IPv6 differently; normalize brackets and mapped IPv4 for merging.
+sub iscsi_normalize_portal {
+ my ($portal) = @_;
+ return $portal if $portal !~ m/^(.+):(\d+)$/;
+ my ($addr, $port) = ($1, $2);
+ $addr =~ s/^\[(.*)\]$/$1/;
+ $addr =~ s/^::ffff:(\d+\.\d+\.\d+\.\d+)$/$1/i;
+ $addr = "[$addr]" if $addr =~ m/:/;
+ return "$addr:$port";
+}
+
+# iscsiadm accepts an IPv6 port only with brackets and no dots in the address. Encode an embedded
+# IPv4 tail as hex groups to address the stored record.
+my sub iscsiadm_portal_arg {
+ my ($portal) = @_;
+ my ($addr, $port) = $portal =~ m/^(.+):(\d+)$/;
+ return $portal if !defined($addr) || $addr !~ m/:/;
+ if ($addr =~ m/^(.*:)(\d+)\.(\d+)\.(\d+)\.(\d+)$/) {
+ $addr = sprintf("%s%x:%x", $1, ($2 << 8) + $3, ($4 << 8) + $5);
+ }
+ return "[$addr]:$port";
+}
+
+# Field names from the Timeouts and Negotiated iSCSI params sections of iscsiadm -m session -P 2.
+my $session_param_map = {
+ 'HeaderDigest' => ['header-digest', 'string'],
+ 'DataDigest' => ['data-digest', 'string'],
+ 'MaxRecvDataSegmentLength' => ['max-recv-data-segment-length', 'integer'],
+ 'MaxXmitDataSegmentLength' => ['max-xmit-data-segment-length', 'integer'],
+ 'FirstBurstLength' => ['first-burst-length', 'integer'],
+ 'MaxBurstLength' => ['max-burst-length', 'integer'],
+ 'ImmediateData' => ['immediate-data', 'string'],
+ 'InitialR2T' => ['initial-r2t', 'string'],
+ 'MaxOutstandingR2T' => ['max-outstanding-r2t', 'integer'],
+ 'Recovery Timeout' => ['recovery-timeout', 'integer'],
+ 'Abort Timeout' => ['abort-timeout', 'integer'],
+ 'LUN Reset Timeout' => ['lu-reset-timeout', 'integer'],
+ 'Target Reset Timeout' => ['tgt-reset-timeout', 'integer'],
+};
+
+sub iscsi_session_parameters {
+ my ($session_id) = @_;
+
+ my $res = {};
+ run_command(
+ [$ISCSIADM, '--mode', 'session', '--sid', $session_id, '-P', '2'],
+ errmsg => "reading parameters of iSCSI session $session_id failed",
+ outfunc => sub {
+ my ($line) = @_;
+ return if $line !~ m/^\s*([^:]+?)\s*:\s*(.+?)\s*$/;
+ my ($key, $value) = ($1, $2);
+ my $field = $session_param_map->{$key} or return;
+ my ($name, $type) = @$field;
+ if ($type eq 'integer') {
+ return if $value !~ m/^\d+$/;
+ $value = int($value);
+ }
+ $res->{$name} = $value;
+ },
+ );
+ return $res;
+}
+
+sub iscsi_delete_records {
+ my ($target, $portal) = @_;
+
+ my $sessions = iscsi_session_list()->{$target} // [];
+ for my $session (@$sessions) {
+ next if defined($portal) && iscsi_normalize_portal($session->{portal}) ne $portal;
+ die "target '$target' still has an active session - log out first\n";
+ }
+
+ if (!defined($portal)) {
+ run_command(
+ [$ISCSIADM, '--mode', 'node', '--targetname', $target, '--op', 'delete'],
+ errmsg => "removing the node records of target '$target' failed",
+ outfunc => sub { },
+ );
+ return undef;
+ }
+
+ # Compare normalized portals but address records in their stored representation.
+ my @records =
+ grep { $_->{target} eq $target && iscsi_normalize_portal($_->{portal}) eq $portal }
+ @{ iscsi_node_list() };
+ die "no node record for target '$target', portal '$portal' found\n" if !scalar(@records);
+
+ for my $record (@records) {
+ run_command(
+ [
+ $ISCSIADM,
+ '--mode',
+ 'node',
+ '--targetname',
+ $target,
+ '--portal',
+ iscsiadm_portal_arg($record->{portal}),
+ '--op',
+ 'delete',
+ ],
+ errmsg => "removing the node record of target '$target' failed",
+ outfunc => sub { },
+ );
+ }
+ return undef;
+}
+
+sub iscsi_rescan_session {
+ my ($sid) = @_;
+
+ run_command(
+ [$ISCSIADM, '--mode', 'session', '--sid', $sid, '--rescan'],
+ errmsg => "rescan of iSCSI session $sid failed",
+ outfunc => sub { },
+ );
+ return undef;
+}
+
sub iscsi_test_session {
my ($sid) = @_;
diff --git a/src/test/run_iscsi_session_tests.pl b/src/test/run_iscsi_session_tests.pl
index a338b9b4..c68b54fc 100755
--- a/src/test/run_iscsi_session_tests.pl
+++ b/src/test/run_iscsi_session_tests.pl
@@ -37,20 +37,23 @@ my sub sessions {
for my $transport (qw(tcp iser bnx2i be2iscsi cxgb4i qedi qla4xxx)) {
is_deeply(
sessions("$transport: [3] 192.0.2.1:3260,1 iqn.test (non-flash)"),
- { 'iqn.test' => [{ session_id => 3, portal => '192.0.2.1:3260' }] },
+ {
+ 'iqn.test' =>
+ [{ session_id => 3, portal => '192.0.2.1:3260', transport => $transport }],
+ },
"$transport session listed",
);
}
is_deeply(
sessions('qla4xxx: [4] 192.0.2.1:3260,1 iqn.test (flash)'),
- { 'iqn.test' => [{ session_id => 4, portal => '192.0.2.1:3260' }] },
+ { 'iqn.test' => [{ session_id => 4, portal => '192.0.2.1:3260', transport => 'qla4xxx' }] },
'flash node session listed',
);
is_deeply(
sessions('iser: [2] [2001:db8::1]:3260,1 iqn.test (non-flash)'),
- { 'iqn.test' => [{ session_id => 2, portal => '[2001:db8::1]:3260' }] },
+ { 'iqn.test' => [{ session_id => 2, portal => '[2001:db8::1]:3260', transport => 'iser' }] },
'IPv6 portal keeps its brackets',
);
@@ -62,10 +65,10 @@ is_deeply(
),
{
'iqn.a' => [
- { session_id => 1, portal => '192.0.2.1:3260' },
- { session_id => 2, portal => '192.0.2.2:3260' },
+ { session_id => 1, portal => '192.0.2.1:3260', transport => 'tcp' },
+ { session_id => 2, portal => '192.0.2.2:3260', transport => 'iser' },
],
- 'iqn.b' => [{ session_id => 3, portal => '192.0.2.3:3260' }],
+ 'iqn.b' => [{ session_id => 3, portal => '192.0.2.3:3260', transport => 'bnx2i' }],
},
'sessions of one target are grouped across transports',
);
diff --git a/src/test/run_san_api_tests.pl b/src/test/run_san_api_tests.pl
index ac5a8be3..00d719a0 100755
--- a/src/test/run_san_api_tests.pl
+++ b/src/test/run_san_api_tests.pl
@@ -7,10 +7,26 @@ use Test::MockModule;
use Test::More;
use lib ('.', '..');
+use PVE::API2::ISCSI;
use PVE::API2::Storage::Scan;
+subtest 'iSCSI status fields' => sub {
+ my $method = PVE::API2::ISCSI->map_method_by_name('status');
+ ok($method->{returns}->{properties}->{'initiator-name'}, 'name schema');
+ ok($method->{returns}->{properties}->{'initiator-alias'}, 'alias schema');
+ my $enum =
+ PVE::API2::ISCSI->map_method_by_name('targets')->{returns}->{items}->{properties}->{health}
+ ->{enum};
+ is_deeply(
+ $enum,
+ ['logged-in', 'recovering', 'not-connected', 'unknown'],
+ 'only produced session states',
+ );
+};
+
subtest 'node SAN read permissions' => sub {
for my $case (
+ ['PVE::API2::ISCSI', qw(status targets session_params)],
['PVE::API2::Storage::Scan', 'sanlunscan'],
) {
my ($class, @methods) = @$case;
@@ -26,6 +42,18 @@ subtest 'node SAN read permissions' => sub {
);
}
}
+ for my $case (
+ ['PVE::API2::ISCSI', qw(target_delete rescan)],
+ ) {
+ my ($class, @methods) = @$case;
+ for my $name (@methods) {
+ is_deeply(
+ $class->map_method_by_name($name)->{permissions}->{check},
+ ['perm', '/', ['Sys.Modify']],
+ "$class $name write gate",
+ );
+ }
+ }
};
subtest 'SAN CLI columns leave machine output complete' => sub {
@@ -94,4 +122,64 @@ subtest 'SAN LUN path counts' => sub {
}
};
+subtest 'iSCSI transport helpers preserve command arguments' => sub {
+ my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+ my (@commands, $sessions);
+ $plugin->mock(
+ iscsi_session_list => sub { { 'iqn.test' => $sessions // [] } },
+ iscsi_node_list =>
+ sub { [{ target => 'iqn.test', portal => '::ffff:192.0.2.1:3260' }] },
+ run_command => sub {
+ my ($cmd, %param) = @_;
+ push @commands, $cmd;
+ $param{outfunc}->($_)
+ for 'HeaderDigest: None', 'Recovery Timeout: 120',
+ 'MaxBurstLength: invalid';
+ },
+ );
+ my $call = sub { PVE::API2::ISCSI->map_method_by_name($_[0])->{code}->($_[1]) };
+ is_deeply(
+ $call->('session_params', { 'session-id' => 7 }),
+ { 'header-digest' => 'None', 'recovery-timeout' => 120 },
+ 'session parser retains recognized values and rejects invalid integers',
+ );
+ is_deeply(
+ $commands[-1],
+ ['/usr/bin/iscsiadm', '--mode', 'session', '--sid', 7, '-P', '2'],
+ 'session query arguments',
+ );
+ $call->('rescan', { 'session-id' => 7 });
+ is_deeply(
+ $commands[-1],
+ ['/usr/bin/iscsiadm', '--mode', 'session', '--sid', 7, '--rescan'],
+ 'rescan arguments',
+ );
+ $call->('target_delete', { target => 'iqn.test', portal => '192.0.2.1:3260' });
+ is_deeply(
+ $commands[-1],
+ [
+ '/usr/bin/iscsiadm',
+ '--mode',
+ 'node',
+ '--targetname',
+ 'iqn.test',
+ '--portal',
+ '[::ffff:c000:201]:3260',
+ '--op',
+ 'delete',
+ ],
+ 'mapped IPv4 record uses its stored address with an iscsiadm-compatible spelling',
+ );
+ $sessions = [{ portal => '[::ffff:192.0.2.1]:3260' }];
+ @commands = ();
+ eval { $call->('target_delete', { target => 'iqn.test', portal => '192.0.2.1:3260' }) };
+ like($@, qr/still has an active session/, 'active normalized portal refuses deletion');
+ is_deeply(\@commands, [], 'refusal does not invoke delete');
+ is(
+ PVE::Storage::ISCSIPlugin::iscsi_normalize_portal('2001:db8::1:3260'),
+ '[2001:db8::1]:3260',
+ 'IPv6 comparison key',
+ );
+};
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 27/76] api: iscsi: add session logout with last-path protection
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (25 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 26/76] api: add node-level iSCSI status and maintenance Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 28/76] multipath: add a cluster-wide configuration Thomas Lamprecht
` (48 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Session logout removes its LUN devices. Losing the last path makes
guests lose their disks or hang if the multipath map queues I/O.
Refuse logout if it would remove the last usable path of a LUN with
local users. Check users of a multipath map unless another path is
active, its checker found it ready or as an ALUA standby, and it
belongs to a logged-in iSCSI session or is a running Fibre Channel
path. For a plain disk, check the disk and its partitions. Always check
direct users of departing paths, excluding multipathd's path-checker
descriptors.
Claim devices losing their last path exclusively until logout returns.
Serialize logouts so two requests cannot each rely on a path the other
removes. Configured storage ownership alone does not block logout:
storage activation can log in again.
This exposes the logout endpoint from Dietmar Maurer's iSCSI API RFC
with these guards.
Link: https://lore.proxmox.com/pve-devel/20260731102156.3947857-1-dietmar@proxmox.com/
Co-authored-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/ISCSI.pm | 27 +++
src/PVE/Diskmanage.pm | 9 +
src/PVE/Storage/ISCSIPlugin.pm | 190 +++++++++++++++-
src/test/Makefile | 3 +-
src/test/run_block_device_users_tests.pl | 37 +++-
src/test/run_iscsi_logout_tests.pl | 271 +++++++++++++++++++++++
src/test/run_san_api_tests.pl | 2 +-
7 files changed, 525 insertions(+), 14 deletions(-)
create mode 100755 src/test/run_iscsi_logout_tests.pl
diff --git a/src/PVE/API2/ISCSI.pm b/src/PVE/API2/ISCSI.pm
index 28c5a6e4..a3d55845 100644
--- a/src/PVE/API2/ISCSI.pm
+++ b/src/PVE/API2/ISCSI.pm
@@ -265,6 +265,7 @@ __PACKAGE__->register_method({
code => sub {
return [
{ name => 'rescan' },
+ { name => 'session-logout' },
{ name => 'session-params' },
{ name => 'status' },
{ name => 'target' },
@@ -387,6 +388,32 @@ __PACKAGE__->register_method({
},
});
+__PACKAGE__->register_method({
+ name => 'session_logout',
+ path => 'session-logout',
+ method => 'POST',
+ protected => 1,
+ proxyto => 'node',
+ description => "Log out of a single iSCSI session. Refused if a device being disconnected"
+ . " is used directly or a LUN in use would lose its last usable path.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ node => get_standard_option('pve-node'),
+ 'session-id' => $session_id_schema,
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+ PVE::Storage::ISCSIPlugin::iscsi_logout_guarded($param->{'session-id'});
+ return undef;
+ },
+});
+
__PACKAGE__->register_method({
name => 'rescan',
path => 'rescan',
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index 655cee37..0aa005d2 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -70,11 +70,15 @@ sub block_device_number {
return PVE::Tools::dev_t_major($st[6]) . ':' . PVE::Tools::dev_t_minor($st[6]);
}
+# Never read device data. Probe failures are not evidence of non-use. Excluding a holder must not
+# exclude direct users; multipathd can be exempted because its checkers keep member paths open.
sub block_device_users {
my ($devices, $options) = @_;
$options //= {};
my $sys = $options->{sysfs} // '/sys';
my $proc = $options->{procfs} // '/proc';
+ my $ignore = $options->{'ignore-holders'} // {};
+ my $ignore_executables = $options->{'ignore-executables'} // {};
my $entries = sub {
my ($dir, $optional) = @_;
@@ -110,6 +114,7 @@ sub block_device_users {
$visit->($part);
}
for my $holder ($entries->("$dir/holders")->@*) {
+ next if $ignore->{$holder};
push $users->@*, "$name held by $holder";
$visit->($holder);
}
@@ -199,6 +204,10 @@ sub block_device_users {
die $err;
}
next if !defined($dev) || !defined($numbers->{$dev});
+ if (%$ignore_executables) {
+ my $exe = readlink("$proc/$pid/exe");
+ next if defined($exe) && $ignore_executables->{$exe};
+ }
my $comm = file_read_firstline("$proc/$pid/comm") // '?';
push $users->@*, "$comm (PID $pid) has $numbers->{$dev} open";
}
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index e7d1ff3f..d10a5549 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -3,12 +3,15 @@ package PVE::Storage::ISCSIPlugin;
use strict;
use warnings;
+use Cwd qw(abs_path);
use Digest::SHA qw(sha256_hex);
-use Fcntl qw(LOCK_EX LOCK_NB);
+use Fcntl qw(O_RDONLY O_EXCL O_NONBLOCK LOCK_EX LOCK_NB);
use IO::Dir;
use IO::File;
+use PVE::Diskmanage;
use PVE::JSONSchema qw(get_standard_option);
+use PVE::Multipath;
use PVE::Storage::Plugin;
use PVE::Tools
qw(run_command file_read_firstline trim dir_glob_regex dir_glob_foreach $IPV4RE $IPV6RE);
@@ -19,6 +22,7 @@ use base qw(PVE::Storage::Plugin);
my $ISCSIADM = '/usr/bin/iscsiadm';
our $SYSFS = '/sys';
+my $MULTIPATHD_EXECUTABLES = { '/usr/sbin/multipathd' => 1, '/sbin/multipathd' => 1 };
my $found_iscsi_adm_exe;
my sub assert_iscsi_support {
@@ -434,17 +438,32 @@ sub load_stable_scsi_paths {
return $stable_paths;
}
+# Strict callers die on unreadable sysfs directories instead of skipping them, so missing topology
+# cannot be mistaken for an absent device.
+my sub foreach_entry {
+ my ($strict, $dir, $regex, $code) = @_;
+
+ return dir_glob_foreach($dir, $regex, $code) if !$strict;
+ opendir(my $dh, $dir) or die "unable to inspect '$dir' - $!\n";
+ while (defined(my $entry = readdir($dh))) {
+ $code->($entry, $entry =~ m/^$regex$/) if $entry =~ m/^$regex$/;
+ }
+ closedir($dh) or die "unable to close '$dir' - $!\n";
+}
+
# Offload initiators can run several sessions on one SCSI host, so the LUNs of a session are the
# ones below its own target, not all LUNs of its host.
my sub foreach_session_lun {
- my ($sessiondir, $code) = @_;
+ my ($sessiondir, $code, $strict) = @_;
- dir_glob_foreach(
+ foreach_entry(
+ $strict,
$sessiondir,
'target\d+:\d+:\d+',
sub {
my ($targetdir) = @_;
- dir_glob_foreach(
+ foreach_entry(
+ $strict,
"$sessiondir/$targetdir",
'\d+:(\d+):(\d+):(\d+)',
sub {
@@ -456,27 +475,42 @@ my sub foreach_session_lun {
);
}
+# Destructive callers request strict probing so missing topology cannot be mistaken for a session
+# without devices.
sub iscsi_session_device_map {
+ my ($strict) = @_;
my $res = {};
my $dirname = "$SYSFS/class/iscsi_session";
- dir_glob_foreach(
+ foreach_entry(
+ $strict,
$dirname,
'session(\d+)',
sub {
my ($ent, $session) = @_;
my $target = file_read_firstline("$dirname/$ent/targetname");
+ die "unable to identify iSCSI session $session - missing target\n"
+ if $strict && !$target;
return if !$target;
+ my $device = abs_path("$dirname/$ent/device") // '';
+ my ($host) = $device =~ m{/host(\d+)/session\d+$};
+ die "unable to identify iSCSI session $session - missing host\n"
+ if $strict && !defined($host);
+ return if !defined($host);
+
foreach_session_lun(
- "$dirname/$ent/device",
+ $device,
sub {
my ($devdir, $channel, $id, $lun) = @_;
+ my ($tmp) = $devdir =~ m{([^/]+)$};
my $type = file_read_firstline("$devdir/type");
+ die "unable to identify SCSI device '$tmp' - missing type\n"
+ if $strict && !defined($type);
return if !defined($type) || $type ne '0'; # list disks only
my $bdev;
@@ -485,23 +519,165 @@ sub iscsi_session_device_map {
} else {
(undef, $bdev) = dir_glob_regex($devdir, 'block:(\S+)');
}
+ die "unable to identify SCSI device '$tmp' - missing block device\n"
+ if $strict && !$bdev;
return if !$bdev;
$res->{$bdev} = {
target => $target,
+ session => int($session),
+ host => int($host),
channel => int($channel),
id => int($id),
lun => int($lun),
};
},
+ $strict,
);
-
},
);
return $res;
}
+# block device names that end up in commands and device opens, untainted by their expected form
+my sub block_entries {
+ my ($dir) = @_;
+ opendir(my $dh, $dir) or die "unable to inspect '$dir' - $!\n";
+ my @entries = sort grep { $_ ne '.' && $_ ne '..' } readdir($dh);
+ closedir($dh) or die "unable to close '$dir' - $!\n";
+ return [map { m/^([\w.:+-]+)$/ ? $1 : die "unexpected entry '$_' in '$dir'\n" } @entries];
+}
+
+sub iscsi_claim_device {
+ my ($name, $number) = @_;
+
+ sysopen(my $fh, "/dev/$name", O_RDONLY | O_EXCL | O_NONBLOCK)
+ or die "unable to claim '/dev/$name' - $!\n";
+ die "device '/dev/$name' changed during logout\n"
+ if (PVE::Diskmanage::block_device_number($fh) // '') ne $number;
+ return $fh;
+}
+
+# Serialize path removals so concurrent logouts cannot each depend on the other's surviving path.
+# Storage ownership is not a user: an enabled storage can reconnect after a safe logout.
+sub iscsi_logout_guarded {
+ my ($sid) = @_;
+
+ my $result = PVE::Tools::lock_file(
+ '/run/lock/pve-iscsi-lifecycle.lock',
+ 10,
+ sub {
+ my $session_dir = "$SYSFS/class/iscsi_session/session$sid";
+ my $session_ino = (CORE::stat($session_dir))[1]
+ // die "unable to identify iSCSI session $sid\n";
+ my $devices = iscsi_session_device_map(1);
+ my @departing = sort grep { $devices->{$_}->{session} == $sid } keys %$devices;
+ my %departing = map { $_ => 1 } @departing;
+ my (%claim, @users, @claims);
+ my $maps;
+ for my $name (@departing) {
+ my $dir = "$SYSFS/class/block/$name";
+ my @holders = block_entries("$dir/holders")->@*;
+ my @maps;
+ for my $holder (@holders) {
+ next if $holder !~ m/^dm-\d+$/;
+ my $uuid = file_read_firstline("$SYSFS/class/block/$holder/dm/uuid")
+ // die "unable to identify holder '$holder'\n";
+ push @maps, [$holder, $1] if $uuid =~ m/^mpath-(\S+)$/;
+ }
+ die "device '$name' belongs to multiple multipath maps\n" if @maps > 1;
+ my $ignore = {};
+ if (@maps) {
+ my ($map_name, $wwid) = $maps[0]->@*;
+ $ignore->{$map_name} = 1;
+ $maps //= PVE::Multipath::get_maps();
+ my ($map) =
+ grep { ($_->{sysfs} // '') eq $map_name && $_->{wwid} eq $wwid } @$maps;
+ die "unable to inspect multipath map '$map_name'\n" if !$map;
+ my %slaves =
+ map { $_ => 1 } block_entries("$SYSFS/class/block/$map_name/slaves")->@*;
+ # Another path counts if device-mapper uses it and its checker last found it
+ # usable; an unverified or failing checker state is no evidence of survival.
+ my $survives = 0;
+ for my $path (map { $_->{paths}->@* } $map->{'path-groups'}->@*) {
+ my $other = $path->{dev};
+ next
+ if !$slaves{$other}
+ || $departing{$other}
+ || ($path->{'dm-state'} // '') ne 'active'
+ || ($path->{'check-state'} // '') !~ m/^(?:ready|ghost)$/;
+ if (my $session = $devices->{$other}) {
+ my $state = file_read_firstline(
+ "$SYSFS/class/iscsi_session/session$session->{session}/state");
+ $survives = 1 if defined($state) && $state eq 'LOGGED_IN';
+ } else {
+ my $link = abs_path("$SYSFS/class/block/$other/device") // '';
+ my $state =
+ file_read_firstline("$SYSFS/class/block/$other/device/state") // '';
+ $survives = 1 if $link =~ m{/rport-} && $state eq 'running';
+ }
+ }
+ if (!$survives && !$claim{$map_name}) {
+ my $open = '';
+ run_command(
+ [
+ 'dmsetup',
+ 'info',
+ '-c',
+ '--noheadings',
+ '-o',
+ 'open',
+ "/dev/$map_name",
+ ],
+ timeout => 10,
+ outfunc => sub { $open .= "$_[0]\n" },
+ );
+ die "unable to inspect map '$map_name' - malformed open count\n"
+ if $open !~ m/^\s*(\d+)\s*$/;
+ $open = $1;
+ push @users,
+ "$map_name has $open open " . ($open == 1 ? 'reference' : 'references')
+ if $open > 0;
+ push @users, PVE::Diskmanage::block_device_users([$map_name])->@*;
+ $claim{$map_name} = 1;
+ }
+ } else {
+ $claim{$name} = 1;
+ }
+ # multipathd's checkers keep every path of a map open, which does not count as use
+ my $daemon = @maps ? $MULTIPATHD_EXECUTABLES : {};
+ push @users,
+ PVE::Diskmanage::block_device_users(
+ [$name],
+ { 'ignore-holders' => $ignore, 'ignore-executables' => $daemon },
+ )->@*;
+ }
+ die
+ "cannot log out of iSCSI session '$sid' - a device is used directly or a LUN in use"
+ . " would lose its last usable path: "
+ . join(', ', @users) . "\n"
+ if @users;
+ for my $name (sort keys %claim) {
+ my $number = file_read_firstline("$SYSFS/class/block/$name/dev");
+ die "unable to identify device '$name'\n"
+ if !defined($number) || $number !~ m/^\d+:\d+$/;
+ push @claims, iscsi_claim_device($name, $number);
+ }
+ die "iSCSI session $sid changed during logout\n"
+ if ((CORE::stat($session_dir))[1] // '') ne $session_ino;
+ run_command(
+ [$ISCSIADM, '--mode', 'session', '--sid', $sid, '--logout'],
+ errmsg => "logout of iSCSI session $sid failed",
+ outfunc => sub { },
+ );
+ return;
+ },
+ );
+ die $@ if $@;
+ return $result;
+}
+
sub iscsi_device_list {
my $res = {};
diff --git a/src/test/Makefile b/src/test/Makefile
index 9962bdfd..3517db6e 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -55,7 +55,8 @@ test_multipath: run_multipath_tests.pl
test_san_api: run_san_api_tests.pl
./run_san_api_tests.pl
-test_iscsi: run_iscsi_discovery_tests.pl run_iscsi_rescan_tests.pl run_iscsi_device_list_tests.pl run_iscsi_session_tests.pl run_iscsi_volume_tests.pl
+test_iscsi: run_iscsi_logout_tests.pl run_iscsi_discovery_tests.pl run_iscsi_rescan_tests.pl run_iscsi_device_list_tests.pl run_iscsi_session_tests.pl run_iscsi_volume_tests.pl
+ ./run_iscsi_logout_tests.pl
./run_iscsi_discovery_tests.pl
./run_iscsi_rescan_tests.pl
./run_iscsi_device_list_tests.pl
diff --git a/src/test/run_block_device_users_tests.pl b/src/test/run_block_device_users_tests.pl
index 5a27b503..1c040f60 100755
--- a/src/test/run_block_device_users_tests.pl
+++ b/src/test/run_block_device_users_tests.pl
@@ -72,22 +72,49 @@ $numbers{"$root/proc/999999/fd/3"} = '8:1';
write_file('/proc/999999/comm', "qemu\n");
is_deeply(
PVE::Diskmanage::block_device_users(
- ['sda'], $options,
+ ['sda'],
+ { %$options, 'ignore-holders' => { 'dm-0' => 1 } },
),
- ['sda held by dm-0', 'qemu (PID 999999) has sda1 open'],
- 'holders and direct users of a member partition are both listed',
+ ['qemu (PID 999999) has sda1 open'],
+ 'ignoring a map does not hide direct users of a member partition',
);
unlink("$root/proc/999999/fd/3") or die $!;
symlink('/srv/raw-disk', "$root/proc/999999/fd/3") or die $!;
is_deeply(
PVE::Diskmanage::block_device_users(
- ['sda'], $options,
+ ['sda'],
+ { %$options, 'ignore-holders' => { 'dm-0' => 1 } },
),
- ['sda held by dm-0', 'qemu (PID 999999) has sda1 open'],
+ ['qemu (PID 999999) has sda1 open'],
'device node outside /dev is not silently skipped',
);
unlink("$root/proc/999999/fd/3") or die $!;
+symlink('/dev/alias', "$root/proc/999999/fd/3") or die $!;
+symlink('/usr/sbin/multipathd', "$root/proc/999999/exe") or die $!;
+write_file('/proc/999999/comm', "multipathd\n");
+is_deeply(
+ PVE::Diskmanage::block_device_users(
+ ['sda'],
+ {
+ %$options,
+ 'ignore-holders' => { 'dm-0' => 1 },
+ 'ignore-executables' => { '/usr/sbin/multipathd' => 1 },
+ },
+ ),
+ [],
+ 'an ignored executable holding a member open is no user',
+);
+is_deeply(
+ PVE::Diskmanage::block_device_users(
+ ['sda'],
+ { %$options, 'ignore-holders' => { 'dm-0' => 1 } },
+ ),
+ ['multipathd (PID 999999) has sda1 open'],
+ 'other callers still see the executable',
+);
+unlink("$root/proc/999999/fd/3", "$root/proc/999999/exe") or die $!;
+
write_file('/sys/class/block/loop0/loop/backing_file', "/dev/alias\n");
write_file('/sys/kernel/config/nvmet/subsystems/test/namespaces/1/device_path', "/dev/alias\n");
write_file('/sys/kernel/config/target/core/iblock_0/test/udev_path', "/dev/alias\n");
diff --git a/src/test/run_iscsi_logout_tests.pl b/src/test/run_iscsi_logout_tests.pl
new file mode 100755
index 00000000..b88e5740
--- /dev/null
+++ b/src/test/run_iscsi_logout_tests.pl
@@ -0,0 +1,271 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use File::Path qw(make_path);
+use File::Temp qw(tempdir);
+use Scalar::Util qw(weaken);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Storage::ISCSIPlugin;
+
+my $root = tempdir(CLEANUP => 1);
+local $PVE::Storage::ISCSIPlugin::SYSFS = "$root/sys";
+my sub put {
+ my ($path, $value) = @_;
+ my ($dir) = "$root$path" =~ m{^(.*)/[^/]+$};
+ make_path($dir);
+ open(my $fh, '>', "$root$path") or die $!;
+ print $fh "$value\n";
+ close($fh) or die $!;
+}
+# The sysfs layout of the kernel's iSCSI transport class: the SCSI target and its LUNs are children
+# of the session device, and the SCSI bus lists the LUNs by H:C:T:L.
+my sub session {
+ my ($sid, $host, $target, $hctl, $dev, $number) = @_;
+ put("/sys/class/iscsi_session/session$sid/targetname", $target);
+ put("/sys/class/iscsi_session/session$sid/state", 'LOGGED_IN');
+ my $sessiondir = "/sys/devices/host$host/session$sid";
+ make_path("$root$sessiondir");
+ symlink("$root$sessiondir", "$root/sys/class/iscsi_session/session$sid/device") or die $!;
+ my ($tdir) = $hctl =~ m/^(\d+:\d+:\d+):\d+$/;
+ my $lundir = "$sessiondir/target$tdir/$hctl";
+ put("$lundir/type", '0');
+ make_path("$root$lundir/block/$dev", "$root/sys/bus/scsi/devices");
+ symlink("$root$lundir", "$root/sys/bus/scsi/devices/$hctl") or die $!;
+ put("/sys/class/block/$dev/dev", $number);
+ make_path("$root/sys/class/block/$dev/holders");
+}
+session(1, 1, 'iqn.same', '1:0:0:1', 'sda', '8:0');
+session(2, 2, 'iqn.same', '2:0:0:1', 'sdb', '8:16');
+put('/sys/class/block/dm-0/dev', '253:0');
+put('/sys/class/block/dm-0/dm/uuid', 'mpath-test');
+make_path("$root/sys/class/block/dm-0/slaves/sda", "$root/sys/class/block/dm-0/slaves/sdb");
+symlink('../../dm-0', "$root/sys/class/block/sda/holders/dm-0") or die $!;
+symlink('../../dm-0', "$root/sys/class/block/sdb/holders/dm-0") or die $!;
+my $devices = PVE::Storage::ISCSIPlugin::iscsi_session_device_map(1);
+is($devices->{sda}->{session}, 1, 'first session identified');
+is($devices->{sdb}->{session}, 2, 'same target on another session distinguished');
+is($devices->{sdb}->{host}, 2, 'host identified');
+
+my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+my $disk = Test::MockModule->new('PVE::Diskmanage');
+my $mp = Test::MockModule->new('PVE::Multipath');
+my $tools = Test::MockModule->new('PVE::Tools');
+my (
+ $locked,
+ $dm_active,
+ $check_state,
+ $fc_state,
+ $opens,
+ $map_users,
+ $direct_users,
+ $fail_probe,
+ $fail_claim,
+);
+my (@checks, @logouts, %claims);
+$tools->mock(
+ lock_file => sub {
+ my ($path, $timeout, $code) = @_;
+ like($path, qr/iscsi-lifecycle/, 'node-wide lifecycle lock');
+ die "overlapping logout\n" if $locked;
+ $locked = 1;
+ my $res = eval { $code->() };
+ my $err = $@;
+ $locked = 0;
+ die $err if $err;
+ return $res;
+ },
+);
+$mp->mock(
+ get_maps => sub {
+ ok($locked, 'map topology under lock');
+ return [{
+ sysfs => 'dm-0',
+ wwid => 'test',
+ 'path-groups' => [{
+ paths => [
+ { dev => 'sda', 'dm-state' => 'active', 'check-state' => 'ready' },
+ {
+ dev => 'sdb',
+ 'dm-state' => $dm_active ? 'active' : 'failed',
+ defined($check_state) ? ('check-state' => $check_state) : (),
+ },
+ defined($fc_state)
+ ? ({ dev => 'sdf', 'dm-state' => 'active', 'check-state' => $fc_state })
+ : (),
+ ],
+ }],
+ }];
+ },
+);
+$disk->mock(
+ block_device_users => sub {
+ my ($names, $options) = @_;
+ ok($locked, 'user check under lock');
+ my $name = $names->[0];
+ push @checks, $name;
+ die "user probe failed\n" if $fail_probe;
+ if ($name eq 'sda' && -e "$root/sys/class/block/sda/holders/dm-0") {
+ is_deeply($options->{'ignore-holders'}, { 'dm-0' => 1 },
+ 'only map holder excluded');
+ ok(
+ $options->{'ignore-executables'}->{'/usr/sbin/multipathd'},
+ 'multipathd is no user',
+ );
+ }
+ return $name eq 'dm-0' ? $map_users : $direct_users;
+ },
+);
+$plugin->mock(
+ run_command => sub {
+ my ($cmd, %opt) = @_;
+ ok($locked, 'commands under lock');
+ if ($cmd->[0] eq 'dmsetup') {
+ $opt{outfunc}->($opens);
+ } else {
+ push @logouts, $cmd;
+ ok(defined(fileno($claims{$_})), 'exclusive claim held during logout')
+ for keys %claims;
+ }
+ },
+ iscsi_claim_device => sub {
+ my ($name, $number) = @_;
+ ok($locked, 'claim under lock');
+ die "claim failed\n" if $fail_claim;
+ open(my $fh, '<', "$root/sys/class/block/$name/dev") or die $!;
+ $claims{$name} = $fh;
+ weaken($claims{$name});
+ return $fh;
+ },
+);
+my sub reset_case {
+ (
+ $dm_active, $check_state, $fc_state, $opens, $map_users, $direct_users, $fail_probe,
+ $fail_claim,
+ ) = (1, 'ready', undef, 0, [], [], 0, 0);
+ @checks = ();
+ @logouts = ();
+ %claims = ();
+ put('/sys/class/iscsi_session/session2/state', 'LOGGED_IN');
+}
+my sub logout {
+ my ($sid) = @_;
+ eval { PVE::Storage::ISCSIPlugin::iscsi_logout_guarded($sid // 1) };
+ return $@;
+}
+reset_case();
+$map_users = ['guest uses map'];
+is(logout(), '', 'in-use map retains another usable path');
+is_deeply(\@checks, ['sda'], 'departing member checked, not surviving map');
+is(scalar(@logouts), 1, 'logout executed');
+reset_case();
+($check_state, $map_users) = ('ghost', ['guest uses map']);
+is(logout(), '', 'an ALUA standby path is a usable alternate');
+
+# a running Fibre Channel path of the map is an alternate as well, by the same checker states
+my $rport = "$root/sys/devices/host5/rport-5:0-0/target5:0:0/5:0:0:1";
+make_path($rport, "$root/sys/class/block/sdf", "$root/sys/class/block/dm-0/slaves/sdf");
+put('/sys/devices/host5/rport-5:0-0/target5:0:0/5:0:0:1/state', 'running');
+symlink($rport, "$root/sys/class/block/sdf/device") or die $!;
+for my $case (['ready', 1], ['ghost', 1], ['faulty', 0], ['shaky', 0]) {
+ my ($state, $usable) = @$case;
+ reset_case();
+ ($dm_active, $fc_state, $map_users) = (0, $state, ['guest uses map']);
+ if ($usable) {
+ is(logout(), '', "a Fibre Channel alternate with checker state $state is usable");
+ } else {
+ like(logout(), qr/last usable path/, "a Fibre Channel alternate in state $state is not");
+ is(scalar(@logouts), 0, 'no logout on refusal');
+ }
+}
+rmdir("$root/sys/class/block/dm-0/slaves/sdf") or die $!;
+
+for my $case (
+ [
+ 'FAILED alternate session',
+ sub {
+ put('/sys/class/iscsi_session/session2/state', 'FAILED');
+ $map_users = ['guest uses map'];
+ },
+ ],
+ ['failed dm path', sub { $dm_active = 0; $map_users = ['guest uses map'] }],
+ (
+ map {
+ my $state = $_;
+ [
+ 'alternate path with checker state ' . ($state // 'missing'),
+ sub { ($check_state, $map_users) = ($state, ['guest uses map']) },
+ ]
+ } ('faulty', 'shaky', 'i/o pending', undef)
+ ),
+ ['dm opens', sub { $dm_active = 0; $opens = 1 }],
+ ['direct member user', sub { $direct_users = ['qemu has sda open'] }],
+ ['users probe failure', sub { $fail_probe = 1 }],
+ ['malformed open count', sub { $dm_active = 0; $opens = 'bad' }],
+ ['claim failure', sub { $dm_active = 0; $fail_claim = 1 }],
+) {
+ reset_case();
+ $case->[1]->();
+ ok(logout(), "$case->[0] refused");
+ is(scalar(@logouts), 0, 'no logout on refusal');
+}
+for my $count (1, 2) {
+ reset_case();
+ ($dm_active, $opens) = (0, $count);
+ my $reference = $count == 1 ? 'reference' : 'references';
+ like(logout(), qr/dm-0 has $count open $reference\n$/,
+ "open count $count uses correct grammar");
+ is(scalar(@logouts), 0, 'open references still prevent logout');
+}
+reset_case();
+$dm_active = 0;
+is(logout(), '', 'unused last-path map can be removed');
+is_deeply([sort keys %claims], ['dm-0'], 'map claimed');
+ok(!defined($claims{'dm-0'}), 'claim released after logout');
+
+unlink("$root/sys/class/block/sda/holders/dm-0") or die $!;
+reset_case();
+is(logout(), '', 'unused non-multipath disk can be removed');
+is_deeply([sort keys %claims], ['sda'], 'non-multipath disk claimed');
+reset_case();
+$direct_users = ['sda1 mounted'];
+like(logout(), qr/last usable path.*sda1 mounted/, 'partition user refuses non-multipath logout');
+
+# Offload initiators can run several sessions on one SCSI host; each LUN still belongs only to the
+# session whose target holds it.
+session(3, 9, 'iqn.other-a', '9:0:0:1', 'sdc', '8:32');
+session(4, 9, 'iqn.other-b', '9:0:1:1', 'sdd', '8:48');
+$devices = PVE::Storage::ISCSIPlugin::iscsi_session_device_map(1);
+is_deeply(
+ { map { $_ => [$devices->{$_}->@{qw(session target)}] } qw(sdc sdd) },
+ { sdc => [3, 'iqn.other-a'], sdd => [4, 'iqn.other-b'] },
+ 'sessions sharing a SCSI host keep their own LUNs',
+);
+reset_case();
+$direct_users = ['qemu has sdc open'];
+like(logout(3), qr/last usable path.*qemu has sdc open/, 'user of a shared-host session refuses');
+is_deeply(\@checks, ['sdc'], 'the LUN of the shared-host session was checked');
+is(scalar(@logouts), 0, 'no logout of a shared-host session in use');
+
+reset_case();
+unlink("$root/sys/class/iscsi_session/session4/device") or die $!;
+like(logout(3), qr/session 4 - missing host/, 'a session without its device link fails closed');
+is(scalar(@logouts), 0, 'no logout with an unknown session topology');
+symlink("$root/sys/devices/host9/session4", "$root/sys/class/iscsi_session/session4/device")
+ or die $!;
+
+reset_case();
+rmdir("$root/sys/devices/host9/session4/target9:0:1/9:0:1:1/block/sdd") or die $!;
+like(logout(3), qr/missing block device/, 'incomplete topology of any session fails closed');
+make_path("$root/sys/devices/host9/session4/target9:0:1/9:0:1:1/block/sdd");
+
+unlink("$root/sys/class/iscsi_session/session1/targetname") or die $!;
+reset_case();
+like(logout(), qr/missing target/, 'missing identity fails closed');
+is(scalar(@logouts), 0, 'no logout without identity');
+
+done_testing();
diff --git a/src/test/run_san_api_tests.pl b/src/test/run_san_api_tests.pl
index 00d719a0..cf382a11 100755
--- a/src/test/run_san_api_tests.pl
+++ b/src/test/run_san_api_tests.pl
@@ -43,7 +43,7 @@ subtest 'node SAN read permissions' => sub {
}
}
for my $case (
- ['PVE::API2::ISCSI', qw(target_delete rescan)],
+ ['PVE::API2::ISCSI', qw(session_logout target_delete rescan)],
) {
my ($class, @methods) = @$case;
for my $name (@methods) {
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 28/76] multipath: add a cluster-wide configuration
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (26 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 27/76] api: iscsi: add session logout with last-path protection Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 29/76] iscsi: add an option to manage storage LUNs with multipath Thomas Lamprecht
` (47 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Per-node multipath configuration can drift, leaving some nodes using
individual paths while others use maps. Keep the allow-list, map names,
and daemon settings in the cluster file system instead.
Provide global defaults, per-LUN entries with aliases and node
restrictions, and a verbatim file for hardware-specific settings.
Use find_multipaths strict to map only allow-listed LUNs, WWID-based
names for consistency across nodes, and auto_resize grow_only so maps
follow target-side LUN growth before the PV is enlarged.
Place global settings also supplied by device profiles, such as
no_path_retry, in an overrides section so the profiles cannot supersede
them. Applying the configuration is the generator's responsibility.
Refuse entries whose WWID multipath would read as an option or a
device number, also in a hand-edited file.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Makefile | 2 +
src/PVE/Multipath/ClusterConfig.pm | 64 ++++
src/PVE/Multipath/Config.pm | 451 +++++++++++++++++++++++++++++
src/test/run_multipath_tests.pl | 255 ++++++++++++++++
src/test/run_san_library_tests.pl | 36 +++
5 files changed, 808 insertions(+)
create mode 100644 src/PVE/Multipath/ClusterConfig.pm
create mode 100644 src/PVE/Multipath/Config.pm
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index 91f22282..b92059bd 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -5,6 +5,8 @@ install:
install -D -m 0644 Storage.pm ${DESTDIR}${PERLDIR}/PVE/Storage.pm
install -D -m 0644 Diskmanage.pm ${DESTDIR}${PERLDIR}/PVE/Diskmanage.pm
install -D -m 0644 Multipath.pm ${DESTDIR}${PERLDIR}/PVE/Multipath.pm
+ install -D -m 0644 Multipath/Config.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Config.pm
+ install -D -m 0644 Multipath/ClusterConfig.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/ClusterConfig.pm
install -D -m 0644 CephConfig.pm ${DESTDIR}${PERLDIR}/PVE/CephConfig.pm
install -D -m 0644 GuestImport.pm ${DESTDIR}${PERLDIR}/PVE/GuestImport.pm
make -C Storage install
diff --git a/src/PVE/Multipath/ClusterConfig.pm b/src/PVE/Multipath/ClusterConfig.pm
new file mode 100644
index 00000000..5d5fa765
--- /dev/null
+++ b/src/PVE/Multipath/ClusterConfig.pm
@@ -0,0 +1,64 @@
+package PVE::Multipath::ClusterConfig;
+
+use v5.36;
+
+use Digest::SHA ();
+
+use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_write_file cfs_lock_file);
+
+use PVE::Multipath::Config;
+
+my $FILENAME = 'multipath.cfg';
+my $OVERRIDES_FILENAME = 'multipath-overrides.conf';
+
+cfs_register_file(
+ $FILENAME,
+ sub($filename, $data) { PVE::Multipath::Config->parse_config($filename, $data); },
+ sub($filename, $data) { PVE::Multipath::Config->write_config($filename, $data); },
+);
+
+cfs_register_file(
+ $OVERRIDES_FILENAME,
+ \&PVE::Multipath::Config::parse_overrides,
+ \&PVE::Multipath::Config::write_overrides,
+);
+
+sub read_config() {
+ return cfs_read_file($FILENAME);
+}
+
+sub write_config($cfg) {
+ cfs_write_file($FILENAME, $cfg);
+}
+
+sub read_overrides() {
+ return cfs_read_file($OVERRIDES_FILENAME);
+}
+
+sub write_overrides($text) {
+ cfs_write_file($OVERRIDES_FILENAME, $text);
+}
+
+# Digest over the raw override text, to detect concurrent modifications; the SectionConfig digest
+# of multipath.cfg does not cover this separate file.
+sub overrides_digest($text) {
+ return Digest::SHA::sha1_hex($text // '');
+}
+
+# Preserve exception objects so parameter errors retain their API error type.
+my sub rethrow($err, $errmsg) {
+ die $err if ref($err) || !$errmsg;
+ die "$errmsg - $err";
+}
+
+sub lock_config($code, $errmsg = undef) {
+ cfs_lock_file($FILENAME, undef, $code);
+ rethrow($@, $errmsg) if $@;
+}
+
+sub lock_overrides($code, $errmsg = undef) {
+ cfs_lock_file($OVERRIDES_FILENAME, undef, $code);
+ rethrow($@, $errmsg) if $@;
+}
+
+1;
diff --git a/src/PVE/Multipath/Config.pm b/src/PVE/Multipath/Config.pm
new file mode 100644
index 00000000..4d9901e0
--- /dev/null
+++ b/src/PVE/Multipath/Config.pm
@@ -0,0 +1,451 @@
+package PVE::Multipath::Config;
+
+use v5.36;
+
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::SectionConfig;
+use PVE::Tools;
+
+use base qw(PVE::SectionConfig);
+
+# Strict discovery leaves unrelated and boot disks alone; WWID names avoid node-local bindings.
+# Grow-only resizing follows target expansion without shrinking a map under its consumers.
+my $MANAGED_DEFAULTS = {
+ 'find-multipaths' => 'strict',
+ 'user-friendly-names' => 'no',
+ 'polling-interval' => 5,
+ 'auto-resize' => 'grow_only',
+};
+
+sub managed_defaults() { return { $MANAGED_DEFAULTS->%* }; }
+
+# multipath reads an argument starting with '-' as an option and one starting with
+# '<major>:<minor>' as a device number, so a WWID must not start with either.
+our $WWID_PATTERN = '(?!-|\d+:-?\d)[a-zA-Z0-9._:-]+';
+our $WWID_RE = qr/^(?:$WWID_PATTERN)\z/;
+
+my $shared_knobs = {
+ 'no-path-retry' => {
+ type => 'string',
+ pattern => '(?:queue|fail|\d+)',
+ typetext => 'queue|fail|<count>',
+ description =>
+ "How to react when all paths are down: keep queuing, fail at once, or retry"
+ . " for the given number of polling intervals.",
+ optional => 1,
+ },
+ 'path-grouping-policy' => {
+ type => 'string',
+ enum =>
+ [qw(failover multibus group_by_serial group_by_prio group_by_node_name group_by_tpg)],
+ description => "How paths are grouped into priority groups. 'group_by_tpg' groups by"
+ . " the ALUA target port group and needs the alua prioritizer, which the hardware"
+ . " table of multipath-tools selects for arrays that support it.",
+ optional => 1,
+ },
+ failback => {
+ type => 'string',
+ pattern => '(?:manual|immediate|followover|\d+)',
+ typetext => 'manual|immediate|followover|<seconds>',
+ description => "When to fail back to a restored higher-priority path group.",
+ optional => 1,
+ },
+ 'path-selector' => {
+ type => 'string',
+ pattern =>
+ '(?:round-robin|queue-length|service-time|historical-service-time|io-affinity) \d+',
+ typetext => '<selector> <number>',
+ description => "Path selector algorithm used within a priority group, for example"
+ . " 'service-time 0'.",
+ optional => 1,
+ },
+};
+
+my $defaults_only_knobs = {
+ 'find-multipaths' => {
+ type => 'string',
+ enum => [qw(yes no strict greedy smart)],
+ default => 'strict',
+ description => "Which devices multipathd assembles into a map. 'strict' only uses WWIDs"
+ . " in the node's allow-list, including those added for multipath-enabled storages.",
+ optional => 1,
+ },
+ 'user-friendly-names' => {
+ type => 'string',
+ enum => [qw(yes no)],
+ default => 'no',
+ description => "Whether to use node-local mpathN names. Keep 'no' for stable WWID-based"
+ . " names across the cluster.",
+ optional => 1,
+ },
+ 'polling-interval' => {
+ type => 'integer',
+ minimum => 1,
+ default => 5,
+ description => "Interval between path checks, in seconds.",
+ optional => 1,
+ },
+ 'auto-resize' => {
+ type => 'string',
+ enum => [qw(never grow_only grow_shrink)],
+ default => 'grow_only',
+ description => "Whether multipathd resizes a map once all its paths report a new size."
+ . " 'grow_only' follows a LUN grown on the target, 'never' leaves it to"
+ . " 'multipathd resize map'.",
+ optional => 1,
+ },
+};
+
+my $wwid_only_knobs = {
+ alias => {
+ type => 'string',
+ pattern => '[a-zA-Z0-9][a-zA-Z0-9._-]*',
+ maxLength => 64,
+ description => "Human-readable name that multipathd uses for this WWID's map.",
+ optional => 1,
+ },
+ 'rr-min-io-rq' => {
+ type => 'integer',
+ minimum => 1,
+ description => "Number of I/O requests to send through a path before switching to"
+ . " another path, for request-based multipath devices.",
+ optional => 1,
+ },
+ 'rr-weight' => {
+ type => 'string',
+ enum => [qw(priorities uniform)],
+ description => "Whether to weight paths by priority when balancing I/O.",
+ optional => 1,
+ },
+};
+
+# Per-LUN settings that are not multipathd knobs and stay out of the rendered configuration.
+my $wwid_only_props = {
+ nodes => get_standard_option(
+ 'pve-node-list',
+ {
+ description => "The nodes that see this LUN and are expected to assemble its map."
+ . " Other nodes leave it off their allow-list. Unset means every node.",
+ optional => 1,
+ },
+ ),
+};
+
+my $defaultData = {
+ propertyList => {
+ type => { description => "Section type ('defaults' or 'wwid')." },
+ id => {
+ type => 'string',
+ description =>
+ "Section ID: the literal 'defaults', or a LUN WWID for 'wwid' sections.",
+ pattern => $WWID_PATTERN,
+ maxLength => 128,
+ },
+ $shared_knobs->%*,
+ $defaults_only_knobs->%*,
+ $wwid_only_knobs->%*,
+ $wwid_only_props->%*,
+ },
+};
+
+sub private($class) { return $defaultData; }
+
+# The file can be edited by hand; skip WWID sections whose ID multipath would misread.
+sub parse_section_header($class, $line) {
+ my ($type, $id, $errmsg, $config) = $class->SUPER::parse_section_header($line);
+ return undef if !defined($type);
+ $errmsg //= "invalid WWID '$id'" if $type eq 'wwid' && $id !~ $WWID_RE;
+ return ($type, $id, $errmsg, $config);
+}
+
+sub decode_value($class, $type, $key, $value) {
+ if ($key eq 'nodes') {
+ my $res = {};
+ for my $node (PVE::Tools::split_list($value)) {
+ $res->{$node} = 1 if PVE::JSONSchema::pve_verify_node_name($node);
+ }
+ return $res;
+ }
+ return $value;
+}
+
+sub encode_value($class, $type, $key, $value) {
+ return join(',', sort keys %$value) if $key eq 'nodes';
+ return $value;
+}
+
+package PVE::Multipath::Config::Defaults;
+
+use base qw(PVE::Multipath::Config);
+
+sub type($class) { return 'defaults'; }
+
+sub options($class) {
+ return {
+ 'find-multipaths' => { optional => 1 },
+ 'user-friendly-names' => { optional => 1 },
+ 'polling-interval' => { optional => 1 },
+ 'auto-resize' => { optional => 1 },
+ 'no-path-retry' => { optional => 1 },
+ 'path-grouping-policy' => { optional => 1 },
+ failback => { optional => 1 },
+ 'path-selector' => { optional => 1 },
+ };
+}
+
+__PACKAGE__->register();
+
+package PVE::Multipath::Config::Wwid;
+
+use base qw(PVE::Multipath::Config);
+
+sub type($class) { return 'wwid'; }
+
+sub options($class) {
+ return {
+ alias => { optional => 1 },
+ nodes => { optional => 1 },
+ 'no-path-retry' => { optional => 1 },
+ 'path-grouping-policy' => { optional => 1 },
+ failback => { optional => 1 },
+ 'path-selector' => { optional => 1 },
+ 'rr-min-io-rq' => { optional => 1 },
+ 'rr-weight' => { optional => 1 },
+ };
+}
+
+__PACKAGE__->register();
+
+package PVE::Multipath::Config;
+
+__PACKAGE__->init();
+
+# multipaths is generated from WWID sections and must not also occur in overrides.
+my $OVERRIDE_KEYWORDS =
+ { devices => 1, overrides => 1, defaults => 1, blacklist => 1, blacklist_exceptions => 1 };
+
+# This is a structural guard, not a complete multipath.conf parser.
+sub check_overrides($text) {
+ return if !defined($text) || $text !~ /\S/;
+
+ # walk every block opener and brace in order, so a section keyword after a closing brace on
+ # the same line is still checked at its real depth
+ my $depth = 0;
+ for my $line (split(/\n/, $text)) {
+ next if $line =~ /^\s*#/;
+ while ($line =~ /(\w+)\s*\{|([{}])/g) {
+ if (defined($1)) {
+ my $kw = $1;
+ die "multipath overrides: 'multipaths' sections are generated from the multipath"
+ . " entries, do not set them here\n"
+ if $kw eq 'multipaths';
+ die "multipath overrides: unknown top-level section '$kw'\n"
+ if $depth == 0 && !$OVERRIDE_KEYWORDS->{$kw};
+ $depth++;
+ } elsif ($2 eq '{') {
+ $depth++;
+ } else {
+ die "multipath overrides: unbalanced braces\n" if $depth == 0;
+ $depth--;
+ }
+ }
+ }
+ die "multipath overrides: unbalanced braces\n" if $depth != 0;
+
+ return;
+}
+
+sub parse_overrides($filename, $raw) {
+ return $raw // '';
+}
+
+sub write_overrides($filename, $text) {
+ $text //= '';
+ $text =~ s/\s+$//;
+ return length($text) ? "$text\n" : '';
+}
+
+my $MANAGED_HEADER =
+ "# This file is managed by Proxmox VE - do not edit by hand.\n"
+ . "# Hardware-/node-specific overrides belong in the override config.\n";
+
+# multipath.conf requires quotes around whitespace; schemas exclude embedded quotes.
+my sub render_value($value) {
+ return $value =~ /\s/ ? "\"$value\"" : $value;
+}
+
+my sub render_section($name, $kv) {
+ my $out = "$name {\n";
+ for my $key (sort keys $kv->%*) {
+ (my $keyword = $key) =~ tr/-/_/;
+ $out .= "\t$keyword " . render_value($kv->{$key}) . "\n";
+ }
+ $out .= "}\n";
+ return $out;
+}
+
+# Device profiles override defaults, so these global settings need an overrides section.
+# Per-LUN settings still take precedence. Admin overrides use a separate file to avoid duplicate
+# section keywords in one file.
+my $OVERRIDE_KNOBS = {
+ 'no-path-retry' => 1,
+ 'path-grouping-policy' => 1,
+ failback => 1,
+ 'path-selector' => 1,
+};
+
+sub generate_managed_conf($defaults = undef) {
+ $defaults //= managed_defaults();
+
+ my ($global, $overrides) = ({}, {});
+ ($OVERRIDE_KNOBS->{$_} ? $overrides : $global)->{$_} = $defaults->{$_} for keys %$defaults;
+ # the path grouping that multipathd detects for ALUA arrays overrides even the policy of a
+ # single LUN, so turn the detection off when a policy is configured
+ $overrides->{'detect-pgpolicy'} = 'no' if defined($overrides->{'path-grouping-policy'});
+
+ my $conf = $MANAGED_HEADER . "\n" . render_section('defaults', $global);
+ $conf .= "\n" . render_section('overrides', $overrides) if %$overrides;
+ return $conf;
+}
+
+# The WWID allow-list file (/etc/multipath/wwids) holds one '/<wwid>/' per line.
+sub parse_wwids($text) {
+ my $wwids = [];
+ for my $line (split(/\n/, $text // '')) {
+ next if $line =~ /^\s*#/;
+ next if $line =~ /^\s*$/;
+ if ($line =~ m{^/(.+)/\s*$}) {
+ push $wwids->@*, $1;
+ }
+ }
+ return $wwids;
+}
+
+sub parse_managed_wwids($text) {
+ my $record = {};
+ for my $line (split(/\n/, $text // '')) {
+ next if $line =~ /^\s*#/;
+ next if $line !~ m/^\s*(\S+)\s+(\S+)\s*$/;
+ my ($wwid, $sources) = ($1, $2);
+ my @sources = grep { m/^(?:config|storage:.+)$/ } split(/,/, $sources);
+ $record->{$wwid} = [sort @sources] if scalar(@sources);
+ }
+ return $record;
+}
+
+sub format_managed_wwids($record) {
+ my $out = "# Multipath WWIDs added by Proxmox VE, with the reasons each was added\n";
+ $out .= "$_ " . join(',', sort $record->{$_}->@*) . "\n" for sort keys %$record;
+ return $out;
+}
+
+sub build_multipaths_block($wwid_opts) {
+ my @entries = grep { %{ $wwid_opts->{$_} } } sort keys %$wwid_opts;
+ return '' if !@entries;
+
+ my $out = "multipaths {\n";
+ for my $wwid (@entries) {
+ my $opts = $wwid_opts->{$wwid};
+ $out .= "\tmultipath {\n";
+ $out .= "\t\twwid $wwid\n";
+ for my $key (sort keys %$opts) {
+ (my $keyword = $key) =~ tr/-/_/;
+ $out .= "\t\t$keyword " . render_value($opts->{$key}) . "\n";
+ }
+ $out .= "\t}\n";
+ }
+ $out .= "}\n";
+ return $out;
+}
+
+sub apply_section_update($section, $param, $delete, $settable) {
+ my @delete = PVE::Tools::split_list($delete // '');
+ for my $key (@delete) {
+ die "'$key' is not a settable property\n" if !$settable->{$key};
+ die "cannot set and delete property '$key' at the same time\n" if defined($param->{$key});
+ }
+ for my $key (sort keys %$param) {
+ die "'$key' is not a settable property\n" if !$settable->{$key};
+ $section->{$key} = __PACKAGE__->decode_value($section->{type}, $key, $param->{$key});
+ }
+ delete $section->{$_} for @delete;
+ return;
+}
+
+# Omitted update parameters must remain unchanged, not inherit schema defaults.
+my sub api_schema($props) {
+ my $res = {};
+ for my $key (keys %$props) {
+ $res->{$key} = { $props->{$key}->%* };
+ delete $res->{$key}->{default};
+ }
+ return $res;
+}
+
+sub defaults_api_schema() {
+ return api_schema({ $shared_knobs->%*, $defaults_only_knobs->%* });
+}
+
+sub wwid_api_schema() {
+ return api_schema({ $shared_knobs->%*, $wwid_only_knobs->%*, $wwid_only_props->%* });
+}
+
+sub effective_defaults($cfg) {
+ my $defaults = managed_defaults();
+ if (my $section = $cfg->{ids}->{defaults}) {
+ $defaults->{$_} = $section->{$_} for grep { $_ ne 'type' } keys %$section;
+ }
+ return $defaults;
+}
+
+sub wwid_list($cfg) {
+ return [sort grep { ($cfg->{ids}->{$_}->{type} // '') eq 'wwid' } keys $cfg->{ids}->%*];
+}
+
+sub wwids_for_node($cfg, $node) {
+ return [
+ grep {
+ my $nodes = $cfg->{ids}->{$_}->{nodes};
+ !$nodes || !%$nodes || $nodes->{$node}
+ } wwid_list($cfg)->@*
+ ];
+}
+
+# The per-WWID sections as the API reports them, { wwid => { alias?, nodes?, knob => value } },
+# with the node restriction as a list.
+sub wwid_sections($cfg) {
+ my $res = {};
+ for my $wwid (wwid_list($cfg)->@*) {
+ my $section = $cfg->{ids}->{$wwid};
+ my $entry = { map { $_ => $section->{$_} } grep { $_ ne 'type' } keys %$section };
+ $entry->{nodes} = __PACKAGE__->encode_value('wwid', 'nodes', $entry->{nodes})
+ if $entry->{nodes};
+ $res->{$wwid} = $entry;
+ }
+ return $res;
+}
+
+# { wwid => alias } for the WWIDs that have one.
+sub aliases($cfg) {
+ my $res = {};
+ for my $wwid (keys $cfg->{ids}->%*) {
+ my $section = $cfg->{ids}->{$wwid};
+ next if ($section->{type} // '') ne 'wwid';
+ $res->{$wwid} = $section->{alias} if defined($section->{alias});
+ }
+ return $res;
+}
+
+sub wwid_opts($cfg) {
+ my $res = {};
+ for my $wwid (wwid_list($cfg)->@*) {
+ my $section = $cfg->{ids}->{$wwid};
+ $res->{$wwid} = {
+ map { $_ => $section->{$_} }
+ grep { $_ ne 'type' && !$wwid_only_props->{$_} } keys %$section
+ };
+ }
+ return $res;
+}
+
+1;
diff --git a/src/test/run_multipath_tests.pl b/src/test/run_multipath_tests.pl
index 484e9348..0e247284 100755
--- a/src/test/run_multipath_tests.pl
+++ b/src/test/run_multipath_tests.pl
@@ -7,6 +7,7 @@ use Test::More;
use lib ('.', '..');
use PVE::Multipath;
+use PVE::Multipath::Config;
# A recorded `multipathd show maps json` reply with three maps exercising each
# health state: an all-active map, a partially-failed map and an all-failed map.
@@ -130,4 +131,258 @@ is_deeply($empty, [], 'no maps parses to empty list');
eval { PVE::Multipath::parse_maps_json('not json') };
ok($@ =~ m/could not parse multipathd maps JSON/, 'invalid JSON raises a clear error');
+# --- config generation / WWID allow-list ---
+my $conf = PVE::Multipath::Config::generate_managed_conf();
+like($conf, qr/managed by Proxmox VE/, 'managed conf carries the managed header');
+like($conf, qr/user_friendly_names no/, 'baseline sets user_friendly_names no');
+like($conf, qr/find_multipaths strict/, 'baseline opts in explicitly via find_multipaths strict');
+is(
+ scalar(() = $conf =~ /^defaults \{/mg),
+ 1,
+ 'baseline has exactly one defaults block (a second would be a duplicate-keyword error)',
+);
+
+like($conf, qr/^\tauto_resize grow_only$/m, 'baseline lets maps grow with their LUN');
+unlike($conf, qr/^overrides \{/m, 'baseline overrides nothing of the device profiles');
+
+# multipath.conf only allows whitespace inside double-quoted values
+like(
+ PVE::Multipath::Config::generate_managed_conf({
+ PVE::Multipath::Config::managed_defaults()->%*, 'path-selector' => 'service-time 0',
+ }),
+ qr/^\tpath_selector "service-time 0"$/m,
+ 'a value containing whitespace renders double-quoted, single words stay bare',
+);
+
+# what the device profiles set too goes into overrides, or the profiles would beat it
+my $with_policy = PVE::Multipath::Config::generate_managed_conf({
+ PVE::Multipath::Config::managed_defaults()->%*,
+ 'no-path-retry' => 'fail',
+ 'path-grouping-policy' => 'multibus',
+});
+my ($defaults_block) = $with_policy =~ m/^(defaults \{.*?^\})/ms;
+my ($overrides_block) = $with_policy =~ m/^(overrides \{.*?^\})/ms;
+unlike($defaults_block, qr/no_path_retry|path_grouping_policy/, 'device knobs: not in defaults');
+like($overrides_block, qr/^\tno_path_retry fail$/m, 'device knobs: in overrides');
+like(
+ $overrides_block,
+ qr/^\tdetect_pgpolicy no$/m,
+ 'device knobs: a grouping policy turns off the detected grouping',
+);
+
+my $wwids = PVE::Multipath::Config::parse_wwids("# Multipath wwids\n/3600abc/\n/3600def/\n");
+is_deeply($wwids, ['3600abc', '3600def'], 'parse_wwids extracts the wwids');
+
+# --- cluster config (pmxcfs source of truth): SectionConfig parse/write ---
+my $raw =
+ "defaults: defaults\n\tfind-multipaths strict\n\tno-path-retry queue\n\n"
+ . "wwid: 3600def\n\talias san-b-lun0\n\n"
+ . "wwid: 3600abc\n\talias san-a-lun0\n\tno-path-retry 18\n";
+my $cc = PVE::Multipath::Config->parse_config('multipath.cfg', $raw);
+is_deeply(
+ PVE::Multipath::Config::wwid_list($cc),
+ ['3600abc', '3600def'],
+ 'wwid sections become the allow-list (sorted)',
+);
+is_deeply(
+ PVE::Multipath::Config::aliases($cc),
+ { '3600abc' => 'san-a-lun0', '3600def' => 'san-b-lun0' },
+ 'aliases read from the wwid sections',
+);
+is(
+ PVE::Multipath::Config::effective_defaults($cc)->{'no-path-retry'},
+ 'queue',
+ 'defaults section knob is read',
+);
+is(
+ PVE::Multipath::Config::effective_defaults($cc)->{'user-friendly-names'},
+ 'no',
+ 'an unset defaults knob falls back to the managed default',
+);
+
+my $written = PVE::Multipath::Config->write_config('multipath.cfg', $cc);
+my $cc2 = PVE::Multipath::Config->parse_config('multipath.cfg', $written);
+is_deeply(
+ PVE::Multipath::Config::wwid_list($cc2),
+ ['3600abc', '3600def'],
+ 'wwids survive the SectionConfig round-trip',
+);
+is_deeply(
+ PVE::Multipath::Config::aliases($cc2),
+ PVE::Multipath::Config::aliases($cc),
+ 'aliases survive the round-trip',
+);
+is($cc2->{ids}->{'3600abc'}->{'no-path-retry'}, 18, 'a per-WWID knob survives the round-trip');
+
+is_deeply(
+ PVE::Multipath::Config::wwid_list(PVE::Multipath::Config->parse_config('multipath.cfg', '')),
+ [],
+ 'an empty cluster config has no WWIDs',
+);
+{
+ my @warnings;
+ local $SIG{__WARN__} = sub { push @warnings, @_ };
+ my $cfg = PVE::Multipath::Config->parse_config(
+ 'multipath.cfg',
+ "wwid: -W\n\nwwid: 8:0\n\nwwid: 3600abc\n\nwwid: 36001405:ab\n",
+ );
+ is_deeply(
+ PVE::Multipath::Config::wwid_list($cfg),
+ ['36001405:ab', '3600abc'],
+ 'a WWID that multipath would read as an option or a device number is no section',
+ );
+ is(scalar(@warnings), 2, 'both rejected sections are reported');
+}
+
+# a node restriction stays out of the rendered multipathd settings and filters per node
+my $restricted = PVE::Multipath::Config->parse_config(
+ 'multipath.cfg',
+ "wwid: 3600abc\n\talias fc-lun0\n\tnodes n2,n1\n\nwwid: 3600def\n",
+);
+is_deeply(
+ $restricted->{ids}->{'3600abc'}->{nodes},
+ { n1 => 1, n2 => 1 },
+ 'the node list is decoded into a set',
+);
+is_deeply(
+ PVE::Multipath::Config::wwid_opts($restricted)->{'3600abc'},
+ { alias => 'fc-lun0' },
+ 'the rendered per-WWID options omit the node restriction',
+);
+is_deeply(
+ PVE::Multipath::Config::wwid_sections($restricted)->{'3600abc'},
+ { alias => 'fc-lun0', nodes => 'n1,n2' },
+ 'the API view lists the node restriction as a sorted list',
+);
+is_deeply(
+ PVE::Multipath::Config::wwids_for_node($restricted, 'n1'),
+ ['3600abc', '3600def'],
+ 'a listed node carries the restricted and the unrestricted WWID',
+);
+is_deeply(
+ PVE::Multipath::Config::wwids_for_node($restricted, 'n3'),
+ ['3600def'],
+ 'an unlisted node carries only the unrestricted WWID',
+);
+like(
+ PVE::Multipath::Config->write_config('multipath.cfg', $restricted),
+ qr/^\tnodes n1,n2$/m,
+ 'the node list is written back as a list',
+);
+
+# --- API updates land in the config decoded, guarded to the settable properties ---
+{
+ my $settable = PVE::Multipath::Config::wwid_api_schema();
+ my $section = { type => 'wwid', alias => 'old', 'no-path-retry' => 'queue' };
+ PVE::Multipath::Config::apply_section_update(
+ $section,
+ { nodes => 'n2,n1', alias => 'new' },
+ 'no-path-retry',
+ $settable,
+ );
+ is_deeply(
+ $section,
+ { type => 'wwid', alias => 'new', nodes => { n1 => 1, n2 => 1 } },
+ 'a node list is decoded into the set the config layer works with, deletes are applied',
+ );
+ like(
+ PVE::Multipath::Config->write_config('multipath.cfg', { ids => { w => $section } }),
+ qr/^\tnodes n1,n2$/m,
+ 'the updated section writes back as a config',
+ );
+
+ eval {
+ PVE::Multipath::Config::apply_section_update(
+ $section,
+ { type => 'defaults' },
+ undef,
+ $settable,
+ );
+ };
+ like($@, qr/not a settable property/, 'the section type cannot be set through an update');
+ eval { PVE::Multipath::Config::apply_section_update($section, {}, 'type', $settable) };
+ like($@, qr/not a settable property/, 'the section type cannot be deleted through an update');
+ eval {
+ PVE::Multipath::Config::apply_section_update(
+ $section,
+ { alias => 'x' },
+ 'alias',
+ $settable,
+ );
+ };
+ like($@, qr/set and delete/, 'a property is never set and deleted at once');
+}
+
+# --- multipaths{} block (alias plus per-WWID knobs) ---
+my $block = PVE::Multipath::Config::build_multipaths_block({
+ '3600def' => { alias => 'san-b-lun0' },
+ '3600abc' => { alias => 'san-a-lun0', 'no-path-retry' => 18 },
+ '3600nul' => {},
+});
+like($block, qr/^multipaths \{/m, 'block opens with multipaths {');
+is(
+ scalar(() = $block =~ /^\tmultipath \{/mg),
+ 2,
+ 'one multipath{} per WWID that has an alias or a knob (the empty WWID is skipped)',
+);
+like(
+ $block,
+ qr/wwid 3600abc.*?alias san-a-lun0.*?no_path_retry 18/s,
+ 'block carries the alias and the per-WWID knob',
+);
+my $abc_pos = index($block, 'wwid 3600abc');
+my $def_pos = index($block, 'wwid 3600def');
+ok($abc_pos < $def_pos, 'block emits entries in WWID-sorted order');
+is(PVE::Multipath::Config::build_multipaths_block({}), '', 'no WWIDs render to the empty string');
+like(
+ PVE::Multipath::Config::build_multipaths_block({
+ '3600abc' => { 'path-selector' => 'round-robin 0' },
+ }),
+ qr/^\t\tpath_selector "round-robin 0"$/m,
+ 'a per-WWID value containing whitespace renders double-quoted',
+);
+
+# --- ownership record file format ---
+is_deeply(
+ PVE::Multipath::Config::parse_managed_wwids(
+ "# header\n3600abc config\n3600def storage:san1,config\nbogus line here\n/3600old/\n"
+ . "3600bad nonsense\n"
+ ),
+ { '3600abc' => ['config'], '3600def' => ['config', 'storage:san1'] },
+ 'parse_managed_wwids reads wwid and reasons, ignores what does not fit',
+);
+is(
+ PVE::Multipath::Config::format_managed_wwids({
+ '3600def' => ['storage:san1', 'config'],
+ '3600abc' => ['config'],
+ }),
+ "# Multipath WWIDs added by Proxmox VE, with the reasons each was added\n"
+ . "3600abc config\n3600def config,storage:san1\n",
+ 'format_managed_wwids sorts and round-trips',
+);
+# --- override guard ---
+eval { PVE::Multipath::Config::check_overrides("devices {\n\tdevice {\n\t\tvendor X\n\t}\n}\n") };
+is($@, '', 'a well-formed devices{} block passes the guard');
+eval { PVE::Multipath::Config::check_overrides("multipaths {\n}\n") };
+like($@, qr/generated from the multipath/, 'a multipaths{} block is rejected, it is generated');
+eval { PVE::Multipath::Config::check_overrides("devices {\n") };
+like($@, qr/unbalanced braces/, 'unbalanced braces are rejected');
+eval { PVE::Multipath::Config::check_overrides("frobnicate {\n}\n") };
+like($@, qr/unknown top-level section/, 'an unknown top-level section is rejected');
+eval { PVE::Multipath::Config::check_overrides("blacklist { } multipaths {\n}\n") };
+like(
+ $@,
+ qr/generated from the multipath/,
+ 'a multipaths{} block cannot hide behind a same-line close',
+);
+eval { PVE::Multipath::Config::check_overrides("blacklist { } frobnicate {\n}\n") };
+like($@, qr/unknown top-level section/, 'an unknown section cannot hide behind a same-line close');
+eval { PVE::Multipath::Config::check_overrides("}\ndevices {\n") };
+like($@, qr/unbalanced braces/, 'a closing brace before any open is rejected');
+is(
+ PVE::Multipath::Config::write_overrides('x', "text \n\n"),
+ "text\n",
+ 'the overrides writer trims trailing whitespace',
+);
+
done_testing();
diff --git a/src/test/run_san_library_tests.pl b/src/test/run_san_library_tests.pl
index 6d509979..84a8539c 100755
--- a/src/test/run_san_library_tests.pl
+++ b/src/test/run_san_library_tests.pl
@@ -11,6 +11,42 @@ use lib ('.', '..');
use PVE::Cluster;
use PVE::Storage::StatusCommand;
+my %registered;
+my $cluster = Test::MockModule->new('PVE::Cluster');
+$cluster->mock(cfs_register_file => sub { $registered{ $_[0] } = [@_[1, 2]] });
+require PVE::Multipath::ClusterConfig;
+
+subtest 'CFS callbacks preserve filenames, data and method receivers' => sub {
+ my ($parse, $write) = $registered{'multipath.cfg'}->@*;
+ my $cfg = $parse->('multipath.cfg', undef);
+ is_deeply($cfg->{ids}, {}, 'an absent configuration parses as empty');
+ $cfg = $parse->('multipath.cfg', "wwid: 3600abc\n\talias san-lun\n\tnodes n2,n1\n");
+ is_deeply($cfg->{ids}->{'3600abc'}->{nodes}, { n1 => 1, n2 => 1 }, 'nodes are decoded');
+ my $again = $parse->('multipath.cfg', $write->('multipath.cfg', $cfg));
+ is_deeply($again->{ids}, $cfg->{ids}, 'sections survive the registered writer and parser');
+ ($parse, $write) = $registered{'multipath-overrides.conf'}->@*;
+ is($parse->('multipath-overrides.conf', undef), '', 'absent overrides parse as empty');
+ is($write->('multipath-overrides.conf', undef), '', 'undefined overrides write as empty');
+ is($write->('multipath-overrides.conf', "devices { } \n"), "devices { }\n",
+ 'writer trims');
+};
+
+subtest 'omitted optional parameters retain explicit undef behavior' => sub {
+ my $config = Test::MockModule->new('PVE::Multipath::ClusterConfig');
+ $config->mock(cfs_lock_file => sub { $@ = "lock unavailable\n" });
+ for my $lock (
+ \&PVE::Multipath::ClusterConfig::lock_config,
+ \&PVE::Multipath::ClusterConfig::lock_overrides,
+ ) {
+ for my $args ([], [undef]) {
+ eval {
+ $lock->(sub { die "unexpected callback\n" }, @$args);
+ };
+ is($@, "lock unavailable\n", 'an omitted error prefix leaves the error intact');
+ }
+ }
+};
+
subtest 'shared status retains callback context and named command parameters' => sub {
for my $cache (undef, { status_only => 1 }) {
my $code = sub { return wantarray ? ('a', 'b') : 'scalar' };
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 29/76] iscsi: add an option to manage storage LUNs with multipath
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (27 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 28/76] multipath: add a cluster-wide configuration Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 30/76] lvm: activate a missing base volume before using the volume group Thomas Lamprecht
` (46 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Let iSCSI storages request multipath maps so LVM uses a stable LUN
identity rather than an arbitrary path. Require new LVM base volumes
to name the map and wait for it before initialization. Existing
path-based base volumes also wait for their map. Refuse disabling
multipath while LVM depends on one.
Resolve identities from maps and daemon inventory, rejecting changed
paths or conflicting evidence. Preserve unresolved identities so callers
do not mistake an unchecked LUN for a device without a SAN identity.
Older nodes drop unknown storage options when rewriting configuration.
Require support reports from every cluster node and multipath tools on
the storage nodes before accepting the option.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Makefile | 2 +
src/PVE/Multipath.pm | 45 ++
src/PVE/Multipath/Identity.pm | 221 ++++++++++
src/PVE/Multipath/Status.pm | 70 ++++
src/PVE/Storage.pm | 31 ++
src/PVE/Storage/ISCSIPlugin.pm | 147 ++++++-
src/PVE/Storage/LVMPlugin.pm | 8 +-
src/test/Makefile | 17 +-
src/test/run_iscsi_activation_tests.pl | 171 ++++++++
src/test/run_lun_identity_tests.pl | 557 +++++++++++++++++++++++++
src/test/run_multipath_api_tests.pl | 92 ++++
11 files changed, 1353 insertions(+), 8 deletions(-)
create mode 100644 src/PVE/Multipath/Identity.pm
create mode 100644 src/PVE/Multipath/Status.pm
create mode 100755 src/test/run_iscsi_activation_tests.pl
create mode 100755 src/test/run_lun_identity_tests.pl
create mode 100755 src/test/run_multipath_api_tests.pl
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index b92059bd..29d0146d 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -7,6 +7,8 @@ install:
install -D -m 0644 Multipath.pm ${DESTDIR}${PERLDIR}/PVE/Multipath.pm
install -D -m 0644 Multipath/Config.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Config.pm
install -D -m 0644 Multipath/ClusterConfig.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/ClusterConfig.pm
+ install -D -m 0644 Multipath/Identity.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Identity.pm
+ install -D -m 0644 Multipath/Status.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Status.pm
install -D -m 0644 CephConfig.pm ${DESTDIR}${PERLDIR}/PVE/CephConfig.pm
install -D -m 0644 GuestImport.pm ${DESTDIR}${PERLDIR}/PVE/GuestImport.pm
make -C Storage install
diff --git a/src/PVE/Multipath.pm b/src/PVE/Multipath.pm
index 69589ac3..e2f7ea40 100644
--- a/src/PVE/Multipath.pm
+++ b/src/PVE/Multipath.pm
@@ -2,6 +2,8 @@ package PVE::Multipath;
use v5.36;
+use JSON;
+
use PVE::Tools qw(run_command file_read_firstline);
use PVE::RS::Multipath;
@@ -14,6 +16,9 @@ my $QUERY_TIMEOUT = 10;
my $TOOL_TIMEOUT = 30;
my $RECONFIGURE_TIMEOUT = 120;
+# Allow multipathd's default four-second socket timeout during startup or reconfiguration.
+my $PATH_QUERY_TIMEOUT = 5;
+
my $supported;
# Cached only while true: a long-lived daemon like pvestatd must notice a later multipath-tools
@@ -64,6 +69,46 @@ sub parse_maps_json($json) {
return PVE::RS::Multipath::parse_maps_json($json);
}
+my $PATH_FORMAT = '%d|%D|%i|%I|%n|%w';
+
+# Daemon placeholders are not identity evidence; callers must validate them. A non-list reply
+# means the daemon could not answer, unlike an empty list.
+sub parse_path_identities($text) {
+ my @lines = grep { length } split(/\n/, $text // '');
+ # the reply to a list without entries
+ return {} if scalar(@lines) == 1 && $lines[0] eq 'ok';
+
+ my ($res, $count) = ({}, {});
+ for my $line (@lines) {
+ my ($dev, $devt, $hctl, $init, $target, $wwid) = split(/\|/, $line, 6);
+ return undef if !defined($wwid);
+ return undef
+ if $dev !~ m/^\S+$/
+ || $devt !~ m/^(?:#:#|\d+:\d+)$/
+ || $hctl !~ m/^(?:#:#:#:#|\d+:\d+:\d+:\d+)$/
+ || $init !~ m/^[a-z-]+$/;
+ next if $dev eq '-';
+ $count->{$dev}++;
+ $res->{$dev} =
+ { devt => $devt, hctl => $hctl, init => $init, target => $target, wwid => $wwid };
+ }
+ delete $res->{$_} for grep { $count->{$_} > 1 } keys %$count;
+ return $res;
+}
+
+sub query_path_identities() {
+ assert_supported();
+
+ my $output = '';
+ run_command(
+ [$MULTIPATHD, 'show', 'paths', 'raw', 'format', $PATH_FORMAT],
+ timeout => $PATH_QUERY_TIMEOUT,
+ outfunc => sub($line) { $output .= "$line\n"; },
+ errfunc => sub($line) { },
+ );
+ return parse_path_identities($output) // die "multipathd returned an invalid path list\n";
+}
+
# Expected paths include missing paths that the daemon no longer counts.
sub derive_health($paths_active, $paths_total, $paths_expected = undef) {
return PVE::RS::Multipath::derive_health($paths_active, $paths_total, $paths_expected);
diff --git a/src/PVE/Multipath/Identity.pm b/src/PVE/Multipath/Identity.pm
new file mode 100644
index 00000000..2a8d3d20
--- /dev/null
+++ b/src/PVE/Multipath/Identity.pm
@@ -0,0 +1,221 @@
+package PVE::Multipath::Identity;
+
+# Use multipathd's configured identity, not a guessed sysfs identifier. Maps pin their paths;
+# unmapped names can be reused, so daemon entries must match dev_t, H:C:T:L and target, plus
+# the cached VPD NAA designator when applicable. Only iSCSI has grouping evidence before resolution.
+
+use v5.36;
+
+use Cwd qw(abs_path);
+use File::Basename qw(basename);
+
+use PVE::Multipath;
+use PVE::Multipath::Config;
+use PVE::Tools qw(dir_glob_foreach dir_glob_regex file_get_contents file_read_firstline);
+
+our $SYSFS = '/sys';
+
+my sub sysfs_line($path) {
+ return file_read_firstline("$SYSFS/$path");
+}
+
+# The designators of a VPD page 0x83 as [code set, association, type, value], undef if the page is
+# malformed.
+sub vpd_designators($page) {
+ return undef if !defined($page) || length($page) < 4;
+ my ($code, $length) = unpack('x C n', $page);
+ my $end = 4 + $length;
+ return undef if $code != 0x83 || $end > length($page);
+ my ($pos, @designators) = (4);
+ while ($pos < $end) {
+ return undef if $pos + 4 > $end;
+ my ($set, $kind, $size) = unpack("x$pos C C x C", $page);
+ my $next = $pos + 4 + $size;
+ return undef if $next > $end;
+ push @designators,
+ [$set & 0x0f, ($kind & 0x30) >> 4, $kind & 0x0f, substr($page, $pos + 4, $size)];
+ $pos = $next;
+ }
+ return \@designators;
+}
+
+sub vpd_has_naa($page, $hex) {
+ my $designators = vpd_designators($page) // return 0;
+ # code set 1 (binary), association 0 (logical unit), designator type 3 (NAA)
+ my @match = grep {
+ my ($set, $association, $type, $value) = $_->@*;
+ $set == 1 && $association == 0 && $type == 3 && unpack('H*', $value) eq lc($hex)
+ } $designators->@*;
+ return @match ? 1 : 0;
+}
+
+my sub line_identifies($disk, $line, $naa_ok) {
+ return 0
+ if !$line || $line->{init} ne 'ok' || $line->{wwid} !~ $PVE::Multipath::Config::WWID_RE;
+ return 0 if $line->{devt} ne $disk->{devt} || $line->{hctl} ne $disk->{hctl};
+ return 0 if defined($disk->{target}) && $line->{target} ne $disk->{target};
+ my ($hex) = $line->{wwid} =~ m/^3([0-9a-fA-F]+)$/;
+ return defined($hex) ? $naa_ok->($disk, $hex) : 1;
+}
+
+# An unlisted path alone does not make its LUN unresolved when the daemon answered and there is no
+# other identity evidence.
+sub resolve($disks, $daemon, $naa_ok) {
+ my $luns = {};
+ for my $name (sort keys %$disks) {
+ my $disk = $disks->{$name};
+ my $key = "$disk->{target}\0$disk->{lun}";
+ $luns->{$key} //= { target => $disk->{target}, lun => $disk->{lun}, paths => [] };
+ push $luns->{$key}->{paths}->@*, $name;
+ }
+
+ for my $lun (values %$luns) {
+ my ($held, $listed, $unlisted) = ({}, {}, 0);
+ my $unresolved = !$daemon;
+ for my $name ($lun->{paths}->@*) {
+ my $disk = $disks->{$name};
+ $held->{ $disk->{map} } = 1 if defined($disk->{map});
+ my $line = $daemon ? $daemon->{$name} : undef;
+ if (line_identifies($disk, $line, $naa_ok)) {
+ $listed->{ $line->{wwid} } = 1;
+ } else {
+ $unlisted = 1;
+ $unresolved = 1 if $line;
+ }
+ }
+ my @held = keys %$held;
+ if (@held) {
+ # A missing daemon identity does not count against a trusted map identity.
+ $lun->{wwid} = $held[0]
+ if @held == 1 && !grep { $_ ne $held[0] } keys %$listed;
+ } elsif (scalar(keys %$listed) == 1 && !$unlisted) {
+ ($lun->{wwid}) = keys %$listed;
+ }
+ $lun->{unresolved} = 1 if !defined($lun->{wwid}) && ($unresolved || @held || %$listed);
+ }
+ return $luns;
+}
+
+# The iSCSI sessions: { sid => { target, host } }.
+my sub iscsi_sessions() {
+ my $res = {};
+ dir_glob_foreach(
+ "$SYSFS/class/iscsi_session",
+ 'session(\d+)',
+ sub($ent, $sid) {
+ my $target = sysfs_line("class/iscsi_session/$ent/targetname") // return;
+ my (undef, $host) =
+ dir_glob_regex("$SYSFS/class/iscsi_session/$ent/device", 'target(\d+):\d+:\d+');
+ return if !defined($host);
+ $res->{$sid} = { target => $target, host => $host };
+ },
+ );
+ return $res;
+}
+
+# The target of each LUN, from the session whose SCSI target holds it: offload initiators can run
+# sessions to several targets on one SCSI host.
+my sub lun_targets($sessions) {
+ my $res = {};
+ for my $sid (keys %$sessions) {
+ my $dir = "$SYSFS/class/iscsi_session/session$sid/device";
+ dir_glob_foreach(
+ $dir,
+ 'target\d+:\d+:\d+',
+ sub($targetdir) {
+ dir_glob_foreach(
+ "$dir/$targetdir",
+ '\d+:\d+:\d+:\d+',
+ sub($hctl) { $res->{$hctl} = $sessions->{$sid}->{target}; },
+ );
+ },
+ );
+ }
+ return $res;
+}
+
+my sub session_disks($sessions, $targets) {
+ my $lun_target = lun_targets($sessions);
+ my $res = {};
+ dir_glob_foreach(
+ "$SYSFS/bus/scsi/devices",
+ '\d+:\d+:\d+:(\d+)',
+ sub($hctl, $lun) {
+ my $target = $lun_target->{$hctl} // return;
+ return if $targets && !$targets->{$target};
+ my $dir = "bus/scsi/devices/$hctl";
+ return if (sysfs_line("$dir/type") // '') ne '0'; # disks only
+ my (undef, $name) = dir_glob_regex("$SYSFS/$dir/block", '([a-z]\S*)');
+ return if !defined($name);
+ my $devt = sysfs_line("$dir/block/$name/dev") // return;
+ my $disk =
+ { target => $target, lun => int($lun), devt => $devt, hctl => $hctl, dir => $dir };
+ # Read the holder through the SCSI device, so neither name can refer to another device
+ # meanwhile.
+ my (undef, $holder) = dir_glob_regex("$SYSFS/$dir/block/$name/holders", '(dm-\d+)');
+ if (defined($holder)) {
+ my $uuid = sysfs_line("$dir/block/$name/holders/$holder/dm/uuid") // '';
+ ($disk->{map}) = $uuid =~ m/^mpath-(\S+)$/;
+ }
+ $res->{$name} = $disk;
+ },
+ );
+ return $res;
+}
+
+sub collect(%param) {
+ my $sessions = iscsi_sessions();
+ my $disks = session_disks($sessions, $param{targets});
+
+ my ($daemon, $error);
+ if (grep { !defined($_->{map}) } values %$disks) {
+ $daemon = eval { PVE::Multipath::query_path_identities() };
+ chomp($error = $@) if $@;
+ }
+ my $naa_ok = sub($disk, $hex) {
+ my $page = eval { file_get_contents("$SYSFS/$disk->{dir}/vpd_pg83") };
+ return vpd_has_naa($page, $hex);
+ };
+ my $luns = resolve($disks, $daemon, $naa_ok);
+ my $paths = {};
+ for my $lun (values %$luns) {
+ for my $name ($lun->{paths}->@*) {
+ my $disk = $disks->{$name};
+ $paths->{$name} = {
+ $disk->%{qw(devt hctl)},
+ wwid => $lun->{wwid},
+ $lun->{unresolved} ? (unresolved => 1) : (),
+ };
+ }
+ }
+ return {
+ luns => $luns,
+ paths => $paths,
+ defined($error) ? (error => $error) : (),
+ };
+}
+
+sub target_luns($ids, $target) {
+ return [grep { $_->{target} eq $target } values $ids->{luns}->%*];
+}
+
+# The WWID of the LUN of the path $name in $ids, if it was resolved and the name still denotes the
+# same device: the kernel may have given it to another one since the collection.
+sub path_wwid($ids, $name) {
+ my $path = $ids->{paths}->{$name} // return undef;
+ return undef if !defined($path->{wwid});
+ my $devt = sysfs_line("class/block/$name/dev");
+ my $device = abs_path("$SYSFS/class/block/$name/device");
+ if (
+ !defined($devt)
+ || !defined($device)
+ || $devt ne $path->{devt}
+ || basename($device) ne $path->{hctl}
+ ) {
+ $path->{unresolved} = 1;
+ return undef;
+ }
+ return $path->{wwid};
+}
+
+1;
diff --git a/src/PVE/Multipath/Status.pm b/src/PVE/Multipath/Status.pm
new file mode 100644
index 00000000..e3dccf08
--- /dev/null
+++ b/src/PVE/Multipath/Status.pm
@@ -0,0 +1,70 @@
+package PVE::Multipath::Status;
+
+use v5.36;
+
+use JSON;
+use PVE::Multipath;
+use PVE::Multipath::Config;
+
+my $KV_KEY = 'multipath';
+
+# Older nodes can drop the unknown multipath option when rewriting storage.cfg; offline nodes
+# may also lack a report, so silence is not evidence of compatibility.
+sub nodes_without_status() {
+ require PVE::Cluster;
+ require PVE::INotify;
+
+ my $local = PVE::INotify::nodename();
+ my $raw = PVE::Cluster::get_node_kv($KV_KEY);
+ my $reported = sub($node) {
+ my $status = eval { decode_json($raw->{$node} // '') };
+ return ref($status) eq 'HASH';
+ };
+ return [sort grep { $_ ne $local && !$reported->($_) } PVE::Cluster::get_nodelist()->@*];
+}
+
+# The nodes out of $nodes ({ node => 1 }, empty or undef for every node) whose status says that
+# multipath-tools is not installed; nodes without a status are left to nodes_without_status().
+sub nodes_without_tools($nodes) {
+ require PVE::Cluster;
+ require PVE::INotify;
+
+ my $local = PVE::INotify::nodename();
+ my $raw = PVE::Cluster::get_node_kv($KV_KEY);
+ my $res = [];
+ for my $node (sort { $a cmp $b } PVE::Cluster::get_nodelist()->@*) {
+ next if $nodes && %$nodes && !$nodes->{$node};
+ my $supported;
+ if ($node eq $local) {
+ $supported = PVE::Multipath::is_supported();
+ } else {
+ next if !defined($raw->{$node});
+ my $status = eval { decode_json($raw->{$node}) };
+ $supported = ref($status) eq 'HASH' && $status->{supported};
+ }
+ push $res->@*, $node if !$supported;
+ }
+ return $res;
+}
+
+sub assert_multipath_tools($nodes) {
+ my $no_tools = nodes_without_tools($nodes);
+ die "multipath-tools is not installed on node(s) "
+ . join(', ', $no_tools->@*)
+ . ", which the storage is available on\n"
+ if scalar($no_tools->@*);
+ return;
+}
+
+sub assert_cluster_supports_multipath($nodes) {
+ my $silent = nodes_without_status();
+ die "node(s) "
+ . join(', ', $silent->@*)
+ . " did not report multipath support yet - upgrade"
+ . " every node and make sure it is online before enabling multipath on a storage\n"
+ if scalar($silent->@*);
+ assert_multipath_tools($nodes);
+ return;
+}
+
+1;
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 2fc91a82..88134473 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -19,6 +19,7 @@ use PVE::Cluster qw(cfs_read_file cfs_write_file cfs_lock_file);
use PVE::DataCenterConfig;
use PVE::Exception qw(raise_param_exc raise);
use PVE::JSONSchema;
+use PVE::Multipath::Identity;
use PVE::INotify;
use PVE::RPCEnvironment;
use PVE::SSHInfo;
@@ -298,6 +299,36 @@ sub storage_has_feature {
return PVE::Storage::Plugin::storage_has_feature($scfg->{type}, $feature);
}
+# Retain empty entries on failure: absence means the storage is no longer enabled here, not merely
+# unreachable.
+sub derived_multipath_wwids {
+ my ($cfg, $ids) = @_;
+
+ my @storeids;
+ for my $storeid (sort keys $cfg->{ids}->%*) {
+ my $scfg = $cfg->{ids}->{$storeid};
+ next if !$scfg->{multipath} || $scfg->{type} ne 'iscsi';
+ next if !storage_check_enabled($cfg, $storeid, undef, 1);
+ push @storeids, $storeid;
+ }
+ return {} if !@storeids;
+ $ids //= PVE::Multipath::Identity::collect(
+ targets => { map { $cfg->{ids}->{$_}->{target} => 1 } @storeids },
+ );
+
+ my $derived = {};
+ for my $storeid (@storeids) {
+ my $scfg = $cfg->{ids}->{$storeid};
+ my $set = eval { PVE::Storage::ISCSIPlugin->multipath_lun_wwids($scfg, $ids) };
+ if (my $err = $@) {
+ chomp $err;
+ $set = { wwids => {}, error => $err };
+ }
+ $derived->{$storeid} = $set;
+ }
+ return $derived;
+}
+
# storage_can_replicate:
# return true if storage supports replication
# (volumes allocated with vdisk_alloc() has replication feature)
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index d10a5549..0e3b6df3 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -8,13 +8,16 @@ use Digest::SHA qw(sha256_hex);
use Fcntl qw(O_RDONLY O_EXCL O_NONBLOCK LOCK_EX LOCK_NB);
use IO::Dir;
use IO::File;
+use Time::HiRes qw(clock_gettime CLOCK_MONOTONIC);
use PVE::Diskmanage;
use PVE::JSONSchema qw(get_standard_option);
use PVE::Multipath;
+use PVE::Multipath::Identity;
use PVE::Storage::Plugin;
use PVE::Tools
qw(run_command file_read_firstline trim dir_glob_regex dir_glob_foreach $IPV4RE $IPV6RE);
+use PVE::Multipath::Status;
use base qw(PVE::Storage::Plugin);
@@ -678,11 +681,15 @@ sub iscsi_logout_guarded {
return $result;
}
+# The LUNs of the iSCSI sessions, { target => { volname => info } }. With $map_names a LUN is named
+# after its multipath map, see PVE::Multipath::Identity, and only the LUNs of $target are listed.
sub iscsi_device_list {
+ my ($map_names, $target) = @_;
my $res = {};
my $stable_paths = load_stable_scsi_paths();
+ my $ids = $map_names ? PVE::Multipath::Identity::collect(targets => { $target => 1 }) : undef;
my $device_map = iscsi_session_device_map();
@@ -690,6 +697,8 @@ sub iscsi_device_list {
my $info = $device_map->{$bdev};
my ($channel, $id, $lun) = $info->@{qw(channel id lun)};
+ my $path_dev = $bdev;
+
#check multipath
if (-d "$SYSFS/block/$bdev/holders") {
my $multipathdev = dir_glob_regex("$SYSFS/block/$bdev/holders", '[A-Za-z]\S*');
@@ -697,6 +706,15 @@ sub iscsi_device_list {
}
my $blockdev = $stable_paths->{$bdev};
+ # scsi-<wwid> can name a member before assembly; dm-uuid-mpath-<wwid> only names the map.
+ # Managed volumes keep the map name even while assembly is pending.
+ if ($map_names) {
+ my $wwid =
+ $info->{target} eq $target
+ ? PVE::Multipath::Identity::path_wwid($ids, $path_dev)
+ : undef;
+ $blockdev = defined($wwid) ? "dm-uuid-mpath-$wwid" : undef;
+ }
next if !$blockdev;
my $size = file_read_firstline("$SYSFS/block/$bdev/size");
@@ -717,6 +735,26 @@ sub iscsi_device_list {
return $res;
}
+# Expect one path per node-record portal.
+sub multipath_lun_wwids {
+ my ($class, $scfg, $ids) = @_;
+
+ my $res = { wwids => {} };
+ my $target = $scfg->{target};
+
+ $ids //= PVE::Multipath::Identity::collect(targets => { $target => 1 });
+ my $luns = PVE::Multipath::Identity::target_luns($ids, $target);
+ return $res if !scalar(@$luns);
+
+ my $paths_expected = scalar(iscsi_portals($target, $scfg->{portal})->@*);
+ for my $lun (@$luns) {
+ next if !defined($lun->{wwid});
+ $res->{wwids}->{ $lun->{wwid} } = { 'paths-expected' => $paths_expected };
+ }
+
+ return $res;
+}
+
# Configuration
sub type {
@@ -742,6 +780,11 @@ sub properties {
type => 'string',
format => 'pve-storage-portal-dns',
},
+ multipath => {
+ description => "Manage multipath access to the storage's LUNs and include them in"
+ . " the cluster-wide path-health monitoring.",
+ type => 'boolean',
+ },
};
}
@@ -753,9 +796,62 @@ sub options {
disable => { optional => 1 },
content => { optional => 1 },
bwlimit => { optional => 1 },
+ multipath => { optional => 1 },
};
}
+my sub base_volume_consumers {
+ my ($cfg, $storeid, $volname_re) = @_;
+
+ my $consumers = [];
+ for my $sid (sort keys $cfg->{ids}->%*) {
+ my $base = $cfg->{ids}->{$sid}->{base} or next;
+ my ($baseid, $volname) = eval { PVE::Storage::Plugin::parse_volume_id($base) };
+ next if !defined($baseid) || $baseid ne $storeid;
+ push $consumers->@*, $sid if $volname =~ $volname_re;
+ }
+ return $consumers;
+}
+
+sub on_add_hook {
+ my ($class, $storeid, $scfg, %param) = @_;
+
+ PVE::Multipath::Status::assert_cluster_supports_multipath($scfg->{nodes}) if $scfg->{multipath};
+
+ return $class->SUPER::on_add_hook($storeid, $scfg, %param);
+}
+
+sub on_update_hook_full {
+ my ($class, $storeid, $scfg, $update, $delete, $sensitive) = @_;
+
+ my $disables_multipath = (defined($update->{multipath}) && !$update->{multipath})
+ || grep { $_ eq 'multipath' } ($delete // [])->@*;
+
+ if ($scfg->{multipath} && $disables_multipath) {
+ my $cfg = PVE::Storage::config();
+ my $consumers = base_volume_consumers($cfg, $storeid, qr/\.dm-uuid-mpath-\S+$/);
+ die "cannot disable multipath - storage '$consumers->[0]' uses a multipath map of this"
+ . " storage as its base volume\n"
+ if scalar($consumers->@*);
+ } elsif (!$scfg->{multipath} && $update->{multipath}) {
+ my $nodes = $update->{nodes} // $scfg->{nodes};
+ $nodes = undef if grep { $_ eq 'nodes' } ($delete // [])->@*;
+ PVE::Multipath::Status::assert_cluster_supports_multipath($nodes);
+ }
+
+ my $drops_restriction = grep { $_ eq 'nodes' } ($delete // [])->@*;
+ if (
+ $scfg->{multipath}
+ && !$disables_multipath
+ && (defined($update->{nodes}) || $drops_restriction)
+ ) {
+ PVE::Multipath::Status::assert_multipath_tools(
+ $drops_restriction ? undef : $update->{nodes});
+ }
+
+ return $class->SUPER::on_update_hook_full($storeid, $scfg, $update, $delete, $sensitive);
+}
+
# Storage implementation
sub parse_volname {
@@ -823,13 +919,14 @@ sub list_images {
my $res = [];
- $cache->{iscsi_devices} = iscsi_device_list() if !$cache->{iscsi_devices};
+ my $cache_key = $scfg->{multipath} ? "iscsi_devices_mpath:$scfg->{target}" : 'iscsi_devices';
+ $cache->{$cache_key} //= iscsi_device_list($scfg->{multipath}, $scfg->{target});
# we have no owner for iscsi devices
my $target = $scfg->{target};
- if (my $dat = $cache->{iscsi_devices}->{$target}) {
+ if (my $dat = $cache->{$cache_key}->{$target}) {
foreach my $volname (keys %$dat) {
@@ -997,9 +1094,55 @@ $resolve_virtual_devices = sub {
return $resolved;
};
+# The polling budget excludes an in-flight identity or map query (bounded by five and ten
+# seconds respectively). Legacy aliases need not be repointed at the map, so this only waits for
+# an identified map; normal activation must resolve and validate the original alias afterwards.
+sub wait_for_path_map {
+ my ($scfg, $path, $budget) = @_;
+ $budget //= 10;
+ my $deadline = clock_gettime(CLOCK_MONOTONIC) + $budget;
+ while (clock_gettime(CLOCK_MONOTONIC) < $deadline) {
+ my $real = Cwd::realpath($path) // '';
+ if ($real =~ m{^/dev/(dm-\d+)$}) {
+ my $uuid = file_read_firstline("/sys/class/block/$1/dm/uuid") // '';
+ return 1 if $uuid =~ m/^mpath-\S+$/ && -b $real;
+ } elsif ($real =~ m{^/dev/(sd[a-z]+)$}) {
+ my $name = $1;
+ my $wwid = eval {
+ my $ids =
+ PVE::Multipath::Identity::collect(targets => { $scfg->{target} => 1 });
+ PVE::Multipath::Identity::path_wwid($ids, $name);
+ };
+ last if clock_gettime(CLOCK_MONOTONIC) >= $deadline;
+ if (defined($wwid)) {
+ # One query per round leaves the outer monotonic clock in charge of polling.
+ return 1 if eval { PVE::Multipath::wait_for_map($wwid, 0) };
+ }
+ }
+ my $remaining = $deadline - clock_gettime(CLOCK_MONOTONIC);
+ last if $remaining <= 0;
+ Time::HiRes::sleep($remaining < 1 ? $remaining : 1);
+ }
+ return 0;
+}
+
sub activate_volume {
my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+ if ($scfg->{multipath}) {
+ if (my ($wwid) = $volname =~ m/\.dm-uuid-mpath-(\S+)$/) {
+ # Activation can race map assembly at login or boot; wait before resolving the link.
+ PVE::Multipath::wait_for_map($wwid)
+ or die "multipath map for WWID '$wwid' did not assemble in time"
+ . " - check the multipath status of this node\n";
+ } else {
+ my $path = $class->filesystem_path($scfg, $volname, $snapname);
+ warn "volume '$volname' of storage '$storeid': timed out waiting to confirm a multipath"
+ . " map; continuing normal volume activation - check this node's multipath status\n"
+ if !wait_for_path_map($scfg, $path);
+ }
+ }
+
my $path = $class->filesystem_path($scfg, $volname, $snapname);
my $real_path = Cwd::realpath($path);
die "failed to get realpath for '$path': $!\n" if !$real_path;
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index 723ed555..11a64c97 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -1083,11 +1083,17 @@ sub on_add_hook {
die "unsupported base type '$basecfg->{type}'"
if $basecfg->{type} ne 'iscsi';
+ # Managed multipath requires the coalesced map, never a transient single-path PV.
+ die "base volume '$volname' does not reference a multipath map - retry in a few seconds"
+ . " once the map has assembled, or check the multipath status of this node\n"
+ if $basecfg->{multipath} && $volname !~ m/\.dm-uuid-mpath-\S+$/;
+
my $path = PVE::Storage::path($cfg, $base);
PVE::Storage::activate_storage($cfg, $baseid);
- # Activating the base volume verifies that it belongs to its target.
+ # the base's activate_volume checks target membership and, with managed multipath, waits
+ # for the map to assemble before the VG is created on it
PVE::Storage::activate_volumes($cfg, [$base]);
# a failing hook does not remove a volume group it created, so check the device first
diff --git a/src/test/Makefile b/src/test/Makefile
index 3517db6e..97b28929 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_iscsi test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_san_api test_san_libraries
+test: test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_iscsi test_lun_identity test_multipath_api test_san_api test_san_libraries
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -52,16 +52,23 @@ test_volume_access: run_volume_access_tests.pl
test_multipath: run_multipath_tests.pl
./run_multipath_tests.pl
-test_san_api: run_san_api_tests.pl
- ./run_san_api_tests.pl
-
-test_iscsi: run_iscsi_logout_tests.pl run_iscsi_discovery_tests.pl run_iscsi_rescan_tests.pl run_iscsi_device_list_tests.pl run_iscsi_session_tests.pl run_iscsi_volume_tests.pl
+test_iscsi: run_iscsi_logout_tests.pl run_iscsi_activation_tests.pl run_iscsi_discovery_tests.pl run_iscsi_rescan_tests.pl run_iscsi_device_list_tests.pl run_iscsi_session_tests.pl run_iscsi_volume_tests.pl
./run_iscsi_logout_tests.pl
+ ./run_iscsi_activation_tests.pl
./run_iscsi_discovery_tests.pl
./run_iscsi_rescan_tests.pl
./run_iscsi_device_list_tests.pl
./run_iscsi_session_tests.pl
./run_iscsi_volume_tests.pl
+test_lun_identity: run_lun_identity_tests.pl
+ ./run_lun_identity_tests.pl
+
+test_multipath_api: run_multipath_api_tests.pl
+ ./run_multipath_api_tests.pl
+
+test_san_api: run_san_api_tests.pl
+ ./run_san_api_tests.pl
+
test_san_libraries: run_san_library_tests.pl
./run_san_library_tests.pl
diff --git a/src/test/run_iscsi_activation_tests.pl b/src/test/run_iscsi_activation_tests.pl
new file mode 100755
index 00000000..39a1a735
--- /dev/null
+++ b/src/test/run_iscsi_activation_tests.pl
@@ -0,0 +1,171 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Storage::ISCSIPlugin;
+
+my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+my $identity = Test::MockModule->new('PVE::Multipath::Identity');
+my $mp = Test::MockModule->new('PVE::Multipath');
+my $cwd = Test::MockModule->new('Cwd');
+my $hires = Test::MockModule->new('Time::HiRes');
+my ($now, $link_ready, $identity_ready, $map_ready, $identity_error, $id_cost, $map_cost);
+my ($resolved, $target, $reads, $queries, $waits, $last_udev);
+my @sleeps;
+my $scfg = { multipath => 1, target => 'iqn.test' };
+my $volname = '0.0.1.scsi-not-the-wwid';
+$plugin->mock(
+ clock_gettime => sub { $now },
+ filesystem_path => sub { '/dev/disk/by-id/legacy' },
+ file_read_firstline => sub {
+ return 'mpath-identified' if $_[0] =~ m{/dm/uuid$};
+ return $target if $_[0] =~ m{/targetname$};
+ die "unexpected read $_[0]";
+ },
+ run_command => sub {
+ my ($cmd, %options) = @_;
+ die 'unexpected command' if $cmd->[0] ne 'udevadm';
+ $last_udev = $cmd->[-1];
+ $options{outfunc}->('/devices/platform/host1/session1/target1:0:0/1:0:0:1/block/sda');
+ },
+);
+$cwd->mock(
+ realpath => sub {
+ $reads++;
+ return $now >= $link_ready ? $resolved : undef;
+ },
+);
+$hires->mock(
+ sleep => sub {
+ push @sleeps, [$now, $_[0]];
+ $now += $_[0];
+ },
+);
+$identity->mock(
+ collect => sub {
+ my %param = @_;
+ is_deeply(
+ $param{targets},
+ { 'iqn.test' => 1 },
+ 'identity collection restricted to target',
+ );
+ $queries++;
+ $now += $id_cost;
+ die "identity query failed\n" if $identity_error;
+ return {};
+ },
+ path_wwid => sub {
+ is($_[1], 'sda', 'resolved path, not alias suffix, identifies the LUN');
+ return $now >= $identity_ready ? 'identified' : undef;
+ },
+);
+$mp->mock(
+ wait_for_map => sub {
+ $waits++;
+ is($_[0], 'identified', 'identified WWID passed to map wait');
+ is($_[1], 0, 'single map query per budget round') if @_ > 1;
+ $now += $map_cost;
+ return $now >= $map_ready ? { wwid => 'identified' } : undef;
+ },
+);
+my sub reset_case {
+ ($now, $link_ready, $identity_ready, $map_ready, $identity_error, $id_cost, $map_cost) =
+ (0, 0, 0, 0, 0, 0, 0);
+ ($resolved, $target, $reads, $queries, $waits, $last_udev) =
+ ('/dev/sda', 'iqn.test', 0, 0, 0, undef);
+ @sleeps = ();
+}
+my sub activate {
+ my ($name, $config) = @_;
+ my $warnings = '';
+ local $SIG{__WARN__} = sub { $warnings .= $_[0] };
+ eval { PVE::Storage::ISCSIPlugin->activate_volume('san', $config // $scfg, $name // $volname) };
+ return ($@, $warnings);
+}
+
+for my $case (
+ ['delayed link', sub { $link_ready = 3 }],
+ ['delayed identity', sub { $identity_ready = 3 }],
+ ['sd path identity', sub { }],
+) {
+ reset_case();
+ $case->[1]->();
+ ok(PVE::Storage::ISCSIPlugin::wait_for_path_map($scfg, '/dev/disk/by-id/legacy'), $case->[0]);
+ ok($now <= 10, 'within polling budget');
+}
+for my $case (
+ ['failed identity queries', sub { $identity_error = 1 }, 10],
+ ['slow identity query', sub { $id_cost = 11 }, 11],
+ ['slow map query', sub { $id_cost = 4; $map_cost = 10; $map_ready = 100 }, 14],
+) {
+ reset_case();
+ $case->[1]->();
+ ok(
+ !PVE::Storage::ISCSIPlugin::wait_for_path_map($scfg, '/dev/disk/by-id/legacy'),
+ "$case->[0] times out",
+ );
+ is($now, $case->[2], 'budget plus in-flight query only');
+ ok(!(grep { $_->[0] >= 10 } @sleeps), 'no sleep starts after budget expires');
+ is($waits, 0, 'no map query starts after identity consumes budget') if $id_cost == 11;
+}
+
+SKIP: {
+ # Only stat metadata of these existing nodes is used. All device queries are mocked.
+ skip 'no /dev/sda block node for activation validation', 10 if !-b '/dev/sda';
+ reset_case();
+ my ($err, $warn) = activate();
+ is($err, '', 'successful map wait with alias still naming a member');
+ is($warn, '', 'no warning after confirmation');
+ is($last_udev, '/dev/sda', 'post-wait resolution validates the original alias');
+ is($reads, 2, 'alias resolved again after wait');
+ reset_case();
+ $map_ready = 100;
+ ($err, $warn) = activate();
+ is($err, '', 'timeout continues normal activation of a valid member');
+ is(
+ $warn,
+ "volume '$volname' of storage 'san': timed out waiting to confirm a multipath map; continuing normal volume activation - check this node's multipath status\n",
+ 'exact compatibility warning',
+ );
+ reset_case();
+ $target = 'iqn.wrong';
+ ($err, $warn) = activate();
+ like($err, qr/not part of target/, 'wrong-target device still refused');
+ reset_case();
+ ($err, $warn) = activate(undef, { %$scfg, multipath => 0 });
+ is($err, '', 'storage without multipath unchanged');
+ is($waits + $queries, 0, 'no wait or identity query without multipath');
+ reset_case();
+ ($err, $warn) = activate('0.0.1.dm-uuid-mpath-identified');
+ is($queries, 0, 'map-named volumes keep their direct map wait');
+}
+reset_case();
+$link_ready = 100;
+my ($err, $warn) = activate();
+like($err, qr/failed to get realpath/, 'permanently absent link still fails full activation');
+like($warn, qr/continuing normal volume activation/, 'absent link warns before normal validation');
+reset_case();
+$resolved = '/dev/null';
+($err, $warn) = activate();
+like($err, qr/not a block device/, 'non-block device still refused');
+reset_case();
+$map_ready = 100;
+($err, $warn) = activate('0.0.1.dm-uuid-mpath-identified');
+like($err, qr/did not assemble in time/, 'map-named timeout remains fatal');
+SKIP: {
+ skip 'no /dev/dm-0 block node for map validation', 2 if !-b '/dev/dm-0';
+ reset_case();
+ $resolved = '/dev/dm-0';
+ ok(
+ PVE::Storage::ISCSIPlugin::wait_for_path_map($scfg, '/dev/disk/by-id/legacy'),
+ 'alias already resolves to map',
+ );
+ is($queries + $waits, 0, 'existing map needs no daemon query');
+}
+
+done_testing();
diff --git a/src/test/run_lun_identity_tests.pl b/src/test/run_lun_identity_tests.pl
new file mode 100755
index 00000000..df248b38
--- /dev/null
+++ b/src/test/run_lun_identity_tests.pl
@@ -0,0 +1,557 @@
+#!/usr/bin/perl
+
+# Unit test for the identities that dm-multipath gives the LUNs of iSCSI sessions: the path list of
+# multipathd, the NAA designator of the cached VPD page 0x83, the resolution of a LUN from its maps
+# and the daemon's lines bound to its devices, the collection from sysfs (a fake tree below a
+# temporary directory) and the iSCSI plugin's use of it. The daemon is mocked.
+
+use strict;
+use warnings;
+
+use lib '..';
+use File::Path qw(make_path remove_tree);
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use PVE::Multipath;
+use PVE::Multipath::Identity;
+use PVE::Storage;
+use PVE::Storage::ISCSIPlugin;
+use PVE::Multipath::Status;
+
+my $T = 'iqn.1992-08.com.netapp:sn.1:vs.5';
+my $O = 'iqn.2003-01.org.linux-iscsi.other:sn.2';
+
+# --- the daemon's path list ---
+
+my $paths = PVE::Multipath::parse_path_identities(join(
+ "\n",
+ "sda|8:0|0:0:0:0|ok|[undef]|0QEMU_QEMU_HARDDISK_drive-scsi0",
+ "sdb|8:16|7:0:0:0|ok|$T|3600a0980383235536724",
+ "sdc|8:32|8:0:0:0|udev-missing|$T|",
+ "-|#:#|#:#:#:#|undef|[undef]|",
+ "sdd|8:48|9:0:0:1|ok|$T|3600a0980dup",
+ "sdd|8:64|9:0:0:2|ok|$T|3600a0980dup",
+ '',
+));
+is_deeply(
+ $paths,
+ {
+ sda => {
+ devt => '8:0',
+ hctl => '0:0:0:0',
+ init => 'ok',
+ target => '[undef]',
+ wwid => '0QEMU_QEMU_HARDDISK_drive-scsi0',
+ },
+ sdb => {
+ devt => '8:16',
+ hctl => '7:0:0:0',
+ init => 'ok',
+ target => $T,
+ wwid => '3600a0980383235536724',
+ },
+ sdc =>
+ { devt => '8:32', hctl => '8:0:0:0', init => 'udev-missing', target => $T, wwid => '' },
+ },
+ 'path list: every path, placeholders kept, a name listed twice left out',
+);
+is_deeply(PVE::Multipath::parse_path_identities("ok\n"), {}, 'path list: no paths');
+for my $case (
+ ["fail\n", 'a busy daemon'],
+ ["timeout\n", 'a timeout'],
+ ["multipath-tools v0.11.1\nCLI commands reference:\n list|show paths\n", 'the help text'],
+ ["sdb|8:16|7:0:0:0|ok|$T\n", 'a line without all fields'],
+ ["sdb|8:x|7:0:0:0|ok|$T|3600a\n", 'a malformed field'],
+) {
+ is(PVE::Multipath::parse_path_identities($case->[0]), undef, "path list: none from $case->[1]");
+}
+
+# --- the NAA designator of the VPD page 0x83, as the kernel cached it for a NetApp LUN: T10
+# vendor, NAA and EUI-64 of the LUN, NAA of the target port, port and group numbers, SCSI name ---
+
+my $vpd = pack(
+ 'H*',
+ '0083009c020100204e45544150502020204c554e203832355367242f747a716b7a202020202020200103'
+ . '0010600a09803832355367242f747a716b7a01020010242f747a716b7a0000a0983832355367011300'
+ . '10600a098000000002ac16036a00000cbc011400040000000201150004000003e8032800286e61612e'
+ . '363030413039383033383332333535333637323432463734374137313642373900000000',
+);
+ok(
+ PVE::Multipath::Identity::vpd_has_naa($vpd, '600A09803832355367242F747A716B7A'),
+ 'VPD: the NAA designator of the logical unit',
+);
+ok(
+ !PVE::Multipath::Identity::vpd_has_naa($vpd, '600a098000000002ac16036a00000cbc'),
+ 'VPD: not the one of the target port',
+);
+ok(
+ !PVE::Multipath::Identity::vpd_has_naa($vpd, '600a09803832355367242f747a716c30'),
+ 'VPD: not another LUN',
+);
+ok(
+ !PVE::Multipath::Identity::vpd_has_naa(substr($vpd, 0, 60), '600a09803832355367242f747a716b7a'),
+ 'VPD: nothing from a truncated page',
+);
+{
+ # the NAA designator of the LUN, then one that claims 32 bytes the page does not have
+ my $page =
+ pack('C C n', 0, 0x83, 24)
+ . pack('C C x C', 0x01, 0x03, 16)
+ . pack('H*', '600a09803832355367242f747a716b7a')
+ . pack('C C x C', 0x01, 0x02, 32);
+ ok(
+ !PVE::Multipath::Identity::vpd_has_naa($page, '600a09803832355367242f747a716b7a'),
+ 'VPD: nothing from a page whose designators do not add up, even after a match',
+ );
+}
+ok(!PVE::Multipath::Identity::vpd_has_naa(undef, '600a'), 'VPD: nothing without a page');
+
+# --- resolving LUNs ---
+
+my sub disk {
+ my ($lun, $n, %extra) = @_;
+ return (
+ "sd$n" => {
+ target => $extra{target} // $T,
+ lun => $lun,
+ devt => "8:$n",
+ hctl => "$n:0:0:$lun",
+ defined($extra{map}) ? (map => $extra{map}) : (),
+ },
+ );
+}
+
+my sub line {
+ my ($n, $lun, $wwid, %extra) = @_;
+ return (
+ "sd$n" => {
+ devt => $extra{devt} // "8:$n",
+ hctl => $extra{hctl} // "$n:0:0:$lun",
+ init => $extra{init} // 'ok',
+ target => $extra{target} // $T,
+ wwid => $wwid,
+ },
+ );
+}
+
+my $naa_checked = [];
+my $naa_ok = sub {
+ my ($disk, $hex) = @_;
+ push $naa_checked->@*, $hex;
+ return $hex ne 'bad';
+};
+
+my sub resolved {
+ my ($disks, $daemon) = @_;
+ my $luns = PVE::Multipath::Identity::resolve($disks, $daemon, $naa_ok);
+ return { map { $_->{lun} => $_->{wwid} } values %$luns };
+}
+
+is_deeply(
+ resolved({ disk(0, 1), disk(0, 2, map => '3W0'), disk(0, 3) }, undef),
+ { 0 => '3W0' },
+ 'resolve: a map on a later path, the others not listed',
+);
+is_deeply(
+ resolved({ disk(0, 1, map => '3W0'), disk(0, 2, map => '3X0') }, undef),
+ { 0 => undef },
+ 'resolve: maps that disagree',
+);
+is_deeply(
+ resolved({ disk(0, 1, map => '3W0'), disk(0, 2) }, { line(2, 0, '3X0') }),
+ { 0 => undef },
+ 'resolve: a map and a path the daemon names differently',
+);
+is_deeply(
+ resolved({ disk(0, 1, map => '3W0'), disk(0, 2) }, { line(2, 0, '3W0') }),
+ { 0 => '3W0' },
+ 'resolve: a map and a path the daemon names alike',
+);
+is_deeply(
+ resolved({ disk(1, 1), disk(1, 2) }, { line(1, 1, '3W1'), line(2, 1, '3W1') }),
+ { 1 => '3W1' },
+ 'resolve: paths without a map that the daemon names alike',
+);
+is_deeply(
+ resolved({ disk(2, 1), disk(2, 2) }, { line(1, 2, '3W2'), line(2, 2, '3X2') }),
+ { 2 => undef },
+ 'resolve: paths the daemon names differently',
+);
+is_deeply(
+ resolved({ disk(3, 1), disk(3, 2) }, { line(1, 3, '3W3') }),
+ { 3 => undef },
+ 'resolve: a path the daemon does not list',
+);
+
+for my $case (
+ [{ init => 'udev-missing' }, 'a path the daemon did not initialize'],
+ [{ hctl => '9:0:0:4' }, 'a line of another SCSI device under a reused name'],
+ [{ devt => '8:99' }, 'a line of another block device'],
+ [{ target => $O }, 'a line of another target'],
+) {
+ my ($extra, $what) = @$case;
+ is_deeply(
+ resolved({ disk(4, 1), disk(4, 2) }, { line(1, 4, '3W4'), line(2, 4, '3W4', %$extra) }),
+ { 4 => undef },
+ "resolve: not with $what",
+ );
+}
+is_deeply(
+ resolved({ disk(5, 1) }, { line(1, 5, '') }),
+ { 5 => undef },
+ 'resolve: not with an empty identity',
+);
+is_deeply(
+ resolved({ disk(6, 1) }, { line(1, 6, '3bad') }),
+ { 6 => undef },
+ 'resolve: not with an NAA identity the device does not report',
+);
+$naa_checked = [];
+is_deeply(
+ resolved({ disk(7, 1) }, { line(1, 7, 'SNETAPP_LUN_825S') }),
+ { 7 => 'SNETAPP_LUN_825S' },
+ 'resolve: an identity of another kind',
+);
+is_deeply($naa_checked, [], 'which is not checked against the VPD page');
+is_deeply(
+ resolved({ disk(0, 1, map => '3W0'), disk(8, 2) }, undef),
+ { 0 => '3W0', 8 => undef },
+ 'resolve: without the daemon only LUNs with a map',
+);
+is_deeply(
+ {
+ map { ("$_->{target} $_->{lun}" => $_->{wwid}) } values PVE::Multipath::Identity::resolve(
+ { disk(9, 1), disk(9, 2, target => $O) },
+ { line(1, 9, '3W9'), line(2, 9, '3X9', target => $O) },
+ $naa_ok,
+ )->%*
+ },
+ { "$T 9" => '3W9', "$O 9" => '3X9' },
+ 'resolve: the same LUN number of another target is another LUN',
+);
+
+for my $case (
+ [{ disk(0, 1) }, undef, 1, 'daemon unavailable'],
+ [{ disk(0, 1) }, {}, 0, 'unlisted path'],
+ [{ disk(0, 1) }, { line(1, 0, '') }, 1, 'listed without identity'],
+ [{ disk(0, 1), disk(0, 2) }, { line(1, 0, 'A'), line(2, 0, 'B') }, 1, 'paths disagree'],
+ [{ disk(0, 1, map => 'A'), disk(0, 2, map => 'B') }, undef, 1, 'maps disagree'],
+ [{ disk(0, 1, map => 'A'), disk(0, 2) }, undef, 0, 'trusted map without daemon'],
+) {
+ my ($disks, $daemon, $expected, $description) = @$case;
+ my ($lun) = values PVE::Multipath::Identity::resolve($disks, $daemon, $naa_ok)->%*;
+ is($lun->{unresolved} // 0, $expected, "uncertainty: $description");
+}
+
+# --- collecting from sysfs ---
+
+my $sys = tempdir(CLEANUP => 1);
+$PVE::Multipath::Identity::SYSFS = $sys;
+
+my sub put {
+ my ($path, $content) = @_;
+ my $file = "$sys/$path";
+ make_path($file =~ s|/[^/]+$||r);
+ open(my $fh, '>', $file) or die "$file: $!\n";
+ print $fh $content;
+ close($fh);
+}
+
+# An iSCSI session as the kernel's transport class shows it: its SCSI target below the session
+# device, which the session class links to.
+my %session_dirs;
+my sub session {
+ my ($sid, $target, $scsi_target) = @_;
+ my ($host) = $scsi_target =~ m/^(\d+):/;
+ my $dir = "devices/platform/host$host/session$sid";
+ make_path("$sys/$dir/target$scsi_target");
+ put("class/iscsi_session/session$sid/targetname", "$target\n");
+ symlink("$sys/$dir", "$sys/class/iscsi_session/session$sid/device") or die "$!\n";
+ $session_dirs{$scsi_target} = "$dir/target$scsi_target";
+}
+
+# A SCSI device, below the target of its session if it has one, listed on the SCSI bus.
+my sub scsi_device {
+ my ($hctl) = @_;
+ my ($scsi_target) = $hctl =~ m/^(\d+:\d+:\d+):\d+$/;
+ my $parent = $session_dirs{$scsi_target} // return;
+ make_path("$sys/$parent/$hctl", "$sys/bus/scsi/devices");
+ symlink("$sys/$parent/$hctl", "$sys/bus/scsi/devices/$hctl")
+ if !-l "$sys/bus/scsi/devices/$hctl";
+}
+
+# a disk as sysfs shows it: below its SCSI device, linked from class/block, with a link back
+my sub disk_device {
+ my ($hctl, $name, $devt) = @_;
+ my $dir = "bus/scsi/devices/$hctl/block/$name";
+ scsi_device($hctl);
+ put("bus/scsi/devices/$hctl/type", "0\n");
+ put("$dir/dev", "$devt\n");
+ make_path("$sys/$dir/holders", "$sys/class/block");
+ unlink("$sys/$dir/device");
+ symlink("$sys/bus/scsi/devices/$hctl", "$sys/$dir/device") or die "$!\n";
+ unlink("$sys/class/block/$name");
+ symlink("$sys/$dir", "$sys/class/block/$name") or die "$!\n";
+}
+
+# the name of a disk going to another SCSI device, as the kernel does when it reuses the name
+my sub move_disk {
+ my ($from, $to, $name, $devt) = @_;
+ remove_tree("$sys/bus/scsi/devices/$from/block/$name");
+ disk_device($to, $name, $devt);
+}
+
+# a map holding a path, linked from the holders of the path
+my sub hold {
+ my ($hctl, $name, $dm, $wwid) = @_;
+ put("devices/virtual/block/$dm/dm/uuid", "mpath-$wwid\n");
+ symlink("$sys/devices/virtual/block/$dm", "$sys/bus/scsi/devices/$hctl/block/$name/holders/$dm")
+ or die "$!\n";
+}
+
+my sub wwids_of {
+ my ($ids) = @_;
+ return {
+ map {
+ my $wwid = PVE::Multipath::Identity::path_wwid($ids, $_);
+ defined($wwid) ? ($_ => $wwid) : ()
+ } qw(sdb sdc sdd sde sdf)
+ };
+}
+
+# session 1 to $T on host 7 with LUN 0 (a map) and LUN 1 (no map), session 2 to $O on host 8
+session(1, $T, '7:0:0');
+session(2, $O, '8:0:0');
+disk_device('7:0:0:0', 'sdb', '8:16');
+disk_device('7:0:0:1', 'sdc', '8:32');
+disk_device('8:0:0:0', 'sdd', '8:48');
+hold('7:0:0:0', 'sdb', 'dm-0', '3600a09803832355367242f747a716b7a');
+# what /sys/block would show for dm-0 if the number went to another map meanwhile
+put('block/dm-0/dm/uuid', "mpath-3DECOY\n");
+put('bus/scsi/devices/7:0:0:1/vpd_pg83', $vpd);
+# a tape drive of the session does not count
+scsi_device('7:0:0:9');
+put('bus/scsi/devices/7:0:0:9/type', "1\n");
+
+my ($queries, $reply) = (0);
+my $multipath = Test::MockModule->new('PVE::Multipath');
+$multipath->mock(
+ query_path_identities => sub {
+ $queries++;
+ die $reply if !ref($reply);
+ return $reply;
+ },
+);
+
+$reply = { line(9, 0, 'unused') };
+my $ids = PVE::Multipath::Identity::collect(targets => { $O => 1 });
+is($queries, 1, 'collect: the daemon is asked for a path without a map');
+$queries = 0;
+hold('8:0:0:0', 'sdd', 'dm-1', '3OTHER');
+$ids = PVE::Multipath::Identity::collect(targets => { $O => 1 });
+is($queries, 0, 'collect: not if every path of the targets has a map');
+is_deeply(wwids_of($ids), { sdd => '3OTHER' }, 'collect: only the LUNs of the targets asked for');
+
+$reply = {
+ sdc => {
+ devt => '8:32',
+ hctl => '7:0:0:1',
+ init => 'ok',
+ target => $T,
+ wwid => '3600a09803832355367242f747a716b7a',
+ },
+};
+$ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+is_deeply(
+ { map { $_->{lun} => $_->{wwid} } PVE::Multipath::Identity::target_luns($ids, $T)->@* },
+ { 0 => '3600a09803832355367242f747a716b7a', 1 => '3600a09803832355367242f747a716b7a' },
+ 'collect: a map, and a path the daemon and the VPD page of the device name alike',
+);
+$reply->{sdc}->{wwid} = '3600a09803832355367242f747a716c30';
+$ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+is_deeply(
+ { map { $_->{lun} => $_->{wwid} } PVE::Multipath::Identity::target_luns($ids, $T)->@* },
+ { 0 => '3600a09803832355367242f747a716b7a', 1 => undef },
+ 'collect: not a path whose VPD page names another LUN',
+);
+is($ids->{paths}->{sdc}->{unresolved}, 1, 'collect: rejected identity marks the path unresolved');
+$reply = {};
+$ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+ok(!$ids->{paths}->{sdc}->{unresolved}, 'collect: genuinely unlisted path not marked unresolved');
+$reply = "multipathd returned an invalid path list\n";
+$ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+is($ids->{paths}->{sdc}->{unresolved}, 1, 'collect: failed daemon marks unmapped path unresolved');
+ok(!$ids->{paths}->{sdb}->{unresolved}, 'collect: trusted map not marked unresolved');
+is($ids->{error}, 'multipathd returned an invalid path list', 'collect: a failed query is kept');
+is_deeply(
+ wwids_of($ids),
+ { sdb => '3600a09803832355367242f747a716b7a' },
+ 'collect: and the LUNs with a map still resolve, through the holder link of their paths',
+);
+
+# a later device under the name of a path: another LUN, the same dev_t
+$reply = {
+ sdc => {
+ devt => '8:32',
+ hctl => '7:0:0:1',
+ init => 'ok',
+ target => $T,
+ wwid => '3600a09803832355367242f747a716b7a',
+ },
+};
+$ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+is(
+ PVE::Multipath::Identity::path_wwid($ids, 'sdc'),
+ '3600a09803832355367242f747a716b7a',
+ 'path: the WWID of its LUN',
+);
+move_disk('7:0:0:1', '7:0:0:5', 'sdc', '8:32');
+is(PVE::Multipath::Identity::path_wwid($ids, 'sdc'), undef, 'path: not once the name is reused');
+is($ids->{paths}->{sdc}->{unresolved}, 1, 'path: changed H:C:T:L marks identity unresolved');
+move_disk('7:0:0:5', '7:0:0:1', 'sdc', '8:32');
+ok(!-e "$sys/bus/scsi/devices/7:0:0:5/block/sdc", 'and the fixture keeps one device per name');
+
+$ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+put('bus/scsi/devices/7:0:0:1/block/sdc/dev', "8:99\n");
+is(PVE::Multipath::Identity::path_wwid($ids, 'sdc'), undef, 'path: changed device number rejected');
+is($ids->{paths}->{sdc}->{unresolved}, 1, 'path: changed device number marks identity unresolved');
+put('bus/scsi/devices/7:0:0:1/block/sdc/dev', "8:32\n");
+
+# the only LUN of a third target: a path with a map, one that the daemon names differently
+my $P = 'iqn.2003-01.org.linux-iscsi.third:sn.3';
+session(3, $P, '9:0:0');
+# a second session to it on the same SCSI host, as offload initiators run them
+session(4, $P, '9:0:1');
+disk_device('9:0:0:0', 'sde', '8:64');
+disk_device('9:0:1:0', 'sdf', '8:80');
+hold('9:0:0:0', 'sde', 'dm-2', '3W9');
+($queries, $reply) = (
+ 0,
+ { sdf => { devt => '8:80', hctl => '9:0:1:0', init => 'ok', target => $P, wwid => '3X9' } },
+);
+$ids = PVE::Multipath::Identity::collect(targets => { $P => 1 });
+is($queries, 1, 'collect: the daemon is asked for a LUN that a map holds on one path only');
+is($ids->{paths}->{$_}->{unresolved}, 1, "collect: disagreeing LUN marks $_ unresolved")
+ for qw(sde sdf);
+is_deeply(
+ wwids_of($ids),
+ {},
+ 'collect: and the LUN unresolved if it names the other path differently',
+);
+
+# two targets through one SCSI host, as offload initiators run them: each LUN stays with the target
+# of its own session
+my ($Q, $R) = ('iqn.2003-01.org.linux-iscsi.fourth:sn.4', 'iqn.2003-01.org.linux-iscsi.fifth:sn.5');
+session(5, $Q, '10:0:0');
+session(6, $R, '10:0:1');
+disk_device('10:0:0:0', 'sdg', '8:96');
+disk_device('10:0:1:0', 'sdh', '8:112');
+hold('10:0:0:0', 'sdg', 'dm-3', '3WQ');
+hold('10:0:1:0', 'sdh', 'dm-4', '3WR');
+
+for my $case ([$Q, '3WQ', 'sdg'], [$R, '3WR', 'sdh']) {
+ my ($target, $wwid, $path) = @$case;
+ $ids = PVE::Multipath::Identity::collect(targets => { $target => 1 });
+ is_deeply(
+ [
+ map { [$_->{wwid}, $_->{paths}] }
+ PVE::Multipath::Identity::target_luns($ids, $target)->@*
+ ],
+ [[$wwid, [$path]]],
+ "collect: a shared SCSI host keeps the LUN of $target with its target",
+ );
+}
+
+{
+ # the daemon command itself fails
+ $multipath->unmock('query_path_identities');
+ $multipath->mock(
+ is_supported => sub { 1 },
+ run_command => sub { die "command 'multipathd show paths' failed: exit code 1\n" },
+ );
+ $ids = PVE::Multipath::Identity::collect(targets => { $P => 1 });
+ like($ids->{error}, qr/exit code 1/, 'collect: a failed daemon command is kept');
+ is_deeply(
+ wwids_of($ids),
+ { sde => '3W9', sdf => '3W9' },
+ 'collect: and the LUN of the map, a path without a line not counting against it',
+ );
+ $multipath->unmock('is_supported', 'run_command');
+ $multipath->mock(
+ query_path_identities => sub { $queries++; die $reply if !ref($reply); return $reply });
+}
+
+# --- the iSCSI plugin ---
+
+$ids = {
+ luns => PVE::Multipath::Identity::resolve(
+ {
+ disk(0, 1, map => '3W0'),
+ disk(0, 2),
+ disk(1, 3),
+ disk(1, 4),
+ disk(2, 5, target => $O),
+ },
+ { line(3, 1, '3W1'), line(4, 1, '3X1') },
+ $naa_ok,
+ ),
+};
+my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+$iscsi->mock(iscsi_portals => sub { ['192.0.2.10:3260', '192.0.2.11:3260'] });
+is_deeply(
+ PVE::Storage::ISCSIPlugin->multipath_lun_wwids({ target => $T, portal => '192.0.2.10' }, $ids),
+ { wwids => { '3W0' => { 'paths-expected' => 2 } } },
+ 'derived: the resolved LUNs, with a path per portal expected',
+);
+
+my $identity = Test::MockModule->new('PVE::Multipath::Identity');
+$identity->mock(
+ collect => sub { $ids },
+ path_wwid => sub {
+ my ($ids, $name) = @_;
+ my ($lun) = grep {
+ grep { $_ eq $name } $_->{paths}->@*
+ } values $ids->{luns}->%*;
+ return $lun ? $lun->{wwid} : undef;
+ },
+);
+$iscsi->mock(
+ iscsi_session_device_map => sub {
+ my $disks =
+ { disk(0, 1), disk(0, 2), disk(1, 3), disk(1, 4), disk(2, 5, target => $O) };
+ return {
+ map {
+ $_ => {
+ target => $disks->{$_}->{target},
+ channel => 0,
+ id => 0,
+ lun => $disks->{$_}->{lun},
+ }
+ }
+ keys %$disks
+ };
+ },
+ load_stable_scsi_paths => sub { {} },
+ file_read_firstline => sub { $_[0] =~ m|^/sys/block/\w+/size$| ? 2048 : undef },
+);
+is_deeply(
+ [sort keys PVE::Storage::ISCSIPlugin::iscsi_device_list(1, $T)->{$T}->%*],
+ ['0.0.0.dm-uuid-mpath-3W0'],
+ 'volumes: every path of a resolved LUN named after its map, an unresolved LUN left out',
+);
+
+my $collected = 0;
+$identity->mock(collect => sub { $collected++; return { luns => {} } });
+my $derived = PVE::Storage::derived_multipath_wwids({
+ ids => {
+ a => { type => 'iscsi', target => $T, portal => '192.0.2.10', multipath => 1 },
+ b => { type => 'iscsi', target => $O, portal => '192.0.2.20', multipath => 1 },
+ c => { type => 'iscsi', target => $O, portal => '192.0.2.20' },
+ },
+});
+is_deeply([sort keys %$derived], ['a', 'b'], 'derived: every storage with the option');
+is($collected, 1, 'derived: the identities are collected once for all of them');
+
+done_testing();
diff --git a/src/test/run_multipath_api_tests.pl b/src/test/run_multipath_api_tests.pl
new file mode 100755
index 00000000..c6fdfb7c
--- /dev/null
+++ b/src/test/run_multipath_api_tests.pl
@@ -0,0 +1,92 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib ('.', '..');
+use Test::More;
+use Test::MockModule;
+
+use PVE::Multipath;
+use PVE::Storage;
+use PVE::Storage::ISCSIPlugin;
+use PVE::Multipath::Status;
+
+subtest 'enabling the multipath option needs every node to know it' => sub {
+ my $cluster = Test::MockModule->new('PVE::Cluster');
+ my $inotify = Test::MockModule->new('PVE::INotify');
+ $inotify->mock(nodename => sub { 'n1' });
+ $cluster->mock(get_nodelist => sub { ['n1', 'n2', 'n3'] });
+ my $kv = { n2 => '{"supported":1}' };
+ $cluster->mock(get_node_kv => sub { +{%$kv} });
+ my $mp = Test::MockModule->new('PVE::Multipath');
+ $mp->mock(is_supported => sub { 1 });
+
+ is_deeply(
+ PVE::Multipath::Status::nodes_without_status(),
+ ['n3'],
+ 'the local node is not waited for',
+ );
+ my $storage = Test::MockModule->new('PVE::Storage');
+ $storage->mock(config => sub { +{ ids => {} } });
+ my $enable = sub {
+ my ($scfg, $update, $delete) = @_;
+ eval {
+ PVE::Storage::ISCSIPlugin->on_update_hook_full(
+ 'san',
+ { type => 'iscsi', %$scfg },
+ $update,
+ $delete // [],
+ {},
+ );
+ };
+ return $@;
+ };
+ like(
+ $enable->({}, { multipath => 1 }),
+ qr/node\(s\) n3 did not report/,
+ 'enabling is refused',
+ );
+ eval { PVE::Storage::ISCSIPlugin->on_add_hook('san', { type => 'iscsi', multipath => 1 }) };
+ like($@, qr/did not report/, 'as is adding a storage with it');
+ is($enable->({}, { content => 'none' }), '', 'other changes are not affected');
+
+ $kv->{n3} = '{"supported":0}';
+ like(
+ $enable->({}, { multipath => 1 }),
+ qr/multipath-tools is not installed on node\(s\) n3/,
+ 'a node of the storage without multipath-tools refuses it too',
+ );
+ is(
+ $enable->({ nodes => { n1 => 1, n2 => 1 } }, { multipath => 1 }),
+ '',
+ 'unless the storage is not available on that node',
+ );
+ like(
+ $enable->({ nodes => { n1 => 1 } }, { multipath => 1 }, ['nodes']),
+ qr/not installed on node\(s\) n3/,
+ 'dropping the restriction in the same update counts',
+ );
+ my $widen = sub {
+ my ($update, $delete) = @_;
+ return $enable->({ multipath => 1, nodes => { n1 => 1, n2 => 1 } }, $update, $delete);
+ };
+ like(
+ $widen->({ nodes => { n1 => 1, n3 => 1 } }),
+ qr/not installed on node\(s\) n3/,
+ 'an enabled storage cannot be extended to a node without the tools',
+ );
+ like($widen->({}, ['nodes']), qr/n3/, 'nor lose its node restriction');
+ is($widen->({ nodes => { n1 => 1 } }), '', 'restricting it further is fine');
+
+ $kv->{n3} = '{"supported":1}';
+ is($enable->({}, { multipath => 1 }), '', 'once every node supports it, it can be enabled');
+ $kv->{n3} = 'not json';
+ is_deeply(
+ PVE::Multipath::Status::nodes_without_status(),
+ ['n3'],
+ 'a broken status does not count',
+ );
+};
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 30/76] lvm: activate a missing base volume before using the volume group
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (28 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 29/76] iscsi: add an option to manage storage LUNs with multipath Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 31/76] multipath: apply the configuration on every node Thomas Lamprecht
` (45 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Activating LVM storage activates its base storage, but not the base
volume. After login, that device may still be appearing, for example
while multipathd assembles its map. Guest startup then fails to find the
volume group even though the device arrives shortly afterward.
If the group is missing, activate the base volume to wait for its
device, then retry the lookup, bypassing LVM hints if necessary. Skip
this wait during status collection so an absent device cannot stall
every status pass. Activation of an already visible group is unchanged.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Storage/LVMPlugin.pm | 30 ++++++
src/test/Makefile | 5 +-
src/test/run_test_lvm_base_activation.pl | 130 +++++++++++++++++++++++
3 files changed, 164 insertions(+), 1 deletion(-)
create mode 100755 src/test/run_test_lvm_base_activation.pl
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index 11a64c97..ed7cdb8c 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -1604,6 +1604,32 @@ my sub find_unhinted_vgs {
delete $cache->{lvs};
}
+# The base device may still be appearing after transport activation; wait before retrying LVM.
+my sub activate_missing_base {
+ my ($storeid, $scfg, $cache) = @_;
+
+ # the volume group list of the cache can predate the device
+ $cache->{vgs} = lvm_vgs();
+ return if $cache->{vgs}->{ $scfg->{vgname} };
+
+ eval {
+ my $cfg = PVE::Storage::config();
+ my ($baseid, $volname) = PVE::Storage::Plugin::parse_volume_id($scfg->{base});
+ my $basecfg = PVE::Storage::storage_config($cfg, $baseid);
+ my $plugin = PVE::Storage::Plugin->lookup($basecfg->{type});
+ # the activation of this storage activated the base storage with the same cache before
+ $plugin->activate_volume($baseid, $basecfg, $volname, undef, $cache);
+ };
+ warn "storage '$storeid': activating base volume '$scfg->{base}' failed - $@" if $@;
+
+ # A new device invalidates LVM's hints, but a PV written remotely to an existing device does
+ # not; retry without hints after each missing-base activation.
+ $cache->{vgs} = lvm_vgs();
+ # the logical volume list of the cache can predate the volume group too
+ delete $cache->{lvs};
+ find_unhinted_vgs($cache) if !$cache->{vgs}->{ $scfg->{vgname} };
+}
+
sub activate_storage {
my ($class, $storeid, $scfg, $cache) = @_;
@@ -1614,6 +1640,10 @@ sub activate_storage {
sub {
$cache->{vgs} = lvm_vgs(undef, undef, $cache) if !$cache->{vgs};
+ # Waiting for a missing base must not stall periodic status collection.
+ activate_missing_base($storeid, $scfg, $cache)
+ if $scfg->{base} && !$cache->{status_only} && !$cache->{vgs}->{ $scfg->{vgname} };
+
find_unhinted_vgs($cache)
if !$cache->{vgscaned} && !$cache->{vgs}->{ $scfg->{vgname} };
},
diff --git a/src/test/Makefile b/src/test/Makefile
index 97b28929..97d9db4d 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_iscsi test_lun_identity test_multipath_api test_san_api test_san_libraries
+test: test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_base_activation test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_iscsi test_lun_identity test_multipath_api test_san_api test_san_libraries
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -23,6 +23,9 @@ test_lvm_base_volume: run_test_lvm_base_volume.pl
test_lvm_hints: run_test_lvm_hints.pl
./run_test_lvm_hints.pl
+test_lvm_base_activation: run_test_lvm_base_activation.pl
+ ./run_test_lvm_base_activation.pl
+
test_lvm_status: run_test_lvm_status.pl
./run_test_lvm_status.pl
diff --git a/src/test/run_test_lvm_base_activation.pl b/src/test/run_test_lvm_base_activation.pl
new file mode 100755
index 00000000..4d448163
--- /dev/null
+++ b/src/test/run_test_lvm_base_activation.pl
@@ -0,0 +1,130 @@
+#!/usr/bin/perl
+
+# Test the activation of an LVM storage whose volume group sits on a base volume of another storage.
+# If the volume group is missing, activation activates the base volume, waits for a device that is
+# still appearing, and looks for the volume group again, without the LVM device hints if needed.
+# Status-only activation skips the base volume and only lists without the hints. vgs, pvscan and the
+# base plugin are mocked, so the test needs neither root nor a real LVM setup.
+
+use strict;
+use warnings;
+
+use lib '..';
+use Test::More;
+use Test::MockModule;
+
+use PVE::Storage;
+use PVE::Storage::LVMPlugin;
+
+my $CLASS = 'PVE::Storage::LVMPlugin';
+
+# the volume groups vgs lists through the device hints, those only a listing without them finds
+my ($vgs, $unhinted, @calls) = ([], []);
+
+my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
+$lvm->mock(
+ lvm_vgs => sub {
+ my ($includepvs, $nohints) = @_;
+ push @calls, $nohints ? 'vgs --nohints' : 'vgs';
+ return { map { $_ => {} } $vgs->@*, ($nohints ? $unhinted->@* : ()) };
+ },
+);
+my $status_command = Test::MockModule->new('PVE::Storage::StatusCommand');
+$status_command->mock(
+ run_for_cache => sub {
+ my ($cache, $cmd) = @_;
+ push @calls, join(' ', $cmd->[0] =~ s|^/sbin/||r, $cmd->@[1 .. $#$cmd]);
+ },
+);
+
+my $storage = Test::MockModule->new('PVE::Storage');
+$storage->mock(config => sub { return { ids => {} } });
+$storage->mock(storage_config => sub { return { type => 'fakebase' } });
+
+my $base_fails;
+my $plugin = Test::MockModule->new('PVE::Storage::Plugin');
+$plugin->mock(lookup => sub { return 'FakeBasePlugin' });
+
+{
+ no warnings 'once';
+ *FakeBasePlugin::activate_volume = sub {
+ my ($class, $storeid, $scfg, $volname, $snapname, $cache) = @_;
+ push @calls, "base $storeid:$volname" . ($cache->{marker} ? ' shared-cache' : '');
+ die "volume '$volname' is not available on this node\n" if $base_fails;
+ # the device appeared while waiting, and with it the volume group
+ push $vgs->@*, $volname =~ s/^vol-//r;
+ };
+}
+
+my sub activate {
+ my ($scfg, $cache, $vgs_now, $unhinted_now) = @_;
+
+ ($vgs, $unhinted, @calls) = ($vgs_now // [], $unhinted_now // []);
+ my @warnings;
+ local $SIG{__WARN__} = sub { push @warnings, $_[0] };
+ $CLASS->activate_storage('lvm', $scfg, $cache);
+ return (join(', ', @calls), \@warnings);
+}
+
+my $with_base = { vgname => 'vg1', base => 'san:vol-vg1' };
+
+my ($calls, $warnings) = activate($with_base, { vgs => { vg1 => {} } });
+is($calls, '', 'present volume group: no base activation, no scan');
+
+($calls) = activate($with_base, { status_only => 1, vgs => {} });
+is($calls, 'vgs --nohints',
+ 'status collection: only the listing without hints, no base activation');
+
+($calls) = activate($with_base, { vgs => {} }, ['vg1']);
+is($calls, 'vgs', 'volume group appeared meanwhile: a fresh list, no base activation');
+
+my $cache = { marker => 1, vgs => {}, lvs => {} };
+($calls) = activate($with_base, $cache);
+is(
+ $calls,
+ 'vgs, base san:vol-vg1 shared-cache, vgs',
+ 'missing volume group: base activated with the shared cache, the device brings it into the list',
+);
+ok($cache->{vgs}->{vg1}, 'and the volume group list of the cache is refreshed');
+ok(!exists($cache->{lvs}), 'and its logical volume list is dropped');
+
+# the volume group another node created on a device this node had before, after an earlier probe
+$cache = { vgs => {}, vgscaned => 1 };
+{
+ no warnings 'redefine';
+ local *FakeBasePlugin::activate_volume = sub { push @calls, 'base present' };
+ ($calls) = activate($with_base, $cache, [], ['vg1']);
+}
+is(
+ $calls,
+ 'vgs, base present, vgs, vgs --nohints, pvscan --cache',
+ 'volume group missing from the hints: found without them despite an earlier probe',
+);
+ok($cache->{vgs}->{vg1}, 'and added to the volume group list of the cache');
+
+$cache = { vgs => {} };
+activate($with_base, $cache);
+($calls) = activate({ vgname => 'vg2', base => 'san:vol-vg2' }, $cache, ['vg1']);
+is($calls, 'vgs, base san:vol-vg2, vgs', 'a second base in the same cache is activated');
+
+$base_fails = 1;
+($calls, $warnings) = activate($with_base, { vgs => {} });
+is(
+ $calls,
+ 'vgs, base san:vol-vg1, vgs, vgs --nohints',
+ 'a failed base activation still looks for the volume group',
+);
+like(
+ $warnings->[0] // '',
+ qr/^storage 'lvm': activating base volume 'san:vol-vg1' failed - volume/,
+ 'and warns about it',
+);
+$base_fails = 0;
+
+$cache = { vgs => {} };
+($calls) = activate({ vgname => 'vg1' }, $cache);
+is($calls, 'vgs --nohints', 'without base: one listing without hints per cache');
+($calls) = activate({ vgname => 'vg3' }, $cache);
+is($calls, '', 'and not again for the next storage');
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 31/76] multipath: apply the configuration on every node
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (29 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 30/76] lvm: activate a missing base volume before using the volume group Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 32/76] iscsi: share the initiator identity reader Thomas Lamprecht
` (44 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Each node needs local multipath-tools configuration and allow-list
entries for its explicitly managed and storage-derived LUNs. Add a
generator for periodic use by the status daemon. It writes drop-ins,
updates the allow-list through multipath, and reloads multipathd,
leaving nodes that do not use the feature unchanged.
Track which entries Proxmox VE added and why. Preserve entries added
by hand or for boot from SAN. Remove managed entries only when no
configuration needs them, not when a target is temporarily unreachable.
Keep maps with local users allow-listed to avoid falling back to single
paths after reboot. Read signatures from the device, since udev may not
know about volume groups created by other nodes. Unreadable maps and
maps without usable paths count as in use, as do maps held open by a
process, even without signatures or holders. The paths can fail during
the read, so it runs as a bounded status command that cannot stall the
status daemon if the map queues I/O.
Record a pending reload before changing anything so failures are retried
even if the configuration files are already up to date.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Makefile | 1 +
src/PVE/Multipath.pm | 145 +++++++++++++-
src/PVE/Multipath/Generator.pm | 294 ++++++++++++++++++++++++++++
src/test/Makefile | 5 +-
src/test/run_lun_identity_tests.pl | 1 +
src/test/run_multipath_api_tests.pl | 192 ++++++++++++++++++
src/test/run_multipath_tests.pl | 247 +++++++++++++++++++++++
src/test/run_san_library_tests.pl | 6 +
src/test/run_san_taint_tests.pl | 80 ++++++++
9 files changed, 968 insertions(+), 3 deletions(-)
create mode 100644 src/PVE/Multipath/Generator.pm
create mode 100755 src/test/run_san_taint_tests.pl
diff --git a/src/PVE/Makefile b/src/PVE/Makefile
index 29d0146d..4edc609d 100644
--- a/src/PVE/Makefile
+++ b/src/PVE/Makefile
@@ -7,6 +7,7 @@ install:
install -D -m 0644 Multipath.pm ${DESTDIR}${PERLDIR}/PVE/Multipath.pm
install -D -m 0644 Multipath/Config.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Config.pm
install -D -m 0644 Multipath/ClusterConfig.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/ClusterConfig.pm
+ install -D -m 0644 Multipath/Generator.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Generator.pm
install -D -m 0644 Multipath/Identity.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Identity.pm
install -D -m 0644 Multipath/Status.pm ${DESTDIR}${PERLDIR}/PVE/Multipath/Status.pm
install -D -m 0644 CephConfig.pm ${DESTDIR}${PERLDIR}/PVE/CephConfig.pm
diff --git a/src/PVE/Multipath.pm b/src/PVE/Multipath.pm
index e2f7ea40..96f1e84d 100644
--- a/src/PVE/Multipath.pm
+++ b/src/PVE/Multipath.pm
@@ -2,11 +2,13 @@ package PVE::Multipath;
use v5.36;
-use JSON;
+use PVE::Diskmanage;
-use PVE::Tools qw(run_command file_read_firstline);
+use PVE::Tools qw(run_command file_read_firstline file_get_contents file_set_contents);
+use PVE::Multipath::Config;
use PVE::RS::Multipath;
+use PVE::Storage::StatusCommand;
my $MULTIPATH = '/sbin/multipath';
my $MULTIPATHD = '/sbin/multipathd';
@@ -122,6 +124,14 @@ my sub dm_size_bytes($sysfs) {
return int($sectors) * 512;
}
+my sub dir_has_entries($dir) {
+ return 0 if !-d $dir;
+ opendir(my $dh, $dir) or return 0;
+ my @entries = grep { $_ ne '.' && $_ ne '..' } readdir($dh);
+ closedir($dh);
+ return scalar(@entries) ? 1 : 0;
+}
+
my sub path_transport($dev) {
return undef if !$dev;
my $link = readlink("/sys/block/$dev");
@@ -201,4 +211,135 @@ sub wait_for_map($wwid, $timeout = undef) {
}
}
+my $WWIDS_FILE = '/etc/multipath/wwids';
+
+# The WWIDs in /etc/multipath/wwids, the on-disk allow-list that multipathd assembles from with
+# 'find_multipaths strict'. Distinct from the cluster config's desired set (Config::wwid_list).
+sub list_etc_multipath_wwids() {
+ return [] if !-e $WWIDS_FILE;
+ return PVE::Multipath::Config::parse_wwids(file_get_contents($WWIDS_FILE));
+}
+
+# multipath reads other arguments as options or device numbers, see Config::WWID_PATTERN.
+my sub assert_wwid($wwid) {
+ die "invalid WWID '$wwid'\n" if $wwid !~ $PVE::Multipath::Config::WWID_RE;
+}
+
+sub add_wwid($wwid) {
+ assert_supported();
+ assert_wwid($wwid);
+ run_command([$MULTIPATH, '-a', '--', $wwid], timeout => $TOOL_TIMEOUT);
+}
+
+sub remove_wwid($wwid) {
+ assert_supported();
+ assert_wwid($wwid);
+ run_command([$MULTIPATH, '-w', '--', $wwid], timeout => $TOOL_TIMEOUT);
+}
+
+# Read the device: udev can miss another node's writes. Without active paths, I/O can queue
+# indefinitely; only a successful empty probe proves non-use. Removing an in-use map's allow-list
+# entry could expose its PV through a single path after reboot. The paths can still fail during the
+# read, so it runs as a status command, which leaves a reader stuck in I/O behind instead of
+# waiting for it. Probes sharing $cache share its time budget.
+sub map_user($map, $cache = undef) {
+ my ($dev) = ($map->{sysfs} // '') =~ m/\A(dm-\d+)\z/;
+ return 'unknown' if !defined($dev);
+ return 'Device Mapper' if dir_has_entries("/sys/block/$dev/holders");
+ return 'unknown' if !$map->{'paths-active'};
+
+ my $open = '';
+ eval {
+ run_command(
+ ['dmsetup', 'info', '-c', '--noheadings', '-o', 'open', "/dev/$dev"],
+ timeout => $QUERY_TIMEOUT,
+ outfunc => sub($line) { $open .= "$line\n" },
+ errfunc => sub($line) { },
+ );
+ };
+ return 'unknown' if $@ || $open !~ m/^\s*(\d+)\s*$/;
+ return 'open device' if $1 > 0;
+
+ my $output = '';
+ eval {
+ local $PVE::Storage::StatusCommand::BUDGET = $QUERY_TIMEOUT;
+ PVE::Storage::StatusCommand::run(
+ $cache // {},
+ ['/sbin/wipefs', '--no-act', '--json', "/dev/$dev"],
+ outfunc => sub($line) { $output .= "$line\n" },
+ );
+ };
+ return 'unknown' if $@;
+ my $signatures = eval { PVE::Diskmanage::parse_wipefs_signatures($output) };
+ return 'unknown' if $@;
+
+ my %types = map { $_->{type} => 1 } $signatures->@*;
+ return undef if !%types;
+ return 'LVM' if $types{LVM2_member};
+ return 'ZFS' if $types{zfs_member};
+ return 'partitions' if grep { $types{$_} } qw(gpt PMBR dos);
+ return (sort keys %types)[0];
+}
+
+# Avoid repeatedly reading the SAN while removal of an in-use map is pending.
+our $IN_USE_RECHECK_SECONDS = 60;
+my $in_use_seen = {};
+
+# { wwid => user } for those of the given WWIDs whose map is assembled on this node and in use, see
+# map_user(). Every WWID counts as in use when the maps cannot be queried.
+sub maps_in_use($wwids) {
+ my $maps = eval { get_maps() };
+ if (my $err = $@) {
+ warn "multipath: querying the maps failed - $err";
+ return { map { $_ => 'unknown' } $wwids->@* };
+ }
+ my %map_of = map { $_->{wwid} => $_ } grep { defined($_->{wwid}) } $maps->@*;
+
+ my $now = time();
+ my $in_use = {};
+ # one time budget for all probes, so a pass does not grow with the number of maps
+ my $probes = { 'status-command-budget' => $QUERY_TIMEOUT };
+ for my $wwid ($wwids->@*) {
+ my $map = $map_of{$wwid} // next;
+ my $seen = $in_use_seen->{$wwid};
+ my $user;
+ if ($seen && $now - $seen->{time} < $IN_USE_RECHECK_SECONDS) {
+ $user = $seen->{user};
+ } else {
+ $user = map_user($map, $probes);
+ $in_use_seen->{$wwid} = { time => $now, user => $user } if defined($user);
+ }
+ next if !defined($user);
+ $in_use->{$wwid} = $user;
+ }
+ delete $in_use_seen->{$_} for grep { !$in_use->{$_} } keys %$in_use_seen;
+ return $in_use;
+}
+
+# Track only PVE-added entries so pruning never removes hand-managed or boot-from-SAN WWIDs.
+# Removal waits until every ownership reason disappears.
+my $MANAGED_WWIDS_FILE = '/etc/multipath/wwids.pve';
+
+# The ownership record, { wwid => [sources] } (empty if Proxmox VE added nothing yet).
+sub managed_wwid_record() {
+ return {} if !-e $MANAGED_WWIDS_FILE;
+ return PVE::Multipath::Config::parse_managed_wwids(file_get_contents($MANAGED_WWIDS_FILE));
+}
+
+sub set_managed_wwid_record($record) {
+ if (!%$record) {
+ if (-e $MANAGED_WWIDS_FILE) {
+ unlink($MANAGED_WWIDS_FILE)
+ or die "could not remove '$MANAGED_WWIDS_FILE' - $!\n";
+ }
+ return;
+ }
+ file_set_contents($MANAGED_WWIDS_FILE, PVE::Multipath::Config::format_managed_wwids($record));
+}
+
+sub reconfigure() {
+ assert_supported();
+ run_command([$MULTIPATHD, 'reconfigure'], timeout => $RECONFIGURE_TIMEOUT);
+}
+
1;
diff --git a/src/PVE/Multipath/Generator.pm b/src/PVE/Multipath/Generator.pm
new file mode 100644
index 00000000..b7aec227
--- /dev/null
+++ b/src/PVE/Multipath/Generator.pm
@@ -0,0 +1,294 @@
+package PVE::Multipath::Generator;
+
+use v5.36;
+
+use File::Path qw(make_path);
+
+use PVE::Tools qw(file_get_contents file_set_contents);
+
+use PVE::Multipath;
+use PVE::Multipath::Config;
+use PVE::Multipath::ClusterConfig;
+
+# Keep rendered files local so multipathd can start before pmxcfs, using the last successful render.
+# multipath rejects duplicate section keywords within one file. Separate baseline, overrides and
+# aliases so they merge across files; overrides must sort after the baseline.
+our $DEFAULTS_DROPIN = '/etc/multipath/conf.d/pve-defaults.conf';
+our $OVERRIDES_DROPIN = '/etc/multipath/conf.d/pve-overrides.conf';
+our $ALIASES_DROPIN = '/etc/multipath/conf.d/pve-aliases.conf';
+
+# Set before changing files and clear only after a successful reload, so partial passes and
+# failed reloads retry even when files already match. Reboot loads the files without this marker.
+our $RECONFIGURE_PENDING = '/run/pve-multipath.reconfigure-pending';
+
+my sub mark_reconfigure_pending() {
+ file_set_contents($RECONFIGURE_PENDING, '') if !-e $RECONFIGURE_PENDING;
+}
+
+my sub write_if_changed($path, $content) {
+ my $old = -e $path ? eval { file_get_contents($path) } : undef;
+ return 0 if defined($old) && $old eq $content;
+
+ mark_reconfigure_pending();
+ my $dir = $path =~ s!/[^/]+$!!r;
+ make_path($dir) if !-d $dir;
+ file_set_contents($path, $content);
+ return 1;
+}
+
+my sub remove_if_present($path) {
+ return 0 if !-e $path;
+ mark_reconfigure_pending();
+ unlink($path) or die "could not remove '$path' - $!\n";
+ return 1;
+}
+
+# Derived storages need the baseline even without explicit sections; remove unused generated files
+# so hand-managed configuration takes over.
+my sub config_in_use($cfg, $overrides, $derived, $node) {
+ return
+ scalar(PVE::Multipath::Config::wwids_for_node($cfg, $node)->@*)
+ || (defined($overrides) && length($overrides))
+ || $cfg->{ids}->{defaults}
+ || scalar(keys(($derived // {})->%*));
+}
+
+# Retain owned storage entries while their storage is enabled here: an outage or unsettled scan
+# is not evidence that the LUN was withdrawn. Only configuration changes remove ownership reasons.
+sub plan_wwid_changes($explicit, $derived, $current, $managed) {
+ my %desired;
+ push $desired{$_}->@*, 'config' for keys %$explicit;
+ for my $storeid (sort keys %$derived) {
+ push $desired{$_}->@*, "storage:$storeid" for keys $derived->{$storeid}->{wwids}->%*;
+ }
+ for my $wwid (keys %$managed) {
+ my %have = map { $_ => 1 } ($desired{$wwid} // [])->@*;
+ for my $source ($managed->{$wwid}->@*) {
+ next if $have{$source};
+ my ($storeid) = $source =~ m/^storage:(.+)$/;
+ push $desired{$wwid}->@*, $source if defined($storeid) && $derived->{$storeid};
+ }
+ }
+ $desired{$_} = [sort $desired{$_}->@*] for keys %desired;
+
+ my @to_add = sort grep { !$current->{$_} } keys %desired;
+ my @to_remove = sort grep { !$desired{$_} && $current->{$_} } keys %$managed;
+ return (\@to_add, \@to_remove, \%desired);
+}
+
+# Do not claim pre-existing foreign entries: removing a storage must not prune hand-managed WWIDs.
+# Pending or failed removals must retain their recorded reasons for retry.
+sub owned_wwid_record($desired, $managed, $current, $keep) {
+ my %owned = map { $_ => $managed->{$_} } grep { defined($managed->{$_}) } keys %$keep;
+ for my $wwid (keys %$desired) {
+ $owned{$wwid} = $desired->{$wwid} if $managed->{$wwid} || !$current->{$wwid};
+ }
+ return \%owned;
+}
+
+sub wwid_records_differ($a, $b) {
+ my $flat = sub($record) {
+ join("\0", map { "$_=" . join(',', sort $record->{$_}->@*) } sort keys $record->%*);
+ };
+ return $flat->($a) ne $flat->($b);
+}
+
+# Storages can expose a LUN through different portal sets, so judge map health against the largest
+# expected path count.
+sub desired_wwid_info($desired, $derived) {
+ my $wwids = {};
+ for my $wwid (keys %$desired) {
+ my $info = {};
+ for my $source ($desired->{$wwid}->@*) {
+ my ($storeid) = $source =~ m/^storage:(.+)$/ or next;
+ push $info->{storages}->@*, $storeid;
+ my $luns = ($derived->{$storeid} // {})->{wwids} // {};
+ my $expected = ($luns->{$wwid} // {})->{'paths-expected'};
+ $info->{'paths-expected'} = $expected
+ if defined($expected) && $expected > ($info->{'paths-expected'} // 0);
+ }
+ $wwids->{$wwid} = $info;
+ }
+ return $wwids;
+}
+
+# Collect per-operation errors instead of losing the WWID state needed by health reports.
+sub regenerate($cfg = undef, $overrides = undef, $derived = undef) {
+ $cfg //= PVE::Multipath::ClusterConfig::read_config();
+ $overrides //= PVE::Multipath::ClusterConfig::read_overrides();
+ $derived //= {};
+
+ require PVE::INotify;
+ my $node = PVE::INotify::nodename();
+
+ my $changed = 0;
+
+ if (config_in_use($cfg, $overrides, $derived, $node)) {
+ my $defaults = PVE::Multipath::Config::effective_defaults($cfg);
+ $changed = 1
+ if write_if_changed(
+ $DEFAULTS_DROPIN,
+ PVE::Multipath::Config::generate_managed_conf($defaults),
+ );
+ } else {
+ $changed = 1 if remove_if_present($DEFAULTS_DROPIN);
+ }
+
+ if (defined($overrides) && length($overrides)) {
+ my $content =
+ "# Managed by Proxmox VE - edit overrides in /etc/pve/multipath-overrides.conf.\n\n"
+ . "$overrides\n";
+ $changed = 1 if write_if_changed($OVERRIDES_DROPIN, $content);
+ } else {
+ $changed = 1 if remove_if_present($OVERRIDES_DROPIN);
+ }
+
+ my $block =
+ PVE::Multipath::Config::build_multipaths_block(PVE::Multipath::Config::wwid_opts($cfg));
+ if (length($block)) {
+ my $content =
+ "# Managed by Proxmox VE - edit aliases and per-LUN options in /etc/pve/multipath.cfg.\n\n"
+ . $block;
+ $changed = 1 if write_if_changed($ALIASES_DROPIN, $content);
+ } else {
+ $changed = 1 if remove_if_present($ALIASES_DROPIN);
+ }
+
+ # Use multipath's own allow-list writer. Isolate operations so a failing WWID cannot prevent
+ # the others from converging; failed operations are retried on the next pass.
+ my %explicit = map { $_ => 1 } PVE::Multipath::Config::wwids_for_node($cfg, $node)->@*;
+ my %current = map { $_ => 1 } PVE::Multipath::list_etc_multipath_wwids()->@*;
+ my $managed = PVE::Multipath::managed_wwid_record();
+ my ($to_add, $to_remove, $desired) =
+ plan_wwid_changes(\%explicit, $derived, \%current, $managed);
+
+ my @errors;
+ for my $storeid (sort keys %$derived) {
+ my $err = $derived->{$storeid}->{error} // next;
+ push @errors, "identifying the LUN WWIDs of storage '$storeid' failed - $err";
+ }
+
+ # Keep ownership while a map is in use, so removal can be retried after release.
+ my $in_use = scalar($to_remove->@*) ? PVE::Multipath::maps_in_use($to_remove) : {};
+ my @removable = grep { !$in_use->{$_} } $to_remove->@*;
+
+ # Record ownership before adding: a successful add followed by a failed record write would
+ # leak an unprunable entry. Keep pending removals owned until the entries are actually gone.
+ my %pending = map { $_ => 1 } $to_remove->@*;
+ my $record = owned_wwid_record($desired, $managed, \%current, \%pending);
+ my $record_ok = 1;
+ if (wwid_records_differ($record, $managed)) {
+ eval { PVE::Multipath::set_managed_wwid_record($record); };
+ if (my $err = $@) {
+ push @errors, "recording managed WWIDs failed - $err";
+ $record_ok = 0;
+ $record = $managed;
+ }
+ }
+
+ mark_reconfigure_pending() if scalar($to_add->@*) || scalar(@removable);
+ if ($record_ok) {
+ for my $wwid ($to_add->@*) {
+ eval { PVE::Multipath::add_wwid($wwid); };
+ if (my $err = $@) {
+ push @errors, "adding WWID '$wwid' failed - $err";
+ } else {
+ $changed = 1;
+ }
+ }
+ }
+ my %kept = map { $_ => 1 } keys %$in_use;
+ for my $wwid (@removable) {
+ eval { PVE::Multipath::remove_wwid($wwid); };
+ if (my $err = $@) {
+ push @errors, "removing WWID '$wwid' failed - $err";
+ $kept{$wwid} = 1;
+ } else {
+ $changed = 1;
+ }
+ }
+
+ # A failed ownership release is safe: absent entries leave the record on the next pass.
+ if (scalar(@removable)) {
+ my $final = owned_wwid_record($desired, $managed, \%current, \%kept);
+ if (wwid_records_differ($final, $record)) {
+ eval { PVE::Multipath::set_managed_wwid_record($final); };
+ push @errors, "recording managed WWIDs failed - $@" if $@;
+ }
+ }
+
+ # Reload successful changes even if other operations failed; retain the marker until success.
+ if (-e $RECONFIGURE_PENDING && PVE::Multipath::is_running()) {
+ eval { PVE::Multipath::reconfigure(); };
+ if (my $err = $@) {
+ push @errors, "reconfigure failed - $err";
+ } else {
+ unlink($RECONFIGURE_PENDING);
+ }
+ }
+
+ my $wwids = desired_wwid_info($desired, $derived);
+ $wwids->{$_} = { kept => $in_use->{$_} } for keys %$in_use;
+
+ my $error = join('', map { $_ =~ m/\n\z/ ? $_ : "$_\n" } @errors);
+
+ return {
+ changed => $changed,
+ wwids => $wwids,
+ length($error) ? (error => $error) : (),
+ };
+}
+
+# On config-read failure, retain WWIDs from the ownership record so managed LUNs remain visible.
+sub sync() {
+ my $record = eval { PVE::Multipath::managed_wwid_record() };
+ return { wwids => {}, error => "reading managed WWIDs failed - $@" } if $@;
+ my $state = { wwids => desired_wwid_info($record, {}) };
+
+ my $cfg = eval { PVE::Multipath::ClusterConfig::read_config() };
+ return { $state->%*, error => "reading cluster config failed - $@" } if $@;
+
+ require PVE::INotify;
+ my $node = PVE::INotify::nodename();
+ my %explicit = map { $_ => 1 } PVE::Multipath::Config::wwids_for_node($cfg, $node)->@*;
+ $state->{wwids}->{$_} //= {} for keys %explicit;
+
+ # Without storage.cfg, an empty derived set could incorrectly prune storage-owned WWIDs.
+ my $derived = eval {
+ require PVE::Storage;
+ PVE::Storage::derived_multipath_wwids(PVE::Storage::config());
+ };
+ return { $state->%*, error => "reading the storage configuration failed - $@" } if $@;
+
+ my (undef, undef, $desired) =
+ plan_wwid_changes(\%explicit, $derived, {}, $record);
+ $state->{wwids} = desired_wwid_info($desired, $derived);
+ my $overrides = eval { PVE::Multipath::ClusterConfig::read_overrides() };
+ return { $state->%*, error => "reading cluster overrides failed - $@" } if $@;
+
+ my $in_use = config_in_use($cfg, $overrides, $derived, $node);
+
+ if (!PVE::Multipath::is_supported()) {
+ return $state if !scalar(keys %$derived) && !%explicit;
+ return { $state->%*, error => "multipath-tools is not installed\n" };
+ }
+
+ # Leftover generated files or ownership records require another teardown pass.
+ return $state
+ if !$in_use
+ && !-e $DEFAULTS_DROPIN
+ && !-e $OVERRIDES_DROPIN
+ && !-e $ALIASES_DROPIN
+ && !-e $RECONFIGURE_PENDING
+ && !%$record;
+
+ my $res = eval { regenerate($cfg, $overrides, $derived) };
+ return { $state->%*, error => "$@" } if $@;
+
+ return {
+ wwids => $res->{wwids},
+ defined($res->{error}) ? (error => $res->{error}) : (),
+ };
+}
+
+1;
diff --git a/src/test/Makefile b/src/test/Makefile
index 97d9db4d..9ecd59d4 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_base_activation test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_iscsi test_lun_identity test_multipath_api test_san_api test_san_libraries
+test: test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_base_activation test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_iscsi test_lun_identity test_multipath_api test_san_api test_san_libraries test_san_taint
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -75,3 +75,6 @@ test_san_api: run_san_api_tests.pl
test_san_libraries: run_san_library_tests.pl
./run_san_library_tests.pl
+
+test_san_taint: run_san_taint_tests.pl
+ perl -T ./run_san_taint_tests.pl
diff --git a/src/test/run_lun_identity_tests.pl b/src/test/run_lun_identity_tests.pl
index df248b38..33b92d7c 100755
--- a/src/test/run_lun_identity_tests.pl
+++ b/src/test/run_lun_identity_tests.pl
@@ -16,6 +16,7 @@ use Test::More;
use PVE::Multipath;
use PVE::Multipath::Identity;
+use PVE::Multipath::Generator;
use PVE::Storage;
use PVE::Storage::ISCSIPlugin;
use PVE::Multipath::Status;
diff --git a/src/test/run_multipath_api_tests.pl b/src/test/run_multipath_api_tests.pl
index c6fdfb7c..b013ba38 100755
--- a/src/test/run_multipath_api_tests.pl
+++ b/src/test/run_multipath_api_tests.pl
@@ -3,11 +3,21 @@
use strict;
use warnings;
+# a fixed clock for the boundaries of the report ages, the real one otherwise; installed before the
+# modules are compiled, so their time() binds to it
+our $NOW;
+
+BEGIN {
+ *CORE::GLOBAL::time = sub { return $NOW // CORE::time() };
+}
+
use lib ('.', '..');
+use Storable qw(dclone);
use Test::More;
use Test::MockModule;
use PVE::Multipath;
+use PVE::Multipath::Generator;
use PVE::Storage;
use PVE::Storage::ISCSIPlugin;
use PVE::Multipath::Status;
@@ -89,4 +99,186 @@ subtest 'enabling the multipath option needs every node to know it' => sub {
);
};
+subtest 'an entry stays allow-listed while its map is in use' => sub {
+ my $mp = Test::MockModule->new('PVE::Multipath');
+ my $map = { wwid => 'w1', sysfs => 'dm-999999', 'paths-active' => 2 };
+ $mp->mock(get_maps => sub { [$map, { wwid => 'w2', sysfs => 'dm-999998' }] });
+ my ($wipefs, $ran, @caches) = ('', 0);
+ $mp->mock(
+ run_command => sub {
+ my ($cmd, %param) = @_;
+ $ran++;
+ die "unexpected command @$cmd\n" if $cmd->[0] ne 'dmsetup';
+ return $param{outfunc}->('0');
+ },
+ );
+ my $status_command = Test::MockModule->new('PVE::Storage::StatusCommand');
+ $status_command->mock(
+ run => sub {
+ my ($cache, $cmd, %param) = @_;
+ $ran++;
+ push @caches, $cache;
+ is($cmd->[0], '/sbin/wipefs', 'the device is read as a bounded status command');
+ die "status command 'wipefs' failed: exit code 1\n" if $wipefs eq 'unreadable';
+ die "another status command is still running\n" if $wipefs eq 'contended';
+ $param{outfunc}->($wipefs);
+ },
+ );
+ my $signatures = sub {
+ return '{"signatures":[' . join(',', map { qq({"type":"$_"}) } @_) . ']}';
+ };
+ local $PVE::Multipath::IN_USE_RECHECK_SECONDS = 0;
+
+ $wipefs = $signatures->('LVM2_member');
+ is_deeply(
+ PVE::Multipath::maps_in_use(['w1', 'w3']),
+ { w1 => 'LVM' },
+ 'a physical volume keeps the entry, a LUN without a map here does not',
+ );
+ $wipefs = $signatures->();
+ is_deeply(PVE::Multipath::maps_in_use(['w1']), {},
+ 'a map without signatures is not in use');
+ $wipefs = $signatures->('gpt', 'PMBR');
+ is_deeply(PVE::Multipath::maps_in_use(['w1']), { w1 => 'partitions' }, 'a partition table');
+ $wipefs = 'unreadable';
+ is_deeply(PVE::Multipath::maps_in_use(['w1']), { w1 => 'unknown' }, 'an unreadable map');
+ $wipefs = 'contended';
+ is_deeply(PVE::Multipath::maps_in_use(['w1']), { w1 => 'unknown' }, 'a contended probe');
+
+ # the probes of one pass share a time budget
+ $mp->mock(
+ get_maps => sub {
+ [map { { wwid => $_, sysfs => 'dm-999999', 'paths-active' => 1 } } qw(w1 w4)];
+ },
+ );
+ ($wipefs, @caches) = ($signatures->());
+ is_deeply(PVE::Multipath::maps_in_use(['w1', 'w4']), {}, 'two unused maps');
+ is(scalar(@caches), 2, 'both maps were probed');
+ ok($caches[0] == $caches[1], 'with one shared budget');
+ is($caches[0]->{'status-command-budget'}, 10, 'of ten seconds');
+ $mp->mock(get_maps => sub { [$map, { wwid => 'w2', sysfs => 'dm-999998' }] });
+
+ # an unused map leaves no remembered result behind
+ $wipefs = $signatures->();
+ PVE::Multipath::maps_in_use(['w1']);
+ $PVE::Multipath::IN_USE_RECHECK_SECONDS = 60;
+ $wipefs = $signatures->('LVM2_member');
+ PVE::Multipath::maps_in_use(['w1']);
+ $ran = 0;
+ $wipefs = $signatures->();
+ is_deeply(
+ PVE::Multipath::maps_in_use(['w1']),
+ { w1 => 'LVM' },
+ 'a map in use is not reread',
+ );
+ is($ran, 0, 'until the recheck interval passed');
+ $PVE::Multipath::IN_USE_RECHECK_SECONDS = 0;
+
+ $ran = 0;
+ is_deeply(
+ PVE::Multipath::maps_in_use(['w2']),
+ { w2 => 'unknown' },
+ 'a map without a usable path counts as in use',
+ );
+ is($ran, 0, 'without being probed, as its I/O could block');
+
+ $mp->mock(get_maps => sub { die "daemon gone\n" });
+ {
+ local $SIG{__WARN__} = sub { };
+ is_deeply(
+ PVE::Multipath::maps_in_use(['w1', 'w3']),
+ { w1 => 'unknown', w3 => 'unknown' },
+ 'nothing is pruned while the maps cannot be queried',
+ );
+ }
+};
+
+subtest 'regeneration prunes only conclusively closed blank maps' => sub {
+ require File::Temp;
+ my $dir = File::Temp::tempdir(CLEANUP => 1);
+ local $PVE::Multipath::Generator::DEFAULTS_DROPIN = "$dir/defaults";
+ local $PVE::Multipath::Generator::OVERRIDES_DROPIN = "$dir/overrides";
+ local $PVE::Multipath::Generator::ALIASES_DROPIN = "$dir/aliases";
+ local $PVE::Multipath::Generator::RECONFIGURE_PENDING = "$dir/pending";
+ local $PVE::Multipath::IN_USE_RECHECK_SECONDS = 60;
+ local $NOW = 100_000;
+ my $mp = Test::MockModule->new('PVE::Multipath');
+ my $inotify = Test::MockModule->new('PVE::INotify');
+ $inotify->mock(nodename => sub { 'n1' });
+ my $cfg = PVE::Multipath::Config->parse_config('multipath.cfg', "wwid: w2\n");
+ my ($opens, $signatures, $failed) = ('1', '{"signatures":[]}', '');
+ my ($record, $current, @removed);
+ $mp->mock(
+ get_maps => sub { [{ wwid => 'w1', sysfs => 'dm-999999', 'paths-active' => 2 }] },
+ is_running => sub { 0 },
+ managed_wwid_record => sub { dclone($record) },
+ set_managed_wwid_record => sub { $record = dclone($_[0]) },
+ list_etc_multipath_wwids => sub { [keys %$current] },
+ assert_supported => sub { 1 },
+ run_command => sub {
+ my ($cmd, %param) = @_;
+ if ($cmd->[0] eq 'dmsetup') {
+ is($param{timeout}, 10, 'probe has a bounded timeout');
+ die "probe failed\n" if $failed eq $cmd->[0];
+ $param{outfunc}->($opens);
+ } elsif ($cmd->[0] eq '/sbin/multipath' && $cmd->[1] eq '-w') {
+ is($cmd->[2], '--', 'the WWID is no option');
+ push @removed, $cmd->[3];
+ delete $current->{ $cmd->[3] };
+ } else {
+ die "unexpected command @$cmd\n";
+ }
+ },
+ );
+ my $status_command = Test::MockModule->new('PVE::Storage::StatusCommand');
+ $status_command->mock(
+ run => sub {
+ my ($cache, $cmd, %param) = @_;
+ is($cmd->[0], '/sbin/wipefs', 'only the device read is a status command');
+ die "status command 'wipefs' failed: exit code 1\n" if $failed eq $cmd->[0];
+ $param{outfunc}->($signatures);
+ },
+ );
+ for my $case (
+ ['raw opener', '1', '{"signatures":[]}', ''],
+ ['missing open count', '', '{"signatures":[]}', ''],
+ ['malformed open count', 'zero', '{"signatures":[]}', ''],
+ ['multiple open counts', "0\n0", '{"signatures":[]}', ''],
+ ['failed open probe', '0', '{"signatures":[]}', 'dmsetup'],
+ ['malformed signature', '0', '{"signatures":[{}]}', ''],
+ ['missing signatures', '0', '{}', ''],
+ ['failed signature probe', '0', '{"signatures":[]}', '/sbin/wipefs'],
+ ) {
+ my $name;
+ ($name, $opens, $signatures, $failed) = @$case;
+ $record = { w1 => ['config'], w2 => ['config'] };
+ $current = { w1 => 1, w2 => 1 };
+ @removed = ();
+ $NOW += 61;
+ my $state = PVE::Multipath::Generator::regenerate($cfg, '', {});
+ is_deeply(\@removed, [], "$name never runs multipath -w");
+ ok($record->{w1}, "$name retains ownership");
+ ok($state->{wwids}->{w1}->{kept}, "$name reports retention");
+ ($opens, $signatures, $failed) = ('0', '{"signatures":[]}', '');
+ PVE::Multipath::Generator::regenerate($cfg, '', {});
+ is_deeply(\@removed, [], 'positive in-use result remains cached');
+ $NOW += 61;
+ PVE::Multipath::Generator::regenerate($cfg, '', {});
+ is_deeply(\@removed, ['w1'], 'closed and blank map is eventually pruned');
+ ok(!$record->{w1}, 'pruning releases ownership');
+ ok($record->{w2}, 'other managed entry is retained');
+ }
+};
+
+subtest 'unreadable configuration keeps the known LUNs reported' => sub {
+ my $mp = Test::MockModule->new('PVE::Multipath');
+ $mp->mock(managed_wwid_record => sub { +{ '3600abc' => ['storage:san'] } });
+ my $cc = Test::MockModule->new('PVE::Multipath::ClusterConfig');
+ $cc->mock(read_config => sub { die "config unavailable\n" });
+ my $state = PVE::Multipath::Generator::sync();
+ like($state->{error}, qr/config unavailable/, 'the failure is reported');
+ is_deeply($state->{wwids}->{'3600abc'}->{storages}, ['san'],
+ 'the recorded LUN stays known');
+};
+
done_testing();
diff --git a/src/test/run_multipath_tests.pl b/src/test/run_multipath_tests.pl
index 0e247284..992dd3dd 100755
--- a/src/test/run_multipath_tests.pl
+++ b/src/test/run_multipath_tests.pl
@@ -3,11 +3,14 @@
use strict;
use warnings;
+use Test::MockModule;
use Test::More;
use lib ('.', '..');
use PVE::Multipath;
use PVE::Multipath::Config;
+use PVE::Multipath::Generator;
+use PVE::Multipath::Status;
# A recorded `multipathd show maps json` reply with three maps exercising each
# health state: an all-active map, a partially-failed map and an all-failed map.
@@ -360,6 +363,230 @@ is(
. "3600abc config\n3600def config,storage:san1\n",
'format_managed_wwids sorts and round-trips',
);
+is_deeply(
+ PVE::Multipath::Generator::desired_wwid_info(
+ { w1 => ['config', 'storage:a', 'storage:b'], w2 => ['config'] },
+ {
+ a => { wwids => { w1 => { 'paths-expected' => 2 } } },
+ b => { wwids => { w1 => { 'paths-expected' => 4 } } },
+ },
+ ),
+ { w1 => { storages => ['a', 'b'], 'paths-expected' => 4 }, w2 => {} },
+ 'every deriving storage is named, the largest path expectation wins',
+);
+
+# --- WWID allow-list reconcile plan: only prune what PVE itself added, and only once every ---
+# --- reason it was added for is gone from the configuration ---
+# 'f' is a foreign, hand-added WWID present in the local allow-list but never managed by PVE.
+{
+ my @cases = (
+ {
+ name => 'explicit-only config adds missing WWIDs, foreign entries stay untouched',
+ explicit => { a => 1, b => 1 },
+ current => { b => 1, f => 1 },
+ managed => { b => ['config'] },
+ add => ['a'],
+ remove => [],
+ desired => { a => ['config'], b => ['config'] },
+ },
+ {
+ name => 'an owned WWID dropped from an explicit-only config is pruned',
+ explicit => { a => 1 },
+ current => { a => 1, b => 1, f => 1 },
+ managed => { a => ['config'], b => ['config'] },
+ add => [],
+ remove => ['b'],
+ },
+ {
+ name => 'derived WWIDs are added like explicit ones, sourced to their storage',
+ derived => { san1 => { wwids => { d1 => {}, d2 => {} } } },
+ current => { d1 => 1 },
+ managed => { d1 => ['storage:san1'] },
+ add => ['d2'],
+ remove => [],
+ desired => { d1 => ['storage:san1'], d2 => ['storage:san1'] },
+ },
+ {
+ name => 'a LUN gone from the target stays while its storage is still enabled',
+ derived => { san1 => { wwids => { d1 => {} } } },
+ current => { d1 => 1, gone => 1, f => 1 },
+ managed => { d1 => ['storage:san1'], gone => ['storage:san1'] },
+ add => [],
+ remove => [],
+ desired => { d1 => ['storage:san1'], gone => ['storage:san1'] },
+ },
+ {
+ name => 'a storage no longer enabled on the node prunes its owned WWIDs',
+ derived => {},
+ current => { d1 => 1, f => 1 },
+ managed => { d1 => ['storage:san1'] },
+ add => [],
+ remove => ['d1'],
+ },
+ {
+ name => 'an outage (nothing visible) keeps every owned WWID of the storage',
+ derived => { san1 => { wwids => {} } },
+ current => { d1 => 1, f => 1 },
+ managed => { d1 => ['storage:san1'] },
+ add => [],
+ remove => [],
+ desired => { d1 => ['storage:san1'] },
+ },
+ {
+ name => 'a failing derivation of one storage does not touch the others',
+ derived => {
+ san1 => { wwids => { d1 => {} } },
+ san2 => { wwids => {}, error => 'boom' },
+ },
+ current => { d1 => 1, d2 => 1 },
+ managed => { d1 => ['storage:san1'], d2 => ['storage:san2'] },
+ add => [],
+ remove => [],
+ },
+ {
+ name => 'an explicit section on a derived WWID adds a reason, it replaces none',
+ explicit => { p => 1 },
+ derived => { san1 => { wwids => { p => {} } } },
+ current => { p => 1 },
+ managed => { p => ['storage:san1'] },
+ add => [],
+ remove => [],
+ desired => { p => ['config', 'storage:san1'] },
+ },
+ {
+ name => 'unpinning during an outage keeps the WWID for its storage',
+ explicit => {},
+ derived => { san1 => { wwids => {} } },
+ current => { p => 1 },
+ managed => { p => ['config', 'storage:san1'] },
+ add => [],
+ remove => [],
+ desired => { p => ['storage:san1'] },
+ },
+ {
+ name => 'unpinning a WWID whose storage is gone prunes it',
+ explicit => {},
+ derived => {},
+ current => { p => 1 },
+ managed => { p => ['config', 'storage:san1'] },
+ add => [],
+ remove => ['p'],
+ },
+ {
+ name => 'a WWID handed from one storage to another follows the deriving storage',
+ derived => { san2 => { wwids => { d => {} } } },
+ current => { d => 1 },
+ managed => { d => ['storage:san1'] },
+ add => [],
+ remove => [],
+ desired => { d => ['storage:san2'] },
+ },
+ {
+ name => 'an owned config WWID missing from the allow-list is restored',
+ explicit => { a => 1 },
+ current => {},
+ managed => { a => ['config'] },
+ add => ['a'],
+ remove => [],
+ },
+ {
+ name =>
+ 'an owned derived WWID missing from the allow-list is restored during an outage',
+ derived => { san1 => { wwids => {} } },
+ current => {},
+ managed => { d1 => ['storage:san1'] },
+ add => ['d1'],
+ remove => [],
+ },
+ {
+ name =>
+ 'an owned WWID neither in the allow-list nor wanted anymore is just forgotten',
+ explicit => {},
+ current => { f => 1 },
+ managed => { stale => ['config'] },
+ add => [],
+ remove => [],
+ desired => {},
+ },
+ );
+ for my $case (@cases) {
+ my ($add, $remove, $desired) = PVE::Multipath::Generator::plan_wwid_changes(
+ $case->{explicit} // {},
+ $case->{derived} // {},
+ $case->{current} // {},
+ $case->{managed} // {},
+ );
+ is_deeply($add, $case->{add}, "$case->{name}: additions");
+ is_deeply($remove, $case->{remove}, "$case->{name}: removals");
+ is_deeply($desired, $case->{desired}, "$case->{name}: desired reasons")
+ if exists $case->{desired};
+ }
+}
+
+# --- ownership record: never adopt a pre-existing hand-made allow-list entry ---
+{
+ # 'h' was already in the allow-list when it appeared in the cluster config, so PVE never added
+ # it and it must stay foreign even while desired; dropping it from the config later must not
+ # prune the hand-made entry (think a boot-from-SAN LUN added just for health monitoring)
+ my $owned = PVE::Multipath::Generator::owned_wwid_record(
+ { a => ['config'], h => ['config'] }, # desired
+ { a => ['config'] }, # managed before
+ { a => 1, h => 1 }, # both already in the allow-list
+ {},
+ );
+ is_deeply($owned, { a => ['config'] }, 'a desired but hand-added WWID is not adopted');
+
+ $owned = PVE::Multipath::Generator::owned_wwid_record(
+ { a => ['config'], n => ['storage:san1'] }, # desired
+ { a => ['config'] }, # managed before
+ { a => 1 }, # 'n' is absent, so PVE is about to add it
+ {},
+ );
+ is_deeply(
+ $owned,
+ { a => ['config'], n => ['storage:san1'] },
+ 'a WWID PVE is about to add becomes owned before the add, with its reasons',
+ );
+
+ $owned = PVE::Multipath::Generator::owned_wwid_record(
+ { a => ['config'] }, # desired
+ { a => ['config'], gone => ['storage:san1'] }, # managed; 'gone' is no longer wanted
+ { a => 1, gone => 1 },
+ { gone => 1 }, # but its removal is pending or failed
+ );
+ is_deeply(
+ $owned,
+ { a => ['config'], gone => ['storage:san1'] },
+ 'a pending or failed prune keeps ownership with the recorded reasons',
+ );
+
+ $owned = PVE::Multipath::Generator::owned_wwid_record(
+ { d => ['config', 'storage:san1'] }, # now also pinned by an explicit section
+ { d => ['storage:san1'] },
+ { d => 1 },
+ {},
+ );
+ is_deeply($owned, { d => ['config', 'storage:san1'] }, 'an owned WWID takes over new reasons');
+
+ is_deeply(
+ PVE::Multipath::Generator::owned_wwid_record({}, {}, {}, {}),
+ {},
+ 'empty in, empty out',
+ );
+
+ ok(
+ PVE::Multipath::Generator::wwid_records_differ({ a => ['config'] }, { a => ['storage:x'] }),
+ 'records with different reasons differ',
+ );
+ ok(
+ !PVE::Multipath::Generator::wwid_records_differ(
+ { a => ['config', 'storage:x'], b => ['storage:x'] },
+ { b => ['storage:x'], a => ['storage:x', 'config'] },
+ ),
+ 'records with the same reasons do not differ',
+ );
+}
+
# --- override guard ---
eval { PVE::Multipath::Config::check_overrides("devices {\n\tdevice {\n\t\tvendor X\n\t}\n}\n") };
is($@, '', 'a well-formed devices{} block passes the guard');
@@ -385,4 +612,24 @@ is(
'the overrides writer trims trailing whitespace',
);
+{
+ my $mp = Test::MockModule->new('PVE::Multipath');
+ my @commands;
+ $mp->mock(assert_supported => sub { 1 }, run_command => sub { push @commands, $_[0] });
+ for my $wwid ('-W', '-F', '8:0') {
+ eval { PVE::Multipath::add_wwid($wwid) };
+ like($@, qr/^invalid WWID/, "allow-list addition refuses '$wwid'");
+ eval { PVE::Multipath::remove_wwid($wwid) };
+ like($@, qr/^invalid WWID/, "allow-list removal refuses '$wwid'");
+ }
+ is_deeply(\@commands, [], 'multipath never ran for them');
+ PVE::Multipath::add_wwid('3600abc');
+ PVE::Multipath::remove_wwid('3600abc');
+ is_deeply(
+ \@commands,
+ [['/sbin/multipath', '-a', '--', '3600abc'], ['/sbin/multipath', '-w', '--', '3600abc']],
+ 'a WWID is passed after the end of the options',
+ );
+}
+
done_testing();
diff --git a/src/test/run_san_library_tests.pl b/src/test/run_san_library_tests.pl
index 84a8539c..e1564deb 100755
--- a/src/test/run_san_library_tests.pl
+++ b/src/test/run_san_library_tests.pl
@@ -15,6 +15,7 @@ my %registered;
my $cluster = Test::MockModule->new('PVE::Cluster');
$cluster->mock(cfs_register_file => sub { $registered{ $_[0] } = [@_[1, 2]] });
require PVE::Multipath::ClusterConfig;
+require PVE::Multipath::Generator;
subtest 'CFS callbacks preserve filenames, data and method receivers' => sub {
my ($parse, $write) = $registered{'multipath.cfg'}->@*;
@@ -33,6 +34,11 @@ subtest 'CFS callbacks preserve filenames, data and method receivers' => sub {
subtest 'omitted optional parameters retain explicit undef behavior' => sub {
my $config = Test::MockModule->new('PVE::Multipath::ClusterConfig');
+ $config->mock(read_config => sub { die "configuration unavailable\n" });
+ for my $args ([], [undef, undef, undef]) {
+ eval { PVE::Multipath::Generator::regenerate(@$args) };
+ is($@, "configuration unavailable\n", 'regeneration reads the default configuration');
+ }
$config->mock(cfs_lock_file => sub { $@ = "lock unavailable\n" });
for my $lock (
\&PVE::Multipath::ClusterConfig::lock_config,
diff --git a/src/test/run_san_taint_tests.pl b/src/test/run_san_taint_tests.pl
new file mode 100755
index 00000000..fd992171
--- /dev/null
+++ b/src/test/run_san_taint_tests.pl
@@ -0,0 +1,80 @@
+#!/usr/bin/perl -T
+
+use strict;
+use warnings;
+
+use File::Temp qw(tempdir);
+use JSON qw(decode_json encode_json);
+use Scalar::Util qw(tainted);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Multipath;
+use PVE::Tools;
+
+die "run this test with perl -T\n" if !${^TAINT};
+$ENV{PATH} = '/usr/sbin:/usr/bin:/sbin:/bin';
+delete @ENV{qw(IFS CDPATH ENV BASH_ENV)};
+
+my $root = tempdir(CLEANUP => 1);
+my sub from_file {
+ my ($data) = @_;
+ my $file = "$root/reply.json";
+ open(my $fh, '>', $file) or die $!;
+ print $fh encode_json($data);
+ close($fh) or die $!;
+ return decode_json(PVE::Tools::file_get_contents($file));
+}
+
+my $mp = Test::MockModule->new('PVE::Multipath');
+my @commands;
+my $signature;
+$mp->mock(
+ dir_has_entries => sub { 0 },
+ run_command => sub {
+ my ($cmd, %opts) = @_;
+ push @commands, [@$cmd];
+ PVE::Tools::run_command(['/usr/bin/true', @$cmd]);
+ die "unexpected command @$cmd\n" if $cmd->[0] ne 'dmsetup';
+ $opts{outfunc}->('0');
+ },
+);
+my $status_command = Test::MockModule->new('PVE::Storage::StatusCommand');
+$status_command->mock(
+ run => sub {
+ my ($cache, $cmd, %opts) = @_;
+ push @commands, [@$cmd];
+ PVE::Tools::run_command(['/usr/bin/true', @$cmd]);
+ die "unexpected command @$cmd\n" if $cmd->[0] ne '/sbin/wipefs';
+ $opts{outfunc}
+ ->(encode_json({ signatures => $signature ? [{ type => $signature }] : [] }));
+ },
+);
+
+for my $type (undef, 'LVM2_member') {
+ @commands = ();
+ $signature = $type;
+ my $map = from_file({ sysfs => 'dm-900', 'paths-active' => 1 });
+ ok(tainted($map->{sysfs}), 'map name from JSON file is tainted');
+ is(PVE::Multipath::map_user($map), $type ? 'LVM' : undef,
+ 'taint does not make map use unknown');
+ is_deeply(
+ \@commands,
+ [
+ ['dmsetup', 'info', '-c', '--noheadings', '-o', 'open', '/dev/dm-900'],
+ ['/sbin/wipefs', '--no-act', '--json', '/dev/dm-900'],
+ ],
+ 'both map probes cross a real benign exec boundary',
+ );
+ ok(!tainted($_->[-1]), 'map probe uses captured device name') for @commands;
+}
+
+for my $name (undef, '', 'dm-x', 'sda', '../dm-900', "dm-900\n", 'dm-900 extra') {
+ @commands = ();
+ my $map = from_file({ sysfs => $name, 'paths-active' => 1 });
+ is(PVE::Multipath::map_user($map), 'unknown', 'invalid map name stays inconclusive');
+ is_deeply(\@commands, [], 'invalid map name reaches no command');
+}
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 32/76] iscsi: share the initiator identity reader
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (30 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 31/76] multipath: apply the configuration on every node Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 33/76] multipath: publish per-node status Thomas Lamprecht
` (43 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
The node API and multipath status publication need the same initiator
identity. Share the open-iscsi configuration reader through the plugin
so both consumers interpret its name and alias consistently.
Bound the read to 64 KiB: a valid identity file is small, and an
oversized or unreadable file must not stall reporting or produce a
truncated identity. Missing names remain unavailable rather than being
invented.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/ISCSI.pm | 22 ++---------------
src/PVE/Storage/ISCSIPlugin.pm | 16 +++++++++++++
src/test/run_multipath_api_tests.pl | 37 +++++++++++++++++++++++++++++
src/test/run_san_api_tests.pl | 21 ++++++++++++++++
4 files changed, 76 insertions(+), 20 deletions(-)
diff --git a/src/PVE/API2/ISCSI.pm b/src/PVE/API2/ISCSI.pm
index a3d55845..0f7bde83 100644
--- a/src/PVE/API2/ISCSI.pm
+++ b/src/PVE/API2/ISCSI.pm
@@ -7,13 +7,10 @@ use PVE::JSONSchema qw(get_standard_option);
use PVE::RESTHandler;
use PVE::Storage;
use PVE::Storage::ISCSIPlugin;
-use PVE::Tools qw(run_command file_get_contents file_read_firstline);
+use PVE::Tools qw(file_read_firstline);
use base qw(PVE::RESTHandler);
-my $ISCSIADM = '/usr/bin/iscsiadm';
-my $INITIATOR_NAME_FILE = '/etc/iscsi/initiatorname.iscsi';
-
my $session_id_schema = {
description => "Kernel iSCSI session id.",
type => 'integer',
@@ -166,21 +163,6 @@ my $session_params_return = {
},
};
-my sub read_initiator_info {
- my $content = eval { file_get_contents($INITIATOR_NAME_FILE) };
- return undef if !defined($content);
-
- my $res = {};
- for my $line (split(/\n/, $content)) {
- if ($line =~ m/^\s*InitiatorName\s*=\s*(\S+)\s*$/) {
- $res->{name} = $1;
- } elsif ($line =~ m/^\s*InitiatorAlias\s*=\s*(.+?)\s*$/) {
- $res->{alias} = $1;
- }
- }
- return defined($res->{name}) ? $res : undef;
-}
-
my sub session_health {
my ($kernel_state) = @_;
@@ -299,7 +281,7 @@ __PACKAGE__->register_method({
code => sub {
my $res = { supported => PVE::Storage::ISCSIPlugin::iscsi_supported() };
if ($res->{supported}) {
- my $initiator = read_initiator_info();
+ my $initiator = PVE::Storage::ISCSIPlugin::read_initiator_info();
if ($initiator) {
$res->{'initiator-name'} = $initiator->{name} if defined($initiator->{name});
$res->{'initiator-alias'} = $initiator->{alias} if defined($initiator->{alias});
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index 0e3b6df3..4831d2d4 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -24,9 +24,25 @@ use base qw(PVE::Storage::Plugin);
# iscsi helper function
my $ISCSIADM = '/usr/bin/iscsiadm';
+our $INITIATOR_NAME_FILE = '/etc/iscsi/initiatorname.iscsi';
our $SYSFS = '/sys';
my $MULTIPATHD_EXECUTABLES = { '/usr/sbin/multipathd' => 1, '/sbin/multipathd' => 1 };
+sub read_initiator_info {
+ my $content = eval { PVE::Tools::file_get_contents($INITIATOR_NAME_FILE, 64 * 1024) };
+ return undef if !defined($content);
+
+ my $res = {};
+ for my $line (split(/\n/, $content)) {
+ if ($line =~ m/^\s*InitiatorName\s*=\s*(\S+)\s*$/) {
+ $res->{name} = $1;
+ } elsif ($line =~ m/^\s*InitiatorAlias\s*=\s*(.+?)\s*$/) {
+ $res->{alias} = $1;
+ }
+ }
+ return defined($res->{name}) ? $res : undef;
+}
+
my $found_iscsi_adm_exe;
my sub assert_iscsi_support {
my ($noerr) = @_;
diff --git a/src/test/run_multipath_api_tests.pl b/src/test/run_multipath_api_tests.pl
index b013ba38..fdd9ec39 100755
--- a/src/test/run_multipath_api_tests.pl
+++ b/src/test/run_multipath_api_tests.pl
@@ -99,6 +99,43 @@ subtest 'enabling the multipath option needs every node to know it' => sub {
);
};
+subtest 'the iSCSI initiator identity reader' => sub {
+ # the initiator identity reader with open-iscsi configuration files
+ require File::Temp;
+ my $file = File::Temp->new();
+ local $PVE::Storage::ISCSIPlugin::INITIATOR_NAME_FILE = $file->filename;
+ my $read = sub {
+ my ($text) = @_;
+ open(my $fh, '>', $file->filename) or die "open failed - $!\n";
+ print $fh $text;
+ close($fh);
+ return PVE::Storage::ISCSIPlugin::read_initiator_info();
+ };
+ is_deeply(
+ $read->(
+ "## a comment\nInitiatorName=iqn.2026-10.test.lab:n1\nInitiatorAlias= node 1 \n"),
+ { name => 'iqn.2026-10.test.lab:n1', alias => 'node 1' },
+ 'the reader takes the name and the alias',
+ );
+ is(
+ $read->("InitiatorName=iqn.2026-10.test.lab:n2\n")->{name},
+ 'iqn.2026-10.test.lab:n2',
+ 'and reads the file again',
+ );
+ ok(!defined($read->("## only a comment\n")), 'a file without a name gives none');
+ ok(
+ !defined($read->('InitiatorName=iqn.2026-10.test.lab:' . ('x' x 70000))),
+ 'an oversized file gives none rather than a cut name',
+ );
+ {
+ local $PVE::Storage::ISCSIPlugin::INITIATOR_NAME_FILE = $file->filename . '.missing';
+ ok(
+ !defined(PVE::Storage::ISCSIPlugin::read_initiator_info()),
+ 'a missing file gives none',
+ );
+ }
+};
+
subtest 'an entry stays allow-listed while its map is in use' => sub {
my $mp = Test::MockModule->new('PVE::Multipath');
my $map = { wwid => 'w1', sysfs => 'dm-999999', 'paths-active' => 2 };
diff --git a/src/test/run_san_api_tests.pl b/src/test/run_san_api_tests.pl
index cf382a11..94ee74f0 100755
--- a/src/test/run_san_api_tests.pl
+++ b/src/test/run_san_api_tests.pl
@@ -11,9 +11,30 @@ use PVE::API2::ISCSI;
use PVE::API2::Storage::Scan;
subtest 'iSCSI status fields' => sub {
+ my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+ my $identity = { name => 'iqn.test', alias => 'test' };
+ $plugin->mock(read_initiator_info => sub { $identity });
my $method = PVE::API2::ISCSI->map_method_by_name('status');
ok($method->{returns}->{properties}->{'initiator-name'}, 'name schema');
ok($method->{returns}->{properties}->{'initiator-alias'}, 'alias schema');
+ SKIP: {
+ skip 'iscsiadm not installed', 2 if !-x '/usr/bin/iscsiadm';
+ is_deeply(
+ $method->{code}->({ node => 'test' }),
+ {
+ supported => 1,
+ 'initiator-name' => 'iqn.test',
+ 'initiator-alias' => 'test',
+ },
+ 'flat identity',
+ );
+ $identity = undef;
+ is_deeply(
+ $method->{code}->({ node => 'test' }),
+ { supported => 1 },
+ 'missing identity omitted',
+ );
+ }
my $enum =
PVE::API2::ISCSI->map_method_by_name('targets')->{returns}->{items}->{properties}->{health}
->{enum};
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 33/76] multipath: publish per-node status
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (31 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 32/76] iscsi: share the initiator identity reader Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 34/76] api: add endpoints for the cluster-wide multipath configuration Thomas Lamprecht
` (42 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Path health can differ between nodes. Publish each node's map health,
path counts, tool and daemon availability, and configuration errors in
the cluster key-value store. Include wanted LUNs even without maps, the
number of unmanaged maps, and the count of unresolved LUN identities.
For storage-derived LUNs, also report whether the target still presents
them. Establish absence only after a session on every portal has
finished scanning and every discovered LUN is identified. This
distinguishes a withdrawn LUN from a map that has not assembled.
Include the iSCSI initiator name for target access rules, omitting
invalid names rather than truncating them. Resend unchanged reports once
a minute so readers can detect stale reports. Nodes without wanted LUNs
reuse a successful map listing for a minute.
The store limits values to 32 KiB, enough for roughly 200 LUNs. Larger
reports contain only counts per state, never a partial list that could
make omitted LUNs look missing.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Multipath/Generator.pm | 28 +++-
src/PVE/Multipath/Identity.pm | 51 +++++-
src/PVE/Multipath/Status.pm | 184 +++++++++++++++++++++
src/PVE/Storage.pm | 2 +-
src/PVE/Storage/ISCSIPlugin.pm | 17 +-
src/test/run_lun_identity_tests.pl | 237 +++++++++++++++++++++++++---
src/test/run_multipath_api_tests.pl | 151 ++++++++++++++++++
src/test/run_multipath_tests.pl | 114 ++++++++++++-
8 files changed, 742 insertions(+), 42 deletions(-)
diff --git a/src/PVE/Multipath/Generator.pm b/src/PVE/Multipath/Generator.pm
index b7aec227..5a5ad96c 100644
--- a/src/PVE/Multipath/Generator.pm
+++ b/src/PVE/Multipath/Generator.pm
@@ -93,20 +93,37 @@ sub wwid_records_differ($a, $b) {
return $flat->($a) ne $flat->($b);
}
-# Storages can expose a LUN through different portal sets, so judge map health against the largest
-# expected path count.
+sub unresolved_lun_count($derived) {
+ my %luns = map { $_ => 1 } map { ($_->{unresolved} // [])->@* } values(($derived // {})->%*);
+ return scalar(keys %luns);
+}
+
+# Omit 'derived' when no source finds the LUN and at least one could not scan: a failed scan cannot
+# prove absence. Storages can expose a LUN through different portal sets, so judge map health
+# against the largest expected path count.
sub desired_wwid_info($desired, $derived) {
my $wwids = {};
for my $wwid (keys %$desired) {
my $info = {};
+ my ($found, $unseen) = (0, 0);
for my $source ($desired->{$wwid}->@*) {
my ($storeid) = $source =~ m/^storage:(.+)$/ or next;
push $info->{storages}->@*, $storeid;
- my $luns = ($derived->{$storeid} // {})->{wwids} // {};
- my $expected = ($luns->{$wwid} // {})->{'paths-expected'};
+ my $set = $derived->{$storeid} // {};
+ my $lun = ($set->{wwids} // {})->{$wwid};
+ if ($lun) {
+ $found = 1;
+ } elsif (!$set->{scanned}) {
+ $unseen = 1;
+ }
+ my $expected = ($lun // {})->{'paths-expected'};
$info->{'paths-expected'} = $expected
if defined($expected) && $expected > ($info->{'paths-expected'} // 0);
}
+ if ($info->{storages}) {
+ $info->{derived} = 1 if $found;
+ $info->{derived} = 0 if !$found && !$unseen;
+ }
$wwids->{$wwid} = $info;
}
return $wwids;
@@ -259,6 +276,8 @@ sub sync() {
PVE::Storage::derived_multipath_wwids(PVE::Storage::config());
};
return { $state->%*, error => "reading the storage configuration failed - $@" } if $@;
+ my $unresolved = unresolved_lun_count($derived);
+ $state->{unresolved} = $unresolved if $unresolved;
my (undef, undef, $desired) =
plan_wwid_changes(\%explicit, $derived, {}, $record);
@@ -287,6 +306,7 @@ sub sync() {
return {
wwids => $res->{wwids},
+ $unresolved ? (unresolved => $unresolved) : (),
defined($res->{error}) ? (error => $res->{error}) : (),
};
}
diff --git a/src/PVE/Multipath/Identity.pm b/src/PVE/Multipath/Identity.pm
index 2a8d3d20..86141e98 100644
--- a/src/PVE/Multipath/Identity.pm
+++ b/src/PVE/Multipath/Identity.pm
@@ -96,18 +96,35 @@ sub resolve($disks, $daemon, $naa_ok) {
return $luns;
}
-# The iSCSI sessions: { sid => { target, host } }.
+# SCANNED means the initial LUN scan after login finished.
my sub iscsi_sessions() {
my $res = {};
dir_glob_foreach(
"$SYSFS/class/iscsi_session",
'session(\d+)',
sub($ent, $sid) {
- my $target = sysfs_line("class/iscsi_session/$ent/targetname") // return;
- my (undef, $host) =
- dir_glob_regex("$SYSFS/class/iscsi_session/$ent/device", 'target(\d+):\d+:\d+');
- return if !defined($host);
- $res->{$sid} = { target => $target, host => $host };
+ my $dir = "class/iscsi_session/$ent";
+ my $target = sysfs_line("$dir/targetname") // return;
+ # the SCSI host of the session, also when it has no target left, after its last LUN
+ # was removed
+ my $device = abs_path("$SYSFS/$dir/device") // return;
+ my ($host) = $device =~ m|/host(\d+)/session\d+$| or return;
+ my $session = {
+ target => $target,
+ host => $host,
+ state => sysfs_line("$dir/state") // '',
+ scan => sysfs_line("$dir/target_state") // '',
+ portal => '',
+ };
+ my (undef, $conn) = dir_glob_regex("$SYSFS/$dir/device", '(connection\d+:\d+)');
+ if (defined($conn)) {
+ my $cdir = "$dir/device/$conn/iscsi_connection/$conn";
+ my $address = sysfs_line("$cdir/persistent_address") // '';
+ my $port = sysfs_line("$cdir/persistent_port") // '';
+ $address = "[$address]" if $address =~ m/:/;
+ $session->{portal} = "$address:$port";
+ }
+ $res->{$sid} = $session;
},
);
return $res;
@@ -191,10 +208,32 @@ sub collect(%param) {
return {
luns => $luns,
paths => $paths,
+ sessions => $sessions,
+ 'sessions-after' => iscsi_sessions(),
defined($error) ? (error => $error) : (),
};
}
+# Whether every portal of $portals has a logged-in session to $target in $ids (see collect()) that
+# had finished its LUN scan before the LUNs were collected and did not change until afterwards.
+sub portals_covered($ids, $target, $portals) {
+ my ($before, $after) = $ids->@{qw(sessions sessions-after)};
+ for my $portal (@$portals) {
+ my ($sid) = grep {
+ my $session = $before->{$_};
+ $session->{target} eq $target
+ && $session->{portal} eq $portal
+ && $session->{state} eq 'LOGGED_IN'
+ && $session->{scan} eq 'SCANNED'
+ } sort keys %$before;
+ return 0 if !defined($sid) || !$after->{$sid};
+ return 0 if grep {
+ $after->{$sid}->{$_} ne $before->{$sid}->{$_}
+ } keys $before->{$sid}->%*;
+ }
+ return scalar(@$portals) ? 1 : 0;
+}
+
sub target_luns($ids, $target) {
return [grep { $_->{target} eq $target } values $ids->{luns}->%*];
}
diff --git a/src/PVE/Multipath/Status.pm b/src/PVE/Multipath/Status.pm
index e3dccf08..cc8fe49d 100644
--- a/src/PVE/Multipath/Status.pm
+++ b/src/PVE/Multipath/Status.pm
@@ -6,7 +6,191 @@ use JSON;
use PVE::Multipath;
use PVE::Multipath::Config;
+# Include wanted WWIDs without assembled maps so failed assembly remains visible. Expected path
+# counts include paths the daemon no longer lists.
+sub status_luns($wanted, $maps) {
+ my $luns = {};
+ for my $wwid (keys %$wanted) {
+ my $info = $wanted->{$wwid};
+ $luns->{$wwid} = { map { defined($info->{$_}) ? ($_ => $info->{$_}) : () }
+ qw(storages paths-expected derived kept) };
+ }
+
+ my $unmanaged = 0;
+ for my $map (($maps // [])->@*) {
+ my $wwid = $map->{wwid} // next;
+ my $lun = $luns->{$wwid};
+ if (!$lun) {
+ $unmanaged++;
+ next;
+ }
+
+ my ($active, $total) = ($map->{'paths-active'} // 0, $map->{'paths-total'} // 0);
+ my $expected = $lun->{'paths-expected'};
+ $lun->{state} =
+ defined($expected)
+ ? PVE::Multipath::derive_health($active, $total, $expected)
+ : $map->{health};
+ $lun->{'paths-active'} = $active;
+ $lun->{'paths-total'} = $total;
+ $lun->{transport} = $map->{transport} if defined($map->{transport});
+ $lun->{size} = $map->{size} if defined($map->{size});
+ }
+ return ($luns, $unmanaged);
+}
+
+# The pmxcfs node KV store rejects values of 32 KiB and more.
+my $KV_VALUE_LIMIT = 30 * 1024;
+my $ERROR_LIMIT = 2048;
my $KV_KEY = 'multipath';
+my $status_json = JSON->new->canonical->utf8;
+
+our $KV_REFRESH_SECONDS = 60;
+our $KV_STALE_SECONDS = 3 * $KV_REFRESH_SECONDS;
+
+# Replace oversized details with a summary: a partial list is indistinguishable from missing LUNs.
+sub encode_status($status, $limit = undef) {
+ $limit //= $KV_VALUE_LIMIT;
+ my $data = { $status->%* };
+ for my $key (qw(apply-error report-error)) {
+ $data->{$key} = substr($data->{$key}, 0, $ERROR_LIMIT) . '...'
+ if defined($data->{$key}) && length($data->{$key}) > $ERROR_LIMIT;
+ }
+ my $encoded = $status_json->encode($data);
+ return $encoded if length($encoded) <= $limit;
+
+ my $luns = delete($data->{luns}) // {};
+ my $summary = { luns => scalar(keys %$luns) };
+ $summary->{ $_->{state} // 'missing' }++ for values %$luns;
+ $data->{summary} = $summary;
+ return $status_json->encode($data);
+}
+
+# Send on change, and unchanged once per refresh interval so readers can tell a current report from
+# the last one of a node that stopped reporting. A failed send is retried on the next call.
+my $last_kv;
+
+my sub update_node_kv($status) {
+ require PVE::Cluster;
+
+ my $content = $status_json->encode($status);
+ my $now = time();
+ return
+ if $last_kv
+ && $content eq $last_kv->{content}
+ && $now - $last_kv->{time} < $KV_REFRESH_SECONDS;
+
+ eval {
+ # the cluster library only warns when the send fails
+ local $SIG{__WARN__} = sub($line) { die $line };
+ PVE::Cluster::broadcast_node_kv($KV_KEY, encode_status({ $status->%*, time => $now }));
+ };
+ if (my $err = $@) {
+ warn "multipath: broadcasting the status failed - $err";
+ return;
+ }
+ $last_kv = { content => $content, time => $now };
+}
+
+# Unmanaged-map counts need no finer freshness than the broadcast refresh interval.
+my $idle_maps;
+
+# Reject malformed or oversized initiator names rather than truncate identities used in target ACLs.
+my sub valid_iscsi_name($name) {
+ return 0 if !defined($name) || length($name) > 223;
+ my $valid = $name =~ m/^(?:
+ iqn\.\d{4}-\d{2}\.[A-Za-z0-9][A-Za-z0-9._:-]*
+ | eui\.[0-9A-Fa-f]{16}
+ | naa\.[0-9A-Fa-f]{16}(?:[0-9A-Fa-f]{16})?
+ )$/x;
+ return $valid ? 1 : 0;
+}
+
+# Publish even without tools: the report's presence also signals support for the storage option.
+sub broadcast_status($apply) {
+ $apply //= {};
+ my $wanted = $apply->{wwids} // {};
+ my $status = {
+ supported => PVE::Multipath::is_supported() ? 1 : 0,
+ defined($apply->{error}) ? ('apply-error' => "$apply->{error}") : (),
+ $apply->{unresolved} ? ('unresolved-luns' => $apply->{unresolved}) : (),
+ };
+
+ require PVE::Storage::ISCSIPlugin;
+ my $initiator = eval { PVE::Storage::ISCSIPlugin::read_initiator_info() };
+ $status->{'iscsi-initiator-name'} = $initiator->{name}
+ if $initiator && valid_iscsi_name($initiator->{name});
+
+ my $maps = [];
+ if ($status->{supported}) {
+ $status->{running} = PVE::Multipath::is_running() ? 1 : 0;
+ if ($status->{running}) {
+ if (!%$wanted && $idle_maps && time() - $idle_maps->{time} < $KV_REFRESH_SECONDS) {
+ $maps = $idle_maps->{maps};
+ } else {
+ $maps = eval { PVE::Multipath::get_maps() };
+ if (my $err = $@) {
+ $status->{'report-error'} = "querying multipathd failed - $err";
+ $maps = [];
+ $idle_maps = undef;
+ } else {
+ $idle_maps = %$wanted ? undef : { time => time(), maps => $maps };
+ }
+ }
+ }
+ }
+
+ my ($luns, $unmanaged) = status_luns($wanted, $maps);
+ $status->{luns} = $luns;
+ $status->{unmanaged} = $unmanaged if $unmanaged;
+
+ eval { update_node_kv($status) };
+ warn "multipath: $@" if $@;
+}
+
+sub read_status_broadcasts() {
+ require PVE::Cluster;
+
+ my $now = time();
+ my $members = PVE::Cluster::get_members() // {};
+ my $raw = PVE::Cluster::get_node_kv($KV_KEY);
+ my $res = {};
+ for my $node (keys %$raw) {
+ my $decoded = eval { decode_json($raw->{$node}) };
+ next if ref($decoded) ne 'HASH';
+
+ my $entry = {
+ supported => $decoded->{supported} ? 1 : 0,
+ running => $decoded->{running} ? 1 : 0,
+ luns => ref($decoded->{luns}) eq 'HASH' ? $decoded->{luns} : undef,
+ time => $decoded->{time},
+ };
+ for my $key (qw(apply-error report-error unmanaged unresolved-luns summary)) {
+ $entry->{$key} = $decoded->{$key} if defined($decoded->{$key});
+ }
+ # a report of an older release has none, a buggy peer may send anything
+ $entry->{'iscsi-initiator-name'} = $decoded->{'iscsi-initiator-name'}
+ if valid_iscsi_name($decoded->{'iscsi-initiator-name'});
+
+ my $reason;
+ if (%$members && (!$members->{$node} || !$members->{$node}->{online})) {
+ $reason = 'offline';
+ } elsif (!defined($decoded->{time}) || $now - $decoded->{time} > $KV_STALE_SECONDS) {
+ $reason = 'stale';
+ } elsif (!$entry->{supported}) {
+ $reason = 'unsupported';
+ } elsif (!$entry->{running}) {
+ $reason = 'not-running';
+ } elsif (defined($entry->{'report-error'})) {
+ $reason = 'query-error';
+ }
+ $entry->{fresh} = defined($reason) ? 0 : 1;
+ $entry->{reason} = $reason if defined($reason);
+
+ $res->{$node} = $entry;
+ }
+ return $res;
+}
# Older nodes can drop the unknown multipath option when rewriting storage.cfg; offline nodes
# may also lack a report, so silence is not evidence of compatibility.
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 88134473..8cbe9308 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -300,7 +300,7 @@ sub storage_has_feature {
}
# Retain empty entries on failure: absence means the storage is no longer enabled here, not merely
-# unreachable.
+# unreachable. scanned requires every portal's completed scan and identified LUNs.
sub derived_multipath_wwids {
my ($cfg, $ids) = @_;
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index 4831d2d4..ad3afaf6 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -751,7 +751,8 @@ sub iscsi_device_list {
return $res;
}
-# Expect one path per node-record portal.
+# Expect one path per node-record portal. scanned requires completed, stable sessions on every
+# portal and all LUNs identified; an incomplete scan must not prove absence or remove owned entries.
sub multipath_lun_wwids {
my ($class, $scfg, $ids) = @_;
@@ -760,13 +761,19 @@ sub multipath_lun_wwids {
$ids //= PVE::Multipath::Identity::collect(targets => { $target => 1 });
my $luns = PVE::Multipath::Identity::target_luns($ids, $target);
- return $res if !scalar(@$luns);
- my $paths_expected = scalar(iscsi_portals($target, $scfg->{portal})->@*);
+ my $portals = iscsi_portals($target, $scfg->{portal});
+ my @unresolved;
for my $lun (@$luns) {
- next if !defined($lun->{wwid});
- $res->{wwids}->{ $lun->{wwid} } = { 'paths-expected' => $paths_expected };
+ if (!defined($lun->{wwid})) {
+ push @unresolved, "$target $lun->{lun}";
+ next;
+ }
+ $res->{wwids}->{ $lun->{wwid} } = { 'paths-expected' => scalar(@$portals) };
}
+ $res->{unresolved} = [sort @unresolved] if @unresolved;
+ $res->{scanned} = 1
+ if !@unresolved && PVE::Multipath::Identity::portals_covered($ids, $target, $portals);
return $res;
}
diff --git a/src/test/run_lun_identity_tests.pl b/src/test/run_lun_identity_tests.pl
index 33b92d7c..783fac18 100755
--- a/src/test/run_lun_identity_tests.pl
+++ b/src/test/run_lun_identity_tests.pl
@@ -260,18 +260,8 @@ my sub put {
close($fh);
}
-# An iSCSI session as the kernel's transport class shows it: its SCSI target below the session
-# device, which the session class links to.
+# the SCSI targets of the sessions, see session_device()
my %session_dirs;
-my sub session {
- my ($sid, $target, $scsi_target) = @_;
- my ($host) = $scsi_target =~ m/^(\d+):/;
- my $dir = "devices/platform/host$host/session$sid";
- make_path("$sys/$dir/target$scsi_target");
- put("class/iscsi_session/session$sid/targetname", "$target\n");
- symlink("$sys/$dir", "$sys/class/iscsi_session/session$sid/device") or die "$!\n";
- $session_dirs{$scsi_target} = "$dir/target$scsi_target";
-}
# A SCSI device, below the target of its session if it has one, listed on the SCSI bus.
my sub scsi_device {
@@ -322,9 +312,27 @@ my sub wwids_of {
};
}
+# an iSCSI session on its SCSI host, logged in to $portal and done with its scan for LUNs
+my sub session_device {
+ my ($sid, $host, $target, $portal, $id) = @_;
+ $id //= 0;
+ my $dir = "devices/platform/host$host/session$sid";
+ make_path("$sys/$dir/target$host:0:$id");
+ $session_dirs{"$host:0:$id"} = "$dir/target$host:0:$id";
+ my $class = "class/iscsi_session/session$sid";
+ put("$class/targetname", "$target\n");
+ put("$class/state", "LOGGED_IN\n");
+ put("$class/target_state", "SCANNED\n");
+ my ($address, $port) = $portal =~ m/^(.+):(\d+)$/;
+ my $conn = "$dir/connection$sid:0/iscsi_connection/connection$sid:0";
+ put("$conn/persistent_address", "$address\n");
+ put("$conn/persistent_port", "$port\n");
+ symlink("$sys/$dir", "$sys/$class/device") or die "$!\n";
+}
+
# session 1 to $T on host 7 with LUN 0 (a map) and LUN 1 (no map), session 2 to $O on host 8
-session(1, $T, '7:0:0');
-session(2, $O, '8:0:0');
+session_device(1, 7, $T, '192.0.2.10:3260');
+session_device(2, 8, $O, '192.0.2.20:3260');
disk_device('7:0:0:0', 'sdb', '8:16');
disk_device('7:0:0:1', 'sdc', '8:32');
disk_device('8:0:0:0', 'sdd', '8:48');
@@ -422,9 +430,9 @@ put('bus/scsi/devices/7:0:0:1/block/sdc/dev', "8:32\n");
# the only LUN of a third target: a path with a map, one that the daemon names differently
my $P = 'iqn.2003-01.org.linux-iscsi.third:sn.3';
-session(3, $P, '9:0:0');
+session_device(3, 9, $P, '192.0.2.30:3260');
# a second session to it on the same SCSI host, as offload initiators run them
-session(4, $P, '9:0:1');
+session_device(7, 9, $P, '192.0.2.31:3260', 1);
disk_device('9:0:0:0', 'sde', '8:64');
disk_device('9:0:1:0', 'sdf', '8:80');
hold('9:0:0:0', 'sde', 'dm-2', '3W9');
@@ -445,12 +453,12 @@ is_deeply(
# two targets through one SCSI host, as offload initiators run them: each LUN stays with the target
# of its own session
my ($Q, $R) = ('iqn.2003-01.org.linux-iscsi.fourth:sn.4', 'iqn.2003-01.org.linux-iscsi.fifth:sn.5');
-session(5, $Q, '10:0:0');
-session(6, $R, '10:0:1');
-disk_device('10:0:0:0', 'sdg', '8:96');
-disk_device('10:0:1:0', 'sdh', '8:112');
-hold('10:0:0:0', 'sdg', 'dm-3', '3WQ');
-hold('10:0:1:0', 'sdh', 'dm-4', '3WR');
+session_device(8, 11, $Q, '192.0.2.60:3260');
+session_device(9, 11, $R, '192.0.2.70:3260', 1);
+disk_device('11:0:0:0', 'sdg', '8:96');
+disk_device('11:0:1:0', 'sdh', '8:112');
+hold('11:0:0:0', 'sdg', 'dm-3', '3WQ');
+hold('11:0:1:0', 'sdh', 'dm-4', '3WR');
for my $case ([$Q, '3WQ', 'sdg'], [$R, '3WR', 'sdh']) {
my ($target, $wwid, $path) = @$case;
@@ -503,8 +511,8 @@ my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
$iscsi->mock(iscsi_portals => sub { ['192.0.2.10:3260', '192.0.2.11:3260'] });
is_deeply(
PVE::Storage::ISCSIPlugin->multipath_lun_wwids({ target => $T, portal => '192.0.2.10' }, $ids),
- { wwids => { '3W0' => { 'paths-expected' => 2 } } },
- 'derived: the resolved LUNs, with a path per portal expected',
+ { wwids => { '3W0' => { 'paths-expected' => 2 } }, unresolved => ["$T 1"] },
+ 'derived: the resolved LUNs, with a path per portal expected, and the others',
);
my $identity = Test::MockModule->new('PVE::Multipath::Identity');
@@ -555,4 +563,187 @@ my $derived = PVE::Storage::derived_multipath_wwids({
is_deeply([sort keys %$derived], ['a', 'b'], 'derived: every storage with the option');
is($collected, 1, 'derived: the identities are collected once for all of them');
+# --- whether a storage looked at all its LUNs ---
+
+my sub session {
+ my ($sid, $target, $portal, $scan, %extra) = @_;
+ return (
+ $sid => {
+ target => $target,
+ host => $sid + 6,
+ state => $extra{state} // 'LOGGED_IN',
+ scan => $scan,
+ portal => $portal,
+ },
+ );
+}
+my $portals = ['192.0.2.10:3260', '192.0.2.11:3260'];
+my $both = { session(1, $T, $portals->[0], 'SCANNED'), session(2, $T, $portals->[1], 'SCANNED') };
+my sub covered {
+ my ($before, $after) = @_;
+ return PVE::Multipath::Identity::portals_covered(
+ { sessions => $before, 'sessions-after' => $after // $before },
+ $T,
+ $portals,
+ );
+}
+ok(covered($both), 'covered: every portal with a session that finished its scan');
+ok(!covered({ session(1, $T, $portals->[0], 'SCANNED') }),
+ 'covered: not with a portal without one');
+ok(
+ !covered({ $both->%*, session(2, $T, $portals->[1], 'ALLOCATED') }),
+ 'covered: not while the scan of a session still runs',
+);
+ok(
+ !covered({ $both->%*, session(2, $T, $portals->[1], '') }),
+ 'covered: not without the scan state',
+);
+ok(
+ !covered({ $both->%*, session(2, $T, $portals->[1], 'SCANNED', state => 'FAILED') }),
+ 'covered: not with a session that is not logged in',
+);
+ok(
+ !covered(
+ $both,
+ { session(1, $T, $portals->[0], 'SCANNED'), session(3, $T, $portals->[1], 'SCANNED') },
+ ),
+ 'covered: not if a session changed while the LUNs were collected',
+);
+ok(
+ !PVE::Multipath::Identity::portals_covered({ sessions => {}, 'sessions-after' => {} }, $T, []),
+ 'covered: not without any portal',
+);
+
+$ids = {
+ luns => PVE::Multipath::Identity::resolve(
+ { disk(0, 1, map => '3W0'), disk(1, 2) },
+ { line(2, 1, '3W1') },
+ $naa_ok,
+ ),
+ sessions => $both,
+ 'sessions-after' => $both,
+};
+$iscsi->mock(iscsi_portals => sub { $portals });
+my $scfg = { target => $T, portal => '192.0.2.10' };
+is_deeply(
+ PVE::Storage::ISCSIPlugin->multipath_lun_wwids($scfg, $ids),
+ {
+ wwids => { '3W0' => { 'paths-expected' => 2 }, '3W1' => { 'paths-expected' => 2 } },
+ scanned => 1,
+ },
+ 'derived: every LUN identified on every portal: looked at all of them',
+);
+$ids->{luns} = PVE::Multipath::Identity::resolve(
+ { disk(0, 1, map => '3W0'), disk(1, 2) },
+ { line(2, 1, '3X1', init => 'udev-missing') },
+ $naa_ok,
+);
+is_deeply(
+ PVE::Storage::ISCSIPlugin->multipath_lun_wwids($scfg, $ids),
+ { wwids => { '3W0' => { 'paths-expected' => 2 } }, unresolved => ["$T 1"] },
+ 'derived: a LUN not identified: the others still, but not looked at all of them',
+);
+$ids->{luns} = {};
+is_deeply(
+ PVE::Storage::ISCSIPlugin->multipath_lun_wwids($scfg, $ids),
+ { wwids => {}, scanned => 1 },
+ 'derived: no LUN left on a target that every portal reaches',
+);
+$ids->{sessions} =
+ { session(1, $T, $portals->[0], 'ALLOCATED'), session(2, $T, $portals->[1], 'SCANNED') };
+is_deeply(
+ PVE::Storage::ISCSIPlugin->multipath_lun_wwids($scfg, $ids),
+ { wwids => {} },
+ 'derived: but not while a scan of the target runs',
+);
+
+is(
+ PVE::Multipath::Generator::unresolved_lun_count({
+ a => { unresolved => ["$T 1", "$T 2"] },
+ b => { unresolved => ["$T 2"] },
+ c => {},
+ }),
+ 2,
+ 'unresolved: a LUN listed by two storages counts once',
+);
+
+# --- whether a storage looked at all its LUNs, from sysfs ---
+
+{
+ $iscsi->unmock_all();
+ $identity->unmock_all();
+ my $portal = '192.0.2.10:3260';
+ $iscsi->mock(iscsi_portals => sub { [$portal] });
+ my $scfg = { target => $T, portal => '192.0.2.10', multipath => 1 };
+ $reply = {
+ sdc => {
+ devt => '8:32',
+ hctl => '7:0:0:1',
+ init => 'ok',
+ target => $T,
+ wwid => '3600a09803832355367242f747a716b7a',
+ },
+ };
+ my sub derived { PVE::Storage::ISCSIPlugin->multipath_lun_wwids($scfg) }
+
+ ok(derived()->{scanned}, 'sysfs: every portal covered, every LUN identified');
+ $iscsi->mock(iscsi_portals => sub { [$portal, '192.0.2.11:3260'] });
+ ok(!derived()->{scanned}, 'sysfs: not with a portal without a session');
+ $iscsi->mock(iscsi_portals => sub { [$portal] });
+ put('class/iscsi_session/session1/target_state', "ALLOCATED\n");
+ ok(
+ !derived()->{scanned} && !derived()->{scanned},
+ 'sysfs: not while the scan still runs, again and again',
+ );
+ put('class/iscsi_session/session1/target_state', "SCANNED\n");
+ {
+ # the session goes through a recovery while the LUNs are collected
+ $multipath->mock(
+ query_path_identities => sub {
+ put('class/iscsi_session/session1/state', "FAILED\n");
+ return $reply;
+ },
+ );
+ ok(!derived()->{scanned}, 'sysfs: not if the session changed during the collection');
+ put('class/iscsi_session/session1/state', "LOGGED_IN\n");
+ $multipath->mock(query_path_identities =>
+ sub { $queries++; die $reply if !ref($reply); return $reply });
+ }
+
+ # removing the last LUN of a target also makes the kernel remove the session's SCSI target
+ my $empty = 'iqn.2003-01.org.linux-iscsi.empty:sn.4';
+ session_device(4, 10, $empty, '192.0.2.40:3260');
+ rmdir("$sys/devices/platform/host10/session4/target10:0:0") or die "$!\n";
+ $iscsi->mock(iscsi_portals => sub { ['192.0.2.40:3260'] });
+ is_deeply(
+ PVE::Storage::ISCSIPlugin->multipath_lun_wwids({
+ target => $empty,
+ portal => '192.0.2.40',
+ }),
+ { wwids => {}, scanned => 1 },
+ 'sysfs: a target without LUNs, all of them gone, looked at',
+ );
+}
+
+# --- the count of LUNs a node cannot identify, through its report ---
+
+{
+ my $cluster = Test::MockModule->new('PVE::Cluster');
+ my $sent;
+ $cluster->mock(
+ broadcast_node_kv => sub { $sent = $_[1] },
+ get_node_kv => sub { { lab1 => $sent } },
+ get_members => sub { { lab1 => { online => 1 } } },
+ rrd_dump => sub { {} },
+ );
+ $multipath->mock(is_supported => sub { 0 });
+ PVE::Multipath::Status::broadcast_status({ wwids => {}, unresolved => 2 });
+ is(
+ PVE::Multipath::Status::read_status_broadcasts()->{lab1}->{'unresolved-luns'},
+ 2,
+ 'report: the count of LUNs the node cannot identify',
+ );
+ $multipath->unmock('is_supported');
+}
+
done_testing();
diff --git a/src/test/run_multipath_api_tests.pl b/src/test/run_multipath_api_tests.pl
index fdd9ec39..3f0c8a92 100755
--- a/src/test/run_multipath_api_tests.pl
+++ b/src/test/run_multipath_api_tests.pl
@@ -22,6 +22,80 @@ use PVE::Storage;
use PVE::Storage::ISCSIPlugin;
use PVE::Multipath::Status;
+subtest 'every node sends a bounded status report' => sub {
+ my $kv = {};
+ my $cluster = Test::MockModule->new('PVE::Cluster');
+ $cluster->mock(
+ get_node_kv => sub {
+ +{ map { $_ => $kv->{$_} } keys %$kv };
+ },
+ );
+ $cluster->mock(get_members => sub { {} });
+ my $node = 'n1';
+ $cluster->mock(
+ broadcast_node_kv => sub {
+ die "oversized message\n" if length($_[1]) >= 32768;
+ $kv->{$node} = $_[1];
+ },
+ );
+ my $mp = Test::MockModule->new('PVE::Multipath');
+ my $supported = 0;
+ $mp->mock(is_supported => sub { $supported });
+ $mp->mock(is_running => sub { 1 });
+ my $maps = [];
+ $mp->mock(get_maps => sub { $maps });
+
+ PVE::Multipath::Status::broadcast_status({});
+ my $reports = PVE::Multipath::Status::read_status_broadcasts();
+ ok($reports->{n1}, 'a node without multipath-tools still reports');
+ is($reports->{n1}->{reason}, 'unsupported', 'and says why it has no map states');
+
+ $node = 'n2';
+ $supported = 1;
+ $maps = [{ wwid => 'w1', health => 'optimal', 'paths-active' => 2, 'paths-total' => 2 }];
+ PVE::Multipath::Status::broadcast_status({
+ wwids => { w1 => {}, w2 => { storages => ['san'] } } });
+ $reports = PVE::Multipath::Status::read_status_broadcasts();
+ ok($reports->{n2}->{fresh}, 'a node with a running daemon reports current states');
+ is($reports->{n2}->{luns}->{w1}->{state}, 'optimal', 'with the state of an assembled map');
+ is_deeply($reports->{n2}->{luns}->{w2}, { storages => ['san'] }, 'and the unassembled LUN');
+
+ $node = 'n3';
+ $mp->mock(get_maps => sub { die "query failed\n" });
+ PVE::Multipath::Status::broadcast_status({ wwids => { w1 => {} } });
+ $reports = PVE::Multipath::Status::read_status_broadcasts();
+ is(
+ $reports->{n3}->{reason},
+ 'query-error',
+ 'a failing query is not evidence of a missing map',
+ );
+ ok($reports->{n3}->{luns}->{w1}, 'and keeps the wanted LUNs listed');
+
+ $node = 'n5';
+ my $queries = 0;
+ $mp->mock(get_maps => sub { $queries++; die "query failed\n" });
+ PVE::Multipath::Status::broadcast_status({});
+ $mp->mock(get_maps => sub { $queries++; [] });
+ PVE::Multipath::Status::broadcast_status({});
+ is($queries, 2, 'a failed query of an idle node is not reused');
+ ok(
+ PVE::Multipath::Status::read_status_broadcasts()->{n5}->{fresh},
+ 'and the next one counts',
+ );
+ PVE::Multipath::Status::broadcast_status({});
+ is($queries, 2, 'while a successful one is');
+
+ $node = 'n4';
+ $mp->mock(get_maps => sub { [] });
+ my $wanted = { map { (sprintf('3600%032x', $_) => { storages => ['san'] }) } 1 .. 1000 };
+ PVE::Multipath::Status::broadcast_status({ wwids => $wanted, error => 'timeout ' x 10000 });
+ $reports = PVE::Multipath::Status::read_status_broadcasts();
+ ok($reports->{n4}->{fresh}, 'a node with more LUNs than fit still reports');
+ ok(!defined($reports->{n4}->{luns}), 'without a partial list');
+ is_deeply($reports->{n4}->{summary}, { luns => 1000, missing => 1000 }, 'but with counts');
+ ok(length($reports->{n4}->{'apply-error'}) < 2100, 'and a bounded apply error');
+};
+
subtest 'enabling the multipath option needs every node to know it' => sub {
my $cluster = Test::MockModule->new('PVE::Cluster');
my $inotify = Test::MockModule->new('PVE::INotify');
@@ -136,6 +210,83 @@ subtest 'the iSCSI initiator identity reader' => sub {
}
};
+subtest 'the report carries the iSCSI initiator name of the node' => sub {
+ my $kv = {};
+ my $cluster = Test::MockModule->new('PVE::Cluster');
+ $cluster->mock(get_node_kv => sub { +{%$kv} });
+ $cluster->mock(get_members => sub { {} });
+ $cluster->mock(rrd_dump => sub { {} });
+ $cluster->mock(broadcast_node_kv => sub { $kv->{n1} = $_[1] });
+ my $mp = Test::MockModule->new('PVE::Multipath');
+ my $supported = 0;
+ $mp->mock(is_supported => sub { $supported });
+ $mp->mock(is_running => sub { 1 });
+ $mp->mock(get_maps => sub { [] });
+ my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+ my $initiator;
+ $iscsi->mock(read_initiator_info => sub { $initiator });
+ my $sent = sub {
+ my ($apply) = @_;
+ PVE::Multipath::Status::broadcast_status($apply // {});
+ return PVE::Multipath::Status::read_status_broadcasts()->{n1};
+ };
+
+ my $iqn = 'iqn.2026-10.test.lab:n1';
+ $initiator = { name => $iqn };
+ is($sent->()->{'iscsi-initiator-name'}, $iqn, 'a node without multipath-tools sends it');
+
+ $supported = 1;
+ $initiator = { name => 'iqn.2026-10.test.lab:renamed', alias => 'n1' };
+ is(
+ $sent->()->{'iscsi-initiator-name'},
+ 'iqn.2026-10.test.lab:renamed',
+ 'a changed name goes out with the next report',
+ );
+
+ $initiator = undef;
+ ok(!exists($sent->()->{'iscsi-initiator-name'}), 'a removed one is gone from it');
+
+ $iscsi->mock(read_initiator_info => sub { die "unreadable\n" });
+ ok(!exists($sent->()->{'iscsi-initiator-name'}), 'an unreadable file only leaves it out');
+ $iscsi->mock(read_initiator_info => sub { $initiator });
+
+ my $invalid = [
+ ['iqn.2026-10.test.lab:' . ('x' x 210), 'a name too long to be an iSCSI name'],
+ ['iqn.bogus', 'an IQN without a date'],
+ ['iqn.2026-10.a b', 'one with a blank'],
+ ];
+ for my $case ($invalid->@*) {
+ $initiator = { name => $case->[0] };
+ ok(!exists($sent->()->{'iscsi-initiator-name'}), "is left out, not cut: $case->[1]");
+ }
+ for my $name ('eui.02004567A425678D', 'naa.52004567BA64678D') {
+ $initiator = { name => $name };
+ is($sent->()->{'iscsi-initiator-name'}, $name, "the other name formats: $name");
+ }
+
+ $initiator = { name => $iqn };
+ my $wanted = { map { (sprintf('3600%032x', $_) => { storages => ['san'] }) } 1 .. 1000 };
+ my $report = $sent->({ wwids => $wanted });
+ ok($report->{summary}, 'a node with too many LUNs sends a summary');
+ is($report->{'iscsi-initiator-name'}, $iqn, 'with the name');
+
+ $kv->{n1} = JSON::encode_json({ supported => 1, running => 1, luns => {}, time => time() });
+ ok(
+ !exists(
+ PVE::Multipath::Status::read_status_broadcasts()->{n1}->{'iscsi-initiator-name'}
+ ),
+ 'a report of an older release reads without one',
+ );
+ $kv->{n1} =
+ JSON::encode_json({ luns => {}, time => time(), 'iscsi-initiator-name' => ['x'] });
+ ok(
+ !exists(
+ PVE::Multipath::Status::read_status_broadcasts()->{n1}->{'iscsi-initiator-name'}
+ ),
+ 'and a garbled one from a peer is dropped',
+ );
+};
+
subtest 'an entry stays allow-listed while its map is in use' => sub {
my $mp = Test::MockModule->new('PVE::Multipath');
my $map = { wwid => 'w1', sysfs => 'dm-999999', 'paths-active' => 2 };
diff --git a/src/test/run_multipath_tests.pl b/src/test/run_multipath_tests.pl
index 992dd3dd..1c28ac86 100755
--- a/src/test/run_multipath_tests.pl
+++ b/src/test/run_multipath_tests.pl
@@ -3,6 +3,7 @@
use strict;
use warnings;
+use JSON;
use Test::MockModule;
use Test::More;
@@ -367,13 +368,35 @@ is_deeply(
PVE::Multipath::Generator::desired_wwid_info(
{ w1 => ['config', 'storage:a', 'storage:b'], w2 => ['config'] },
{
- a => { wwids => { w1 => { 'paths-expected' => 2 } } },
- b => { wwids => { w1 => { 'paths-expected' => 4 } } },
+ a => { wwids => { w1 => { 'paths-expected' => 2 } }, scanned => 1 },
+ b => { wwids => { w1 => { 'paths-expected' => 4 } }, scanned => 1 },
},
),
- { w1 => { storages => ['a', 'b'], 'paths-expected' => 4 }, w2 => {} },
+ { w1 => { storages => ['a', 'b'], 'paths-expected' => 4, derived => 1 }, w2 => {} },
'every deriving storage is named, the largest path expectation wins',
);
+is_deeply(
+ PVE::Multipath::Generator::desired_wwid_info(
+ {
+ gone => ['storage:a'],
+ unseen => ['storage:a', 'storage:b'],
+ once => ['storage:a', 'storage:b'],
+ failed => ['storage:c'],
+ },
+ {
+ a => { wwids => { once => {} }, scanned => 1 },
+ b => { wwids => {} },
+ c => { wwids => {}, error => 'boom' },
+ },
+ ),
+ {
+ gone => { storages => ['a'], derived => 0 },
+ unseen => { storages => ['a', 'b'] },
+ once => { storages => ['a', 'b'], derived => 1 },
+ failed => { storages => ['c'] },
+ },
+ 'derived: 0 only if every storage looked and none found it, unknown if one could not look',
+);
# --- WWID allow-list reconcile plan: only prune what PVE itself added, and only once every ---
# --- reason it was added for is gone from the configuration ---
@@ -632,4 +655,89 @@ is(
);
}
+# --- per-LUN status report ---
+my $wanted_all = { map { $_->{wwid} => {} } $maps->@* };
+my ($luns, $unmanaged) = PVE::Multipath::Status::status_luns($wanted_all, $maps);
+is_deeply([sort keys %$luns], [sort keys %$wanted_all], 'every wanted LUN is reported');
+is($unmanaged, 0, 'no unmanaged maps when everything is wanted');
+is($luns->{ $a->{wwid} }->{state}, 'optimal', 'optimal map reported as optimal');
+is($luns->{ $a->{wwid} }->{'paths-active'}, 2, 'active path count carried');
+is($luns->{ $a->{wwid} }->{'paths-total'}, 2, 'total path count carried');
+is($luns->{ $b->{wwid} }->{state}, 'degraded', 'degraded map reported as degraded');
+is($luns->{ $c->{wwid} }->{state}, 'failed', 'failed map reported as failed');
+ok(!exists $luns->{ $a->{wwid} }->{transport}, 'transport omitted when not derived');
+
+($luns, $unmanaged) = PVE::Multipath::Status::status_luns({}, []);
+is_deeply($luns, {}, 'nothing wanted and nothing assembled reports nothing');
+
+($luns, $unmanaged) = PVE::Multipath::Status::status_luns(
+ { $a->{wwid} => { storages => ['san1'], 'paths-expected' => 4 }, w404 => {} }, $maps,
+);
+is_deeply([sort keys %$luns], [sort($a->{wwid}, 'w404')], 'only wanted LUNs are listed');
+is($unmanaged, 2, 'other maps are counted as unmanaged');
+is(
+ $luns->{ $a->{wwid} }->{state},
+ 'degraded',
+ 'two active paths of four expected read as degraded, the missing ones are gone from the total',
+);
+is_deeply($luns->{ $a->{wwid} }->{storages}, ['san1'], 'the deriving storages are carried');
+is_deeply($luns->{w404}, {}, 'a wanted LUN without a map is listed without a state');
+($luns) =
+ PVE::Multipath::Status::status_luns({ w404 => { storages => ['san1'], derived => 0 } }, $maps);
+is_deeply(
+ $luns->{w404},
+ { storages => ['san1'], derived => 0 },
+ 'whether the storage still derives the LUN is carried',
+);
+($luns) = PVE::Multipath::Status::status_luns({ $a->{wwid} => { 'paths-expected' => 2 } }, $maps);
+is($luns->{ $a->{wwid} }->{state}, 'optimal', 'two of two expected stay optimal');
+
+my $enriched = [{
+ wwid => '3600x',
+ health => 'optimal',
+ 'paths-active' => 2,
+ 'paths-total' => 2,
+ transport => 'iscsi',
+ size => 34359738368,
+}];
+($luns) = PVE::Multipath::Status::status_luns({ '3600x' => { kept => 'LVM' } }, $enriched);
+is($luns->{'3600x'}->{transport}, 'iscsi', 'transport carried into the report');
+is($luns->{'3600x'}->{size}, 34359738368, 'size carried into the report');
+is($luns->{'3600x'}->{kept}, 'LVM', 'what keeps an unconfigured LUN is carried');
+
+# --- encoding within the KV size limit ---
+my $small = PVE::Multipath::Status::encode_status({
+ supported => 1,
+ running => 1,
+ luns => { w1 => { state => 'optimal' } },
+ 'apply-error' => 'x' x 5000,
+});
+my $decoded = decode_json($small);
+is_deeply($decoded->{luns}, { w1 => { state => 'optimal' } }, 'a small status keeps its LUNs');
+ok(length($decoded->{'apply-error'}) < 2100, 'an oversized apply error is cut down');
+
+my $many = {
+ map { (
+ sprintf('3600140500000000000000000000%04x', $_) => {
+ storages => ['san1'],
+ 'paths-expected' => 4,
+ state => $_ % 100 ? 'optimal' : 'degraded',
+ 'paths-active' => 4,
+ 'paths-total' => 4,
+ transport => 'iscsi',
+ size => 1099511627776,
+ },
+ ) } 0 .. 999
+};
+$many->{wNone} = {};
+my $big = PVE::Multipath::Status::encode_status({ supported => 1, running => 1, luns => $many });
+ok(length($big) <= 30 * 1024, 'a large status stays within the KV limit');
+$decoded = decode_json($big);
+ok(!exists $decoded->{luns}, 'a large status sends no partial LUN list');
+is_deeply(
+ $decoded->{summary},
+ { luns => 1001, optimal => 990, degraded => 10, missing => 1 },
+ 'but how many LUNs are in which state',
+);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 34/76] api: add endpoints for the cluster-wide multipath configuration
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (32 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 33/76] multipath: publish per-node status Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 35/76] api: multipath: add the cluster-wide health status Thomas Lamprecht
` (41 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Expose defaults, verbatim overrides, and explicit LUN entries under
/cluster/multipath, with digests protecting against concurrent edits.
Reject aliases matching another LUN's alias or WWID: multipathd drops
one of two maps claiming the same name. Refuse entry removal or node
exclusion while LVM uses the map there as a base volume, unless another
configuration keeps it allow-listed. For LVM without a base volume,
such as Fibre Channel setups, the generator's local checks keep the LUN
allow-listed while its map holds the volume group.
Adding entries or expanding node coverage requires cluster support and
the necessary tools. Maintenance, narrowing, and removal remain
available during upgrades.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Makefile | 1 +
src/PVE/API2/Multipath.pm | 501 ++++++++++++++++++++++++++++
src/PVE/Multipath.pm | 24 ++
src/PVE/Multipath/Status.pm | 8 +-
src/test/run_multipath_api_tests.pl | 162 ++++++++-
src/test/run_multipath_tests.pl | 38 +++
6 files changed, 720 insertions(+), 14 deletions(-)
create mode 100644 src/PVE/API2/Multipath.pm
diff --git a/src/PVE/API2/Makefile b/src/PVE/API2/Makefile
index 10e18f95..9ab978b0 100644
--- a/src/PVE/API2/Makefile
+++ b/src/PVE/API2/Makefile
@@ -4,5 +4,6 @@
install:
install -D -m 0644 Disks.pm ${DESTDIR}${PERLDIR}/PVE/API2/Disks.pm
install -D -m 0644 ISCSI.pm ${DESTDIR}${PERLDIR}/PVE/API2/ISCSI.pm
+ install -D -m 0644 Multipath.pm ${DESTDIR}${PERLDIR}/PVE/API2/Multipath.pm
make -C Storage install
make -C Disks install
diff --git a/src/PVE/API2/Multipath.pm b/src/PVE/API2/Multipath.pm
new file mode 100644
index 00000000..595cb00b
--- /dev/null
+++ b/src/PVE/API2/Multipath.pm
@@ -0,0 +1,501 @@
+package PVE::API2::Multipath;
+
+use strict;
+use warnings;
+
+use PVE::Cluster;
+use PVE::Exception qw(raise_param_exc);
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::Storage;
+use PVE::Tools qw(extract_param);
+
+use PVE::Multipath;
+use PVE::Multipath::Config;
+use PVE::Multipath::ClusterConfig;
+
+use PVE::RESTHandler;
+use PVE::Multipath::Status;
+
+use base qw(PVE::RESTHandler);
+
+# multipathd resolves aliases against all maps; an alias colliding with another alias or WWID
+# can make it drop a map. Include derived LUNs, which need not have config sections.
+my sub assert_map_name_free {
+ my ($cfg, $wwid, $name) = @_;
+
+ for my $other (sort keys $cfg->{ids}->%*) {
+ next if $other eq $wwid;
+ my $section = $cfg->{ids}->{$other};
+ next if ($section->{type} // '') ne 'wwid';
+ die "'$name' is already the map name of WWID '$other'\n"
+ if $other eq $name || (defined($section->{alias}) && $section->{alias} eq $name);
+ }
+ return if $name eq $wwid;
+
+ my $reports = PVE::Multipath::Status::read_status_broadcasts();
+ for my $node (sort keys %$reports) {
+ my $luns = $reports->{$node}->{luns} // next;
+ die "'$name' is the WWID of a LUN with a multipath map on node '$node'\n" if $luns->{$name};
+ }
+}
+
+# An undefined entry node set means removal, whereas an empty set covers every node.
+my sub assert_consumers_covered {
+ my ($wwid, $entry_nodes) = @_;
+
+ my $uncovered = PVE::Multipath::uncovered_consumer_nodes(
+ $wwid,
+ $entry_nodes,
+ PVE::Storage::config(),
+ PVE::Cluster::get_nodelist(),
+ );
+ for my $storeid (sort keys %$uncovered) {
+ die "storage '$storeid' uses the multipath map of WWID '$wwid' on node(s) "
+ . join(', ', $uncovered->{$storeid}->@*)
+ . ", which would no longer allow-list the LUN\n";
+ }
+}
+
+my sub apply_section_update {
+ my ($section, $param, $delete, $settable) = @_;
+
+ eval { PVE::Multipath::Config::apply_section_update($section, $param, $delete, $settable) };
+ raise_param_exc({ delete => $@ }) if $@;
+}
+
+# Caller must hold the config lock through the subsequent write.
+my sub read_config_unlocked {
+ my ($digest) = @_;
+
+ my $cfg = PVE::Multipath::ClusterConfig::read_config();
+ PVE::Tools::assert_if_modified($cfg->{digest}, $digest);
+ return $cfg;
+}
+
+my sub edit_config_locked {
+ my ($digest, $errmsg, $code) = @_;
+
+ PVE::Multipath::ClusterConfig::lock_config(
+ sub {
+ my $cfg = read_config_unlocked($digest);
+ $code->($cfg);
+ PVE::Multipath::ClusterConfig::write_config($cfg);
+ },
+ $errmsg,
+ );
+}
+
+# The global defaults section shares the WWID namespace and must not be accessible here.
+my sub assert_valid_wwid {
+ my ($wwid) = @_;
+
+ raise_param_exc({ wwid => "does not look like a valid WWID" })
+ if !defined($wwid)
+ || $wwid !~ $PVE::Multipath::Config::WWID_RE
+ || $wwid eq 'defaults';
+}
+
+__PACKAGE__->register_method({
+ name => 'index',
+ path => '',
+ method => 'GET',
+ description => "Directory index.",
+ permissions => { user => 'all' },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => {
+ name => { type => 'string' },
+ },
+ },
+ links => [{ rel => 'child', href => "{name}" }],
+ },
+ code => sub {
+ return [map { +{ name => $_ } } qw(defaults overrides wwid)];
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'read_overrides',
+ path => 'overrides',
+ method => 'GET',
+ description => "Read the verbatim multipath.conf override sections.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ overrides => {
+ type => 'string',
+ description => 'Verbatim override sections.',
+ optional => 1,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ code => sub {
+ my $overrides = PVE::Multipath::ClusterConfig::read_overrides();
+ return {
+ digest => PVE::Multipath::ClusterConfig::overrides_digest($overrides),
+ defined($overrides) && length($overrides) ? (overrides => $overrides) : (),
+ };
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'set_overrides',
+ path => 'overrides',
+ method => 'PUT',
+ protected => 1,
+ description => "Set the verbatim multipath.conf override sections, such as "
+ . "hardware-specific 'device {}' or 'overrides {}' blocks.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ overrides => {
+ type => 'string',
+ description => 'Verbatim override sections. Omit to clear them.',
+ maxLength => 128 * 1024,
+ optional => 1,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $overrides = $param->{overrides};
+ my $digest = extract_param($param, 'digest');
+ eval { PVE::Multipath::Config::check_overrides($overrides) };
+ raise_param_exc({ overrides => $@ }) if $@;
+
+ PVE::Multipath::ClusterConfig::lock_overrides(
+ sub {
+ my $current = PVE::Multipath::ClusterConfig::read_overrides();
+ PVE::Tools::assert_if_modified(
+ PVE::Multipath::ClusterConfig::overrides_digest($current), $digest,
+ );
+ PVE::Multipath::ClusterConfig::write_overrides($overrides // '');
+ },
+ "updating multipath overrides failed",
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'read_defaults',
+ path => 'defaults',
+ method => 'GET',
+ description => "Read the global multipathd settings applied on every node, including"
+ . " configured values and managed defaults.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ PVE::Multipath::Config::defaults_api_schema()->%*,
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ code => sub {
+ my $cfg = PVE::Multipath::ClusterConfig::read_config();
+ my $res = PVE::Multipath::Config::effective_defaults($cfg);
+ $res->{digest} = $cfg->{digest};
+ return $res;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'set_defaults',
+ path => 'defaults',
+ method => 'PUT',
+ protected => 1,
+ description => "Set the global multipathd defaults applied on every node.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ PVE::Multipath::Config::defaults_api_schema()->%*,
+ delete => {
+ type => 'string',
+ format => 'pve-configid-list',
+ description => 'Comma-separated list of settings to reset to their defaults.',
+ optional => 1,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $delete = extract_param($param, 'delete');
+ my $digest = extract_param($param, 'digest');
+ my $settable = PVE::Multipath::Config::defaults_api_schema();
+
+ edit_config_locked(
+ $digest,
+ "updating multipath defaults failed",
+ sub {
+ my ($cfg) = @_;
+ my $section = $cfg->{ids}->{defaults} //= { type => 'defaults' };
+ apply_section_update($section, $param, $delete, $settable);
+ # An empty defaults section keeps the generator engaged and has no removal endpoint.
+ delete $cfg->{ids}->{defaults} if !grep { $_ ne 'type' } keys %$section;
+ },
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'wwid_index',
+ path => 'wwid',
+ method => 'GET',
+ description => "List the explicit multipath entries with their alias and per-LUN"
+ . " multipathd settings.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'array',
+ items => {
+ type => 'object',
+ properties => {
+ wwid => { type => 'string', description => 'The LUN WWID.' },
+ PVE::Multipath::Config::wwid_api_schema()->%*,
+ },
+ },
+ links => [{ rel => 'child', href => '{wwid}' }],
+ },
+ code => sub {
+ my $cfg = PVE::Multipath::ClusterConfig::read_config();
+ my $sections = PVE::Multipath::Config::wwid_sections($cfg);
+ return [map { { wwid => $_, $sections->{$_}->%* } } sort keys %$sections];
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'read_wwid',
+ path => 'wwid/{wwid}',
+ method => 'GET',
+ description => "Read the explicit multipath entry of a WWID, with the digest of the"
+ . " configuration it was read from.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Audit']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID to read.',
+ maxLength => 128,
+ },
+ },
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ wwid => { type => 'string', description => 'The LUN WWID.' },
+ PVE::Multipath::Config::wwid_api_schema()->%*,
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ code => sub {
+ my ($param) = @_;
+
+ my $wwid = $param->{wwid};
+ assert_valid_wwid($wwid);
+
+ my $cfg = PVE::Multipath::ClusterConfig::read_config();
+ my $entry = PVE::Multipath::Config::wwid_sections($cfg)->{$wwid}
+ or die "WWID '$wwid' is not explicitly allow-listed\n";
+ return { wwid => $wwid, $entry->%*, digest => $cfg->{digest} };
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'add_wwid',
+ path => 'wwid',
+ method => 'POST',
+ protected => 1,
+ description => "Add an explicit multipath entry for a LUN, with an optional alias, node"
+ . " restriction, and multipathd settings. For a LUN of an iSCSI storage with multipath"
+ . " enabled, an entry is only needed to set an alias or per-LUN settings. The entry"
+ . " keeps the LUN managed even if the storage's multipath option is disabled.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID of the LUN to manage via multipath.',
+ maxLength => 128,
+ },
+ PVE::Multipath::Config::wwid_api_schema()->%*,
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $wwid = extract_param($param, 'wwid');
+ my $digest = extract_param($param, 'digest');
+ my $settable = PVE::Multipath::Config::wwid_api_schema();
+ assert_valid_wwid($wwid);
+
+ edit_config_locked(
+ $digest,
+ "adding WWID '$wwid' failed",
+ sub {
+ my ($cfg) = @_;
+ raise_param_exc({ wwid => "WWID '$wwid' is already allow-listed" })
+ if $cfg->{ids}->{$wwid};
+ assert_map_name_free($cfg, $wwid, $wwid);
+ assert_map_name_free($cfg, $wwid, $param->{alias}) if defined($param->{alias});
+ my $section = $cfg->{ids}->{$wwid} = { type => 'wwid' };
+ apply_section_update($section, $param, undef, $settable);
+ PVE::Multipath::Status::assert_cluster_supports_multipath($section->{nodes});
+ },
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'update_wwid',
+ path => 'wwid/{wwid}',
+ method => 'PUT',
+ protected => 1,
+ description => "Update the alias, node restriction, and multipathd settings of an"
+ . " explicit multipath entry.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID to update.',
+ maxLength => 128,
+ },
+ PVE::Multipath::Config::wwid_api_schema()->%*,
+ delete => {
+ type => 'string',
+ format => 'pve-configid-list',
+ description => 'Comma-separated list of settings to unset.',
+ optional => 1,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $wwid = extract_param($param, 'wwid');
+ my $delete = extract_param($param, 'delete');
+ my $digest = extract_param($param, 'digest');
+ my $settable = PVE::Multipath::Config::wwid_api_schema();
+ assert_valid_wwid($wwid);
+
+ edit_config_locked(
+ $digest,
+ "updating WWID '$wwid' failed",
+ sub {
+ my ($cfg) = @_;
+ my $section = $cfg->{ids}->{$wwid}
+ or die "WWID '$wwid' is not explicitly allow-listed, add it first\n";
+ assert_map_name_free($cfg, $wwid, $param->{alias}) if defined($param->{alias});
+ my $old_nodes = $section->{nodes} // {};
+ apply_section_update($section, $param, $delete, $settable);
+ my $nodes = $section->{nodes} // {};
+ if (%$old_nodes && (!%$nodes || grep { !$old_nodes->{$_} } keys %$nodes)) {
+ PVE::Multipath::Status::assert_cluster_supports_multipath($nodes);
+ }
+ assert_consumers_covered($wwid, $section->{nodes} // {})
+ if defined($param->{nodes})
+ || grep { $_ eq 'nodes' } PVE::Tools::split_list($delete // '');
+ },
+ );
+ return undef;
+ },
+});
+
+__PACKAGE__->register_method({
+ name => 'remove_wwid',
+ path => 'wwid/{wwid}',
+ method => 'DELETE',
+ protected => 1,
+ description => "Remove an explicit multipath entry from the multipath allow-list. A"
+ . " node keeps the LUN allow-listed while its map is still in use there.",
+ permissions => {
+ check => ['perm', '/', ['Sys.Modify']],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {
+ wwid => {
+ type => 'string',
+ description => 'The WWID to remove.',
+ maxLength => 128,
+ },
+ digest => get_standard_option('pve-config-digest'),
+ },
+ },
+ returns => { type => 'null' },
+ code => sub {
+ my ($param) = @_;
+
+ my $wwid = $param->{wwid};
+ my $digest = extract_param($param, 'digest');
+ assert_valid_wwid($wwid);
+
+ edit_config_locked(
+ $digest,
+ "removing WWID '$wwid' failed",
+ sub {
+ my ($cfg) = @_;
+ die "WWID '$wwid' is not explicitly allow-listed - a LUN of a multipath-enabled"
+ . " storage is managed through that storage's multipath option\n"
+ if !$cfg->{ids}->{$wwid};
+ assert_consumers_covered($wwid, undef);
+ delete $cfg->{ids}->{$wwid};
+ },
+ );
+ return undef;
+ },
+});
+
+1;
diff --git a/src/PVE/Multipath.pm b/src/PVE/Multipath.pm
index 96f1e84d..acca5a11 100644
--- a/src/PVE/Multipath.pm
+++ b/src/PVE/Multipath.pm
@@ -337,6 +337,30 @@ sub set_managed_wwid_record($record) {
file_set_contents($MANAGED_WWIDS_FILE, PVE::Multipath::Config::format_managed_wwids($record));
}
+# An undefined entry node set means removal, whereas an empty set covers every node.
+sub uncovered_consumer_nodes($wwid, $entry_nodes, $storage_cfg, $all_nodes) {
+ my $ids = $storage_cfg->{ids} // {};
+ my $node_set = sub($nodes) {
+ return $nodes && %$nodes ? {%$nodes} : { map { $_ => 1 } $all_nodes->@* };
+ };
+ my $entry = defined($entry_nodes) ? $node_set->($entry_nodes) : {};
+
+ my $res = {};
+ for my $storeid (sort keys %$ids) {
+ my $scfg = $ids->{$storeid};
+ next if ($scfg->{type} // '') ne 'lvm' || !defined($scfg->{base});
+ my ($baseid, $volname) = $scfg->{base} =~ m/^([a-zA-Z][a-zA-Z0-9\-_.]*):(\S+)$/;
+ next if !defined($volname) || $volname !~ m/(?:^|\.)dm-uuid-mpath-\Q$wwid\E$/;
+
+ my $base = $ids->{$baseid};
+ my $derived = $base && $base->{multipath} ? $node_set->($base->{nodes}) : {};
+ my @uncovered =
+ grep { !$entry->{$_} && !$derived->{$_} } sort keys $node_set->($scfg->{nodes})->%*;
+ $res->{$storeid} = \@uncovered if @uncovered;
+ }
+ return $res;
+}
+
sub reconfigure() {
assert_supported();
run_command([$MULTIPATHD, 'reconfigure'], timeout => $RECONFIGURE_TIMEOUT);
diff --git a/src/PVE/Multipath/Status.pm b/src/PVE/Multipath/Status.pm
index cc8fe49d..67cc42b9 100644
--- a/src/PVE/Multipath/Status.pm
+++ b/src/PVE/Multipath/Status.pm
@@ -233,9 +233,7 @@ sub nodes_without_tools($nodes) {
sub assert_multipath_tools($nodes) {
my $no_tools = nodes_without_tools($nodes);
- die "multipath-tools is not installed on node(s) "
- . join(', ', $no_tools->@*)
- . ", which the storage is available on\n"
+ die "multipath-tools is not installed on node(s) " . join(', ', $no_tools->@*) . "\n"
if scalar($no_tools->@*);
return;
}
@@ -244,8 +242,8 @@ sub assert_cluster_supports_multipath($nodes) {
my $silent = nodes_without_status();
die "node(s) "
. join(', ', $silent->@*)
- . " did not report multipath support yet - upgrade"
- . " every node and make sure it is online before enabling multipath on a storage\n"
+ . " have not reported multipath support - upgrade every node and"
+ . " bring it online before enabling multipath\n"
if scalar($silent->@*);
assert_multipath_tools($nodes);
return;
diff --git a/src/test/run_multipath_api_tests.pl b/src/test/run_multipath_api_tests.pl
index 3f0c8a92..055ade62 100755
--- a/src/test/run_multipath_api_tests.pl
+++ b/src/test/run_multipath_api_tests.pl
@@ -16,7 +16,9 @@ use Storable qw(dclone);
use Test::More;
use Test::MockModule;
+use PVE::API2::Multipath;
use PVE::Multipath;
+use PVE::Multipath::Config;
use PVE::Multipath::Generator;
use PVE::Storage;
use PVE::Storage::ISCSIPlugin;
@@ -128,17 +130,17 @@ subtest 'enabling the multipath option needs every node to know it' => sub {
};
like(
$enable->({}, { multipath => 1 }),
- qr/node\(s\) n3 did not report/,
+ qr/node\(s\) n3 have not reported/,
'enabling is refused',
);
eval { PVE::Storage::ISCSIPlugin->on_add_hook('san', { type => 'iscsi', multipath => 1 }) };
- like($@, qr/did not report/, 'as is adding a storage with it');
+ like($@, qr/have not reported/, 'as is adding a storage with it');
is($enable->({}, { content => 'none' }), '', 'other changes are not affected');
$kv->{n3} = '{"supported":0}';
- like(
+ is(
$enable->({}, { multipath => 1 }),
- qr/multipath-tools is not installed on node\(s\) n3/,
+ "multipath-tools is not installed on node(s) n3\n",
'a node of the storage without multipath-tools refuses it too',
);
is(
@@ -146,21 +148,25 @@ subtest 'enabling the multipath option needs every node to know it' => sub {
'',
'unless the storage is not available on that node',
);
- like(
+ is(
$enable->({ nodes => { n1 => 1 } }, { multipath => 1 }, ['nodes']),
- qr/not installed on node\(s\) n3/,
+ "multipath-tools is not installed on node(s) n3\n",
'dropping the restriction in the same update counts',
);
my $widen = sub {
my ($update, $delete) = @_;
return $enable->({ multipath => 1, nodes => { n1 => 1, n2 => 1 } }, $update, $delete);
};
- like(
+ is(
$widen->({ nodes => { n1 => 1, n3 => 1 } }),
- qr/not installed on node\(s\) n3/,
+ "multipath-tools is not installed on node(s) n3\n",
'an enabled storage cannot be extended to a node without the tools',
);
- like($widen->({}, ['nodes']), qr/n3/, 'nor lose its node restriction');
+ is(
+ $widen->({}, ['nodes']),
+ "multipath-tools is not installed on node(s) n3\n",
+ 'nor lose its node restriction',
+ );
is($widen->({ nodes => { n1 => 1 } }), '', 'restricting it further is fine');
$kv->{n3} = '{"supported":1}';
@@ -458,6 +464,144 @@ subtest 'regeneration prunes only conclusively closed blank maps' => sub {
}
};
+my sub mock_config {
+ my ($raw_ref, $storage_cfg) = @_;
+
+ my $cc = Test::MockModule->new('PVE::Multipath::ClusterConfig');
+ $cc->mock(
+ read_config => sub { PVE::Multipath::Config->parse_config('multipath.cfg', $$raw_ref) },
+ );
+ $cc->mock(write_config =>
+ sub { $$raw_ref = PVE::Multipath::Config->write_config('multipath.cfg', $_[0]) });
+ $cc->mock(lock_config => sub { $_[0]->() });
+ my $storage = Test::MockModule->new('PVE::Storage');
+ $storage->mock(config => sub { dclone($storage_cfg) });
+ my $cluster = Test::MockModule->new('PVE::Cluster');
+ $cluster->mock(get_nodelist => sub { ['n1', 'n2'] });
+ my $mp = Test::MockModule->new('PVE::Multipath');
+ my $status = Test::MockModule->new('PVE::Multipath::Status');
+ $status->mock(read_status_broadcasts => sub { {} });
+ return [$cc, $storage, $cluster, $mp];
+}
+
+subtest 'the configuration keeps the LUNs storages use allow-listed' => sub {
+ my $raw = "wwid: 3600abc\n\talias lun-a\n";
+ my $storage_cfg = {
+ ids => {
+ san => { type => 'iscsi', nodes => { n1 => 1 } },
+ vg => { type => 'lvm', base => 'san:0.0.1.dm-uuid-mpath-3600abc', vgname => 'vg' },
+ },
+ };
+ my $mocks = mock_config(\$raw, $storage_cfg);
+
+ eval { PVE::API2::Multipath->remove_wwid({ wwid => '3600abc' }) };
+ like($@, qr/storage 'vg' uses the multipath map .* on node\(s\) n1, n2/, 'removal refused');
+ like($raw, qr/3600abc/, 'and the entry is kept');
+
+ eval { PVE::API2::Multipath->update_wwid({ wwid => '3600abc', nodes => 'n1' }) };
+ like($@, qr/on node\(s\) n2,/,
+ 'restricting the entry away from a consumer node is refused');
+ unlike($raw, qr/nodes/, 'and leaves the entry as it was');
+
+ PVE::API2::Multipath->update_wwid({ wwid => '3600abc', alias => 'lun-b' });
+ like($raw, qr/alias lun-b/, 'other changes go through');
+
+ $storage_cfg->{ids}->{san} = { type => 'iscsi', multipath => 1 };
+ $mocks = mock_config(\$raw, $storage_cfg);
+ PVE::API2::Multipath->update_wwid({ wwid => '3600abc', nodes => 'n1' });
+ like($raw, qr/nodes n1/, 'a base storage deriving the LUN covers the other nodes');
+ PVE::API2::Multipath->remove_wwid({ wwid => '3600abc' });
+ unlike($raw, qr/3600abc/, 'and lets the entry go');
+};
+
+subtest 'a WWID that multipath would misread is refused' => sub {
+ my $raw = '';
+ my $mocks = mock_config(\$raw, { ids => {} });
+ my ($cc) = @$mocks;
+ my $writes = 0;
+ $cc->mock(write_config => sub { $writes++ });
+ for my $wwid ('-W', '-F', '8:0') {
+ eval { PVE::API2::Multipath->add_wwid({ wwid => $wwid }) };
+ like($@, qr/does not look like a valid WWID/, "'$wwid' is refused");
+ }
+ is($writes, 0, 'without writing the configuration');
+};
+
+subtest 'WWID enablement checks support before writing' => sub {
+ my $raw = '';
+ my $mocks = mock_config(\$raw, { ids => {} });
+ my ($cc, $storage, $cluster, $mp) = @$mocks;
+ my $writes = 0;
+ $cc->mock(
+ write_config => sub {
+ $writes++;
+ $raw = PVE::Multipath::Config->write_config('multipath.cfg', $_[0]);
+ },
+ );
+ my $inotify = Test::MockModule->new('PVE::INotify');
+ $inotify->mock(nodename => sub { 'n1' });
+ my $kv = {};
+ $cluster->mock(get_node_kv => sub { $kv });
+ my $supported = 1;
+ $mp->mock(is_supported => sub { $supported });
+ my $support_error = "node(s) n2 have not reported multipath support - upgrade every node"
+ . " and bring it online before enabling multipath\n";
+ for my $report (undef, 'old-node', '[]') {
+ $kv = defined($report) ? { n2 => $report } : {};
+ eval { PVE::API2::Multipath->add_wwid({ wwid => 'w1', nodes => 'n1' }) };
+ is($@, $support_error, 'silent or incompatible peer prevents a new entry');
+ is($writes, 0, 'refusal does not write configuration');
+ }
+ $kv = { n2 => '{"supported":0}' };
+ eval { PVE::API2::Multipath->add_wwid({ wwid => 'w1' }) };
+ is($@, "multipath-tools is not installed on node(s) n2\n", 'remote tools required');
+ is($writes, 0, 'missing remote tools never write');
+ $supported = 0;
+ eval { PVE::API2::Multipath->add_wwid({ wwid => 'w1', nodes => 'n1' }) };
+ is($@, "multipath-tools is not installed on node(s) n1\n", 'local tools required');
+ is($writes, 0, 'missing local tools never write');
+ eval { PVE::API2::Multipath->add_wwid({ wwid => 'w1' }) };
+ is(
+ $@,
+ "multipath-tools is not installed on node(s) n1, n2\n",
+ 'all affected nodes are listed',
+ );
+ is($writes, 0, 'missing tools on multiple nodes never write');
+ $supported = 1;
+ PVE::API2::Multipath->add_wwid({ wwid => 'w1', nodes => 'n1' });
+ is($writes, 1, 'supported restricted entry can be added');
+
+ for my $report (undef, '{"supported":0}') {
+ $kv = defined($report) ? { n2 => $report } : {};
+ for my $param ({ nodes => 'n1,n2' }, { delete => 'nodes' }) {
+ eval { PVE::API2::Multipath->update_wwid({ wwid => 'w1', %$param }) };
+ is(
+ $@,
+ defined($report)
+ ? "multipath-tools is not installed on node(s) n2\n"
+ : $support_error,
+ 'widening coverage is refused',
+ );
+ is($writes, 1, 'refused expansion does not write');
+ }
+ }
+ $kv = {};
+ PVE::API2::Multipath->update_wwid({
+ wwid => 'w1',
+ alias => 'lun',
+ 'no-path-retry' => 'fail',
+ });
+ is($writes, 2, 'maintenance works with silent peers');
+ $kv = { n2 => '{"supported":1}' };
+ PVE::API2::Multipath->update_wwid({ wwid => 'w1', nodes => 'n1,n2' });
+ is($writes, 3, 'supported expansion works');
+ $kv = {};
+ PVE::API2::Multipath->update_wwid({ wwid => 'w1', nodes => 'n1' });
+ is($writes, 4, 'narrowing works during upgrades');
+ PVE::API2::Multipath->remove_wwid({ wwid => 'w1' });
+ is($writes, 5, 'removal works during upgrades');
+};
+
subtest 'unreadable configuration keeps the known LUNs reported' => sub {
my $mp = Test::MockModule->new('PVE::Multipath');
$mp->mock(managed_wwid_record => sub { +{ '3600abc' => ['storage:san'] } });
diff --git a/src/test/run_multipath_tests.pl b/src/test/run_multipath_tests.pl
index 1c28ac86..94683a50 100755
--- a/src/test/run_multipath_tests.pl
+++ b/src/test/run_multipath_tests.pl
@@ -740,4 +740,42 @@ is_deeply(
'but how many LUNs are in which state',
);
+# --- consumers the configuration would leave without an allow-list entry ---
+my $scfg = {
+ ids => {
+ san => { type => 'iscsi', multipath => 1, nodes => { n1 => 1, n2 => 1 } },
+ plain => { type => 'iscsi' },
+ shared => { type => 'lvm', base => 'san:0.0.1.dm-uuid-mpath-w1' },
+ other =>
+ { type => 'lvm', base => 'plain:0.0.2.dm-uuid-mpath-w2', nodes => { n1 => 1 } },
+ local => { type => 'lvm', vgname => 'vg0' },
+ },
+};
+my $all = ['n1', 'n2', 'n3'];
+is_deeply(
+ PVE::Multipath::uncovered_consumer_nodes('w1', undef, $scfg, $all),
+ { shared => ['n3'] },
+ 'removing the entry uncovers the nodes the deriving storage does not reach',
+);
+is_deeply(
+ PVE::Multipath::uncovered_consumer_nodes('w1', {}, $scfg, $all),
+ {},
+ 'an unrestricted entry covers every node',
+);
+is_deeply(
+ PVE::Multipath::uncovered_consumer_nodes('w1', { n3 => 1 }, $scfg, $all),
+ {},
+ 'an entry for the remaining node covers it',
+);
+is_deeply(
+ PVE::Multipath::uncovered_consumer_nodes('w2', { n2 => 1 }, $scfg, $all),
+ { other => ['n1'] },
+ 'a base without the multipath option derives nothing',
+);
+is_deeply(
+ PVE::Multipath::uncovered_consumer_nodes('w3', undef, $scfg, $all),
+ {},
+ 'a LUN nothing uses has no consumers to uncover',
+);
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 35/76] api: multipath: add the cluster-wide health status
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (33 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 34/76] api: add endpoints for the cluster-wide multipath configuration Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 36/76] multipath: collect identities of non-iSCSI SCSI paths Thomas Lamprecht
` (40 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Combine node reports into per-LUN node and cluster states. Derive
expected nodes from explicit entries and storages that supply or consume
the LUN, using all nodes for unrestricted entries.
An expected node reporting no map counts as missing; one without a
current report counts as unknown. Missing reports do not worsen a known
cluster state, but an expected node reporting unavailable tools or an
unresponsive daemon makes the LUN degraded. Do not mark unexpected nodes
missing merely because SAN zoning excludes them.
Storage-derived LUNs stay allow-listed after withdrawal because a scan
cannot distinguish deliberate removal from failure. Mark a LUN as not
observed, with unknown cluster state, when every expected node reports
it absent and no node has its map. Explicit entries and LUNs with a
consuming storage remain missing instead.
A blocked pvestatd loop can leave an optimal report visible until it
expires after three minutes. Detect stalled loops from node RRD entries
older than a minute: show their optimal maps as unknown, but retain
reported degraded, missing, or failed states in the cluster result.
Keep the report-age limit too, since loop progress does not establish
observation freshness.
Include reported iSCSI initiator names and allow storage administrators
as well as auditors to read status, since target setup needs both the
names and LUN health.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Multipath.pm | 190 +++++++++++-
src/PVE/Multipath/Status.pm | 234 ++++++++++++++
src/test/run_lun_identity_tests.pl | 135 ++++++++
src/test/run_multipath_api_tests.pl | 77 +++++
src/test/run_multipath_tests.pl | 459 ++++++++++++++++++++++++++++
src/test/run_san_api_tests.pl | 137 +++++----
6 files changed, 1171 insertions(+), 61 deletions(-)
diff --git a/src/PVE/API2/Multipath.pm b/src/PVE/API2/Multipath.pm
index 595cb00b..bcb5502a 100644
--- a/src/PVE/API2/Multipath.pm
+++ b/src/PVE/API2/Multipath.pm
@@ -116,7 +116,195 @@ __PACKAGE__->register_method({
links => [{ rel => 'child', href => "{name}" }],
},
code => sub {
- return [map { +{ name => $_ } } qw(defaults overrides wwid)];
+ return [map { +{ name => $_ } } qw(defaults overrides status wwid)];
+ },
+});
+
+my $NODE_FIELDS = [
+ qw(apply-error report-error unmanaged unresolved-luns summary iscsi-initiator-name),
+ qw(reason stalled time),
+];
+
+my $reason_schema = {
+ type => 'string',
+ optional => 1,
+ description => "Known reasons: offline, stale, not-running, unsupported, query-error,"
+ . " summary-only, no-report, stalled, not-derived. Newer nodes may report other reasons.",
+};
+my $lun_node_schema = {
+ type => 'object',
+ additionalProperties => 0,
+ properties => {
+ state =>
+ { type => 'string', enum => ['optimal', 'degraded', 'missing', 'failed', 'unknown'] },
+ reason => $reason_schema,
+ transport => { type => 'string', optional => 1 },
+ kept =>
+ { type => 'string', optional => 1, description => 'What keeps the map in use.' },
+ map { $_ => { type => 'integer', minimum => 0, optional => 1 } }
+ qw(paths-active paths-total paths-expected),
+ },
+};
+my $node_schema = {
+ type => 'object',
+ additionalProperties => 0,
+ properties => {
+ supported => { type => 'boolean' },
+ running => { type => 'boolean' },
+ reason => $reason_schema,
+ stalled => { type => 'boolean', optional => 1 },
+ summary => {
+ type => 'object',
+ optional => 1,
+ description =>
+ 'LUN counts by state when the detailed report exceeds the size limit.',
+ additionalProperties => { type => 'integer', minimum => 0 },
+ },
+ (
+ map { $_ => { type => 'integer', minimum => 0, optional => 1 } }
+ qw(time unmanaged unresolved-luns)
+ ),
+ (
+ map { $_ => { type => 'string', optional => 1 } }
+ qw(apply-error report-error iscsi-initiator-name)
+ ),
+ },
+};
+
+__PACKAGE__->register_method({
+ name => 'status',
+ path => 'status',
+ method => 'GET',
+ description => "Report the multipath health of each managed LUN on each node, and the worst"
+ . " state among the nodes that should see the LUN.",
+ # Storage administrators need path health and initiator names to configure SAN target ACLs.
+ permissions => {
+ check => [
+ 'or', ['perm', '/', ['Sys.Audit']], ['perm', '/storage', ['Datastore.Allocate']],
+ ],
+ },
+ parameters => {
+ additionalProperties => 0,
+ properties => {},
+ },
+ returns => {
+ type => 'object',
+ properties => {
+ luns => {
+ type => 'array',
+ description =>
+ "One entry per LUN, identified by its WWID, with the state of its"
+ . " map on each node and the overall cluster state.",
+ items => {
+ type => 'object',
+ additionalProperties => 1,
+ properties => {
+ wwid => { type => 'string', description => 'The LUN WWID.' },
+ source => {
+ type => 'string',
+ description =>
+ "Why the LUN is allow-listed: 'manual' for an explicit entry,"
+ . " 'storage' for a LUN of a multipath-enabled storage, 'kept'"
+ . " for a LUN no longer configured that a node keeps while its"
+ . " map is in use there.",
+ enum => ['manual', 'storage', 'kept'],
+ },
+ 'multipath-storage' => {
+ type => 'string',
+ description => "The multipath-enabled storage the LUN belongs to,"
+ . " as reported by the nodes that see it.",
+ optional => 1,
+ },
+ alias => {
+ type => 'string',
+ description => 'The configured alias, if any.',
+ optional => 1,
+ },
+ 'used-by' => {
+ type => 'string',
+ description =>
+ 'The LVM storage using the map as its base volume, if any.',
+ optional => 1,
+ },
+ size => {
+ type => 'integer',
+ description => 'LUN size in bytes, as reported by a node.',
+ optional => 1,
+ },
+ 'cluster-state' => {
+ type => 'string',
+ description => "Worst map state across the nodes that should see"
+ . " the LUN: 'optimal', 'degraded' (some paths down or missing"
+ . " on a node, or a node that cannot manage multipath),"
+ . " 'missing' (a node has not assembled it), 'failed' (no active"
+ . " path), or 'unknown' (no current report from any of them, or"
+ . " the LUN is not observed, see 'not-observed').",
+ enum => ['optimal', 'degraded', 'missing', 'failed', 'unknown'],
+ },
+ 'not-observed' => {
+ type => 'boolean',
+ description =>
+ "Every expected node reports that the storage no longer"
+ . " finds the LUN and has no map of it. The storage keeps it allow-listed.",
+ optional => 1,
+ },
+ nodes => {
+ type => 'object',
+ description => "Map health keyed by node name.",
+ additionalProperties => $lun_node_schema,
+ },
+ },
+ },
+ },
+ nodes => {
+ type => 'object',
+ description => "Available node reports, keyed by node name.",
+ additionalProperties => $node_schema,
+ },
+ 'required-nodes' => {
+ type => 'array',
+ description => "Nodes configured for enabled multipath storages or explicit"
+ . " multipath entries, including nodes whose LUNs have not yet been identified.",
+ items => { type => 'string' },
+ },
+ },
+ },
+ code => sub {
+ my $cfg = PVE::Multipath::ClusterConfig::read_config();
+ my $reports = PVE::Multipath::Status::read_status_broadcasts();
+
+ my $expectations = PVE::Multipath::Status::cluster_storage_expectations();
+ my $explicit = PVE::Multipath::Config::wwid_list($cfg);
+
+ my $expected = PVE::Multipath::Status::expected_wwid_nodes(
+ $cfg,
+ $expectations,
+ $reports,
+ PVE::Cluster::get_nodelist(),
+ );
+
+ my $luns = PVE::Multipath::Status::aggregate_cluster_status(
+ $explicit,
+ PVE::Multipath::Config::aliases($cfg),
+ $expectations->{consumers},
+ $reports,
+ $expected,
+ );
+
+ my $nodes = {};
+ for my $node (keys %$reports) {
+ my $report = $reports->{$node};
+ $nodes->{$node} = {
+ supported => $report->{supported},
+ running => $report->{running},
+ map { defined($report->{$_}) ? ($_ => $report->{$_}) : () } $NODE_FIELDS->@*,
+ };
+ }
+
+ my $required = { $expectations->{nodes}->%* };
+ $required->{$_} = 1 for map { keys %$_ } values %$expected;
+
+ return { luns => $luns, nodes => $nodes, 'required-nodes' => [sort keys %$required] };
},
});
diff --git a/src/PVE/Multipath/Status.pm b/src/PVE/Multipath/Status.pm
index 67cc42b9..a7a2d4a0 100644
--- a/src/PVE/Multipath/Status.pm
+++ b/src/PVE/Multipath/Status.pm
@@ -6,6 +6,105 @@ use JSON;
use PVE::Multipath;
use PVE::Multipath::Config;
+# An old node RRD entry indicates a stalled status loop, not the age of this report. The loop
+# can block before or after broadcasting; the report's timestamp bounds observation age.
+our $STATUS_LOOP_STALE_SECONDS = 60;
+
+# Missing, invalid or future RRD timestamps do not prove that a status loop stalled.
+sub status_loop_ages($now, $dump) {
+ my $newest = {};
+ for my $key (keys %$dump) {
+ my ($node) = $key =~ m!^(?:pve-node-9\.0|pve2-node)/(.+)$! or next;
+ my $time = ref($dump->{$key}) eq 'ARRAY' ? $dump->{$key}->[2] : undef;
+ next if !defined($time) || $time !~ m/^\d+$/ || $time == 0;
+ $newest->{$node} = $time if !defined($newest->{$node}) || $time > $newest->{$node};
+ }
+ return { map { $_ => $now - $newest->{$_} } grep { $newest->{$_} <= $now } keys %$newest };
+}
+
+# An unknown observation does not worsen the known states; the node states show it instead.
+my $STATE_RANK = {
+ unknown => -1,
+ optimal => 0,
+ degraded => 1,
+ missing => 2,
+ failed => 3,
+};
+
+# Unknown states from newer or malformed peer reports must remain rankable.
+sub known_state($state) {
+ return defined($state) && exists($STATE_RANK->{$state}) ? $state : 'unknown';
+}
+
+sub worse_state($state, $other) {
+ return $STATE_RANK->{$other} > $STATE_RANK->{$state} ? $other : $state;
+}
+
+sub storage_expectations($storage_cfg, $all_nodes) {
+ my $res = { consumers => {}, 'consumer-bases' => {}, nodes => {}, 'storage-nodes' => {} };
+ my $ids = $storage_cfg->{ids} // {};
+
+ my $storage_nodes = sub($scfg) {
+ return { $scfg->{nodes}->%* } if $scfg->{nodes} && %{ $scfg->{nodes} };
+ return { map { $_ => 1 } $all_nodes->@* };
+ };
+
+ for my $storeid (sort keys %$ids) {
+ my $scfg = $ids->{$storeid};
+
+ if ($scfg->{multipath} && !$scfg->{disable}) {
+ my $nodes = $storage_nodes->($scfg);
+ $res->{'storage-nodes'}->{$storeid} = $nodes;
+ $res->{nodes}->{$_} = 1 for keys %$nodes;
+ }
+
+ next if ($scfg->{type} // '') ne 'lvm' || !defined($scfg->{base});
+
+ # Read raw configuration conservatively, including bare map names. The add hook requires
+ # '.dm-uuid-mpath-' because valid iSCSI volume names include a numeric address prefix.
+ my ($baseid, $volname) = $scfg->{base} =~ m/^([a-zA-Z][a-zA-Z0-9\-_.]*):(\S+)$/;
+ next if !defined($volname);
+ my ($wwid) = $volname =~ m/(?:^|\.)dm-uuid-mpath-(\S+)$/;
+ next if !defined($wwid);
+ next if !$ids->{$baseid};
+
+ # Disabling multipath does not make an existing LVM consumer safe to drop.
+ $res->{consumers}->{$wwid} = $storeid;
+ push $res->{'consumer-bases'}->{$wwid}->@*, $baseid;
+ }
+
+ return $res;
+}
+
+sub cluster_storage_expectations() {
+ require PVE::Storage;
+ require PVE::Cluster;
+
+ return storage_expectations(PVE::Storage::config(), PVE::Cluster::get_nodelist());
+}
+
+sub expected_wwid_nodes($cfg, $expectations, $reports, $all_nodes) {
+ my $expected = {};
+ for my $wwid (PVE::Multipath::Config::wwid_list($cfg)->@*) {
+ my $nodes = $cfg->{ids}->{$wwid}->{nodes};
+ $expected->{$wwid} = $nodes && %$nodes ? {%$nodes} : { map { $_ => 1 } $all_nodes->@* };
+ }
+ my $add_storage = sub($wwid, $storeid) {
+ my $nodes = $expectations->{'storage-nodes'}->{$storeid} // {};
+ $expected->{$wwid}->{$_} = 1 for keys %$nodes;
+ };
+ for my $report (values %$reports) {
+ my $luns = $report->{luns} // {};
+ for my $wwid (keys %$luns) {
+ $add_storage->($wwid, $_) for ($luns->{$wwid}->{storages} // [])->@*;
+ }
+ }
+ for my $wwid (keys $expectations->{'consumer-bases'}->%*) {
+ $add_storage->($wwid, $_) for $expectations->{'consumer-bases'}->{$wwid}->@*;
+ }
+ return $expected;
+}
+
# Include wanted WWIDs without assembled maps so failed assembly remains visible. Expected path
# counts include paths the daemon no longer lists.
sub status_luns($wanted, $maps) {
@@ -154,6 +253,7 @@ sub read_status_broadcasts() {
my $now = time();
my $members = PVE::Cluster::get_members() // {};
my $raw = PVE::Cluster::get_node_kv($KV_KEY);
+ my $loop_ages = status_loop_ages($now, PVE::Cluster::rrd_dump() // {});
my $res = {};
for my $node (keys %$raw) {
my $decoded = eval { decode_json($raw->{$node}) };
@@ -186,6 +286,10 @@ sub read_status_broadcasts() {
}
$entry->{fresh} = defined($reason) ? 0 : 1;
$entry->{reason} = $reason if defined($reason);
+ $entry->{stalled} = 1
+ if $entry->{fresh}
+ && defined($loop_ages->{$node})
+ && $loop_ages->{$node} > $STATUS_LOOP_STALE_SECONDS;
$res->{$node} = $entry;
}
@@ -249,4 +353,134 @@ sub assert_cluster_supports_multipath($nodes) {
return;
}
+# The reasons for an unknown state that mean a node cannot manage multipath at all, as opposed to
+# not being heard from.
+my $CANNOT_MANAGE = { unsupported => 1, 'not-running' => 1, 'query-error' => 1 };
+
+# Absence requires fresh, complete, unstalled reports without apply errors and with explicit
+# non-derivation. Missing evidence cannot distinguish withdrawal from a failed scan.
+my sub not_observed($wwid, $reports, $exp) {
+ return 0 if grep { !$reports->{$_} } keys %$exp;
+ my $evidence = 0;
+ for my $node (sort keys %$reports) {
+ my $report = $reports->{$node};
+ my $lun = ($report->{luns} // {})->{$wwid};
+ $lun = undef if ref($lun) ne 'HASH';
+ return 0 if $lun && defined($lun->{state});
+ next if !$lun && !$exp->{$node};
+ return 0 if !$report->{fresh} || $report->{stalled} || !$report->{luns};
+ return 0 if defined($report->{'apply-error'});
+ return 0 if !$lun || defined($lun->{kept}) || ($lun->{derived} // 1);
+ $evidence = 1;
+ }
+ return $evidence;
+}
+
+# Unexpected nodes must not become missing, but their known states still count. Preserve problems
+# from stalled loops, not optimal claims that may predate blocked I/O. An unconsumed, non-explicit
+# LUN absent from every reliable scan is unknown: deliberate withdrawal and accidental loss look
+# alike.
+sub aggregate_cluster_status($explicit, $aliases, $used_by, $reports, $expected = undef) {
+ $explicit //= [];
+ $aliases //= {};
+ $used_by //= {};
+ $reports //= {};
+ $expected //= {};
+
+ my %manual = map { $_ => 1 } $explicit->@*;
+
+ my %wwids = %manual;
+ $wwids{$_} = 1 for grep { %{ $expected->{$_} // {} } } keys %$used_by;
+ for my $report (values %$reports) {
+ $wwids{$_} = 1 for keys(($report->{luns} // {})->%*);
+ }
+
+ my $res = [];
+ for my $wwid (sort keys %wwids) {
+ my $nodes = {};
+ my $worst = 'optimal';
+ my $have_state = 0;
+ my ($size, $storage, $kept);
+
+ my $exp = $expected->{$wwid} // {};
+ my $unobserved =
+ !$manual{$wwid} && !defined($used_by->{$wwid}) && not_observed($wwid, $reports, $exp);
+
+ for my $node (sort keys %$reports) {
+ my $report = $reports->{$node};
+ my $lun = ($report->{luns} // {})->{$wwid};
+ $lun = undef if ref($lun) ne 'HASH';
+
+ if ($lun) {
+ $storage //= ($lun->{storages} // [])->[0];
+ $kept = 1 if defined($lun->{kept});
+ }
+
+ if (!$report->{fresh}) {
+ my $reason = $report->{reason} // 'stale';
+ $nodes->{$node} = { state => 'unknown', reason => $reason }
+ if $lun || $exp->{$node};
+ if ($exp->{$node} && $CANNOT_MANAGE->{$reason}) {
+ $have_state = 1;
+ $worst = 'degraded' if $STATE_RANK->{degraded} > $STATE_RANK->{$worst};
+ }
+ next;
+ }
+ if (!$report->{luns}) {
+ $nodes->{$node} = { state => 'unknown', reason => 'summary-only' }
+ if $exp->{$node};
+ next;
+ }
+
+ my @stalled = $report->{stalled} ? (reason => 'stalled') : ();
+ if ($lun && defined($lun->{state})) {
+ my $state = known_state($lun->{state});
+ $size //= $lun->{size};
+ if (@stalled && $state eq 'optimal') {
+ $nodes->{$node} = { state => 'unknown', @stalled };
+ next;
+ }
+ $have_state = 1 if $state ne 'unknown';
+ $nodes->{$node} = {
+ state => $state,
+ map({ defined($lun->{$_}) ? ($_ => $lun->{$_}) : () }
+ qw(paths-active paths-total paths-expected transport kept)),
+ @stalled,
+ };
+ $worst = $state if $STATE_RANK->{$state} > $STATE_RANK->{$worst};
+ } elsif ($lun || $exp->{$node}) {
+ if ($unobserved) {
+ $nodes->{$node} = { state => 'missing', reason => 'not-derived' };
+ next;
+ }
+ $have_state = 1;
+ $nodes->{$node} = { state => 'missing', @stalled };
+ $worst = 'missing' if $STATE_RANK->{missing} > $STATE_RANK->{$worst};
+ }
+ }
+
+ # Missing reports must not hide expected nodes or drive the health roll-up.
+ for my $node (sort keys %$exp) {
+ next if exists $nodes->{$node};
+ $nodes->{$node} = { state => 'unknown', reason => 'no-report' };
+ }
+
+ my $source = $manual{$wwid} ? 'manual' : !defined($storage) && $kept ? 'kept' : 'storage';
+ push $res->@*,
+ {
+ wwid => $wwid,
+ source => $source,
+ defined($storage) ? ('multipath-storage' => $storage) : (),
+ defined($aliases->{$wwid}) ? (alias => $aliases->{$wwid}) : (),
+ defined($used_by->{$wwid}) ? ('used-by' => $used_by->{$wwid}) : (),
+ defined($size) ? (size => $size) : (),
+ $unobserved ? ('not-observed' => 1) : (),
+ 'cluster-state' => $have_state ? $worst : 'unknown',
+ nodes => $nodes,
+ };
+ }
+
+ return $res;
+}
+
1;
diff --git a/src/test/run_lun_identity_tests.pl b/src/test/run_lun_identity_tests.pl
index 783fac18..4e9507a0 100755
--- a/src/test/run_lun_identity_tests.pl
+++ b/src/test/run_lun_identity_tests.pl
@@ -14,6 +14,7 @@ use File::Temp qw(tempdir);
use Test::MockModule;
use Test::More;
+use PVE::API2::Multipath;
use PVE::Multipath;
use PVE::Multipath::Identity;
use PVE::Multipath::Generator;
@@ -746,4 +747,138 @@ is(
$multipath->unmock('is_supported');
}
+# --- from the sysfs of two nodes to the cluster view: an owned LUN gone from the target is
+# not observed only once both nodes could look at all LUNs ---
+
+$identity->unmock_all();
+$iscsi->unmock_all();
+$iscsi->mock(iscsi_portals => sub { ['192.0.2.10:3260'] });
+my sub node_report {
+ my ($scan) = @_;
+ put('class/iscsi_session/session1/state', "LOGGED_IN\n");
+ put('class/iscsi_session/session1/target_state', "$scan\n");
+ put(
+ 'class/iscsi_session/session1/device/connection1:0/iscsi_connection/connection1:0/persistent_address',
+ "192.0.2.10\n",
+ );
+ put(
+ 'class/iscsi_session/session1/device/connection1:0/iscsi_connection/connection1:0/persistent_port',
+ "3260\n",
+ );
+ my $cfg =
+ { ids =>
+ { san => { type => 'iscsi', target => $T, portal => '192.0.2.10', multipath => 1 } } };
+ my $derived = PVE::Storage::derived_multipath_wwids($cfg);
+ # the node owns a LUN for the storage that the target no longer has
+ my $desired = {
+ %{ { map { $_ => ['storage:san'] } keys $derived->{san}->{wwids}->%* } },
+ '3GONE' => ['storage:san'],
+ };
+ my ($luns) = PVE::Multipath::Status::status_luns(
+ PVE::Multipath::Generator::desired_wwid_info($desired, $derived), [],
+ );
+ return { fresh => 1, luns => $luns };
+}
+$reply = {
+ sdc => {
+ devt => '8:32',
+ hctl => '7:0:0:1',
+ init => 'ok',
+ target => $T,
+ wwid => '3600a09803832355367242f747a716b7a',
+ },
+};
+my $exp = { '3GONE' => { lab1 => 1, lab2 => 1 } };
+my $rows = PVE::Multipath::Status::aggregate_cluster_status(
+ [], {}, {}, { lab1 => node_report('SCANNED'), lab2 => node_report('ALLOCATED') }, $exp,
+);
+my ($gone) = grep { $_->{wwid} eq '3GONE' } @$rows;
+is($gone->{'cluster-state'}, 'missing', 'end to end: missing while one node scans the target');
+$rows = PVE::Multipath::Status::aggregate_cluster_status(
+ [], {}, {}, { lab1 => node_report('SCANNED'), lab2 => node_report('SCANNED') }, $exp,
+);
+($gone) = grep { $_->{wwid} eq '3GONE' } @$rows;
+ok($gone->{'not-observed'}, 'end to end: not observed once both could look at all LUNs');
+$reply->{sdc}->{init} = 'udev-missing';
+$rows = PVE::Multipath::Status::aggregate_cluster_status(
+ [], {}, {}, { lab1 => node_report('SCANNED'), lab2 => node_report('SCANNED') }, $exp,
+);
+($gone) = grep { $_->{wwid} eq '3GONE' } @$rows;
+is($gone->{'cluster-state'}, 'missing', 'end to end: missing while a LUN cannot be identified');
+$reply->{sdc}->{init} = 'ok';
+{
+ # An invalid daemon response cannot establish that a LUN was withdrawn.
+ $multipath->mock(
+ query_path_identities => sub { die "multipathd returned an invalid path list\n" });
+ $rows = PVE::Multipath::Status::aggregate_cluster_status(
+ [], {}, {}, { lab1 => node_report('SCANNED'), lab2 => node_report('SCANNED') }, $exp,
+ );
+ ($gone) = grep { $_->{wwid} eq '3GONE' } @$rows;
+ is(
+ $gone->{'cluster-state'},
+ 'missing',
+ 'end to end: missing while the daemon returns an invalid path list',
+ );
+ $multipath->mock(
+ query_path_identities => sub { $queries++; die $reply if !ref($reply); return $reply },
+ );
+}
+
+{
+ # the count of LUNs a node cannot identify, through the status API
+ my $cluster = Test::MockModule->new('PVE::Cluster');
+ $cluster->mock(
+ get_node_kv => sub {
+ return { lab1 => '{"supported":1,"running":1,"time":'
+ . time()
+ . ',"unresolved-luns":3}' };
+ },
+ get_members => sub { { lab1 => { online => 1 } } },
+ rrd_dump => sub { {} },
+ get_nodelist => sub { ['lab1'] },
+ );
+ my $config = Test::MockModule->new('PVE::Multipath::ClusterConfig');
+ $config->mock(read_config => sub { { ids => {} } });
+ my $status = Test::MockModule->new('PVE::Multipath::Status');
+ $status->mock(
+ cluster_storage_expectations => sub {
+ { consumers => {}, 'consumer-bases' => {}, nodes => {}, 'storage-nodes' => {} }
+ },
+ );
+ is(
+ PVE::API2::Multipath->status({})->{nodes}->{lab1}->{'unresolved-luns'},
+ 3,
+ 'API: the count of LUNs a node cannot identify',
+ );
+}
+
+# the last LUN of a target retired as documented: the session stays, its target is gone
+my sub empty_report {
+ my $cfg = {
+ ids => {
+ e => {
+ type => 'iscsi',
+ target => 'iqn.2003-01.org.linux-iscsi.empty:sn.4',
+ portal => '192.0.2.40',
+ multipath => 1,
+ },
+ },
+ };
+ $iscsi->mock(iscsi_portals => sub { ['192.0.2.40:3260'] });
+ my $derived = PVE::Storage::derived_multipath_wwids($cfg);
+ my ($luns) = PVE::Multipath::Status::status_luns(
+ PVE::Multipath::Generator::desired_wwid_info({ '3LAST' => ['storage:e'] }, $derived),
+ [],
+ );
+ return { fresh => 1, luns => $luns };
+}
+$rows = PVE::Multipath::Status::aggregate_cluster_status(
+ [],
+ {},
+ {},
+ { lab1 => empty_report(), lab2 => empty_report() },
+ { '3LAST' => { lab1 => 1, lab2 => 1 } },
+);
+ok($rows->[0]->{'not-observed'}, 'end to end: the last LUN of a target retired, not observed');
+
done_testing();
diff --git a/src/test/run_multipath_api_tests.pl b/src/test/run_multipath_api_tests.pl
index 055ade62..370b6684 100755
--- a/src/test/run_multipath_api_tests.pl
+++ b/src/test/run_multipath_api_tests.pl
@@ -12,6 +12,7 @@ BEGIN {
}
use lib ('.', '..');
+use JSON qw(encode_json);
use Storable qw(dclone);
use Test::More;
use Test::MockModule;
@@ -33,6 +34,7 @@ subtest 'every node sends a bounded status report' => sub {
},
);
$cluster->mock(get_members => sub { {} });
+ $cluster->mock(rrd_dump => sub { {} });
my $node = 'n1';
$cluster->mock(
broadcast_node_kv => sub {
@@ -98,6 +100,49 @@ subtest 'every node sends a bounded status report' => sub {
ok(length($reports->{n4}->{'apply-error'}) < 2100, 'and a bounded apply error');
};
+subtest 'reports of a node whose status loop stalled' => sub {
+ local $NOW = 100_000;
+ my $report =
+ sub { encode_json({ supported => 1, running => 1, luns => {}, time => $_[0] }) };
+ my $cluster = Test::MockModule->new('PVE::Cluster');
+ $cluster->mock(
+ get_members => sub {
+ +{ map { ($_ => { online => 1 }) } qw(n1 n2 n3 n4 n5) };
+ },
+ );
+ $cluster->mock(
+ get_node_kv => sub {
+ return {
+ n1 => $report->($NOW),
+ n2 => $report->($NOW),
+ n3 => $report->($NOW - 181),
+ n4 => $report->($NOW - 180),
+ n5 => $report->($NOW),
+ };
+ },
+ );
+ $cluster->mock(
+ rrd_dump => sub {
+ return {
+ 'pve-node-9.0/n1' => [1, undef, $NOW - 60],
+ 'pve-node-9.0/n2' => [1, undef, $NOW - 61],
+ 'pve-node-9.0/n3' => [1, undef, $NOW - 61],
+ 'pve-node-9.0/n4' => [1, undef, $NOW],
+ };
+ },
+ );
+ my $reports = PVE::Multipath::Status::read_status_broadcasts();
+ ok(
+ $reports->{n1}->{fresh} && !$reports->{n1}->{stalled},
+ 'a loop 60 s behind is not stalled',
+ );
+ ok($reports->{n2}->{fresh} && $reports->{n2}->{stalled}, 'one 61 s behind is');
+ is($reports->{n3}->{reason}, 'stale', 'a report older than 180 s is stale');
+ ok(!$reports->{n3}->{stalled}, 'and not marked stalled on top');
+ ok($reports->{n4}->{fresh}, 'one of 180 s still counts');
+ ok($reports->{n5}->{fresh} && !$reports->{n5}->{stalled}, 'no RRD entry gives no opinion');
+};
+
subtest 'enabling the multipath option needs every node to know it' => sub {
my $cluster = Test::MockModule->new('PVE::Cluster');
my $inotify = Test::MockModule->new('PVE::INotify');
@@ -291,6 +336,38 @@ subtest 'the report carries the iSCSI initiator name of the node' => sub {
),
'and a garbled one from a peer is dropped',
);
+ $kv->{n1} =
+ JSON::encode_json({ luns => {}, time => time(), 'iscsi-initiator-name' => $iqn });
+
+ my $cc = Test::MockModule->new('PVE::Multipath::ClusterConfig');
+ $cc->mock(read_config => sub { PVE::Multipath::Config->parse_config('multipath.cfg', '') });
+ my $storage = Test::MockModule->new('PVE::Storage');
+ $storage->mock(config => sub { { ids => {} } });
+ $cluster->mock(get_nodelist => sub { ['n1'] });
+ is(
+ PVE::API2::Multipath->status({})->{nodes}->{n1}->{'iscsi-initiator-name'},
+ $iqn,
+ 'the status lists it per node',
+ );
+ is_deeply(
+ PVE::API2::Multipath->status({})->{'required-nodes'},
+ [],
+ 'no node has to carry LUNs',
+ );
+ $storage->mock(
+ config =>
+ sub { { ids => { san => { type => 'iscsi', multipath => 1, nodes => { n1 => 1 } } } } },
+ );
+ is_deeply(
+ PVE::API2::Multipath->status({})->{'required-nodes'},
+ ['n1'],
+ 'the nodes of a multipath storage do, before any of its LUNs is identified',
+ );
+ is_deeply(
+ PVE::API2::Multipath->map_method_by_name('status')->{permissions}->{check},
+ ['or', ['perm', '/', ['Sys.Audit']], ['perm', '/storage', ['Datastore.Allocate']]],
+ 'for auditors and storage admins alike',
+ );
};
subtest 'an entry stays allow-listed while its map is in use' => sub {
diff --git a/src/test/run_multipath_tests.pl b/src/test/run_multipath_tests.pl
index 94683a50..c2dcd51a 100755
--- a/src/test/run_multipath_tests.pl
+++ b/src/test/run_multipath_tests.pl
@@ -610,6 +610,50 @@ is_deeply(
);
}
+# --- storage-derived expectations (consumers and per-storage node sets) ---
+{
+ my $storage_cfg = {
+ ids => {
+ san => { type => 'iscsi', multipath => 1 },
+ sanb => { type => 'iscsi', multipath => 1, nodes => { n1 => 1, n2 => 1 } },
+ sanoff => { type => 'iscsi', multipath => 1, disable => 1, nodes => { n9 => 1 } },
+ sanplain => { type => 'iscsi' },
+ lvm1 => {
+ type => 'lvm',
+ base => 'san:0.0.0.dm-uuid-mpath-3600abc',
+ nodes => { n1 => 1 },
+ },
+ lvm2 => { type => 'lvm', base => 'sanb:0.0.1.dm-uuid-mpath-3600def' },
+ lvmsingle => { type => 'lvm', base => 'sanplain:0.0.2.scsi-3600fff' },
+ lvmoff => { type => 'lvm', base => 'sanplain:0.0.3.dm-uuid-mpath-3600aaa' },
+ lvmplain => { type => 'lvm' },
+ other => { type => 'nfs' },
+ },
+ };
+ my $exp = PVE::Multipath::Status::storage_expectations($storage_cfg, ['n1', 'n2', 'n3']);
+ is_deeply(
+ $exp->{consumers},
+ { '3600abc' => 'lvm1', '3600def' => 'lvm2', '3600aaa' => 'lvmoff' },
+ 'LVM storages over a map-named base volume are found as consumers, scsi- names are not',
+ );
+ is_deeply(
+ $exp->{nodes},
+ { n1 => 1, n2 => 1, n3 => 1 },
+ 'expected union covers all nodes of unrestricted multipath-enabled storages, skips'
+ . ' disabled ones and storages without the option',
+ );
+ is_deeply(
+ $exp->{'storage-nodes'},
+ { san => { n1 => 1, n2 => 1, n3 => 1 }, sanb => { n1 => 1, n2 => 1 } },
+ 'per-storage node sets follow the restriction, disabled storages are absent',
+ );
+ is_deeply(
+ $exp->{'consumer-bases'},
+ { '3600abc' => ['san'], '3600def' => ['sanb'], '3600aaa' => ['sanplain'] },
+ 'the transport storage of every consumed LUN is known',
+ );
+}
+
# --- override guard ---
eval { PVE::Multipath::Config::check_overrides("devices {\n\tdevice {\n\t\tvendor X\n\t}\n}\n") };
is($@, '', 'a well-formed devices{} block passes the guard');
@@ -740,6 +784,421 @@ is_deeply(
'but how many LUNs are in which state',
);
+# --- cluster status aggregation ---
+my $mk = sub {
+ my ($state, $active, $total, %extra) = @_;
+ return {
+ state => $state,
+ 'paths-active' => $active,
+ 'paths-total' => $total,
+ %extra,
+ };
+};
+my $reports = {
+ nodeA => {
+ fresh => 1,
+ luns => {
+ wA => $mk->('optimal', 2, 2, transport => 'iscsi'),
+ wB => $mk->('optimal', 2, 2, transport => 'iscsi', size => 42),
+ },
+ },
+ nodeB => {
+ fresh => 1,
+ luns => { wA => $mk->('degraded', 1, 2, transport => 'iscsi') },
+ # nodeB is active but does not see wB
+ },
+ nodeC => {
+ # last report from a node that is now offline
+ fresh => 0,
+ reason => 'offline',
+ luns => { wA => $mk->('optimal', 2, 2) },
+ },
+};
+my $everyone = { nodeA => 1, nodeB => 1 };
+my $agg = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wA', 'wB', 'wZ'], # explicit incl. an unseen WWID
+ { wA => 'lun-a' }, # alias
+ { wB => 'mptank' }, # used-by
+ $reports,
+ { wA => $everyone, wB => $everyone, wZ => $everyone },
+);
+my %by_wwid = map { $_->{wwid} => $_ } $agg->@*;
+
+is_deeply([sort keys %by_wwid], ['wA', 'wB', 'wZ'], 'matrix covers explicit and seen WWIDs');
+
+is($by_wwid{wA}->{alias}, 'lun-a', 'alias surfaced on the WWID row');
+is($by_wwid{wA}->{source}, 'manual', 'an explicitly allow-listed LUN is marked manual');
+is($by_wwid{wA}->{'cluster-state'}, 'degraded', 'degraded on one active node rolls up to degraded');
+is($by_wwid{wA}->{nodes}->{nodeA}->{state}, 'optimal', 'per-node optimal cell kept');
+is($by_wwid{wA}->{nodes}->{nodeB}->{state}, 'degraded', 'per-node degraded cell kept');
+is($by_wwid{wA}->{nodes}->{nodeC}->{state}, 'unknown', 'offline node with old data shows unknown');
+is($by_wwid{wA}->{nodes}->{nodeC}->{reason}, 'offline', 'with the reason why');
+
+is($by_wwid{wB}->{'used-by'}, 'mptank', 'consuming storage surfaced as used-by');
+is($by_wwid{wB}->{size}, 42, 'LUN size surfaced from a reporting node');
+is(
+ $by_wwid{wB}->{'cluster-state'},
+ 'missing',
+ 'active node not assembling the LUN rolls up to missing',
+);
+is($by_wwid{wB}->{nodes}->{nodeB}->{state}, 'missing', 'missing marked on the node lacking it');
+
+is($by_wwid{wZ}->{'cluster-state'}, 'missing', 'an explicit WWID nobody assembled is missing');
+is($by_wwid{wZ}->{nodes}->{nodeA}->{state}, 'missing', 'each expected node misses it');
+ok(!exists $by_wwid{wZ}->{nodes}->{nodeC}, 'a non-expected offline node contributes no cell');
+
+# a WWID only an offline node ever reported, with no fresh node, is unknown
+my $agg_off = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ { dead => { fresh => 0, reason => 'offline', luns => { wA => $mk->('optimal', 2, 2) } } },
+ { wA => {} },
+);
+is($agg_off->[0]->{'cluster-state'}, 'unknown', 'no fresh node leaves the cluster-state unknown');
+is($agg_off->[0]->{nodes}->{dead}->{state}, 'unknown', 'the offline node is shown as unknown');
+
+# failure outranks degraded in the roll-up
+my $agg2 = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ {
+ n1 => { fresh => 1, luns => { wA => $mk->('degraded', 1, 2) } },
+ n2 => { fresh => 1, luns => { wA => $mk->('failed', 0, 2) } },
+ },
+ { wA => { n1 => 1, n2 => 1 } },
+);
+is($agg2->[0]->{'cluster-state'}, 'failed', 'failed outranks degraded in the cluster roll-up');
+
+# --- expected-node set: a node that lost all paths still reports (with the LUN listed but not
+# assembled, or not at all) and must show as missing; one that does not report at all (an older
+# release) is unknown
+my $exp_reports = {
+ nodeA => { fresh => 1, luns => { wA => $mk->('optimal', 2, 2) } },
+ nodeS => { fresh => 1, luns => {} },
+ nodeStale => { fresh => 0, reason => 'stale', luns => { wA => $mk->('optimal', 2, 2) } },
+ nodeDown => { fresh => 0, reason => 'not-running', luns => {} },
+};
+my $eagg = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ $exp_reports,
+ { wA => { nodeA => 1, nodeS => 1, nodeStale => 1, nodeDown => 1, nodeOld => 1 } },
+);
+my $row = $eagg->[0];
+is($row->{nodes}->{nodeA}->{state}, 'optimal', 'reporting node keeps its real state');
+is($row->{nodes}->{nodeS}->{state}, 'missing', 'a participant without the map is missing');
+is($row->{nodes}->{nodeStale}->{state}, 'unknown', 'a stale reporter is unknown');
+is($row->{nodes}->{nodeStale}->{reason}, 'stale', 'and says so');
+is($row->{nodes}->{nodeDown}->{reason}, 'not-running', 'a node whose daemon is down says so');
+is($row->{nodes}->{nodeOld}->{state}, 'unknown', 'an expected node without any report is unknown');
+is($row->{nodes}->{nodeOld}->{reason}, 'no-report', 'with no-report as the reason');
+is($row->{'cluster-state'}, 'missing', 'the participant lacking the map drags the roll-up');
+
+# an expected node that cannot manage multipath at all degrades the LUN, one not heard from does not
+my $down = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ {
+ n1 => { fresh => 1, luns => { wA => $mk->('optimal', 2, 2) } },
+ n2 => { fresh => 0, reason => 'not-running', luns => {} },
+ n3 => { fresh => 0, reason => 'offline', luns => {} },
+ },
+ { wA => { n1 => 1, n2 => 1, n3 => 1 } },
+);
+is($down->[0]->{'cluster-state'}, 'degraded', 'a stopped daemon on an expected node degrades');
+my $tools = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ {
+ n1 => { fresh => 1, luns => { wA => $mk->('optimal', 2, 2) } },
+ n2 => { fresh => 0, reason => 'unsupported', luns => {} },
+ n3 => { fresh => 0, reason => 'unsupported', luns => {} },
+ },
+ { wA => { n1 => 1, n2 => 1 } },
+);
+is($tools->[0]->{'cluster-state'}, 'degraded', 'so do missing tools on an expected node');
+ok(!exists $tools->[0]->{nodes}->{n3}, 'a node without tools that is not expected is left out');
+
+# a LUN that a node merely happens to report, with no expectation info at all, must not
+# synthesize missing cells on other nodes
+my $offlist = PVE::Multipath::Status::aggregate_cluster_status(
+ [],
+ {},
+ {},
+ {
+ nodeA => { fresh => 1, luns => { wX => $mk->('optimal', 1, 1) } },
+ nodeS => { fresh => 1, luns => {} },
+ },
+ {},
+);
+ok(
+ !exists $offlist->[0]->{nodes}->{nodeS},
+ 'a WWID without expectation info does not synthesize missing cells for other nodes',
+);
+
+# a derived LUN (no explicit section) carries its storage from the reports and gets missing
+# detection through its expected set, and through a node that wants the LUN but has no map for it
+my $derived_agg = PVE::Multipath::Status::aggregate_cluster_status(
+ [],
+ {},
+ { wD => 'lvmshared' },
+ {
+ n1 => {
+ fresh => 1,
+ luns => {
+ wD => $mk->('optimal', 2, 2, storages => ['san1'], 'paths-expected' => 2),
+ },
+ },
+ n2 => { fresh => 1, luns => { wD => { storages => ['san1'], 'paths-expected' => 2 } } },
+ n3 => { fresh => 1, luns => {} },
+ },
+ { wD => { n1 => 1, n2 => 1 } },
+);
+is($derived_agg->[0]->{source}, 'storage',
+ 'a LUN without an explicit section comes from a storage');
+is($derived_agg->[0]->{'multipath-storage'}, 'san1', 'the deriving storage is named');
+is(
+ $derived_agg->[0]->{nodes}->{n1}->{'paths-expected'},
+ 2,
+ 'the expected path count is in the cell',
+);
+is(
+ $derived_agg->[0]->{nodes}->{n2}->{state},
+ 'missing',
+ 'a node wanting the LUN without a map is missing',
+);
+ok(!exists $derived_agg->[0]->{nodes}->{n3}, 'an unrelated node gets no cell');
+is($derived_agg->[0]->{'cluster-state'}, 'missing', 'the derived LUN roll-up includes the gap');
+
+# a LUN no node has assembled still has a row: from the wanted set of a node, or from an LVM
+# storage consuming it where some node is supposed to carry it
+my $unassembled = PVE::Multipath::Status::aggregate_cluster_status(
+ [],
+ {},
+ { wC => 'lvmc', wU => 'lvmu' },
+ { n1 => { fresh => 1, luns => { wW => { storages => ['san2'] } } } },
+ { wW => { n1 => 1 }, wC => { n1 => 1 } },
+);
+my %rows = map { $_->{wwid} => $_ } $unassembled->@*;
+is(
+ $rows{wW}->{'multipath-storage'},
+ 'san2',
+ 'a wanted but unassembled LUN has a row naming its storage',
+);
+is($rows{wW}->{nodes}->{n1}->{state}, 'missing', 'and is missing on the node wanting it');
+is($rows{wC}->{'used-by'}, 'lvmc', 'a consumed LUN nobody reports has a row');
+is($rows{wC}->{nodes}->{n1}->{state}, 'missing', 'and is missing where it is expected');
+ok(!$rows{wU}, 'a consumed LUN no node is supposed to carry has no row');
+
+# a node that sent only a summary cannot say the LUN is missing
+my $cut = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ {
+ n1 => { fresh => 1, summary => { luns => 300, optimal => 300 } },
+ n2 => { fresh => 1, luns => { wA => $mk->('optimal', 2, 2) } },
+ },
+ { wA => { n1 => 1, n2 => 1 } },
+);
+is($cut->[0]->{nodes}->{n1}->{state}, 'unknown', 'a LUN of a summary-only node is unknown');
+is($cut->[0]->{nodes}->{n1}->{reason}, 'summary-only', 'with the summary as the reason');
+is($cut->[0]->{'cluster-state'}, 'optimal', 'and does not read as missing');
+
+# a LUN dropped from the configuration that a node keeps while its map is in use
+my $kept = PVE::Multipath::Status::aggregate_cluster_status(
+ [],
+ {},
+ {},
+ { n1 => { fresh => 1, luns => { wK => $mk->('optimal', 2, 2, kept => 'LVM') } } },
+ {},
+);
+is($kept->[0]->{source}, 'kept', 'a LUN only kept for its user is marked as such');
+is($kept->[0]->{nodes}->{n1}->{kept}, 'LVM', 'with what keeps it on the node');
+
+# a reporting node outside a LUN's expected set is never marked missing (a SAN zoned to only
+# some nodes)
+my $zoned = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ {
+ n1 => { fresh => 1, luns => { wA => $mk->('optimal', 2, 2) } },
+ n2 => { fresh => 1, luns => { wOther => $mk->('degraded', 1, 2) } },
+ },
+ { wA => { n1 => 1 } },
+);
+my %zoned_rows = map { $_->{wwid} => $_ } $zoned->@*;
+ok(!exists $zoned_rows{wA}->{nodes}->{n2}, 'a node outside the expected set is not marked missing');
+is($zoned_rows{wA}->{'cluster-state'}, 'optimal', 'the zoned LUN stays green on its own nodes');
+ok(!exists $zoned_rows{wOther}->{nodes}->{n1}, 'a LUN of another node does not drag n1 in');
+is($zoned_rows{wOther}->{'cluster-state'}, 'degraded', 'and still surfaces its real state');
+
+# an unrecognized state from a (newer or buggy) peer clamps to unknown and does not fake a report
+my $clamped = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wA'],
+ {},
+ {},
+ { n1 => { fresh => 1, luns => { wA => $mk->('frobnicated', 1, 2) } } },
+ { wA => { n1 => 1 } },
+);
+is($clamped->[0]->{nodes}->{n1}->{state}, 'unknown', 'unrecognized peer state clamps to unknown');
+is($clamped->[0]->{'cluster-state'}, 'unknown', 'a clamped state does not count as a report');
+
+# a LUN of a storage that no node observes any more: neutral only with affirmative evidence from
+# every node that lists it or should carry it
+{
+ my $gone = { storages => ['san'], derived => 0 };
+ my %luns =
+ map { $_ => $gone }
+ qw(gone manual used failed noreport offline errored summary kept stalled);
+ my $reports = {
+ n1 => { fresh => 1, luns => { %luns, old => $gone } },
+ n2 => {
+ fresh => 1,
+ luns => {
+ %luns,
+ old => { storages => ['san'] },
+ failed =>
+ { %$gone, state => 'failed', 'paths-active' => 0, 'paths-total' => 0 },
+ kept => { kept => 'vm-100-disk-0' },
+ },
+ },
+ n3 => { fresh => 0, reason => 'offline', luns => { offline => $gone } },
+ n4 => { fresh => 1, 'apply-error' => "boom\n", luns => { errored => $gone } },
+ n5 => { fresh => 1, summary => { luns => 300 } },
+ n7 => { fresh => 1, stalled => 1, luns => { stalled => $gone } },
+ };
+ my $both = { n1 => 1, n2 => 1 };
+ my $rows = PVE::Multipath::Status::aggregate_cluster_status(
+ ['manual'],
+ {},
+ { used => 'lvm' },
+ $reports,
+ {
+ (map { $_ => $both } qw(gone old manual used failed kept)),
+ noreport => { %$both, n6 => 1 },
+ offline => { %$both, n3 => 1 },
+ errored => { %$both, n4 => 1 },
+ summary => { %$both, n5 => 1 },
+ stalled => { %$both, n7 => 1 },
+ },
+ );
+ my %row = map { $_->{wwid} => $_ } $rows->@*;
+ is_deeply(
+ [$row{gone}->@{qw(source cluster-state not-observed)}],
+ ['storage', 'unknown', 1],
+ 'not observed: a LUN every node no longer derives and none has a map of is neutral',
+ );
+ is_deeply(
+ $row{gone}->{nodes},
+ {
+ n1 => { state => 'missing', reason => 'not-derived' },
+ n2 => { state => 'missing', reason => 'not-derived' },
+ },
+ 'not observed: the nodes still show the map as missing, with the reason',
+ );
+
+ for my $case (
+ ['old', 'missing', 'a report without the flag'],
+ ['manual', 'missing', 'an explicit entry'],
+ ['used', 'missing', 'a consuming storage'],
+ ['failed', 'failed', 'a map without paths on one node'],
+ ['noreport', 'missing', 'an expected node without a report'],
+ ['offline', 'missing', 'an offline node listing it'],
+ ['errored', 'missing', 'a node with an apply error'],
+ ['summary', 'missing', 'a node sending only a summary'],
+ ['kept', 'missing', 'a node keeping the LUN'],
+ ['stalled', 'missing', 'a node whose status loop stalled'],
+ ) {
+ my ($wwid, $state, $what) = @$case;
+ ok(!$row{$wwid}->{'not-observed'}, "not observed: not with $what");
+ is($row{$wwid}->{'cluster-state'}, $state, "not observed: $what keeps the state");
+ }
+}
+
+# --- stalled status loops ---
+{
+ my $now = 1000;
+ is_deeply(
+ PVE::Multipath::Status::status_loop_ages(
+ $now,
+ {
+ 'pve-node-9.0/n1' => [100, undef, 990],
+ 'pve2-node/n1' => [100, undef, 900],
+ 'pve2-node/n2' => [100, undef, 930],
+ 'pve-node-9.0/n3' => [100, undef, 1005],
+ 'pve2-node/n3' => [100, undef, 990],
+ 'pve-node-9.0/n4' => [100, undef, 'U'],
+ 'pve-node-9.0/n5' => [100, undef],
+ 'pve-node-9.0/n6' => 'garbage',
+ 'pve2-vm/100' => [100, undef, 990],
+ },
+ ),
+ { n1 => 10, n2 => 70 },
+ 'loop ages: newest of both layouts, nothing for a future, invalid or missing time',
+ );
+ is_deeply(PVE::Multipath::Status::status_loop_ages($now, {}), {}, 'loop ages: an empty dump');
+}
+
+my $stalled_rows = PVE::Multipath::Status::aggregate_cluster_status(
+ ['wO', 'wD', 'wM', 'wS'],
+ {},
+ {},
+ {
+ n1 => {
+ fresh => 1,
+ stalled => 1,
+ luns => {
+ wO => $mk->('optimal', 2, 2),
+ wD => $mk->('degraded', 1, 2),
+ wS => $mk->('optimal', 2, 2),
+ },
+ },
+ n2 => { fresh => 1, luns => { wO => $mk->('optimal', 2, 2) } },
+ },
+ { wO => { n1 => 1, n2 => 1 }, wD => { n1 => 1 }, wM => { n1 => 1 }, wS => { n1 => 1 } },
+);
+my %stalled_by = map { $_->{wwid} => $_ } $stalled_rows->@*;
+is_deeply(
+ $stalled_by{wO}->{nodes}->{n1},
+ { state => 'unknown', reason => 'stalled' },
+ 'stalled loop: an optimal map no longer counts as optimal',
+);
+is($stalled_by{wO}->{'cluster-state'}, 'optimal', 'while another node still reports it optimal');
+is($stalled_by{wS}->{'cluster-state'}, 'unknown', 'and it alone gives no state to the cluster');
+is_deeply(
+ [$stalled_by{wD}->{nodes}->{n1}->@{qw(state reason)}],
+ ['degraded', 'stalled'],
+ 'stalled loop: a reported problem stays, with the reason',
+);
+is($stalled_by{wD}->{'cluster-state'}, 'degraded', 'and still drives the cluster state');
+is_deeply(
+ $stalled_by{wM}->{nodes}->{n1},
+ { state => 'missing', reason => 'stalled' },
+ 'stalled loop: a map that was not assembled stays missing',
+);
+
+# --- expected nodes: explicit entries and the storages deriving or providing a LUN ---
+my $expected = PVE::Multipath::Status::expected_wwid_nodes(
+ { ids => { wE => { type => 'wwid', nodes => { n2 => 1 } }, wAll => { type => 'wwid' } } },
+ {
+ 'storage-nodes' => { s1 => { n1 => 1 }, s2 => { n3 => 1 }, s3 => { n2 => 1 } },
+ 'consumer-bases' => { wC => ['s3'] },
+ },
+ { n1 => { luns => { wD => { storages => ['s1', 's2'] } } } },
+ ['n1', 'n2', 'n3'],
+);
+is_deeply($expected->{wE}, { n2 => 1 }, 'a restricted entry expects its nodes');
+is_deeply($expected->{wAll}, { n1 => 1, n2 => 1, n3 => 1 }, 'an unrestricted one every node');
+is_deeply($expected->{wD}, { n1 => 1, n3 => 1 }, 'a derived LUN the nodes of all its storages');
+is_deeply($expected->{wC}, { n2 => 1 }, 'a consumed LUN the nodes of the consumer base');
+
# --- consumers the configuration would leave without an allow-list entry ---
my $scfg = {
ids => {
diff --git a/src/test/run_san_api_tests.pl b/src/test/run_san_api_tests.pl
index 94ee74f0..249185eb 100755
--- a/src/test/run_san_api_tests.pl
+++ b/src/test/run_san_api_tests.pl
@@ -8,6 +8,7 @@ use Test::More;
use lib ('.', '..');
use PVE::API2::ISCSI;
+use PVE::API2::Multipath;
use PVE::API2::Storage::Scan;
subtest 'iSCSI status fields' => sub {
@@ -45,6 +46,82 @@ subtest 'iSCSI status fields' => sub {
);
};
+subtest 'iSCSI transport helpers preserve command arguments' => sub {
+ my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+ my (@commands, $sessions);
+ $plugin->mock(
+ iscsi_session_list => sub { { 'iqn.test' => $sessions // [] } },
+ iscsi_node_list =>
+ sub { [{ target => 'iqn.test', portal => '::ffff:192.0.2.1:3260' }] },
+ run_command => sub {
+ my ($cmd, %param) = @_;
+ push @commands, $cmd;
+ $param{outfunc}->($_)
+ for 'HeaderDigest: None', 'Recovery Timeout: 120',
+ 'MaxBurstLength: invalid';
+ },
+ );
+ my $call = sub { PVE::API2::ISCSI->map_method_by_name($_[0])->{code}->($_[1]) };
+ is_deeply(
+ $call->('session_params', { 'session-id' => 7 }),
+ { 'header-digest' => 'None', 'recovery-timeout' => 120 },
+ 'session parser retains recognized values and rejects invalid integers',
+ );
+ is_deeply(
+ $commands[-1],
+ ['/usr/bin/iscsiadm', '--mode', 'session', '--sid', 7, '-P', '2'],
+ 'session query arguments',
+ );
+ $call->('rescan', { 'session-id' => 7 });
+ is_deeply(
+ $commands[-1],
+ ['/usr/bin/iscsiadm', '--mode', 'session', '--sid', 7, '--rescan'],
+ 'rescan arguments',
+ );
+ $call->('target_delete', { target => 'iqn.test', portal => '192.0.2.1:3260' });
+ is_deeply(
+ $commands[-1],
+ [
+ '/usr/bin/iscsiadm',
+ '--mode',
+ 'node',
+ '--targetname',
+ 'iqn.test',
+ '--portal',
+ '[::ffff:c000:201]:3260',
+ '--op',
+ 'delete',
+ ],
+ 'mapped IPv4 record uses its stored address with an iscsiadm-compatible spelling',
+ );
+ $sessions = [{ portal => '[::ffff:192.0.2.1]:3260' }];
+ @commands = ();
+ eval { $call->('target_delete', { target => 'iqn.test', portal => '192.0.2.1:3260' }) };
+ like($@, qr/still has an active session/, 'active normalized portal refuses deletion');
+ is_deeply(\@commands, [], 'refusal does not invoke delete');
+ is(
+ PVE::Storage::ISCSIPlugin::iscsi_normalize_portal('2001:db8::1:3260'),
+ '[2001:db8::1]:3260',
+ 'IPv6 comparison key',
+ );
+};
+
+subtest 'node-keyed status schemas validate nested values' => sub {
+ for my $case (['PVE::API2::Multipath', 'luns']) {
+ my ($class, $rows) = @$case;
+ my $schema = $class->map_method_by_name('status')->{returns}->{properties};
+ my $nested = $schema->{$rows}->{items}->{properties}->{nodes};
+ is(ref($nested->{additionalProperties}), 'HASH', "$class typed node map");
+ my $data = { node1 => { state => 'unknown', reason => 'newer-peer-reason' } };
+ eval { PVE::JSONSchema::validate($data, $nested) };
+ is($@, '', 'future reason accepted');
+ $data->{node1}->{'paths-total'} = 'not a count';
+ eval { PVE::JSONSchema::validate($data, $nested) };
+ ok($@, 'bad nested count rejected');
+ is(ref($schema->{nodes}->{additionalProperties}), 'HASH', 'typed node report');
+ }
+};
+
subtest 'node SAN read permissions' => sub {
for my $case (
['PVE::API2::ISCSI', qw(status targets session_params)],
@@ -143,64 +220,4 @@ subtest 'SAN LUN path counts' => sub {
}
};
-subtest 'iSCSI transport helpers preserve command arguments' => sub {
- my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
- my (@commands, $sessions);
- $plugin->mock(
- iscsi_session_list => sub { { 'iqn.test' => $sessions // [] } },
- iscsi_node_list =>
- sub { [{ target => 'iqn.test', portal => '::ffff:192.0.2.1:3260' }] },
- run_command => sub {
- my ($cmd, %param) = @_;
- push @commands, $cmd;
- $param{outfunc}->($_)
- for 'HeaderDigest: None', 'Recovery Timeout: 120',
- 'MaxBurstLength: invalid';
- },
- );
- my $call = sub { PVE::API2::ISCSI->map_method_by_name($_[0])->{code}->($_[1]) };
- is_deeply(
- $call->('session_params', { 'session-id' => 7 }),
- { 'header-digest' => 'None', 'recovery-timeout' => 120 },
- 'session parser retains recognized values and rejects invalid integers',
- );
- is_deeply(
- $commands[-1],
- ['/usr/bin/iscsiadm', '--mode', 'session', '--sid', 7, '-P', '2'],
- 'session query arguments',
- );
- $call->('rescan', { 'session-id' => 7 });
- is_deeply(
- $commands[-1],
- ['/usr/bin/iscsiadm', '--mode', 'session', '--sid', 7, '--rescan'],
- 'rescan arguments',
- );
- $call->('target_delete', { target => 'iqn.test', portal => '192.0.2.1:3260' });
- is_deeply(
- $commands[-1],
- [
- '/usr/bin/iscsiadm',
- '--mode',
- 'node',
- '--targetname',
- 'iqn.test',
- '--portal',
- '[::ffff:c000:201]:3260',
- '--op',
- 'delete',
- ],
- 'mapped IPv4 record uses its stored address with an iscsiadm-compatible spelling',
- );
- $sessions = [{ portal => '[::ffff:192.0.2.1]:3260' }];
- @commands = ();
- eval { $call->('target_delete', { target => 'iqn.test', portal => '192.0.2.1:3260' }) };
- like($@, qr/still has an active session/, 'active normalized portal refuses deletion');
- is_deeply(\@commands, [], 'refusal does not invoke delete');
- is(
- PVE::Storage::ISCSIPlugin::iscsi_normalize_portal('2001:db8::1:3260'),
- '[2001:db8::1]:3260',
- 'IPv6 comparison key',
- );
-};
-
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 36/76] multipath: collect identities of non-iSCSI SCSI paths
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (34 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 35/76] api: multipath: add the cluster-wide health status Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 37/76] api: scan: san-luns: report the multipath state of each LUN Thomas Lamprecht
` (39 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Fibre Channel paths need the same trusted WWID evidence as iSCSI paths,
but have no session target to group them by. Allow collection across
SCSI transports, treating each non-iSCSI path independently until its
identity is established.
Keep unresolved identities distinct from paths with no identity
evidence, so consumers do not mistake failed discovery for an absent
LUN.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Multipath/Identity.pm | 27 ++++++++++------
src/test/run_lun_identity_tests.pl | 49 ++++++++++++++++++++++++++++++
2 files changed, 67 insertions(+), 9 deletions(-)
diff --git a/src/PVE/Multipath/Identity.pm b/src/PVE/Multipath/Identity.pm
index 86141e98..3281c6d4 100644
--- a/src/PVE/Multipath/Identity.pm
+++ b/src/PVE/Multipath/Identity.pm
@@ -64,8 +64,13 @@ sub resolve($disks, $daemon, $naa_ok) {
my $luns = {};
for my $name (sort keys %$disks) {
my $disk = $disks->{$name};
- my $key = "$disk->{target}\0$disk->{lun}";
- $luns->{$key} //= { target => $disk->{target}, lun => $disk->{lun}, paths => [] };
+ my $target = $disk->{target};
+ my $key = defined($target) ? "$target\0$disk->{lun}" : "\0$name";
+ $luns->{$key} //= {
+ defined($target) ? (target => $target) : (),
+ lun => $disk->{lun},
+ paths => [],
+ };
push $luns->{$key}->{paths}->@*, $name;
}
@@ -151,22 +156,26 @@ my sub lun_targets($sessions) {
return $res;
}
-my sub session_disks($sessions, $targets) {
+my sub scsi_disks($sessions, $targets, $all) {
my $lun_target = lun_targets($sessions);
my $res = {};
dir_glob_foreach(
"$SYSFS/bus/scsi/devices",
'\d+:\d+:\d+:(\d+)',
sub($hctl, $lun) {
- my $target = $lun_target->{$hctl} // return;
- return if $targets && !$targets->{$target};
+ my $target = $lun_target->{$hctl};
+ if (defined($target)) {
+ return if $targets && !$targets->{$target} && !$all;
+ } else {
+ return if !$all;
+ }
my $dir = "bus/scsi/devices/$hctl";
return if (sysfs_line("$dir/type") // '') ne '0'; # disks only
my (undef, $name) = dir_glob_regex("$SYSFS/$dir/block", '([a-z]\S*)');
return if !defined($name);
my $devt = sysfs_line("$dir/block/$name/dev") // return;
- my $disk =
- { target => $target, lun => int($lun), devt => $devt, hctl => $hctl, dir => $dir };
+ my $disk = { lun => int($lun), devt => $devt, hctl => $hctl, dir => $dir };
+ $disk->{target} = $target if defined($target);
# Read the holder through the SCSI device, so neither name can refer to another device
# meanwhile.
my (undef, $holder) = dir_glob_regex("$SYSFS/$dir/block/$name/holders", '(dm-\d+)');
@@ -182,7 +191,7 @@ my sub session_disks($sessions, $targets) {
sub collect(%param) {
my $sessions = iscsi_sessions();
- my $disks = session_disks($sessions, $param{targets});
+ my $disks = scsi_disks($sessions, $param{targets}, $param{all});
my ($daemon, $error);
if (grep { !defined($_->{map}) } values %$disks) {
@@ -235,7 +244,7 @@ sub portals_covered($ids, $target, $portals) {
}
sub target_luns($ids, $target) {
- return [grep { $_->{target} eq $target } values $ids->{luns}->%*];
+ return [grep { ($_->{target} // '') eq $target } values $ids->{luns}->%*];
}
# The WWID of the LUN of the path $name in $ids, if it was resolved and the name still denotes the
diff --git a/src/test/run_lun_identity_tests.pl b/src/test/run_lun_identity_tests.pl
index 4e9507a0..9f89c923 100755
--- a/src/test/run_lun_identity_tests.pl
+++ b/src/test/run_lun_identity_tests.pl
@@ -881,4 +881,53 @@ $rows = PVE::Multipath::Status::aggregate_cluster_status(
);
ok($rows->[0]->{'not-observed'}, 'end to end: the last LUN of a target retired, not observed');
+# --- paths of other transports ---
+
+{
+ # a Fibre Channel LUN on host 20, on a node without any iSCSI session: each path counts on
+ # its own
+ $identity->unmock_all();
+ my $iscsi_sys = $sys;
+ $sys = tempdir(CLEANUP => 1);
+ $PVE::Multipath::Identity::SYSFS = $sys;
+ make_path("$sys/class/iscsi_session");
+ disk_device('20:0:0:0', 'sdg', '8:96');
+ disk_device('20:0:1:0', 'sdh', '8:112');
+ $queries = 0;
+ $reply = {
+ sdg => {
+ devt => '8:96',
+ hctl => '20:0:0:0',
+ init => 'ok',
+ target => '0x200',
+ wwid => 'SFC_LUN',
+ },
+ sdh => {
+ devt => '8:112',
+ hctl => '20:0:1:0',
+ init => 'ok',
+ target => '0x201',
+ wwid => 'SFC_LUN',
+ },
+ };
+ my $ids = PVE::Multipath::Identity::collect(targets => {});
+ is(
+ PVE::Multipath::Identity::path_wwid($ids, 'sdg'),
+ undef,
+ 'other transports: not without asking for them',
+ );
+ $ids = PVE::Multipath::Identity::collect(all => 1);
+ is_deeply(
+ [map { PVE::Multipath::Identity::path_wwid($ids, $_) } qw(sdg sdh)],
+ ['SFC_LUN', 'SFC_LUN'],
+ 'other transports: every path identified on its own',
+ );
+ is(
+ scalar(grep { !defined($_->{target}) } values $ids->{luns}->%*),
+ 2,
+ 'other transports: as LUNs of their own, as nothing groups the paths',
+ );
+ ($sys, $PVE::Multipath::Identity::SYSFS) = ($iscsi_sys, $iscsi_sys);
+}
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 37/76] api: scan: san-luns: report the multipath state of each LUN
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (35 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 36/76] multipath: collect identities of non-iSCSI SCSI paths Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 38/76] api: scan: flag duplicate physical volumes on distinct LUNs Thomas Lamprecht
` (38 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Clients need a LUN's WWID and local management state to offer multipath
configuration actions. For paths without maps, use the WWID from
multipathd so it matches storage-derived identities. On nodes without
multipath-tools, take NAA identities from the udev property that
multipathd reads, checked against the VPD page of the device, so plain
iSCSI LUNs keep their identity there.
Distinguish maps managed by a storage or explicit entry, maps retained
while in use, unmanaged maps, pending maps, and LUNs not allow-listed.
Include the associated storage, iSCSI target and LUN number, and
expected path count where known.
Keep unresolved SCSI identities distinct from devices without identity
evidence. Changed or disagreeing paths and an unavailable daemon must
not be mistaken for a LUN the node cannot see.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Disks.pm | 2 +-
src/PVE/API2/Storage/Scan.pm | 55 +++++-
src/PVE/CLI/pvesm.pm | 4 +-
src/PVE/Multipath.pm | 6 +
src/PVE/Multipath/Identity.pm | 38 +++-
src/PVE/Storage/SANScan.pm | 106 +++++++++--
src/test/Makefile | 5 +-
src/test/run_lun_identity_tests.pl | 45 +++++
src/test/run_san_api_tests.pl | 47 +----
src/test/run_san_lun_scan_tests.pl | 295 +++++++++++++++++++++++++++++
10 files changed, 529 insertions(+), 74 deletions(-)
create mode 100755 src/test/run_san_lun_scan_tests.pl
diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index dd3524b5..58a89c67 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -101,7 +101,7 @@ __PACKAGE__->register_method({
'include-remote' => {
description => "Also include devices attached through a remote storage"
. " transport (for example FC, iSCSI, or NVMe over TCP), which are"
- . " excluded by default.",
+ . " excluded by default. See scan/san-luns for paths and multipath state.",
type => 'boolean',
optional => 1,
default => 0,
diff --git a/src/PVE/API2/Storage/Scan.pm b/src/PVE/API2/Storage/Scan.pm
index 73632a1f..93c78ac2 100644
--- a/src/PVE/API2/Storage/Scan.pm
+++ b/src/PVE/API2/Storage/Scan.pm
@@ -380,10 +380,9 @@ __PACKAGE__->register_method({
name => 'sanlunscan',
path => 'san-luns',
method => 'GET',
- description => "List block devices (disks and multipath devices) together with their"
- . " usage, for example as candidates for a SAN backed LVM volume group. For multipath"
- . " mapped devices and NVMe namespaces handled by native NVMe multipath, the state"
- . " of the member paths is included.",
+ description => "List the block devices of the node with their SAN details: paths, multipath"
+ . " state, target, and usage. Local disks are included; the web UI filters them by"
+ . " transport.",
protected => 1,
proxyto => "node",
permissions => {
@@ -438,6 +437,13 @@ __PACKAGE__->register_method({
type => 'integer',
optional => 1,
},
+ 'paths-expected' => {
+ description => "The number of paths the LUN should have according to the"
+ . " storage configuration, one per portal for iSCSI; absent when it is"
+ . " not known.",
+ type => 'integer',
+ optional => 1,
+ },
'path-faults' => {
description => "The number of path failures of a multipath mapped"
. " device since its creation, as reported by multipathd.",
@@ -514,6 +520,47 @@ __PACKAGE__->register_method({
type => 'string',
optional => 1,
},
+ wwid => {
+ description => "The WWID of the LUN as used by dm-multipath, for SCSI"
+ . " devices with a designator, and for multipath mapped devices.",
+ type => 'string',
+ optional => 1,
+ },
+ 'wwid-unknown' => {
+ description => "The SCSI path has no trusted LUN identity because identity"
+ . " discovery failed, disagreed, or the path changed during the scan.",
+ type => 'boolean',
+ optional => 1,
+ },
+ multipath => {
+ description => "Multipath management on this node: 'storage' for a map"
+ . " allow-listed by a multipath-enabled storage; 'manual' for a map"
+ . " allow-listed by an explicit cluster-wide entry; 'kept' for a map no"
+ . " longer configured but still allow-listed while in use; 'unmanaged'"
+ . " for a map assembled outside the managed configuration; 'pending' for"
+ . " a path of an allow-listed LUN whose map is not yet assembled; or"
+ . " 'none' for a path of a LUN that is not allow-listed.",
+ type => 'string',
+ enum => ['none', 'pending', 'storage', 'manual', 'kept', 'unmanaged'],
+ optional => 1,
+ },
+ 'multipath-storage' => {
+ description =>
+ "The multipath-enabled storage the LUN belongs to, whether or"
+ . " not it also has an explicit multipath entry.",
+ type => 'string',
+ optional => 1,
+ },
+ 'iscsi-target' => {
+ description => "The IQN of the iSCSI target the LUN is attached through.",
+ type => 'string',
+ optional => 1,
+ },
+ 'iscsi-lun' => {
+ description => "The number of the LUN on its iSCSI target.",
+ type => 'integer',
+ optional => 1,
+ },
detail => {
description =>
"Details about other usage, for example the file system type.",
diff --git a/src/PVE/CLI/pvesm.pm b/src/PVE/CLI/pvesm.pm
index 67419da2..dfe6c25b 100755
--- a/src/PVE/CLI/pvesm.pm
+++ b/src/PVE/CLI/pvesm.pm
@@ -743,8 +743,8 @@ our $cmddef = {
{ node => $nodename },
sub {
my ($data, $schema, $options) = @_;
- my $columns =
- [qw(devpath size transport paths-active usage vgname used-by-storage)];
+ my $columns = [qw(devpath size transport paths-active paths-expected multipath
+ usage vgname used-by-storage wwid)];
PVE::CLIFormatter::print_api_result($data, $schema, $columns, $options);
},
$PVE::RESTHandler::standard_output_options,
diff --git a/src/PVE/Multipath.pm b/src/PVE/Multipath.pm
index acca5a11..81f6d867 100644
--- a/src/PVE/Multipath.pm
+++ b/src/PVE/Multipath.pm
@@ -36,6 +36,12 @@ sub assert_supported() {
return 1;
}
+# Neither tool is installed, so there is no daemon to ask for identities. A partial installation
+# does not count, as a daemon with its own identity settings may still run.
+sub is_absent() {
+ return !-e $MULTIPATH && !-e $MULTIPATHD;
+}
+
sub is_running() {
return 0 if !is_supported();
diff --git a/src/PVE/Multipath/Identity.pm b/src/PVE/Multipath/Identity.pm
index 3281c6d4..5fa6542d 100644
--- a/src/PVE/Multipath/Identity.pm
+++ b/src/PVE/Multipath/Identity.pm
@@ -1,8 +1,9 @@
package PVE::Multipath::Identity;
-# Use multipathd's configured identity, not a guessed sysfs identifier. Maps pin their paths;
-# unmapped names can be reused, so daemon entries must match dev_t, H:C:T:L and target, plus
-# the cached VPD NAA designator when applicable. Only iSCSI has grouping evidence before resolution.
+# Use multipathd's configured identity, not a guessed sysfs identifier; on nodes without
+# multipath-tools, NAA identities from the udev property it reads. Maps pin their paths; unmapped
+# names can be reused, so daemon entries must match dev_t, H:C:T:L and target, plus the cached VPD
+# NAA designator when applicable. Only iSCSI has grouping evidence before resolution.
use v5.36;
@@ -14,6 +15,7 @@ use PVE::Multipath::Config;
use PVE::Tools qw(dir_glob_foreach dir_glob_regex file_get_contents file_read_firstline);
our $SYSFS = '/sys';
+our $UDEV_DATA = '/run/udev/data';
my sub sysfs_line($path) {
return file_read_firstline("$SYSFS/$path");
@@ -189,6 +191,28 @@ my sub scsi_disks($sessions, $targets, $all) {
return $res;
}
+# Without multipath-tools there is no daemon to ask. Take the udev property that multipathd reads
+# the WWID of a SCSI path from by default, in the form of its path list. Unlike a daemon entry, a
+# udev record does not name the address of its device and can outlive it, so only an NAA identity
+# counts, which line_identifies() checks against the VPD page of the current device. Paths with
+# other identities count as not listed.
+my sub udev_path_identities($disks) {
+ my $res = {};
+ for my $name (sort keys %$disks) {
+ my $disk = $disks->{$name};
+ my $data = eval { file_get_contents("$UDEV_DATA/b$disk->{devt}") } // next;
+ my ($wwid) = $data =~ m/^E:ID_SERIAL=(3(?:[0-9a-fA-F]{16}){1,2})$/m or next;
+ $res->{$name} = {
+ devt => $disk->{devt},
+ hctl => $disk->{hctl},
+ init => 'ok',
+ target => $disk->{target} // '[undef]',
+ wwid => $wwid,
+ };
+ }
+ return $res;
+}
+
sub collect(%param) {
my $sessions = iscsi_sessions();
my $disks = scsi_disks($sessions, $param{targets}, $param{all});
@@ -196,7 +220,13 @@ sub collect(%param) {
my ($daemon, $error);
if (grep { !defined($_->{map}) } values %$disks) {
$daemon = eval { PVE::Multipath::query_path_identities() };
- chomp($error = $@) if $@;
+ if (my $err = $@) {
+ if (PVE::Multipath::is_absent()) {
+ $daemon = udev_path_identities($disks);
+ } else {
+ chomp($error = $err);
+ }
+ }
}
my $naa_ok = sub($disk, $hex) {
my $page = eval { file_get_contents("$SYSFS/$disk->{dir}/vpd_pg83") };
diff --git a/src/PVE/Storage/SANScan.pm b/src/PVE/Storage/SANScan.pm
index ee311cb9..415efdba 100644
--- a/src/PVE/Storage/SANScan.pm
+++ b/src/PVE/Storage/SANScan.pm
@@ -4,11 +4,16 @@ use v5.36;
use PVE::Diskmanage;
use PVE::Multipath;
+use PVE::Multipath::ClusterConfig;
+use PVE::Multipath::Config;
+use PVE::Multipath::Identity;
use PVE::Storage;
use PVE::Storage::ISCSIPlugin;
use PVE::Storage::LVMPlugin;
my sub collect_inventory($cfg, $node) {
+ # Collect identities before disks so path_wwid() rejects names reused during enumeration.
+ my $ids = PVE::Multipath::Identity::collect(all => 1);
my $disks = PVE::Diskmanage::get_disks(undef, 1, 0);
my $multipath = PVE::Diskmanage::get_multipath_disks();
@@ -61,31 +66,74 @@ my sub collect_inventory($cfg, $node) {
}
}
- # map kernel device name to the storage using its iSCSI target
+ # map kernel device name to its iSCSI session and to the storage using that target
+ my $device_map = PVE::Storage::ISCSIPlugin::iscsi_session_device_map();
my $iscsi_used_by = {};
- if (scalar(keys %$iscsi_target_used_by)) {
- my $device_map = PVE::Storage::ISCSIPlugin::iscsi_session_device_map();
- for my $bdev (sort keys %$device_map) {
- my $sid = $iscsi_target_used_by->{ $device_map->{$bdev}->{target} } // next;
- $iscsi_used_by->{$bdev} = $sid;
+ for my $bdev (sort keys %$device_map) {
+ my $sid = $iscsi_target_used_by->{ $device_map->{$bdev}->{target} } // next;
+ $iscsi_used_by->{$bdev} = $sid;
+ }
+
+ # A pre-existing allow-list entry can be managed for a storage without being owned by PVE.
+ my $mp_current = { map { $_ => 1 } PVE::Multipath::list_etc_multipath_wwids()->@* };
+ my $mp_cfg = eval { PVE::Multipath::ClusterConfig::read_config() };
+ warn $@ if $@;
+ my $mp_explicit = { map { $_ => 1 }
+ $mp_cfg ? PVE::Multipath::Config::wwids_for_node($mp_cfg, $node)->@* : () };
+ my $mp_owned = eval { PVE::Multipath::managed_wwid_record() } // {};
+ warn $@ if $@;
+ my ($mp_storage, $mp_expected) = ({}, {});
+ my $derived = PVE::Storage::derived_multipath_wwids($cfg, $ids);
+
+ for my $sid (sort keys %$derived) {
+ my $luns = $derived->{$sid}->{wwids};
+ for my $wwid (keys %$luns) {
+ $mp_storage->{$wwid} //= $sid;
+ my $expected = $luns->{$wwid}->{'paths-expected'};
+ $mp_expected->{$wwid} = $expected
+ if defined($expected) && $expected > ($mp_expected->{$wwid} // 0);
}
}
return {
+ ids => $ids,
disks => $disks,
multipath => $multipath,
mp_status => $mp_status,
nvme_status => $nvme_status,
+ device_map => $device_map,
vg_used_by => $vg_used_by,
iscsi_target_used_by => $iscsi_target_used_by,
+ mp_current => $mp_current,
+ mp_explicit => $mp_explicit,
+ mp_owned => $mp_owned,
+ mp_storage => $mp_storage,
+ mp_expected => $mp_expected,
find_vg => $find_vg,
iscsi_used_by => $iscsi_used_by,
};
}
my sub build_lun_rows($cfg, $inventory) {
- my ($disks, $multipath, $mp_status, $nvme_status, $vg_used_by, $find_vg, $iscsi_used_by) =
- $inventory->@{ qw(disks multipath mp_status nvme_status vg_used_by find_vg
- iscsi_used_by) };
+ my (
+ $ids,
+ $disks,
+ $multipath,
+ $mp_status,
+ $nvme_status,
+ $device_map,
+ $vg_used_by,
+ $mp_current,
+ $mp_explicit,
+ $mp_owned,
+ $mp_storage,
+ $mp_expected,
+ $find_vg,
+ $iscsi_used_by,
+ )
+ = $inventory->@{
+ qw(ids disks multipath mp_status nvme_status device_map
+ vg_used_by mp_current mp_explicit mp_owned mp_storage mp_expected find_vg iscsi_used_by)
+ };
my $records = [];
my $add_entry = sub($disk) {
@@ -126,14 +174,38 @@ my sub build_lun_rows($cfg, $inventory) {
$entry->{usage} = 'other';
$entry->{detail} = $disk->{used};
}
- if (!defined($entry->{'used-by-storage'})) {
- my @members =
- map { $_ =~ s|^/dev/||r } ($disk->{slaves} // [$disk->{devpath}])->@*;
- for my $member (@members) {
- if (defined(my $sid = $iscsi_used_by->{$member})) {
- $entry->{'used-by-storage'} = $sid;
- last;
- }
+ my @members = map { $_ =~ s|^/dev/||r } ($disk->{slaves} // [$disk->{devpath}])->@*;
+ for my $member (@members) {
+ my $session_device = $device_map->{$member} // next;
+ $entry->{'iscsi-target'} //= $session_device->{target};
+ $entry->{'iscsi-lun'} //= $session_device->{lun};
+ $entry->{'used-by-storage'} //= $iscsi_used_by->{$member}
+ if defined($iscsi_used_by->{$member});
+ }
+ my $wwid;
+ if (defined($disk->{slaves})) {
+ $wwid = $disk->{wwn};
+ } elsif ($disk->{devpath} =~ m|^/dev/(sd[a-z]+)$|) {
+ my $name = $1;
+ $wwid = PVE::Multipath::Identity::path_wwid($ids, $name);
+ $entry->{'wwid-unknown'} = 1
+ if !defined($wwid) && $ids->{paths}->{$name}->{unresolved};
+ }
+ if (defined($wwid)) {
+ $entry->{wwid} = $wwid;
+ $entry->{'multipath-storage'} = $mp_storage->{$wwid} if $mp_storage->{$wwid};
+ $entry->{'paths-expected'} = $mp_expected->{$wwid}
+ if defined($mp_expected->{$wwid});
+ if (!defined($disk->{slaves})) {
+ $entry->{multipath} = $mp_current->{$wwid} ? 'pending' : 'none';
+ } elsif ($mp_explicit->{$wwid}) {
+ $entry->{multipath} = 'manual';
+ } elsif ($mp_storage->{$wwid}) {
+ $entry->{multipath} = 'storage';
+ } elsif ($mp_owned->{$wwid}) {
+ $entry->{multipath} = 'kept';
+ } else {
+ $entry->{multipath} = 'unmanaged';
}
}
push $records->@*, { row => $entry, disk => $disk };
diff --git a/src/test/Makefile b/src/test/Makefile
index 9ecd59d4..91c685e7 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_base_activation test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_iscsi test_lun_identity test_multipath_api test_san_api test_san_libraries test_san_taint
+test: test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_base_activation test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_iscsi test_lun_identity test_multipath_api test_san_lun_scan test_san_api test_san_libraries test_san_taint
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -70,6 +70,9 @@ test_lun_identity: run_lun_identity_tests.pl
test_multipath_api: run_multipath_api_tests.pl
./run_multipath_api_tests.pl
+test_san_lun_scan: run_san_lun_scan_tests.pl
+ ./run_san_lun_scan_tests.pl
+
test_san_api: run_san_api_tests.pl
./run_san_api_tests.pl
diff --git a/src/test/run_lun_identity_tests.pl b/src/test/run_lun_identity_tests.pl
index 9f89c923..c5363e46 100755
--- a/src/test/run_lun_identity_tests.pl
+++ b/src/test/run_lun_identity_tests.pl
@@ -348,6 +348,9 @@ put('bus/scsi/devices/7:0:0:9/type', "1\n");
my ($queries, $reply) = (0);
my $multipath = Test::MockModule->new('PVE::Multipath');
$multipath->mock(
+ # a failed query means a broken daemon, not a node without multipath-tools, whatever the build
+ # host has installed
+ is_absent => sub { 0 },
query_path_identities => sub {
$queries++;
die $reply if !ref($reply);
@@ -400,6 +403,48 @@ is_deeply(
{ sdb => '3600a09803832355367242f747a716b7a' },
'collect: and the LUNs with a map still resolve, through the holder link of their paths',
);
+{
+ # a node without multipath-tools: the udev property that multipathd would read
+ local $PVE::Multipath::Identity::UDEV_DATA = "$sys/udev";
+ $multipath->mock(is_absent => sub { 1 });
+ $reply = "multipath-tools is not installed\n";
+ put(
+ 'udev/b8:32',
+ "S:disk/by-id/scsi-3600a09803832355367242f747a716b7a\n"
+ . "E:ID_SERIAL=3600a09803832355367242f747a716b7a\n",
+ );
+ $ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+ is($ids->{error}, undef, 'collect: no multipath-tools is not a failed query');
+ is_deeply(
+ { map { $_->{lun} => $_->{wwid} } PVE::Multipath::Identity::target_luns($ids, $T)->@* },
+ { 0 => '3600a09803832355367242f747a716b7a', 1 => '3600a09803832355367242f747a716b7a' },
+ 'collect: without multipath-tools, a path udev and the VPD page of the device name alike',
+ );
+ put('udev/b8:32', "E:ID_SERIAL=3600a09803832355367242f747a716c30\n");
+ $ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+ is($ids->{paths}->{sdc}->{unresolved}, 1, 'collect: but not one that the VPD page contradicts');
+ unlink("$sys/udev/b8:32") or die "$!\n";
+ $ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+ ok(
+ !$ids->{paths}->{sdc}->{unresolved},
+ 'collect: a path without udev identity is not unresolved',
+ );
+ is(PVE::Multipath::Identity::path_wwid($ids, 'sdc'), undef, 'collect: and has no identity');
+ # a serial without an NAA designator that the VPD page could confirm, maybe of the device that
+ # had the number before
+ put('udev/b8:32', "E:ID_SERIAL=20011223344556677\n");
+ $ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+ ok(
+ !$ids->{paths}->{sdc}->{unresolved},
+ 'collect: a path with another kind of serial is not listed',
+ );
+ is(
+ PVE::Multipath::Identity::path_wwid($ids, 'sdc'),
+ undef,
+ 'collect: and has no identity either',
+ );
+ $multipath->mock(is_absent => sub { 0 });
+}
# a later device under the name of a path: another LUN, the same dev_t
$reply = {
diff --git a/src/test/run_san_api_tests.pl b/src/test/run_san_api_tests.pl
index 249185eb..c6a3bb54 100755
--- a/src/test/run_san_api_tests.pl
+++ b/src/test/run_san_api_tests.pl
@@ -165,7 +165,8 @@ subtest 'SAN CLI columns leave machine output complete' => sub {
is($_[1], $schema, 'complete schema forwarded');
is_deeply(
$_[2],
- [qw(devpath size transport paths-active usage vgname used-by-storage)],
+ [qw(devpath size transport paths-active paths-expected multipath
+ usage vgname used-by-storage wwid)],
'concise text columns',
);
is($_[3]->{'output-format'}, 'json', 'format option unchanged');
@@ -176,48 +177,4 @@ subtest 'SAN CLI columns leave machine output complete' => sub {
is($schema->{items}->{properties}->{size}->{renderer}, 'bytes', 'size renderer');
};
-subtest 'SAN LUN path counts' => sub {
- my $disk = Test::MockModule->new('PVE::Diskmanage');
- my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
- my $storage = Test::MockModule->new('PVE::Storage');
- my $disks =
- { map { ("/dev/$_" => { devpath => "/dev/$_", size => 1024, multipath => 'm0' }) }
- qw(sdb sdc) };
- $disk->mock(
- get_disks => sub { $disks },
- get_multipath_disks => sub {
- return {
- '/dev/mapper/m0' => {
- devpath => '/dev/mapper/m0',
- size => 1024,
- slaves => ['/dev/sdb', '/dev/sdc'],
- paths => 2,
- },
- };
- },
- );
- $lvm->mock(lvm_vgs => sub { {} });
- $storage->mock(config => sub { { ids => {} } });
- # native NVMe multipath: two controllers, one of them usable
- $disks->{'/dev/nvme0n1'} = { devpath => '/dev/nvme0n1', size => 1024 };
- my $paths = [{ device => '/dev/nvme0' }, { device => '/dev/nvme1' }];
- $disk->mock(
- get_nvme_path_status => sub { { nvme0n1 => { active => 1, paths => $paths } } });
- my $multipath = Test::MockModule->new('PVE::Multipath');
- $multipath->mock(path_status_by_wwid => sub { {} });
- my $declared =
- PVE::API2::Storage::Scan->map_method_by_name('sanlunscan')->{returns}->{items}
- ->{properties};
- my $rows = PVE::API2::Storage::Scan->sanlunscan({ node => 'test' });
- my $by_dev = { map { ($_->{devpath} => $_) } @$rows };
- is($by_dev->{'/dev/mapper/m0'}->{'paths-total'}, 2, 'map path count');
- # native NVMe multipath
- is($by_dev->{'/dev/nvme0n1'}->{'paths-total'}, 2, 'namespace path count');
- is($by_dev->{'/dev/nvme0n1'}->{'paths-active'}, 1, 'namespace usable paths');
- for my $row (@$rows) {
- my @undeclared = grep { !$declared->{$_} } sort keys %$row;
- is_deeply(\@undeclared, [], "$row->{devpath}: every field declared");
- }
-};
-
done_testing();
diff --git a/src/test/run_san_lun_scan_tests.pl b/src/test/run_san_lun_scan_tests.pl
new file mode 100755
index 00000000..c9f130ff
--- /dev/null
+++ b/src/test/run_san_lun_scan_tests.pl
@@ -0,0 +1,295 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib '..';
+use File::Path;
+use File::Temp;
+use Test::MockModule;
+use Test::More;
+
+use PVE::API2::Storage::Scan;
+use PVE::Multipath::Identity;
+
+my $T = 'iqn.1992-08.com.netapp:sn.1:vs.5';
+
+# the devices, with the PV each carries: { name => { disk => get_disks entry, pv => uuid } }, and
+# the maps: { wwid => { slaves => [names], pv => uuid } }
+my ($devices, $maps, $resolvable, $namespace_ids) = ({}, {}, 1, {});
+
+my sub sd {
+ my ($name, $lun, $pv, %extra) = @_;
+ return (
+ $name => {
+ disk => {
+ devpath => "/dev/$name",
+ size => 1024,
+ transport => 'iscsi',
+ wwn => $extra{wwn} // 'unknown',
+ defined($pv) ? (used => 'LVM') : (),
+ defined($extra{map}) ? (multipath => $extra{map}) : (),
+ },
+ lun => $lun,
+ pv => $pv,
+ map => $extra{map},
+ daemon => $extra{daemon},
+ },
+ );
+}
+
+# what the scan collects in which order
+my @order;
+my $diskmanage = Test::MockModule->new('PVE::Diskmanage');
+$diskmanage->mock(
+ get_disks => sub {
+ push @order, 'disks';
+ return { map { ("/dev/$_" => $devices->{$_}->{disk}) } keys %$devices };
+ },
+ get_multipath_disks => sub {
+ return {
+ map {
+ my $wwid = $_;
+ (
+ "/dev/mapper/$wwid" => {
+ devpath => "/dev/mapper/$wwid",
+ size => 1024,
+ wwn => $wwid,
+ slaves => [map { "/dev/$_" } $maps->{$wwid}->{slaves}->@*],
+ paths => scalar($maps->{$wwid}->{slaves}->@*),
+ used => 'LVM',
+ },
+ )
+ } keys %$maps
+ };
+ },
+ get_nvme_path_status => sub { {} },
+);
+my $multipath = Test::MockModule->new('PVE::Multipath');
+$multipath->mock(
+ path_status_by_wwid => sub { {} },
+ list_etc_multipath_wwids => sub { [] },
+ managed_wwid_record => sub { {} },
+);
+my $cluster_config = Test::MockModule->new('PVE::Multipath::ClusterConfig');
+$cluster_config->mock(read_config => sub { { ids => {} } });
+my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
+$lvm->mock(lvm_vgs => sub { {} });
+my $storage = Test::MockModule->new('PVE::Storage');
+# names whose device changed since the collection, and what the storages derive from
+my ($reused, $collected, $derived_with) = ({});
+$storage->mock(
+ config => sub { { ids => {} } },
+ derived_multipath_wwids => sub { $derived_with = $_[1]; return {} },
+);
+my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+$iscsi->mock(
+ iscsi_session_device_map => sub {
+ return {
+ map { $_ => { target => $T, channel => 0, id => 0, lun => $devices->{$_}->{lun} } }
+ grep { defined($devices->{$_}->{lun}) } keys %$devices
+ };
+ },
+);
+my $identity = Test::MockModule->new('PVE::Multipath::Identity');
+$identity->mock(
+ path_wwid => sub {
+ my ($ids, $name) = @_;
+ if ($reused->{$name}) {
+ $ids->{paths}->{$name}->{unresolved} = 1;
+ return undef;
+ }
+ my ($lun) = grep {
+ grep { $_ eq $name } $_->{paths}->@*
+ } values $ids->{luns}->%*;
+ return $lun ? $lun->{wwid} : undef;
+ },
+ collect => sub {
+ push @order, 'identities';
+ my $disks = {};
+ my $daemon = {};
+ for my $name (keys %$devices) {
+ my $dev = $devices->{$name};
+ next if $name !~ m/^sd/;
+ my $hctl = "7:0:0:" . ($dev->{lun} // 99);
+ $disks->{$name} = {
+ defined($dev->{lun}) ? (target => $T) : (),
+ lun => $dev->{lun} // 99,
+ devt => "8:$name",
+ hctl => $hctl,
+ defined($dev->{map}) ? (map => $dev->{map}) : (),
+ };
+ $daemon->{$name} = {
+ devt => "8:$name",
+ hctl => $hctl,
+ init => 'ok',
+ target => $T,
+ wwid => $dev->{daemon},
+ }
+ if defined($dev->{daemon});
+ }
+ my $luns =
+ PVE::Multipath::Identity::resolve($disks, $resolvable ? $daemon : undef, sub { 1 });
+ my $paths = {};
+ for my $lun (values %$luns) {
+ $paths->{$_} = { unresolved => $lun->{unresolved} } for $lun->{paths}->@*;
+ }
+ $collected = { luns => $luns, paths => $paths };
+ return $collected;
+ },
+);
+
+# the schema must declare every field of a row, as the CLI and the web UI rely on it
+my $declared =
+ PVE::API2::Storage::Scan->map_method_by_name('sanlunscan')->{returns}->{items}->{properties};
+my sub scan {
+ my $res = PVE::API2::Storage::Scan->sanlunscan({ node => 'lab1' });
+ for my $row (@$res) {
+ my @undeclared = grep { !$declared->{$_} } sort keys %$row;
+ fail("$row->{devpath} has undeclared fields: @undeclared") if @undeclared;
+ }
+ return { map { $_->{devpath} =~ s|^/dev/(mapper/)?||r => $_ } @$res };
+}
+# a map of LUN 0 over sdb and sdc, and sdd, a path of it without a map yet
+($devices, $maps) = (
+ {
+ sd('sdb', 0, undef, map => '3W0'),
+ sd('sdc', 0, undef, map => '3W0'),
+ sd('sdd', 0, 'PV0', daemon => '3W0'),
+ },
+ { '3W0' => { slaves => ['sdb', 'sdc'], pv => 'PV0' } },
+);
+is(scan()->{'3W0'}->{'paths-total'}, 2, 'map path count of the disk scan');
+my $res = scan();
+is($res->{sdd}->{wwid}, '3W0', 'wwid: a path without a map has the identity of its LUN');
+ok($derived_with && $derived_with == $collected, 'the storages derive from the same collection');
+is_deeply([@order[0, 1]], ['identities', 'disks'], 'the identities are collected before the disks');
+$reused->{sdd} = 1;
+ok(!defined(scan()->{sdd}->{wwid}), 'wwid: none for a name that denotes another device by now');
+is(scan()->{sdd}->{'wwid-unknown'}, 1, 'changed path exports identity uncertainty');
+$reused = {};
+$resolvable = 0;
+$res = scan();
+is($res->{sdd}->{wwid}, '3W0', 'wwid: also without the daemon, through the map of its LUN');
+ok(!$res->{sdd}->{'wwid-unknown'}, 'trusted map identity is not uncertain');
+ok(!$res->{'3W0'}->{'wwid-unknown'}, 'map row retains trusted identity');
+
+# LUN 5 without any map
+($devices, $maps) = (
+ { sd('sdl', 5, 'PV0', daemon => '3W5'), sd('sdb', 0, undef, map => '3W0') },
+ { '3W0' => { slaves => ['sdb'], pv => 'PV0' } },
+);
+$res = scan();
+ok(
+ !defined($res->{sdl}->{wwid}),
+ 'wwid: none for a LUN without a map while the daemon cannot be asked',
+);
+ok(!defined($res->{sdl}->{multipath}), 'and no multipath state either, as it is not known');
+is($res->{sdl}->{'wwid-unknown'}, 1, 'daemon unavailable exports uncertainty');
+$resolvable = 1;
+
+# two paths of LUN 1 without a map
+($devices, $maps) =
+ ({ sd('sde', 1, 'PV1', daemon => '3W1'), sd('sdf', 1, 'PV1', daemon => '3W1') }, {});
+$devices->{sdf}->{daemon} = '3OTHER';
+$res = scan();
+is($res->{$_}->{'wwid-unknown'}, 1, "disagreeing LUN path $_ is uncertain") for qw(sde sdf);
+$devices->{sdf}->{daemon} = '3W1';
+
+# local disks without any identity
+($devices, $maps) = ({ sd('sdj', undef, 'PV5'), sd('sdk', undef, 'PV5') }, {});
+ok(!scan()->{sdj}->{'wwid-unknown'}, 'unlisted path is not marked uncertain');
+$devices->{sdj}->{daemon} = '';
+is(scan()->{sdj}->{'wwid-unknown'}, 1, 'listed path without identity is uncertain');
+
+# the real collection and derivation on a fake sysfs: a failing daemon query runs only once, and
+# both the scan and the storage derivation do without its data
+{
+ $identity->unmock_all();
+ $storage->unmock('derived_multipath_wwids');
+ my $sys = File::Temp::tempdir(CLEANUP => 1);
+ $PVE::Multipath::Identity::SYSFS = $sys;
+ my sub put {
+ my ($path, $content) = @_;
+ File::Path::make_path("$sys/$path" =~ s|/[^/]+$||r);
+ open(my $fh, '>', "$sys/$path") or die "$!\n";
+ print $fh $content;
+ close($fh);
+ }
+ # the LUN below the target of its session, as the iSCSI transport class places it
+ my $lun = "$sys/devices/platform/host7/session1/target7:0:0/7:0:0:6";
+ File::Path::make_path($lun, "$sys/bus/scsi/devices");
+ put('class/iscsi_session/session1/targetname', "$T\n");
+ symlink("$sys/devices/platform/host7/session1", "$sys/class/iscsi_session/session1/device")
+ or die "$!\n";
+ symlink($lun, "$sys/bus/scsi/devices/7:0:0:6") or die "$!\n";
+ put('bus/scsi/devices/7:0:0:6/type', "0\n");
+ put('bus/scsi/devices/7:0:0:6/block/sdq/dev', "65:0\n");
+ File::Path::make_path("$sys/bus/scsi/devices/7:0:0:6/block/sdq/holders");
+ my $queries = 0;
+ $multipath->mock(
+ is_absent => sub { 0 },
+ is_supported => sub { 1 },
+ query_path_identities =>
+ sub { $queries++; die "multipathd returned an invalid path list\n" },
+ );
+ $storage->mock(
+ config => sub { {
+ ids => {
+ san =>
+ { type => 'iscsi', target => $T, portal => '192.0.2.10', multipath => 1 },
+ },
+ } },
+ );
+ $iscsi->mock(iscsi_portals => sub { ['192.0.2.10:3260'] });
+ ($devices, $maps) = ({ sd('sdq', 6, 'PVQ') }, {});
+ $res = scan();
+ is($queries, 1, 'shared: the daemon is asked once for the scan and the storages');
+ ok(!defined($res->{sdq}->{wwid}), 'shared: and the path has no WWID without it');
+ is($res->{sdq}->{'wwid-unknown'}, 1, 'real collection exports uncertainty to scan');
+ my $ids = PVE::Multipath::Identity::collect(targets => { $T => 1 });
+ is_deeply(
+ [
+ map { [$_->@{qw(lun paths unresolved)}] }
+ PVE::Multipath::Identity::target_luns($ids, $T)->@*
+ ],
+ [[6, ['sdq'], 1]],
+ 'shared: the path counts as an unresolved LUN of the target of its session',
+ );
+}
+
+subtest 'direct image consumers win target attribution' => sub {
+ ($devices, $maps) = ({ sd('sdb', 0, undef, daemon => '3W0') }, {});
+ for my $type ('iscsi', 'iscsidirect', 'zfs', 'vendor') {
+ $storage->mock(
+ config => sub {
+ return {
+ ids => {
+ a => { type => 'iscsi', target => $T, content => { none => 1 } },
+ z => {
+ type => $type,
+ target => $T,
+ content => { images => 1 },
+ disable => 1,
+ nodes => { elsewhere => 1 },
+ },
+ zz => { type => $type, target => $T, content => { images => 1 } },
+ },
+ };
+ },
+ );
+ is(scan()->{sdb}->{'used-by-storage'}, 'z', "$type image consumer, deterministic tie");
+ }
+ $devices = { nvme0n1 => { disk => { devpath => '/dev/nvme0n1', size => 1024 } } };
+ $diskmanage->mock(
+ get_nvme_path_status => sub {
+ return { nvme0n1 => { active => 1, paths => [{ device => '/dev/nvme0' }] } };
+ },
+ );
+ my $row = scan()->{nvme0n1};
+ is($row->{'paths-active'}, 1, 'active path count');
+ is($row->{'paths-total'}, 1, 'total path count');
+ is(ref($row->{'path-list'}), 'ARRAY', 'path list');
+};
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 38/76] api: scan: flag duplicate physical volumes on distinct LUNs
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (36 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 37/76] api: scan: san-luns: report the multipath state of each LUN Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 39/76] api: disks: recheck local users and retain wipe device references Thomas Lamprecht
` (37 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
A snapshot clone presented beside its original can carry the same LVM
physical volume identity. LVM refuses to activate or change a volume
group with such a duplicate, so flag the affected LUNs in the inventory.
Compare SCSI LUNs by their multipath WWID and NVMe namespaces by their
kernel identity, not the udev WWN that distinct LUNs of an array can
share. Devices without a trusted identity do not take part. Only
whole-device PVs recorded by udev are compared: this is a diagnostic
hint, not a safety check.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Storage/Scan.pm | 9 +++
src/PVE/Diskmanage.pm | 9 +++
src/PVE/Storage/SANScan.pm | 52 ++++++++++++-
src/test/run_san_lun_scan_tests.pl | 116 +++++++++++++++++++++++++++++
4 files changed, 185 insertions(+), 1 deletion(-)
diff --git a/src/PVE/API2/Storage/Scan.pm b/src/PVE/API2/Storage/Scan.pm
index 93c78ac2..f6263150 100644
--- a/src/PVE/API2/Storage/Scan.pm
+++ b/src/PVE/API2/Storage/Scan.pm
@@ -513,6 +513,15 @@ __PACKAGE__->register_method({
type => 'string',
optional => 1,
},
+ 'duplicate-pv' => {
+ description =>
+ "Another listed LUN has the same LVM physical volume UUID, as"
+ . " can happen with a snapshot clone. LVM refuses to activate or change a"
+ . " volume group with such a duplicate. The check compares only udev"
+ . " records for physical volumes on whole devices, so it may miss duplicates.",
+ type => 'boolean',
+ optional => 1,
+ },
'used-by-storage' => {
description => "The ID of an existing storage definition that uses this"
. " device, either through the LVM volume group on it or because it"
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index 0aa005d2..58a627a7 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -245,6 +245,15 @@ sub parse_wipefs_signatures {
return $probe->{signatures};
}
+# Read cached metadata only; never probe the device.
+sub udev_properties {
+ my ($dev) = @_;
+
+ my $devno = file_read_firstline("/sys/class/block/$dev/dev") // return {};
+ my $data = eval { file_get_contents("/run/udev/data/b$devno") } // return {};
+ return { $data =~ m/^E:([^=\n]+)=(.*)$/mg };
+}
+
sub is_iscsi_device {
my ($dev) = @_;
diff --git a/src/PVE/Storage/SANScan.pm b/src/PVE/Storage/SANScan.pm
index 415efdba..4015361b 100644
--- a/src/PVE/Storage/SANScan.pm
+++ b/src/PVE/Storage/SANScan.pm
@@ -2,6 +2,9 @@ package PVE::Storage::SANScan;
use v5.36;
+use Cwd qw(abs_path);
+use File::Basename qw(basename);
+
use PVE::Diskmanage;
use PVE::Multipath;
use PVE::Multipath::ClusterConfig;
@@ -10,6 +13,51 @@ use PVE::Multipath::Identity;
use PVE::Storage;
use PVE::Storage::ISCSIPlugin;
use PVE::Storage::LVMPlugin;
+use PVE::Tools qw(dir_glob_foreach file_read_firstline);
+
+# NVMe maps use their members' namespace identity, not multipathd's WWID. Follow current slave
+# links to avoid reused names.
+my sub lun_compare_key($entry, $disk) {
+ my $scsi_key = defined($entry->{wwid}) ? "scsi $entry->{wwid}" : undef;
+ if (!defined($disk->{slaves})) {
+ my ($namespace) = $entry->{devpath} =~ m|^/dev/(nvme\d+n\d+)$|;
+ return $scsi_key if !defined($namespace);
+ my $id = file_read_firstline("/sys/block/$namespace/wwid");
+ return defined($id) && length($id) ? "nvme $id" : undef;
+ }
+
+ my $dm = basename(abs_path($entry->{devpath}) // return undef);
+ my @members;
+ dir_glob_foreach(
+ "/sys/block/$dm/slaves",
+ '([a-z]\S*)',
+ sub($name, $match) { push @members, $name },
+ );
+ return undef if !@members;
+ return $scsi_key if !grep { !m/^sd/ } @members;
+ return undef if grep { !m/^nvme/ } @members;
+ my %ids = map { (file_read_firstline("/sys/block/$dm/slaves/$_/wwid") // '') => 1 } @members;
+ return undef if scalar(keys %ids) != 1 || exists($ids{''});
+ return 'nvme ' . (keys %ids)[0];
+}
+
+my sub flag_duplicate_pvs($records) {
+ my ($pv_of, $pv_luns) = ({}, {});
+ for my $record ($records->@*) {
+ my $key = $record->{identity} // next;
+ my $dev = abs_path($record->{row}->{devpath}) // next;
+ my $props = PVE::Diskmanage::udev_properties(basename($dev));
+ next if ($props->{ID_FS_TYPE} // '') ne 'LVM2_member';
+ # the rules for device-mapper devices only keep the encoded form
+ my $uuid = $props->{ID_FS_UUID_ENC} // $props->{ID_FS_UUID} // next;
+ $pv_of->{$record} = $uuid;
+ $pv_luns->{$uuid}->{$key} = 1;
+ }
+ for my $record ($records->@*) {
+ my $uuid = $pv_of->{$record} // next;
+ $record->{row}->{'duplicate-pv'} = 1 if scalar(keys $pv_luns->{$uuid}->%*) > 1;
+ }
+}
my sub collect_inventory($cfg, $node) {
# Collect identities before disks so path_wwid() rejects names reused during enumeration.
@@ -208,7 +256,8 @@ my sub build_lun_rows($cfg, $inventory) {
$entry->{multipath} = 'unmanaged';
}
}
- push $records->@*, { row => $entry, disk => $disk };
+ push $records->@*,
+ { row => $entry, disk => $disk, identity => lun_compare_key($entry, $disk) };
};
for my $dev (sort keys %$disks) {
my $disk = $disks->{$dev};
@@ -223,6 +272,7 @@ sub scan($cfg, $node) {
my $inventory = collect_inventory($cfg, $node);
my $records = build_lun_rows($cfg, $inventory);
+ flag_duplicate_pvs($records);
return [map { $_->{row} } $records->@*];
}
diff --git a/src/test/run_san_lun_scan_tests.pl b/src/test/run_san_lun_scan_tests.pl
index c9f130ff..1771a5c8 100755
--- a/src/test/run_san_lun_scan_tests.pl
+++ b/src/test/run_san_lun_scan_tests.pl
@@ -1,5 +1,10 @@
#!/usr/bin/perl
+# Unit test for the SAN LUN scan: the WWID it reports for a path without a map, from the LUN
+# identities of PVE::Multipath::Identity, and the duplicate-PV flag, which compares LUNs only under
+# an identity that is known, never under the fallback names of a device. The disk listing, udev,
+# the LUN identities and the multipath state are mocked.
+
use strict;
use warnings;
@@ -64,6 +69,32 @@ $diskmanage->mock(
};
},
get_nvme_path_status => sub { {} },
+ udev_properties => sub {
+ my ($dev) = @_;
+ my $pv = $dev =~ m/^dm-(.+)$/ ? $maps->{$1}->{pv} : ($devices->{$dev} // {})->{pv};
+ return defined($pv) ? { ID_FS_TYPE => 'LVM2_member', ID_FS_UUID_ENC => $pv } : {};
+ },
+);
+# a map /dev/mapper/<wwid> is the dm device dm-<wwid>, its current members are 'members', or else
+# 'slaves' as listed
+my $scan = Test::MockModule->new('PVE::Storage::SANScan');
+$scan->mock(
+ abs_path => sub { $_[0] =~ s|^/dev/mapper/|/dev/dm-|r },
+ dir_glob_foreach => sub {
+ my ($dir, $regex, $func) = @_;
+ my ($wwid) = $dir =~ m|^/sys/block/dm-(.+)/slaves$| or return;
+ my $map = $maps->{$wwid} // return;
+ for my $name (($map->{members} // $map->{slaves})->@*) {
+ my @matches = $name =~ m/^($regex)$/;
+ $func->(@matches) if @matches;
+ }
+ },
+ file_read_firstline => sub {
+ my ($file) = @_;
+ return $namespace_ids->{$1} if $file =~ m|^/sys/block/(\w+)/wwid$|;
+ return $namespace_ids->{$1} if $file =~ m|^/sys/block/dm-[^/]+/slaves/(\w+)/wwid$|;
+ return undef;
+ },
);
my $multipath = Test::MockModule->new('PVE::Multipath');
$multipath->mock(
@@ -150,6 +181,11 @@ my sub scan {
}
return { map { $_->{devpath} =~ s|^/dev/(mapper/)?||r => $_ } @$res };
}
+my sub flagged {
+ my ($res) = @_;
+ return [sort grep { $res->{$_}->{'duplicate-pv'} } keys %$res];
+}
+
# a map of LUN 0 over sdb and sdc, and sdd, a path of it without a map yet
($devices, $maps) = (
{
@@ -168,6 +204,7 @@ $reused->{sdd} = 1;
ok(!defined(scan()->{sdd}->{wwid}), 'wwid: none for a name that denotes another device by now');
is(scan()->{sdd}->{'wwid-unknown'}, 1, 'changed path exports identity uncertainty');
$reused = {};
+is_deeply(flagged($res), [], 'duplicate: not a map and a path of the same LUN');
$resolvable = 0;
$res = scan();
is($res->{sdd}->{wwid}, '3W0', 'wwid: also without the daemon, through the map of its LUN');
@@ -186,22 +223,101 @@ ok(
);
ok(!defined($res->{sdl}->{multipath}), 'and no multipath state either, as it is not known');
is($res->{sdl}->{'wwid-unknown'}, 1, 'daemon unavailable exports uncertainty');
+is_deeply(flagged($res), [], 'duplicate: and not flagged then');
$resolvable = 1;
+is_deeply(flagged(scan()), ['3W0', 'sdl'], 'duplicate: but once the daemon names it');
# two paths of LUN 1 without a map
($devices, $maps) =
({ sd('sde', 1, 'PV1', daemon => '3W1'), sd('sdf', 1, 'PV1', daemon => '3W1') }, {});
+is_deeply(flagged(scan()), [], 'duplicate: not two paths of one LUN');
$devices->{sdf}->{daemon} = '3OTHER';
$res = scan();
is($res->{$_}->{'wwid-unknown'}, 1, "disagreeing LUN path $_ is uncertain") for qw(sde sdf);
$devices->{sdf}->{daemon} = '3W1';
+# a clone, LUN 2, of the PV of the map of LUN 0
+($devices, $maps) = (
+ { sd('sdb', 0, undef, map => '3W0'), sd('sdg', 2, 'PV0', daemon => '3W2') },
+ { '3W0' => { slaves => ['sdb'], pv => 'PV0' } },
+);
+is_deeply(flagged(scan()), ['3W0', 'sdg'], 'duplicate: a clone and its original, both flagged');
+
+# LUNs 3 and 4 share the truncated udev WWN of their array
+($devices, $maps) = (
+ {
+ sd('sdh', 3, 'PV3', daemon => '3W3', wwn => '0x600a098038323553'),
+ sd('sdi', 4, 'PV4', daemon => '3W4', wwn => '0x600a098038323553'),
+ },
+ {},
+);
+is_deeply(flagged(scan()), [], 'duplicate: not distinct PVs on LUNs sharing a WWN');
+$devices->{sdi}->{pv} = 'PV3';
+is_deeply(flagged(scan()), ['sdh', 'sdi'], 'duplicate: one PV on two LUNs sharing a WWN');
+
# local disks without any identity
($devices, $maps) = ({ sd('sdj', undef, 'PV5'), sd('sdk', undef, 'PV5') }, {});
+is_deeply(flagged(scan()), [], 'duplicate: not devices without an identity, whatever their WWN');
ok(!scan()->{sdj}->{'wwid-unknown'}, 'unlisted path is not marked uncertain');
$devices->{sdj}->{daemon} = '';
is(scan()->{sdj}->{'wwid-unknown'}, 1, 'listed path without identity is uncertain');
+# NVMe namespaces, and maps over NVMe paths
+($devices, $maps) = (
+ {
+ nvme0n1 =>
+ { disk => { devpath => '/dev/nvme0n1', size => 1024, used => 'LVM' }, pv => 'PV6' },
+ nvme1n1 =>
+ { disk => { devpath => '/dev/nvme1n1', size => 1024, used => 'LVM' }, pv => 'PV6' },
+ },
+ {},
+);
+$namespace_ids = { nvme0n1 => 'eui.0001', nvme1n1 => 'eui.0002' };
+is_deeply(flagged(scan()), ['nvme0n1', 'nvme1n1'], 'duplicate: two namespaces with one PV');
+$namespace_ids->{nvme1n1} = 'eui.0001';
+is_deeply(flagged(scan()), [], 'duplicate: not two heads of one namespace');
+
+$devices =
+ { nvme4n1 =>
+ { disk => { devpath => '/dev/nvme4n1', size => 1024, used => 'LVM' }, pv => 'PV7' } };
+$maps = { CUSTOM => { slaves => ['nvme2n1', 'nvme3n1'], pv => 'PV7' } };
+$namespace_ids = { nvme2n1 => 'uuid.a', nvme3n1 => 'uuid.a', nvme4n1 => 'uuid.a' };
+is_deeply(
+ flagged(scan()),
+ [],
+ 'duplicate: not a map over NVMe paths that multipathd names its own way, and a path of it',
+);
+$namespace_ids->{nvme3n1} = 'uuid.b';
+$namespace_ids->{nvme4n1} = 'uuid.c';
+is_deeply(flagged(scan()), [], 'duplicate: not a map whose paths disagree');
+$namespace_ids->{nvme3n1} = 'uuid.a';
+is_deeply(flagged(scan()), ['CUSTOM', 'nvme4n1'], 'duplicate: a map over NVMe paths and a clone');
+
+# a map that has lost all its members, next to a namespace with the same PV
+$devices =
+ { nvme5n1 =>
+ { disk => { devpath => '/dev/nvme5n1', size => 1024, used => 'LVM' }, pv => 'PV8' } };
+$maps = { EMPTY => { slaves => ['nvme6n1'], members => [], pv => 'PV8' } };
+$namespace_ids = { nvme5n1 => 'uuid.e', nvme6n1 => 'uuid.e' };
+is_deeply(flagged(scan()), [], 'duplicate: not a map without members, whatever the listing says');
+
+# a member without an identity, and members of two transports
+$maps = { NOID => { slaves => ['nvme6n1', 'nvme7n1'], pv => 'PV8' } };
+is_deeply(flagged(scan()), [], 'duplicate: not a map with a member without an identity');
+$maps = { MIXED => { slaves => ['nvme6n1', 'sdz'], pv => 'PV8' } };
+is_deeply(flagged(scan()), [], 'duplicate: not a map with members of two transports');
+
+# a namespace identity longer than multipathd keeps
+my $long = 'nvme.1af4-' . ('4e4554415050' x 20);
+$devices =
+ { nvme8n1 =>
+ { disk => { devpath => '/dev/nvme8n1', size => 1024, used => 'LVM' }, pv => 'PV9' } };
+$maps = { SHORTENED => { slaves => ['nvme9n1'], pv => 'PV9' } };
+$namespace_ids = { nvme8n1 => $long, nvme9n1 => $long };
+is_deeply(flagged(scan()), [], 'duplicate: a map and a path of one namespace with a long identity');
+$namespace_ids->{nvme8n1} = "${long}0";
+is_deeply(flagged(scan()), ['SHORTENED', 'nvme8n1'], 'duplicate: and two namespaces');
+
# the real collection and derivation on a fake sysfs: a failing daemon query runs only once, and
# both the scan and the storage derivation do without its data
{
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 39/76] api: disks: recheck local users and retain wipe device references
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (37 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 38/76] api: scan: flag duplicate physical volumes on distinct LUNs Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 40/76] iscsi: name plain LUNs after their identity link Thomas Lamprecht
` (36 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Checking mounts and holders before queueing a wipe misses raw device
users and changes while the task waits. Recheck local users under the
disk lock in the worker, refusing the wipe if a probe fails.
Recheck device instance and partition topology before destructive
commands. Retain open device references so reused kernel names cannot
redirect writes. Local wipes remain independent of quorum.
Share reference retention with SAN volume-group creation: keep its
descriptor through completion, clear sectors through it, and verify the
device instance after probes and before each LVM write.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Disks.pm | 9 +-
src/PVE/API2/Disks/LVM.pm | 5 +
src/PVE/Diskmanage.pm | 185 +++++++++++++++++++++++++--
src/PVE/Storage/LVMPlugin.pm | 17 ++-
src/test/Makefile | 5 +-
src/test/disk_api_test.pm | 3 +-
src/test/run_san_lvm_create_tests.pl | 34 ++++-
src/test/run_wipe_partition_tests.pl | 122 ++++++++++++++++++
src/test/run_wipe_reference_tests.pl | 123 ++++++++++++++++++
src/test/run_wipedisk_tests.pl | 64 +++++++++
10 files changed, 545 insertions(+), 22 deletions(-)
create mode 100755 src/test/run_wipe_partition_tests.pl
create mode 100755 src/test/run_wipe_reference_tests.pl
create mode 100755 src/test/run_wipedisk_tests.pl
diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index 58a89c67..ebc73103 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -329,7 +329,9 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- my $disk = PVE::Diskmanage::verify_blockdev_path($param->{disk});
+ my $reference = $param->{disk};
+ my $expected = PVE::Diskmanage::block_device_reference($reference);
+ my $disk = $expected->{device};
my $mounted = PVE::Diskmanage::is_mounted($disk);
die "disk/partition '${mounted}' is mounted\n" if $mounted;
@@ -341,8 +343,9 @@ __PACKAGE__->register_method({
my $authuser = $rpcenv->get_user();
my $worker = sub {
- PVE::Diskmanage::wipe_blockdev($disk);
- PVE::Diskmanage::udevadm_trigger($disk);
+ PVE::Diskmanage::locked_disk_action(sub {
+ PVE::Diskmanage::wipe_blockdev_checked($reference, $expected);
+ });
};
my $basename = basename($disk); # avoid '/' in the ID
diff --git a/src/PVE/API2/Disks/LVM.pm b/src/PVE/API2/Disks/LVM.pm
index feab0a3b..e2cc8118 100644
--- a/src/PVE/API2/Disks/LVM.pm
+++ b/src/PVE/API2/Disks/LVM.pm
@@ -160,6 +160,8 @@ __PACKAGE__->register_method({
|| PVE::Diskmanage::is_san_device($device);
};
my $san = $is_san->($dev);
+ my $expected = $san ? PVE::Diskmanage::block_device_reference($reference) : undef;
+ $dev = $expected->{device} if $san;
PVE::Diskmanage::assert_disk_unused($dev, 1);
@@ -187,6 +189,8 @@ __PACKAGE__->register_method({
my $create = sub {
if ($san) {
$dev = PVE::Diskmanage::verify_blockdev_path($reference);
+ $dev = PVE::Diskmanage::block_device_reference($reference, $expected)
+ ->{device};
}
die "cannot use '$dev' - device topology changed\n"
if !$san && $is_san->($dev);
@@ -202,6 +206,7 @@ __PACKAGE__->register_method({
my $guard = {
reference => $san ? $reference : $dev,
san => $san,
+ expected => $expected,
};
PVE::Storage::LVMPlugin::lvm_create_volume_group(
$guard->{reference},
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index 58a627a7..d0e1dd0d 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -7,7 +7,7 @@ use PVE::ProcFSTools;
use Data::Dumper;
use Cwd qw(abs_path);
use Errno qw(ENOENT ESRCH);
-use Fcntl ':mode';
+use Fcntl qw(:mode O_RDWR O_NONBLOCK);
use File::Basename;
use File::stat;
use JSON;
@@ -70,6 +70,92 @@ sub block_device_number {
return PVE::Tools::dev_t_major($st[6]) . ':' . PVE::Tools::dev_t_minor($st[6]);
}
+# The sysfs inode distinguishes device instances even when the kernel reuses a name and dev_t.
+sub block_device_stamp {
+ my ($device, $sys) = @_;
+ $sys //= '/sys';
+ my $dir = "$sys/class/block/" . basename($device);
+ my $ino = (CORE::stat($dir))[1] // die "cannot inspect '$device' - missing sysfs device\n";
+ my $number = file_read_firstline("$dir/dev");
+ die "cannot inspect '$device' - invalid device number\n"
+ if !defined($number) || $number !~ m/^\d+:\d+$/;
+ return "$ino:$number";
+}
+
+# Retain the device instance before queueing a worker; kernel names can be reused while it waits.
+sub block_device_reference {
+ my ($reference, $expected, $sys) = @_;
+ $sys //= '/sys';
+ my $device = verify_blockdev_path($reference);
+ my $stamp = block_device_stamp($device, $sys);
+ die "cannot use '$reference' - device identity changed\n"
+ if $expected && $expected->{stamp} ne $stamp;
+ return { device => $device, stamp => $stamp };
+}
+
+sub assert_block_device_instance {
+ my ($reference, $current) = @_;
+
+ my $now = block_device_reference($reference, $current);
+ die "cannot use '$current->{device}' - device topology changed\n"
+ if $now->{device} ne $current->{device} || $now->{stamp} ne $current->{stamp};
+ return;
+}
+
+# Retain the checked device instance even if its kernel name or udev alias is reused.
+sub open_block_device {
+ my ($device, $stamp) = @_;
+ sysopen(my $fh, $device, O_RDWR | O_NONBLOCK)
+ or die "cannot open '$device' - $!\n";
+ my $number = block_device_number($fh) // '';
+ die "cannot use '$device' - device changed while opening it\n"
+ if block_device_stamp($device) ne $stamp || $stamp !~ m/:\Q$number\E$/;
+ return $fh;
+}
+
+# Untaint sysfs names before passing them to commands in taint-mode daemons.
+my sub block_entries {
+ my ($dir) = @_;
+ opendir(my $dh, $dir) or die "unable to list '$dir' - $!\n";
+ my @names = sort grep { $_ ne '.' && $_ ne '..' } readdir($dh);
+ closedir($dh) or die "unable to close '$dir' - $!\n";
+ return [map { m/^([\w.:+-]+)$/ ? $1 : die "unexpected entry '$_' in '$dir'\n" } @names];
+}
+
+sub block_device_partitions {
+ my ($name, $sys) = @_;
+ $sys //= '/sys';
+ my $dir = "$sys/class/block/$name";
+ my @parts;
+ for my $part (block_entries($dir)->@*) {
+ next if !-e "$dir/$part/partition";
+ my $number = file_read_firstline("$dir/$part/partition");
+ die "invalid partition number for '$part'\n"
+ if !defined($number) || $number !~ m/^[1-9]\d*$/;
+ push @parts, $part;
+ }
+ return \@parts;
+}
+
+# Postorder keeps child partitions accessible until their signatures have been erased.
+sub wipe_partitions {
+ my ($device, $sys) = @_;
+ $sys //= '/sys';
+ my (@parts, %seen);
+ my $visit;
+ $visit = sub {
+ my ($name) = @_;
+ die "cannot inspect '$device' - cyclic partition topology\n" if $seen{$name}++;
+ for my $part (block_device_partitions($name, $sys)->@*) {
+ $visit->($part);
+ push @parts, $part;
+ }
+ };
+ $visit->(basename($device));
+ undef $visit;
+ return \@parts;
+}
+
# Never read device data. Probe failures are not evidence of non-use. Excluding a holder must not
# exclude direct users; multipathd can be exempted because its checkers keep member paths open.
sub block_device_users {
@@ -1378,11 +1464,75 @@ sub change_parttype {
run_command(['sgdisk', "-t${partnum}:${parttype}", $blockdev], errmsg => $err);
}
+# Caller holds the disk lock through verification, wiping and the device trigger.
+sub wipe_blockdev_checked {
+ my ($reference, $expected) = @_;
+
+ my $current = block_device_reference($reference, $expected);
+ my $disk = $current->{device};
+ my $partitions = wipe_partitions($disk);
+ my $children = { map { $_ => block_device_reference("/dev/$_") } $partitions->@* };
+ my $verify = sub {
+ my ($removed) = @_;
+ $removed //= {};
+ assert_block_device_instance($reference, $current);
+ my @remaining = grep { !$removed->{$_} } $partitions->@*;
+ my $parts = wipe_partitions($disk);
+ die "cannot wipe '$reference' - partition topology changed\n"
+ if join(',', @remaining) ne join(',', $parts->@*);
+ for my $part (@remaining) {
+ my $now = block_device_reference(
+ "/dev/$part", $children->{$part},
+ );
+ die "cannot wipe '$reference' - partition '$part' changed\n"
+ if $now->{stamp} ne $children->{$part}->{stamp};
+ }
+ };
+ $verify->();
+ my $handle = open_block_device($disk, $current->{stamp});
+ my $users = block_device_users(
+ [basename($disk)], {},
+ );
+ die "cannot wipe '$disk' - " . join(', ', $users->@*) . "\n" if $users->@*;
+ $verify->();
+ wipe_blockdev(
+ $disk,
+ $partitions,
+ { handle => $handle, verify => $verify, children => $children },
+ );
+ udevadm_trigger(
+ $disk, map { "/dev/$_" } grep { -e "/sys/class/block/$_" } $partitions->@*,
+ );
+}
+
# Wipes all labels and the first 200 MiB of a disk/partition (or the whole if it is smaller).
# If called with a partition, also sets the partition type to 0x83 'Linux filesystem'.
# Expected to be called with a result of verify_blockdev_path().
sub wipe_blockdev {
- my ($devpath) = @_;
+ my ($devpath, $partitions, $options) = @_;
+ $options //= {};
+ my $removed = {};
+ my $verify = sub { $options->{verify}->($removed) if $options->{verify} };
+ my $target = $options->{handle} ? "/proc/$$/fd/" . fileno($options->{handle}) : $devpath;
+
+ if ($partitions) {
+ my $current = wipe_partitions($devpath);
+ die "cannot wipe '$devpath' - partition topology changed\n"
+ if join(',', $current->@*) ne join(',', $partitions->@*);
+ my $users = block_device_users([basename($devpath)], {});
+ die "cannot wipe '$devpath' - " . join(', ', $users->@*) . "\n" if $users->@*;
+ for my $part ($partitions->@*) {
+ $verify->();
+ my $child = $options->{children}->{$part};
+ my $handle = $child ? open_block_device("/dev/$part", $child->{stamp}) : undef;
+ my $path = $handle ? "/proc/$$/fd/" . fileno($handle) : "/dev/$part";
+ run_command(['wipefs', '--all', $path], errmsg => "error wiping '/dev/$part'");
+ close($handle) or die "error closing '/dev/$part' - $!\n" if $handle;
+ }
+ # New holders make the parent unsafe even after its child signatures have been erased.
+ $users = block_device_users([basename($devpath)], {});
+ die "cannot wipe '$devpath' - " . join(', ', $users->@*) . "\n" if $users->@*;
+ }
my $devname = basename($devpath);
my $dev_size = PVE::Tools::file_get_contents("/sys/class/block/$devname/size");
@@ -1394,29 +1544,40 @@ sub wipe_blockdev {
my $count = ($size < 200) ? $size : 200;
my $to_wipe = [];
- dir_glob_foreach(
- "/sys/class/block/${devname}",
- "${devname}.+",
- sub {
- my ($part) = @_;
- push $to_wipe->@*, "/dev/${part}" if -b "/dev/${part}";
- },
- );
+ if (!$partitions) {
+ dir_glob_foreach(
+ "/sys/class/block/${devname}",
+ "${devname}.+",
+ sub {
+ my ($part) = @_;
+ push $to_wipe->@*, "/dev/${part}" if -b "/dev/${part}";
+ },
+ );
+ }
if (scalar($to_wipe->@*) > 0) {
print "found child partitions to wipe: " . join(', ', $to_wipe->@*) . "\n";
}
- push $to_wipe->@*, $devpath; # put actual device last
+ push $to_wipe->@*, $target; # put actual device last
print "wiping block device ${devpath}\n";
+ $verify->();
run_command(['wipefs', '--all', $to_wipe->@*], errmsg => "error wiping '${devpath}'");
+ if ($partitions) {
+ # Erasing the partition table can remove kernel partitions through BLKRRPART, as expected
+ # here; changed or newly appearing devices still fail verification.
+ my %remaining = map { $_ => 1 } wipe_partitions($devpath)->@*;
+ $removed->{$_} = 1 for grep { !/^dm-\d+$/ && !$remaining{$_} } $partitions->@*;
+ }
+ $verify->();
run_command(
- ['dd', 'if=/dev/zero', "of=${devpath}", 'bs=1M', 'conv=fdatasync', "count=${count}"],
+ ['dd', 'if=/dev/zero', "of=${target}", 'bs=1M', 'conv=fdatasync', "count=${count}"],
errmsg => "error wiping '${devpath}'",
);
+ $verify->();
if (is_partition($devpath)) {
eval { change_parttype($devpath, '8300'); };
warn $@ if $@;
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index ed7cdb8c..5ba56519 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -154,7 +154,18 @@ sub lvm_create_volume_group {
$device = PVE::Diskmanage::verify_blockdev_path($requested_device);
my $san = $guard && $guard->{san};
if ($guard) {
- if (!$san) {
+ if ($san) {
+ my $current = $guard->{current} = PVE::Diskmanage::block_device_reference(
+ $requested_device, $guard->{expected},
+ );
+ $device = $current->{device};
+ $guard->{handle} = PVE::Diskmanage::open_block_device($device, $current->{stamp});
+ $guard->{verify} = sub {
+ PVE::Diskmanage::assert_block_device_instance($requested_device, $current);
+ PVE::Diskmanage::assert_block_device_instance($_, $current)
+ for grep { defined } @_;
+ };
+ } else {
$guard->{verify} = sub {
die "cannot use '$device' - device topology changed\n"
if PVE::Diskmanage::is_multipath_device($device)
@@ -163,7 +174,7 @@ sub lvm_create_volume_group {
$guard->{verify}->();
}
}
- my $check_instance = sub { $guard->{verify}->() if $guard && $guard->{verify} };
+ my $check_instance = sub { $guard->{verify}->() if $guard };
my $check_safe = sub {
$check_instance->();
if ($san) {
@@ -248,7 +259,7 @@ sub lvm_create_volume_group {
$refuse_data->(keys %signatures) if %signatures;
$check_safe->();
- clear_first_sector($device);
+ clear_first_sector($san ? '/proc/' . $$ . '/fd/' . fileno($guard->{handle}) : $device);
# we use --metadatasize 250k, which reseults in "pe_start = 512"
# so pe_start is aligned on a 128k boundary (advantage for SSDs)
diff --git a/src/test/Makefile b/src/test/Makefile
index 91c685e7..3ae2639f 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -37,8 +37,11 @@ test_disklist: run_disk_tests.pl run_block_device_users_tests.pl
./run_disk_tests.pl
./run_block_device_users_tests.pl
-test_disk_api: run_disk_api_tests.pl
+test_disk_api: run_disk_api_tests.pl run_wipedisk_tests.pl run_wipe_reference_tests.pl run_wipe_partition_tests.pl
./run_disk_api_tests.pl
+ ./run_wipedisk_tests.pl
+ ./run_wipe_reference_tests.pl
+ ./run_wipe_partition_tests.pl
test_bwlimit: run_bwlimit_tests.pl
./run_bwlimit_tests.pl
diff --git a/src/test/disk_api_test.pm b/src/test/disk_api_test.pm
index d1fc6465..2371a953 100644
--- a/src/test/disk_api_test.pm
+++ b/src/test/disk_api_test.pm
@@ -131,6 +131,7 @@ subtest 'a volume group on a map is created under the storage configuration lock
$storage->mock(lock_storage_config => sub { $lock->('storage.cfg', $_[0]) });
$disk->mock(locked_disk_action => sub { $lock->('disk', $_[0]) });
$disk->mock(verify_blockdev_path => sub { $_[0] });
+ $disk->mock(block_device_reference => sub { { device => $_[0] } });
$disk->mock(assert_disk_unused => sub { });
$disk->mock(is_partition => sub { 0 });
$disk->mock(udevadm_trigger => sub { });
@@ -164,7 +165,7 @@ subtest 'a volume group on a map is created under the storage configuration lock
);
$disk->unmock($_)
for qw(
- locked_disk_action verify_blockdev_path assert_disk_unused is_partition
+ locked_disk_action verify_blockdev_path block_device_reference assert_disk_unused is_partition
udevadm_trigger is_multipath_device is_san_device
);
};
diff --git a/src/test/run_san_lvm_create_tests.pl b/src/test/run_san_lvm_create_tests.pl
index 200fd7cb..8d6bbfdc 100755
--- a/src/test/run_san_lvm_create_tests.pl
+++ b/src/test/run_san_lvm_create_tests.pl
@@ -7,6 +7,7 @@ use File::Basename qw(basename);
use File::Path qw(make_path);
use File::Temp qw(tempdir);
use JSON qw(encode_json);
+use Scalar::Util qw(weaken);
use Storable qw(dclone);
use Test::MockModule;
use Test::More;
@@ -22,8 +23,10 @@ my $rpc = Test::MockModule->new('PVE::RPCEnvironment');
my $inotify = Test::MockModule->new('PVE::INotify');
my $tools = Test::MockModule->new('PVE::Tools');
my $glob = \&PVE::Tools::dir_glob_foreach;
+my $reference = \&PVE::Diskmanage::block_device_reference;
+my $stamp = \&PVE::Diskmanage::block_device_stamp;
my $users = \&PVE::Diskmanage::block_device_users;
-my ($root, $resolved, $worker, $cfg, $local, $pv, $during_probe, $inventory_used);
+my ($root, $resolved, $worker, $cfg, $handle, $local, $pv, $during_probe, $inventory_used);
my (@locks, @writes);
my $uuid = 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg';
my $canonical = '/dev/mapper/lun';
@@ -46,6 +49,8 @@ my sub locked {
}
my sub written {
my ($tool, $path) = @_;
+ ok($handle && defined(fileno($handle)), 'production retains SAN handle through write')
+ if !$local;
push @writes, [$tool, $path];
}
{
@@ -70,6 +75,14 @@ $disk->mock(
return PVE::Tools::file_read_firstline(
"$root/sys/class/block/" . basename($path) . '/dev');
},
+ block_device_reference => sub { $reference->($_[0], $_[1], "$root/sys") },
+ block_device_stamp => sub { $stamp->($_[0], "$root/sys") },
+ open_block_device => sub {
+ open(my $opened, '+<', "$root/handle") or die $!;
+ $handle = $opened;
+ weaken($handle);
+ return $opened;
+ },
is_mounted => sub { undef },
has_holder => sub { undef },
assert_disk_unused => sub { die "device '$_[0]' is already in use\n" if $inventory_used },
@@ -140,8 +153,9 @@ my sub reset_case {
put('sys/class/block/dm-900/slaves/sda/device/state', 'running');
put('proc/self/mountinfo', '1 0 0:1 / / rw - rootfs rootfs rw');
put('proc/swaps', "Filename\tType\tSize\tUsed\tPriority");
+ put('handle', 'retained descriptor fixture');
($resolved, $local, $pv) = ('/dev/dm-900', 0, 'blank');
- ($during_probe, $worker) = (undef, undef);
+ ($during_probe, $worker, $handle) = (undef, undef, undef);
$cfg =
{ ids => { base => { type => 'iscsi', target => 'iqn.test', content => { none => 1 } } } };
@writes = ();
@@ -227,4 +241,20 @@ eval { $worker->() };
like($@, qr/cannot use.*device topology changed/, 'queued local-to-SAN transition refuses');
is_deeply(\@writes, [], 'transition does not bypass storage lock');
+for my $route ('disk', 'base') {
+ reset_case();
+ $during_probe = sub { put('sys/class/block/dm-900/dev', '253:999') };
+ like(run_route($route), qr/device identity changed/, "$route changed instance refuses");
+ is_deeply(\@writes, [], 'replacement after probing receives no destructive command');
+ reset_case();
+ is(run_route($route), '', "$route initializes unchanged instance");
+ like($writes[0]->[1], qr{^/proc/\d+/fd/\d+$}, 'sector clear uses retained descriptor');
+ ok(!defined($handle), 'production releases the retained handle after completion');
+}
+reset_case();
+queue();
+put('sys/class/block/dm-900/dev', '253:999');
+eval { $worker->() };
+like($@, qr/device identity changed/, 'queued replacement refuses');
+is_deeply(\@writes, [], 'queued replacement receives no destructive command');
done_testing();
diff --git a/src/test/run_wipe_partition_tests.pl b/src/test/run_wipe_partition_tests.pl
new file mode 100755
index 00000000..29ba74a2
--- /dev/null
+++ b/src/test/run_wipe_partition_tests.pl
@@ -0,0 +1,122 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use File::Path qw(make_path remove_tree);
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Diskmanage;
+
+my $root = tempdir(CLEANUP => 1);
+my $disk = Test::MockModule->new('PVE::Diskmanage');
+my $tools = Test::MockModule->new('PVE::Tools');
+my $users = \&PVE::Diskmanage::block_device_users;
+my $parts = \&PVE::Diskmanage::wipe_partitions;
+my (@commands, $drop_kernel);
+my sub put {
+ my ($path, $value) = @_;
+ my ($dir) = "$root/$path" =~ m{^(.*)/[^/]+$};
+ make_path($dir);
+ open(my $fh, '>', "$root/$path") or die $!;
+ print $fh "$value\n";
+ close($fh) or die $!;
+}
+my sub device {
+ my ($name, $number, $uuid) = @_;
+ put("sys/class/block/$name/dev", $number);
+ put("sys/class/block/$name/dm/uuid", $uuid);
+ make_path("$root/sys/class/block/$name/holders");
+}
+my sub reset_case {
+ remove_tree("$root/sys", "$root/proc");
+ device('sda', '8:0', '');
+ device('sda1', '8:1', '');
+ put('sys/class/block/sda1/partition', '1');
+ symlink('../sda1', "$root/sys/class/block/sda/sda1") or die $!;
+ put('proc/self/mountinfo', '1 0 0:1 / / rw - rootfs rootfs rw');
+ put('proc/swaps', "Filename\tType\tSize\tUsed\tPriority");
+ @commands = ();
+ $drop_kernel = 0;
+}
+$disk->mock(
+ wipe_partitions => sub { $parts->($_[0], "$root/sys") },
+ block_device_users => sub {
+ $users->($_[0], { %{ $_[1] // {} }, sysfs => "$root/sys", procfs => "$root/proc" });
+ },
+ block_device_number => sub { '8:1' },
+ is_partition => sub { 0 },
+ dir_glob_foreach => sub { },
+ run_command => sub {
+ my ($cmd) = @_;
+ push @commands, $cmd;
+ if ($drop_kernel && $cmd->[0] eq 'wipefs' && $cmd->[-1] eq '/dev/sda') {
+ unlink("$root/sys/class/block/sda/sda1") or die $!;
+ }
+ },
+);
+$tools->mock(file_get_contents => sub { "8192\n" });
+reset_case();
+$drop_kernel = 1;
+my $removed_kernel;
+eval {
+ PVE::Diskmanage::wipe_blockdev(
+ '/dev/sda',
+ ['sda1'],
+ {
+ verify => sub {
+ $removed_kernel = $_[0]->{sda1};
+ my $expected = $removed_kernel ? [] : ['sda1'];
+ is_deeply(
+ PVE::Diskmanage::wipe_partitions('/dev/sda'),
+ $expected,
+ 'kernel partition topology matches deliberate removals',
+ );
+ },
+ },
+ );
+};
+is($@, '', 'BLKRRPART removal does not abort the parent wipe');
+ok($removed_kernel, 'kernel partition disappearance recorded after parent signature wipe');
+is($commands[-1]->[0], 'dd', 'zeroing still follows partition table erasure');
+
+reset_case();
+put('handle', 'not a device');
+open(my $parent, '<', "$root/handle") or die $!;
+my $parent_fd = fileno($parent);
+my $child_fd;
+$disk->mock(
+ open_block_device => sub {
+ open(my $fh, '<', "$root/handle") or die $!;
+ $child_fd = fileno($fh);
+ return $fh;
+ },
+);
+my @verified;
+PVE::Diskmanage::wipe_blockdev(
+ '/dev/sda',
+ ['sda1'],
+ {
+ handle => $parent,
+ children => { 'sda1' => { stamp => 'test', dm_uuid => 'part1-mpath-test' } },
+ verify => sub { push @verified, { %{ $_[0] } } },
+ },
+);
+is_deeply(
+ $commands[0],
+ ['wipefs', '--all', "/proc/$$/fd/$child_fd"],
+ 'child write uses retained descriptor',
+);
+is_deeply(
+ $commands[1],
+ ['wipefs', '--all', "/proc/$$/fd/$parent_fd"],
+ 'parent signatures use retained descriptor',
+);
+like($commands[2]->[2], qr{^of=/proc/$$/fd/$parent_fd$}, 'zeroing uses retained descriptor');
+ok(@verified >= 4, 'topology verified between destructive commands');
+is_deeply($verified[-1], {}, 'unchanged kernel partitions remain verified');
+
+done_testing();
diff --git a/src/test/run_wipe_reference_tests.pl b/src/test/run_wipe_reference_tests.pl
new file mode 100755
index 00000000..610cb691
--- /dev/null
+++ b/src/test/run_wipe_reference_tests.pl
@@ -0,0 +1,123 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use File::Path qw(make_path);
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::API2::Disks;
+
+my $root = tempdir(CLEANUP => 1);
+my $disk = Test::MockModule->new('PVE::Diskmanage');
+my $rpc = Test::MockModule->new('PVE::RPCEnvironment');
+my $reference = \&PVE::Diskmanage::block_device_reference;
+my ($resolved, $worker, $locked, $written);
+my sub put {
+ my ($path, $value) = @_;
+ my ($dir) = "$root/$path" =~ m{^(.*)/[^/]+$};
+ make_path($dir);
+ open(my $fh, '>', "$root/$path") or die $!;
+ print $fh "$value\n";
+ close($fh) or die $!;
+}
+put('class/block/sda/dev', '8:0');
+put('class/block/sdb/dev', '8:16');
+{
+
+ package ReferenceTestRPC;
+ sub get_user { 'root@pam' }
+ sub fork_worker { $worker = $_[4]; 'test-upid' }
+}
+$rpc->mock(get => sub { bless {}, 'ReferenceTestRPC' });
+$disk->mock(
+ verify_blockdev_path => sub { $resolved },
+ is_partition => sub { 0 },
+ block_device_reference => sub {
+ ok($locked, 're-resolution occurs after locks') if $_[1];
+ $reference->($_[0], $_[1], $root);
+ },
+ locked_disk_action => sub {
+ $locked = 1;
+ eval { $_[0]->() };
+ my $err = $@;
+ $locked = 0;
+ die $err if $err;
+ },
+ block_device_users => sub { [] },
+ is_mounted => sub { undef },
+ has_holder => sub { undef },
+ wipe_partitions => sub { [] },
+ open_block_device => sub { \*STDOUT },
+ wipe_blockdev => sub {
+ $_[2]->{verify}->();
+ ok($_[2]->{handle}, 'verified handle is retained through wiping');
+ $written = $_[0];
+ },
+ udevadm_trigger => sub { },
+);
+my $method = PVE::API2::Disks->map_method_by_name('wipe_disk')->{code};
+my sub queue {
+ my ($path) = @_;
+ $written = undef;
+ $resolved = '/dev/sda';
+ $method->({ node => 'test', disk => $path });
+}
+queue('/dev/sda');
+eval { $worker->() };
+is($@, '', 'delayed worker accepts the same device instance');
+is($written, '/dev/sda', 'worker uses the verified device');
+queue('/dev/sda');
+$resolved = '/dev/sdb';
+eval { $worker->() };
+like($@, qr/identity changed/, 'rebound reference refuses');
+ok(!defined($written), 'rebound reference does not write');
+
+$disk->unmock('open_block_device');
+put('handle', 'not a device');
+$disk->mock(block_device_stamp => sub { '1:8:0' }, block_device_number => sub { '8:16' });
+eval { PVE::Diskmanage::open_block_device("$root/handle", '1:8:0') };
+like($@, qr/device changed while opening/, 'opened device number must match verified snapshot');
+$disk->mock(block_device_number => sub { '8:0' });
+my $handle = PVE::Diskmanage::open_block_device("$root/handle", '1:8:0');
+ok(defined(fileno($handle)), 'matching opened reference retained');
+close($handle) or die $!;
+
+subtest 'checked wipe keeps verification order and retained handle' => sub {
+ my @events;
+ my $handle;
+ $disk->mock(
+ block_device_reference =>
+ sub { push @events, 'reference'; { device => '/dev/sda', stamp => 's' } },
+ wipe_partitions => sub { push @events, 'partitions'; [] },
+ assert_block_device_ready => sub { push @events, 'ready' },
+ open_block_device => sub {
+ push @events, 'open';
+ open($handle, '<', "$root/handle") or die $!;
+ return $handle;
+ },
+ block_device_users => sub { push @events, 'users'; [] },
+ wipe_blockdev => sub {
+ push @events, 'wipe';
+ is($_[2]->{handle}, $handle, 'wipe receives retained parent handle');
+ $_[2]->{verify}->();
+ },
+ udevadm_trigger => sub {
+ push @events, 'trigger';
+ ok(defined(fileno($handle)), 'parent handle stays open through trigger');
+ },
+ );
+ PVE::Diskmanage::wipe_blockdev_checked('/dev/sda', {});
+ is_deeply(
+ \@events,
+ [qw(reference partitions reference partitions open users
+ reference partitions wipe reference partitions trigger)],
+ 'retained open, users, verification, wipe and trigger order',
+ );
+ close($handle);
+};
+
+done_testing();
diff --git a/src/test/run_wipedisk_tests.pl b/src/test/run_wipedisk_tests.pl
new file mode 100755
index 00000000..f110a2f4
--- /dev/null
+++ b/src/test/run_wipedisk_tests.pl
@@ -0,0 +1,64 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::API2::Disks;
+
+my $disk = Test::MockModule->new('PVE::Diskmanage');
+my $rpc = Test::MockModule->new('PVE::RPCEnvironment');
+my ($worker, $users, $probe_error, $wiped);
+my $inside_worker = 0;
+{
+
+ package WipeTestRPC;
+ sub get_user { 'root@pam' }
+ sub fork_worker { $worker = $_[4]; return 'test-upid' }
+}
+$rpc->mock(get => sub { bless {}, 'WipeTestRPC' });
+$disk->mock(
+ verify_blockdev_path => sub { '/dev/sda' },
+ block_device_reference => sub {
+ return {
+ device => '/dev/sda',
+ stamp => '1:8:0',
+ key => 'scsi:test',
+ partition => '',
+ };
+ },
+ open_block_device => sub { \*STDOUT },
+ locked_disk_action => sub { $_[0]->() },
+ is_mounted => sub { undef },
+ has_holder => sub { undef },
+ wipe_partitions => sub { [] },
+ block_device_users => sub {
+ ok($inside_worker, 'users checked in the worker');
+ is_deeply($_[0], ['sda'], 'resolved name checked, including partitions');
+ die $probe_error if $probe_error;
+ return $users;
+ },
+ wipe_blockdev => sub { $wiped++ },
+ udevadm_trigger => sub { },
+);
+my $method = PVE::API2::Disks->map_method_by_name('wipe_disk');
+for my $case (
+ [[], undef, 1, qr/^$/],
+ [['qemu (PID 7) has sda open'], undef, 0, qr/cannot wipe.*qemu/],
+ [[], "probe failed\n", 0, qr/probe failed/],
+) {
+ ($users, $probe_error, my $expected, my $error) = @$case;
+ $wiped = 0;
+ $inside_worker = 0;
+ $method->{code}->({ node => 'test', disk => '/dev/sda' });
+ is($wiped, 0, 'request does not wipe before worker');
+ $inside_worker = 1;
+ eval { $worker->() };
+ like($@, $error, 'worker outcome');
+ is($wiped, $expected, 'only unused devices wiped');
+}
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 40/76] iscsi: name plain LUNs after their identity link
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (38 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 39/76] api: disks: recheck local users and retain wipe device references Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 41/76] api: disks: guard SAN wipes with stable identities and storage checks Thomas Lamprecht
` (35 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Storages without the multipath option name each LUN after the last of
its by-id links. udev can add aliases, such as SCSI name strings, that
sort after scsi-<WWID> but do not encode the LUN identity, so the
volume name alone does not tell which LUN it denotes. Use the
scsi-<WWID> link for such LUNs instead, so operations on the volume can
verify the LUN. The identities are only collected when a LUN has such
aliases.
Existing references to an alias name still resolve.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/Storage/ISCSIPlugin.pm | 15 ++++--
src/test/run_iscsi_device_list_tests.pl | 2 +-
src/test/run_lun_identity_tests.pl | 70 +++++++++++++++++++++++++
3 files changed, 83 insertions(+), 4 deletions(-)
diff --git a/src/PVE/Storage/ISCSIPlugin.pm b/src/PVE/Storage/ISCSIPlugin.pm
index ad3afaf6..cadbda8a 100644
--- a/src/PVE/Storage/ISCSIPlugin.pm
+++ b/src/PVE/Storage/ISCSIPlugin.pm
@@ -26,6 +26,7 @@ use base qw(PVE::Storage::Plugin);
my $ISCSIADM = '/usr/bin/iscsiadm';
our $INITIATOR_NAME_FILE = '/etc/iscsi/initiatorname.iscsi';
our $SYSFS = '/sys';
+our $BY_ID_DIR = '/dev/disk/by-id';
my $MULTIPATHD_EXECUTABLES = { '/usr/sbin/multipathd' => 1, '/sbin/multipathd' => 1 };
sub read_initiator_info {
@@ -437,7 +438,7 @@ sub load_stable_scsi_paths {
my $stable_paths = {};
- my $stabledir = "/dev/disk/by-id";
+ my $stabledir = $BY_ID_DIR;
if (my $dh = IO::Dir->new($stabledir)) {
foreach my $tmp (sort $dh->read) {
@@ -448,7 +449,7 @@ sub load_stable_scsi_paths {
my $path = "$stabledir/$tmp";
my $bdevdest = readlink($path);
if ($bdevdest && $bdevdest =~ m|^../../([^/]+)|) {
- $stable_paths->{$1} = $tmp;
+ push $stable_paths->{$1}->@*, $tmp;
}
}
}
@@ -721,7 +722,8 @@ sub iscsi_device_list {
$bdev = $multipathdev if $multipathdev;
}
- my $blockdev = $stable_paths->{$bdev};
+ my $names = $stable_paths->{$bdev} // [];
+ my $blockdev = $names->[-1];
# scsi-<wwid> can name a member before assembly; dm-uuid-mpath-<wwid> only names the map.
# Managed volumes keep the map name even while assembly is pending.
if ($map_names) {
@@ -730,6 +732,13 @@ sub iscsi_device_list {
? PVE::Multipath::Identity::path_wwid($ids, $path_dev)
: undef;
$blockdev = defined($wwid) ? "dm-uuid-mpath-$wwid" : undef;
+ } elsif ((grep { m/^scsi-/ } @$names) > 1) {
+ # udev can add aliases, such as SCSI name strings, that sort after scsi-<wwid> but do
+ # not encode the LUN identity; prefer the name that does, so operations on the volume
+ # can verify which LUN it denotes.
+ $ids //= PVE::Multipath::Identity::collect(all => 1);
+ my $wwid = PVE::Multipath::Identity::path_wwid($ids, $path_dev);
+ $blockdev = "scsi-$wwid" if defined($wwid) && grep { $_ eq "scsi-$wwid" } @$names;
}
next if !$blockdev;
diff --git a/src/test/run_iscsi_device_list_tests.pl b/src/test/run_iscsi_device_list_tests.pl
index 6df3a416..00ce2684 100755
--- a/src/test/run_iscsi_device_list_tests.pl
+++ b/src/test/run_iscsi_device_list_tests.pl
@@ -55,7 +55,7 @@ session(4, 9, 'iqn.target-d');
my $plugin = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
$plugin->mock(
load_stable_scsi_paths => sub {
- return { map { $_ => "scsi-3$_" } qw(sda sdb sdc sdd) };
+ return { map { $_ => ["scsi-3$_"] } qw(sda sdb sdc sdd) };
},
);
diff --git a/src/test/run_lun_identity_tests.pl b/src/test/run_lun_identity_tests.pl
index c5363e46..a49f27c4 100755
--- a/src/test/run_lun_identity_tests.pl
+++ b/src/test/run_lun_identity_tests.pl
@@ -597,6 +597,76 @@ is_deeply(
'volumes: every path of a resolved LUN named after its map, an unresolved LUN left out',
);
+{
+ # Without the option, LUNs are named after their by-id links, here in a scratch directory:
+ # { device => [lun, wwid, links] }
+ my $fixture = {
+ sdzzb => [0, '3W0', ['scsi-3W0', 'scsi-3W0-part1', 'scsi-8naa.W0']],
+ sdzzc => [1, '2E1', ['scsi-2E1', 'scsi-8eui.E1']],
+ sdzzd => [2, '3W2', ['scsi-3W2', 'scsi-SLIO-ORG_disk_W2']],
+ sdzze => [3, '3W3', ['scsi-3W3']],
+ sdzzf => [4, undef, ['scsi-3W4', 'scsi-8W4']],
+ sdzzg => [5, '3W5', ['scsi-8W5', 'scsi-SW5']],
+ };
+ my ($disks, $lines) = ({}, {});
+ for my $name (keys %$fixture) {
+ my ($lun, $wwid) = $fixture->{$name}->@*;
+ my $disk = { target => $T, lun => $lun, devt => "8:$lun", hctl => "9:0:0:$lun" };
+ $disks->{$name} = $disk;
+ $lines->{$name} = { $disk->%{qw(devt hctl target)}, init => 'ok', wwid => $wwid }
+ if defined($wwid);
+ }
+ my $plain = { luns => PVE::Multipath::Identity::resolve($disks, $lines, $naa_ok) };
+ my $by_id = tempdir(CLEANUP => 1);
+ my sub link_devices {
+ unlink(glob("$by_id/*"));
+ for my $name (@_) {
+ symlink("../../$name", "$by_id/$_") or die "$!\n" for $fixture->{$name}->[2]->@*;
+ }
+ }
+ local $PVE::Storage::ISCSIPlugin::BY_ID_DIR = $by_id;
+ my $collected = 0;
+ my $session_device_map = \&PVE::Storage::ISCSIPlugin::iscsi_session_device_map;
+ $identity->mock(collect => sub { $collected++; $plain });
+ $iscsi->unmock('load_stable_scsi_paths');
+ $iscsi->mock(
+ iscsi_session_device_map => sub {
+ return {
+ map {
+ $_ => { target => $T, channel => 0, id => 0, lun => $disks->{$_}->{lun} }
+ }
+ keys %$disks
+ };
+ },
+ );
+ my $list = sub { [sort keys PVE::Storage::ISCSIPlugin::iscsi_device_list(0, $T)->{$T}->%*] };
+ link_devices(keys %$fixture);
+ my $listed = $list->();
+ is_deeply(
+ $listed,
+ [
+ '0.0.0.scsi-3W0',
+ '0.0.1.scsi-2E1',
+ '0.0.2.scsi-3W2',
+ '0.0.3.scsi-3W3',
+ '0.0.4.scsi-8W4',
+ '0.0.5.scsi-SW5',
+ ],
+ 'plain volumes: the identity name over aliases, else the last name as before',
+ );
+ is($collected, 1, 'plain volumes: identities collected once');
+ link_devices('sdzze');
+ $collected = 0;
+ is_deeply($list->(), ['0.0.3.scsi-3W3'], 'plain volumes: a LUN without aliases');
+ is($collected, 0, 'plain volumes: needs no identities');
+
+ $iscsi->mock(
+ iscsi_session_device_map => $session_device_map,
+ load_stable_scsi_paths => sub { {} },
+ );
+ $identity->mock(collect => sub { $ids });
+}
+
my $collected = 0;
$identity->mock(collect => sub { $collected++; return { luns => {} } });
my $derived = PVE::Storage::derived_multipath_wwids({
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 41/76] api: disks: guard SAN wipes with stable identities and storage checks
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (39 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 40/76] iscsi: name plain LUNs after their identity link Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 42/76] api: scan: refresh SAN usage with bounded probes Thomas Lamprecht
` (34 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
A SAN LUN without local users may still belong to storage used on
another node. Before wiping, check storage references and PV identities
under the cluster storage configuration lock. Read PV membership without
LVM hints; missing volume-group metadata does not prove a PV is
orphaned.
Retain SAN identities across worker delays, require usable paths before
probes, and wipe same-LUN partition maps before removing them. Expose
usage hints and verified device references in the scan, while keeping
the worker checks authoritative. Local wipes remain independent of
quorum.
A path without an identity, such as a local disk that multipathd
ignores, counts as a possible path of the LUN: the write is refused
unless the cached VPD page of the path shows another logical unit.
Apply the same identity, readiness, and ownership checks before each
SAN volume-group initialization write, through both the disk API and
storage-add. Accept only whole LUNs and reject raw guest disks and
LUNs on targets used by storage for disk images.
pvcreate and vgcreate cannot use an open descriptor. Give them a
checked, identity-bearing by-id name, but a race remains before LVM
resolves and opens it. Cluster locks do not serialize hosts outside the
cluster.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Disks.pm | 35 +-
src/PVE/API2/Disks/LVM.pm | 7 +
src/PVE/API2/Storage/Scan.pm | 24 +
src/PVE/Diskmanage.pm | 390 ++++++++++++-
src/PVE/Multipath/Identity.pm | 119 +++-
src/PVE/Storage.pm | 219 ++++++++
src/PVE/Storage/LVMPlugin.pm | 27 +-
src/PVE/Storage/SANScan.pm | 7 +
src/test/Makefile | 15 +-
src/test/disk_api_test.pm | 1 +
src/test/run_block_device_users_tests.pl | 11 +
src/test/run_lun_identity_tests.pl | 214 +++++++
src/test/run_san_lun_scan_tests.pl | 26 +
src/test/run_san_lvm_create_tests.pl | 453 +++++++++++++--
src/test/run_san_wipe_tests.pl | 673 +++++++++++++++++++++++
src/test/run_test_lvm_base_volume.pl | 8 +-
src/test/run_wipe_partition_tests.pl | 297 +++++++++-
src/test/run_wipe_readiness_tests.pl | 138 +++++
src/test/run_wipe_reference_tests.pl | 168 +++++-
src/test/run_wipedisk_tests.pl | 58 +-
20 files changed, 2743 insertions(+), 147 deletions(-)
create mode 100755 src/test/run_san_wipe_tests.pl
create mode 100755 src/test/run_wipe_readiness_tests.pl
diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index ebc73103..37a2df31 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -321,7 +321,7 @@ __PACKAGE__->register_method({
disk => {
type => 'string',
description => "Block device name",
- pattern => '^/dev/[a-zA-Z0-9\/]+$',
+ pattern => '^/dev/[a-zA-Z0-9._:/-]+$',
},
},
},
@@ -333,18 +333,39 @@ __PACKAGE__->register_method({
my $expected = PVE::Diskmanage::block_device_reference($reference);
my $disk = $expected->{device};
- my $mounted = PVE::Diskmanage::is_mounted($disk);
- die "disk/partition '${mounted}' is mounted\n" if $mounted;
-
- my $held = PVE::Diskmanage::has_holder($disk);
- die "disk/partition '${held}' has a holder\n" if $held;
+ my $users = PVE::Diskmanage::block_device_users(
+ [basename($disk)],
+ { 'partition-maps' => 1, 'metadata-only' => 1 },
+ );
+ die "cannot wipe '$disk' - " . join(', ', $users->@*) . "\n" if $users->@*;
my $rpcenv = PVE::RPCEnvironment::get();
my $authuser = $rpcenv->get_user();
+ # A SAN LUN may be another storage's, which is checked against the cluster-wide storage
+ # configuration under its lock. A local disk is not, and its wipe must work without quorum.
+ my $is_san = sub {
+ my ($dev) = @_;
+ return PVE::Diskmanage::is_san_device($dev)
+ || PVE::Diskmanage::is_multipath_device($dev);
+ };
+ my $san = $is_san->($disk);
+
my $worker = sub {
PVE::Diskmanage::locked_disk_action(sub {
- PVE::Diskmanage::wipe_blockdev_checked($reference, $expected);
+ my $locked = $san ? \&PVE::Storage::lock_storage_config : sub { $_[0]->() };
+ $locked->(sub {
+ my $cfg = $san ? PVE::Storage::config() : undef;
+ my $check_storage = $san
+ ? sub {
+ my ($disk, $partitions) = @_;
+ PVE::Storage::assert_san_write_safe($cfg, $disk, $partitions);
+ }
+ : undef;
+ PVE::Diskmanage::wipe_blockdev_checked(
+ $reference, $expected, $check_storage,
+ );
+ });
});
};
diff --git a/src/PVE/API2/Disks/LVM.pm b/src/PVE/API2/Disks/LVM.pm
index e2cc8118..6129df56 100644
--- a/src/PVE/API2/Disks/LVM.pm
+++ b/src/PVE/API2/Disks/LVM.pm
@@ -160,6 +160,7 @@ __PACKAGE__->register_method({
|| PVE::Diskmanage::is_san_device($device);
};
my $san = $is_san->($dev);
+ PVE::Storage::LVMPlugin::assert_whole_lun($dev) if $san;
my $expected = $san ? PVE::Diskmanage::block_device_reference($reference) : undef;
$dev = $expected->{device} if $san;
@@ -189,6 +190,7 @@ __PACKAGE__->register_method({
my $create = sub {
if ($san) {
$dev = PVE::Diskmanage::verify_blockdev_path($reference);
+ PVE::Storage::LVMPlugin::assert_whole_lun($dev);
$dev = PVE::Diskmanage::block_device_reference($reference, $expected)
->{device};
}
@@ -208,6 +210,11 @@ __PACKAGE__->register_method({
san => $san,
expected => $expected,
};
+ if ($san) {
+ my $cfg = PVE::Storage::config();
+ $guard->{check_storage} =
+ sub { PVE::Storage::assert_san_write_safe($cfg, $_[0]) };
+ }
PVE::Storage::LVMPlugin::lvm_create_volume_group(
$guard->{reference},
$name,
diff --git a/src/PVE/API2/Storage/Scan.pm b/src/PVE/API2/Storage/Scan.pm
index f6263150..e7f6eba8 100644
--- a/src/PVE/API2/Storage/Scan.pm
+++ b/src/PVE/API2/Storage/Scan.pm
@@ -513,6 +513,30 @@ __PACKAGE__->register_method({
type => 'string',
optional => 1,
},
+ 'in-use' => {
+ description =>
+ "Local usage indicated by mounts and devices using this device,"
+ . " including its partitions and kpartx partition maps for the same LUN."
+ . " 'no' does not rule out open file descriptors, swap, or other uses"
+ . " checked before wiping. 'unknown' means the metadata check failed.",
+ type => 'string',
+ enum => ['yes', 'no', 'unknown'],
+ },
+ 'in-use-by' => {
+ description =>
+ "Local devices and mounts using this device, or why its usage"
+ . " could not be checked. Device mapper devices are identified by their"
+ . " map name when available.",
+ type => 'string',
+ },
+ 'by-id' => {
+ description =>
+ "A path under /dev/disk/by-id that encodes the device's identity"
+ . " and has been verified to resolve to this device. Absent when no such"
+ . " link could be verified.",
+ type => 'string',
+ optional => 1,
+ },
'duplicate-pv' => {
description =>
"Another listed LUN has the same LVM physical volume UUID, as"
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index d0e1dd0d..796d4eb4 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -70,6 +70,23 @@ sub block_device_number {
return PVE::Tools::dev_t_major($st[6]) . ':' . PVE::Tools::dev_t_minor($st[6]);
}
+# Canonical udev names that encode the kernel's LUN identity, unlike model/serial aliases.
+# Partition names carry the identity of the containing LUN as well as their partition number.
+sub block_identity_from_id {
+ my ($name) = @_;
+ if ($name =~ m/^dm-uuid-(?:part([1-9]\d*)-)?mpath-([a-zA-Z0-9_.:+-]+)$/) {
+ return { key => "scsi:$2", partition => $1 // '' };
+ }
+ if (
+ $name =~ m/^nvme-((?:eui\.(?:[0-9a-f]{16}|[0-9a-f]{32})
+ |uuid\.[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}
+ |nvme\.[0-9a-f]{4}-[0-9a-f]+-[0-9a-f]+-[0-9a-f]{8}))(?:-part([1-9]\d*))?$/x
+ ) {
+ return { key => "nvme:$1", partition => $2 // '' };
+ }
+ return undef;
+}
+
# The sysfs inode distinguishes device instances even when the kernel reuses a name and dev_t.
sub block_device_stamp {
my ($device, $sys) = @_;
@@ -82,15 +99,64 @@ sub block_device_stamp {
return "$ino:$number";
}
-# Retain the device instance before queueing a worker; kernel names can be reused while it waits.
+# Retain both the original reference and the identity it denotes before queueing a worker. Only
+# identities established from kernel metadata may survive renumbering while the worker waits.
sub block_device_reference {
my ($reference, $expected, $sys) = @_;
$sys //= '/sys';
my $device = verify_blockdev_path($reference);
my $stamp = block_device_stamp($device, $sys);
+ my $whole = is_partition($device) ? get_blockdev($device) : $device;
+ my $name = basename($whole);
+ my $dir = "$sys/class/block/$name";
+ my $partition =
+ $whole ne $device
+ ? file_read_firstline("$sys/class/block/" . basename($device) . '/partition')
+ : '';
+ my ($identity, $dm_uuid);
+
+ if ($name =~ m/^dm-\d+$/) {
+ my $uuid = file_read_firstline("$dir/dm/uuid")
+ // die "cannot use '$reference' - missing device mapper identity\n";
+ # passed on to dmsetup, the daemons run with taint checks
+ ($dm_uuid) = $uuid =~ m/^([\w.:+-]+)$/
+ or die "unable to identify '$device' - unexpected device mapper identity\n";
+ $identity = block_identity_from_id("dm-uuid-$uuid");
+ $identity //= { key => "dm:$uuid", partition => '' };
+ } elsif ($name =~ m/^nvme\d+n\d+$/) {
+ my $wwid = file_read_firstline("$dir/wwid")
+ // die "cannot use '$reference' - missing namespace identity\n";
+ $identity = { key => "nvme:$wwid", partition => $partition };
+ } elsif ($name =~ m/^sd[a-z]+$/) {
+ require PVE::Multipath::Identity;
+ my $ids = PVE::Multipath::Identity::collect(all => 1);
+ my $wwid = PVE::Multipath::Identity::path_wwid($ids, $name);
+ $identity = { key => "scsi:$wwid", partition => $partition } if defined($wwid);
+ }
+ $identity //= { key => "instance:$stamp", partition => $partition };
+ if ($reference =~ m{^/dev/disk/by-id/([^/]+)$}) {
+ my $id = $1;
+ my $encoded = block_identity_from_id($id);
+ if (!$encoded && $id =~ m/^scsi-(.+?)(?:-part([1-9]\d*))?$/) {
+ $encoded = { key => "scsi:$1", partition => $2 // '' };
+ } elsif (!$encoded && $id =~ m/^wwn-0x([0-9a-f]+)(?:-part([1-9]\d*))?$/) {
+ my ($wwn, $part) = ($1, $2 // '');
+ die "cannot use '$reference' - device identity does not match its by-id name\n"
+ if $identity->{key} !~ m/^scsi:[23]\Q$wwn\E$/;
+ $encoded = { key => $identity->{key}, partition => $part };
+ }
+ die "cannot use '$reference' - device identity does not match its by-id name\n"
+ if $encoded
+ && ($encoded->{key} ne $identity->{key}
+ || $encoded->{partition} ne $identity->{partition});
+ }
die "cannot use '$reference' - device identity changed\n"
- if $expected && $expected->{stamp} ne $stamp;
- return { device => $device, stamp => $stamp };
+ if $expected
+ && ($expected->{key} ne $identity->{key}
+ || $expected->{partition} ne $identity->{partition});
+ die "cannot use '$reference' - device changed while identifying it\n"
+ if block_device_stamp($device, $sys) ne $stamp;
+ return { %$identity, device => $device, stamp => $stamp, dm_uuid => $dm_uuid };
}
sub assert_block_device_instance {
@@ -102,6 +168,107 @@ sub assert_block_device_instance {
return;
}
+# A scan must not invent a stable name from a friendly mapper alias or an opaque serial string.
+# Return a canonical identity-bearing link only while it resolves to the device being reported.
+sub canonical_blockdev_link {
+ my ($device, $ids, $expected_wwid, $sys) = @_;
+ $sys //= '/sys';
+ return eval {
+ $device = verify_blockdev_path($device);
+ my $stamp = block_device_stamp($device, $sys);
+ my $whole = is_partition($device) ? get_blockdev($device) : $device;
+ my $name = basename($whole);
+ my $suffix = '';
+ if ($whole ne $device) {
+ my $part =
+ file_read_firstline("$sys/class/block/" . basename($device) . '/partition');
+ die "invalid partition number\n" if !defined($part) || $part !~ m/^[1-9]\d*$/;
+ $suffix = "-part$part";
+ }
+ my @links;
+ if ($name =~ m/^dm-\d+$/) {
+ my $uuid = file_read_firstline("$sys/class/block/$name/dm/uuid") // '';
+ if ($uuid =~ m/^(?:part[1-9]\d*-)?mpath-([a-zA-Z0-9_.:+-]+)$/) {
+ die "map identity changed\n" if defined($expected_wwid) && $1 ne $expected_wwid;
+ my $id = "dm-uuid-$uuid";
+ $id = "dm-uuid-part$1-$uuid" if $suffix =~ m/^-part([1-9]\d*)$/;
+ push @links, $id if block_identity_from_id($id);
+ }
+ } elsif ($name =~ m/^nvme\d+n\d+$/) {
+ my $wwid = file_read_firstline("$sys/class/block/$name/wwid") // '';
+ push @links, "nvme-$wwid$suffix" if block_identity_from_id("nvme-$wwid$suffix");
+ } elsif ($name =~ m/^sd[a-z]+$/) {
+ require PVE::Multipath::Identity;
+ $ids //= PVE::Multipath::Identity::collect(all => 1);
+ die "cannot inspect SAN identities\n" if $ids->{error};
+ my $wwid = PVE::Multipath::Identity::path_wwid($ids, $name);
+ if (defined($wwid) && $wwid =~ m/^[a-zA-Z0-9_.:+-]+$/) {
+ die "path identity changed\n"
+ if defined($expected_wwid) && $wwid ne $expected_wwid;
+ push @links, "scsi-$wwid$suffix";
+ push @links, "wwn-0x$1$suffix" if $wwid =~ m/^[23]([0-9a-f]+)$/;
+ }
+ }
+ my $link;
+ for my $id (@links) {
+ my ($path) = "/dev/disk/by-id/$id" =~ m{\A(/dev/disk/by-id/[a-zA-Z0-9_.:+-]+)\z};
+ next if !defined($path);
+ next if (eval { verify_blockdev_path($path) } // '') ne $device;
+ $link = $path;
+ last;
+ }
+ die "device changed\n" if block_device_stamp($device, $sys) ne $stamp;
+ $link;
+ };
+}
+
+# This is a metadata-only hint, not a certificate that the full destructive-operation checks pass.
+# With 'map-wwid', the device is the multipath map of that WWID and the users of its current paths
+# count as well, for example a logical volume that LVM activated on a path before the map existed.
+# A user found stays a user if a later check fails.
+sub block_device_usage_hint {
+ my ($device, $options) = @_;
+ my %options = (%{ $options // {} }, 'partition-maps' => 1, 'metadata-only' => 1);
+ my $wwid = delete $options{'map-wwid'};
+ my (@users, $error);
+ my $check = sub {
+ my ($code) = @_;
+ eval { $code->() };
+ chomp($error //= $@) if $@;
+ };
+ my $name;
+ $check->(sub {
+ $name = basename(verify_blockdev_path($device));
+ push @users, block_device_users([$name], \%options)->@*;
+ });
+ if (defined($wwid) && defined($name)) {
+ my $dir = ($options{sysfs} // '/sys') . "/class/block/$name";
+ my $topology = sub {
+ my $uuid = file_read_firstline("$dir/dm/uuid") // '';
+ die "'$name' is not the multipath map of '$wwid'\n" if $uuid ne "mpath-$wwid";
+ opendir(my $dh, "$dir/slaves") or die "unable to list '$dir/slaves' - $!\n";
+ my @paths = sort grep { $_ ne '.' && $_ ne '..' } readdir($dh);
+ closedir($dh);
+ return \@paths;
+ };
+ my %path_options = (
+ %options, 'ignore-holders' => { %{ $options{'ignore-holders'} // {} }, $name => 1 },
+ );
+ $check->(sub {
+ my $paths = $topology->();
+ # one path at a time, so that a failed check keeps the users of the others
+ for my $path ($paths->@*) {
+ $check->(sub { push @users, block_device_users([$path], \%path_options)->@* });
+ }
+ die "the paths of '$name' changed during the check\n"
+ if join(' ', $topology->()->@*) ne join(' ', $paths->@*);
+ });
+ }
+ return { 'in-use' => 'yes', 'in-use-by' => join(', ', @users) } if @users;
+ return { 'in-use' => 'unknown', 'in-use-by' => "check failed: $error" } if defined($error);
+ return { 'in-use' => 'no', 'in-use-by' => '' };
+}
+
# Retain the checked device instance even if its kernel name or udev alias is reused.
sub open_block_device {
my ($device, $stamp) = @_;
@@ -122,8 +289,9 @@ my sub block_entries {
return [map { m/^([\w.:+-]+)$/ ? $1 : die "unexpected entry '$_' in '$dir'\n" } @names];
}
+# Only exact same-LUN kpartx UUIDs make a holder a partition, not its name or its table type.
sub block_device_partitions {
- my ($name, $sys) = @_;
+ my ($name, $sys, $partition_maps) = @_;
$sys //= '/sys';
my $dir = "$sys/class/block/$name";
my @parts;
@@ -134,10 +302,22 @@ sub block_device_partitions {
if !defined($number) || $number !~ m/^[1-9]\d*$/;
push @parts, $part;
}
+ if ($partition_maps && $name =~ m/^dm-\d+$/) {
+ my $uuid = file_read_firstline("$dir/dm/uuid")
+ // die "unable to identify '$name' - missing device mapper identity\n";
+ if (my ($wwid) = $uuid =~ m/^(?:part[1-9]\d*-)?mpath-(\S+)$/) {
+ for my $holder (block_entries("$dir/holders")->@*) {
+ next if $holder !~ m/^dm-\d+$/;
+ my $child = file_read_firstline("$sys/class/block/$holder/dm/uuid")
+ // die "unable to identify holder '$holder'\n";
+ push @parts, $holder if $child =~ m/^part[1-9]\d*-mpath-\Q$wwid\E$/;
+ }
+ }
+ }
return \@parts;
}
-# Postorder keeps child partitions accessible until their signatures have been erased.
+# Postorder keeps nested partition maps accessible until their own signatures have been erased.
sub wipe_partitions {
my ($device, $sys) = @_;
$sys //= '/sys';
@@ -146,7 +326,7 @@ sub wipe_partitions {
$visit = sub {
my ($name) = @_;
die "cannot inspect '$device' - cyclic partition topology\n" if $seen{$name}++;
- for my $part (block_device_partitions($name, $sys)->@*) {
+ for my $part (block_device_partitions($name, $sys, 1)->@*) {
$visit->($part);
push @parts, $part;
}
@@ -156,6 +336,116 @@ sub wipe_partitions {
return \@parts;
}
+# Readiness is checked before signature probes: even read-only I/O can stall on a queueing map.
+# A map's direct member users matter too, including same-identity paths not yet added to the map.
+sub assert_block_device_ready {
+ my ($device, $options) = @_;
+ $options //= {};
+ my $sys = $options->{sysfs} // '/sys';
+ my $whole = is_partition($device) ? get_blockdev($device) : $device;
+ my $name = basename($whole);
+ my $dir = "$sys/class/block/$name";
+ my $size = file_read_firstline("$dir/size");
+ die "cannot use '$device' - device is unavailable\n"
+ if !defined($size) || $size !~ m/^\d+$/ || !$size;
+ my $uuid = $name =~ m/^dm-\d+$/ ? file_read_firstline("$dir/dm/uuid") : undef;
+ die "cannot use '$device' - missing device mapper identity\n"
+ if $name =~ m/^dm-\d+$/ && !defined($uuid);
+
+ if (defined($uuid) && $uuid =~ m/^part[1-9]\d*-mpath-/) {
+ my $parents = block_entries("$dir/slaves");
+ die "cannot use '$device' - cannot identify containing map\n" if @$parents != 1;
+ return assert_block_device_ready("/dev/$parents->[0]", $options);
+ }
+ if ($name =~ m/^nvme\d+n\d+$/) {
+ my $status = get_nvme_path_status()->{$name};
+ my $state = file_read_firstline("$dir/device/state") // '';
+ die "cannot use '$device' - namespace has no usable path\n"
+ if $status ? !$status->{active} : $state ne 'live';
+ my $id = file_read_firstline("$dir/wwid")
+ // die "cannot use '$device' - missing namespace identity\n";
+ for my $other (
+ grep { /^nvme\d+n\d+$/ && $_ ne $name }
+ block_entries("$sys/class/block")->@*
+ ) {
+ my $other_id = file_read_firstline("$sys/class/block/$other/wwid")
+ // die "cannot use '$device' - unresolved namespace '$other'\n";
+ die "cannot use '$device' - namespace has multiple devices without a selected map\n"
+ if $id eq $other_id;
+ }
+ return;
+ }
+
+ my ($wwid) = defined($uuid) ? $uuid =~ m/^mpath-(\S+)$/ : ();
+ my %members;
+ if (defined($wwid)) {
+ require PVE::Multipath;
+ %members = map { $_ => 1 } block_entries("$dir/slaves")->@*;
+ my ($map) = grep { ($_->{sysfs} // '') eq $name && ($_->{wwid} // '') eq $wwid }
+ PVE::Multipath::get_maps()->@*;
+ my $usable = 0;
+ if ($map) {
+ for my $path (map { $_->{paths}->@* } $map->{'path-groups'}->@*) {
+ $usable++
+ if $members{ $path->{dev} }
+ && ($path->{'dm-state'} // '') eq 'active'
+ && ($path->{'check-state'} // '') =~ m/^(?:ready|ghost)$/;
+ }
+ }
+ die "cannot use '$device' - multipath map has no usable path\n" if !$usable;
+ } elsif (!is_san_device($whole)) {
+ return;
+ }
+
+ require PVE::Multipath::Identity;
+ my $ids = PVE::Multipath::Identity::collect(all => 1);
+ die "cannot use '$device' - cannot inspect sibling paths: $ids->{error}\n" if $ids->{error};
+ $wwid //= PVE::Multipath::Identity::path_wwid($ids, $name)
+ // die "cannot use '$device' - unresolved LUN identity\n";
+ for my $path (sort keys $ids->{paths}->%*) {
+ my $id = PVE::Multipath::Identity::path_wwid($ids, $path);
+ if (!defined($id)) {
+ next if PVE::Multipath::Identity::path_is_other_lun($ids, $path, $wwid);
+ die "cannot use '$device' - cannot exclude '$path' as another path of this LUN because"
+ . " its identity could not be verified; check the path identity and multipathd"
+ . " configuration\n";
+ }
+ next if $id ne $wwid;
+ die "cannot use '$device' - LUN has multiple paths without a selected multipath map\n"
+ if !defined($uuid) && $path ne $name;
+ $members{$path} = 1 if defined($uuid);
+ }
+ if (defined($uuid)) {
+ my %namespace_ids;
+ for my $member (grep { /^nvme/ } keys %members) {
+ my $id = file_read_firstline("$sys/class/block/$member/wwid")
+ // die "cannot use '$device' - unresolved namespace '$member'\n";
+ $namespace_ids{$id} = 1;
+ }
+ if (%namespace_ids) {
+ for my $other (grep { /^nvme\d+n\d+$/ } block_entries("$sys/class/block")->@*) {
+ my $id = file_read_firstline("$sys/class/block/$other/wwid")
+ // die "cannot use '$device' - unresolved namespace '$other'\n";
+ $members{$other} = 1 if $namespace_ids{$id};
+ }
+ }
+ my $users = block_device_users(
+ [sort keys %members],
+ {
+ %$options,
+ 'ignore-holders' => { $name => 1 },
+ 'ignore-executables' =>
+ { '/sbin/multipathd' => 1, '/usr/sbin/multipathd' => 1 },
+ },
+ );
+ die "cannot use '$device' - " . join(', ', $users->@*) . "\n" if $users->@*;
+ } else {
+ my $state = file_read_firstline("$dir/device/state") // '';
+ die "cannot use '$device' - path is not running\n" if $state ne 'running';
+ }
+ return;
+}
+
# Never read device data. Probe failures are not evidence of non-use. Excluding a holder must not
# exclude direct users; multipathd can be exempted because its checkers keep member paths open.
sub block_device_users {
@@ -183,6 +473,11 @@ sub block_device_users {
die "unable to read '$file' - missing value\n" if !defined($value);
return $value;
};
+ my $display_name = sub {
+ my ($name) = @_;
+ return $name if $name !~ m/^dm-\d+$/;
+ return file_read_firstline("$sys/class/block/$name/dm/name") || $name;
+ };
my ($numbers, $seen, $users) = ({}, {}, []);
my $visit;
$visit = sub {
@@ -192,16 +487,15 @@ sub block_device_users {
my $dir = "$sys/class/block/$name";
my $dev = $read->("$dir/dev");
die "invalid device number for '$name'\n" if $dev !~ m/^\d+:\d+$/;
- $numbers->{$dev} = $name;
- for my $part ($entries->($dir)->@*) {
- next if !-e "$dir/$part/partition";
- my $number = $read->("$dir/$part/partition");
- die "invalid partition number for '$part'\n" if $number !~ m/^[1-9]\d*$/;
- $visit->($part);
- }
+ my $display = $display_name->($name);
+ $numbers->{$dev} = $display;
+ my $parts = block_device_partitions($name, $sys, $options->{'partition-maps'});
+ my %parts = map { $_ => 1 } $parts->@*;
+ $visit->($_) for $parts->@*;
+
for my $holder ($entries->("$dir/holders")->@*) {
- next if $ignore->{$holder};
- push $users->@*, "$name held by $holder";
+ next if $ignore->{$holder} || $parts{$holder};
+ push $users->@*, "$display held by " . $display_name->($holder);
$visit->($holder);
}
};
@@ -215,9 +509,12 @@ sub block_device_users {
my (undef, undef, $dev, undef, $target) = split(/\s+/, $line);
die "unable to check mounts - malformed mountinfo\n"
if !defined($target) || $dev !~ m/^\d+:\d+$/ || $line !~ m/ - /;
+ $target =~ s/\\([0-7]{3})/chr(oct($1))/ge;
push $users->@*, "$numbers->{$dev} mounted at $target" if defined($numbers->{$dev});
}
+ return $users if $options->{'metadata-only'};
+
my $by_path = sub {
my ($path, $what) = @_;
return if $path eq '';
@@ -331,6 +628,27 @@ sub parse_wipefs_signatures {
return $probe->{signatures};
}
+sub parse_pv_report {
+ my ($json) = @_;
+
+ my $report = decode_json($json);
+ die "Malformed pvs report\n"
+ if ref($report) ne 'HASH'
+ || ref($report->{report}) ne 'ARRAY'
+ || $report->{report}->@* != 1
+ || ref($report->{report}->[0]) ne 'HASH'
+ || ref($report->{report}->[0]->{pv}) ne 'ARRAY';
+ for my $pv ($report->{report}->[0]->{pv}->@*) {
+ die "Malformed PV in pvs report\n"
+ if ref($pv) ne 'HASH'
+ || grep { !defined($pv->{$_}) || ref($pv->{$_}) } qw(pv_name vg_name pv_in_use);
+ die "Invalid PV in pvs report\n"
+ if ($pv->{vg_name} ne '[unknown]' && $pv->{vg_name} !~ m/^[a-zA-Z0-9+_.-]*$/)
+ || $pv->{pv_in_use} !~ m/^(?:0|1|unknown|)$/;
+ }
+ return $report->{report}->[0]->{pv};
+}
+
# Read cached metadata only; never probe the device.
sub udev_properties {
my ($dev) = @_;
@@ -774,6 +1092,15 @@ sub dir_is_empty {
return 1;
}
+# Some controllers pad the NQN or derive it from padded serial/model fields; trim trailing space.
+sub read_subsysnqn {
+ my ($path) = @_;
+
+ my $nqn = file_read_firstline($path) // return undef;
+ $nqn =~ s/\s+$//;
+ return $nqn;
+}
+
# Returns the NVMe transport (pcie, fc, tcp, rdma, loop) of a namespace, either directly from
# its controller, or from any controller of its subsystem for native NVMe multipath setups.
sub get_nvme_transport {
@@ -1464,12 +1791,15 @@ sub change_parttype {
run_command(['sgdisk', "-t${partnum}:${parttype}", $blockdev], errmsg => $err);
}
-# Caller holds the disk lock through verification, wiping and the device trigger.
+# Caller holds the disk lock and, for SAN devices, the storage config lock. Without a storage
+# callback the device was classified local; a topology change must not bypass that lock.
sub wipe_blockdev_checked {
- my ($reference, $expected) = @_;
+ my ($reference, $expected, $check_storage) = @_;
my $current = block_device_reference($reference, $expected);
my $disk = $current->{device};
+ die "cannot wipe '$reference' - device topology changed\n"
+ if !$check_storage && (is_san_device($disk) || is_multipath_device($disk));
my $partitions = wipe_partitions($disk);
my $children = { map { $_ => block_device_reference("/dev/$_") } $partitions->@* };
my $verify = sub {
@@ -1481,18 +1811,17 @@ sub wipe_blockdev_checked {
die "cannot wipe '$reference' - partition topology changed\n"
if join(',', @remaining) ne join(',', $parts->@*);
for my $part (@remaining) {
- my $now = block_device_reference(
- "/dev/$part", $children->{$part},
- );
+ my $now = block_device_reference("/dev/$part", $children->{$part});
die "cannot wipe '$reference' - partition '$part' changed\n"
if $now->{stamp} ne $children->{$part}->{stamp};
}
};
+ assert_block_device_ready($disk);
$verify->();
my $handle = open_block_device($disk, $current->{stamp});
- my $users = block_device_users(
- [basename($disk)], {},
- );
+ $check_storage->($disk, $partitions) if $check_storage;
+ assert_block_device_ready($disk);
+ my $users = block_device_users([basename($disk)], { 'partition-maps' => 1 });
die "cannot wipe '$disk' - " . join(', ', $users->@*) . "\n" if $users->@*;
$verify->();
wipe_blockdev(
@@ -1519,7 +1848,7 @@ sub wipe_blockdev {
my $current = wipe_partitions($devpath);
die "cannot wipe '$devpath' - partition topology changed\n"
if join(',', $current->@*) ne join(',', $partitions->@*);
- my $users = block_device_users([basename($devpath)], {});
+ my $users = block_device_users([basename($devpath)], { 'partition-maps' => 1 });
die "cannot wipe '$devpath' - " . join(', ', $users->@*) . "\n" if $users->@*;
for my $part ($partitions->@*) {
$verify->();
@@ -1528,8 +1857,17 @@ sub wipe_blockdev {
my $path = $handle ? "/proc/$$/fd/" . fileno($handle) : "/dev/$part";
run_command(['wipefs', '--all', $path], errmsg => "error wiping '/dev/$part'");
close($handle) or die "error closing '/dev/$part' - $!\n" if $handle;
+ next if $part !~ m/^dm-\d+$/;
+ $verify->();
+ my @selector = $child ? ('--uuid', $child->{dm_uuid}) : ("/dev/$part");
+ # udev probes the wiped partition map right away and keeps it open meanwhile
+ run_command(
+ ['dmsetup', 'remove', '--retry', @selector],
+ errmsg => "error removing '/dev/$part'",
+ );
+ $removed->{$part} = 1;
}
- # New holders make the parent unsafe even after its child signatures have been erased.
+ # Recreated partition maps, new holders and new openers all make the parent unsafe.
$users = block_device_users([basename($devpath)], {});
die "cannot wipe '$devpath' - " . join(', ', $users->@*) . "\n" if $users->@*;
}
@@ -1591,7 +1929,7 @@ sub udevadm_trigger {
return if scalar(@devs) == 0;
- eval { run_command(['udevadm', 'trigger', @devs]); };
+ eval { run_command(['udevadm', 'trigger', '--settle', @devs]); };
warn $@ if $@;
}
diff --git a/src/PVE/Multipath/Identity.pm b/src/PVE/Multipath/Identity.pm
index 5fa6542d..ed02373e 100644
--- a/src/PVE/Multipath/Identity.pm
+++ b/src/PVE/Multipath/Identity.pm
@@ -81,7 +81,7 @@ sub resolve($disks, $daemon, $naa_ok) {
my $unresolved = !$daemon;
for my $name ($lun->{paths}->@*) {
my $disk = $disks->{$name};
- $held->{ $disk->{map} } = 1 if defined($disk->{map});
+ $held->{$_} = 1 for ($disk->{maps} // [$disk->{map} // ()])->@*;
my $line = $daemon ? $daemon->{$name} : undef;
if (line_identifies($disk, $line, $naa_ok)) {
$listed->{ $line->{wwid} } = 1;
@@ -178,13 +178,21 @@ my sub scsi_disks($sessions, $targets, $all) {
my $devt = sysfs_line("$dir/block/$name/dev") // return;
my $disk = { lun => int($lun), devt => $devt, hctl => $hctl, dir => $dir };
$disk->{target} = $target if defined($target);
- # Read the holder through the SCSI device, so neither name can refer to another device
- # meanwhile.
- my (undef, $holder) = dir_glob_regex("$SYSFS/$dir/block/$name/holders", '(dm-\d+)');
- if (defined($holder)) {
- my $uuid = sysfs_line("$dir/block/$name/holders/$holder/dm/uuid") // '';
- ($disk->{map}) = $uuid =~ m/^mpath-(\S+)$/;
- }
+ # Read the holders through the SCSI device, so neither name can refer to another device
+ # meanwhile. Check every holder, as a logical volume can hold a path next to its map.
+ my %maps;
+ dir_glob_foreach(
+ "$SYSFS/$dir/block/$name/holders",
+ 'dm-\d+',
+ sub($holder, @) {
+ my $uuid = sysfs_line("$dir/block/$name/holders/$holder/dm/uuid") // '';
+ $maps{$1} = 1 if $uuid =~ m/^mpath-(\S+)$/;
+ },
+ );
+ my @maps = sort keys %maps;
+ $disk->{map} = $maps[0] if @maps;
+ # several maps on one path leave its LUN unresolved
+ $disk->{maps} = \@maps if @maps > 1;
$res->{$name} = $disk;
},
);
@@ -237,10 +245,12 @@ sub collect(%param) {
for my $lun (values %$luns) {
for my $name ($lun->{paths}->@*) {
my $disk = $disks->{$name};
+ # claimed: a map or the daemon names an identity, even one that did not resolve
$paths->{$name} = {
$disk->%{qw(devt hctl)},
wwid => $lun->{wwid},
$lun->{unresolved} ? (unresolved => 1) : (),
+ defined($disk->{map}) || ($daemon && $daemon->{$name}) ? (claimed => 1) : (),
};
}
}
@@ -296,4 +306,97 @@ sub path_wwid($ids, $name) {
return $path->{wwid};
}
+my sub valid_naa($value) {
+ my $naa = length($value) ? ord($value) >> 4 : 0;
+ return length($value) == 16 ? $naa == 6 : length($value) == 8 && grep { $naa == $_ } 2, 3, 5;
+}
+
+# The VPD page 0x83 of the path $name of $ids, if the name denotes the collected device before and
+# after reading it.
+my sub current_page($ids, $name) {
+ my $path = $ids->{paths}->{$name} // return undef;
+ my $current = sub {
+ my $devt = sysfs_line("class/block/$name/dev");
+ my $device = abs_path("$SYSFS/class/block/$name/device");
+ return
+ defined($devt)
+ && defined($device)
+ && $devt eq $path->{devt}
+ && basename($device) eq $path->{hctl};
+ };
+ return undef if !$current->();
+ my $page = eval { file_get_contents("$SYSFS/class/block/$name/device/vpd_pg83") };
+ return undef if !$current->();
+ return $page;
+}
+
+# Return the binary NAA designators of the logical unit in VPD page 0x83, in hex, and the number of
+# its other designators. Return undef for a malformed page, and if any other logical unit designator
+# is malformed or of a kind that multipathd might turn into an NAA WWID ('3...'), for example a SCSI
+# name string 'naa.<hex>'.
+my sub unit_designators($page) {
+ my $designators = vpd_designators($page) // return undef;
+ my ($naa, $other) = ([], 0);
+ for my $designator ($designators->@*) {
+ my ($set, $association, $type, $value) = $designator->@*;
+ next if $association != 0; # target port and target device designators
+ if ($type == 3) {
+ return undef if $set != 1 || !valid_naa($value);
+ push $naa->@*, unpack('H*', $value);
+ next;
+ }
+ my $length = length($value);
+ # printable ASCII with at least one character, padded with spaces or NUL bytes
+ my $ascii = $value =~ m/\A[\x20-\x7e]*[\x21-\x7e][\x20-\x7e]*\0*\z/;
+ if ($type == 0) { # vendor specific
+ return undef if $set == 1 ? !$length : $set != 2 || !$ascii;
+ } elsif ($type == 1) { # T10 vendor identification
+ return undef if $set != 2 || $length < 8 || !$ascii;
+ } elsif ($type == 2) { # EUI-64 based
+ return undef if $set != 1 || ($length != 8 && $length != 12 && $length != 16);
+ } elsif ($type == 8) { # SCSI name string, only the ASCII forms of iSCSI and EUI names
+ my $name = $value =~ s/\0+\z//r;
+ return undef
+ if $set != 3
+ || ($name !~ m/\Aiqn\.[0-9]{4}-[0-9]{2}\.[a-zA-Z0-9.:_-]+\z/
+ && $name !~ m/\Aeui\.[0-9a-fA-F]{16}(?:[0-9a-fA-F]{8}){0,2}\z/);
+ } else {
+ return undef;
+ }
+ $other++;
+ }
+ return { naa => $naa, other => $other };
+}
+
+# Whether the path $name of $ids, which has no resolved identity, is not a path of the LUN with
+# the NAA WWID $wwid. Only a path that neither a map nor multipathd names, for example a local disk
+# that multipathd does not list, can qualify: an identity from the configured source that did not
+# resolve may still be this LUN. Its VPD page 0x83 must name the logical unit only with designators
+# that multipathd cannot turn into $wwid, and a current path of the LUN must have $wwid as a binary
+# NAA designator, as other sources, like a SCSI name string or a configured udev property, can
+# yield the same spelling. The kernel caches the page when it scans the device, as multipathd
+# caches its identity.
+sub path_is_other_lun($ids, $name, $wwid) {
+ my ($hex) = $wwid =~ m/^3((?:[0-9a-fA-F]{16}){1,2})$/ or return 0;
+ $hex = lc($hex);
+ my $path = $ids->{paths}->{$name} // return 0;
+ return 0 if $path->{claimed};
+ my $unit = unit_designators(current_page($ids, $name)) // return 0;
+ return 0 if !$unit->{other} && !$unit->{naa}->@*;
+ return 0 if grep { $_ eq $hex } $unit->{naa}->@*;
+ for my $member (sort keys $ids->{paths}->%*) {
+ next if (path_wwid($ids, $member) // '') ne $wwid;
+ my $designators = vpd_designators(current_page($ids, $member)) // next;
+ return 1 if grep {
+ my ($set, $association, $type, $value) = $_->@*;
+ $set == 1
+ && $association == 0
+ && $type == 3
+ && valid_naa($value)
+ && unpack('H*', $value) eq $hex
+ } $designators->@*;
+ }
+ return 0;
+}
+
1;
diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm
index 8cbe9308..cb7a62f9 100755
--- a/src/PVE/Storage.pm
+++ b/src/PVE/Storage.pm
@@ -329,6 +329,225 @@ sub derived_multipath_wwids {
return $derived;
}
+# Compare whole LUN identities, not aliases or the spelling of a configured base volume. A
+# partition belongs to its containing LUN for storage-reference checks.
+sub san_device_identity {
+ my ($device, $ids, $sysfs, $allow_member) = @_;
+ $sysfs //= '/sys';
+
+ $device = PVE::Diskmanage::verify_blockdev_path($device);
+ $device = PVE::Diskmanage::get_blockdev($device) if PVE::Diskmanage::is_partition($device);
+ my $name = basename($device);
+ my $dir = "$sysfs/class/block/$name";
+ my $number = file_read_firstline("$dir/dev");
+ die "unable to identify '$device' - missing device number\n"
+ if !defined($number) || $number !~ m/^\d+:\d+$/;
+ opendir(my $dh, "$dir/holders") or die "unable to inspect '$device' - $!\n";
+ my @holders = grep { $_ ne '.' && $_ ne '..' } readdir($dh);
+ closedir($dh);
+ my $wwid;
+
+ for my $holder (@holders) {
+ next if $holder !~ m/^dm-\d+$/;
+ my $uuid = file_read_firstline("$sysfs/class/block/$holder/dm/uuid");
+ die "unable to identify holder '$holder'\n" if !defined($uuid);
+ if ($uuid =~ m/^mpath-(\S+)$/) {
+ die "cannot use '$device' - member path of multipath map '$holder'\n"
+ if !$allow_member;
+ $wwid = $1;
+ }
+ }
+
+ my $res = { key => "local:$number", targets => {} };
+ if ($name =~ m/^dm-\d+$/) {
+ my $uuid = file_read_firstline("$dir/dm/uuid")
+ // die "unable to identify '$device' - missing device mapper identity\n";
+ ($wwid) = $uuid =~ m/^(?:part\d+-)?mpath-(\S+)$/;
+ } elsif ($name =~ m/^nvme\d+n\d+$/) {
+ my $nqn = PVE::Diskmanage::read_subsysnqn("$dir/device/subsysnqn")
+ // die "unable to identify '$device' - missing subsystem\n";
+ my $id = file_read_firstline("$dir/wwid")
+ // die "unable to identify '$device' - missing namespace identity\n";
+ die "unable to identify '$device' - empty namespace identity\n" if !$id;
+ $res->{key} = "nvme:$id";
+ $res->{subsystem} = $nqn;
+ } elsif (PVE::Diskmanage::is_san_device($device)) {
+ $ids = $ids->() if ref($ids) eq 'CODE';
+ $wwid = PVE::Multipath::Identity::path_wwid($ids, $name)
+ // die "unable to identify '$device' - unresolved LUN identity\n";
+ }
+ if (defined($wwid)) {
+ $ids = $ids->() if ref($ids) eq 'CODE';
+ $res->{key} = "scsi:$wwid";
+ my $observed = 0;
+ for my $lun (values $ids->{luns}->%*) {
+ $observed = 1 if ($lun->{wwid} // '') eq $wwid;
+ next if !defined($lun->{target});
+ if (!defined($lun->{wwid})) {
+ my @paths = sort(($lun->{paths} // [])->@*);
+ next
+ if @paths
+ && !grep { !PVE::Multipath::Identity::path_is_other_lun($ids, $_, $wwid) }
+ @paths;
+ die "unable to inspect target '$lun->{target}' - the identity of '"
+ . join(', ', @paths)
+ . "' could not be verified; check the path identity and multipathd"
+ . " configuration\n";
+ }
+ $res->{targets}->{ $lun->{target} } = 1 if $lun->{wwid} eq $wwid;
+ }
+ die "unable to inspect '$device' - no identified member paths\n" if !$observed;
+ }
+ return $res;
+}
+
+my sub normalized_pv_uuid {
+ my ($uuid) = @_;
+ die "unable to inspect physical volume - invalid PV UUID\n"
+ if !defined($uuid)
+ || ref($uuid)
+ || $uuid !~ m/^(?:[a-zA-Z0-9]{32}|[a-zA-Z0-9]{6}(?:-[a-zA-Z0-9]{4}){5}-[a-zA-Z0-9]{6})$/;
+ $uuid =~ s/-//g;
+ return $uuid;
+}
+
+# Called under the storage configuration lock before writing a SAN device. Scan attribution is
+# intentionally not used: it selects only one owner and can be stale.
+sub assert_san_write_safe {
+ my ($cfg, $device, $partitions) = @_;
+ $partitions //= PVE::Diskmanage::wipe_partitions($device);
+ PVE::Diskmanage::assert_block_device_ready($device);
+
+ my $collected;
+ my $ids = sub {
+ return $collected if $collected;
+ my $result = PVE::Multipath::Identity::collect(all => 1);
+ die "unable to inspect SAN identities - $result->{error}\n" if $result->{error};
+ if (-d "$PVE::Storage::ISCSIPlugin::SYSFS/class/iscsi_session") {
+ my $sessions = PVE::Storage::ISCSIPlugin::iscsi_session_device_map(1);
+ # san_device_identity() decides about devices without an identity
+ for my $name (keys %$sessions) {
+ $result->{luns}->{"$name\0strict"} = {
+ wwid => PVE::Multipath::Identity::path_wwid($result, $name),
+ target => $sessions->{$name}->{target},
+ paths => [$name],
+ };
+ }
+ }
+ return $collected = $result;
+ };
+ my $identity = san_device_identity($device, $ids);
+ for my $sid (sort keys $cfg->{ids}->%*) {
+ my $scfg = $cfg->{ids}->{$sid};
+ if (
+ $scfg->{content}->{images} || $scfg->{type} eq 'iscsidirect' || $scfg->{type} eq 'zfs'
+ ) {
+ die "cannot use '$device' - storage '$sid' serves disk images from this target\n"
+ if $scfg->{target} && $identity->{targets}->{ $scfg->{target} };
+ die "cannot use '$device' - storage '$sid' serves disk images from this subsystem\n"
+ if $scfg->{subsysnqn} && ($identity->{subsystem} // '') eq $scfg->{subsysnqn};
+ }
+ next if $scfg->{type} ne 'lvm' || !$scfg->{base};
+ my $base = eval {
+ my ($base_sid) = parse_volume_id($scfg->{base});
+ my (undef, $name) = parse_volname($cfg, $scfg->{base});
+ my $type = $cfg->{ids}->{$base_sid}->{type};
+ my $canonical =
+ $type eq 'iscsi' ? PVE::Diskmanage::block_identity_from_id($name) : undef;
+ $canonical
+ // san_device_identity(scalar(path($cfg, $scfg->{base})), $ids, undef, 1);
+ };
+ if (!$base) {
+ my $err = $@ // '';
+ chomp($err);
+ die "cannot use '$device' - cannot check storage '$sid', its base volume"
+ . " '$scfg->{base}' is not visible on this node\n"
+ if $err =~ m/not a valid block device|failed to get absolute path/ || !$err;
+ die "cannot use '$device' - cannot check base volume '$scfg->{base}' of storage '$sid'"
+ . " - $err\n";
+ }
+ die "cannot use '$device' - storage '$sid' uses this LUN as its base volume\n"
+ if $base->{key} eq $identity->{key};
+ }
+
+ # LVM can select a clone instead of the candidate when both have the same PV UUID. Probe
+ # every device being destroyed directly rather than relying on that selected device list.
+ my %candidate_pvs;
+ for my $candidate ($device, map { "/dev/$_" } $partitions->@*) {
+ my $output = '';
+ run_command(
+ ['wipefs', '--no-act', '--json', '--output', 'TYPE,UUID', $candidate],
+ timeout => 10,
+ outfunc => sub { $output .= "$_[0]\n" },
+ );
+ my $probe = eval { PVE::Diskmanage::parse_wipefs_signatures($output) };
+ if (my $err = $@) {
+ my $kind = $err eq "Malformed wipefs signature\n" ? 'signature' : 'output';
+ die "unable to probe '$candidate' - malformed wipefs $kind\n";
+ }
+ for my $signature ($probe->@*) {
+ next if $signature->{type} ne 'LVM2_member';
+ $candidate_pvs{ normalized_pv_uuid($signature->{uuid}) } = 1;
+ }
+ }
+
+ my $output = '';
+ run_command(
+ [
+ '/sbin/pvs',
+ '--nohints',
+ '--readonly',
+ '--binary',
+ '--reportformat',
+ 'json',
+ '--options',
+ 'pv_name,vg_name,pv_uuid,pv_in_use',
+ ],
+ timeout => 10,
+ outfunc => sub { $output .= "$_[0]\n" },
+ );
+ my $report = eval { PVE::Diskmanage::parse_pv_report($output) };
+ if (my $err = $@) {
+ my $kind = $err =~ m/^(?:Malformed|Invalid) PV in pvs report\n$/ ? 'PV' : 'pvs output';
+ die "unable to inspect volume groups - malformed $kind\n";
+ }
+ my %reported;
+ for my $pv ($report->@*) {
+ die "unable to inspect volume groups - malformed PV\n" if $pv->{pv_name} !~ m{^/dev/};
+ $reported{ normalized_pv_uuid($pv->{pv_uuid}) } = 1;
+ }
+ die "cannot use '$device' - LVM does not report its physical volume,"
+ . " so its volume group is unknown\n"
+ if grep { !$reported{$_} } keys %candidate_pvs;
+
+ for my $pv ($report->@*) {
+ my $uuid = normalized_pv_uuid($pv->{pv_uuid});
+ # The sentinel means missing metadata even if the use flag contradicts it. A matching UUID
+ # already proves membership; resolving an alias first could hide that proof if it vanished.
+ my $unknown_vg = $pv->{vg_name} eq '[unknown]'
+ || ($pv->{vg_name} eq '' && $pv->{pv_in_use} ne '0');
+ if ($unknown_vg) {
+ die "cannot use '$device' - physical volume belongs to a volume group"
+ . " whose metadata is not visible on this node\n"
+ if $candidate_pvs{$uuid}
+ || san_device_identity($pv->{pv_name}, $ids, undef, 1)->{key} eq $identity->{key};
+ next;
+ }
+ next if $pv->{vg_name} eq '';
+ my @storages = grep {
+ my $scfg = $cfg->{ids}->{$_};
+ ($scfg->{type} eq 'lvm' || $scfg->{type} eq 'lvmthin')
+ && ($scfg->{vgname} // '') eq $pv->{vg_name};
+ } sort keys $cfg->{ids}->%*;
+ next if !@storages;
+ die "cannot use '$device' - volume group '$pv->{vg_name}' is used by storage '"
+ . $storages[0] . "'\n"
+ if $candidate_pvs{$uuid}
+ || san_device_identity($pv->{pv_name}, $ids, undef, 1)->{key} eq $identity->{key};
+ }
+ return;
+}
+
# storage_can_replicate:
# return true if storage supports replication
# (volumes allocated with vdisk_alloc() has replication feature)
diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm
index 5ba56519..0660529a 100644
--- a/src/PVE/Storage/LVMPlugin.pm
+++ b/src/PVE/Storage/LVMPlugin.pm
@@ -146,6 +146,15 @@ sub clear_first_sector {
}
}
+sub assert_whole_lun {
+ my ($device) = @_;
+
+ my $uuid = file_read_firstline('/sys/class/block/' . basename($device) . '/dm/uuid') // '';
+ die "cannot use '$device' - create the volume group on the whole LUN\n"
+ if PVE::Diskmanage::is_partition($device) || $uuid =~ m/^part[1-9]\d*-mpath-/;
+ return;
+}
+
sub lvm_create_volume_group {
my ($device, $vgname, $shared, $guard) = @_;
$guard = {%$guard} if $guard;
@@ -153,12 +162,15 @@ sub lvm_create_volume_group {
my $requested_device = $guard ? $guard->{reference} : $device;
$device = PVE::Diskmanage::verify_blockdev_path($requested_device);
my $san = $guard && $guard->{san};
+ my $stable_device;
if ($guard) {
if ($san) {
+ assert_whole_lun($device);
my $current = $guard->{current} = PVE::Diskmanage::block_device_reference(
$requested_device, $guard->{expected},
);
$device = $current->{device};
+ PVE::Diskmanage::assert_block_device_ready($device);
$guard->{handle} = PVE::Diskmanage::open_block_device($device, $current->{stamp});
$guard->{verify} = sub {
PVE::Diskmanage::assert_block_device_instance($requested_device, $current);
@@ -174,10 +186,12 @@ sub lvm_create_volume_group {
$guard->{verify}->();
}
}
- my $check_instance = sub { $guard->{verify}->() if $guard };
+ my $check_instance = sub { $guard->{verify}->($stable_device) if $guard };
my $check_safe = sub {
$check_instance->();
if ($san) {
+ $guard->{check_storage}->($device);
+ PVE::Diskmanage::assert_block_device_ready($device);
my $users = PVE::Diskmanage::block_device_users(
[basename($device)], { 'partition-maps' => 1 },
);
@@ -258,17 +272,21 @@ sub lvm_create_volume_group {
delete $signatures{LVM2_member} if $res && !$res->{vgname};
$refuse_data->(keys %signatures) if %signatures;
+ if ($san) {
+ $stable_device = PVE::Diskmanage::canonical_blockdev_link($device)
+ // die "cannot use '$device' - cannot verify a stable name under /dev/disk/by-id\n";
+ }
$check_safe->();
clear_first_sector($san ? '/proc/' . $$ . '/fd/' . fileno($guard->{handle}) : $device);
# we use --metadatasize 250k, which reseults in "pe_start = 512"
# so pe_start is aligned on a 128k boundary (advantage for SSDs)
- my $cmd = ['/sbin/pvcreate', '--metadatasize', '250k', $device];
+ my $cmd = ['/sbin/pvcreate', '--metadatasize', '250k', $stable_device // $device];
$check_safe->();
run_command($cmd, errmsg => "pvcreate '$device' error");
- $cmd = ['/sbin/vgcreate', $vgname, $device];
+ $cmd = ['/sbin/vgcreate', $vgname, $stable_device // $device];
# push @$cmd, '-c', 'y' if $shared; # we do not use this yet
$check_safe->();
@@ -1107,6 +1125,8 @@ sub on_add_hook {
# for the map to assemble before the VG is created on it
PVE::Storage::activate_volumes($cfg, [$base]);
+ assert_whole_lun(PVE::Diskmanage::verify_blockdev_path($path));
+
# a failing hook does not remove a volume group it created, so check the device first
assert_device_discard_supported($path)
if discard_on_remove_requested($scfg->{'on-volume-remove'});
@@ -1120,6 +1140,7 @@ sub on_add_hook {
{
reference => $path,
san => 1,
+ check_storage => sub { PVE::Storage::assert_san_write_safe($cfg, $_[0]) },
},
);
}
diff --git a/src/PVE/Storage/SANScan.pm b/src/PVE/Storage/SANScan.pm
index 4015361b..7efdbfd5 100644
--- a/src/PVE/Storage/SANScan.pm
+++ b/src/PVE/Storage/SANScan.pm
@@ -256,6 +256,13 @@ my sub build_lun_rows($cfg, $inventory) {
$entry->{multipath} = 'unmanaged';
}
}
+ my $hint = PVE::Diskmanage::block_device_usage_hint(
+ $disk->{devpath},
+ defined($disk->{slaves}) ? { 'map-wwid' => $disk->{wwn} } : undef,
+ );
+ $entry->@{qw(in-use in-use-by)} = $hint->@{qw(in-use in-use-by)};
+ my $link = PVE::Diskmanage::canonical_blockdev_link($disk->{devpath}, $ids, $wwid);
+ $entry->{'by-id'} = $link if defined($link);
push $records->@*,
{ row => $entry, disk => $disk, identity => lun_compare_key($entry, $disk) };
};
diff --git a/src/test/Makefile b/src/test/Makefile
index 3ae2639f..49f6413c 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_base_activation test_lvm_status test_status_command test_disklist test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_iscsi test_lun_identity test_multipath_api test_san_lun_scan test_san_api test_san_libraries test_san_taint
+test: test_zfspoolplugin test_lvmplugin test_lvm_create test_lvm_lock_retry test_lvm_base_volume test_lvm_hints test_lvm_base_activation test_lvm_status test_status_command test_disklist test_wipe test_disk_api test_bwlimit test_plugin test_ovf test_volume_access test_multipath test_iscsi test_lun_identity test_multipath_api test_san_lun_scan test_san_api test_san_libraries test_san_taint
test_zfspoolplugin: run_test_zfspoolplugin.pl
./run_test_zfspoolplugin.pl
@@ -37,11 +37,15 @@ test_disklist: run_disk_tests.pl run_block_device_users_tests.pl
./run_disk_tests.pl
./run_block_device_users_tests.pl
-test_disk_api: run_disk_api_tests.pl run_wipedisk_tests.pl run_wipe_reference_tests.pl run_wipe_partition_tests.pl
- ./run_disk_api_tests.pl
+test_wipe: run_wipedisk_tests.pl run_wipe_partition_tests.pl run_wipe_readiness_tests.pl run_wipe_reference_tests.pl
./run_wipedisk_tests.pl
- ./run_wipe_reference_tests.pl
./run_wipe_partition_tests.pl
+ ./run_wipe_readiness_tests.pl
+ ./run_wipe_reference_tests.pl
+
+test_disk_api: run_disk_api_tests.pl run_san_wipe_tests.pl
+ ./run_disk_api_tests.pl
+ ./run_san_wipe_tests.pl
test_bwlimit: run_bwlimit_tests.pl
./run_bwlimit_tests.pl
@@ -82,5 +86,6 @@ test_san_api: run_san_api_tests.pl
test_san_libraries: run_san_library_tests.pl
./run_san_library_tests.pl
-test_san_taint: run_san_taint_tests.pl
+test_san_taint: run_san_lvm_create_tests.pl run_san_taint_tests.pl
+ perl -T ./run_san_lvm_create_tests.pl
perl -T ./run_san_taint_tests.pl
diff --git a/src/test/disk_api_test.pm b/src/test/disk_api_test.pm
index 2371a953..2d30d2e5 100644
--- a/src/test/disk_api_test.pm
+++ b/src/test/disk_api_test.pm
@@ -132,6 +132,7 @@ subtest 'a volume group on a map is created under the storage configuration lock
$disk->mock(locked_disk_action => sub { $lock->('disk', $_[0]) });
$disk->mock(verify_blockdev_path => sub { $_[0] });
$disk->mock(block_device_reference => sub { { device => $_[0] } });
+ $storage->mock(config => sub { { ids => {} } });
$disk->mock(assert_disk_unused => sub { });
$disk->mock(is_partition => sub { 0 });
$disk->mock(udevadm_trigger => sub { });
diff --git a/src/test/run_block_device_users_tests.pl b/src/test/run_block_device_users_tests.pl
index 1c040f60..b041201f 100755
--- a/src/test/run_block_device_users_tests.pl
+++ b/src/test/run_block_device_users_tests.pl
@@ -16,6 +16,7 @@ my $options = { sysfs => "$root/sys", procfs => "$root/proc" };
my $disk = Test::MockModule->new('PVE::Diskmanage');
my %numbers = ('/dev/alias' => '8:1');
$disk->mock(
+ verify_blockdev_path => sub { $_[0] },
block_device_number => sub {
my ($path) = @_;
die "stat failed\n" if !exists($numbers{$path});
@@ -53,6 +54,11 @@ device('sda1', '8:1', 'sda');
is_deeply(users(), [], 'unused disk and partition');
write_file('/proc/self/mountinfo', "1 0 8:1 / /mnt rw - ext4 /dev/alias rw\n");
is_deeply(users(), ['sda1 mounted at /mnt'], 'mounts matched by number');
+is_deeply(
+ PVE::Diskmanage::block_device_usage_hint('/dev/sda', $options),
+ { 'in-use' => 'yes', 'in-use-by' => 'sda1 mounted at /mnt' },
+ 'metadata hint includes kernel partition mounts',
+);
write_file('/proc/self/mountinfo', "1 0 0:1 / / rw - rootfs rootfs rw\n");
write_file('/proc/swaps', "Filename\tType\tSize\tUsed\tPriority\n/dev/alias partition 1 0 -2\n");
is_deeply(users(), ['sda1 used as swap'], 'swap alias');
@@ -135,6 +141,11 @@ is_deeply(
unlink("$root/sys/class/block/sda1/dev") or die $!;
eval { users() };
like($@, qr/missing value/, 'missing partition identity fails closed');
+is(
+ PVE::Diskmanage::block_device_usage_hint('/dev/sda', $options)->{'in-use'},
+ 'unknown',
+ 'missing kernel partition identity gives unknown hint',
+);
write_file('/sys/class/block/sda1/dev', "8:1\n");
remove_tree("$root/sys/class/block/sda1/holders");
eval { users() };
diff --git a/src/test/run_lun_identity_tests.pl b/src/test/run_lun_identity_tests.pl
index a49f27c4..5a5fdced 100755
--- a/src/test/run_lun_identity_tests.pl
+++ b/src/test/run_lun_identity_tests.pl
@@ -538,6 +538,200 @@ for my $case ([$Q, '3WQ', 'sdg'], [$R, '3WR', 'sdh']) {
query_path_identities => sub { $queries++; die $reply if !ref($reply); return $reply });
}
+# --- paths without an identity that are provably another LUN ---
+
+{
+ my $W = '3600a09803832355367242f747a716b7a';
+ my sub page {
+ my $body =
+ join('', map { pack('C C x C', $_->[0], $_->[1], length($_->[2])) . $_->[2] } @_);
+ return pack('C C n', 0, 0x83, length($body)) . $body;
+ }
+ my sub naa { [0x01, 0x03, pack('H*', $_[0])] }
+ # a local QEMU disk, as lab nodes have it: one vendor specific ASCII designator
+ my $qemu = pack('H*', '0083000f0200000b64726976652d7363736930');
+ disk_device('0:0:0:0', 'sda', '8:0');
+ put('bus/scsi/devices/0:0:0:0/type', "0\n");
+ # the map's path names its NAA, the other path of the same LUN is not listed by the daemon
+ put('bus/scsi/devices/7:0:0:0/vpd_pg83', $vpd);
+ $reply = {};
+ my sub other {
+ my ($name, $wwid) = @_;
+ $ids = PVE::Multipath::Identity::collect(all => 1);
+ return PVE::Multipath::Identity::path_is_other_lun($ids, $name, $wwid // $W);
+ }
+ my sub sda_page { put('bus/scsi/devices/0:0:0:0/vpd_pg83', $_[0]) }
+
+ sda_page($qemu);
+ ok(other('sda'), 'other LUN: a local disk with a vendor specific designator only');
+ is($ids->{paths}->{sda}->{wwid}, undef, 'other LUN: and without an identity of its own');
+ ok(!other('sdc'), 'other LUN: not the unlisted path of the same LUN');
+ ok(!other('sda', '3OTHER'), 'other LUN: undecided for a WWID that is not an NAA');
+ ok(!other('sda', '2600a09803832355367242f747a716b7a'), 'other LUN: undecided for EUI-64');
+ ok(!other('sda', '3600a0980383235536724'), 'other LUN: undecided for a short NAA');
+
+ sda_page(page(naa('600a09803832355367242f747a716c30')));
+ ok(other('sda'), 'other LUN: a disk with the NAA of another LUN');
+ sda_page(page(naa('600a09803832355367242f747a716c30'), naa(substr($W, 1))));
+ ok(!other('sda'), 'other LUN: not with the NAA of the LUN after another one');
+ sda_page(page([0x01, 0x13, pack('H*', '600a098000000002ac16036a00000cbc')]));
+ ok(!other('sda'), 'other LUN: undecided with target port designators only');
+ sda_page(page());
+ ok(!other('sda'), 'other LUN: undecided with no designator');
+ sda_page(page([0x02, 0x03, '600a09803832355367242f747a716c30']));
+ ok(!other('sda'), 'other LUN: undecided with an NAA designator in ASCII');
+ sda_page(page(naa('600a0980383235536724')));
+ ok(!other('sda'), 'other LUN: undecided with an NAA designator of a wrong length');
+ sda_page(page(naa('500a09803832355367242f747a716c30')));
+ ok(!other('sda'), 'other LUN: undecided with an NAA type that does not fit its length');
+ sda_page(page([0x03, 0x08, 'naa.600A09803832355367242F747A716B7A']));
+ ok(!other('sda'), 'other LUN: undecided with a SCSI name string the daemon reads as an NAA');
+ sda_page(page([0x03, 0x08, 'iqn.2003-01.org.example:disk']));
+ ok(other('sda'), 'other LUN: a SCSI name string that is no NAA');
+ sda_page(substr($qemu, 0, 10));
+ ok(!other('sda'), 'other LUN: undecided with a truncated page');
+ unlink("$sys/bus/scsi/devices/0:0:0:0/vpd_pg83");
+ ok(!other('sda'), 'other LUN: undecided without a page');
+
+ for my $case (
+ [[0x00, 0x0f, 'x'], 'a designator of a reserved type'],
+ [[0x02, 0x00, ''], 'an empty vendor specific designator'],
+ [[0x03, 0x08, 'bad'], 'a SCSI name string without a known prefix'],
+ [[0x01, 0x07, pack('H*', '00' x 16)], 'an MD5 logical unit identifier'],
+ [[0x01, 0x02, pack('H*', '0123')], 'an EUI-64 designator of a wrong length'],
+ [[0x03, 0x08, "iqn.2003-01.org.example:\xff"], 'a SCSI name string that is not ASCII'],
+ [[0x03, 0x08, 'eui.x'], 'a SCSI name string with a malformed EUI'],
+ [[0x02, 0x00, "drive\xff"], 'a vendor specific designator that is not ASCII'],
+ [[0x02, 0x01, "QEMU \xff"], 'a T10 designator that is not ASCII'],
+ [[0x02, 0x00, "drive-scsi0\n"], 'a vendor specific designator ending in a newline'],
+ [[0x02, 0x01, "QEMU X\n"], 'a T10 designator ending in a newline'],
+ [[0x03, 0x08, "iqn.2003-01.org.example:disk\n"], 'an iSCSI name ending in a newline'],
+ [[0x03, 0x08, "eui.0123456789abcdef\n"], 'an EUI name ending in a newline'],
+ ) {
+ sda_page(page($case->[0]));
+ ok(!other('sda'), "other LUN: undecided with $case->[1]");
+ }
+ sda_page(page([0x01, 0x02, pack('H*', '0123456789abcdef')]));
+ ok(other('sda'), 'other LUN: a disk with an EUI-64 designator only');
+ sda_page(page([0x03, 0x08, "eui.0123456789ABCDEF\0\0\0\0"]));
+ ok(other('sda'), 'other LUN: a disk with an EUI name string only');
+ # the daemon names it, with an identity that its page does not confirm
+ $reply = { sda => { devt => '8:0', hctl => '0:0:0:0', init => 'ok', wwid => $W } };
+ ok(
+ !other('sda'),
+ 'other LUN: undecided if the daemon names the disk with an identity that did not resolve',
+ );
+ $reply = {};
+
+ sda_page($qemu);
+ $ids = PVE::Multipath::Identity::collect(all => 1);
+ put('bus/scsi/devices/0:0:0:0/block/sda/dev', "8:99\n");
+ ok(
+ !PVE::Multipath::Identity::path_is_other_lun($ids, 'sda', $W),
+ 'other LUN: undecided once the name denotes another device',
+ );
+ put('bus/scsi/devices/0:0:0:0/block/sda/dev', "8:0\n");
+ {
+ $ids = PVE::Multipath::Identity::collect(all => 1);
+ my $module = Test::MockModule->new('PVE::Multipath::Identity');
+ my $read = \&PVE::Multipath::Identity::file_get_contents;
+ my $change = 'sda';
+ $module->mock(
+ file_get_contents => sub {
+ my $page = $read->($_[0]);
+ put("bus/scsi/devices/$1/block/$change/dev", "8:99\n")
+ if $_[0] =~ m{/$change/device/vpd_pg83$}
+ && readlink("$sys/class/block/$change") =~ m{/devices/([^/]+)/block/};
+ return $page;
+ },
+ );
+ ok(
+ !PVE::Multipath::Identity::path_is_other_lun($ids, 'sda', $W),
+ 'other LUN: undecided if the device changes while its page is read',
+ );
+ put('bus/scsi/devices/0:0:0:0/block/sda/dev', "8:0\n");
+ $change = 'sdb';
+ $ids = PVE::Multipath::Identity::collect(all => 1);
+ ok(
+ !PVE::Multipath::Identity::path_is_other_lun($ids, 'sda', $W),
+ 'other LUN: undecided if the path of the LUN changes while its page is read',
+ );
+ put('bus/scsi/devices/7:0:0:0/block/sdb/dev', "8:16\n");
+ $module->unmock('file_get_contents');
+ $ids = PVE::Multipath::Identity::collect(all => 1);
+ ok(
+ PVE::Multipath::Identity::path_is_other_lun($ids, 'sda', $W),
+ 'other LUN: and decided again with both devices unchanged',
+ );
+ put('bus/scsi/devices/7:0:0:0/block/sdb/dev', "8:99\n");
+ ok(
+ !PVE::Multipath::Identity::path_is_other_lun($ids, 'sda', $W),
+ 'other LUN: undecided once the path of the LUN changed since an earlier answer',
+ );
+ put('bus/scsi/devices/7:0:0:0/block/sdb/dev', "8:16\n");
+ }
+ {
+ # a logical volume can hold a path next to its map
+ put('devices/virtual/block/dm-20/dm/uuid', "LVM-oldvg\n");
+ symlink(
+ "$sys/devices/virtual/block/dm-20",
+ "$sys/bus/scsi/devices/0:0:0:0/block/sda/holders/dm-20",
+ ) or die "$!\n";
+ hold('0:0:0:0', 'sda', 'dm-21', $W);
+ sda_page(page([0x01, 0x02, pack('H*', '0123456789abcdef')]));
+ $ids = PVE::Multipath::Identity::collect(all => 1);
+ is(
+ PVE::Multipath::Identity::path_wwid($ids, 'sda'),
+ $W,
+ 'holders: the map of a path that a logical volume holds as well',
+ );
+ # and in the other order, as the order of the holder links is the directory's
+ unlink("$sys/bus/scsi/devices/0:0:0:0/block/sda/holders/$_") for qw(dm-20 dm-21);
+ hold('0:0:0:0', 'sda', 'dm-21', $W);
+ symlink(
+ "$sys/devices/virtual/block/dm-20",
+ "$sys/bus/scsi/devices/0:0:0:0/block/sda/holders/dm-20",
+ ) or die "$!\n";
+ $ids = PVE::Multipath::Identity::collect(all => 1);
+ is(
+ PVE::Multipath::Identity::path_wwid($ids, 'sda'),
+ $W,
+ 'holders: the map of a path that a logical volume holds as well, linked the other way',
+ );
+ hold('0:0:0:0', 'sda', 'dm-22', '3600a09803832355367242f747a716c30');
+ $ids = PVE::Multipath::Identity::collect(all => 1);
+ is(PVE::Multipath::Identity::path_wwid($ids, 'sda'), undef, 'holders: none with two maps');
+ ok(
+ $ids->{paths}->{sda}->{claimed} && $ids->{paths}->{sda}->{unresolved},
+ 'holders: but claimed and unresolved',
+ );
+ ok(
+ !PVE::Multipath::Identity::path_is_other_lun($ids, 'sda', $W),
+ 'other LUN: undecided for a path that maps hold',
+ );
+ unlink("$sys/bus/scsi/devices/0:0:0:0/block/sda/holders/$_") for qw(dm-20 dm-21 dm-22);
+ }
+ {
+ # NAA format 5 has 8 bytes, so a 16-byte one cannot stand for the WWID
+ my $W5 = '35' . substr($W, 2);
+ hold('7:0:0:1', 'sdc', 'dm-9', $W5);
+ put('bus/scsi/devices/7:0:0:1/vpd_pg83', page(naa(substr($W5, 1))));
+ ok(!other('sda', $W5), 'other LUN: undecided if the NAA of the LUN is not a valid one');
+ unlink("$sys/bus/scsi/devices/7:0:0:1/block/sdc/holders/dm-9") or die "$!\n";
+ put('bus/scsi/devices/7:0:0:1/vpd_pg83', $vpd);
+ }
+
+ # only if a path of the LUN names its NAA in binary, as a name string can give the same WWID
+ unlink("$sys/bus/scsi/devices/7:0:0:0/vpd_pg83");
+ ok(!other('sda'), 'other LUN: undecided if no path of the LUN names its NAA');
+ put('bus/scsi/devices/7:0:0:0/vpd_pg83', page([0x03, 0x08, 'naa.' . uc(substr($W, 1))]));
+ ok(!other('sda'), 'other LUN: undecided if the LUN names its NAA as a SCSI name string only');
+
+ unlink("$sys/bus/scsi/devices/7:0:0:0/vpd_pg83");
+ remove_tree("$sys/bus/scsi/devices/0:0:0:0");
+ unlink("$sys/class/block/sda");
+}
+
# --- the iSCSI plugin ---
$ids = {
@@ -660,6 +854,26 @@ is_deeply(
is_deeply($list->(), ['0.0.3.scsi-3W3'], 'plain volumes: a LUN without aliases');
is($collected, 0, 'plain volumes: needs no identities');
+ my $device = {
+ map {
+ my $name = $_;
+ map { $_ => $name } $fixture->{$name}->[2]->@*
+ } keys %$fixture
+ };
+ my $diskmanage = Test::MockModule->new('PVE::Diskmanage');
+ $diskmanage->mock(
+ verify_blockdev_path => sub { '/dev/' . $device->{ $_[0] =~ s|^/dev/disk/by-id/||r } },
+ block_device_stamp => sub { '1:8:0' },
+ is_partition => sub { 0 },
+ );
+ for my $volname ($listed->@[0 .. 3]) {
+ my ($id) = $volname =~ m/^\d+\.\d+\.\d+\.(.+)$/;
+ eval { PVE::Diskmanage::block_device_reference("/dev/disk/by-id/$id") };
+ is($@, '', "plain volumes: $id passes the volume group creation identity check");
+ }
+ eval { PVE::Diskmanage::block_device_reference('/dev/disk/by-id/scsi-8naa.W0') };
+ like($@, qr/does not match its by-id name/, 'plain volumes: while an alias would not');
+
$iscsi->mock(
iscsi_session_device_map => $session_device_map,
load_stable_scsi_paths => sub { {} },
diff --git a/src/test/run_san_lun_scan_tests.pl b/src/test/run_san_lun_scan_tests.pl
index 1771a5c8..af8b4870 100755
--- a/src/test/run_san_lun_scan_tests.pl
+++ b/src/test/run_san_lun_scan_tests.pl
@@ -46,6 +46,9 @@ my sub sd {
# what the scan collects in which order
my @order;
my $diskmanage = Test::MockModule->new('PVE::Diskmanage');
+my $hint = { 'in-use' => 'no', 'in-use-by' => '' };
+my @hint_calls;
+my $by_id;
$diskmanage->mock(
get_disks => sub {
push @order, 'disks';
@@ -69,6 +72,8 @@ $diskmanage->mock(
};
},
get_nvme_path_status => sub { {} },
+ block_device_usage_hint => sub { push @hint_calls, [@_]; $hint },
+ canonical_blockdev_link => sub { $by_id },
udev_properties => sub {
my ($dev) = @_;
my $pv = $dev =~ m/^dm-(.+)$/ ? $maps->{$1}->{pv} : ($devices->{$dev} // {})->{pv};
@@ -196,7 +201,28 @@ my sub flagged {
{ '3W0' => { slaves => ['sdb', 'sdc'], pv => 'PV0' } },
);
is(scan()->{'3W0'}->{'paths-total'}, 2, 'map path count of the disk scan');
+@hint_calls = ();
my $res = scan();
+is($res->{sdd}->{'in-use'}, 'no', 'scan exposes metadata usage hint');
+is_deeply(
+ { map { $_->[0] => $_->[1] } @hint_calls },
+ {
+ '/dev/mapper/3W0' => { 'map-wwid' => '3W0' },
+ '/dev/sdd' => undef,
+ },
+ 'usage hint: a map also for its paths, a path without a map on its own',
+);
+ok(!exists($res->{sdd}->{'by-id'}), 'scan omits unverified by-id path');
+$hint = { 'in-use' => 'yes', 'in-use-by' => 'array held by vg-data' };
+$by_id = '/dev/disk/by-id/dm-uuid-mpath-3W0';
+my $busy = scan()->{'3W0'};
+is($busy->{'in-use'}, 'yes', 'scan propagates positive usage hint');
+is($busy->{'in-use-by'}, 'array held by vg-data', 'scan propagates user names');
+is($busy->{'by-id'}, $by_id, 'scan emits the verified canonical link');
+$hint = { 'in-use' => 'unknown', 'in-use-by' => 'check failed' };
+is(scan()->{sdd}->{'in-use'}, 'unknown', 'scan preserves failed-probe state');
+$hint = { 'in-use' => 'no', 'in-use-by' => '' };
+$by_id = undef;
is($res->{sdd}->{wwid}, '3W0', 'wwid: a path without a map has the identity of its LUN');
ok($derived_with && $derived_with == $collected, 'the storages derive from the same collection');
is_deeply([@order[0, 1]], ['identities', 'disks'], 'the identities are collected before the disks');
diff --git a/src/test/run_san_lvm_create_tests.pl b/src/test/run_san_lvm_create_tests.pl
index 8d6bbfdc..c41142da 100755
--- a/src/test/run_san_lvm_create_tests.pl
+++ b/src/test/run_san_lvm_create_tests.pl
@@ -7,7 +7,7 @@ use File::Basename qw(basename);
use File::Path qw(make_path);
use File::Temp qw(tempdir);
use JSON qw(encode_json);
-use Scalar::Util qw(weaken);
+use Scalar::Util qw(refaddr tainted weaken);
use Storable qw(dclone);
use Test::MockModule;
use Test::More;
@@ -16,20 +16,40 @@ use lib ('.', '..');
use PVE::API2::Disks::LVM;
use PVE::API2::Storage::Config;
+$ENV{PATH} = '/usr/sbin:/usr/bin:/sbin:/bin';
+delete @ENV{qw(IFS CDPATH ENV BASH_ENV)};
+
+my sub command_boundary {
+ my ($cmd) = @_;
+
+ # Exercise exec's taint check without letting any device command reach a real device.
+ PVE::Tools::run_command(['/usr/bin/true', @$cmd]) if ${^TAINT};
+}
+
my $disk = Test::MockModule->new('PVE::Diskmanage');
my $storage = Test::MockModule->new('PVE::Storage');
my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
+my $mp = Test::MockModule->new('PVE::Multipath');
+my $identity = Test::MockModule->new('PVE::Multipath::Identity');
+my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
my $rpc = Test::MockModule->new('PVE::RPCEnvironment');
-my $inotify = Test::MockModule->new('PVE::INotify');
my $tools = Test::MockModule->new('PVE::Tools');
-my $glob = \&PVE::Tools::dir_glob_foreach;
+my $inotify = Test::MockModule->new('PVE::INotify');
my $reference = \&PVE::Diskmanage::block_device_reference;
my $stamp = \&PVE::Diskmanage::block_device_stamp;
+my $ready = \&PVE::Diskmanage::assert_block_device_ready;
my $users = \&PVE::Diskmanage::block_device_users;
-my ($root, $resolved, $worker, $cfg, $handle, $local, $pv, $during_probe, $inventory_used);
-my (@locks, @writes);
+my $partitions = \&PVE::Diskmanage::wipe_partitions;
+my $san_identity = \&PVE::Storage::san_device_identity;
+my $storage_guard = \&PVE::Storage::assert_san_write_safe;
+my $canonical_link = \&PVE::Diskmanage::canonical_blockdev_link;
+my $glob = \&PVE::Tools::dir_glob_foreach;
+my ($root, $resolved, $worker, $cfg, $handle, $local, $partition, $pv, $during_probe, $after_write);
+my (@locks, @writes, @probes, $config_writes, $ids, $missing_links, $inventory_used);
+my ($config_reads, @guard_configs);
my $uuid = 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg';
-my $canonical = '/dev/mapper/lun';
+my $canonical = '/dev/disk/by-id/dm-uuid-mpath-test';
+my $nvme_id = 'eui.0123456789abcdef';
my sub put {
my ($path, $value) = @_;
my ($dir) = "$root/$path" =~ m{^(.*)/[^/]+$};
@@ -38,6 +58,13 @@ my sub put {
print $fh "$value\n";
close($fh) or die $!;
}
+my sub device {
+ my ($name, $number) = @_;
+ put("sys/class/block/$name/dev", $number);
+ put("sys/class/block/$name/size", '8192');
+ put("sys/class/block/$name/device/state", 'running');
+ make_path("$root/sys/class/block/$name/holders");
+}
my sub locked {
my ($name, $code) = @_;
push @locks, $name;
@@ -52,6 +79,7 @@ my sub written {
ok($handle && defined(fileno($handle)), 'production retains SAN handle through write')
if !$local;
push @writes, [$tool, $path];
+ $after_write->($tool) if $after_write;
}
{
@@ -63,10 +91,22 @@ my sub written {
$rpc->mock(get => sub { bless {}, 'SanLVMTestRPC' });
$inotify->mock(nodename => sub { 'n1' });
$disk->mock(
- verify_blockdev_path => sub { $_[0] =~ m{^/dev/mapper/} ? $resolved : $_[0] },
+ verify_blockdev_path => sub {
+ die "missing link\n" if $missing_links->{ $_[0] };
+ return '/dev/sda' if $_[0] eq '/dev/disk/by-id/scsi-test';
+ return $_[0] =~ m{^/dev/(?:disk|mapper)/} ? $resolved : $_[0];
+ },
+ canonical_blockdev_link => sub { $canonical_link->($_[0], $_[1], $_[2], "$root/sys") },
is_san_device => sub { !$local },
is_multipath_device => sub { !$local && $_[0] =~ /dm-/ },
- is_partition => sub { 0 },
+ is_partition => sub { $partition && $_[0] eq $resolved },
+ get_blockdev => sub { '/dev/sda' },
+ block_device_reference => sub { $reference->($_[0], $_[1], "$root/sys") },
+ block_device_stamp => sub { $stamp->($_[0], "$root/sys") },
+ assert_block_device_ready => sub {
+ push @probes, 'readiness';
+ $ready->($_[0], { sysfs => "$root/sys", procfs => "$root/proc" });
+ },
block_device_users => sub {
$users->($_[0], { %{ $_[1] // {} }, sysfs => "$root/sys", procfs => "$root/proc" });
},
@@ -75,9 +115,10 @@ $disk->mock(
return PVE::Tools::file_read_firstline(
"$root/sys/class/block/" . basename($path) . '/dev');
},
- block_device_reference => sub { $reference->($_[0], $_[1], "$root/sys") },
- block_device_stamp => sub { $stamp->($_[0], "$root/sys") },
+ wipe_partitions => sub { $partitions->($_[0], "$root/sys") },
open_block_device => sub {
+ push @probes, 'open';
+ is($_[1], $stamp->($_[0], "$root/sys"), 'open receives current instance stamp');
open(my $opened, '+<', "$root/handle") or die $!;
$handle = $opened;
weaken($handle);
@@ -85,9 +126,50 @@ $disk->mock(
},
is_mounted => sub { undef },
has_holder => sub { undef },
- assert_disk_unused => sub { die "device '$_[0]' is already in use\n" if $inventory_used },
+ assert_disk_unused => sub {
+ push @probes, 'inventory';
+ die "device '$_[0]' is already in use\n" if $inventory_used;
+ },
locked_disk_action => sub { locked('disk', $_[0]) },
+ get_partnum => sub { 1 },
+ get_disks => sub {
+ push @probes, 'disk';
+ return { sda => { gpt => 1 } };
+ },
+ run_command => sub {
+ my ($cmd) = @_;
+ command_boundary($cmd);
+ die "unexpected command @$cmd\n" if $cmd->[0] ne 'sgdisk';
+ written('parttype', $cmd->[-1]);
+ },
udevadm_trigger => sub { },
+ get_nvme_path_status => sub { { nvme900n1 => { active => 1 } } },
+);
+$tools->mock(
+ dir_glob_foreach => sub {
+ my $path = $_[0];
+ $path =~ s{^/sys}{$root/sys};
+ $glob->($path, $_[1], $_[2]);
+ },
+);
+$identity->mock(
+ collect => sub { push @probes, 'identity'; $ids },
+ path_wwid => sub { $_[0]->{paths}->{ $_[1] }->{wwid} },
+);
+$iscsi->mock(iscsi_session_device_map => sub { {} });
+$mp->mock(
+ get_maps => sub {
+ return [
+ map { {
+ sysfs => "dm-$_",
+ wwid => 'test',
+ 'path-groups' => [{
+ paths =>
+ [{ dev => 'sda', 'dm-state' => 'active', 'check-state' => 'ready' }],
+ }],
+ } } (900, 901)
+ ];
+ },
);
$storage->mock(
lock_storage_config => sub {
@@ -98,18 +180,45 @@ $storage->mock(
);
locked('storage', $_[0]);
},
- config =>
- sub { is($locks[-1], 'storage', 'configuration read under storage lock'); dclone($cfg) },
- write_config => sub { },
+ config => sub {
+ is($locks[-1], 'storage', 'configuration read under storage lock');
+ $config_reads++;
+ dclone($cfg);
+ },
+ assert_san_write_safe => sub {
+ push @guard_configs, refaddr($_[0]);
+ $storage_guard->(@_);
+ },
+ write_config => sub { $config_writes++ },
path => sub { $canonical },
activate_storage => sub { },
activate_volumes => sub { },
-);
-$tools->mock(
- dir_glob_foreach => sub {
- my $path = $_[0];
- $path =~ s{^/sys}{$root/sys};
- $glob->($path, $_[1], $_[2]);
+ san_device_identity => sub { $san_identity->($_[0], $_[1], "$root/sys", $_[3]) },
+ run_command => sub {
+ my ($cmd, %opt) = @_;
+ command_boundary($cmd);
+ push @probes, $cmd->[0];
+ if ($cmd->[0] eq 'wipefs') {
+ $opt{outfunc}->(encode_json({
+ signatures => $pv eq 'blank'
+ ? []
+ : [{ type => 'LVM2_member', uuid => $uuid }],
+ }));
+ } elsif ($cmd->[0] eq '/sbin/pvs') {
+ my $rows =
+ $pv eq 'blank'
+ ? []
+ : [{
+ pv_name => $resolved,
+ pv_uuid => $uuid,
+ vg_name => $pv eq 'same' ? 'vg' : '',
+ pv_in_use => $pv eq 'same' ? '1' : '0',
+ }];
+ $opt{outfunc}->(encode_json({ report => [{ pv => $rows }] }));
+ $during_probe->('storage') if $during_probe;
+ } else {
+ die "unexpected storage command @$cmd\n";
+ }
},
);
$lvm->mock(
@@ -120,18 +229,23 @@ $lvm->mock(
},
abs_path => sub { $_[0] },
clear_first_sector => sub { written('clear', $_[0]) },
- lvm_vgs => sub { $pv eq 'same' ? { vg => {} } : {} },
+ lvm_vgs => sub { push @probes, 'vgs'; $pv eq 'same' ? { vg => {} } : {} },
assert_discard_supported => sub { },
+ assert_device_discard_supported => sub { push @probes, 'discard' },
run_command => sub {
my ($cmd, %opt) = @_;
+ command_boundary($cmd);
if ($cmd->[0] eq 'wipefs') {
+ push @probes, 'wipefs';
$opt{outfunc}
->(encode_json({ signatures => $pv eq 'blank' ? [] : [{ type => 'LVM2_member' }] })
);
} elsif ($cmd->[0] eq '/usr/bin/file') {
+ push @probes, 'file';
$opt{outfunc}->("$cmd->[-1]: " . ($pv eq 'blank' ? 'data' : 'LVM2 PV'));
- $during_probe->() if $during_probe;
+ $during_probe->('lvm') if $during_probe;
} elsif ($cmd->[0] eq '/sbin/pvs') {
+ push @probes, 'pvs';
$opt{outfunc}->("$resolved:8192:"
. ($pv eq 'same' ? 'vg' : '')
. ":$uuid:"
@@ -144,29 +258,45 @@ $lvm->mock(
);
my sub reset_case {
$root = tempdir(CLEANUP => 1);
- for my $name ('dm-900', 'vdz') {
- put("sys/class/block/$name/dev", $name eq 'vdz' ? '252:0' : '253:900');
- put("sys/class/block/$name/size", '8192');
- make_path("$root/sys/class/block/$name/holders");
+ for my $n (900, 901, 902) {
+ device("dm-$n", "253:$n");
+ put("sys/class/block/dm-$n/dm/uuid", $n == 902 ? 'mpath-other' : 'mpath-test');
+ make_path("$root/sys/class/block/dm-$n/slaves");
+ symlink('../../sda', "$root/sys/class/block/dm-$n/slaves/sda") or die $!;
}
- put('sys/class/block/dm-900/dm/uuid', 'mpath-test');
- put('sys/class/block/dm-900/slaves/sda/device/state', 'running');
+ device('sda', '8:0');
+ device('sdb', '8:16');
+ device('vdz', '252:0');
+ device('nvme900n1', '259:900');
+ put('sys/class/block/nvme900n1/wwid', $nvme_id);
+ put('sys/class/block/nvme900n1/device/subsysnqn', 'nqn.test');
put('proc/self/mountinfo', '1 0 0:1 / / rw - rootfs rootfs rw');
put('proc/swaps', "Filename\tType\tSize\tUsed\tPriority");
put('handle', 'retained descriptor fixture');
- ($resolved, $local, $pv) = ('/dev/dm-900', 0, 'blank');
- ($during_probe, $worker, $handle) = (undef, undef, undef);
+ ($resolved, $local, $partition, $pv) = ('/dev/dm-900', 0, 0, 'blank');
+ ($during_probe, $after_write, $worker, $handle) = (undef, undef, undef, undef);
$cfg =
{ ids => { base => { type => 'iscsi', target => 'iqn.test', content => { none => 1 } } } };
+ $ids = {
+ paths => { sda => { wwid => 'test' }, sdb => { wwid => 'test' } },
+ luns => { lun => { wwid => 'test', target => 'iqn.test' } },
+ };
@writes = ();
+ @probes = ();
@locks = ();
+ $config_writes = 0;
+ $config_reads = 0;
+ @guard_configs = ();
+ $missing_links = {};
$inventory_used = 0;
+ $canonical = '/dev/disk/by-id/dm-uuid-mpath-test';
}
my sub queue {
- PVE::API2::Disks::LVM->create({ node => 'n1', name => 'vg', device => $_[0] // $canonical });
+ my ($path) = @_;
+ PVE::API2::Disks::LVM->create({ node => 'n1', name => 'vg', device => $path // $canonical });
}
my sub run_route {
- my ($route) = @_;
+ my ($route, $options) = @_;
eval {
if ($route eq 'disk') {
queue();
@@ -176,27 +306,116 @@ my sub run_route {
type => 'lvm',
storage => 'new',
vgname => 'vg',
- base => 'base:0.0.1.lun',
+ %{ $options // {} },
+ base => 'base:0.0.1.dm-uuid-mpath-test',
});
}
};
return $@;
}
-for my $route ('disk', 'base') {
- reset_case();
+my sub raw_user {
+ my ($dev) = @_;
make_path("$root/proc/999999/fd");
- symlink('/dev/dm-900', "$root/proc/999999/fd/3") or die $!;
+ symlink("/dev/$dev", "$root/proc/999999/fd/3") or die $!;
put('proc/999999/comm', 'qemu');
- like(
- run_route($route),
- qr/cannot use.*qemu .* has dm-900 open/,
- "$route raw map opener refused",
- );
- is_deeply(\@writes, [], 'raw user receives no destructive command');
+}
+
+if (${^TAINT}) {
+ reset_case();
+ for my $attr ('dm-900/dm/uuid', 'nvme900n1/wwid') {
+ my $value = PVE::Tools::file_read_firstline("$root/sys/class/block/$attr");
+ ok(tainted($value), "$attr fixture retains taint from its file read");
+ eval { command_boundary(['/sbin/pvcreate', $value]) };
+ like($@, qr/Insecure dependency/, 'benign exec boundary rejects an uncaptured value');
+ }
+}
+
+for my $route ('disk', 'base') {
+ for my $transport ($route eq 'disk' ? ('iscsi', 'nvmeof') : ('iscsi')) {
+ for my $restriction ({}, { disable => 1 }, { nodes => { elsewhere => 1 } }) {
+ reset_case();
+ if ($transport eq 'nvmeof') {
+ $resolved = '/dev/nvme900n1';
+ $canonical = "/dev/disk/by-id/nvme-$nvme_id";
+ $cfg->{ids}->{base} =
+ { type => 'nvmeof', subsysnqn => 'nqn.test', content => { none => 1 } };
+ }
+ $cfg->{ids}->{images} =
+ { %{ $cfg->{ids}->{base} }, content => { images => 1 }, %$restriction };
+ my $kind = $transport eq 'nvmeof' ? 'subsystem' : 'target';
+ my $prefix = $route eq 'disk' ? "cannot use '$resolved' - " : '';
+ is(
+ run_route($route),
+ "${prefix}storage 'images' serves disk images from this $kind\n",
+ "$route $transport image owner refused",
+ );
+ is_deeply(\@writes, [], 'signatureless image volume receives no destructive command');
+ is($config_writes, 0, 'refusal does not publish storage');
+ $canonical = '/dev/disk/by-id/dm-uuid-mpath-test';
+ }
+ }
+ for my $dev ('dm-900', 'sda', 'sdb') {
+ reset_case();
+ raw_user($dev);
+ like(
+ run_route($route),
+ qr/cannot use.*qemu .* has $dev open/,
+ "$route raw map, member or same-identity sibling opener refused",
+ );
+ is_deeply(\@writes, [], 'raw user receives no destructive command');
+ }
reset_case();
unlink("$root/proc/swaps") or die $!;
like(run_route($route), qr{can't open '.*/proc/swaps'}, "$route inconclusive users refuse");
is_deeply(\@writes, [], 'inconclusive users receive no destructive command');
+ for my $dev ('sda', 'sdb') {
+ for my $probe_number (1 .. 3) {
+ reset_case();
+ my $probes = 0;
+ $during_probe = sub {
+ raw_user($dev) if $_[0] eq 'storage' && ++$probes == $probe_number;
+ };
+ like(
+ run_route($route),
+ qr/cannot use.*qemu .* has $dev open/,
+ "$route opener during storage probe $probe_number refuses the next write",
+ );
+ is(scalar(@writes), $probe_number - 1, 'only writes preceding the opener occurred');
+ is_deeply(\@writes, [], 'opener during first ownership probe causes zero writes')
+ if $probe_number == 1;
+ }
+ }
+ for my $probe ('lvm', 'storage') {
+ reset_case();
+ $during_probe = sub { $resolved = '/dev/dm-902' if $_[0] eq $probe };
+ like(
+ run_route($route),
+ qr/identity does not match/,
+ "$route replacement after $probe probe refused",
+ );
+ is_deeply(\@writes, [], 'replacement after probing receives no destructive command');
+ }
+ if ($route eq 'disk') {
+ reset_case();
+ $resolved = '/dev/nvme900n1';
+ $canonical = "/dev/disk/by-id/nvme-$nvme_id";
+ $cfg->{ids}->{base} =
+ { type => 'nvmeof', subsysnqn => 'nqn.test', content => { none => 1 } };
+ raw_user('nvme900n1');
+ like(
+ run_route($route),
+ qr/cannot use.*qemu .* has nvme900n1 open/,
+ "$route raw namespace refuses",
+ );
+ is_deeply(\@writes, [], 'raw namespace receives no destructive command');
+ unlink("$root/proc/999999/fd/3") or die $!;
+ is(run_route($route), '', "$route closed namespace permits initialization");
+ is(scalar(@writes), 3, 'closed namespace gets all initialization writes');
+ is($writes[1]->[1], $canonical, 'PV operand encodes namespace identity');
+ is($writes[2]->[1], $canonical, 'VG operand encodes namespace identity');
+ $canonical = '/dev/disk/by-id/dm-uuid-mpath-test';
+
+ }
for my $kind ('blank', 'orphan', 'same') {
reset_case();
@@ -224,8 +443,67 @@ for my $route ('disk', 'base') {
'only the base-route same-VG retry succeeds without writes',
);
}
+ if ($kind ne 'same') {
+ is(
+ $config_reads,
+ $route eq 'disk' ? 1 : 2,
+ 'guard uses one snapshot; base API retains its separate pre-hook config read',
+ );
+ is(scalar(@guard_configs), 3, 'ownership is checked before every write');
+ is_deeply(
+ \@guard_configs,
+ [($guard_configs[0]) x 3],
+ 'every guard check receives the same locked configuration snapshot',
+ );
+ }
+ ok(!defined($handle), 'production releases the retained handle after completion');
+ like($writes[0]->[1], qr{^/proc/\d+/fd/\d+$}, 'sector clear uses retained descriptor')
+ if @writes;
}
}
+
+reset_case();
+queue();
+$resolved = '/dev/dm-901';
+eval { $worker->() };
+is($@, '', 'identity-preserving queued renumbering succeeds');
+is($writes[1]->[1], $canonical, 'LVM uses the verified identity-encoding name after renumbering');
+is($writes[2]->[1], $canonical, 'VG creation also uses the verified stable name');
+for my $path ($canonical, '/dev/mapper/friendly', '/dev/dm-900') {
+ reset_case();
+ queue($path);
+ if ($path eq '/dev/dm-900') {
+ put('sys/class/block/dm-900/dm/uuid', 'mpath-other');
+ } else {
+ $resolved = '/dev/dm-902';
+ }
+ eval { $worker->() };
+ like(
+ $@,
+ qr/cannot use.*identity (?:changed|does not match)/,
+ 'moved by-id, rebound alias or recycled dm number refuses',
+ );
+ is_deeply(\@writes, [], 'queued identity replacement receives no destructive command');
+}
+for my $tool ('clear', '/sbin/pvcreate') {
+ reset_case();
+ $after_write = sub { $resolved = '/dev/dm-902' if $_[0] eq $tool };
+ like(
+ run_route('disk'),
+ qr/identity does not match/,
+ "replacement after $tool prevents next write",
+ );
+ is(
+ scalar(@writes),
+ $tool eq 'clear' ? 1 : 2,
+ 'only preceding writes to the checked instance occurred',
+ );
+}
+reset_case();
+$during_probe = sub { put('sys/class/block/dm-900/dev', '253:999') };
+like(run_route('disk'), qr/device topology changed/, 'instance change after probing refuses');
+is_deeply(\@writes, [], 'changed instance receives no destructive command');
+
reset_case();
$local = 1;
$resolved = '/dev/vdz';
@@ -241,20 +519,93 @@ eval { $worker->() };
like($@, qr/cannot use.*device topology changed/, 'queued local-to-SAN transition refuses');
is_deeply(\@writes, [], 'transition does not bypass storage lock');
+reset_case();
+$local = 1;
+$resolved = '/dev/sda1';
+$partition = 1;
+device('sda1', '8:1');
+put('sys/class/block/sda1/partition', '1');
+queue($resolved);
+eval { $worker->() };
+is($@, '', 'local partition initialization still works without quorum');
+is_deeply(
+ \@writes,
+ [
+ ['parttype', '/dev/sda'],
+ ['clear', $resolved],
+ ['/sbin/pvcreate', $resolved],
+ ['/sbin/vgcreate', $resolved],
+ ],
+ 'local partitions retain unguarded type changes and normal LVM operands',
+);
+
for my $route ('disk', 'base') {
reset_case();
- $during_probe = sub { put('sys/class/block/dm-900/dev', '253:999') };
- like(run_route($route), qr/device identity changed/, "$route changed instance refuses");
- is_deeply(\@writes, [], 'replacement after probing receives no destructive command');
+ $canonical = '/dev/mapper/friendly';
+ is(run_route($route), '', "$route converts friendly alias to canonical operand");
+ is($writes[1]->[1], '/dev/disk/by-id/dm-uuid-mpath-test',
+ 'PV operand encodes the map identity');
+ is($writes[2]->[1], '/dev/disk/by-id/dm-uuid-mpath-test',
+ 'VG operand encodes the map identity');
reset_case();
- is(run_route($route), '', "$route initializes unchanged instance");
- like($writes[0]->[1], qr{^/proc/\d+/fd/\d+$}, 'sector clear uses retained descriptor');
- ok(!defined($handle), 'production releases the retained handle after completion');
+ $canonical = '/dev/mapper/friendly';
+ $missing_links->{'/dev/disk/by-id/dm-uuid-mpath-test'} = 1;
+ is(
+ run_route($route),
+ "cannot use '/dev/dm-900' - cannot verify a stable name under /dev/disk/by-id\n",
+ "$route refuses missing canonical link",
+ );
+ is_deeply(\@writes, [], 'missing stable operand refuses before sector clearing');
}
+
+for my $route ('disk', 'base') {
+ for my $kind ($route eq 'disk' ? ('scsi', 'nvme', 'map') : ('scsi', 'map')) {
+ reset_case();
+ if ($kind eq 'map') {
+ device('dm-903', '253:903');
+ put('sys/class/block/dm-903/dm/uuid', 'part2-mpath-test');
+ $resolved = '/dev/dm-903';
+ $canonical = '/dev/mapper/friendly';
+ } else {
+ my $name = $kind eq 'scsi' ? 'sda1' : 'nvme900n1p1';
+ device($name, $kind eq 'scsi' ? '8:1' : '259:901');
+ put("sys/class/block/$name/partition", '1');
+ $resolved = "/dev/$name";
+ $partition = 1;
+ $canonical =
+ $kind eq 'scsi'
+ ? '/dev/disk/by-id/scsi-test-part1'
+ : "/dev/disk/by-id/nvme-$nvme_id-part1";
+ $cfg->{ids}->{base} =
+ { type => 'nvmeof', subsysnqn => 'nqn.test', content => { none => 1 } }
+ if $kind eq 'nvme';
+ }
+ is(
+ run_route($route, { 'on-volume-remove' => 'discard=1' }),
+ "cannot use '$resolved' - create the volume group on the whole LUN\n",
+ "$route refuses a $kind SAN partition",
+ );
+ is_deeply(
+ \@probes,
+ [],
+ 'partition refused before identity, readiness, inventory or content probes',
+ );
+ is_deeply(\@writes, [], 'partition refusal issues no destructive command');
+ is($config_writes, 0, 'partition refusal writes no storage configuration');
+ }
+}
+
reset_case();
queue();
-put('sys/class/block/dm-900/dev', '253:999');
+@probes = ();
+put('sys/class/block/dm-900/dm/uuid', 'part2-mpath-test');
eval { $worker->() };
-like($@, qr/device identity changed/, 'queued replacement refuses');
-is_deeply(\@writes, [], 'queued replacement receives no destructive command');
+is(
+ $@,
+ "cannot use '/dev/dm-900' - create the volume group on the whole LUN\n",
+ 'queued whole-LUN reference replaced by a partition refuses',
+);
+is_deeply(\@probes, [], 'queued partition refused before probes');
+is_deeply(\@writes, [], 'queued partition receives no destructive command');
+
done_testing();
diff --git a/src/test/run_san_wipe_tests.pl b/src/test/run_san_wipe_tests.pl
new file mode 100755
index 00000000..0779bc85
--- /dev/null
+++ b/src/test/run_san_wipe_tests.pl
@@ -0,0 +1,673 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use File::Path qw(make_path);
+use File::Temp qw(tempdir);
+use FindBin qw($Bin);
+use JSON;
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Storage;
+
+my $storage = Test::MockModule->new('PVE::Storage');
+my $disk = Test::MockModule->new('PVE::Diskmanage');
+my ($ready, $readiness_error);
+$disk->mock(
+ assert_block_device_ready => sub {
+ die "no usable path\n" if $readiness_error;
+ $ready = 1;
+ },
+);
+my $identity = Test::MockModule->new('PVE::Multipath::Identity');
+my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
+$iscsi->mock(iscsi_session_device_map => sub { {} });
+$identity->mock(collect => sub { { luns => {} } });
+my ($report, $failure, $cfg, $key, $subsystem, $path, $signatures, $partitions, $pv_identities);
+my $pv_uuid = 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg';
+my (@probed, @resolved, @writes);
+$storage->mock(
+ san_device_identity => sub {
+ push @resolved, $_[0];
+ # a base path taken in list context would shift the arguments
+ die "san_device_identity called with " . scalar(@_) . " arguments\n" if @_ > 4;
+ # as verify_blockdev_path() does for a link that is not there
+ die "failed to get absolute path to $_[0]\n" if $_[0] eq '/dev/missing';
+ return {
+ key => $pv_identities->{ $_[0] } // ($_[0] eq '/dev/other' ? 'scsi:other' : $key),
+ targets => { 'iqn.test' => 1 },
+ subsystem => $subsystem,
+ };
+ },
+ # as the real one: path, owner and type in list context
+ path => sub { wantarray ? ($path, undef, 'images') : $path },
+ run_command => sub {
+ my ($cmd, %opt) = @_;
+ ok($ready, 'readiness precedes every device-reading probe');
+ is($opt{timeout}, 10, 'PV probe bounded');
+ if ($cmd->[0] eq 'wipefs') {
+ push @probed, $cmd->[-1];
+ die "wipefs failed\n" if $failure eq 'wipefs';
+ $opt{outfunc}->($signatures->{ $cmd->[-1] } // '{"signatures":[]}');
+ } elsif ($cmd->[0] eq '/sbin/pvs') {
+ is_deeply(
+ $cmd,
+ [
+ '/sbin/pvs',
+ '--nohints',
+ '--readonly',
+ '--binary',
+ '--reportformat',
+ 'json',
+ '--options',
+ 'pv_name,vg_name,pv_uuid,pv_in_use',
+ ],
+ 'full hint-free read-only PV report includes independent use flag',
+ );
+ die "pvs failed\n" if $failure;
+ $opt{outfunc}->($report);
+ } else {
+ push @writes, $cmd;
+ die "unexpected command @$cmd\n";
+ }
+ },
+);
+my sub reset_case {
+ $report = encode_json({ report => [{ pv => [] }] });
+ $cfg = { ids => {} };
+ $failure = 0;
+ $key = 'scsi:test';
+ $subsystem = undef;
+ $path = '/dev/alias';
+ $signatures = {};
+ $partitions = [];
+ $pv_identities = {};
+ @probed = ();
+ @resolved = ();
+ @writes = ();
+ ($ready, $readiness_error) = (0, 0);
+}
+my sub check {
+ eval { PVE::Storage::assert_san_write_safe($cfg, '/dev/dm-0', $partitions) };
+ return $@;
+}
+reset_case();
+is(check(), '', 'unreferenced LUN allowed');
+for my $type ('iscsi', 'iscsidirect', 'zfs', 'vendor') {
+ reset_case();
+ $cfg->{ids} = {
+ base => { type => 'iscsi', target => 'iqn.test', content => { none => 1 } },
+ images => {
+ type => $type,
+ target => 'iqn.test',
+ content => { images => 1 },
+ disable => 1,
+ nodes => { elsewhere => 1 },
+ },
+ };
+ like(check(), qr/storage 'images' serves disk images/, "$type conflicting reference found");
+}
+for my $base ('0.0.1.scsi-alias', '0.0.1.wwn-0x123-part1') {
+ reset_case();
+ $cfg->{ids}->{base} = { type => 'iscsi' };
+ $cfg->{ids}->{lvm} = { type => 'lvm', base => "base:$base", disable => 1 };
+ like(check(), qr/uses this LUN as its base volume/, "$base compared by resolved identity");
+ $path = '/dev/other';
+ is(check(), '', 'different LUN identity allowed');
+ $path = '/dev/missing';
+ like(
+ check(),
+ qr/cannot check storage 'lvm', its base volume 'base:\Q$base\E' is not visible/,
+ 'unresolvable alias refuses with storage context',
+ );
+}
+for my $case (
+ ['iscsi', '0.0.1.dm-uuid-mpath-test', 'scsi:test'],
+ ['iscsi', '0.0.1.dm-uuid-part2-mpath-test', 'scsi:test'],
+) {
+ reset_case();
+ my ($type, $base, $identity) = @$case;
+ $cfg->{ids}->{base} = { type => $type };
+ $cfg->{ids}->{lvm} = { type => 'lvm', base => "base:$base", disable => 1 };
+ $path = '/dev/missing';
+ $key = $identity;
+ like(check(), qr/uses this LUN as its base volume/, "$base matched without resolution");
+ $key = 'scsi:different';
+ is(check(), '', "$base: genuinely different invisible LUN allowed");
+}
+reset_case();
+$cfg->{ids}->{base} = { type => 'iscsi' };
+$cfg->{ids}->{lvm} = { type => 'lvm', base => 'base:dm-uuid-mpath-test' };
+like(
+ check(),
+ qr/cannot check base volume 'base:dm-uuid-mpath-test' of storage 'lvm'/,
+ 'plugin parser must accept the volume name',
+);
+for my $alias (
+ 'prefix-dm-uuid-mpath-test', 'nvme-eui.0123456789abcdef-extra', 'nvme-model_serial',
+) {
+ ok(!PVE::Diskmanage::block_identity_from_id($alias), "$alias is not a canonical identity");
+}
+reset_case();
+$cfg->{ids}->{lvm} = { type => 'lvm', vgname => 'vg', nodes => { elsewhere => 1 } };
+$report = encode_json({
+ report => [{
+ pv =>
+ [{ pv_name => '/dev/alias', vg_name => 'vg', pv_uuid => $pv_uuid, pv_in_use => '1' }],
+ }],
+});
+like(check(), qr/volume group 'vg' is used by storage 'lvm'/, 'VG referenced elsewhere refuses');
+$cfg = { ids => {} };
+is(check(), '', 'unreferenced old VG allowed');
+$failure = 1;
+like(check(), qr/pvs failed/, 'failed PV probe refuses');
+$failure = 0;
+$report = '{}';
+like(check(), qr/malformed pvs/, 'malformed PV report refuses');
+
+reset_case();
+$cfg->{ids}->{lvm} = { type => 'lvmthin', vgname => 'vg', disable => 1 };
+$report = encode_json({
+ report => [{
+ pv =>
+ [{ pv_name => '/dev/other', vg_name => 'vg', pv_uuid => $pv_uuid, pv_in_use => '1' }],
+ }],
+});
+is(check(), '', 'different LUN without a matching PV UUID allowed');
+
+for my $candidate ('/dev/dm-0', '/dev/dm-1', '/dev/sda1') {
+ $partitions = ['dm-1', 'sda1'];
+ $signatures = {
+ $candidate =>
+ encode_json({ signatures => [{ type => 'LVM2_member', uuid => $pv_uuid =~ s/-//gr }] }),
+ };
+ like(
+ check(),
+ qr/volume group 'vg' is used by storage 'lvm'/,
+ "$candidate duplicate UUID refuses even when LVM lists another LUN",
+ );
+}
+is_deeply(
+ [@probed[-3 .. -1]],
+ ['/dev/dm-0', '/dev/dm-1', '/dev/sda1'],
+ 'parent and every destroyed partition directly probed',
+);
+for my $bad (
+ '{}',
+ '{"signatures":[{}]}',
+ '{"signatures":[{"type":"LVM2_member"}]}',
+ '{"signatures":[{"type":"LVM2_member","uuid":"bad"}]}',
+) {
+ $signatures = { '/dev/dm-0' => $bad };
+ like(check(), qr/malformed wipefs|invalid PV UUID/, 'inconclusive direct probe refuses');
+}
+$signatures = {};
+$failure = 'wipefs';
+like(check(), qr/wipefs failed/, 'direct probe failure refuses');
+$failure = 0;
+$report = encode_json({
+ report => [{ pv => [{ pv_name => '/dev/other', vg_name => 'vg', pv_in_use => '1' }] }] });
+like(check(), qr/invalid PV UUID/, 'configured VG inventory requires PV UUIDs');
+
+reset_case();
+$signatures = {
+ '/dev/dm-0' => encode_json({ signatures => [{ type => 'LVM2_member', uuid => $pv_uuid }] }),
+};
+like(check(), qr/LVM does not report its physical volume/, 'missing candidate UUID refuses');
+my $candidate = { pv_name => '/dev/alias', vg_name => '', pv_uuid => $pv_uuid, pv_in_use => '0' };
+$report = encode_json({ report => [{ pv => [$candidate] }] });
+is(check(), '', 'positively unused orphan allowed');
+
+for my $used ('1', '', 'unknown') {
+ $candidate->{pv_in_use} = $used;
+ $report = encode_json({ report => [{ pv => [$candidate] }] });
+ like(
+ check(),
+ qr/belongs to a volume group whose metadata is not visible/,
+ "empty VG with use flag '$used' refuses missing or damaged metadata",
+ );
+}
+$candidate->{vg_name} = 'foreign';
+$candidate->{pv_in_use} = '1';
+$cfg->{ids}->{foreign} = { type => 'lvm', vgname => 'foreign', nodes => { elsewhere => 1 } };
+$report = encode_json({ report => [{ pv => [$candidate] }] });
+like(check(), qr/volume group 'foreign' is used by storage/, 'unhinted PV reference refuses');
+
+subtest 'malformed multi-record reports are refused' => sub {
+ reset_case();
+ $signatures->{'/dev/dm-0'} = encode_json({
+ signatures => [{ type => 'LVM2_member', uuid => 'bad' }, {}],
+ });
+ ok(check(), 'invalid UUID and malformed signature cannot pass');
+ is_deeply(\@writes, [], 'no writes after malformed signatures');
+
+ reset_case();
+ $report = encode_json({
+ report => [{
+ pv => [
+ { pv_name => '/dev/dm-0', vg_name => '', pv_uuid => 'bad', pv_in_use => '0' },
+ {},
+ ],
+ }],
+ });
+ ok(check(), 'invalid UUID and malformed PV cannot pass');
+ is_deeply(\@writes, [], 'no writes after malformed PV report');
+};
+
+subtest 'captured no-metadata PV and unrelated orphan' => sub {
+ my $capture = PVE::Tools::file_get_contents("$Bin/lvm-fixtures/lab1-nomda-orphan.txt");
+ my ($json) = $capture =~ m/\A== devices B,C\n(.*?)^== full nohints /ms;
+ die "missing captured JSON report\n" if !defined($json);
+ my $captured = decode_json($json);
+ is(scalar($captured->{log}->@*), 2, 'both captured log warnings remain in the report');
+ is($_->{log_type}, 'warn', 'captured log entry is a warning') for $captured->{log}->@*;
+ for my $pv ($captured->{report}->[0]->{pv}->@*) {
+ reset_case();
+ $report = $json;
+ $pv_identities =
+ { map { $_->{pv_name} => "scsi:$_->{pv_uuid}" } $captured->{report}->[0]->{pv}->@* };
+ $key = $pv_identities->{ $pv->{pv_name} };
+ $signatures->{'/dev/dm-0'} = encode_json({
+ signatures => [{ type => 'LVM2_member', uuid => $pv->{pv_uuid} }],
+ });
+ if ($pv->{vg_name} eq '[unknown]') {
+ like(
+ check(),
+ qr/physical volume belongs to a volume group whose metadata is not visible/,
+ 'captured no-metadata PV refused specifically, not as a malformed report',
+ );
+ is_deeply(
+ \@resolved,
+ ['/dev/dm-0'],
+ 'captured candidate UUID precedes alias resolution',
+ );
+ } else {
+ is(check(), '', 'captured orphan passes despite the unrelated no-metadata PV');
+ is(
+ scalar(@resolved),
+ 2,
+ 'unrelated no-metadata PV identity is checked before skipping',
+ );
+ }
+ is_deeply(\@writes, [], 'captured report guard performs no writes');
+ }
+};
+
+subtest 'unknown VG membership is isolated without orphan assumptions' => sub {
+ my $other_uuid = 'hhhhhh-iiii-jjjj-kkkk-llll-mmmm-nnnnnn';
+ for my $vg ('', '[unknown]') {
+ for my $flag ('1', '', 'unknown', $vg ? ('0') : ()) {
+ for my $alias ('/dev/alias', '/dev/other', '/dev/missing') {
+ reset_case();
+ $signatures->{'/dev/dm-0'} = encode_json({
+ signatures => [{ type => 'LVM2_member', uuid => $pv_uuid }],
+ });
+ $report = encode_json({
+ report => [{
+ pv => [{
+ pv_name => $alias,
+ vg_name => $vg,
+ pv_uuid => $pv_uuid,
+ pv_in_use => $flag,
+ }],
+ }],
+ });
+ like(
+ check(),
+ qr/physical volume belongs to a volume group whose metadata is not visible/,
+ "VG '$vg', flag '$flag', alias '$alias': UUID proves candidate membership",
+ );
+ is_deeply(\@resolved, ['/dev/dm-0'], 'matching UUID precedes alias resolution');
+ is_deeply(\@writes, [], 'refused candidate causes no writes');
+ }
+ }
+ }
+ for my $alias ('/dev/alias', '/dev/sda1', '/dev/other', '/dev/missing') {
+ reset_case();
+ $signatures->{'/dev/dm-0'} = encode_json({
+ signatures => [{ type => 'LVM2_member', uuid => $pv_uuid }],
+ });
+ $report = encode_json({
+ report => [{
+ pv => [
+ {
+ pv_name => '/dev/dm-0',
+ vg_name => '',
+ pv_uuid => $pv_uuid,
+ pv_in_use => '0',
+ },
+ {
+ pv_name => $alias,
+ vg_name => '[unknown]',
+ pv_uuid => $other_uuid,
+ pv_in_use => '1',
+ },
+ ],
+ }],
+ });
+ my $error = check();
+ if ($alias eq '/dev/other') {
+ is($error, '', 'unrelated unknown-membership PV does not block the orphan');
+ } elsif ($alias eq '/dev/missing') {
+ like(
+ $error,
+ qr/failed to get absolute path/,
+ 'unresolved different UUID fails closed',
+ );
+ } else {
+ like(
+ $error,
+ qr/metadata is not visible/,
+ 'different UUID can share whole-LUN identity',
+ );
+ }
+ is_deeply(
+ \@resolved,
+ ['/dev/dm-0', $alias],
+ 'different UUID still requires identity proof',
+ );
+ is_deeply(\@writes, [], 'guard performs no writes');
+ }
+
+ for my $flag (undef, [], {}, 'maybe') {
+ reset_case();
+ my $pv = {
+ pv_name => '/dev/other',
+ vg_name => '[unknown]',
+ pv_uuid => $pv_uuid,
+ };
+ $pv->{pv_in_use} = $flag if defined($flag);
+ $report = encode_json({ report => [{ pv => [$pv] }] });
+ like(
+ check(),
+ qr/malformed PV/,
+ 'sentinel does not excuse a malformed or missing use flag',
+ );
+ is_deeply(\@writes, [], 'malformed use flag causes no writes');
+ }
+ for my $vg ('[missing]', '[UNKNOWN]', 'bad/name') {
+ reset_case();
+ $report = encode_json({
+ report => [{
+ pv => [{
+ pv_name => '/dev/other',
+ vg_name => $vg,
+ pv_uuid => $pv_uuid,
+ pv_in_use => '0',
+ }],
+ }],
+ });
+ like(check(), qr/malformed PV/, "unrecognized VG '$vg' still fails closed");
+ }
+ reset_case();
+ $report = encode_json({
+ report => [{
+ pv => [{
+ pv_name => '[unknown]',
+ vg_name => '[unknown]',
+ pv_uuid => $pv_uuid,
+ pv_in_use => '1',
+ }],
+ }],
+ });
+ like(check(), qr/malformed PV/, 'missing-device spelling is not a supported VG sentinel');
+
+ reset_case();
+ $cfg->{ids}->{lvm} = { type => 'lvm', vgname => 'unknown' };
+ $signatures->{'/dev/dm-0'} = encode_json({
+ signatures => [{ type => 'LVM2_member', uuid => $pv_uuid }],
+ });
+ $report = encode_json({
+ report => [{
+ pv => [{
+ pv_name => '/dev/missing',
+ vg_name => 'unknown',
+ pv_uuid => $pv_uuid,
+ pv_in_use => '1',
+ }],
+ }],
+ });
+ like(
+ check(),
+ qr/volume group 'unknown' is used by storage 'lvm'/,
+ 'unknown is a valid VG name',
+ );
+ is_deeply(\@resolved, ['/dev/dm-0'], 'named VG UUID proof also precedes resolution');
+};
+
+reset_case();
+$readiness_error = 1;
+like(check(), qr/no usable path/, 'readiness refusal propagates');
+is_deeply(\@probed, [], 'no direct signature probe after path loss');
+ok(!$ready, 'pvs is not reached after path loss');
+
+$storage->unmock('san_device_identity');
+my $root = tempdir(CLEANUP => 1);
+my sub put {
+ my ($name, $file, $value) = @_;
+ my $dir = "$root/class/block/$name";
+ make_path("$dir/holders", "$dir/dm", "$dir/device");
+ open(my $fh, '>', "$dir/$file") or die $!;
+ print $fh "$value\n";
+ close($fh) or die $!;
+}
+put('dm-0', 'dev', '253:0');
+put('dm-0', 'dm/uuid', 'mpath-test');
+put('sda', 'dev', '8:0');
+$disk->mock(
+ verify_blockdev_path => sub { $_[0] },
+ is_partition => sub { 0 },
+ is_san_device => sub { 1 },
+);
+$identity->mock(path_wwid => sub { 'test' });
+my $ids = { luns => { one => { target => 'iqn.test', wwid => 'test' } } };
+is(
+ PVE::Storage::san_device_identity('/dev/dm-0', $ids, $root)->{key},
+ 'scsi:test',
+ 'map UUID identity',
+);
+is(
+ PVE::Storage::san_device_identity('/dev/sda', $ids, $root)->{key},
+ 'scsi:test',
+ 'single path identity',
+);
+symlink('../../dm-0', "$root/class/block/sda/holders/dm-0") or die $!;
+eval { PVE::Storage::san_device_identity('/dev/sda', $ids, $root) };
+like($@, qr/member path/, 'member cannot be wiped');
+is(
+ PVE::Storage::san_device_identity('/dev/sda', $ids, $root, 1)->{key},
+ 'scsi:test',
+ 'member-named reference resolves',
+);
+$disk->mock(is_partition => sub { $_[0] eq '/dev/sda1' }, get_blockdev => sub { '/dev/sda' });
+eval { PVE::Storage::san_device_identity('/dev/sda1', $ids, $root) };
+like($@, qr/member path/, 'partition of member cannot be wiped');
+$disk->mock(is_partition => sub { 0 });
+put('nvme0n1', 'dev', '259:0');
+put('nvme0n1', 'wwid', 'eui.test');
+# padded, as some controllers report it
+put('nvme0n1', 'device/subsysnqn', 'nqn.test ');
+my $ns =
+ PVE::Storage::san_device_identity('/dev/nvme0n1', sub { die 'unexpected SCSI probe' }, $root);
+is($ns->{key}, 'nvme:eui.test', 'native namespace identity');
+is($ns->{subsystem}, 'nqn.test', 'native subsystem attribution');
+
+# Offload initiators can run sessions to several targets on one SCSI host. Through the real
+# collectors, only the LUN of the target that serves disk images is refused, not its peer.
+{
+ my $sys = tempdir(CLEANUP => 1);
+ local $PVE::Multipath::Identity::SYSFS = $sys;
+ local $PVE::Storage::ISCSIPlugin::SYSFS = $sys;
+ my sub write_file {
+ my ($path, $value) = @_;
+ make_path("$sys/$path" =~ s|/[^/]+$||r);
+ open(my $fh, '>', "$sys/$path") or die $!;
+ print $fh "$value\n";
+ close($fh) or die $!;
+ }
+ my $lines = {};
+ for my $lun (
+ [1, 'iqn.images', '9:0:0:1', 'sdx', '8:240', '2EUIX'],
+ [2, 'iqn.other', '9:0:1:2', 'sdy', '65:0', '2EUIY'],
+ ) {
+ my ($sid, $target, $hctl, $name, $devt, $wwid) = @$lun;
+ my ($scsi_target) = $hctl =~ m/^(\d+:\d+:\d+):/;
+ my $session = "devices/platform/host9/session$sid";
+ my $dir = "$session/target$scsi_target/$hctl";
+ write_file("class/iscsi_session/session$sid/targetname", $target);
+ symlink("$sys/$session", "$sys/class/iscsi_session/session$sid/device") or die $!;
+ write_file("$dir/type", '0');
+ write_file("$dir/block/$name/dev", $devt);
+ make_path("$sys/$dir/block/$name/holders", "$sys/bus/scsi/devices", "$sys/class/block");
+ symlink("$sys/$dir", "$sys/bus/scsi/devices/$hctl") or die $!;
+ symlink("$sys/$dir", "$sys/$dir/block/$name/device") or die $!;
+ symlink("$sys/$dir/block/$name", "$sys/class/block/$name") or die $!;
+ $lines->{$name} =
+ { devt => $devt, hctl => $hctl, init => 'ok', target => $target, wwid => $wwid };
+ }
+ $readiness_error = 0;
+ $identity->unmock($_) for grep { $identity->is_mocked($_) } qw(collect path_wwid);
+ $iscsi->unmock('iscsi_session_device_map');
+ my $multipath = Test::MockModule->new('PVE::Multipath');
+ $multipath->mock(query_path_identities => sub { $lines });
+ $disk->mock(
+ verify_blockdev_path => sub { $_[0] },
+ is_partition => sub { 0 },
+ is_san_device => sub { 1 },
+ );
+ my $san_identity = \&PVE::Storage::san_device_identity;
+ $storage->mock(
+ san_device_identity => sub { $san_identity->($_[0], $_[1], $sys, $_[3]) },
+ run_command => sub {
+ my ($cmd, %opt) = @_;
+ $opt{outfunc}
+ ->($cmd->[0] eq 'wipefs' ? '{"signatures":[]}' : '{"report":[{"pv":[]}]}');
+ },
+ );
+ my $cfg = {
+ ids =>
+ { img => { type => 'iscsi', target => 'iqn.images', content => { images => 1 } } },
+ };
+ eval { PVE::Storage::assert_san_write_safe($cfg, '/dev/sdx', []) };
+ like(
+ $@,
+ qr/storage 'img' serves disk images from this target/,
+ 'shared host: the LUN of the image target is refused',
+ );
+ eval { PVE::Storage::assert_san_write_safe($cfg, '/dev/sdy', []) };
+ is($@, '', 'shared host: the LUN of the other target is not');
+}
+
+# A session device without an identity, for example of a LUN that multipathd is configured to
+# ignore, stops writes to another LUN unless its VPD page proves that it is another one.
+{
+ my $sys = tempdir(CLEANUP => 1);
+ local $PVE::Multipath::Identity::SYSFS = $sys;
+ local $PVE::Storage::ISCSIPlugin::SYSFS = $sys;
+ my sub write_file {
+ my ($path, $value) = @_;
+ make_path("$sys/$path" =~ s|/[^/]+$||r);
+ open(my $fh, '>', "$sys/$path") or die $!;
+ print $fh $value;
+ close($fh) or die $!;
+ }
+ my sub naa_page {
+ return pack('C C n', 0, 0x83, 20) . pack('C C x C', 1, 3, 16) . pack('H*', $_[0]);
+ }
+ my ($W, $V) = ('600a09803832355367242f747a716b7a', '600a09803832355367242f747a716c30');
+ my $lines = {};
+ for my $lun (
+ [1, 'iqn.data', '10:0:0:1', 'sdv', '65:16', "3$W"],
+ [2, 'iqn.ignored', '10:0:1:2', 'sdw', '65:32', undef],
+ ) {
+ my ($sid, $target, $hctl, $name, $devt, $wwid) = @$lun;
+ my ($scsi_target) = $hctl =~ m/^(\d+:\d+:\d+):/;
+ my $session = "devices/platform/host10/session$sid";
+ my $dir = "$session/target$scsi_target/$hctl";
+ write_file("class/iscsi_session/session$sid/targetname", "$target\n");
+ symlink("$sys/$session", "$sys/class/iscsi_session/session$sid/device") or die $!;
+ write_file("$dir/type", "0\n");
+ write_file("$dir/block/$name/dev", "$devt\n");
+ make_path("$sys/$dir/block/$name/holders", "$sys/bus/scsi/devices", "$sys/class/block");
+ symlink("$sys/$dir", "$sys/bus/scsi/devices/$hctl") or die $!;
+ symlink("$sys/$dir", "$sys/$dir/block/$name/device") or die $!;
+ symlink("$sys/$dir/block/$name", "$sys/class/block/$name") or die $!;
+ $lines->{$name} =
+ { devt => $devt, hctl => $hctl, init => 'ok', target => $target, wwid => $wwid }
+ if defined($wwid);
+ }
+ write_file('bus/scsi/devices/10:0:0:1/vpd_pg83', naa_page($W));
+ my $multipath = Test::MockModule->new('PVE::Multipath');
+ $multipath->mock(query_path_identities => sub { $lines });
+ my $san_identity = $storage->original('san_device_identity');
+ $storage->mock(
+ san_device_identity => sub { $san_identity->($_[0], $_[1], $sys, $_[3]) },
+ run_command => sub {
+ my ($cmd, %opt) = @_;
+ $opt{outfunc}
+ ->($cmd->[0] eq 'wipefs' ? '{"signatures":[]}' : '{"report":[{"pv":[]}]}');
+ },
+ );
+ my $write = sub {
+ eval { PVE::Storage::assert_san_write_safe({ ids => {} }, '/dev/sdv', []) };
+ return $@;
+ };
+ like(
+ $write->(),
+ qr/the identity of 'sdw' could not be verified/,
+ 'unidentified session device: refused without a VPD page',
+ );
+ write_file('bus/scsi/devices/10:0:1:2/vpd_pg83', naa_page($W));
+ like(
+ $write->(),
+ qr/the identity of 'sdw' could not be verified/,
+ 'unidentified session device: refused if it names the NAA of the LUN',
+ );
+ write_file('bus/scsi/devices/10:0:1:2/vpd_pg83', naa_page($V));
+ is($write->(), '', 'unidentified session device: not if it names the NAA of another LUN');
+ $lines->{sdv}->{wwid} = '2EUIV';
+ like(
+ $write->(),
+ qr/the identity of 'sdw' could not be verified/,
+ 'unidentified session device: refused for a LUN without an NAA WWID',
+ );
+ $lines->{sdv}->{wwid} = "3$W";
+ $lines->{sdw} = { devt => '65:32', hctl => '10:0:1:2', init => 'ok', wwid => "3$W" };
+ like(
+ $write->(),
+ qr/the identity of 'sdw' could not be verified/,
+ 'unidentified session device: refused if the daemon names it, with an identity that did not'
+ . ' resolve',
+ );
+ delete $lines->{sdw};
+
+ # a session device that only the strict walk of the sessions finds
+ my $collect = \&PVE::Multipath::Identity::collect;
+ $identity->mock(
+ collect => sub {
+ my $result = $collect->(@_);
+ delete $result->{paths}->{sdw};
+ for my $key (keys $result->{luns}->%*) {
+ delete $result->{luns}->{$key}
+ if grep { $_ eq 'sdw' } ($result->{luns}->{$key}->{paths} // [])->@*;
+ }
+ return $result;
+ },
+ );
+ like(
+ $write->(),
+ qr/the identity of 'sdw' could not be verified/,
+ 'unidentified session device: refused if only the strict session walk finds it',
+ );
+ $identity->unmock('collect');
+
+ delete $lines->{sdv};
+ like($write->(), qr/unresolved LUN identity/, 'unidentified device to write: refused');
+ eval { $san_identity->('/dev/sdv', $collect->(all => 1), $sys, 1) };
+ like($@, qr/unresolved LUN identity/, 'unidentified reference: refused as a member, too');
+}
+
+done_testing();
diff --git a/src/test/run_test_lvm_base_volume.pl b/src/test/run_test_lvm_base_volume.pl
index 208e7668..d16c0f70 100755
--- a/src/test/run_test_lvm_base_volume.pl
+++ b/src/test/run_test_lvm_base_volume.pl
@@ -28,8 +28,14 @@ $storage->mock(
},
);
+my $diskmanage = Test::MockModule->new('PVE::Diskmanage');
+$diskmanage->mock(verify_blockdev_path => sub { $_[0] });
+
my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
-$lvm->mock(lvm_create_volume_group => sub { push @calls, 'lvm_create_volume_group' });
+$lvm->mock(
+ assert_whole_lun => sub { },
+ lvm_create_volume_group => sub { push @calls, 'lvm_create_volume_group' },
+);
my $scfg = { base => 'san:0.0.0.scsi-test', vgname => 'vg', shared => 1 };
diff --git a/src/test/run_wipe_partition_tests.pl b/src/test/run_wipe_partition_tests.pl
index 29ba74a2..3faa6032 100755
--- a/src/test/run_wipe_partition_tests.pl
+++ b/src/test/run_wipe_partition_tests.pl
@@ -16,7 +16,7 @@ my $disk = Test::MockModule->new('PVE::Diskmanage');
my $tools = Test::MockModule->new('PVE::Tools');
my $users = \&PVE::Diskmanage::block_device_users;
my $parts = \&PVE::Diskmanage::wipe_partitions;
-my (@commands, $drop_kernel);
+my (@commands, $remove_error, $new_holder, $new_opener, $drop_kernel);
my sub put {
my ($path, $value) = @_;
my ($dir) = "$root/$path" =~ m{^(.*)/[^/]+$};
@@ -33,45 +33,139 @@ my sub device {
}
my sub reset_case {
remove_tree("$root/sys", "$root/proc");
- device('sda', '8:0', '');
- device('sda1', '8:1', '');
- put('sys/class/block/sda1/partition', '1');
- symlink('../sda1', "$root/sys/class/block/sda/sda1") or die $!;
+ device('dm-0', '253:0', 'mpath-test');
+ device('dm-1', '253:1', 'part1-mpath-test');
+ symlink('../../dm-1', "$root/sys/class/block/dm-0/holders/dm-1") or die $!;
put('proc/self/mountinfo', '1 0 0:1 / / rw - rootfs rootfs rw');
put('proc/swaps', "Filename\tType\tSize\tUsed\tPriority");
@commands = ();
- $drop_kernel = 0;
+ ($remove_error, $new_holder, $new_opener, $drop_kernel) = (0, 0, 0, 0);
}
$disk->mock(
wipe_partitions => sub { $parts->($_[0], "$root/sys") },
block_device_users => sub {
$users->($_[0], { %{ $_[1] // {} }, sysfs => "$root/sys", procfs => "$root/proc" });
},
- block_device_number => sub { '8:1' },
+ block_device_number => sub {
+ my ($path) = @_;
+ my $target = -l $path ? readlink($path) : $path;
+ return $target eq '/dev/dm-0' ? '253:0' : '253:1';
+ },
is_partition => sub { 0 },
dir_glob_foreach => sub { },
run_command => sub {
my ($cmd) = @_;
push @commands, $cmd;
- if ($drop_kernel && $cmd->[0] eq 'wipefs' && $cmd->[-1] eq '/dev/sda') {
- unlink("$root/sys/class/block/sda/sda1") or die $!;
+ if ($drop_kernel && $cmd->[0] eq 'wipefs' && $cmd->[-1] eq '/dev/dm-0') {
+ unlink("$root/sys/class/block/dm-0/sda1") or die $!;
+ }
+ if ($cmd->[0] eq 'dmsetup') {
+ die "remove failed\n" if $remove_error;
+ unlink("$root/sys/class/block/dm-0/holders/dm-1") or die $!;
+ if ($new_holder) {
+ device('dm-2', '253:2', 'LVM-test');
+ symlink('../../dm-2', "$root/sys/class/block/dm-0/holders/dm-2") or die $!;
+ }
+ if ($new_opener) {
+ make_path("$root/proc/999998/fd");
+ symlink('/dev/dm-0', "$root/proc/999998/fd/3") or die $!;
+ put('proc/999998/comm', 'qemu');
+ }
}
},
);
$tools->mock(file_get_contents => sub { "8192\n" });
+my sub wipe {
+ eval {
+ PVE::Diskmanage::wipe_blockdev(
+ '/dev/dm-0',
+ PVE::Diskmanage::wipe_partitions('/dev/dm-0'),
+ );
+ };
+ return $@;
+}
reset_case();
+is_deeply(PVE::Diskmanage::wipe_partitions('/dev/dm-0'), ['dm-1'], 'exact kpartx child');
+is(wipe(), '', 'unused partition map can be wiped');
+is_deeply(
+ \@commands,
+ [
+ ['wipefs', '--all', '/dev/dm-1'],
+ ['dmsetup', 'remove', '--retry', '/dev/dm-1'],
+ ['wipefs', '--all', '/dev/dm-0'],
+ ['dd', 'if=/dev/zero', 'of=/dev/dm-0', 'bs=1M', 'conv=fdatasync', 'count=4'],
+ ],
+ 'child signatures erased while mapped, ordinary removal precedes parent writes',
+);
+for my $case (
+ [
+ 'mounted',
+ sub { put('proc/self/mountinfo', '1 0 253:1 / /mnt rw - ext4 /dev/dm-1 rw') },
+ qr/mounted/,
+ ],
+ [
+ 'open',
+ sub {
+ make_path("$root/proc/999999/fd");
+ symlink('/dev/dm-1', "$root/proc/999999/fd/3") or die $!;
+ put('proc/999999/comm', 'qemu');
+ },
+ qr/has dm-1 open/,
+ ],
+ [
+ 'foreign UUID',
+ sub { put('sys/class/block/dm-1/dm/uuid', 'part1-mpath-other') },
+ qr/held by/,
+ ],
+ ['opaque holder', sub { put('sys/class/block/dm-1/dm/uuid', 'LVM-test') }, qr/held by/],
+ [
+ 'missing UUID',
+ sub { unlink("$root/sys/class/block/dm-1/dm/uuid") or die $! },
+ qr/unable to identify/,
+ ],
+) {
+ reset_case();
+ $case->[1]->();
+ like(wipe(), $case->[2], "$case->[0] refuses");
+ is_deeply(\@commands, [], "$case->[0]: no writes");
+}
+reset_case();
+$remove_error = 1;
+like(wipe(), qr/remove failed/, 'failed child removal refuses');
+is(scalar(@commands), 2, 'parent untouched after failed removal');
+reset_case();
+$new_holder = 1;
+like(wipe(), qr/held by dm-2/, 'new holder refuses');
+is(scalar(@commands), 2, 'parent untouched after unexpected holder');
+reset_case();
+$new_opener = 1;
+like(wipe(), qr/has dm-0 open/, 'a process opening the parent meanwhile refuses');
+is(scalar(@commands), 2, 'parent untouched after a new opener');
+reset_case();
+device('dm-2', '253:2', 'part2-mpath-test');
+symlink('../../dm-2', "$root/sys/class/block/dm-1/holders/dm-2") or die $!;
+is_deeply(PVE::Diskmanage::wipe_partitions('/dev/dm-0'), ['dm-2', 'dm-1'], 'recursive postorder');
+put('proc/self/mountinfo', '1 0 253:2 / /nested rw - ext4 /dev/dm-2 rw');
+like(wipe(), qr/dm-2 mounted/, 'nested child mount refuses');
+is_deeply(\@commands, [], 'nested users checked before any write');
+
+reset_case();
+unlink("$root/sys/class/block/dm-0/holders/dm-1") or die $!;
+device('sda1', '8:1', '');
+put('sys/class/block/sda1/partition', '1');
+symlink('../sda1', "$root/sys/class/block/dm-0/sda1") or die $!;
$drop_kernel = 1;
my $removed_kernel;
eval {
PVE::Diskmanage::wipe_blockdev(
- '/dev/sda',
+ '/dev/dm-0',
['sda1'],
{
verify => sub {
$removed_kernel = $_[0]->{sda1};
my $expected = $removed_kernel ? [] : ['sda1'];
is_deeply(
- PVE::Diskmanage::wipe_partitions('/dev/sda'),
+ PVE::Diskmanage::wipe_partitions('/dev/dm-0'),
$expected,
'kernel partition topology matches deliberate removals',
);
@@ -97,11 +191,11 @@ $disk->mock(
);
my @verified;
PVE::Diskmanage::wipe_blockdev(
- '/dev/sda',
- ['sda1'],
+ '/dev/dm-0',
+ ['dm-1'],
{
handle => $parent,
- children => { 'sda1' => { stamp => 'test', dm_uuid => 'part1-mpath-test' } },
+ children => { 'dm-1' => { stamp => 'test', dm_uuid => 'part1-mpath-test' } },
verify => sub { push @verified, { %{ $_[0] } } },
},
);
@@ -112,11 +206,182 @@ is_deeply(
);
is_deeply(
$commands[1],
+ ['dmsetup', 'remove', '--retry', '--uuid', 'part1-mpath-test'],
+ 'remove selects verified UUID, not reusable dm name',
+);
+is_deeply(
+ $commands[2],
['wipefs', '--all', "/proc/$$/fd/$parent_fd"],
'parent signatures use retained descriptor',
);
-like($commands[2]->[2], qr{^of=/proc/$$/fd/$parent_fd$}, 'zeroing uses retained descriptor');
+like($commands[3]->[2], qr{^of=/proc/$$/fd/$parent_fd$}, 'zeroing uses retained descriptor');
ok(@verified >= 4, 'topology verified between destructive commands');
-is_deeply($verified[-1], {}, 'unchanged kernel partitions remain verified');
+is_deeply($verified[-1], { 'dm-1' => 1 }, 'removed children excluded from later verification');
+
+reset_case();
+$disk->mock(verify_blockdev_path => sub { $_[0] });
+is_deeply(
+ PVE::Diskmanage::block_device_usage_hint('/dev/dm-0'),
+ { 'in-use' => 'no', 'in-use-by' => '' },
+ 'unused kpartx children are not users',
+);
+put('sys/class/block/dm-0/dm/name', 'array-lun');
+put('sys/class/block/dm-1/dm/name', 'array-lun-part1');
+put('proc/self/mountinfo', '1 0 253:1 / /mounted\\040data rw - ext4 /dev/dm-1 rw');
+is_deeply(
+ PVE::Diskmanage::block_device_usage_hint('/dev/dm-0'),
+ { 'in-use' => 'yes', 'in-use-by' => 'array-lun-part1 mounted at /mounted data' },
+ 'hint includes named child mounts',
+);
+put('proc/self/mountinfo', '1 0 0:1 / / rw - rootfs rootfs rw');
+device('dm-2', '253:2', 'LVM-test');
+put('sys/class/block/dm-2/dm/name', 'vg-data');
+symlink('../../dm-2', "$root/sys/class/block/dm-1/holders/dm-2") or die $!;
+is_deeply(
+ PVE::Diskmanage::block_device_usage_hint('/dev/dm-0'),
+ { 'in-use' => 'yes', 'in-use-by' => 'array-lun-part1 held by vg-data' },
+ 'hint names non-partition holders below kpartx children',
+);
+like(wipe(), qr/array-lun-part1 held by vg-data/, 'worker refusal names dm holders too');
+unlink("$root/sys/class/block/dm-2/dm/name") or die $!;
+like(
+ PVE::Diskmanage::block_device_usage_hint('/dev/dm-0')->{'in-use-by'},
+ qr/held by dm-2/,
+ 'kernel-name fallback for missing dm name',
+);
+unlink("$root/sys/class/block/dm-1/holders/dm-2") or die $!;
+put('proc/swaps', 'deliberately invalid');
+make_path("$root/proc/999999/fd");
+symlink('/dev/dm-1', "$root/proc/999999/fd/3") or die $!;
+is(
+ PVE::Diskmanage::block_device_usage_hint('/dev/dm-0')->{'in-use'},
+ 'no',
+ 'hint excludes swap and process descriptor probes',
+);
+put('proc/self/mountinfo', 'invalid');
+is(
+ PVE::Diskmanage::block_device_usage_hint('/dev/dm-0')->{'in-use'},
+ 'unknown',
+ 'failed metadata probe is not an unused result',
+);
+
+# the paths of a map count with 'map-wwid', for example a logical volume that LVM activated on a
+# path before the map existed
+reset_case();
+put('sys/class/block/sdb/dev', '8:16');
+make_path("$root/sys/class/block/sdb/holders", "$root/sys/class/block/dm-0/slaves");
+symlink('../../sdb', "$root/sys/class/block/dm-0/slaves/sdb") or die $!;
+symlink('../../dm-0', "$root/sys/class/block/sdb/holders/dm-0") or die $!;
+my sub map_hint {
+ my ($wwid) = @_;
+ return PVE::Diskmanage::block_device_usage_hint(
+ '/dev/dm-0',
+ { 'map-wwid' => $wwid // 'test', sysfs => "$root/sys" },
+ );
+}
+is_deeply(
+ map_hint(),
+ { 'in-use' => 'no', 'in-use-by' => '' },
+ 'map paths: held by the map only, with unused kpartx children',
+);
+device('dm-3', '253:3', 'LVM-other');
+put('sys/class/block/dm-3/dm/name', 'oldvg-data');
+symlink('../../dm-3', "$root/sys/class/block/sdb/holders/dm-3") or die $!;
+is_deeply(
+ map_hint(),
+ { 'in-use' => 'yes', 'in-use-by' => 'sdb held by oldvg-data' },
+ 'map paths: a logical volume on a path is a user',
+);
+is(
+ PVE::Diskmanage::block_device_usage_hint('/dev/dm-0')->{'in-use'},
+ 'no',
+ 'map paths: not looked at without the WWID of the map',
+);
+is(map_hint('other')->{'in-use'}, 'unknown', 'map paths: unknown for a map of another WWID');
+unlink("$root/sys/class/block/sdb/holders/dm-3") or die $!;
+put('proc/self/mountinfo', '1 0 8:16 / /raw rw - ext4 /dev/sdb rw');
+is(map_hint()->{'in-use-by'}, 'sdb mounted at /raw', 'map paths: a mounted path is a user');
+put('proc/self/mountinfo', '1 0 0:1 / / rw - rootfs rootfs rw');
+remove_tree("$root/sys/class/block/sdb");
+is(map_hint()->{'in-use'}, 'unknown', 'map paths: unknown when a path goes away meanwhile');
+device('dm-4', '253:4', 'LVM-map');
+put('sys/class/block/dm-4/dm/name', 'vg-map');
+symlink('../../dm-4', "$root/sys/class/block/dm-0/holders/dm-4") or die $!;
+is_deeply(
+ map_hint(),
+ { 'in-use' => 'yes', 'in-use-by' => 'dm-0 held by vg-map' },
+ 'map paths: a user of the map stays one if the check of a path fails',
+);
+
+# the map changes while its paths are checked
+my $during;
+my sub path_check_mock {
+ $disk->mock(
+ block_device_users => sub {
+ my $found =
+ $users->($_[0], { %{ $_[1] // {} }, sysfs => "$root/sys", procfs => "$root/proc" });
+ $during->() if $during && grep { $_ eq 'sdb' } $_[0]->@*;
+ return $found;
+ },
+ );
+}
+path_check_mock();
+reset_case();
+put('sys/class/block/sdb/dev', '8:16');
+make_path("$root/sys/class/block/sdb/holders", "$root/sys/class/block/dm-0/slaves");
+symlink('../../sdb', "$root/sys/class/block/dm-0/slaves/sdb") or die $!;
+symlink('../../dm-0', "$root/sys/class/block/sdb/holders/dm-0") or die $!;
+$during = sub {
+ put('sys/class/block/sdc/dev', '8:32');
+ make_path("$root/sys/class/block/sdc/holders");
+ symlink('../../sdc', "$root/sys/class/block/dm-0/slaves/sdc");
+ symlink('../../dm-0', "$root/sys/class/block/sdc/holders/dm-0");
+ device('dm-5', '253:5', 'LVM-new');
+ put('sys/class/block/dm-5/dm/name', 'vg-new');
+ symlink('../../dm-5', "$root/sys/class/block/sdc/holders/dm-5");
+};
+like(
+ map_hint()->{'in-use-by'},
+ qr/^check failed: the paths of 'dm-0' changed/,
+ 'map paths: unknown if a path is added during the check',
+);
+$during = undef;
+is(map_hint()->{'in-use-by'}, 'sdc held by vg-new', 'map paths: and its user found the next time');
+unlink("$root/sys/class/block/sdc/holders/dm-5") or die $!;
+$during = sub { put('sys/class/block/dm-0/dm/uuid', 'mpath-other') };
+like(
+ map_hint()->{'in-use-by'},
+ qr/^check failed: 'dm-0' is not the multipath map of 'test'/,
+ 'map paths: unknown if the map changes its identity during the check',
+);
+$during = undef;
+# a user on one path stays one if the check of another path fails
+reset_case();
+put('sys/class/block/sdb/dev', '8:16');
+make_path("$root/sys/class/block/sdb/holders", "$root/sys/class/block/dm-0/slaves");
+symlink('../../sdb', "$root/sys/class/block/dm-0/slaves/sdb") or die $!;
+symlink('../../dm-0', "$root/sys/class/block/sdb/holders/dm-0") or die $!;
+device('dm-3', '253:3', 'LVM-other');
+put('sys/class/block/dm-3/dm/name', 'oldvg-data');
+symlink('../../dm-3', "$root/sys/class/block/sdb/holders/dm-3") or die $!;
+symlink('../../sdc', "$root/sys/class/block/dm-0/slaves/sdc") or die $!;
+is_deeply(
+ map_hint(),
+ { 'in-use' => 'yes', 'in-use-by' => 'sdb held by oldvg-data' },
+ 'map paths: a user of a path stays one if the check of another path fails',
+);
+$disk->mock(
+ block_device_users => sub {
+ $users->($_[0], { %{ $_[1] // {} }, sysfs => "$root/sys", procfs => "$root/proc" });
+ },
+);
+
+@commands = ();
+PVE::Diskmanage::udevadm_trigger('/dev/dm-0', '/dev/sda1');
+is_deeply(
+ \@commands,
+ [['udevadm', 'trigger', '--settle', '/dev/dm-0', '/dev/sda1']],
+ 'device-specific trigger waits for udev processing',
+);
done_testing();
diff --git a/src/test/run_wipe_readiness_tests.pl b/src/test/run_wipe_readiness_tests.pl
new file mode 100755
index 00000000..ef5977a3
--- /dev/null
+++ b/src/test/run_wipe_readiness_tests.pl
@@ -0,0 +1,138 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use File::Path qw(make_path remove_tree);
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+use PVE::Diskmanage;
+use PVE::Multipath;
+use PVE::Multipath::Identity;
+
+my $root = tempdir(CLEANUP => 1);
+my $disk = Test::MockModule->new('PVE::Diskmanage');
+my $mp = Test::MockModule->new('PVE::Multipath');
+my $identity = Test::MockModule->new('PVE::Multipath::Identity');
+my ($ids, $maps, $nvme);
+my sub put {
+ my ($path, $value) = @_;
+ my ($dir) = "$root/$path" =~ m{^(.*)/[^/]+$};
+ make_path($dir);
+ open(my $fh, '>', "$root/$path") or die $!;
+ print $fh "$value\n";
+ close($fh) or die $!;
+}
+my sub device {
+ my ($name, $number) = @_;
+ put("sys/class/block/$name/dev", $number);
+ put("sys/class/block/$name/size", '8192');
+ make_path("$root/sys/class/block/$name/holders");
+}
+$disk->mock(
+ is_partition => sub { 0 },
+ is_san_device => sub { 1 },
+ get_nvme_path_status => sub { $nvme },
+ block_device_number => sub { '8:16' },
+);
+$mp->mock(get_maps => sub { $maps });
+my %other_lun;
+$identity->mock(
+ collect => sub { $ids },
+ path_wwid => sub { $_[0]->{paths}->{ $_[1] }->{wwid} },
+ path_is_other_lun => sub { $other_lun{ $_[1] } && $_[2] eq 'test' },
+);
+my sub reset_case {
+ remove_tree("$root/sys", "$root/proc");
+ device('sda', '8:0');
+ put('sys/class/block/sda/device/state', 'running');
+ $ids = { paths => { sda => { wwid => 'test' } } };
+ $maps = [];
+ $nvme = {};
+ put('proc/self/mountinfo', '1 0 0:1 / / rw - rootfs rootfs rw');
+ put('proc/swaps', "Filename\tType\tSize\tUsed\tPriority");
+}
+my sub check {
+ my ($dev) = @_;
+ eval {
+ PVE::Diskmanage::assert_block_device_ready(
+ "/dev/$dev",
+ { sysfs => "$root/sys", procfs => "$root/proc" },
+ );
+ };
+ return $@;
+}
+reset_case();
+is(check('sda'), '', 'single running path needs no path count');
+put('sys/class/block/sda/device/state', 'offline');
+like(check('sda'), qr/not running/, 'offline single path refuses');
+put('sys/class/block/sda/device/state', 'running');
+device('sdb', '8:16');
+$ids->{paths}->{sdb} = { wwid => 'test' };
+like(check('sda'), qr/multiple paths/, 'unmapped same-identity sibling refuses');
+$ids->{paths}->{sdb}->{wwid} = 'other';
+is(check('sda'), '', 'different identity is not a sibling');
+# a local disk that multipathd does not list
+$ids->{paths}->{sdz} = { wwid => undef };
+like(
+ check('sda'),
+ qr/cannot exclude 'sdz' as another path of this LUN/,
+ 'path without identity refuses',
+);
+$other_lun{sdz} = 1;
+is(check('sda'), '', 'path without identity that is provably another LUN does not');
+delete $ids->{paths}->{sdz};
+$ids->{paths}->{sdb}->{wwid} = 'test';
+device('dm-0', '253:0');
+put('sys/class/block/dm-0/dm/uuid', 'mpath-test');
+make_path("$root/sys/class/block/dm-0/slaves");
+symlink('../../sda', "$root/sys/class/block/dm-0/slaves/sda") or die $!;
+symlink('../../dm-0', "$root/sys/class/block/sda/holders/dm-0") or die $!;
+my $path = { dev => 'sda', 'dm-state' => 'active', 'check-state' => 'ready' };
+$maps = [{ sysfs => 'dm-0', wwid => 'test', 'path-groups' => [{ paths => [$path] }] }];
+is(check('dm-0'), '', 'map with current usable path allowed');
+
+for my $state ('faulty', 'unchecked', 'shaky') {
+ $path->{'check-state'} = $state;
+ like(check('dm-0'), qr/no usable path/, "$state checker refuses despite SCSI running");
+}
+$path->{'check-state'} = 'ready';
+$path->{'dm-state'} = 'failed';
+like(check('dm-0'), qr/no usable path/, 'failed dm path refuses');
+$path->{'dm-state'} = 'active';
+$maps->[0]->{wwid} = 'other';
+like(check('dm-0'), qr/no usable path/, 'daemon map identity must match sysfs');
+$maps->[0]->{wwid} = 'test';
+make_path("$root/proc/999999/fd");
+symlink('/dev/sdb', "$root/proc/999999/fd/3") or die $!;
+put('proc/999999/comm', 'qemu');
+like(check('dm-0'), qr/qemu .* has sdb open/, 'raw open of unmapped sibling refuses map wipe');
+symlink('../../sdb', "$root/sys/class/block/dm-0/slaves/sdb") or die $!;
+symlink('../../dm-0', "$root/sys/class/block/sdb/holders/dm-0") or die $!;
+like(check('dm-0'), qr/qemu .* has sdb open/, 'raw open of map member refuses');
+symlink('/usr/sbin/multipathd', "$root/proc/999999/exe") or die $!;
+is(check('dm-0'), '', 'only multipathd checker descriptors excluded');
+unlink("$root/proc/999999/exe") or die $!;
+symlink('/usr/bin/qemu', "$root/proc/999999/exe") or die $!;
+like(check('dm-0'), qr/has sdb open/, 'other executables remain users');
+reset_case();
+device('nvme0n1', '259:0');
+put('sys/class/block/nvme0n1/wwid', 'eui.0123456789abcdef');
+put('sys/class/block/nvme0n1/device/state', 'live');
+is(check('nvme0n1'), '', 'single live NVMe controller needs no path count');
+$nvme = { nvme0n1 => { active => 0 } };
+like(
+ check('nvme0n1'),
+ qr/no usable path/,
+ 'native head with no ANA-usable path refuses despite live state',
+);
+$nvme->{nvme0n1}->{active} = 1;
+is(check('nvme0n1'), '', 'ANA-usable native head allowed');
+device('nvme1n1', '259:1');
+put('sys/class/block/nvme1n1/wwid', 'eui.0123456789abcdef');
+like(check('nvme0n1'), qr/multiple devices/, 'uncoalesced same-identity NVMe namespaces refuse');
+
+done_testing();
diff --git a/src/test/run_wipe_reference_tests.pl b/src/test/run_wipe_reference_tests.pl
index 610cb691..c47f7c00 100755
--- a/src/test/run_wipe_reference_tests.pl
+++ b/src/test/run_wipe_reference_tests.pl
@@ -13,9 +13,11 @@ use PVE::API2::Disks;
my $root = tempdir(CLEANUP => 1);
my $disk = Test::MockModule->new('PVE::Diskmanage');
+my $storage = Test::MockModule->new('PVE::Storage');
my $rpc = Test::MockModule->new('PVE::RPCEnvironment');
+my $identity = Test::MockModule->new('PVE::Multipath::Identity');
my $reference = \&PVE::Diskmanage::block_device_reference;
-my ($resolved, $worker, $locked, $written);
+my ($resolved, $worker, $locked, $probes, $written, $rebind);
my sub put {
my ($path, $value) = @_;
my ($dir) = "$root/$path" =~ m{^(.*)/[^/]+$};
@@ -24,8 +26,10 @@ my sub put {
print $fh "$value\n";
close($fh) or die $!;
}
-put('class/block/sda/dev', '8:0');
-put('class/block/sdb/dev', '8:16');
+for my $n (0, 1, 2) {
+ put("class/block/dm-$n/dev", "253:$n");
+ put("class/block/dm-$n/dm/uuid", $n == 2 ? 'mpath-other' : 'mpath-test:123');
+}
{
package ReferenceTestRPC;
@@ -40,17 +44,12 @@ $disk->mock(
ok($locked, 're-resolution occurs after locks') if $_[1];
$reference->($_[0], $_[1], $root);
},
- locked_disk_action => sub {
- $locked = 1;
- eval { $_[0]->() };
- my $err = $@;
- $locked = 0;
- die $err if $err;
- },
+ locked_disk_action => sub { $_[0]->() },
block_device_users => sub { [] },
- is_mounted => sub { undef },
- has_holder => sub { undef },
wipe_partitions => sub { [] },
+ is_san_device => sub { 1 },
+ is_multipath_device => sub { 1 },
+ assert_block_device_ready => sub { },
open_block_device => sub { \*STDOUT },
wipe_blockdev => sub {
$_[2]->{verify}->();
@@ -59,22 +58,89 @@ $disk->mock(
},
udevadm_trigger => sub { },
);
+$storage->mock(
+ config => sub { { ids => {} } },
+ lock_storage_config => sub {
+ local $@;
+ $locked = 1;
+ eval { $_[0]->() };
+ my $err = $@;
+ $locked = 0;
+ die $err if $err;
+ },
+ assert_san_write_safe => sub {
+ $probes++;
+ $resolved = $rebind == 2 ? '/dev/dm-1' : '/dev/dm-2' if $rebind;
+ },
+);
my $method = PVE::API2::Disks->map_method_by_name('wipe_disk')->{code};
my sub queue {
my ($path) = @_;
- $written = undef;
- $resolved = '/dev/sda';
+ ($probes, $written, $rebind) = (0, undef, 0);
+ $resolved = '/dev/dm-0';
$method->({ node => 'test', disk => $path });
}
-queue('/dev/sda');
+my $canonical = '/dev/disk/by-id/dm-uuid-mpath-test:123';
+queue($canonical);
+$resolved = '/dev/dm-1';
eval { $worker->() };
-is($@, '', 'delayed worker accepts the same device instance');
-is($written, '/dev/sda', 'worker uses the verified device');
-queue('/dev/sda');
-$resolved = '/dev/sdb';
+is($@, '', 'delayed worker accepts the same identity after renumbering');
+is($written, '/dev/dm-1', 'worker never uses the old reusable kernel name');
+for my $path ($canonical, '/dev/mapper/friendly') {
+ queue($path);
+ $resolved = '/dev/dm-2';
+ eval { $worker->() };
+ like($@, qr/identity (?:changed|does not match)/, 'rebound alias refuses');
+ is($probes, 0, 'rebound alias refuses before device-reading probes');
+ ok(!defined($written), 'rebound alias does not write');
+}
+queue($canonical);
+$rebind = 1;
eval { $worker->() };
-like($@, qr/identity changed/, 'rebound reference refuses');
-ok(!defined($written), 'rebound reference does not write');
+like($@, qr/identity does not match/, 'rebind during guard probes refuses');
+ok(!defined($written), 'rebind during guard probes does not write');
+queue($canonical);
+$rebind = 2;
+eval { $worker->() };
+is(
+ $@,
+ "cannot use '/dev/dm-0' - device topology changed\n",
+ 'shared instance check refuses same-identity replacement after probing',
+);
+ok(!defined($written), 'same-identity replacement during probes does not write');
+$resolved = '/dev/dm-2';
+eval { $method->({ node => 'test', disk => $canonical }) };
+like($@, qr/identity does not match/, 'canonical name verified at request time too');
+
+$disk->unmock('block_device_reference');
+put('class/block/nvme0n1/dev', '259:0');
+put('class/block/nvme0n1/wwid', 'eui.0123456789abcdef');
+$resolved = '/dev/nvme0n1';
+is(
+ $reference->('/dev/disk/by-id/nvme-eui.0123456789abcdef', undef, $root)->{key},
+ 'nvme:eui.0123456789abcdef',
+ 'NVMe canonical identity verified',
+);
+eval { $reference->('/dev/disk/by-id/nvme-eui.1111111111111111', undef, $root) };
+like($@, qr/identity does not match/, 'wrong NVMe identity refuses');
+put('class/block/sda/dev', '8:0');
+$identity->mock(collect => sub { {} }, path_wwid => sub { '3600abc' });
+$resolved = '/dev/sda';
+is(
+ $reference->('/dev/disk/by-id/scsi-3600abc', undef, $root)->{key},
+ 'scsi:3600abc',
+ 'SCSI by-id verified against live identity',
+);
+eval { $reference->('/dev/disk/by-id/scsi-3600def', undef, $root) };
+like($@, qr/identity does not match/, 'wrong SCSI identity refuses');
+
+is(
+ $reference->('/dev/disk/by-id/wwn-0x600abc', undef, $root)->{key},
+ 'scsi:3600abc',
+ 'WWN alias verified against NAA identity',
+);
+eval { $reference->('/dev/disk/by-id/wwn-0x600def', undef, $root) };
+like($@, qr/identity does not match/, 'wrong WWN refuses');
$disk->unmock('open_block_device');
put('handle', 'not a device');
@@ -86,6 +152,56 @@ my $handle = PVE::Diskmanage::open_block_device("$root/handle", '1:8:0');
ok(defined(fileno($handle)), 'matching opened reference retained');
close($handle) or die $!;
+$disk->unmock('block_device_stamp');
+my %links = (
+ '/dev/mapper/friendly' => '/dev/dm-0',
+ '/dev/disk/by-id/dm-uuid-mpath-test:123' => '/dev/dm-0',
+ '/dev/nvme0n1' => '/dev/nvme0n1',
+ '/dev/disk/by-id/nvme-eui.0123456789abcdef' => '/dev/nvme0n1',
+ '/dev/sda' => '/dev/sda',
+ '/dev/disk/by-id/scsi-3600abc' => '/dev/sda',
+);
+$disk->mock(verify_blockdev_path => sub { $links{ $_[0] } // die "missing link\n" });
+is(
+ PVE::Diskmanage::canonical_blockdev_link('/dev/mapper/friendly', {}, 'test:123', $root),
+ $canonical,
+ 'scan canonical map link comes from UUID, not the friendly name',
+);
+is(
+ PVE::Diskmanage::canonical_blockdev_link('/dev/mapper/friendly', {}, 'other', $root),
+ undef,
+ 'scan omits link when the map identity changed',
+);
+$links{$canonical} = '/dev/dm-2';
+is(
+ PVE::Diskmanage::canonical_blockdev_link('/dev/mapper/friendly', {}, 'test:123', $root),
+ undef,
+ 'scan omits a rebound canonical link',
+);
+delete $links{$canonical};
+is(
+ PVE::Diskmanage::canonical_blockdev_link('/dev/mapper/friendly', {}, 'test:123', $root),
+ undef,
+ 'scan omits a missing canonical link',
+);
+is(
+ PVE::Diskmanage::canonical_blockdev_link('/dev/nvme0n1', {}, undef, $root),
+ '/dev/disk/by-id/nvme-eui.0123456789abcdef',
+ 'scan uses identity-bearing NVMe link',
+);
+is(
+ PVE::Diskmanage::canonical_blockdev_link('/dev/sda', {}, '3600abc', $root),
+ '/dev/disk/by-id/scsi-3600abc',
+ 'scan uses verified single-path SCSI link',
+);
+delete $links{'/dev/disk/by-id/scsi-3600abc'};
+$links{'/dev/disk/by-id/wwn-0x600abc'} = '/dev/sda';
+is(
+ PVE::Diskmanage::canonical_blockdev_link('/dev/sda', {}, '3600abc', $root),
+ '/dev/disk/by-id/wwn-0x600abc',
+ 'scan can use the corresponding WWN link',
+);
+
subtest 'checked wipe keeps verification order and retained handle' => sub {
my @events;
my $handle;
@@ -110,14 +226,18 @@ subtest 'checked wipe keeps verification order and retained handle' => sub {
ok(defined(fileno($handle)), 'parent handle stays open through trigger');
},
);
- PVE::Diskmanage::wipe_blockdev_checked('/dev/sda', {});
+ PVE::Diskmanage::wipe_blockdev_checked('/dev/sda', {}, sub { push @events, 'storage' });
is_deeply(
\@events,
- [qw(reference partitions reference partitions open users
+ [qw(reference partitions ready reference partitions open storage ready users
reference partitions wipe reference partitions trigger)],
- 'retained open, users, verification, wipe and trigger order',
+ 'readiness, retained open, storage check, users, verification, wipe and trigger order',
);
close($handle);
+ @events = ();
+ eval { PVE::Diskmanage::wipe_blockdev_checked('/dev/sda', {}, undef) };
+ like($@, qr/device topology changed/, 'local-to-SAN transition refuses without callback');
+ is_deeply(\@events, ['reference'], 'transition refuses before readiness, opens or writes');
};
done_testing();
diff --git a/src/test/run_wipedisk_tests.pl b/src/test/run_wipedisk_tests.pl
index f110a2f4..dafd08a8 100755
--- a/src/test/run_wipedisk_tests.pl
+++ b/src/test/run_wipedisk_tests.pl
@@ -12,6 +12,23 @@ use PVE::API2::Disks;
my $disk = Test::MockModule->new('PVE::Diskmanage');
my $rpc = Test::MockModule->new('PVE::RPCEnvironment');
my ($worker, $users, $probe_error, $wiped);
+my ($san, $storage_locked, $guard_error) = (0, 0, undef);
+my $storage = Test::MockModule->new('PVE::Storage');
+$storage->mock(
+ config => sub { { ids => {} } },
+ lock_storage_config => sub {
+ $storage_locked = 1;
+ my $res = eval { $_[0]->() };
+ my $err = $@;
+ $storage_locked = 0;
+ die $err if $err;
+ return $res;
+ },
+ assert_san_write_safe => sub {
+ ok($storage_locked, 'SAN guard under storage configuration lock');
+ die $guard_error if $guard_error;
+ },
+);
my $inside_worker = 0;
{
@@ -21,23 +38,29 @@ my $inside_worker = 0;
}
$rpc->mock(get => sub { bless {}, 'WipeTestRPC' });
$disk->mock(
- verify_blockdev_path => sub { '/dev/sda' },
+ verify_blockdev_path => sub { '/dev/dm-9' },
block_device_reference => sub {
+ # a local disk is wiped without the cluster-wide lock, which needs quorum
+ is($storage_locked, $san ? 1 : 0, 'storage lock only for a SAN device') if $_[1];
return {
- device => '/dev/sda',
- stamp => '1:8:0',
+ device => '/dev/dm-9',
+ stamp => '1:253:9',
key => 'scsi:test',
partition => '',
};
},
open_block_device => sub { \*STDOUT },
locked_disk_action => sub { $_[0]->() },
+ is_san_device => sub { $san },
+ is_multipath_device => sub { 0 },
is_mounted => sub { undef },
has_holder => sub { undef },
wipe_partitions => sub { [] },
+ assert_block_device_ready => sub { ok($inside_worker, 'readiness checked in worker') },
block_device_users => sub {
+ return [] if $_[1]->{'metadata-only'};
ok($inside_worker, 'users checked in the worker');
- is_deeply($_[0], ['sda'], 'resolved name checked, including partitions');
+ is_deeply($_[0], ['dm-9'], 'resolved name checked, including partitions');
die $probe_error if $probe_error;
return $users;
},
@@ -45,15 +68,26 @@ $disk->mock(
udevadm_trigger => sub { },
);
my $method = PVE::API2::Disks->map_method_by_name('wipe_disk');
+my $pattern = $method->{parameters}->{properties}->{disk}->{pattern};
+for my $path (
+ '/dev/sda1',
+ '/dev/dm-9',
+ '/dev/mapper/array_lun-1.0',
+ '/dev/disk/by-id/scsi-123',
+ '/dev/disk/by-id/dm-uuid-mpath-naa:123',
+) {
+ like($path, qr/$pattern/, 'valid device path');
+}
+unlike('/dev/x;touch', qr/$pattern/, 'metacharacter refused');
for my $case (
[[], undef, 1, qr/^$/],
- [['qemu (PID 7) has sda open'], undef, 0, qr/cannot wipe.*qemu/],
+ [['qemu (PID 7) has dm-9 open'], undef, 0, qr/cannot wipe.*qemu/],
[[], "probe failed\n", 0, qr/probe failed/],
) {
($users, $probe_error, my $expected, my $error) = @$case;
$wiped = 0;
$inside_worker = 0;
- $method->{code}->({ node => 'test', disk => '/dev/sda' });
+ $method->{code}->({ node => 'test', disk => '/dev/mapper/array_lun-1.0' });
is($wiped, 0, 'request does not wipe before worker');
$inside_worker = 1;
eval { $worker->() };
@@ -61,4 +95,16 @@ for my $case (
is($wiped, $expected, 'only unused devices wiped');
}
+$san = 1;
+$users = [];
+$probe_error = undef;
+for my $error (undef, "storage reference\n") {
+ $guard_error = $error;
+ $wiped = 0;
+ $method->{code}->({ node => 'test', disk => '/dev/dm-9' });
+ eval { $worker->() };
+ is($@, $error // '', 'SAN guard outcome');
+ is($wiped, $error ? 0 : 1, 'referenced SAN device not wiped');
+}
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 storage 42/76] api: scan: refresh SAN usage with bounded probes
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (40 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 41/76] api: disks: guard SAN wipes with stable identities and storage checks Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 docs 43/76] storage: lvm: note the rescan after extending a shared volume group Thomas Lamprecht
` (33 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Other nodes can write SAN LUNs without updating local udev data or LVM
hints. The SAN scan inherited those stale usage records from the disk
inventory, so missing usage could make an occupied LUN look unused.
Add bounded signature and hint-free PV-membership probes on usable
paths. Preserve known usage when evidence is incomplete and report
uncertainty rather than offering unchecked devices. Missing volume-group
metadata does not prove a PV is orphaned.
Use path state, session identities, target inventory, and partitions to
probe reachable devices and attribute whole LUNs and partition PVs
without unbounded global LVM queries.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
src/PVE/API2/Storage/Scan.pm | 16 +-
src/PVE/Diskmanage.pm | 14 +-
src/PVE/Storage/SANScan.pm | 229 ++++++++++---
src/test/run_san_lun_scan_tests.pl | 516 ++++++++++++++++++++++++++++-
4 files changed, 719 insertions(+), 56 deletions(-)
diff --git a/src/PVE/API2/Storage/Scan.pm b/src/PVE/API2/Storage/Scan.pm
index e7f6eba8..75d6ec49 100644
--- a/src/PVE/API2/Storage/Scan.pm
+++ b/src/PVE/API2/Storage/Scan.pm
@@ -504,15 +504,25 @@ __PACKAGE__->register_method({
},
usage => {
description => "How the device is currently used. 'lvm' means it is an LVM"
- . " physical volume, see the 'vgname' property.",
+ . " physical volume, see the 'vgname' property. 'unknown' means the"
+ . " device could not be checked for data, see 'detail'.",
type => 'string',
- enum => ['unused', 'lvm', 'other'],
+ enum => ['unused', 'lvm', 'other', 'unknown'],
},
vgname => {
description => "The name of the LVM volume group the device belongs to.",
type => 'string',
optional => 1,
},
+ 'pv-in-use' => {
+ description => "Whether an LVM report collected without cached device hints"
+ . " marks a physical volume on this device as used, even when its volume"
+ . " group metadata is not visible."
+ . " False means all reported physical volumes on it are unused."
+ . " Absent when unknown or no physical volume was reported.",
+ type => 'boolean',
+ optional => 1,
+ },
'in-use' => {
description =>
"Local usage indicated by mounts and devices using this device,"
@@ -596,7 +606,7 @@ __PACKAGE__->register_method({
},
detail => {
description =>
- "Details about other usage, for example the file system type.",
+ "Details about other usage, or why the device could not be checked for data.",
type => 'string',
optional => 1,
},
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index 796d4eb4..4f929784 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -860,7 +860,7 @@ sub get_zfs_devices {
}
sub get_lvm_devices {
- my ($lsblk_info) = @_;
+ my ($lsblk_info, $no_probe) = @_;
my $res = {};
eval {
run_command(
@@ -872,7 +872,7 @@ sub get_lvm_devices {
$res->{$line} = 1;
}
},
- );
+ ) if !$no_probe;
};
# if something goes wrong, we do not want to give up, but indicate an error has occurred
@@ -1185,8 +1185,9 @@ my sub get_multipath_slave_map {
return $slave_map;
}
+# Callers with their own bounded LVM inventory skip both global PV and Ceph LV probes.
sub get_disks {
- my ($disks, $nosmart, $include_partitions) = @_;
+ my ($disks, $nosmart, $include_partitions, $no_lvm_probe) = @_;
my $disklist = {};
my $mounted = mounted_blockdevs();
@@ -1196,11 +1197,11 @@ sub get_disks {
my $lsblk_info = get_lsblk_info();
my $journalhash = get_ceph_journals($lsblk_info);
- my $ceph_volume_infos = get_ceph_volume_infos();
+ my $ceph_volume_infos = $no_lvm_probe ? {} : get_ceph_volume_infos();
my $zfshash = get_zfs_devices($lsblk_info);
- my $lvmhash = get_lvm_devices($lsblk_info);
+ my $lvmhash = get_lvm_devices($lsblk_info, $no_lvm_probe);
my $disk_regex = ".*";
if (defined($disks)) {
@@ -1455,10 +1456,11 @@ sub get_disks {
# Keep multipath maps separate from get_disks(), whose callers manage local disks.
sub get_multipath_disks {
+ my ($no_lvm_probe) = @_;
my $mounted = mounted_blockdevs();
my $lsblk_info = get_lsblk_info();
my $zfshash = get_zfs_devices($lsblk_info);
- my $lvmhash = get_lvm_devices($lsblk_info);
+ my $lvmhash = get_lvm_devices($lsblk_info, $no_lvm_probe);
my $disklist = {};
diff --git a/src/PVE/Storage/SANScan.pm b/src/PVE/Storage/SANScan.pm
index 7efdbfd5..14b67d56 100644
--- a/src/PVE/Storage/SANScan.pm
+++ b/src/PVE/Storage/SANScan.pm
@@ -12,7 +12,7 @@ use PVE::Multipath::Config;
use PVE::Multipath::Identity;
use PVE::Storage;
use PVE::Storage::ISCSIPlugin;
-use PVE::Storage::LVMPlugin;
+use PVE::Storage::StatusCommand;
use PVE::Tools qw(dir_glob_foreach file_read_firstline);
# NVMe maps use their members' namespace identity, not multipathd's WWID. Follow current slave
@@ -59,11 +59,185 @@ my sub flag_duplicate_pvs($records) {
}
}
+my sub san_probe_ready($entry, $disk, $device_map) {
+ return 0 if !$entry->{size};
+ if (defined($disk->{slaves})) {
+ if (my $paths = $entry->{'path-list'}) {
+ return scalar(
+ grep {
+ ($_->{state} // '') eq 'active'
+ && ($_->{'checker-state'} // '') =~ m/^(?:ready|ghost)$/
+ } @$paths
+ );
+ }
+ return scalar(
+ grep {
+ (file_read_firstline('/sys/class/block/' . basename($_) . '/device/state')
+ // '') eq 'running'
+ } $disk->{slaves}->@*
+ );
+ }
+ my $dev = basename($entry->{devpath});
+ if ($dev =~ m/^nvme\d+n\d+$/) {
+ if (my $paths = $entry->{'path-list'}) {
+ return scalar(
+ grep {
+ ($_->{state} // '') eq 'live'
+ && ($_->{'ana-state'} // 'optimized') =~ m/^(?:optimized|non-optimized)$/
+ } @$paths
+ );
+ }
+ return (file_read_firstline("/sys/class/block/$dev/device/state") // '') eq 'live';
+ }
+ return 0
+ if (file_read_firstline("/sys/class/block/$dev/device/state") // '') ne 'running';
+ if (defined($entry->{'iscsi-target'}) || ($entry->{transport} // '') eq 'iscsi') {
+ my $session = $device_map->{$dev}->{session};
+ return 0 if !defined($session);
+ return (file_read_firstline("/sys/class/iscsi_session/session$session/state") // '') eq
+ 'LOGGED_IN';
+ }
+ return 1;
+}
+
+# Cached udev signatures and LVM hints can both predate another node's writes. Bound all label
+# reads together and keep positive usage when a path disappears or a report is incomplete.
+my sub probe_usage($records, $device_map) {
+ local $PVE::Storage::StatusCommand::BUDGET = 10;
+ my $cache = { 'status-command-budget' => 10 };
+ my (%devices, %parents, %eligible, %san_rows);
+ my $uncertain = sub($row, $detail) {
+ return if !$san_rows{$row} || $row->{usage} ne 'unused';
+ chomp($detail);
+ $row->{usage} = 'unknown';
+ $row->{detail} = $detail;
+ if ($row->{'in-use'} ne 'yes') {
+ $row->{'in-use'} = 'unknown';
+ $row->{'in-use-by'} = $detail;
+ }
+ };
+ for my $record ($records->@*) {
+ my ($row, $disk) = $record->@{qw(row disk)};
+ $san_rows{$row} = 1
+ if defined($disk->{slaves})
+ || defined($row->{'iscsi-target'})
+ || ($row->{transport} // '') =~ m/^(?:iscsi|fc|nvme-(?:tcp|rdma|fc))$/;
+ my $dev = abs_path($row->{devpath});
+ if (!defined($dev) || ($san_rows{$row} && !san_probe_ready($row, $disk, $device_map))) {
+ $uncertain->($row, 'No usable path for checking device signatures');
+ next;
+ }
+ my $parts = eval { PVE::Diskmanage::wipe_partitions($dev) };
+ if (!$parts) {
+ $uncertain->($row, "Could not inspect partitions: $@");
+ next;
+ }
+ push $eligible{$dev}->@*, $row;
+ for my $candidate ($dev, map { "/dev/$_" } @$parts) {
+ $candidate = abs_path($candidate) // $candidate;
+ $devices{$candidate} = 1;
+ $parents{$candidate}->{$dev} = 1;
+ }
+ }
+ return if !%devices;
+
+ my $output = '';
+ my $pvs = eval {
+ PVE::Storage::StatusCommand::run(
+ $cache,
+ [
+ '/sbin/pvs',
+ '--nohints',
+ '--readonly',
+ '--binary',
+ '--devices',
+ join(',', sort keys %devices),
+ '--reportformat',
+ 'json',
+ '--options',
+ 'pv_name,vg_name,pv_uuid,pv_in_use',
+ ],
+ outfunc => sub($line) { $output .= "$line\n" },
+ );
+ my $report = PVE::Diskmanage::parse_pv_report($output);
+ my %seen;
+ for my $pv ($report->@*) {
+ die "Malformed PV in pvs report\n"
+ if !defined($pv->{pv_uuid}) || ref($pv->{pv_uuid});
+ my $dev = abs_path($pv->{pv_name}) // $pv->{pv_name};
+ die "Invalid PV in pvs report\n"
+ if !$devices{$dev}
+ || $pv->{pv_uuid} !~ m/^[a-zA-Z0-9]{6}(?:-[a-zA-Z0-9]{4}){5}-[a-zA-Z0-9]{6}$/
+ || $seen{$dev}++;
+ $pv->{pv_name} = $dev;
+ }
+ $report;
+ };
+ if (!$pvs) {
+ my $error = "LVM report failed: $@";
+ $uncertain->($_, $error) for map { @$_ } values %eligible;
+ return;
+ }
+ my %pv_in_use;
+ for my $pv (sort { $a->{vg_name} cmp $b->{vg_name} } @$pvs) {
+ my $unknown_vg = $pv->{vg_name} eq '[unknown]';
+ for my $parent (sort keys $parents{ $pv->{pv_name} }->%*) {
+ # Missing metadata does not prove that a PV is orphaned, whatever its in-use flag says.
+ push $pv_in_use{$parent}->@*, $unknown_vg ? '1' : $pv->{pv_in_use};
+ for my $row ($eligible{$parent}->@*) {
+ $row->{usage} = 'lvm';
+ if ($unknown_vg) {
+ $row->{detail} = 'Volume group metadata is not visible on this node'
+ if !$row->{vgname};
+ } elsif ($pv->{vg_name} ne '') {
+ $row->{vgname} //= $pv->{vg_name};
+ delete $row->{detail};
+ } elsif (!$row->{vgname} && $pv->{pv_in_use} ne '0') {
+ $row->{detail} = 'Volume group metadata is not visible on this node';
+ }
+ }
+ }
+ }
+ for my $dev (sort keys %eligible) {
+ if (my $flags = $pv_in_use{$dev}) {
+ my $in_use = grep { $_ eq '1' } @$flags;
+ my $unknown = grep { $_ ne '0' && $_ ne '1' } @$flags;
+ if ($in_use || !$unknown) {
+ $_->{'pv-in-use'} = $in_use ? 1 : 0 for $eligible{$dev}->@*;
+ }
+ }
+ # Local disks only need VG attribution; keep their disk-scan usage without signature probes.
+ my @rows = grep { $san_rows{$_} && $_->{usage} eq 'unused' } $eligible{$dev}->@*;
+ next if !@rows;
+ my $types = eval {
+ $output = '';
+ PVE::Storage::StatusCommand::run(
+ $cache,
+ ['wipefs', '--no-act', '--json', $dev],
+ outfunc => sub($line) { $output .= "$line\n" },
+ );
+ my $signatures = PVE::Diskmanage::parse_wipefs_signatures($output);
+ +{ map { $_->{type} => 1 } $signatures->@* };
+ };
+ if (!$types) {
+ my $error = "Checking device signatures failed: $@";
+ $uncertain->($_, $error) for @rows;
+ } elsif (%$types) {
+ my $partition_table = grep { /^(?:gpt|PMBR|dos)$/i } keys %$types;
+ my $detail = $partition_table ? 'partitions' : join(', ', sort keys %$types);
+ for my $row (@rows) {
+ $row->{usage} = 'other';
+ $row->{detail} = $detail;
+ }
+ }
+ }
+}
+
my sub collect_inventory($cfg, $node) {
# Collect identities before disks so path_wwid() rejects names reused during enumeration.
my $ids = PVE::Multipath::Identity::collect(all => 1);
- my $disks = PVE::Diskmanage::get_disks(undef, 1, 0);
- my $multipath = PVE::Diskmanage::get_multipath_disks();
+ my $disks = PVE::Diskmanage::get_disks(undef, 1, 0, 1);
+ my $multipath = PVE::Diskmanage::get_multipath_disks(1);
# Path state is auxiliary information; a multipathd failure must not break the scan.
my $mp_status = {};
@@ -74,22 +248,6 @@ my sub collect_inventory($cfg, $node) {
my $nvme_status = eval { PVE::Diskmanage::get_nvme_path_status() } // {};
warn $@ if $@;
- # map PV device to VG name, PVs may sit on a partition of a listed device
- my $vgs = PVE::Storage::LVMPlugin::lvm_vgs(1);
- my $pv2vg = {};
- for my $vgname (sort keys %$vgs) {
- for my $pv (@{ $vgs->{$vgname}->{pvs} // [] }) {
- $pv2vg->{ $pv->{name} } = $vgname;
- }
- }
- my $find_vg = sub($devpath) {
- return $pv2vg->{$devpath} if defined($pv2vg->{$devpath});
- for my $pv (sort keys %$pv2vg) {
- return $pv2vg->{$pv} if $pv =~ m/^\Q$devpath\Ep?\d+$/;
- }
- return undef;
- };
-
my $vg_used_by = {};
my $iscsi_target_used_by = {};
my $serves_images = sub($sid) {
@@ -114,13 +272,7 @@ my sub collect_inventory($cfg, $node) {
}
}
- # map kernel device name to its iSCSI session and to the storage using that target
my $device_map = PVE::Storage::ISCSIPlugin::iscsi_session_device_map();
- my $iscsi_used_by = {};
- for my $bdev (sort keys %$device_map) {
- my $sid = $iscsi_target_used_by->{ $device_map->{$bdev}->{target} } // next;
- $iscsi_used_by->{$bdev} = $sid;
- }
# A pre-existing allow-list entry can be managed for a storage without being owned by PVE.
my $mp_current = { map { $_ => 1 } PVE::Multipath::list_etc_multipath_wwids()->@* };
@@ -156,8 +308,6 @@ my sub collect_inventory($cfg, $node) {
mp_owned => $mp_owned,
mp_storage => $mp_storage,
mp_expected => $mp_expected,
- find_vg => $find_vg,
- iscsi_used_by => $iscsi_used_by,
};
}
@@ -169,19 +319,14 @@ my sub build_lun_rows($cfg, $inventory) {
$mp_status,
$nvme_status,
$device_map,
- $vg_used_by,
$mp_current,
$mp_explicit,
$mp_owned,
$mp_storage,
$mp_expected,
- $find_vg,
- $iscsi_used_by,
)
- = $inventory->@{
- qw(ids disks multipath mp_status nvme_status device_map
- vg_used_by mp_current mp_explicit mp_owned mp_storage mp_expected find_vg iscsi_used_by)
- };
+ = $inventory->@{ qw(ids disks multipath mp_status nvme_status device_map
+ mp_current mp_explicit mp_owned mp_storage mp_expected) };
my $records = [];
my $add_entry = sub($disk) {
@@ -212,12 +357,6 @@ my sub build_lun_rows($cfg, $inventory) {
$entry->{usage} = 'unused';
} elsif ($disk->{used} eq 'LVM') {
$entry->{usage} = 'lvm';
- if (defined(my $vgname = $find_vg->($disk->{devpath}))) {
- $entry->{vgname} = $vgname;
- if (defined(my $sid = $vg_used_by->{$vgname})) {
- $entry->{'used-by-storage'} = $sid;
- }
- }
} else {
$entry->{usage} = 'other';
$entry->{detail} = $disk->{used};
@@ -227,8 +366,6 @@ my sub build_lun_rows($cfg, $inventory) {
my $session_device = $device_map->{$member} // next;
$entry->{'iscsi-target'} //= $session_device->{target};
$entry->{'iscsi-lun'} //= $session_device->{lun};
- $entry->{'used-by-storage'} //= $iscsi_used_by->{$member}
- if defined($iscsi_used_by->{$member});
}
my $wwid;
if (defined($disk->{slaves})) {
@@ -278,7 +415,13 @@ my sub build_lun_rows($cfg, $inventory) {
sub scan($cfg, $node) {
my $inventory = collect_inventory($cfg, $node);
my $records = build_lun_rows($cfg, $inventory);
-
+ probe_usage($records, $inventory->{device_map});
+ for my $record ($records->@*) {
+ my $row = $record->{row};
+ my $sid = $inventory->{vg_used_by}->{ $row->{vgname} // '' }
+ // $inventory->{iscsi_target_used_by}->{ $row->{'iscsi-target'} // '' };
+ $row->{'used-by-storage'} = $sid if defined($sid);
+ }
flag_duplicate_pvs($records);
return [map { $_->{row} } $records->@*];
}
diff --git a/src/test/run_san_lun_scan_tests.pl b/src/test/run_san_lun_scan_tests.pl
index af8b4870..b2a95d32 100755
--- a/src/test/run_san_lun_scan_tests.pl
+++ b/src/test/run_san_lun_scan_tests.pl
@@ -11,6 +11,8 @@ use warnings;
use lib '..';
use File::Path;
use File::Temp;
+use FindBin qw($Bin);
+use JSON qw(decode_json encode_json);
use Test::MockModule;
use Test::More;
@@ -47,14 +49,40 @@ my sub sd {
my @order;
my $diskmanage = Test::MockModule->new('PVE::Diskmanage');
my $hint = { 'in-use' => 'no', 'in-use-by' => '' };
-my @hint_calls;
my $by_id;
+my @hint_calls;
+my ($partitions, $states) = ({}, {});
+my ($pv_report, $probe_output, $probe_failure) = ([], {}, {});
+my @probe_calls;
+my $commands = Test::MockModule->new('PVE::Storage::StatusCommand');
+$commands->mock(
+ run => sub {
+ my ($cache, $cmd, %opt) = @_;
+ push @probe_calls, [$cache, $cmd];
+ die $cache->{'status-command-error'} if $cache->{'status-command-error'};
+ if (my $error = $probe_failure->{ $cmd->[-1] } // $probe_failure->{ $cmd->[0] }) {
+ $cache->{'status-command-error'} = $error;
+ die $error;
+ }
+ if ($cmd->[0] eq '/sbin/pvs') {
+ $opt{outfunc}->(
+ ref($pv_report)
+ ? encode_json({ report => [{ pv => $pv_report }] })
+ : $pv_report
+ );
+ } else {
+ $opt{outfunc}->($probe_output->{ $cmd->[-1] } // '{"signatures":[]}');
+ }
+ },
+);
$diskmanage->mock(
get_disks => sub {
+ die 'unbounded disk LVM inventory' if !$_[3];
push @order, 'disks';
return { map { ("/dev/$_" => $devices->{$_}->{disk}) } keys %$devices };
},
get_multipath_disks => sub {
+ die 'unbounded multipath LVM inventory' if !$_[0];
return {
map {
my $wwid = $_;
@@ -72,6 +100,7 @@ $diskmanage->mock(
};
},
get_nvme_path_status => sub { {} },
+ wipe_partitions => sub { $partitions->{ $_[0] } // [] },
block_device_usage_hint => sub { push @hint_calls, [@_]; $hint },
canonical_blockdev_link => sub { $by_id },
udev_properties => sub {
@@ -98,6 +127,9 @@ $scan->mock(
my ($file) = @_;
return $namespace_ids->{$1} if $file =~ m|^/sys/block/(\w+)/wwid$|;
return $namespace_ids->{$1} if $file =~ m|^/sys/block/dm-[^/]+/slaves/(\w+)/wwid$|;
+ return $states->{$file} if exists($states->{$file});
+ return 'LOGGED_IN' if $file =~ m{/iscsi_session/session\d+/state$};
+ return $file =~ m{/nvme\d+n\d+/} ? 'live' : 'running' if $file =~ m{/device/state$};
return undef;
},
);
@@ -110,7 +142,7 @@ $multipath->mock(
my $cluster_config = Test::MockModule->new('PVE::Multipath::ClusterConfig');
$cluster_config->mock(read_config => sub { { ids => {} } });
my $lvm = Test::MockModule->new('PVE::Storage::LVMPlugin');
-$lvm->mock(lvm_vgs => sub { {} });
+$lvm->mock(lvm_vgs => sub { die 'unbounded VG inventory' });
my $storage = Test::MockModule->new('PVE::Storage');
# names whose device changed since the collection, and what the storages derive from
my ($reused, $collected, $derived_with) = ({});
@@ -122,8 +154,17 @@ my $iscsi = Test::MockModule->new('PVE::Storage::ISCSIPlugin');
$iscsi->mock(
iscsi_session_device_map => sub {
return {
- map { $_ => { target => $T, channel => 0, id => 0, lun => $devices->{$_}->{lun} } }
- grep { defined($devices->{$_}->{lun}) } keys %$devices
+ map {
+ $_ => {
+ target => $T,
+ session => 1,
+ channel => 0,
+ id => 0,
+ lun => $devices->{$_}->{lun},
+ }
+ } grep {
+ defined($devices->{$_}->{lun})
+ } keys %$devices
};
},
);
@@ -434,4 +475,471 @@ subtest 'direct image consumers win target attribution' => sub {
is(ref($row->{'path-list'}), 'ARRAY', 'path list');
};
+subtest 'bounded hint-free usage probes' => sub {
+ ($devices, $maps) = ({ sd('sdb', 0), sd('sdc', 1), sd('sdd', 2) }, {});
+ $storage->mock(config => sub { { ids => {} } });
+ @probe_calls = ();
+ my $rows = scan();
+ is($rows->{sdb}->{usage}, 'unused', 'successful empty probe permits unused');
+ is(scalar(@probe_calls), 4, 'one PV report and one signature probe per candidate');
+ my $cache = $probe_calls[0]->[0];
+ is($cache->{'status-command-budget'}, 10, 'all probes share ten seconds');
+ is_deeply(
+ $probe_calls[0]->[1],
+ [
+ '/sbin/pvs',
+ '--nohints',
+ '--readonly',
+ '--binary',
+ '--devices',
+ '/dev/sdb,/dev/sdc,/dev/sdd',
+ '--reportformat',
+ 'json',
+ '--options',
+ 'pv_name,vg_name,pv_uuid,pv_in_use',
+ ],
+ 'PV inventory is restricted, read-only and hint-free',
+ );
+ is($_->[0], $cache, 'same budget for each command') for @probe_calls;
+
+ for my $type ('gpt', 'PMBR', 'dos', 'ext4', 'LVM2_member') {
+ $probe_output->{'/dev/sdb'} = encode_json({ signatures => [{ type => $type }] });
+ my $row = scan()->{sdb};
+ is($row->{usage}, 'other', "$type detected despite empty cached inventory");
+ is($row->{detail}, $type =~ /gpt|PMBR|dos/ ? 'partitions' : $type, 'normalized detail');
+ }
+ for my $bad ('{}', '{', '{"signatures":[{}]}', '{"signatures":[{"type":[]}]}') {
+ $probe_output->{'/dev/sdb'} = $bad;
+ my $row = scan()->{sdb};
+ is($row->{usage}, 'unknown', 'malformed signature result is not unused');
+ like(
+ $row->{detail},
+ qr/^Checking device signatures failed: /,
+ 'failed probe has diagnostic detail',
+ );
+ unlike($row->{detail}, qr/Could not check/, 'detail does not duplicate the UI prefix');
+ is($row->{'in-use'}, 'unknown', 'later local hint does not erase uncertainty');
+ }
+ $probe_output = {};
+ $probe_failure->{'/dev/sdb'} = "wait budget exhausted\n";
+ $rows = scan();
+ for my $dev (qw(sdb sdc sdd)) {
+ is($rows->{$dev}->{usage}, 'unknown', "$dev failed or suppressed after probe failure");
+ is(
+ $rows->{$dev}->{detail},
+ 'Checking device signatures failed: wait budget exhausted',
+ "$dev signature diagnostic omits the UI prefix and trailing newline",
+ );
+ is(
+ $rows->{$dev}->{'in-use-by'},
+ $rows->{$dev}->{detail},
+ 'usage hint shares diagnostic',
+ );
+ }
+ $hint = { 'in-use' => 'yes', 'in-use-by' => 'held by active-lv' };
+ is(scan()->{sdb}->{'in-use'}, 'yes', 'positive holder hint survives failed data probe');
+ $hint = { 'in-use' => 'no', 'in-use-by' => '' };
+ $probe_failure = { '/sbin/pvs' => "another status command is still running\n" };
+ @probe_calls = ();
+ $rows = scan();
+ is($rows->{sdb}->{usage}, 'unknown', 'stalled inventory makes unused uncertain');
+ is(
+ $rows->{sdb}->{detail},
+ 'LVM report failed: another status command is still running',
+ 'LVM diagnostic omits the UI prefix and trailing newline',
+ );
+ is($rows->{sdb}->{'in-use-by'}, $rows->{sdb}->{detail}, 'usage hint shares LVM diagnostic');
+ is(scalar(@probe_calls), 1, 'no signature probes after failed inventory');
+ $devices->{sdb}->{disk}->{used} = 'LVM';
+ $devices->{sdc}->{disk}->{used} = 'ext4';
+ $rows = scan();
+ is($rows->{sdb}->{usage}, 'lvm', 'positive LVM usage never downgraded');
+ is($rows->{sdc}->{detail}, 'ext4', 'positive filesystem usage never downgraded');
+ $probe_failure = {};
+ $pv_report = '{}';
+ is(scan()->{sdd}->{usage}, 'unknown', 'malformed PV report fails closed');
+ $pv_report = [];
+};
+
+subtest 'usable paths and candidate deduplication' => sub {
+ ($devices, $maps) = ({ sd('sdb', 0), sd('sdc', 1) }, {});
+ $states = { '/sys/class/iscsi_session/session1/state' => 'FAILED' };
+ @probe_calls = ();
+ is(scan()->{sdb}->{usage}, 'unknown', 'logged-out iSCSI path not probed');
+ is(scalar(@probe_calls), 0, 'no device I/O without a logged-in session');
+ $states = { '/sys/class/block/sdb/device/state' => 'offline' };
+ is(scan()->{sdb}->{usage}, 'unknown', 'offline SCSI path not probed');
+ $states = {};
+ $devices->{sdc}->{disk}->{devpath} = '/dev/sdb';
+ @probe_calls = ();
+ scan();
+ is(scalar(@probe_calls), 2, 'same canonical device is probed only once');
+ ($devices, $maps) = ({}, { '3W0' => { slaves => ['sdb'] } });
+ $multipath->mock(
+ path_status_by_wwid => sub {
+ {
+ '3W0' =>
+ { active => 0, paths => [{ state => 'failed', 'checker-state' => 'faulty' }] },
+ };
+ },
+ );
+ @probe_calls = ();
+ scan();
+ is(scalar(@probe_calls), 0, 'failed checker prevents map probe despite running SCSI state');
+ $multipath->mock(path_status_by_wwid => sub { {} });
+ @probe_calls = ();
+ scan();
+ is(scalar(@probe_calls), 1, 'running map member permits PV report without daemon status');
+ $states = { '/sys/class/block/sdb/device/state' => 'offline' };
+ @probe_calls = ();
+ scan();
+ is(scalar(@probe_calls), 0, 'map with no running path not probed');
+ $states = {};
+ ($devices, $maps) = (
+ {
+ nvme0n1 => {
+ disk => {
+ devpath => '/dev/nvme0n1',
+ size => 1024,
+ transport => 'nvme-tcp',
+ },
+ },
+ },
+ {},
+ );
+ $diskmanage->mock(
+ get_nvme_path_status => sub {
+ {
+ nvme0n1 =>
+ { active => 0, paths => [{ state => 'live', 'ana-state' => 'inaccessible' }] },
+ };
+ },
+ );
+ @probe_calls = ();
+ is(scan()->{nvme0n1}->{usage}, 'unknown', 'inaccessible native NVMe not probed');
+ is(scalar(@probe_calls), 0, 'no I/O on inaccessible namespace');
+ $diskmanage->mock(
+ get_nvme_path_status => sub {
+ {
+ nvme0n1 =>
+ { active => 1, paths => [{ state => 'live', 'ana-state' => 'non-optimized' }] },
+ };
+ },
+ );
+ is(scan()->{nvme0n1}->{usage}, 'unused', 'live accessible native NVMe probed');
+ $diskmanage->mock(get_nvme_path_status => sub { {} });
+};
+
+subtest 'fresh VG and storage attribution including partition aliases' => sub {
+ ($devices, $maps) = ({ sd('sdb', 0), sd('sdc', 1, 'PV1') }, {
+ '3W0' => { slaves => ['sdd'] },
+ });
+ $partitions = { '/dev/dm-3W0' => ['dm-9'] };
+ $storage->mock(
+ config => sub { {
+ ids => {
+ base => { type => 'iscsi', target => $T, content => { none => 1 } },
+ data => { type => 'lvm', vgname => 'foreign' },
+ },
+ } },
+ );
+ $pv_report = [
+ map { {
+ pv_name => $_,
+ vg_name => 'foreign',
+ pv_in_use => '1',
+ pv_uuid => 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg',
+ } } qw(/dev/sdb /dev/sdc /dev/mapper/9)
+ ];
+ my $rows = scan();
+ for my $dev (qw(sdb sdc 3W0)) {
+ is($rows->{$dev}->{usage}, 'lvm', "$dev has fresh LVM attribution");
+ is($rows->{$dev}->{vgname}, 'foreign', "$dev has fresh VG name");
+ is(
+ $rows->{$dev}->{'used-by-storage'},
+ 'data',
+ "$dev owner recomputed after classification",
+ );
+ }
+ $devices->{sdb}->{disk}->{used} = 'LVM2_member';
+ is(scan()->{sdb}->{vgname}, 'foreign', 'cached LVM signature receives VG attribution');
+ $pv_report->[0]->{vg_name} = '';
+ like(
+ scan()->{sdb}->{detail},
+ qr/metadata is not visible/,
+ 'used PV without VG is not an orphan',
+ );
+ $pv_report = [];
+ $partitions = {};
+};
+
+subtest 'PV use is explicit orphan proof' => sub {
+ ($devices, $maps) = ({ sd('sdb', 0, 'PV0') }, {});
+ is($declared->{'pv-in-use'}->{type}, 'boolean', 'PV use is a declared boolean');
+ ok($declared->{'pv-in-use'}->{optional}, 'unknown PV use can be omitted');
+ $pv_report = [{
+ pv_name => '/dev/sdb',
+ vg_name => '',
+ pv_in_use => '0',
+ pv_uuid => 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg',
+ }];
+ for my $vg ('', 'foreign', 'unknown', '[unknown]') {
+ $pv_report->[0]->{vg_name} = $vg;
+ for my $flag ('0', '1', 'unknown', '') {
+ $pv_report->[0]->{pv_in_use} = $flag;
+ my $row = scan()->{sdb};
+ is($row->{usage}, 'lvm', "VG '$vg', flag '$flag' retains LVM usage");
+ if ($vg eq '[unknown]') {
+ is(
+ $row->{'pv-in-use'},
+ 1,
+ "sentinel overrides use flag '$flag' without orphan proof",
+ );
+ ok(!exists($row->{vgname}), 'sentinel is not exported as a VG name');
+ is(
+ $row->{detail},
+ 'Volume group metadata is not visible on this node',
+ 'sentinel explains missing metadata',
+ );
+ } else {
+ is($row->{vgname} // '', $vg, 'ordinary VG name retained');
+ if ($flag eq '0' || $flag eq '1') {
+ is($row->{'pv-in-use'}, int($flag), "VG '$vg' exports PV use $flag");
+ } else {
+ ok(
+ !exists($row->{'pv-in-use'}),
+ "VG '$vg', flag '$flag' omits unknown PV use",
+ );
+ }
+ }
+ }
+ }
+ $pv_report = [];
+ ok(!exists(scan()->{sdb}->{'pv-in-use'}), 'unreported cached PV has no orphan proof');
+ $pv_report = '{}';
+ ok(!exists(scan()->{sdb}->{'pv-in-use'}), 'malformed report has no orphan proof');
+ $pv_report = [];
+ $probe_failure = { '/sbin/pvs' => "PV report failed\n" };
+ ok(!exists(scan()->{sdb}->{'pv-in-use'}), 'failed report has no orphan proof');
+ $probe_failure = {};
+
+ $partitions = { '/dev/sdb' => ['sdb1', 'sdb2'] };
+ $pv_report = [
+ map { {
+ pv_name => $_,
+ vg_name => '',
+ pv_in_use => '0',
+ pv_uuid => 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg',
+ } } qw(/dev/sdb1 /dev/sdb2)
+ ];
+ is(scan()->{sdb}->{'pv-in-use'}, 0, 'unused partition PVs report unused');
+ $pv_report->[0]->{pv_in_use} = '1';
+ is(scan()->{sdb}->{'pv-in-use'}, 1, 'used partition PV is not hidden by an unused PV');
+ $pv_report->[0]->{pv_in_use} = 'unknown';
+ ok(!exists(scan()->{sdb}->{'pv-in-use'}), 'unknown partition PV prevents orphan proof');
+ $pv_report->[1]->{pv_in_use} = '1';
+ is(scan()->{sdb}->{'pv-in-use'}, 1, 'positive use survives another unknown PV');
+ $_->{pv_in_use} = '0' for @$pv_report;
+ $pv_report->[0]->{vg_name} = '[unknown]';
+ is(scan()->{sdb}->{'pv-in-use'}, 1, 'sentinel partition prevents whole-disk orphan proof');
+ $pv_report = [];
+ $partitions = {};
+};
+
+subtest 'unknown VG membership preserves unrelated scan rows' => sub {
+ ($devices, $maps) = ({ sd('sdb', 0), sd('sdc', 1), sd('sdd', 2), sd('sde', 3) }, {});
+ $storage->mock(
+ config => sub { { ids => { data => { type => 'lvm', vgname => 'foreign' } } } });
+ $pv_report = [
+ map { {
+ pv_name => $_->[0],
+ vg_name => $_->[1],
+ pv_in_use => $_->[2],
+ pv_uuid => 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg',
+ } }
+ (['/dev/sdb', '[unknown]', '1'], ['/dev/sdc', '', '0'], ['/dev/sdd', 'foreign', '1'])
+ ];
+ my $rows = scan();
+ is($rows->{sdb}->{usage}, 'lvm', 'unknown membership retains LVM usage');
+ ok(!exists($rows->{sdb}->{vgname}), 'sentinel is not a usable VG name');
+ is($rows->{sdb}->{'pv-in-use'}, 1, 'unknown membership has no orphan proof');
+ is($rows->{sdc}->{'pv-in-use'}, 0, 'unrelated orphan remains unused');
+ is($rows->{sdd}->{vgname}, 'foreign', 'unrelated named VG survives');
+ is($rows->{sdd}->{'used-by-storage'}, 'data', 'unrelated VG ownership survives');
+ is($rows->{sde}->{usage}, 'unused', 'unrelated blank LUN remains usable');
+
+ for my $flag (undef, [], {}, 'maybe') {
+ delete $pv_report->[0]->{pv_in_use};
+ $pv_report->[0]->{pv_in_use} = $flag if defined($flag);
+ $rows = scan();
+ is($rows->{sdb}->{usage}, 'unknown', 'malformed or missing use flag fails closed');
+ ok(!exists($rows->{sdb}->{'pv-in-use'}), 'malformed use flag is not orphan proof');
+ }
+ $pv_report->[0]->{pv_in_use} = '1';
+ for my $vg ('[missing]', '[UNKNOWN]', [], undef) {
+ $pv_report->[0]->{vg_name} = $vg;
+ is(scan()->{sdb}->{usage}, 'unknown', 'unrecognized VG field still fails closed');
+ }
+ $pv_report->[0]->{vg_name} = '[unknown]';
+ $pv_report->[0]->{pv_name} = '[unknown]';
+ is(scan()->{sdb}->{usage}, 'unknown', 'missing-device spelling still fails closed');
+ $pv_report = [];
+};
+
+subtest 'captured no-metadata PV and orphan scan report' => sub {
+ my $capture = PVE::Tools::file_get_contents("$Bin/lvm-fixtures/lab1-nomda-orphan.txt");
+ my ($json) = $capture =~ m/\A== devices B,C\n(.*?)^== full nohints /ms;
+ die "missing captured JSON report\n" if !defined($json);
+ my $captured = decode_json($json);
+ is(scalar($captured->{log}->@*), 2, 'both captured log warnings remain in the report');
+ is($_->{log_type}, 'warn', 'captured log entry is a warning') for $captured->{log}->@*;
+ ($devices, $maps) = ({ sd('sde', 2) }, {});
+ $storage->mock(config => sub { { ids => {} } });
+ my $member = 'sdb';
+
+ for my $pv ($captured->{report}->[0]->{pv}->@*) {
+ my $wwid = $pv->{pv_name} =~ s{^/dev/mapper/}{}r;
+ $maps->{$wwid} = { slaves => [$member++], pv => $pv->{pv_uuid} };
+ }
+ $pv_report = $json;
+ @probe_calls = ();
+ my $rows = scan();
+ for my $pv ($captured->{report}->[0]->{pv}->@*) {
+ my $wwid = $pv->{pv_name} =~ s{^/dev/mapper/}{}r;
+ my $row = $rows->{$wwid};
+ is($row->{usage}, 'lvm', 'captured PV is classified as LVM');
+ ok(!exists($row->{vgname}), 'captured PV does not export a usable VG name');
+ is($row->{'pv-in-use'}, int($pv->{pv_in_use}), 'captured PV use flag is preserved');
+ if ($pv->{vg_name} eq '[unknown]') {
+ is(
+ $row->{detail},
+ 'Volume group metadata is not visible on this node',
+ 'captured no-metadata PV has the specific diagnostic',
+ );
+ } else {
+ ok(!exists($row->{detail}), 'captured orphan has no missing-metadata diagnostic');
+ }
+ }
+ is($rows->{sde}->{usage}, 'unused', 'unrelated blank LUN remains usable');
+ is(scalar(@probe_calls), 2, 'captured report succeeds and only the blank LUN needs wipefs');
+ $pv_report = [];
+};
+
+subtest 'local PVs share the bounded report without SAN probes' => sub {
+ ($devices, $maps) = (
+ { map { sd($_, undef) } qw(sda sdb sdc sdd sde sdf) }, {},
+ );
+ my %transport =
+ (sda => 'sata', sdb => 'sas', sdc => 'usb', sdd => 'unknown', sde => 'sata');
+ $devices->{$_}->{disk}->{transport} = $transport{$_} for keys %transport;
+ $devices->{sda}->{disk}->{used} = 'LVM2_member';
+ $devices->{sdb}->{disk}->{used} = 'partitions';
+ $devices->{sdc}->{disk}->{used} = 'ext4';
+ $devices->{sde}->{disk}->{used} = 'LVM';
+ $devices->{sdf}->{lun} = 0;
+ $devices->{nvme0n1} = {
+ disk =>
+ { devpath => '/dev/nvme0n1', size => 1024, transport => 'nvme', used => 'partitions' },
+ };
+ $partitions = { '/dev/sdb' => ['sdb1'], '/dev/nvme0n1' => ['nvme0n1p1'] };
+ # Local rows need no running-path or session evidence for VG attribution.
+ $states = { map { ("/sys/class/block/$_/device/state" => 'offline') }
+ qw(sda sdb sdc sdd sde nvme0n1) };
+ $storage->mock(
+ config => sub { { ids => { localvg => { type => 'lvm', vgname => 'pve' } } } });
+ $pv_report = [
+ map { {
+ pv_name => $_,
+ vg_name => 'pve',
+ pv_in_use => '1',
+ pv_uuid => 'aaaaaa-bbbb-cccc-dddd-eeee-ffff-gggggg',
+ } } qw(/dev/sda /dev/sdb1 /dev/nvme0n1p1)
+ ];
+ @probe_calls = ();
+ my $rows = scan();
+ for my $dev (qw(sda sdb nvme0n1)) {
+ is($rows->{$dev}->{usage}, 'lvm', "$dev local whole-disk or partition PV classified");
+ is($rows->{$dev}->{vgname}, 'pve', "$dev local VG restored");
+ is($rows->{$dev}->{'used-by-storage'}, 'localvg', "$dev local storage attributed");
+ is($rows->{$dev}->{'pv-in-use'}, 1, "$dev local PV use reported");
+ ok(
+ !exists($rows->{$dev}->{detail}),
+ "$dev cached signature or partition detail cleared",
+ );
+ }
+ is_deeply(
+ [map { $_->[1] } @probe_calls],
+ [
+ [
+ '/sbin/pvs',
+ '--nohints',
+ '--readonly',
+ '--binary',
+ '--devices',
+ '/dev/nvme0n1,/dev/nvme0n1p1,/dev/sda,/dev/sdb,/dev/sdb1,/dev/sdc,/dev/sdd,/dev/sde,/dev/sdf',
+ '--reportformat',
+ 'json',
+ '--options',
+ 'pv_name,vg_name,pv_uuid,pv_in_use',
+ ],
+ ['wipefs', '--no-act', '--json', '/dev/sdf'],
+ ],
+ 'one hint-free report includes local disks and partitions; only SAN gets wipefs',
+ );
+ is(
+ $probe_calls[0]->[0],
+ $probe_calls[1]->[0],
+ 'local PVs use the shared SAN command budget',
+ );
+ is($probe_calls[0]->[0]->{'status-command-budget'}, 10,
+ 'shared budget remains ten seconds');
+
+ $pv_report = [];
+ my $baseline = scan();
+ is($baseline->{sdd}->{usage}, 'unused', 'unreported unused local disk stays unused');
+ is($baseline->{sda}->{detail}, 'LVM2_member', 'unreported local signature is retained');
+ is($baseline->{sdb}->{detail}, 'partitions', 'unreported local partitions are retained');
+ for my $failure ('command failure', 'malformed report') {
+ $probe_failure =
+ $failure eq 'command failure' ? { '/sbin/pvs' => "report failed\n" } : {};
+ $pv_report = '{}';
+ @probe_calls = ();
+ $rows = scan();
+ for my $dev (qw(sda sdb sdc sdd sde nvme0n1)) {
+ is_deeply(
+ $rows->{$dev},
+ $baseline->{$dev},
+ "$failure preserves local $dev disk scan",
+ );
+ }
+ is($rows->{sdf}->{usage}, 'unknown', "$failure still marks unused SAN row unknown");
+ is(scalar(@probe_calls), 1, "$failure stops signature probes");
+ }
+ $probe_failure = {};
+ $pv_report = [];
+ $partitions = {};
+ $states = {};
+};
+
+subtest 'disk inventory can skip global LVM reads' => sub {
+ $diskmanage->unmock('get_disks');
+ $diskmanage->unmock('get_multipath_disks');
+ $diskmanage->mock(
+ mounted_blockdevs => sub { {} },
+ get_multipath_slave_map => sub { {} },
+ get_lsblk_info => sub { {} },
+ get_zfs_devices => sub { {} },
+ get_ceph_volume_infos => sub { die 'unexpected global LV scan' },
+ dir_glob_foreach => sub { },
+ run_command => sub { die 'unexpected global PV scan' },
+ );
+ is_deeply(
+ PVE::Diskmanage::get_disks(undef, 1, 0, 1),
+ {},
+ 'disk inventory skips PV and LV scans',
+ );
+ is_deeply(PVE::Diskmanage::get_multipath_disks(1), {},
+ 'map inventory skips global PV scan');
+};
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 docs 43/76] storage: lvm: note the rescan after extending a shared volume group
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (41 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 storage 42/76] api: scan: refresh SAN usage with bounded probes Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 docs 44/76] storage: document the cluster-wide multipath management Thomas Lamprecht
` (32 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
LVM hints can hide a PV added by another node to an existing shared
volume group. Automatic discovery only retries groups missing entirely,
not groups with missing PVs. Document how to rescan after extending a
shared group, including that the rescan resets LVM autoactivation
records.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
pve-storage-lvm.adoc | 9 +++++++++
1 file changed, 9 insertions(+)
diff --git a/pve-storage-lvm.adoc b/pve-storage-lvm.adoc
index ba78663..f00688d 100644
--- a/pve-storage-lvm.adoc
+++ b/pve-storage-lvm.adoc
@@ -121,6 +121,15 @@ A benefit of LVM is that it can be used with shared storage.
For example, an iSCSI LUN. The backend implements proper cluster-wide locking if
the storage is marked as shared in the configuration.
+NOTE: If you extend a shared volume group onto another device on one node, the
+other nodes may only see the added physical volume after LVM has scanned their
+devices again, for example with `pvscan --cache`. This command also resets the
+autoactivation records of LVM, which can let LVM activate logical volumes on its
+own later. {pve} disables autoactivation for the logical volumes it creates.
+Before you run the command, make sure that autoactivation is also disabled for
+all other logical volumes of the volume group, for example with
+`lvchange --setautoactivation n <vg>/<lv>`.
+
TIP: You can use the LVM-thin backend for non-shared local storage. It supports
snapshots and linked clones.
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 docs 44/76] storage: document the cluster-wide multipath management
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (42 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 docs 43/76] storage: lvm: note the rescan after extending a shared volume group Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 45/76] api: ceph: osd: keep iSCSI LUNs out of OSD creation Thomas Lamprecht
` (31 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Document managed multipath setup, health states, and upgrade
requirements so administrators can distinguish incomplete configuration
from unusable paths.
Cover shared LVM setup and recovery in the Remote Storage wizard,
including device checks, unanswered requests, task completion, and
readiness before guest use. Explain safe connection-storage removal,
growing LUNs, and presenting them again without queued I/O blocking
the operation.
Require content none in the iSCSI guidance for LVM, rather than merely
recommending it: initialization is refused while a storage uses the
target for disk images.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
pve-storage-iscsi.adoc | 13 +-
pve-storage-multipath.adoc | 527 +++++++++++++++++++++++++++++++++++++
pvesm.adoc | 13 +-
3 files changed, 545 insertions(+), 8 deletions(-)
create mode 100644 pve-storage-multipath.adoc
diff --git a/pve-storage-iscsi.adoc b/pve-storage-iscsi.adoc
index aea26d2..67055f1 100644
--- a/pve-storage-iscsi.adoc
+++ b/pve-storage-iscsi.adoc
@@ -38,6 +38,13 @@ target::
iSCSI target.
+multipath::
+
+Combine the paths to each LUN of the target into one `dm-multipath` map on every
+node the storage is enabled on, and monitor the maps cluster-wide. A shared LVM
+storage on a target with several portals needs this option. See
+xref:storage_multipath[Multipath].
+
.Configuration Example (`/etc/pve/storage.cfg`)
----
@@ -47,9 +54,9 @@ iscsi: mynas
content none
----
-TIP: If you want to use LVM on top of iSCSI, it make sense to set
-`content none`. That way it is not possible to create VMs using iSCSI
-LUNs directly.
+NOTE: To use LVM on top of iSCSI, set `content none`. When you add an LVM
+storage with a LUN as its base volume, {pve} refuses the operation if any
+configured storage also uses that target for disk images.
File naming conventions
diff --git a/pve-storage-multipath.adoc b/pve-storage-multipath.adoc
new file mode 100644
index 0000000..6f6dc1a
--- /dev/null
+++ b/pve-storage-multipath.adoc
@@ -0,0 +1,527 @@
+[[storage_multipath]]
+Multipath
+---------
+ifdef::wiki[]
+:pve-toplevel:
+:title: Storage: Multipath
+endif::wiki[]
+
+A storage area network (SAN) usually exports each logical unit (LUN) over
+several paths, for example over two iSCSI portals or two Fibre Channel ports.
+Linux sees every path as a separate block device. Device-mapper multipath
+(`dm-multipath`) combines the paths of a LUN into one device, called a
+multipath map, keeps I/O flowing while a path is down, and spreads the load over
+the paths.
+
+{pve} manages `dm-multipath` cluster-wide: it decides which LUNs get a map,
+sets the name and options of each map, and monitors every path on every node. A
+LUN is identified by its World Wide Identifier (WWID), which is the same on
+every node. The configuration is stored in the cluster file system, every node
+generates its local `multipath-tools` configuration from it, and
+*Datacenter -> SAN -> Multipath* shows the health of the maps across the
+cluster.
+
+NOTE: NVMe over Fabrics namespaces use native NVMe multipath in the kernel
+instead of `dm-multipath`. They need no multipath configuration and do not
+appear under *Datacenter -> SAN -> Multipath*. They show up under
+*Node -> SAN*.
+
+Prerequisites
+~~~~~~~~~~~~~
+
+Install `multipath-tools` on every node that accesses the SAN:
+
+----
+# apt install multipath-tools
+----
+
+A node without the package does not take part. *Datacenter -> SAN* shows it as
+*Not installed*, which counts as a problem once a LUN is expected on the node.
+
+Upgrade every node of the cluster before you enable the multipath option of a
+storage. A node with an older version does not know the option and removes it
+from every iSCSI storage the next time it writes the storage configuration,
+even for an unrelated change. The other nodes then stop managing the LUNs of
+those storages. {pve} therefore refuses to enable the option until every
+cluster node has reported multipath support, which a node does while it is
+online. It also refuses while `multipath-tools` is not installed on a node the
+storage is available on, as the volumes of the storage cannot be activated
+there without their maps. Upgrade a node before it joins a cluster that uses the
+option, as the check covers only the nodes that are members when you enable it.
+
+Managing LUNs
+~~~~~~~~~~~~~
+
+{pve} assembles maps only for the LUNs on its allow-list, so unrelated disks,
+the boot disk, and LUNs that other software uses directly stay untouched. A LUN
+gets onto the allow-list in one of two ways, and the *Source* column under
+*Datacenter -> SAN -> Multipath* shows which one applies:
+
+iSCSI storages::
+
+Enable *Multipath* in the edit dialog of the iSCSI storage or in the
+*Remote Storage* wizard, see
+xref:storage_multipath_setup[Setting Up Shared LVM]. Every node the storage is
+enabled on then assembles one map per LUN over the paths through the portals of
+the target. The storage lists its LUNs as multipath maps afterwards, and an LVM
+storage created on top of it uses the map as its base volume, see
+xref:storage_lvm[LVM].
++
+Without the option, {pve} logs in to every portal of the target as well, and
+each LUN shows up once per portal as a separate device. LVM ignores a volume
+group that it finds on several devices, so a shared LVM storage on a target with
+several portals needs the option. The option cannot be disabled while an LVM
+storage uses one of the maps as its base volume.
++
+You do not need to remove and add an existing LVM storage again when you enable
+the option: it can keep its base volume. LVM finds the physical volume by its
+metadata, not by the device name. Keep `multipath_component_detection` enabled
+in the LVM configuration (the default), so that LVM ignores the path devices of
+the LUN and uses the map. Logical volumes that are already active on a single
+path device stay there when you enable the option; before you rely on path
+redundancy, run `lsblk` on each node to see which device they use. Deactivating
+and activating a logical volume again moves it to the map, for example by
+migrating its guest to another node and back.
++
+The same can happen when a node logs in to the target for the first time, for
+example after it joined the cluster, or when the target presents a new LUN: the
+node adds a LUN to its allow-list only after it found it, and until then LVM can
+activate logical volumes with autoactivation enabled on a path device. {pve} 9
+creates new logical volumes for guests with autoactivation disabled. For older
+guest volumes, run
+`/usr/share/pve-manager/migrations/pve-lvm-disable-autoactivation`; for other
+logical volumes, run `lvchange --setautoactivation n <vg>/<lv>`.
++
+If the volume group is not visible yet, for example right after a node booted,
+activating a volume of the LVM storage also activates the base volume and waits
+briefly for the map. For a base volume that is still named by the SCSI
+identifier of the LUN, a timeout only logs a warning and activation continues;
+activation can still fail if the volume group remains unavailable.
+
+Other LUNs::
+
+LUNs attached over Fibre Channel or SAS, and LUNs of an iSCSI target without an
+iSCSI storage, need an explicit entry in the cluster-wide multipath
+configuration, shown as *Explicit entry* in the *Source* column. For Fibre
+Channel, the *Remote Storage* wizard can add the entry together with the volume
+group and the LVM storage, see
+xref:storage_multipath_setup[Setting Up Shared LVM]. To add an entry by hand:
++
+. Under *Node -> SAN*, select the LUN and choose *Enable Multipath*.
+. Review *Nodes*. For a new entry, the dialog preselects this node, the nodes
+ that see the LUN, and the nodes it could not check. Adjust the selection to
+ the nodes that should use the LUN. Optionally set an *Alias*, which becomes
+ the map name on every selected node.
+. Wait for the map to assemble, which takes up to about ten seconds, as every
+ node applies the configuration periodically.
+. Create the volume group on the map with `pvesh create
+ /nodes/<node>/disks/lvm --name <vg> --device
+ /dev/disk/by-id/dm-uuid-mpath-<wwid>` and check its output for errors. Then
+ add the shared LVM storage with
+ `pvesm add lvm <storage> --vgname <vg> --shared 1`.
+
+A map that {pve} does not manage, for example one from a hand-made
+configuration that predates the upgrade, shows up as *Unmanaged* under
+*Node -> SAN*. Select it and choose *Manage* to add it to the cluster-wide
+configuration.
+
+A node keeps a LUN on its allow-list while an explicit entry or an enabled
+storage requires it there. Once neither does, the node removes the LUN from its
+allow-list and `multipathd` releases the map, unless the LUN was added to the
+allow-list outside of {pve}. However, the node keeps the LUN on its allow-list
+while something on the node still uses the map, for example a volume group, a
+file system, or a partition table, as the users would otherwise fall back to a
+single path after the next reboot. The datacenter view and the SAN
+view of the node show such a LUN as *Kept while in use* until the node can
+confirm that the map is unused, including that no data signatures remain. To
+keep the LUN on the allow-list after it is no longer in use, choose *Manage* in
+the SAN view of the node, which adds an explicit entry for it.
+
+{pve} refuses to remove an explicit entry, or to restrict it to fewer nodes,
+while an LVM storage uses the map as its base volume on a node that would then
+drop the LUN from its allow-list. Remove or change that storage first. An LVM
+storage without a base volume, as used for Fibre Channel, is not linked to the
+LUN in the configuration. It is protected only because every node keeps the LUN
+on its allow-list while the volume group is on the map, as described above.
+
+[[storage_multipath_setup]]
+Setting Up Shared LVM
+~~~~~~~~~~~~~~~~~~~~~
+
+The *Remote Storage* wizard under *Datacenter -> Storage -> Add* sets up shared
+LVM storage on a LUN. For iSCSI, it creates the volume group from the node that
+the web interface is connected to. That node must reach the target and be
+included in any node restriction. For Fibre Channel, it uses the selected scan
+node and can enable multipath for the selected LUN, create a volume group or use
+an existing one, and add the LVM storage.
+
+For iSCSI, the wizard first adds or reuses a connection storage: an iSCSI
+storage with `content none` that connects the nodes to the target without
+offering its LUNs as disk images. *Connect and Select LUN* adds the connection
+storage and lists its LUNs. If a connection storage for the target already
+exists, the button is *Select LUN* and lists the LUNs of that storage. Select a
+LUN, then choose *Create Volume Group*. The LVM storage gets the LUN as its
+base volume, so that a node connects to the target before it activates the
+volume group, for example for guests that start at boot. Keep the base volume
+when you add such a storage by hand.
+
+The wizard offers only LUNs that no configured storage uses for data, that are
+neither mounted nor used by another device on the node, and on which the node
+finds no partitions, file system, LVM, or other signature. The node checks the
+device and its users again right before it creates the volume group. If no
+configured storage uses a device for data and nothing on the node uses it,
+`root@pam` can wipe it for reuse: in the wizard after selecting *Allow wiping
+LUNs with data*, or under *Node -> SAN*.
+
+Before you put guests on a new shared storage, check under
+*Datacenter -> SAN -> Multipath* that every node of the storage sees the LUN
+with all its paths.
+
+If a step fails after the wizard has created something, you can no longer
+change the settings in the wizard. *Retry Setup* runs the remaining steps again.
+If creating the LVM storage on iSCSI fails, retries use the same LUN and do not
+return to device selection, as the attempt may have written to it. If the wizard
+could not confirm that it created a storage, *Check and Retry* checks whether
+the storage exists with the expected settings and creates it only if it is
+missing. Closing the wizard does not remove the storages it already created.
+
+If the wizard cannot tell whether a wipe finished, it blocks the device until
+the outcome is known. *Check Task Status* waits for the same wipe task again; it
+does not start another wipe. If the wizard does not know the task ID, check the
+*Task History* of the node before you choose *Dismiss* or use the device.
+
+If the node does not answer while the Fibre Channel wizard creates the volume
+group, the wizard stops, as the task may still be running. Check the
+*Task History* of the node to see whether the task created the volume group.
+Once the task has finished, run the wizard again. If the volume group exists,
+the wizard offers it; select it to complete the setup.
+
+NOTE: When you use the API, a returned task ID only means that the node
+accepted the task, not that the task succeeded. Wait for the task to finish and
+check its exit status before you use the device. `pvesh` runs the task in the
+foreground; check its output for errors.
+
+Health
+~~~~~~
+
+*Datacenter -> SAN* shows one row per node with the state of its multipath
+service, the number of its managed LUNs in each state, and its iSCSI initiator
+name, which the targets of its iSCSI storages have to allow. A node that has not
+reported shows *No report*. A missing `multipath-tools` package or a stopped
+`multipathd` counts as a problem only on a node that should see a LUN.
+
+*Datacenter -> SAN -> Multipath* lists every managed LUN with its state on each
+node:
+
+*Optimal*:: Every path is active and no expected path is missing.
+*Degraded*:: Some paths are down, or fewer paths are present than the storage
+configuration expects.
+*Failed*:: No active path remains.
+*Missing*:: The node should see the LUN but has not assembled its map.
+*Unknown*:: The node has no current report. The view also shows why: the node
+is offline, its report is stale, `multipath-tools` is not installed,
+`multipathd` is not running or did not answer, or the node has too many LUNs to
+list them. A node that runs an older version, or has not reported since its
+cluster file system started, has no report at all.
+
+If the status updates of a node stall, for example while a status query waits
+for I/O to a LUN that lost every path, the view keeps a *Degraded*, *Missing*,
+or *Failed* state from the last report of the node, but shows *Unknown* instead
+of *Optimal*, as that report may be long outdated.
+
+A LUN of an iSCSI storage that no node finds on the target anymore, and that no
+node has a map of, shows *Not currently observed* instead of *Missing*. This
+usually means that the LUN was unmapped on the storage system, on purpose or by
+mistake, so check which one it was. The storage keeps the LUN on the allow-list,
+see xref:storage_multipath_remove[Removing a LUN]. A LUN stays *Missing* if it
+has an explicit entry or an LVM storage uses it. It also stays *Missing* while
+one of its nodes cannot tell whether the LUN is gone: the node runs an older
+version, could not apply the configuration, has no current report or stalled
+status updates, has not completed login and the initial LUN scan on every
+portal of the target, or cannot identify every LUN of the target.
+
+The *Cluster Health* of a LUN is the worst state among the nodes that should see
+it. Nodes without a current report do not count, so *Optimal* means that every
+node with a current report is fine, not that every node has reported. While the
+state on one of these nodes is unknown, the view shows *Optimal (incomplete)*. A
+node that should see the LUN but cannot manage multipath, because
+`multipath-tools` is not installed or `multipathd` is not running or does not
+answer, makes the LUN *Degraded*. An explicit entry without a node restriction
+expects the LUN on every node of the cluster.
+
+Nodes report the state of their LUNs through the cluster file system, which
+limits the size of a report. A node with more than about 200 LUNs reports only
+the number of its LUNs in each state, and the datacenter view refers to the SAN
+view of that node for details.
+
+For an iSCSI storage, {pve} knows how many paths a LUN should have: one per
+portal of the target. If several storages provide the same LUN, the LUN is
+expected on the nodes of all of them, with the highest path count among them.
+For Fibre Channel, {pve} only knows the paths that are currently present. If the
+transport has already removed a failed path, the map shows as *Optimal* with
+fewer paths, so compare the path count with what the SAN should provide.
+
+A node learns the portals of an iSCSI target when it first logs in to it. A
+portal that is added to the target later is neither used nor expected until the
+node discovers the target again. To start using a new portal, run the discovery
+on every node, against any portal of the target:
+
+----
+# iscsiadm --mode discovery --type sendtargets --portal <portal>
+----
+
+The next activation of the storage then also logs in to the new portal.
+
+Above the list, *Datacenter -> SAN -> Multipath* shows the nodes that could not
+apply the configuration, so a node that drifts from the configured state does
+not go unnoticed. It also shows nodes that cannot identify some LUNs of their
+iSCSI storages, because the identifiers of their paths are not available or
+could not be verified, for example while `multipathd` starts. Such a LUN is left
+out until the node can identify it.
+
+*Node -> SAN* lists each LUN of the node with its users and the state of its
+paths, and shows the problems of the node above the list. There, `root@pam` can
+also wipe a LUN that no configured storage uses for data, for example before
+reusing it. The wipe dialog first asks the other nodes what uses the LUN and
+refuses while one of them reports a user. *Node -> SAN -> iSCSI* shows the iSCSI
+targets and sessions of the node and its initiator name.
+
+Permissions
+~~~~~~~~~~~
+
+The multipath views and changes need these privileges:
+
+* Viewing the multipath status: `Sys.Audit` on `/` or `Datastore.Allocate` on
+ `/storage`.
+* Viewing the SAN views of a node: `Sys.Audit` on `/nodes/<node>` or
+ `Datastore.Allocate` on `/storage`.
+* Changing the cluster-wide multipath configuration through the API:
+ `Sys.Modify` on `/`.
+* Using *Enable Multipath*, *Manage*, or *Edit* in the multipath views:
+ `Sys.Modify` and `Sys.Audit` on `/`. The editors read the configuration before
+ changing it.
+* Changing the multipath option of a storage: `Datastore.Allocate` on
+ `/storage`.
+
+[[storage_multipath_grow]]
+Growing a LUN
+~~~~~~~~~~~~~
+
+{pve} lets a map grow with its LUN once every path of the map reports the new
+size, see `auto-resize` under xref:storage_multipath_config[Configuration]. To
+grow a LUN that carries a shared LVM volume group:
+
+. Grow the LUN on the storage system. Never shrink a LUN that is in use.
+. Make every path on every node report the new size. For iSCSI, {pve} rescans
+ the sessions of an active storage about every ten seconds, so this happens on
+ its own; a path of a session that is down follows once the session recovers.
+ For Fibre Channel, rescan each path on every node with
+ `echo 1 > /sys/block/<path>/device/rescan`, where `<path>` is a path device
+ of the map, such as `sdc`. NVMe over Fabrics targets notify the hosts
+ themselves.
+. Check the size of the map on every node, under *Node -> SAN* or with
+ `multipath -ll <wwid>`. If the map of a node did not follow, run
+ `multipathd resize map <wwid>` on that node.
+. Only then, on one node, grow the physical volume with
+ `pvresize /dev/disk/by-id/dm-uuid-mpath-<wwid>`. A node whose map is still
+ smaller cannot activate logical volumes in the added space.
+
+[[storage_multipath_remove]]
+Removing a LUN
+~~~~~~~~~~~~~~
+
+The device profiles of many storage systems make `dm-multipath` queue I/O while
+a map has no path, some of them without a time limit, so that a controller
+failover causes no I/O errors. If such a LUN is unmapped on the storage system,
+every process that reads its map blocks until a path returns or the retries run
+out. This includes LVM commands, which scan every device. The map cannot be
+removed while processes wait for it. The LVM status commands share a 15-second
+time limit. If they fail or time out, {pve} shows the affected LVM storages as
+not active on that node. The rest of the status update can take longer. Guests,
+backups, and other LVM commands can still hang. Therefore, turn off queueing
+before you unmap a LUN:
+
+. Stop the guests that use volumes on the LUN or move their disks to another
+ storage, and remove the LVM storage. Removing the storage entry does not stop
+ its users and keeps the volume group with its data.
+. Deactivate the volume group on every node with `vgchange -an <vg>`. Do not
+ remove the volume group or the physical volume unless you want to destroy the
+ data; removing them is not part of this procedure.
+. Make the map fail I/O instead of queueing it. If the LUN has an explicit
+ entry, note its `no-path-retry` setting and set it to `fail`, for example with
+ `pvesh set /cluster/multipath/wwid/<wwid> --no-path-retry fail`. Otherwise,
+ create a temporary entry for the nodes that see the LUN:
++
+----
+# pvesh create /cluster/multipath/wwid --wwid <wwid> --nodes <nodes> \
+ --no-path-retry fail
+----
+
+. On every node that sees the LUN, check in the output of `multipath -ll <wwid>`
+ that the `features` of the map no longer list `queue_if_no_path`. If a node is
+ offline, wait until it is back and check it as well before you continue.
+. On every node, note the path devices of the map that `multipath -ll <wwid>`
+ lists, and for each of them the identifier in `/sys/block/<path>/device/wwid`.
+ Then unmap the LUN on the storage system.
+. On every node:
+.. Remove the map with `multipath -f <wwid>`. If the map is still in use, do not
+ force the removal: find out what uses it, and retry after you have deleted
+ the path devices.
+.. For each noted path device, check that `/sys/block/<path>/device/wwid` still
+ shows the noted identifier, then delete the device with
+ `echo 1 > /sys/block/<path>/device/delete`. Skip a path device that is gone.
+ If one shows another identifier, stop: its name now belongs to another LUN.
+.. Check that `dmsetup ls` no longer lists the map.
+. Remove the temporary entry with `pvesh delete /cluster/multipath/wwid/<wwid>`,
+ or set `no-path-retry` of the explicit entry back to its previous value. If it
+ was not set before, unset it with
+ `pvesh set /cluster/multipath/wwid/<wwid> --delete no-path-retry`.
+
+If the LUN was already unmapped while its map queued I/O, processes that read
+the map hang, and changing the entry does not stop the queueing while they hold
+the map open. Run `multipathd disablequeueing map <wwid>` on every node that has
+the map to fail the queued I/O, then continue with noting the path devices and
+removing the map.
+
+An iSCSI storage keeps every LUN it has found on the allow-list for as long as
+the storage exists, as it cannot tell a LUN that was withdrawn on purpose from
+one that is only temporarily not visible. The datacenter view therefore keeps
+showing an unmapped LUN as *Not currently observed*. The allow-list entry is
+harmless, and if the LUN is presented again, it is managed again right away.
+Once you remove the storage or disable its multipath option, each node removes
+the allow-list entries that {pve} added, as soon as no other storage or
+explicit entry needs them and the map is unused. Allow-list entries that {pve}
+did not add stay in place.
+
+If the nodes lost a LUN, for example because it was unmapped by mistake, its
+paths report a size of zero after it is presented again, until they are
+rescanned. For iSCSI, the periodic session rescan of {pve} repairs the paths
+that still exist, and `iscsiadm --mode session --rescan` finds the LUNs whose
+paths were deleted. For Fibre Channel, rescan the remaining paths as described
+in xref:storage_multipath_grow[Growing a LUN], and rescan the host adapters to
+find the deleted paths, for example with `rescan-scsi-bus.sh` from the
+`sg3-utils` package.
+
+[[storage_multipath_disconnect]]
+Removing a Connection Storage
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Removing an iSCSI storage leaves its sessions connected and does not erase any
+data. To remove such a connection storage and disconnect the nodes safely:
+
+. Find out what uses each LUN of the target, not only the one that an LVM
+ storage uses. Stop these users or move their data to another storage.
+. Remove the LVM storages on the connection storage, and deactivate their
+ volume groups on every node with `vgchange -an <vg>`. Removing a storage does
+ not deactivate its volumes or remove its data.
+. Remove the connection storage. Make sure that no other enabled storage uses
+ the same target, as it would connect again.
+. On every node, choose *Logout* for the sessions under *Node -> SAN -> iSCSI*.
+ Once the iSCSI sessions are gone, *Remove* deletes the saved target records.
+ A logout is refused if it would take away the last usable path of a LUN that
+ is in use; stop what uses it instead of forcing the removal.
+
+Snapshot Clones
+~~~~~~~~~~~~~~~
+
+Do not present a snapshot clone of a LUN that carries an LVM physical volume to
+the nodes that use the original LUN. LVM then sees the same physical volume
+twice and refuses to activate logical volumes of the volume group or to change
+it, so guests on that storage cannot start or migrate. The SAN view of a node
+flags such LUNs as *duplicate PV*, but that check relies on the information udev
+recorded and can miss a duplicate. To access the data of a clone, present it to
+another host, or give it a new identity with `vgimportclone` on a host that
+sees only the clone before presenting it to the nodes.
+
+[[storage_multipath_config]]
+Configuration
+~~~~~~~~~~~~~
+
+The cluster-wide configuration is stored in `/etc/pve/multipath.cfg`. It has an
+optional `defaults` section for global `multipathd` settings and one `wwid`
+section for each explicit entry. From it, every node generates drop-in files
+with the prefix `pve-` under `/etc/multipath/conf.d/`, and maintains its
+allow-list in `/etc/multipath/wwids` with the LUNs of its multipath-enabled
+storages and the explicit entries that apply to the node. A hand-made
+`/etc/multipath.conf` and allow-list entries that {pve} did not add stay in
+place.
+
+.Example (`/etc/pve/multipath.cfg`)
+----
+defaults: defaults
+ no-path-retry queue
+
+wwid: 3600140558631413f06d4840826b34782
+ alias san1-lun0
+ nodes pve1,pve2
+ path-selector service-time 0
+----
+
+The `defaults` section takes `find-multipaths`, `user-friendly-names`,
+`polling-interval`, `auto-resize`, `no-path-retry`, `path-grouping-policy`,
+`failback`, and `path-selector`. Keep `find-multipaths` at `strict`, so that
+`multipathd` assembles maps only for LUNs on the allow-list, and
+`user-friendly-names` at `no`, so that maps without an explicit alias use their
+WWIDs as names, consistently across nodes and without a node-local bindings
+file. `auto-resize` defaults to `grow_only`, so a map grows with its LUN, see
+xref:storage_multipath_grow[Growing a LUN]; `grow_shrink` also shrinks it, and
+`never` leaves resizing to you.
+
+The built-in device profiles of `multipath-tools` set `no-path-retry` and the
+path settings for many storage systems, and these profiles take precedence over
+a `defaults` section. {pve} therefore writes `no-path-retry`,
+`path-grouping-policy`, `failback`, and `path-selector` from its `defaults`
+section into an `overrides` section, which takes precedence over the device
+profiles. Leave these settings unset to keep the values from the profile of your
+storage system.
+
+A `wwid` section takes `alias`, `nodes`, and the per-LUN settings
+`no-path-retry`, `path-grouping-policy`, `failback`, `path-selector`,
+`rr-min-io-rq`, and `rr-weight`, which take precedence over the defaults and
+the device profiles. See the `multipath.conf` manual page for the meaning of
+each setting.
+
+NOTE: For storage systems with Asymmetric Logical Unit Access (ALUA), whose
+paths all use the `alua` or `sysfs` prioritizer, `multipathd` detects the path
+grouping policy by itself and ignores a configured `path-grouping-policy`, even
+that of a `wwid` section. Setting `path-grouping-policy` in the `defaults`
+section turns this detection off. For a policy that only a `wwid` section sets,
+turn the detection off with `detect_pgpolicy no` in an `overrides` section of
+`/etc/pve/multipath-overrides.conf`.
+
+NOTE: A LUN is identified by the WWID that `multipathd` computes for its paths.
+If you change how `multipathd` computes it, for example with `uid_attrs`, the
+WWIDs of the affected LUNs can change once `multipathd` restarts. The entries of
+their old WWIDs then stay on the allow-list until their storage is removed or
+its multipath option is disabled, see
+xref:storage_multipath_remove[Removing a LUN].
+
+Hardware-specific settings, that is, the `devices`, `overrides`, and
+`blacklist` sections of `multipath.conf`, belong in
+`/etc/pve/multipath-overrides.conf`, written as in `multipath.conf`. Edit them
+with *Edit Overrides* under *Datacenter -> SAN -> Multipath*; every node applies
+them like the rest of the configuration. An `overrides` section in this file
+takes precedence over the managed defaults, but a `devices` section does not
+take precedence over the settings that {pve} writes as overrides.
+
+The API under `/cluster/multipath` manages the configuration and reports the
+health, for example:
+
+----
+# pvesh create /cluster/multipath/wwid --nodes pve1,pve2 \
+ --wwid 3600140558631413f06d4840826b34782
+# pvesh get /cluster/multipath/status
+----
+
+ifdef::wiki[]
+
+See Also
+~~~~~~~~
+
+* link:/wiki/Storage[Storage]
+
+* link:/wiki/Storage:_iSCSI[Storage: iSCSI]
+
+* link:/wiki/Storage:_LVM[Storage: LVM]
+
+endif::wiki[]
diff --git a/pvesm.adoc b/pvesm.adoc
index 5bd24b2..6f4e776 100644
--- a/pvesm.adoc
+++ b/pvesm.adoc
@@ -96,11 +96,10 @@ TiB sized), these operations may take several minutes, or in extreme cases, even
hours. If your setup is affected, create and remove snapshots while the VM is
shut down, expecting a long task duration.
-^3^: With iSCSI, FibreChannel (FC), or SAS block storage as shared storage in a
-cluster, LVM is used to split the LUN into virtual disks. The
-https://pve.proxmox.com/wiki/Multipath[Multipath Guide] explains how to
-configure it in a cluster, including multipathing which is usually a
-requirement in these scenarios.
+^3^: With iSCSI, Fibre Channel (FC), or SAS block storage as shared storage in
+a cluster, LVM is used to split the LUN into virtual disks. A LUN is usually
+reachable over several paths; xref:storage_multipath[Multipath] describes how
+{pve} manages them in a cluster.
^4^: Since {pve} 9, snapshots as a volume chain have been available for VMs.
These snapshots use separate volumes for the snapshot data and layer them. For
@@ -421,6 +420,8 @@ See Also
* link:/wiki/Storage:_iSCSI[Storage: iSCSI]
+* link:/wiki/Storage:_Multipath[Storage: Multipath]
+
* link:/wiki/Storage:_LVM[Storage: LVM]
* link:/wiki/Storage:_LVM_Thin[Storage: LVM Thin]
@@ -463,6 +464,8 @@ include::pve-storage-iscsi.adoc[]
include::pve-storage-iscsidirect.adoc[]
+include::pve-storage-multipath.adoc[]
+
include::pve-storage-rbd.adoc[]
include::pve-storage-cephfs.adoc[]
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 45/76] api: ceph: osd: keep iSCSI LUNs out of OSD creation
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (43 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 docs 44/76] storage: document the cluster-wide multipath management Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 46/76] pvestatd: track pending storage status commands across re-exec Thomas Lamprecht
` (30 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Disk enumeration now includes iSCSI LUNs, which other nodes may use.
Reject them for OSD, DB, and WAL devices, matching the other local disk
operations. Check again in the worker, as a device name can belong to
another device by the time it runs.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
PVE/API2/Ceph/OSD.pm | 3 +++
1 file changed, 3 insertions(+)
diff --git a/PVE/API2/Ceph/OSD.pm b/PVE/API2/Ceph/OSD.pm
index 442b99292..af0fdf069 100644
--- a/PVE/API2/Ceph/OSD.pm
+++ b/PVE/API2/Ceph/OSD.pm
@@ -379,6 +379,9 @@ __PACKAGE__->register_method({
my $test_disk_requirements = sub {
my ($disklist) = @_;
+ # refuse shared SAN devices such as iSCSI LUNs, which get_disks lists as well
+ PVE::Diskmanage::assert_local_device($_->{dev}, $disklist) for values %$devs;
+
my $dev = $devs->{dev}->{dev};
my $devname = $devs->{dev}->{name};
die "unable to get device info for '$dev'\n" if !$disklist->{$devname};
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 46/76] pvestatd: track pending storage status commands across re-exec
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (44 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 45/76] api: ceph: osd: keep iSCSI LUNs out of OSD creation Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 47/76] ui: storage: factor out the scan picker error mask Thomas Lamprecht
` (29 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Storage status commands that outlive their budget must be reaped later.
pvestatd re-exec preserves its children but loses their tracking
records, leaving them as zombies once they exit.
Pass the records through the environment across re-exec. Reap children
on every pass, even if no remaining storage starts new status commands.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
PVE/Service/pvestatd.pm | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/PVE/Service/pvestatd.pm b/PVE/Service/pvestatd.pm
index cec66e001..f682c2f9d 100755
--- a/PVE/Service/pvestatd.pm
+++ b/PVE/Service/pvestatd.pm
@@ -20,6 +20,7 @@ use PVE::RS::SDN::Fabrics;
use PVE::NodeConfig;
use PVE::Cluster qw(cfs_read_file);
use PVE::Storage;
+use PVE::Storage::StatusCommand;
use PVE::QemuServer;
use PVE::QemuServer::Monitor;
use PVE::LXC;
@@ -74,6 +75,15 @@ sub hup {
$restart_request = 1;
}
+# The status commands that a pass left behind, see PVE::Storage::StatusCommand, stay children of
+# this process across a re-exec, but the record of them does not, so hand it over.
+sub restart_daemon {
+ my ($self, @param) = @_;
+
+ $ENV{PVE_STORAGE_STATUS_COMMANDS} = PVE::Storage::StatusCommand::export_pending();
+ $self->SUPER::restart_daemon(@param);
+}
+
my $cached_kvm_version = '';
my $next_flag_update_time;
my $failed_flag_update_delay_sec = 120;
@@ -871,10 +881,15 @@ my $initial_memory_usage;
sub run {
my ($self) = @_;
+ PVE::Storage::StatusCommand::import_pending(delete $ENV{PVE_STORAGE_STATUS_COMMANDS});
+
for (;;) { # forever
$next_update = time() + $updatetime;
+ # reap left-over status commands even when no storage starts a new one
+ PVE::Storage::StatusCommand::reap_pending();
+
if ($cycle) {
my ($ccsec, $cusec) = gettimeofday();
eval {
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 47/76] ui: storage: factor out the scan picker error mask
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (45 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 46/76] pvestatd: track pending storage status commands across re-exec Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 48/76] ui: storage: nfs: show scan errors and empty results Thomas Lamprecht
` (28 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
The CIFS picker masks its list itself because monStoreErrors would mask
the combo field instead. Share the helper so the NFS picker can display
scan errors the same way. This does not change behavior.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Utils.js | 21 +++++++++++++++++++++
www/manager6/storage/CIFSEdit.js | 19 +------------------
2 files changed, 22 insertions(+), 18 deletions(-)
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index 7704404cf..ca850158b 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -439,6 +439,27 @@ Ext.define('PVE.Utils', {
renderNotFound: (what) => Ext.String.format(gettext('No {0} found'), what),
+ // monStoreErrors would mask the field of a combo box, not its picker list
+ monPickerStoreErrors: function (combo, store) {
+ let picker = combo.getPicker();
+ picker.mon(store, 'beforeload', function () {
+ picker.unmask();
+ delete picker.minHeight;
+ });
+ picker.mon(store.proxy, 'afterload', function (proxy, request, success) {
+ if (success) {
+ Proxmox.Utils.setErrorMask(picker, false);
+ return;
+ }
+ let error = request._operation.getError();
+ let msg = Proxmox.Utils.getResponseErrorMessage(error);
+ if (msg) {
+ picker.minHeight = 100;
+ }
+ Proxmox.Utils.setErrorMask(picker, msg);
+ });
+ },
+
get_kvm_osinfo: function (value) {
var info = { base: 'Other' }; // default
if (value) {
diff --git a/www/manager6/storage/CIFSEdit.js b/www/manager6/storage/CIFSEdit.js
index 5fe3fefea..531f390cc 100644
--- a/www/manager6/storage/CIFSEdit.js
+++ b/www/manager6/storage/CIFSEdit.js
@@ -94,24 +94,7 @@ Ext.define('PVE.storage.CIFSScan', {
me.callParent();
- let picker = me.getPicker();
- // don't use monStoreErrors directly, it doesn't copes well with comboboxes
- picker.mon(store, 'beforeload', function (s, operation, eOpts) {
- picker.unmask();
- delete picker.minHeight;
- });
- picker.mon(store.proxy, 'afterload', function (proxy, request, success) {
- if (success) {
- Proxmox.Utils.setErrorMask(picker, false);
- return;
- }
- let error = request._operation.getError();
- let msg = Proxmox.Utils.getResponseErrorMessage(error);
- if (msg) {
- picker.minHeight = 100;
- }
- Proxmox.Utils.setErrorMask(picker, msg);
- });
+ PVE.Utils.monPickerStoreErrors(me, store);
},
});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 48/76] ui: storage: nfs: show scan errors and empty results
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (46 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 47/76] ui: storage: factor out the scan picker error mask Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 49/76] ui: storage: allow changing the node for NFS/CIFS scans Thomas Lamprecht
` (27 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
An empty export picker does not distinguish an unreachable NFS server
from one without exports. Show scan errors on the picker and explain an
empty result, matching the CIFS share picker.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/storage/NFSEdit.js | 3 +++
1 file changed, 3 insertions(+)
diff --git a/www/manager6/storage/NFSEdit.js b/www/manager6/storage/NFSEdit.js
index 72db156f4..91b0e6cc5 100644
--- a/www/manager6/storage/NFSEdit.js
+++ b/www/manager6/storage/NFSEdit.js
@@ -9,6 +9,7 @@ Ext.define('PVE.storage.NFSScan', {
matchFieldWidth: false,
listConfig: {
loadingText: gettext('Scanning...'),
+ emptyText: PVE.Utils.renderNotFound(gettext('NFS export')),
width: 350,
},
doRawQuery: function () {
@@ -55,6 +56,8 @@ Ext.define('PVE.storage.NFSScan', {
});
me.callParent();
+
+ PVE.Utils.monPickerStoreErrors(me, store);
},
});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 49/76] ui: storage: allow changing the node for NFS/CIFS scans
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (47 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 48/76] ui: storage: nfs: show scan errors and empty results Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 50/76] ui: storage: add guided remote storage wizard with NFS support Thomas Lamprecht
` (26 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Unlike the iSCSI and LVM scan combos, NFS and CIFS fix the scan node
when creating their store. Allow callers to change it through
setNodeName() so a node selector can redirect scans. Existing edit
dialogs are unchanged.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/storage/CIFSEdit.js | 10 ++++++++++
www/manager6/storage/NFSEdit.js | 10 ++++++++++
2 files changed, 20 insertions(+)
diff --git a/www/manager6/storage/CIFSEdit.js b/www/manager6/storage/CIFSEdit.js
index 531f390cc..4aa79c66a 100644
--- a/www/manager6/storage/CIFSEdit.js
+++ b/www/manager6/storage/CIFSEdit.js
@@ -72,6 +72,16 @@ Ext.define('PVE.storage.CIFSScan', {
}
},
+ setNodeName: function (nodename) {
+ let me = this;
+ nodename = nodename || 'localhost';
+ if (me.nodename !== nodename) {
+ me.nodename = nodename;
+ me.getStore().getProxy().setUrl(`/api2/json/nodes/${nodename}/scan/cifs`);
+ me.resetProxy();
+ }
+ },
+
initComponent: function () {
var me = this;
diff --git a/www/manager6/storage/NFSEdit.js b/www/manager6/storage/NFSEdit.js
index 91b0e6cc5..7f19116c2 100644
--- a/www/manager6/storage/NFSEdit.js
+++ b/www/manager6/storage/NFSEdit.js
@@ -34,6 +34,16 @@ Ext.define('PVE.storage.NFSScan', {
me.nfsServer = server;
},
+ setNodeName: function (nodename) {
+ let me = this;
+ nodename = nodename || 'localhost';
+ if (me.nodename !== nodename) {
+ me.nodename = nodename;
+ me.getStore().getProxy().setUrl(`/api2/json/nodes/${nodename}/scan/nfs`);
+ me.lastQuery = null;
+ }
+ },
+
initComponent: function () {
var me = this;
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 50/76] ui: storage: add guided remote storage wizard with NFS support
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (48 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 49/76] ui: storage: allow changing the node for NFS/CIFS scans Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 51/76] ui: storage wizard: add SMB/CIFS support Thomas Lamprecht
` (25 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
The storage Add menu assumes users know which plugin and settings they
need. Introduce a wizard that asks for the storage type, then guides the
user through connection, selection, common settings, and confirmation.
Keep each type's steps in a registry. Rebuild the window when the type
changes because field validity tracking is wired only at creation.
Support chained API requests for types requiring several storage
entries.
Include shared settings and the NFS flow; a separate patch adds the
menu entry.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: label and order the confirmation rows by the setup, with positive
labels and readable values; let types present values, add rows and
supply plain-text notes; refresh the summary during setup, keep the
wizard busy and not closable while its request runs and read the scan
node from its selector instead of a copy on the window; widen the
Setting column of the summary so that longer labels fit]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Makefile | 3 +
www/manager6/storage/wizard/CommonSettings.js | 69 ++++
www/manager6/storage/wizard/NFS.js | 89 ++++
www/manager6/storage/wizard/Wizard.js | 389 ++++++++++++++++++
4 files changed, 550 insertions(+)
create mode 100644 www/manager6/storage/wizard/CommonSettings.js
create mode 100644 www/manager6/storage/wizard/NFS.js
create mode 100644 www/manager6/storage/wizard/Wizard.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index d2ea786be..b467e7ef5 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -378,6 +378,9 @@ JSSRC= \
storage/ZFSEdit.js \
storage/ZFSPoolEdit.js \
storage/ESXIEdit.js \
+ storage/wizard/Wizard.js \
+ storage/wizard/CommonSettings.js \
+ storage/wizard/NFS.js \
Workspace.js \
# end of JSSRC list
diff --git a/www/manager6/storage/wizard/CommonSettings.js b/www/manager6/storage/wizard/CommonSettings.js
new file mode 100644
index 000000000..97e49e981
--- /dev/null
+++ b/www/manager6/storage/wizard/CommonSettings.js
@@ -0,0 +1,69 @@
+Ext.define('PVE.storage.wizard.CommonSettings', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pveStorageWizardCommonSettings',
+
+ onlineHelp: 'chapter_storage',
+
+ cts: undefined,
+ defaultContent: undefined,
+ // submitted as content instead of offering a selector
+ fixedContent: undefined,
+ contentFieldConfig: undefined,
+
+ onGetValues: function (values) {
+ values.disable = values.enable ? 0 : 1;
+ delete values.enable;
+ return values;
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ me.column1 = [
+ {
+ xtype: 'textfield',
+ name: 'storage',
+ fieldLabel: 'ID',
+ vtype: 'StorageId',
+ allowBlank: false,
+ },
+ ];
+
+ if (!me.fixedContent) {
+ me.column1.push(
+ Ext.apply(
+ {
+ xtype: 'pveContentTypeSelector',
+ cts: me.cts,
+ name: 'content',
+ value: me.defaultContent,
+ multiSelect: true,
+ fieldLabel: gettext('Content'),
+ allowBlank: false,
+ },
+ me.contentFieldConfig,
+ ),
+ );
+ }
+
+ me.column2 = [
+ {
+ xtype: 'pveNodeSelector',
+ name: 'nodes',
+ fieldLabel: gettext('Nodes'),
+ emptyText: gettext('All') + ' (' + gettext('No restrictions') + ')',
+ multiSelect: true,
+ autoSelect: false,
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'enable',
+ checked: true,
+ uncheckedValue: 0,
+ fieldLabel: gettext('Enable'),
+ },
+ ];
+
+ me.callParent();
+ },
+});
diff --git a/www/manager6/storage/wizard/NFS.js b/www/manager6/storage/wizard/NFS.js
new file mode 100644
index 000000000..8e7a485c4
--- /dev/null
+++ b/www/manager6/storage/wizard/NFS.js
@@ -0,0 +1,89 @@
+Ext.define('PVE.storage.wizard.NFSConnection', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pveStorageWizardNFSConnection',
+
+ onlineHelp: 'storage_nfs',
+
+ onGetValues: function (values) {
+ if (values.nfsversion && values.nfsversion !== '__default__') {
+ values.options = `vers=${values.nfsversion}`;
+ }
+ delete values.nfsversion;
+ return values;
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ me.column1 = [
+ {
+ xtype: 'textfield',
+ name: 'server',
+ fieldLabel: gettext('Server'),
+ allowBlank: false,
+ listeners: {
+ change: function (f, value) {
+ let exportField = me.down('field[name=export]');
+ exportField.setServer(value);
+ exportField.setValue('');
+ },
+ },
+ },
+ {
+ xtype: 'pveNFSScan',
+ name: 'export',
+ fieldLabel: 'Export',
+ allowBlank: false,
+ },
+ ];
+
+ me.column2 = [];
+ if (!PVE.Utils.isStandaloneNode()) {
+ me.column2.push({
+ xtype: 'pveStorageScanNodeSelector',
+ listeners: {
+ change: function (f, value) {
+ me.down('field[name=export]').setNodeName(value);
+ me.up('window').down('field[name=nodes]').setValue(value);
+ },
+ },
+ });
+ }
+
+ me.advancedColumn1 = [
+ {
+ xtype: 'proxmoxKVComboBox',
+ fieldLabel: gettext('NFS Version'),
+ name: 'nfsversion',
+ value: '__default__',
+ deleteEmpty: false,
+ comboItems: [
+ ['__default__', Proxmox.Utils.defaultText],
+ ['3', '3'],
+ ['4', '4'],
+ ['4.1', '4.1'],
+ ['4.2', '4.2'],
+ ],
+ },
+ ];
+
+ me.callParent();
+ },
+});
+
+PVE.storage.wizard.types.nfs = {
+ text: 'NFS',
+ description: gettext('Shared folder on a NAS or file server, accessed via the NFS protocol.'),
+ group: 'nas',
+ apiType: 'nfs',
+ steps: () => [
+ {
+ xtype: 'pveStorageWizardNFSConnection',
+ title: gettext('Connection'),
+ },
+ ],
+ settings: {
+ onlineHelp: 'storage_nfs',
+ defaultContent: ['images'],
+ },
+};
diff --git a/www/manager6/storage/wizard/Wizard.js b/www/manager6/storage/wizard/Wizard.js
new file mode 100644
index 000000000..ce28f6845
--- /dev/null
+++ b/www/manager6/storage/wizard/Wizard.js
@@ -0,0 +1,389 @@
+Ext.ns('PVE.storage.wizard');
+
+PVE.storage.wizard.types = {};
+
+PVE.storage.wizard.groups = [
+ { key: 'nas', text: gettext('File Storage (NAS)') },
+ { key: 'san', text: gettext('Block Storage (SAN)') },
+];
+
+// submitted values without a field to label them: derived in onGetValues or from a radio group
+PVE.storage.wizard.summaryFields = {
+ disable: { label: gettext('Enabled'), isBoolean: true, invert: true, weight: 31 },
+ content: {
+ label: gettext('Content'),
+ render: (value) => PVE.Utils.format_content_types([].concat(value).join(',')),
+ weight: 30,
+ },
+ nodes: {
+ label: gettext('Nodes'),
+ empty: gettext('All') + ' (' + gettext('No restrictions') + ')',
+ weight: 32,
+ },
+ options: { label: gettext('Mount Options') },
+};
+
+Ext.define('PVE.storage.wizard.RemoteStorage', {
+ extend: 'PVE.window.Wizard',
+ alias: 'widget.pveRemoteStorageWizard',
+
+ subject: gettext('Remote Storage'),
+
+ wizardType: 'nfs',
+
+ reloadCallback: undefined,
+
+ // the view model config must be in place before the component config system runs,
+ // initComponent would be too late for the field bindings
+ constructor: function (config) {
+ let me = this;
+ let entry = PVE.storage.wizard.types[config?.wizardType || me.wizardType];
+ if (entry?.viewModel) {
+ config = Ext.apply({ viewModel: Ext.clone(entry.viewModel) }, config);
+ }
+ me.callParent([config]);
+ },
+
+ getScanNode: function () {
+ return this.down('pveStorageScanNodeSelector')?.getValue() || Proxmox.NodeName;
+ },
+
+ changeWizardType: function (type) {
+ let me = this;
+ if (!type || type === me.wizardType) {
+ return;
+ }
+ Ext.create('PVE.storage.wizard.RemoteStorage', {
+ wizardType: type,
+ reloadCallback: me.reloadCallback,
+ autoShow: true,
+ });
+ me.close();
+ },
+
+ busy: false,
+
+ // the continuation lives in this window: block closing and the buttons while waiting, then
+ // restore them
+ setBusy: function (busy, msg) {
+ let me = this;
+ me.busy = busy;
+ me.setLoading(busy ? msg || true : false);
+ let buttons = ['#back', '#next', '#submit']
+ .map((itemId) => me.down(itemId))
+ .filter((button) => !!button);
+ if (busy) {
+ me.buttonStateBeforeBusy ??= buttons.map((button) => button.isDisabled());
+ buttons.forEach((button) => button.setDisabled(true));
+ } else {
+ buttons.forEach((button, idx) =>
+ button.setDisabled(me.buttonStateBeforeBusy?.[idx] ?? false),
+ );
+ delete me.buttonStateBeforeBusy;
+ }
+ },
+
+ // keeps the window busy, so it cannot close with the continuation pending
+ request: function (options) {
+ let me = this;
+ me.setBusy(true, options.msg);
+ Proxmox.Utils.API2Request({
+ url: options.url,
+ method: options.method,
+ params: options.params,
+ success: function (response) {
+ if (me.destroyed) {
+ return;
+ }
+ me.setBusy(false);
+ options.success(response);
+ },
+ failure: function (response) {
+ if (me.destroyed) {
+ return;
+ }
+ me.setBusy(false);
+ options.failure(response);
+ },
+ });
+ },
+
+ // a step calls next() on success and just returns after a failure, so Finish retries from
+ // there
+ runSubmitChain: function (steps) {
+ let me = this;
+ let run = function (idx) {
+ if (me.destroyed) {
+ return;
+ }
+ if (idx >= steps.length) {
+ me.close();
+ return;
+ }
+ steps[idx](() => run(idx + 1));
+ };
+ run(0);
+ },
+
+ // errorHtml (safe HTML) is prepended to an error
+ createStorage: function (params, next, errorHtml) {
+ let me = this;
+ params = Ext.apply({}, params);
+ Ext.Object.each(params, function (key, value) {
+ if (value === undefined || value === null || (Ext.isArray(value) && !value.length)) {
+ delete params[key];
+ }
+ });
+ me.request({
+ url: '/storage',
+ method: 'POST',
+ params: params,
+ success: () => next(),
+ failure: function (response) {
+ let msg = response.htmlStatus;
+ if (errorHtml) {
+ msg = `${errorHtml}<br><br>${msg}`;
+ }
+ Ext.Msg.alert(gettext('Error'), msg);
+ },
+ });
+ },
+
+ summaryField: function (key) {
+ let me = this;
+ let typeEntry = PVE.storage.wizard.types[me.wizardType];
+ return typeEntry.summaryFields?.[key] ?? PVE.storage.wizard.summaryFields[key];
+ },
+
+ summaryLabel: function (key) {
+ let me = this;
+ let entry = me.summaryField(key);
+ if (entry) {
+ return entry.label;
+ }
+ return me.down(`field[name="${key}"]`)?.fieldLabel || key;
+ },
+
+ summaryValue: function (key, value) {
+ let me = this;
+ let entry = me.summaryField(key);
+ let empty = value === undefined || value === null || value === '';
+ if (Ext.isArray(value)) {
+ empty = !value.length;
+ }
+ if (empty && entry?.empty) {
+ return entry.empty;
+ }
+ if (entry?.render) {
+ return entry.render(value);
+ }
+ if (Ext.isArray(value)) {
+ value = value.join(', ');
+ }
+ let labeled = entry?.values?.[value];
+ if (labeled !== undefined) {
+ return labeled;
+ }
+ let field = me.down(`field[name="${key}"]`);
+ // a radio group submits the selected option and is a checkbox as far as ExtJS is concerned
+ let isBoolean =
+ entry?.isBoolean || (!!field && field.isXType('checkbox') && !field.isXType('radio'));
+ if (isBoolean) {
+ let set = !!value && value !== '0';
+ return Proxmox.Utils.format_boolean(entry?.invert ? !set : set);
+ }
+ return empty ? '' : String(value);
+ },
+
+ updateSummary: function () {
+ let me = this;
+ let panel = me.down('#confirm');
+ if (!panel) {
+ return;
+ }
+ let values = me.getValues();
+ let typeEntry = PVE.storage.wizard.types[me.wizardType];
+ let derived = typeEntry.summaryRows?.(values, me) ?? {};
+ let skip = new Set(['delete', ...(derived.skip ?? [])]);
+
+ let data = [];
+ Ext.Object.each(values, function (key, value) {
+ let entry = me.summaryField(key);
+ if (skip.has(key) || entry?.skip?.(values)) {
+ return;
+ }
+ data.push({
+ key: me.summaryLabel(key),
+ value: key === 'password' ? '********' : me.summaryValue(key, value),
+ weight: entry?.weight ?? 50,
+ });
+ });
+ for (const row of derived.rows ?? []) {
+ data.push({ key: row.label, value: row.value, weight: row.weight ?? 20 });
+ }
+
+ let summarystore = panel.down('grid').getStore();
+ summarystore.suspendEvents();
+ summarystore.removeAll();
+ summarystore.add(data);
+ summarystore.sort();
+ summarystore.resumeEvents();
+ summarystore.fireEvent('refresh');
+
+ let notes = panel.down('#summaryNotes');
+ notes.removeAll();
+ for (const note of typeEntry.summaryNotes?.(values) || []) {
+ notes.add({ value: note });
+ }
+ },
+
+ submitSimple: function (values) {
+ let me = this;
+ let entry = PVE.storage.wizard.types[me.wizardType];
+ values.type = entry.apiType;
+ if (entry.settings?.fixedContent) {
+ values.content = entry.settings.fixedContent;
+ }
+ me.runSubmitChain([(next) => me.createStorage(values, next)]);
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ let entry = PVE.storage.wizard.types[me.wizardType];
+ if (!entry) {
+ throw `unknown wizard storage type: ${me.wizardType}`;
+ }
+
+ let chooserItems = [
+ {
+ xtype: 'displayfield',
+ value: gettext('What kind of storage do you want to add?'),
+ },
+ ];
+ for (const group of PVE.storage.wizard.groups) {
+ let typeRadios = Object.entries(PVE.storage.wizard.types)
+ .filter(([, typeEntry]) => typeEntry.group === group.key)
+ .map(([type, typeEntry]) => ({
+ boxLabel: `<b>${typeEntry.text}</b> - ${typeEntry.description}`,
+ name: 'wizardType',
+ inputValue: type,
+ checked: type === me.wizardType,
+ submitValue: false,
+ margin: '0 0 10 0',
+ }));
+ if (!typeRadios.length) {
+ continue;
+ }
+ chooserItems.push({
+ xtype: 'displayfield',
+ value: `<b>${group.text}</b>`,
+ });
+ chooserItems.push({
+ xtype: 'radiogroup',
+ columns: 1,
+ vertical: true,
+ margin: '0 0 0 15',
+ items: typeRadios,
+ listeners: {
+ change: function (rg, value) {
+ // ignore the uncheck fired when a radio in another group takes over
+ if (!value.wizardType) {
+ return;
+ }
+ let wizard = rg.up('window');
+ // let the radio group finish its change handling before it is destroyed
+ Ext.defer(() => wizard.changeWizardType(value.wizardType), 10);
+ },
+ },
+ });
+ }
+
+ me.items = [
+ {
+ xtype: 'inputpanel',
+ title: gettext('Storage Type'),
+ onlineHelp: 'chapter_storage',
+ items: chooserItems,
+ },
+ ...entry.steps(),
+ Ext.apply(
+ {
+ xtype: 'pveStorageWizardCommonSettings',
+ title: gettext('General'),
+ },
+ entry.settings,
+ ),
+ {
+ title: gettext('Confirm'),
+ itemId: 'confirm',
+ layout: { type: 'vbox', align: 'stretch' },
+ defaults: { border: false },
+ items: [
+ {
+ xtype: 'grid',
+ flex: 1,
+ store: {
+ fields: ['key', 'value', 'weight'],
+ sorters: [
+ { property: 'weight', direction: 'ASC' },
+ { property: 'key', direction: 'ASC' },
+ ],
+ },
+ columns: [
+ { header: gettext('Setting'), width: 180, dataIndex: 'key' },
+ {
+ header: gettext('Value'),
+ flex: 1,
+ dataIndex: 'value',
+ renderer: Ext.htmlEncode,
+ },
+ ],
+ },
+ {
+ xtype: 'container',
+ itemId: 'summaryNotes',
+ padding: '5 0 0 0',
+ defaults: {
+ xtype: 'displayfield',
+ margin: 0,
+ },
+ },
+ ],
+ listeners: {
+ show: (panel) => panel.up('window').updateSummary(),
+ },
+ onSubmit: function () {
+ let wizard = this.up('window');
+ let values = wizard.getValues();
+ delete values.delete;
+ let typeEntry = PVE.storage.wizard.types[wizard.wizardType];
+ if (typeEntry.submit) {
+ typeEntry.submit(wizard, values);
+ } else {
+ wizard.submitSimple(values);
+ }
+ },
+ },
+ ];
+
+ me.on('beforeclose', function () {
+ if (!me.busy) {
+ return true;
+ }
+ Ext.Msg.alert(
+ gettext('Please wait'),
+ gettext('The wizard is still waiting for an operation it started.'),
+ );
+ return false;
+ });
+
+ me.on('destroy', function () {
+ if (me.reloadCallback) {
+ me.reloadCallback();
+ }
+ });
+
+ me.callParent();
+ },
+});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 51/76] ui: storage wizard: add SMB/CIFS support
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (49 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 50/76] ui: storage: add guided remote storage wizard with NFS support Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 52/76] ui: storage wizard: preserve settings after partial setup Thomas Lamprecht
` (24 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Reuse the CIFS share scanner from the edit dialog and keep the rarely
needed domain and subdirectory options in the advanced section.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: clear the selected share when the server changes; leave the scan
node to its selector]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Makefile | 1 +
www/manager6/storage/wizard/CIFS.js | 118 ++++++++++++++++++++++++++++
2 files changed, 119 insertions(+)
create mode 100644 www/manager6/storage/wizard/CIFS.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index b467e7ef5..7002518e0 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -381,6 +381,7 @@ JSSRC= \
storage/wizard/Wizard.js \
storage/wizard/CommonSettings.js \
storage/wizard/NFS.js \
+ storage/wizard/CIFS.js \
Workspace.js \
# end of JSSRC list
diff --git a/www/manager6/storage/wizard/CIFS.js b/www/manager6/storage/wizard/CIFS.js
new file mode 100644
index 000000000..4c76a4846
--- /dev/null
+++ b/www/manager6/storage/wizard/CIFS.js
@@ -0,0 +1,118 @@
+Ext.define('PVE.storage.wizard.CIFSConnection', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pveStorageWizardCIFSConnection',
+
+ onlineHelp: 'storage_cifs',
+
+ onGetValues: function (values) {
+ ['username', 'password', 'domain', 'subdir'].forEach(function (opt) {
+ if (values[opt]?.length === 0) {
+ delete values[opt];
+ }
+ });
+ return values;
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ let shareField = () => me.down('field[name=share]');
+
+ me.column1 = [
+ {
+ xtype: 'textfield',
+ name: 'server',
+ fieldLabel: gettext('Server'),
+ allowBlank: false,
+ listeners: {
+ change: function (f, value) {
+ shareField().setServer(value);
+ shareField().setValue('');
+ },
+ },
+ },
+ {
+ xtype: 'textfield',
+ name: 'username',
+ fieldLabel: gettext('Username'),
+ emptyText: gettext('Guest user'),
+ allowBlank: true,
+ listeners: {
+ change: (f, value) => shareField().setUsername(value),
+ },
+ },
+ {
+ xtype: 'textfield',
+ inputType: 'password',
+ name: 'password',
+ fieldLabel: gettext('Password'),
+ emptyText: gettext('None'),
+ minLength: 1,
+ allowBlank: true,
+ listeners: {
+ change: (f, value) => shareField().setPassword(value),
+ },
+ },
+ {
+ xtype: 'pveCIFSScan',
+ name: 'share',
+ fieldLabel: 'Share',
+ allowBlank: false,
+ },
+ ];
+
+ me.column2 = [];
+ if (!PVE.Utils.isStandaloneNode()) {
+ me.column2.push({
+ xtype: 'pveStorageScanNodeSelector',
+ listeners: {
+ change: function (f, value) {
+ shareField().setNodeName(value);
+ me.up('window').down('field[name=nodes]').setValue(value);
+ },
+ },
+ });
+ }
+
+ me.advancedColumn1 = [
+ {
+ xtype: 'textfield',
+ name: 'domain',
+ fieldLabel: gettext('Domain'),
+ allowBlank: true,
+ listeners: {
+ change: (f, value) => shareField().setDomain(value),
+ },
+ },
+ ];
+
+ me.advancedColumn2 = [
+ {
+ xtype: 'textfield',
+ name: 'subdir',
+ fieldLabel: gettext('Subdirectory'),
+ allowBlank: true,
+ emptyText: gettext('/some/path'),
+ },
+ ];
+
+ me.callParent();
+ },
+});
+
+PVE.storage.wizard.types.cifs = {
+ text: 'SMB/CIFS',
+ description: gettext('Shared folder on a Windows server, NAS, or Samba server.'),
+ group: 'nas',
+ apiType: 'cifs',
+ steps: () => [
+ {
+ xtype: 'pveStorageWizardCIFSConnection',
+ title: gettext('Connection'),
+ },
+ ],
+ settings: {
+ onlineHelp: 'storage_cifs',
+ defaultContent: ['images'],
+ },
+};
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 52/76] ui: storage wizard: preserve settings after partial setup
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (50 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 51/76] ui: storage wizard: add SMB/CIFS support Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 53/76] ui: storage wizard: add Fibre Channel shared LVM setup Thomas Lamprecht
` (23 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Once setup creates its first object, keep settings fixed and disable
Back so retries cannot mix unrelated storage IDs or devices. Record
completed steps and retry only missing ones.
Check storage IDs before starting to avoid partial setup on known
conflicts. Allow types to validate node restrictions and explain setup
state in confirmation notes.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/storage/wizard/CommonSettings.js | 20 +++--
www/manager6/storage/wizard/Wizard.js | 84 ++++++++++++++++++-
2 files changed, 95 insertions(+), 9 deletions(-)
diff --git a/www/manager6/storage/wizard/CommonSettings.js b/www/manager6/storage/wizard/CommonSettings.js
index 97e49e981..a2bd240fd 100644
--- a/www/manager6/storage/wizard/CommonSettings.js
+++ b/www/manager6/storage/wizard/CommonSettings.js
@@ -9,6 +9,7 @@ Ext.define('PVE.storage.wizard.CommonSettings', {
// submitted as content instead of offering a selector
fixedContent: undefined,
contentFieldConfig: undefined,
+ nodesFieldConfig: undefined,
onGetValues: function (values) {
values.disable = values.enable ? 0 : 1;
@@ -47,14 +48,17 @@ Ext.define('PVE.storage.wizard.CommonSettings', {
}
me.column2 = [
- {
- xtype: 'pveNodeSelector',
- name: 'nodes',
- fieldLabel: gettext('Nodes'),
- emptyText: gettext('All') + ' (' + gettext('No restrictions') + ')',
- multiSelect: true,
- autoSelect: false,
- },
+ Ext.apply(
+ {
+ xtype: 'pveNodeSelector',
+ name: 'nodes',
+ fieldLabel: gettext('Nodes'),
+ emptyText: gettext('All') + ' (' + gettext('No restrictions') + ')',
+ multiSelect: true,
+ autoSelect: false,
+ },
+ me.nodesFieldConfig,
+ ),
{
xtype: 'proxmoxcheckbox',
name: 'enable',
diff --git a/www/manager6/storage/wizard/Wizard.js b/www/manager6/storage/wizard/Wizard.js
index ce28f6845..ef3e505eb 100644
--- a/www/manager6/storage/wizard/Wizard.js
+++ b/www/manager6/storage/wizard/Wizard.js
@@ -23,6 +23,25 @@ PVE.storage.wizard.summaryFields = {
options: { label: gettext('Mount Options') },
};
+PVE.storage.wizard.lvmNodesValidator = function () {
+ let wizard = this.up('window');
+ if (wizard?.getViewModel().get('usage') !== 'lvm') {
+ return true;
+ }
+ let nodes = []
+ .concat(this.getValue() ?? [])
+ .join(',')
+ .split(',');
+ nodes = nodes.filter((node) => node !== '');
+ if (!nodes.length || nodes.includes(Proxmox.NodeName)) {
+ return true;
+ }
+ return Ext.String.format(
+ gettext('Must include node "{0}", which sets up the volume group.'),
+ Proxmox.NodeName,
+ );
+};
+
Ext.define('PVE.storage.wizard.RemoteStorage', {
extend: 'PVE.window.Wizard',
alias: 'widget.pveRemoteStorageWizard',
@@ -80,6 +99,9 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
button.setDisabled(me.buttonStateBeforeBusy?.[idx] ?? false),
);
delete me.buttonStateBeforeBusy;
+ if (me.frozenValues) {
+ me.down('#back').setDisabled(true);
+ }
}
},
@@ -108,6 +130,31 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
});
},
+ created: undefined,
+
+ // once a first object exists, keep the settings it was created from: lock the other tabs and
+ // return the frozen values
+ freeze: function () {
+ let me = this;
+ if (me.frozenValues) {
+ return;
+ }
+ me.frozenValues = me.getValues();
+ let tp = me.down('#wizcontent');
+ let current = tp.getActiveTab();
+ tp.items.each((tab) => {
+ if (tab !== current) {
+ tab.disable();
+ }
+ });
+ me.down('#back').setDisabled(true);
+ },
+
+ getValues: function (dirtyOnly) {
+ let me = this;
+ return me.frozenValues ? Ext.clone(me.frozenValues) : me.callParent([dirtyOnly]);
+ },
+
// a step calls next() on success and just returns after a failure, so Finish retries from
// there
runSubmitChain: function (steps) {
@@ -125,6 +172,41 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
run(0);
},
+ // check all IDs up front, a conflict found halfway would leave objects behind;
+ // conflict(entries) returns safe HTML
+ assertStorageIdsFree: function (ids, next, conflict) {
+ let me = this;
+ me.request({
+ url: '/storage',
+ method: 'GET',
+ failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
+ success: function (response) {
+ let entries = response.result.data || [];
+ let taken = ids.filter((id) => entries.some((entry) => entry.storage === id));
+ let msg = conflict?.(entries);
+ if (msg) {
+ Ext.Msg.alert(gettext('Error'), msg);
+ return;
+ }
+ if (taken.length) {
+ Ext.Msg.alert(
+ gettext('Error'),
+ Ext.String.format(
+ ngettext(
+ 'Storage ID {0} is already in use.',
+ 'Storage IDs {0} are already in use.',
+ taken.length,
+ ),
+ Ext.htmlEncode(taken.map((id) => `"${id}"`).join(', ')),
+ ),
+ );
+ return;
+ }
+ next();
+ },
+ });
+ },
+
// errorHtml (safe HTML) is prepended to an error
createStorage: function (params, next, errorHtml) {
let me = this;
@@ -232,7 +314,7 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
let notes = panel.down('#summaryNotes');
notes.removeAll();
- for (const note of typeEntry.summaryNotes?.(values) || []) {
+ for (const note of typeEntry.summaryNotes?.(values, me) || []) {
notes.add({ value: note });
}
},
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 53/76] ui: storage wizard: add Fibre Channel shared LVM setup
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (51 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 52/76] ui: storage wizard: preserve settings after partial setup Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 54/76] ui: storage wizard: reconcile unanswered storage creation Thomas Lamprecht
` (22 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Fibre Channel exposes block devices rather than a storage plugin.
Guide shared LVM setup using unused devices or existing volume groups,
showing storage ownership and local usage alongside device identity.
Accept only Fibre Channel devices. Local and unknown transports remain
display-only under "Show all devices": identity alone does not establish
shared access. Require stable by-id links for new groups because kernel
names can be reused before the task runs.
Bind inventory responses and selections to their scan node and request,
rejecting obsolete selections. Reuse these responses for cluster
visibility checks without silently excluding unchecked nodes.
The wizard tracks volume-group creation and keeps partial-setup settings
fixed. Resume status checks for known tasks, but never repeat an
unanswered creation request; direct users to task history if its task ID
is unknown.
Co-authored-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Makefile | 3 +
www/manager6/SANUtils.js | 179 +++++
www/manager6/Utils.js | 4 +
www/manager6/storage/wizard/CommonSettings.js | 2 +
www/manager6/storage/wizard/FibreChannel.js | 674 ++++++++++++++++++
www/manager6/storage/wizard/LunPicker.js | 25 +
www/manager6/storage/wizard/Wizard.js | 169 ++++-
7 files changed, 1054 insertions(+), 2 deletions(-)
create mode 100644 www/manager6/SANUtils.js
create mode 100644 www/manager6/storage/wizard/FibreChannel.js
create mode 100644 www/manager6/storage/wizard/LunPicker.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 7002518e0..602e3b9b9 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -7,6 +7,7 @@ JSSRC= \
StateProvider.js \
qemu/Architecture.js \
Utils.js \
+ SANUtils.js \
UIOptions.js \
Toolkit.js \
VNCConsole.js \
@@ -380,8 +381,10 @@ JSSRC= \
storage/ESXIEdit.js \
storage/wizard/Wizard.js \
storage/wizard/CommonSettings.js \
+ storage/wizard/LunPicker.js \
storage/wizard/NFS.js \
storage/wizard/CIFS.js \
+ storage/wizard/FibreChannel.js \
Workspace.js \
# end of JSSRC list
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
new file mode 100644
index 000000000..3b31ec614
--- /dev/null
+++ b/www/manager6/SANUtils.js
@@ -0,0 +1,179 @@
+Ext.define('PVE.SANUtils', {
+ singleton: true,
+
+ lun_fields: [
+ 'devpath',
+ 'size',
+ 'vendor',
+ 'model',
+ 'paths-total',
+ 'usage',
+ 'used-by-storage',
+ 'vgname',
+ 'pv-in-use',
+ 'detail',
+ 'by-id',
+ ],
+
+ render_san_lun_device: function (value, metaData, rec) {
+ let text = Ext.htmlEncode(value);
+ // only multipath mapped devices report their path count
+ let paths = rec.get('paths-total');
+ if (paths) {
+ text += ` (${paths} ${ngettext('path', 'paths', paths)})`;
+ }
+ return text;
+ },
+
+ // Unchecked nodes (offline, failed or timed out) may still see a LUN.
+ scan_nodes_for_luns: function (
+ callback,
+ request = (options) => Proxmox.Utils.API2Request(options),
+ ) {
+ let wwids = {};
+ let unchecked = [];
+ let nodes = PVE.data.ResourceStore.getNodes();
+ let pending = nodes.length;
+ let finish = function () {
+ callback({
+ sees: (wwid) =>
+ Object.keys(wwids)
+ .filter((node) => wwids[node].has(wwid))
+ .sort(),
+ unchecked: unchecked.sort(),
+ });
+ };
+ if (!pending) {
+ finish();
+ return;
+ }
+ let done = function () {
+ if (--pending === 0) {
+ finish();
+ }
+ };
+ for (const node of nodes) {
+ if (node.status !== 'online') {
+ unchecked.push(node.node);
+ done();
+ continue;
+ }
+ request({
+ url: `/nodes/${node.node}/scan/san-luns`,
+ method: 'GET',
+ timeout: 20000,
+ failure: function () {
+ unchecked.push(node.node);
+ done();
+ },
+ success: function (response) {
+ let rows = (response.result.data || []).filter(
+ (row) => !['sata', 'usb'].includes(row.transport),
+ );
+ if (rows.some((row) => row['wwid-unknown'])) {
+ unchecked.push(node.node);
+ }
+ wwids[node.node] = new Set(rows.map((row) => row.wwid).filter((w) => w));
+ done();
+ },
+ });
+ }
+ },
+
+ request_uncertain: function (response) {
+ return response.timedout || response.status === 0 || response.result?.success === undefined;
+ },
+
+ render_san_lun_usage: function (value, metaData, rec) {
+ let text = PVE.SANUtils.san_lun_usage_text(rec.data);
+ metaData.tdAttr = `data-qtip="${Ext.htmlEncode(text)}"`;
+ return text;
+ },
+
+ san_storage_only_connects: function (storeid) {
+ let rec = PVE.data.ResourceStore.findRecord('storage', storeid, 0, false, true, true);
+ if (!rec || !(rec.get('plugintype') === 'iscsi')) {
+ return false;
+ }
+ let content = `${rec.get('content') ?? ''}`;
+ return content === '' || content === 'none';
+ },
+
+ san_lun_usage_text: function (row) {
+ let value = row.usage;
+ if (value === 'unknown') {
+ return Ext.htmlEncode(
+ Ext.String.format(
+ gettext('Could not check this LUN for data: {0}'),
+ row.detail || gettext('unknown'),
+ ),
+ );
+ }
+ let sid = row['used-by-storage'];
+ // the scan also attributes LUNs to a connection-only storage, which leaves them free
+ let connects = sid && PVE.SANUtils.san_storage_only_connects(sid);
+ if (value === 'unused') {
+ if (connects) {
+ return Ext.htmlEncode(
+ Ext.String.format(gettext('unused, connected through storage "{0}"'), sid),
+ );
+ }
+ if (sid) {
+ return Ext.htmlEncode(Ext.String.format(gettext('used by storage "{0}"'), sid));
+ }
+ return gettext('unused');
+ }
+ if (connects) {
+ sid = undefined;
+ }
+ if (value === 'lvm' || (value === 'other' && row.detail === 'LVM2_member')) {
+ let vgname = row.vgname;
+ if (vgname) {
+ let text = Ext.String.format(gettext('VG "{0}"'), vgname);
+ if (sid === vgname) {
+ text = Ext.String.format(gettext('VG of storage "{0}"'), sid);
+ } else if (sid) {
+ text = Ext.String.format(
+ gettext('VG "{0}", used by storage "{1}"'),
+ vgname,
+ sid,
+ );
+ }
+ return Ext.htmlEncode(text);
+ }
+ let orphan = [0, '0', false].includes(row['pv-in-use']);
+ let text;
+ if (orphan) {
+ text = gettext('LVM physical volume, no volume group');
+ } else {
+ text = gettext('LVM physical volume, volume group not known on this node');
+ }
+ return Ext.htmlEncode(
+ sid ? Ext.String.format(gettext('{0}, used by storage "{1}"'), text, sid) : text,
+ );
+ }
+ return Ext.htmlEncode(row.detail || gettext('in use'));
+ },
+
+ render_san_transport: function (value) {
+ if (!value) {
+ return '';
+ }
+ return (
+ {
+ iscsi: 'iSCSI',
+ iser: 'iSER',
+ fc: 'FC',
+ fcoe: 'FCoE',
+ sas: 'SAS',
+ nvme: 'NVMe',
+ 'nvme-tcp': 'NVMe/TCP',
+ 'nvme-rdma': 'NVMe/RDMA',
+ 'nvme-fc': 'NVMe/FC',
+ 'nvme-loop': 'NVMe loop',
+ tcp: 'TCP',
+ rdma: 'RDMA',
+ }[value] ?? Ext.htmlEncode(value)
+ );
+ },
+});
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index ca850158b..bc5171712 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -274,6 +274,10 @@ Ext.define('PVE.Utils', {
return '<i class="fa fa-' + iconCls + '"></i> ' + value;
},
+ note_text: function (text) {
+ return `<span class="pmx-hint">${gettext('Note:')}</span> ${text}`;
+ },
+
validateZfsBlocksize: function (value) {
if (!value) {
return true;
diff --git a/www/manager6/storage/wizard/CommonSettings.js b/www/manager6/storage/wizard/CommonSettings.js
index a2bd240fd..fedd6dc31 100644
--- a/www/manager6/storage/wizard/CommonSettings.js
+++ b/www/manager6/storage/wizard/CommonSettings.js
@@ -10,6 +10,7 @@ Ext.define('PVE.storage.wizard.CommonSettings', {
fixedContent: undefined,
contentFieldConfig: undefined,
nodesFieldConfig: undefined,
+ extraColumn1: undefined,
onGetValues: function (values) {
values.disable = values.enable ? 0 : 1;
@@ -28,6 +29,7 @@ Ext.define('PVE.storage.wizard.CommonSettings', {
vtype: 'StorageId',
allowBlank: false,
},
+ ...(me.extraColumn1 || []),
];
if (!me.fixedContent) {
diff --git a/www/manager6/storage/wizard/FibreChannel.js b/www/manager6/storage/wizard/FibreChannel.js
new file mode 100644
index 000000000..4affbd8a7
--- /dev/null
+++ b/www/manager6/storage/wizard/FibreChannel.js
@@ -0,0 +1,674 @@
+Ext.define('PVE.storage.wizard.FCLunSelect', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pveStorageWizardFCLunSelect',
+ mixins: ['PVE.storage.wizard.LunPicker'],
+
+ onlineHelp: 'storage_lvm',
+
+ // { lun } of the selected row
+ selection: undefined,
+ lunScan: undefined,
+
+ // include unchecked nodes, one left out cannot use the storage after a migration
+ updateNodes: function () {
+ let me = this;
+ let wizard = me.up('window');
+ let lun = me.selection?.lun;
+ if (PVE.Utils.isStandaloneNode() || !lun?.wwid || !me.lunScan) {
+ return;
+ }
+ let nodes = new Set([
+ wizard.getScanNode(),
+ ...me.lunScan.sees(lun.wwid),
+ ...me.lunScan.unchecked,
+ ]);
+ wizard.down('field[name=nodes]').setValue([...nodes].sort());
+ },
+
+ validator: function () {
+ let wizard = this.up('window');
+ return !wizard.busy && !!wizard.getViewModel().get('vgmode');
+ },
+
+ updateActions: function () {
+ let me = this;
+ let wizard = me.up('window');
+ let blocked = wizard.busy;
+ let grid = me.down('#lunGrid');
+ grid.setDisabled(blocked || me.scanning);
+ me.down('#rescan').setDisabled(blocked);
+ me.down('pveStorageScanNodeSelector')?.setDisabled(blocked);
+ if (wizard.down('#wizcontent').getActiveTab() === me) {
+ wizard.down('#next').setDisabled(!me.validator());
+ }
+ },
+
+ currentSelection: function (values) {
+ let me = this;
+ let wizard = me.up('window');
+ let rec = me.down('#lunGrid').getSelection()[0];
+ if (
+ me.scanning ||
+ !rec?.get('selectable') ||
+ rec.data !== me.selection?.lun ||
+ rec.get('scanNode') !== wizard.getScanNode() ||
+ rec.get('scanGeneration') !== me.scanGeneration
+ ) {
+ return false;
+ }
+ return values.vgmode === 'existing'
+ ? rec.get('usage') === 'lvm' && values.vgname === rec.get('vgname')
+ : values.vgmode === 'create' &&
+ rec.get('usage') === 'unused' &&
+ !!rec.get('by-id') &&
+ values.device === rec.get('by-id');
+ },
+
+ updateEmptyText: function () {
+ let me = this;
+ let text = Ext.String.format(
+ gettext(
+ 'No Fibre Channel devices found on node "{0}". Check zoning and array access, and rescan the host bus adapters on the node. Then press Rescan.',
+ ),
+ Ext.htmlEncode(me.up('window').getScanNode()),
+ );
+ if (!me.down('#showAllDevices').getValue()) {
+ text +=
+ ' ' +
+ gettext(
+ '"Show all devices" also lists local devices and devices with an unknown transport.',
+ );
+ }
+ me.down('#lunGrid').setEmptyText(text);
+ },
+
+ updatePrivilegeNote: function () {
+ let me = this;
+ let note = me.down('#privilegeNote');
+ if (!note || me.canModify) {
+ return;
+ }
+ let rows = me.down('#lunGrid').getStore().getRange();
+ let stuck =
+ me.scanDone &&
+ !rows.some((rec) => rec.get('selectable')) &&
+ rows.some((rec) => rec.get('usage') === 'unused' && !rec.get('reason'));
+ let text = gettext(
+ 'Without the Sys.Modify privilege on "/", only LUNs with an existing volume group can be selected.',
+ );
+ note.setValue(
+ stuck
+ ? `<i class="fa fa-exclamation-triangle warning"></i> ${text}`
+ : PVE.Utils.note_text(text),
+ );
+ note[stuck ? 'addCls' : 'removeCls']('pmx-hint');
+ },
+
+ refreshLuns: function () {
+ let me = this;
+ if (!me.rendered || me.destroyed) {
+ return;
+ }
+ me.updateEmptyText();
+ let wizard = me.up('window');
+ let node = wizard.getScanNode();
+ let generation = (me.scanGeneration = (me.scanGeneration ?? 0) + 1);
+ let current = () =>
+ !me.destroyed && generation === me.scanGeneration && node === wizard.getScanNode();
+ let grid = me.down('#lunGrid');
+ me.scanning = true;
+ me.scanDone = false;
+ grid.getSelectionModel().deselectAll();
+ me.selection = undefined;
+ me.lunScan = undefined;
+ wizard.getViewModel().set({ vgmode: '', device: '', vgname: '' });
+ grid.getStore().setData([]);
+ Proxmox.Utils.setErrorMask(grid, false);
+ grid.setLoading(true);
+ wizard.updateActions();
+ // the grid and the cluster-wide check share the raw local scan within this load
+ let scans = new Map();
+ let request = function (options) {
+ let scan = scans.get(options.url);
+ if (scan) {
+ if (scan.outcome) {
+ options[scan.outcome](scan.response);
+ } else {
+ scan.waiting.push(options);
+ }
+ return;
+ }
+ scan = { waiting: [options] };
+ scans.set(options.url, scan);
+ let finish = function (outcome, response) {
+ scan.outcome = outcome;
+ scan.response = response;
+ scan.waiting.forEach((listener) => listener[outcome](response));
+ scan.waiting = [];
+ };
+ Proxmox.Utils.API2Request({
+ ...options,
+ failure: (response) => finish('failure', response),
+ success: (response) => finish('success', response),
+ });
+ };
+ request({
+ url: `/nodes/${node}/scan/san-luns`,
+ method: 'GET',
+ timeout: 20000,
+ failure: function (response) {
+ if (!current()) {
+ return;
+ }
+ me.scanning = false;
+ grid.setLoading(false);
+ Proxmox.Utils.setErrorMask(grid, response.htmlStatus);
+ me.updatePrivilegeNote();
+ wizard.updateActions();
+ },
+ success: function (response) {
+ if (!current()) {
+ return;
+ }
+ grid.getStore().setData(
+ (response.result.data || []).map((row) => ({
+ ...row,
+ scanNode: node,
+ scanGeneration: generation,
+ })),
+ );
+ me.scanning = false;
+ me.scanDone = true;
+ grid.setLoading(false);
+ me.updatePrivilegeNote();
+ wizard.updateActions();
+ },
+ });
+ if (!PVE.Utils.isStandaloneNode()) {
+ PVE.SANUtils.scan_nodes_for_luns(function (result) {
+ if (!current()) {
+ return;
+ }
+ me.lunScan = result;
+ me.updateNodes();
+ }, request);
+ }
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ let sanTransports = { fc: 1, 'nvme-fc': 1 };
+
+ // these need an initiator connection, which this storage type does not set up on other
+ // nodes
+ let remoteTransports = {
+ fcoe: 1,
+ iscsi: 1,
+ 'nvme-tcp': 1,
+ 'nvme-rdma': 1,
+ 'nvme-loop': 1,
+ };
+
+ // a device on these is attached to this node only; for SAS or an unknown transport the scan
+ // cannot tell whether other nodes reach it
+ let localTransports = { ata: 1, mmc: 1, nvme: 1, sata: 1, sbp: 1, usb: 1, virtio: 1 };
+
+ let sanTransportFilter = new Ext.util.Filter({
+ id: 'san-transport',
+ filterFn: (rec) => !!sanTransports[rec.get('transport')],
+ });
+
+ let remoteTransportFilter = new Ext.util.Filter({
+ id: 'remote-transport',
+ filterFn: (rec) => !remoteTransports[rec.get('transport')],
+ });
+
+ // creating the volume group needs Sys.Modify on /
+ let caps = Ext.state.Manager.get('GuiCap');
+ let canModify = !!caps.dc['Sys.Modify'];
+ me.canModify = canModify;
+
+ let notOffered = function (data) {
+ if (!sanTransports[data.transport]) {
+ return localTransports[data.transport]
+ ? gettext('Not offered: local device')
+ : gettext('Not offered: shared access cannot be verified for this transport');
+ }
+ // a kernel name may point to another device by the time the volume group is created
+ if (data.usage === 'unused' && !data['used-by-storage'] && !data['by-id']) {
+ return gettext('The LUN has no stable device name for creating a volume group.');
+ }
+ return undefined;
+ };
+
+ let columns = me.lunColumns();
+ let store = Ext.create('Ext.data.Store', {
+ fields: [
+ ...PVE.SANUtils.lun_fields,
+ 'scanNode',
+ 'scanGeneration',
+ 'transport',
+ 'serial',
+ 'wwn',
+ 'wwid',
+ 'iscsi-lun',
+ {
+ name: 'reason',
+ calculate: notOffered,
+ },
+ {
+ name: 'selectable',
+ calculate: function (data) {
+ if (notOffered(data) || data['used-by-storage']) {
+ return false;
+ }
+ if (data.usage === 'unused') {
+ return canModify;
+ }
+ return data.usage === 'lvm' && !!data.vgname;
+ },
+ },
+ ],
+ filters: [remoteTransportFilter, sanTransportFilter],
+ sorters: [
+ {
+ sorterFn: function (a, b) {
+ let rank = (rec) => (sanTransports[rec.data.transport] ? 0 : 1);
+ return rank(a) - rank(b) || a.data.devpath.localeCompare(b.data.devpath);
+ },
+ },
+ ],
+ });
+
+ let updateSelection = function (rec) {
+ let wizard = me.up('window');
+ me.selection = rec ? { lun: rec.data } : undefined;
+ let vm = wizard.getViewModel();
+ if (!rec || !rec.get('selectable')) {
+ vm.set({ vgmode: '', device: '', vgname: '' });
+ } else if (rec.get('usage') === 'unused') {
+ vm.set({ vgmode: 'create', device: rec.get('by-id'), vgname: '' });
+ } else {
+ vm.set({ vgmode: 'existing', device: '', vgname: rec.get('vgname') });
+ }
+ me.updateNodes();
+ };
+
+ me.items = [
+ {
+ xtype: 'displayfield',
+ value: gettext('Select the SAN LUN for the volume group:'),
+ },
+ {
+ xtype: 'grid',
+ itemId: 'lunGrid',
+ height: 220,
+ store: store,
+ viewConfig: {
+ trackOver: false,
+ getRowClass: (rec) => (rec.get('selectable') ? '' : 'x-item-disabled'),
+ },
+ listeners: {
+ beforeselect: (grid, rec) => rec.get('selectable'),
+ selectionchange: (sm, selection) => updateSelection(selection[0]),
+ },
+ columns: [
+ {
+ header: gettext('Device'),
+ dataIndex: 'devpath',
+ flex: 3,
+ renderer: PVE.SANUtils.render_san_lun_device,
+ },
+ {
+ header: gettext('Transport'),
+ dataIndex: 'transport',
+ width: 80,
+ renderer: PVE.SANUtils.render_san_transport,
+ },
+ columns.size,
+ columns.model,
+ {
+ header: gettext('Usage'),
+ dataIndex: 'usage',
+ flex: 2,
+ renderer: function (value, metaData, rec) {
+ let reason = rec.get('reason');
+ if (!reason) {
+ return PVE.SANUtils.render_san_lun_usage(value, metaData, rec);
+ }
+ let text = Ext.htmlEncode(reason);
+ metaData.tdAttr = `data-qtip="${Ext.htmlEncode(text)}"`;
+ return text;
+ },
+ },
+ ],
+ },
+ {
+ xtype: 'displayfield',
+ itemId: 'privilegeNote',
+ hidden: canModify,
+ },
+ {
+ xtype: 'container',
+ layout: 'hbox',
+ items: [
+ {
+ xtype: 'button',
+ itemId: 'rescan',
+ text: gettext('Rescan'),
+ iconCls: 'fa fa-search',
+ handler: () => me.refreshLuns(),
+ },
+ ],
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ itemId: 'showAllDevices',
+ boxLabel: gettext('Show all devices'),
+ submitValue: false,
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext('Also list devices with local or unknown transports.'),
+ },
+ listeners: {
+ change: function (f, value) {
+ me.updateEmptyText();
+ let filters = store.getFilters();
+ if (value) {
+ filters.remove(sanTransportFilter);
+ } else {
+ filters.add(sanTransportFilter);
+ let sm = me.down('#lunGrid').getSelectionModel();
+ let selected = sm.getSelection()[0];
+ if (selected && store.indexOf(selected) < 0) {
+ sm.deselectAll();
+ }
+ }
+ me.updatePrivilegeNote();
+ },
+ },
+ },
+ {
+ xtype: 'textfield',
+ name: 'vgmode',
+ hidden: true,
+ allowBlank: false,
+ bind: { value: '{vgmode}' },
+ },
+ {
+ xtype: 'textfield',
+ name: 'device',
+ hidden: true,
+ allowBlank: true,
+ bind: { value: '{device}' },
+ },
+ ];
+
+ if (!PVE.Utils.isStandaloneNode()) {
+ me.items.unshift({
+ xtype: 'pveStorageScanNodeSelector',
+ listeners: {
+ change: function () {
+ me.refreshLuns();
+ },
+ },
+ });
+ }
+
+ // a step is created before its window
+ me.on(
+ 'afterrender',
+ function () {
+ me.mon(me.up('window'), 'setupchange', me.updateActions, me);
+ me.refreshLuns();
+ },
+ me,
+ { single: true },
+ );
+ me.callParent();
+ me.updatePrivilegeNote();
+ },
+});
+
+PVE.storage.wizard.types.fc = {
+ text: gettext('Fibre Channel'),
+ description: gettext('Block storage (LUNs) on a SAN, attached via Fibre Channel.'),
+ group: 'san',
+ apiType: 'lvm',
+ viewModel: {
+ data: {
+ vgmode: '',
+ device: '',
+ vgname: '',
+ },
+ formulas: {
+ vgCreate: (get) => get('vgmode') === 'create',
+ },
+ },
+ steps: () => [
+ {
+ xtype: 'pveStorageWizardFCLunSelect',
+ title: gettext('SAN LUN'),
+ },
+ ],
+ settings: {
+ onlineHelp: 'storage_lvm',
+ cts: ['images', 'rootdir'],
+ defaultContent: ['images', 'rootdir'],
+ extraColumn1: [
+ {
+ xtype: 'textfield',
+ name: 'vgname',
+ fieldLabel: gettext('Volume group'),
+ allowBlank: true,
+ emptyText: gettext('use storage ID'),
+ bind: {
+ value: '{vgname}',
+ readOnly: '{!vgCreate}',
+ },
+ },
+ ],
+ advancedColumn1: [
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'saferemove',
+ uncheckedValue: 0,
+ fieldLabel: gettext('Wipe Removed Volumes'),
+ },
+ ],
+ },
+
+ canSubmit: function (wizard) {
+ return !!wizard.getValues().vgmode;
+ },
+
+ summaryRows: function (values, wizard) {
+ let step = wizard.down('pveStorageWizardFCLunSelect');
+ let { lun = {} } = step.selection ?? {};
+ let rows = [];
+ if (values.vgmode === 'create') {
+ rows.push({
+ label: gettext('Volume group'),
+ value: Ext.String.format(
+ gettext('Create "{0}" on {1} ({2})'),
+ values.vgname || values.storage,
+ values.device,
+ Proxmox.Utils.format_size(lun.size),
+ ),
+ weight: 20,
+ });
+ } else {
+ rows.push({
+ label: gettext('Volume group'),
+ value: Ext.String.format(gettext('Use existing "{0}"'), values.vgname),
+ weight: 20,
+ });
+ }
+ let unchecked = (step.lunScan?.unchecked ?? []).filter((node) =>
+ []
+ .concat(values.nodes ?? [])
+ .join(',')
+ .split(',')
+ .includes(node),
+ );
+ if (unchecked.length) {
+ rows.push({
+ label: gettext('Not checked'),
+ value: Ext.String.format(
+ gettext('Could not check whether these nodes see the LUN: {0}.'),
+ unchecked.join(', '),
+ ),
+ weight: 33,
+ });
+ }
+ return { rows, skip: ['vgmode', 'vgname', 'device'] };
+ },
+
+ submit: function (wizard, values) {
+ let step = wizard.down('pveStorageWizardFCLunSelect');
+ if (!step.currentSelection(values)) {
+ Ext.Msg.alert(
+ gettext('Error'),
+ gettext('Select a LUN from the current scan before continuing.'),
+ );
+ return;
+ }
+ let enc = Ext.htmlEncode;
+ let mode = values.vgmode;
+ delete values.vgmode;
+
+ let params = {
+ storage: values.storage,
+ type: 'lvm',
+ vgname: values.vgname || values.storage,
+ shared: 1,
+ content: values.content,
+ nodes: values.nodes,
+ disable: values.disable,
+ saferemove: values.saferemove,
+ };
+
+ if (mode === 'existing') {
+ wizard.runSubmitChain([(next) => wizard.createStorage(params, next)]);
+ return;
+ }
+
+ let created = (wizard.created ??= {});
+ let node = wizard.getScanNode();
+ let device = values.device;
+
+ let vgCreatedHint = () =>
+ Ext.String.format(
+ gettext(
+ 'The volume group "{0}" was created on node "{1}". Press {2} to retry the remaining steps, or complete the setup under Datacenter > Storage > Add > LVM.',
+ ),
+ enc(params.vgname),
+ enc(node),
+ enc(wizard.down('#submit').text),
+ );
+
+ let volumeGroupFailure = function (msg, final) {
+ if (final) {
+ delete created.task;
+ wizard.unfreeze();
+ } else {
+ msg +=
+ '<br><br>' +
+ Ext.String.format(
+ gettext(
+ 'If the task outcome stays unknown, check whether the volume group "{0}" exists on node "{1}" and complete the setup under Datacenter > Storage > Add > LVM.',
+ ),
+ enc(params.vgname),
+ enc(node),
+ );
+ }
+ wizard.updateActions();
+ Ext.Msg.alert(gettext('Error'), msg);
+ };
+ let unknownVolumeGroup = function () {
+ wizard.unknownVolumeGroup = true;
+ wizard.freeze();
+ wizard.showOperationNotice(
+ Ext.String.format(
+ gettext(
+ 'The node did not answer in time, so a volume group task may still be running on node "{0}". Check its task history. Once the task is done, run the wizard again; it offers the volume group "{1}" if the task created it.',
+ ),
+ enc(node),
+ enc(params.vgname),
+ ),
+ );
+ wizard.updateActions();
+ };
+ let waitForVolumeGroup = function (next) {
+ wizard.waitForTask(node, created.task, {
+ msg: gettext('Creating the volume group...'),
+ texts: {
+ running: Ext.String.format(
+ gettext(
+ 'The task on node "{0}" is still running. Check its task log, or press {1} to wait for it again. If it creates volume group "{2}", you can select that group when you run the wizard again.',
+ ),
+ enc(node),
+ enc(wizard.down('#submit').text),
+ enc(params.vgname),
+ ),
+ },
+ success: function () {
+ created.volumeGroup = params.vgname;
+ delete created.task;
+ next();
+ },
+ failure: volumeGroupFailure,
+ });
+ };
+
+ wizard.runSubmitChain([
+ (next) => {
+ if (created.task || created.volumeGroup) {
+ next();
+ return;
+ }
+ wizard.assertStorageIdsFree([params.storage], next);
+ },
+ (next) => {
+ if (created.volumeGroup) {
+ next();
+ return;
+ }
+ // a task started earlier is still the one to wait for, a second create would
+ // race it on the same device
+ if (created.task) {
+ waitForVolumeGroup(next);
+ return;
+ }
+ wizard.request({
+ url: `/nodes/${node}/disks/lvm`,
+ method: 'POST',
+ params: {
+ name: params.vgname,
+ device: device,
+ },
+ failure: function (response) {
+ if (PVE.SANUtils.request_uncertain(response)) {
+ unknownVolumeGroup();
+ } else {
+ volumeGroupFailure(response.htmlStatus, true);
+ }
+ },
+ success: function (response) {
+ try {
+ Proxmox.Utils.parse_task_upid(response.result.data);
+ } catch (_error) {
+ unknownVolumeGroup();
+ return;
+ }
+ created.task = response.result.data;
+ wizard.freeze();
+ waitForVolumeGroup(next);
+ },
+ });
+ },
+ (next) => wizard.createStorage(params, next, vgCreatedHint()),
+ ]);
+ },
+};
diff --git a/www/manager6/storage/wizard/LunPicker.js b/www/manager6/storage/wizard/LunPicker.js
new file mode 100644
index 000000000..cb3eba7b3
--- /dev/null
+++ b/www/manager6/storage/wizard/LunPicker.js
@@ -0,0 +1,25 @@
+Ext.define('PVE.storage.wizard.LunPicker', {
+ extend: 'Ext.Mixin',
+
+ mixinConfig: {
+ id: 'lunPicker',
+ },
+
+ lunColumns: function () {
+ return {
+ size: {
+ header: gettext('Size'),
+ dataIndex: 'size',
+ width: 90,
+ renderer: Proxmox.Utils.render_size,
+ },
+ model: {
+ header: gettext('Vendor') + '/' + gettext('Model'),
+ dataIndex: 'model',
+ flex: 2,
+ renderer: (value, metaData, rec) =>
+ Ext.htmlEncode(`${rec.get('vendor') || ''} ${value || ''}`.trim()),
+ },
+ };
+ },
+});
diff --git a/www/manager6/storage/wizard/Wizard.js b/www/manager6/storage/wizard/Wizard.js
index ef3e505eb..b4978d296 100644
--- a/www/manager6/storage/wizard/Wizard.js
+++ b/www/manager6/storage/wizard/Wizard.js
@@ -82,6 +82,8 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
busy: false,
+ unknownVolumeGroup: false,
+
// the continuation lives in this window: block closing and the buttons while waiting, then
// restore them
setBusy: function (busy, msg) {
@@ -103,6 +105,42 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
me.down('#back').setDisabled(true);
}
}
+ me.updateActions();
+ },
+
+ canSubmit: function () {
+ let me = this;
+ if (me.busy || me.unknownVolumeGroup) {
+ return false;
+ }
+ let canSubmit = PVE.storage.wizard.types[me.wizardType].canSubmit;
+ if (canSubmit) {
+ return canSubmit(me);
+ }
+ return true;
+ },
+
+ actionText: function () {
+ return gettext('Finish');
+ },
+
+ updateActions: function () {
+ let me = this;
+ if (!me.down('#submit')) {
+ return;
+ }
+ me.down('#submit').setText(me.actionText());
+ // a type with its own submit condition decides the state of the confirm buttons
+ let confirmActive = me.down('#wizcontent').getActiveTab() === me.down('#confirm');
+ if (PVE.storage.wizard.types[me.wizardType].canSubmit && confirmActive) {
+ me.down('#submit').setDisabled(!me.canSubmit());
+ me.down('#back').setDisabled(!!me.frozenValues || me.busy);
+ }
+ me.fireEvent('setupchange', me);
+ // after the listeners, so no step can enable an action again
+ if (me.busy || me.unknownVolumeGroup) {
+ ['#back', '#next', '#submit'].forEach((id) => me.down(id).disable());
+ }
},
// keeps the window busy, so it cannot close with the continuation pending
@@ -150,13 +188,117 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
me.down('#back').setDisabled(true);
},
+ unfreeze: function () {
+ let me = this;
+ if (!me.frozenValues) {
+ return;
+ }
+ delete me.frozenValues;
+ me.down('#wizcontent').items.each((tab) => tab.enable());
+ me.down('#back').setDisabled(false);
+ },
+
getValues: function (dirtyOnly) {
let me = this;
return me.frozenValues ? Ext.clone(me.frozenValues) : me.callParent([dirtyOnly]);
},
- // a step calls next() on success and just returns after a failure, so Finish retries from
- // there
+ // Wait without a TaskProgress window, as closing it would drop the continuation. An unreachable
+ // task status is not final: the next submit resumes waiting.
+ waitForTask: function (node, upid, options) {
+ let me = this;
+ let transportFailures = 0;
+ let deadline = Date.now() + 180000;
+ let ended = false;
+ let lastRunning = 0;
+ let unknown =
+ options.texts?.unknown ??
+ Ext.String.format(
+ gettext('The task outcome is unknown. Press {0} to wait for it again.'),
+ Ext.htmlEncode(me.actionText()),
+ );
+ let timer;
+ let fail = function (msg, final) {
+ if (me.destroyed || ended) {
+ return;
+ }
+ ended = true;
+ clearTimeout(timer);
+ me.setBusy(false);
+ options.failure(msg, final);
+ };
+ timer = Ext.defer(
+ () =>
+ fail(
+ lastRunning && Date.now() - lastRunning < 3000
+ ? (options.texts?.running ?? unknown)
+ : unknown,
+ false,
+ ),
+ 180000,
+ );
+ let poll = function () {
+ if (me.destroyed || ended) {
+ return;
+ }
+ if (Date.now() >= deadline) {
+ fail(unknown, false);
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${node}/tasks/${encodeURIComponent(upid)}/status`,
+ method: 'GET',
+ timeout: Math.max(1, Math.min(30000, deadline - Date.now())),
+ failure: function (response) {
+ if (me.destroyed || ended) {
+ return;
+ }
+ lastRunning = 0;
+ if (++transportFailures <= 10 && Date.now() < deadline) {
+ Ext.defer(poll, 3000);
+ return;
+ }
+ fail(`${unknown}<br><br>${response.htmlStatus}`, false);
+ },
+ success: function (response) {
+ if (me.destroyed || ended) {
+ return;
+ }
+ transportFailures = 0;
+ let task = response.result.data;
+ if (task?.status === 'running') {
+ lastRunning = Date.now();
+ if (Date.now() >= deadline) {
+ fail(options.texts?.running ?? unknown, false);
+ return;
+ }
+ Ext.defer(poll, 1000);
+ return;
+ }
+ let outcome =
+ task?.status === 'stopped' && task.exitstatus
+ ? Proxmox.Utils.parse_task_status(task.exitstatus)
+ : 'unknown';
+ if (outcome === 'unknown') {
+ fail(unknown, false);
+ } else if (outcome === 'error') {
+ fail(Ext.htmlEncode(task.exitstatus), true);
+ } else {
+ // a task that finished with warnings did its job
+ ended = true;
+ clearTimeout(timer);
+ me.setBusy(false);
+ options.success();
+ }
+ },
+ });
+ };
+ me.setBusy(true, options.msg);
+ poll();
+ },
+
+ // a step calls next() on success and just returns after a failure, so the next submit retries
+ // from there
runSubmitChain: function (steps) {
let me = this;
let run = function (idx) {
@@ -207,6 +349,14 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
});
},
+ showOperationNotice: function (text) {
+ let notice = this.down('#operationNotice');
+ notice.setValue(text ? `<i class="fa fa-exclamation-triangle warning"></i> ${text}` : '');
+ notice.setHidden(!text);
+ this.down('#summaryNotes').setHidden(!!text);
+ this.updateActions();
+ },
+
// errorHtml (safe HTML) is prepended to an error
createStorage: function (params, next, errorHtml) {
let me = this;
@@ -317,6 +467,7 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
for (const note of typeEntry.summaryNotes?.(values, me) || []) {
notes.add({ value: note });
}
+ me.updateActions();
},
submitSimple: function (values) {
@@ -422,6 +573,11 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
},
],
},
+ {
+ xtype: 'displayfield',
+ itemId: 'operationNotice',
+ hidden: true,
+ },
{
xtype: 'container',
itemId: 'summaryNotes',
@@ -437,6 +593,9 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
},
onSubmit: function () {
let wizard = this.up('window');
+ if (!wizard.canSubmit()) {
+ return;
+ }
let values = wizard.getValues();
delete values.delete;
let typeEntry = PVE.storage.wizard.types[wizard.wizardType];
@@ -467,5 +626,11 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
});
me.callParent();
+ me.query('field').forEach((field) => {
+ field.on('change', () => me.updateActions());
+ field.on('validitychange', () => me.updateActions());
+ });
+ me.down('#wizcontent').on('tabchange', () => me.updateActions());
+ me.down('#wizcontent').on('beforetabchange', () => !me.busy && !me.frozenValues);
},
});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 54/76] ui: storage wizard: reconcile unanswered storage creation
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (52 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 53/76] ui: storage wizard: add Fibre Channel shared LVM setup Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 55/76] ui: storage wizard: add ZFS over iSCSI support Thomas Lamprecht
` (21 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
A lost POST response does not prove creation failed. Keep the submitted
settings fixed and check the storage before retrying: accept matching
normalized settings, report conflicting entries, and retry creation only
if the entry is absent. The storage API serializes ID checks with the
write. Label the recovery action explicitly so users know it checks the
outcome before continuing.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/storage/wizard/Wizard.js | 126 +++++++++++++++++++++++++-
1 file changed, 124 insertions(+), 2 deletions(-)
diff --git a/www/manager6/storage/wizard/Wizard.js b/www/manager6/storage/wizard/Wizard.js
index b4978d296..ce0eccb73 100644
--- a/www/manager6/storage/wizard/Wizard.js
+++ b/www/manager6/storage/wizard/Wizard.js
@@ -42,6 +42,47 @@ PVE.storage.wizard.lvmNodesValidator = function () {
);
};
+// per submitted storage type, the keys besides type, content, nodes and disable that an existing
+// storage must match, and how to compare them
+PVE.storage.wizard.reconcile = {
+ lvm: { keys: ['vgname', 'base', 'shared', 'saferemove'], booleans: ['shared', 'saferemove'] },
+};
+
+PVE.storage.wizard.storageMatches = function (wanted, actual) {
+ let rule = PVE.storage.wizard.reconcile[wanted.type];
+ if (!rule) {
+ return false;
+ }
+ let booleans = ['disable', ...(rule.booleans ?? [])];
+ let normalize = function (config, key) {
+ let value = config[key];
+ if (booleans.includes(key)) {
+ return value === true || value === 1 || value === '1';
+ }
+ if (key === 'content' && (value === undefined || value === '')) {
+ value = config.type === 'lvm' ? 'images,rootdir' : 'images';
+ }
+ if (['nodes', 'content'].includes(key)) {
+ return [
+ ...new Set(
+ []
+ .concat(value ?? '')
+ .join(',')
+ .split(',')
+ .map((v) => v.trim())
+ .filter(Boolean),
+ ),
+ ]
+ .sort()
+ .join(',');
+ }
+ return String(value === undefined || value === null || value === '' ? '' : value);
+ };
+ return ['type', 'content', 'nodes', 'disable', ...rule.keys].every(
+ (key) => normalize(wanted, key) === normalize(actual, key),
+ );
+};
+
Ext.define('PVE.storage.wizard.RemoteStorage', {
extend: 'PVE.window.Wizard',
alias: 'widget.pveRemoteStorageWizard',
@@ -121,6 +162,10 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
},
actionText: function () {
+ let me = this;
+ if (me.unconfirmedStorage) {
+ return gettext('Check and Retry');
+ }
return gettext('Finish');
},
@@ -349,6 +394,17 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
});
},
+ storageUnknownText: function (html = true) {
+ let encode = html ? Ext.htmlEncode : (value) => value;
+ return Ext.String.format(
+ gettext(
+ 'The creation of storage "{0}" is not confirmed. Press {1} to check for it and to create it again if it is missing.',
+ ),
+ encode(this.unconfirmedStorage.params.storage),
+ encode(this.actionText()),
+ );
+ },
+
showOperationNotice: function (text) {
let notice = this.down('#operationNotice');
notice.setValue(text ? `<i class="fa fa-exclamation-triangle warning"></i> ${text}` : '');
@@ -357,8 +413,54 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
this.updateActions();
},
+ reconcileStorage: function () {
+ let me = this;
+ let pending = me.unconfirmedStorage;
+ if (!pending || me.busy) {
+ return;
+ }
+ let fail = (msg) => {
+ me.showOperationNotice(msg);
+ Ext.Msg.alert(gettext('Error'), msg);
+ };
+ let absent = function () {
+ // Creation checks the ID under the storage configuration lock and writes it last.
+ me.createStorage(pending.params, pending.next, pending.errorHtml, true);
+ };
+ me.request({
+ url: `/storage/${encodeURIComponent(pending.params.storage)}`,
+ method: 'GET',
+ success: function (response) {
+ if (
+ !PVE.storage.wizard.storageMatches(pending.params, response.result.data || {})
+ ) {
+ fail(
+ Ext.String.format(
+ gettext('Storage ID "{0}" is already in use with different settings.'),
+ Ext.htmlEncode(pending.params.storage),
+ ),
+ );
+ return;
+ }
+ delete me.unconfirmedStorage;
+ me.showOperationNotice();
+ pending.next();
+ },
+ failure: function (response) {
+ let missing =
+ response.result?.message?.trim() ===
+ `storage '${pending.params.storage}' does not exist`;
+ if (!PVE.SANUtils.request_uncertain(response) && missing) {
+ absent();
+ } else {
+ fail(me.storageUnknownText() + '<br><br>' + response.htmlStatus);
+ }
+ },
+ });
+ },
+
// errorHtml (safe HTML) is prepended to an error
- createStorage: function (params, next, errorHtml) {
+ createStorage: function (params, next, errorHtml, reconciled) {
let me = this;
params = Ext.apply({}, params);
Ext.Object.each(params, function (key, value) {
@@ -370,8 +472,24 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
url: '/storage',
method: 'POST',
params: params,
- success: () => next(),
+ success: function () {
+ delete me.unconfirmedStorage;
+ me.showOperationNotice();
+ next();
+ },
failure: function (response) {
+ let uncertain = PVE.SANUtils.request_uncertain(response);
+ let exists = /already defined/i.test(response.result?.message || '');
+ if (PVE.storage.wizard.reconcile[params.type] && (uncertain || exists)) {
+ me.unconfirmedStorage = { params: Ext.clone(params), next, errorHtml };
+ me.freeze();
+ me.updateActions();
+ me.showOperationNotice(me.storageUnknownText());
+ if (exists && !reconciled) {
+ me.reconcileStorage();
+ }
+ return;
+ }
let msg = response.htmlStatus;
if (errorHtml) {
msg = `${errorHtml}<br><br>${msg}`;
@@ -596,6 +714,10 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
if (!wizard.canSubmit()) {
return;
}
+ if (wizard.unconfirmedStorage) {
+ wizard.reconcileStorage();
+ return;
+ }
let values = wizard.getValues();
delete values.delete;
let typeEntry = PVE.storage.wizard.types[wizard.wizardType];
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 55/76] ui: storage wizard: add ZFS over iSCSI support
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (53 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 54/76] ui: storage wizard: reconcile unanswered storage creation Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 56/76] ui: dc: storage: add remote storage wizard entry to the add menu Thomas Lamprecht
` (20 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
There is no scan API for this SSH-based storage type, so the wizard
splits the existing single dialog into an appliance connection step,
showing only the fields relevant for the selected iSCSI provider, and
a volume settings step.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: show the write cache positively and only for the providers that
have the option; name the SSH key the appliance is managed with]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Makefile | 1 +
www/manager6/storage/wizard/ZFSOverISCSI.js | 180 ++++++++++++++++++++
2 files changed, 181 insertions(+)
create mode 100644 www/manager6/storage/wizard/ZFSOverISCSI.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 602e3b9b9..b60cfa569 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -385,6 +385,7 @@ JSSRC= \
storage/wizard/NFS.js \
storage/wizard/CIFS.js \
storage/wizard/FibreChannel.js \
+ storage/wizard/ZFSOverISCSI.js \
Workspace.js \
# end of JSSRC list
diff --git a/www/manager6/storage/wizard/ZFSOverISCSI.js b/www/manager6/storage/wizard/ZFSOverISCSI.js
new file mode 100644
index 000000000..c891aa7ea
--- /dev/null
+++ b/www/manager6/storage/wizard/ZFSOverISCSI.js
@@ -0,0 +1,180 @@
+Ext.define('PVE.storage.wizard.ZFSOverISCSIAppliance', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pveStorageWizardZFSOverISCSIAppliance',
+
+ onlineHelp: 'storage_zfs',
+
+ column1: [
+ {
+ xtype: 'pveiScsiProviderSelector',
+ name: 'iscsiprovider',
+ value: 'comstar',
+ fieldLabel: gettext('iSCSI Provider'),
+ allowBlank: false,
+ listeners: {
+ change: function (f, value) {
+ let vm = this.up('window').getViewModel();
+ vm.set('isLIO', value === 'LIO');
+ vm.set('isComstar', value === 'comstar');
+ vm.set('hasWriteCacheOption', value === 'comstar' || value === 'istgt');
+ },
+ },
+ },
+ {
+ xtype: 'textfield',
+ name: 'portal',
+ fieldLabel: gettext('Portal'),
+ allowBlank: false,
+ bind: { value: '{portal}' },
+ },
+ {
+ xtype: 'textfield',
+ name: 'target',
+ fieldLabel: gettext('Target'),
+ allowBlank: false,
+ },
+ ],
+
+ column2: [
+ {
+ xtype: 'textfield',
+ name: 'comstar_tg',
+ fieldLabel: gettext('Target group'),
+ allowBlank: true,
+ bind: {
+ disabled: '{!isComstar}',
+ hidden: '{!isComstar}',
+ },
+ },
+ {
+ xtype: 'textfield',
+ name: 'comstar_hg',
+ fieldLabel: gettext('Host group'),
+ allowBlank: true,
+ bind: {
+ disabled: '{!isComstar}',
+ hidden: '{!isComstar}',
+ },
+ },
+ {
+ xtype: 'textfield',
+ name: 'lio_tpg',
+ fieldLabel: gettext('Target portal group'),
+ allowBlank: false,
+ disabled: true,
+ hidden: true,
+ bind: {
+ disabled: '{!isLIO}',
+ hidden: '{!isLIO}',
+ },
+ },
+ ],
+
+ columnB: [
+ {
+ xtype: 'displayfield',
+ bind: { value: '{sshKeyText}' },
+ },
+ ],
+});
+
+Ext.define('PVE.storage.wizard.ZFSOverISCSIVolume', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pveStorageWizardZFSOverISCSIVolume',
+
+ onlineHelp: 'storage_zfs',
+
+ onGetValues: function (values) {
+ values.nowritecache = values.writecache ? 0 : 1;
+ delete values.writecache;
+ return values;
+ },
+
+ column1: [
+ {
+ xtype: 'textfield',
+ name: 'pool',
+ fieldLabel: gettext('Pool'),
+ allowBlank: false,
+ },
+ {
+ xtype: 'textfield',
+ name: 'blocksize',
+ value: '4k',
+ fieldLabel: gettext('Block Size'),
+ allowBlank: false,
+ validator: PVE.Utils.validateZfsBlocksize,
+ },
+ ],
+
+ column2: [
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'sparse',
+ checked: false,
+ uncheckedValue: 0,
+ fieldLabel: gettext('Thin provision'),
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'writecache',
+ checked: true,
+ uncheckedValue: 0,
+ fieldLabel: gettext('Write cache'),
+ bind: {
+ disabled: '{!hasWriteCacheOption}',
+ hidden: '{!hasWriteCacheOption}',
+ },
+ },
+ ],
+});
+
+PVE.storage.wizard.types['zfs-over-iscsi'] = {
+ text: gettext('ZFS over iSCSI'),
+ description: gettext(
+ 'ZFS storage appliance managed over SSH, exporting volumes via iSCSI with snapshot support.',
+ ),
+ group: 'san',
+ apiType: 'zfs',
+ viewModel: {
+ data: {
+ isLIO: false,
+ isComstar: true,
+ hasWriteCacheOption: true,
+ portal: '',
+ },
+ formulas: {
+ sshKeyText: (get) =>
+ Ext.htmlEncode(
+ Ext.String.format(
+ gettext(
+ 'The appliance is managed as root over SSH. The private key must be stored in {0}, and the appliance must accept its public key.',
+ ),
+ `/etc/pve/priv/zfs/${get('portal') || '<portal>'}_id_rsa`,
+ ),
+ ),
+ },
+ },
+ steps: () => [
+ {
+ xtype: 'pveStorageWizardZFSOverISCSIAppliance',
+ title: gettext('Appliance'),
+ },
+ {
+ xtype: 'pveStorageWizardZFSOverISCSIVolume',
+ title: gettext('Volume Settings'),
+ },
+ ],
+ settings: {
+ onlineHelp: 'storage_zfs',
+ fixedContent: 'images',
+ },
+ summaryFields: {
+ nowritecache: {
+ label: gettext('Write cache'),
+ isBoolean: true,
+ invert: true,
+ skip: (values) => !['comstar', 'istgt'].includes(values.iscsiprovider),
+ },
+ },
+};
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 56/76] ui: dc: storage: add remote storage wizard entry to the add menu
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (54 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 55/76] ui: storage wizard: add ZFS over iSCSI support Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 57/76] ui: node: add SAN LUNs panel Thomas Lamprecht
` (19 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Put guided setup first in the Add menu while keeping the per-type
dialogs available separately.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/dc/StorageView.js | 13 ++++++++++++-
1 file changed, 12 insertions(+), 1 deletion(-)
diff --git a/www/manager6/dc/StorageView.js b/www/manager6/dc/StorageView.js
index e4c6f07db..316019227 100644
--- a/www/manager6/dc/StorageView.js
+++ b/www/manager6/dc/StorageView.js
@@ -72,7 +72,18 @@ Ext.define(
me.createStorageEditWindow(type);
};
};
- let addMenuItems = [];
+ let addMenuItems = [
+ {
+ text: gettext('Remote Storage') + '...',
+ iconCls: 'fa fa-fw fa-magic',
+ handler: () =>
+ Ext.create('PVE.storage.wizard.RemoteStorage', {
+ autoShow: true,
+ reloadCallback: () => store.load(),
+ }),
+ },
+ '-',
+ ];
for (const [type, storage] of Object.entries(PVE.Utils.storageSchema)) {
if (storage.hideAdd) {
continue;
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 57/76] ui: node: add SAN LUNs panel
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (55 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 56/76] ui: dc: storage: add remote storage wizard entry to the add menu Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 58/76] ui: san luns: show multipath path state Thomas Lamprecht
` (18 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Expose the node inventory of SAN LUNs separately from local disk
management. Show device identity, transport, size and usage, with local
or unknown transports hidden unless requested. An empty list names the
node and explains which connectivity and array settings to check.
Allow node auditors and storage administrators to open the view,
matching the inventory API permissions.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: place the view under Node > SAN; use the shared SAN renderers and
scan fields, the scan path-count names and node-specific empty-list
guidance]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Makefile | 1 +
www/manager6/node/Config.js | 15 ++++
www/manager6/node/SANLuns.js | 155 +++++++++++++++++++++++++++++++++++
3 files changed, 171 insertions(+)
create mode 100644 www/manager6/node/SANLuns.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index b60cfa569..2e1b23fed 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -247,6 +247,7 @@ JSSRC= \
node/Directory.js \
node/LVM.js \
node/LVMThin.js \
+ node/SANLuns.js \
node/StatusView.js \
node/Subscription.js \
node/Summary.js \
diff --git a/www/manager6/node/Config.js b/www/manager6/node/Config.js
index 217ee284c..3a84e96f6 100644
--- a/www/manager6/node/Config.js
+++ b/www/manager6/node/Config.js
@@ -357,6 +357,21 @@ Ext.define('PVE.node.Config', {
groups: ['storage'],
xtype: 'pveZFSList',
},
+ );
+ }
+
+ // the san-luns scan accepts either privilege
+ if (caps.nodes['Sys.Audit'] || caps.storage['Datastore.Allocate']) {
+ me.items.push({
+ xtype: 'pveSANLunList',
+ title: 'SAN',
+ itemId: 'san',
+ iconCls: 'fa fa-cubes',
+ });
+ }
+
+ if (caps.nodes['Sys.Audit']) {
+ me.items.push(
{
xtype: 'pveNodeCephStatus',
title: 'Ceph',
diff --git a/www/manager6/node/SANLuns.js b/www/manager6/node/SANLuns.js
new file mode 100644
index 000000000..593bfc519
--- /dev/null
+++ b/www/manager6/node/SANLuns.js
@@ -0,0 +1,155 @@
+Ext.define('PVE.node.SANLunList', {
+ extend: 'Ext.grid.GridPanel',
+ xtype: 'pveSANLunList',
+
+ stateful: true,
+ stateId: 'grid-node-san-luns',
+
+ disableSelection: true,
+
+ emptyDeviceText: function (showAll) {
+ let text = Ext.String.format(
+ gettext(
+ 'No SAN devices found on node "{0}". Check SAN connections and array access, then press Reload.',
+ ),
+ Ext.htmlEncode(this.nodename),
+ );
+ if (!showAll) {
+ text +=
+ ' ' +
+ gettext(
+ '"Show all devices" also lists local devices and devices with an unknown transport.',
+ );
+ }
+ return text;
+ },
+
+ viewConfig: {
+ trackOver: false,
+ },
+
+ columns: [
+ {
+ header: gettext('Device'),
+ dataIndex: 'devpath',
+ flex: 2,
+ renderer: PVE.SANUtils.render_san_lun_device,
+ },
+ {
+ header: gettext('Transport'),
+ dataIndex: 'transport',
+ width: 100,
+ renderer: PVE.SANUtils.render_san_transport,
+ },
+ {
+ header: gettext('Size'),
+ dataIndex: 'size',
+ width: 100,
+ renderer: Proxmox.Utils.render_size,
+ },
+ {
+ header: gettext('Vendor') + '/' + gettext('Model'),
+ dataIndex: 'model',
+ flex: 1,
+ renderer: (value, metaData, rec) =>
+ Ext.htmlEncode(`${rec.get('vendor') || ''} ${value || ''}`.trim()),
+ },
+ {
+ header: 'WWN',
+ dataIndex: 'wwn',
+ width: 200,
+ },
+ {
+ header: gettext('Serial'),
+ dataIndex: 'serial',
+ width: 140,
+ hidden: true,
+ },
+ {
+ header: gettext('Usage'),
+ dataIndex: 'usage',
+ flex: 2,
+ renderer: PVE.SANUtils.render_san_lun_usage,
+ },
+ ],
+
+ reload: function () {
+ this.getStore().load();
+ },
+
+ listeners: {
+ activate: function () {
+ this.reload();
+ },
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ me.nodename = me.pveSelNode.data.node;
+ if (!me.nodename) {
+ throw 'no node name specified';
+ }
+
+ me.emptyText = me.emptyDeviceText(false);
+
+ let sanTransports = {
+ fc: 1,
+ sas: 1,
+ iscsi: 1,
+ 'nvme-fc': 1,
+ 'nvme-tcp': 1,
+ 'nvme-rdma': 1,
+ 'nvme-loop': 1,
+ };
+
+ let sanTransportFilter = new Ext.util.Filter({
+ id: 'san-transport',
+ filterFn: (rec) => !!sanTransports[rec.get('transport')],
+ });
+
+ let store = Ext.create('Ext.data.Store', {
+ fields: [...PVE.SANUtils.lun_fields, 'transport', 'serial', 'wwn'],
+ proxy: {
+ type: 'proxmox',
+ url: `/api2/json/nodes/${me.nodename}/scan/san-luns`,
+ },
+ filters: [sanTransportFilter],
+ sorters: 'devpath',
+ });
+ me.store = store;
+
+ me.tbar = [
+ {
+ text: gettext('Reload'),
+ iconCls: 'fa fa-refresh',
+ handler: () => me.reload(),
+ },
+ '->',
+ {
+ xtype: 'proxmoxcheckbox',
+ boxLabel: gettext('Show all devices'),
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext('Also list devices with local or unknown transports.'),
+ },
+ listeners: {
+ change: function (f, value) {
+ me.setEmptyText(me.emptyDeviceText(value));
+ let filters = store.getFilters();
+ if (value) {
+ filters.remove(sanTransportFilter);
+ } else {
+ filters.add(sanTransportFilter);
+ }
+ },
+ },
+ },
+ ];
+
+ me.callParent();
+
+ Proxmox.Utils.monStoreErrors(me, store);
+ me.reload();
+ },
+});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 58/76] ui: san luns: show multipath path state
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (56 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 57/76] ui: node: add SAN LUNs panel Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 59/76] api: nodes: expose iSCSI status and maintenance Thomas Lamprecht
` (17 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Show active and total path counts with a warning when they differ,
also in the Fibre Channel wizard through the shared renderer.
Expand rows to show path states and fault counters, with checker state
and WWPNs for multipathd or ANA state and controller addresses for
native NVMe multipath. Administrators can inspect these without running
"multipath -ll" or reading sysfs.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: use the active path count under the name the scan uses and a
shared helper for the detail table markup]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/SANUtils.js | 20 +++++++++-
www/manager6/node/SANLuns.js | 76 +++++++++++++++++++++++++++++++++++-
2 files changed, 94 insertions(+), 2 deletions(-)
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index 3b31ec614..25c733263 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -7,6 +7,7 @@ Ext.define('PVE.SANUtils', {
'vendor',
'model',
'paths-total',
+ 'paths-active',
'usage',
'used-by-storage',
'vgname',
@@ -15,12 +16,29 @@ Ext.define('PVE.SANUtils', {
'by-id',
],
+ // headers and cells are HTML
+ render_detail_table: function (headers, rows) {
+ let header = headers.map((text) => `<th style="text-align: left;">${text}</th>`).join('');
+ let body = rows.map((cells) => `<tr><td>${cells.join('</td><td>')}</td></tr>`).join('');
+ return `<table style="border-spacing: 10px 2px;"><tr>${header}</tr>${body}</table>`;
+ },
+
render_san_lun_device: function (value, metaData, rec) {
let text = Ext.htmlEncode(value);
// only multipath mapped devices report their path count
let paths = rec.get('paths-total');
if (paths) {
- text += ` (${paths} ${ngettext('path', 'paths', paths)})`;
+ let active = rec.get('paths-active');
+ if (Ext.isNumeric(active) && active < paths) {
+ let qtip = Ext.htmlEncode(
+ gettext('Some paths failed, the multipath device is degraded'),
+ );
+ text =
+ `<i data-qtip="${qtip}" class="fa fa-exclamation-triangle warning"></i> ` +
+ `${text} (${active}/${paths} ${gettext('paths')})`;
+ } else {
+ text += ` (${paths} ${ngettext('path', 'paths', paths)})`;
+ }
}
return text;
},
diff --git a/www/manager6/node/SANLuns.js b/www/manager6/node/SANLuns.js
index 593bfc519..fe30125d8 100644
--- a/www/manager6/node/SANLuns.js
+++ b/www/manager6/node/SANLuns.js
@@ -28,6 +28,73 @@ Ext.define('PVE.node.SANLunList', {
trackOver: false,
},
+ plugins: [
+ {
+ ptype: 'rowexpander',
+ expandOnDblClick: false,
+ scrollIntoViewOnExpand: false,
+ rowBodyTpl: [
+ '{[this.renderDetails(values)]}',
+ {
+ renderDetails: function (values) {
+ let enc = Ext.htmlEncode;
+ let lines = [];
+ if (values.wwn) {
+ lines.push(`<b>WWN:</b> ${enc(values.wwn)}`);
+ }
+ if (values.serial) {
+ lines.push(`<b>${gettext('Serial')}:</b> ${enc(values.serial)}`);
+ }
+ let faults = values['path-faults'];
+ if (Ext.isNumeric(faults)) {
+ lines.push(
+ `<b>${gettext('Path Failures Since Map Creation')}:</b> ${enc(faults)}`,
+ );
+ }
+ let paths = values['path-list'] || [];
+ if (paths.length) {
+ let goodStates = { active: 1, live: 1 };
+ let goodAna = { optimized: 1, 'non-optimized': 1 };
+ let columns = [
+ { key: 'device', label: gettext('Path') },
+ {
+ key: 'state',
+ label: gettext('State'),
+ bad: (path) => !goodStates[path.state],
+ },
+ { key: 'checker-state', label: gettext('Checker') },
+ {
+ key: 'ana-state',
+ label: 'ANA',
+ bad: (path) => !goodAna[path['ana-state']],
+ },
+ { key: 'host-wwpn', label: 'Host WWPN' },
+ { key: 'target-wwpn', label: 'Target WWPN' },
+ { key: 'address', label: gettext('Address') },
+ ].filter((col) => paths.some((path) => path[col.key] !== undefined));
+ let rows = paths.map((path) =>
+ columns.map((col) => {
+ let text = enc(path[col.key] ?? '');
+ if (text !== '' && col.bad && col.bad(path)) {
+ text = `<i class="fa fa-exclamation-triangle warning"></i> ${text}`;
+ }
+ return text;
+ }),
+ );
+ lines.push(
+ PVE.SANUtils.render_detail_table(
+ columns.map((col) => col.label),
+ rows,
+ ),
+ );
+ }
+ return lines.join('<br>');
+ },
+ },
+ ],
+ },
+ ],
+
columns: [
{
header: gettext('Device'),
@@ -109,7 +176,14 @@ Ext.define('PVE.node.SANLunList', {
});
let store = Ext.create('Ext.data.Store', {
- fields: [...PVE.SANUtils.lun_fields, 'transport', 'serial', 'wwn'],
+ fields: [
+ ...PVE.SANUtils.lun_fields,
+ 'transport',
+ 'serial',
+ 'wwn',
+ 'path-faults',
+ 'path-list',
+ ],
proxy: {
type: 'proxmox',
url: `/api2/json/nodes/${me.nodename}/scan/san-luns`,
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 59/76] api: nodes: expose iSCSI status and maintenance
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (57 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 58/76] ui: san luns: show multipath path state Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 60/76] pvenode: add iscsi commands Thomas Lamprecht
` (16 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Mount the node-level iSCSI API from libpve-storage-perl at
/nodes/{node}/iscsi, so clients can inspect and maintain the targets and
sessions of a node, next to the other storage-backed node endpoints.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: reword the commit message]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
PVE/API2/Nodes.pm | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index df0b2c2d9..ed97205ac 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -50,6 +50,7 @@ use PVE::API2::Certificates;
use PVE::API2::Disks;
use PVE::API2::Firewall::Host;
use PVE::API2::Hardware;
+use PVE::API2::ISCSI;
use PVE::API2::LXC::Status;
use PVE::API2::LXC;
use PVE::API2::Network;
@@ -160,6 +161,11 @@ __PACKAGE__->register_method({
path => 'hardware',
});
+__PACKAGE__->register_method({
+ subclass => "PVE::API2::ISCSI",
+ path => 'iscsi',
+});
+
__PACKAGE__->register_method({
subclass => "PVE::API2::Capabilities",
path => 'capabilities',
@@ -240,6 +246,7 @@ __PACKAGE__->register_method({
{ name => 'firewall' },
{ name => 'hardware' },
{ name => 'hosts' },
+ { name => 'iscsi' },
{ name => 'journal' },
{ name => 'lxc' },
{ name => 'migrateall' },
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 60/76] pvenode: add iscsi commands
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (58 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 59/76] api: nodes: expose iSCSI status and maintenance Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 61/76] ui: node: san: add iSCSI target and session management Thomas Lamprecht
` (15 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
Expose the iSCSI initiator API on the command line, so target and
session state, negotiated parameters, rescan, logout and node record
removal are reachable without the web interface or raw pvesh calls.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: group the commands by object: 'pvenode iscsi target list|remove'
and 'pvenode iscsi session params|logout|rescan'; show the target
first in the target list]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
PVE/CLI/pvenode.pm | 48 ++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 48 insertions(+)
diff --git a/PVE/CLI/pvenode.pm b/PVE/CLI/pvenode.pm
index 7f7176422..0f382e1f5 100644
--- a/PVE/CLI/pvenode.pm
+++ b/PVE/CLI/pvenode.pm
@@ -7,6 +7,7 @@ use PVE::API2::ACME;
use PVE::API2::ACMEAccount;
use PVE::API2::ACMEPlugin;
use PVE::API2::Certificates;
+use PVE::API2::ISCSI;
use PVE::API2::NodeConfig;
use PVE::API2::Nodes;
use PVE::API2::Tasks;
@@ -370,6 +371,53 @@ our $cmddef = {
},
],
+ iscsi => {
+ status => [
+ 'PVE::API2::ISCSI',
+ 'status',
+ [],
+ { node => $nodename },
+ sub {
+ my ($data, $schema, $options) = @_;
+ PVE::CLIFormatter::print_api_result($data, $schema, undef, $options);
+ },
+ $PVE::RESTHandler::standard_output_options,
+ ],
+ target => {
+ list => [
+ 'PVE::API2::ISCSI',
+ 'targets',
+ [],
+ { node => $nodename },
+ sub {
+ my ($data, $schema, $options) = @_;
+ my $columns = [qw(
+ target portal session-id transport health kernel-state used-by-storage
+ )];
+ PVE::CLIFormatter::print_api_result($data, $schema, $columns, $options);
+ },
+ $PVE::RESTHandler::standard_output_options,
+ ],
+ remove => ['PVE::API2::ISCSI', 'target_delete', ['target'], { node => $nodename }],
+ },
+ session => {
+ params => [
+ 'PVE::API2::ISCSI',
+ 'session_params',
+ ['session-id'],
+ { node => $nodename },
+ sub {
+ my ($data, $schema, $options) = @_;
+ PVE::CLIFormatter::print_api_result($data, $schema, undef, $options);
+ },
+ $PVE::RESTHandler::standard_output_options,
+ ],
+ logout =>
+ ['PVE::API2::ISCSI', 'session_logout', ['session-id'], { node => $nodename }],
+ rescan => ['PVE::API2::ISCSI', 'rescan', ['session-id'], { node => $nodename }],
+ },
+ },
+
};
1;
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 61/76] ui: node: san: add iSCSI target and session management
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (59 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 60/76] pvenode: add iscsi commands Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 62/76] pvestatd: apply the multipath configuration and report the node status Thomas Lamprecht
` (14 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
From: Dietmar Maurer <dietmar@proxmox.com>
The LUN inventory cannot explain iSCSI connections. Add a Node > SAN
child view for targets, portals, sessions, and initiator node records.
Keep disconnected targets visible and preserve selection and expansion
across reloads. Include the initiator IQN for array access rules and
negotiated session parameters.
Offer rescan, logout, and removal of logged-out records with Sys.Modify
on "/", explaining when configured storage will reconnect. Show a
missing-package message rather than hiding the view without open-iscsi.
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: adapt the message to the separate SAN child view and
missing-package explanation; distinguish recovering and unknown
sessions, mask the initial listing, report copy failures, explain
storage-owned logins in the logout confirmation, bounded to storages
visible to the user, encode target and initiator names and keep the
session-only renderers with the view]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Makefile | 1 +
www/manager6/SANUtils.js | 18 +
www/manager6/Utils.js | 22 ++
www/manager6/node/Config.js | 23 +-
www/manager6/node/ISCSITargets.js | 615 ++++++++++++++++++++++++++++++
5 files changed, 672 insertions(+), 7 deletions(-)
create mode 100644 www/manager6/node/ISCSITargets.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 2e1b23fed..3b8780790 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -245,6 +245,7 @@ JSSRC= \
node/CmdMenu.js \
node/Config.js \
node/Directory.js \
+ node/ISCSITargets.js \
node/LVM.js \
node/LVMThin.js \
node/SANLuns.js \
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index 25c733263..b48ad969d 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -194,4 +194,22 @@ Ext.define('PVE.SANUtils', {
}[value] ?? Ext.htmlEncode(value)
);
},
+
+ render_san_connection_state: function (value) {
+ if (!value) {
+ return '';
+ }
+ let display = {
+ connected: { icon: 'good fa-check-circle', label: gettext('Connected') },
+ reconnecting: {
+ icon: 'warning fa-exclamation-circle',
+ label: gettext('Reconnecting'),
+ },
+ 'not-connected': { icon: 'faded fa-times-circle', label: gettext('Not connected') },
+ unknown: { icon: 'faded fa-question-circle', label: gettext('Unknown') },
+ }[value];
+ let icon = display?.icon ?? 'faded fa-question-circle';
+ let label = display?.label ?? value;
+ return `<i class="fa fa-fw ${icon}"></i> ${Ext.htmlEncode(label)}`;
+ },
});
diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index bc5171712..68bb7b913 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -1122,6 +1122,28 @@ Ext.define('PVE.Utils', {
return Ext.String.htmlEncode(result);
},
+ copyToClipboard: function (text) {
+ let fail = (err) =>
+ Ext.Msg.alert(
+ gettext('Error'),
+ Ext.htmlEncode(
+ Ext.String.format(
+ gettext('Copying to the clipboard failed: {0}'),
+ `${err}`,
+ ),
+ ),
+ );
+ // navigator.clipboard only exists in a secure context
+ if (!navigator.clipboard?.writeText) {
+ Ext.Msg.alert(
+ gettext('Error'),
+ gettext('The browser does not allow copying to the clipboard on this page.'),
+ );
+ return;
+ }
+ navigator.clipboard.writeText(text).catch(fail);
+ },
+
render_serverity: function (value) {
return PVE.Utils.log_severity_hash[value] || value;
},
diff --git a/www/manager6/node/Config.js b/www/manager6/node/Config.js
index 3a84e96f6..d4a292d68 100644
--- a/www/manager6/node/Config.js
+++ b/www/manager6/node/Config.js
@@ -360,14 +360,23 @@ Ext.define('PVE.node.Config', {
);
}
- // the san-luns scan accepts either privilege
+ // the san-luns and iscsi reads accept either privilege
if (caps.nodes['Sys.Audit'] || caps.storage['Datastore.Allocate']) {
- me.items.push({
- xtype: 'pveSANLunList',
- title: 'SAN',
- itemId: 'san',
- iconCls: 'fa fa-cubes',
- });
+ me.items.push(
+ {
+ xtype: 'pveSANLunList',
+ title: 'SAN',
+ itemId: 'san',
+ iconCls: 'fa fa-cubes',
+ },
+ {
+ xtype: 'pveISCSITargetTree',
+ title: 'iSCSI',
+ itemId: 'san-iscsi',
+ iconCls: 'fa fa-bullseye',
+ groups: ['san'],
+ },
+ );
}
if (caps.nodes['Sys.Audit']) {
diff --git a/www/manager6/node/ISCSITargets.js b/www/manager6/node/ISCSITargets.js
new file mode 100644
index 000000000..a5621ba98
--- /dev/null
+++ b/www/manager6/node/ISCSITargets.js
@@ -0,0 +1,615 @@
+Ext.define('PVE.node.ISCSISessionParams', {
+ extend: 'Proxmox.grid.ObjectGrid',
+ xtype: 'pveISCSISessionParams',
+
+ viewConfig: {
+ enableTextSelection: true,
+ },
+
+ statics: {
+ renderSessionState: function (value) {
+ let state =
+ {
+ 'logged-in': 'connected',
+ recovering: 'reconnecting',
+ 'not-connected': 'not-connected',
+ }[value] ?? (value ? 'unknown' : undefined);
+ return PVE.SANUtils.render_san_connection_state(state);
+ },
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ if (!me.nodename) {
+ throw 'no node name specified';
+ }
+ if (!me.sessionData) {
+ throw 'no session specified';
+ }
+
+ let session = me.sessionData;
+ let seconds = (value) => (value === undefined || value === null ? '' : `${value} s`);
+
+ me.url = `/api2/json/nodes/${me.nodename}/iscsi/session-params`;
+ me.extraParams = { 'session-id': session['session-id'] };
+
+ // identity rows from the list record, shown before the load completes
+ me.rows = {
+ target: { header: gettext('Target'), defaultValue: session.target },
+ 'session-id': {
+ header: gettext('Session ID'),
+ defaultValue: session['session-id'],
+ },
+ portal: { header: gettext('Portal'), defaultValue: session.portal },
+ transport: {
+ header: gettext('Transport'),
+ defaultValue: session.transport,
+ renderer: PVE.SANUtils.render_san_transport,
+ },
+ health: {
+ header: gettext('Status'),
+ defaultValue: session.health,
+ renderer: PVE.node.ISCSISessionParams.renderSessionState,
+ },
+ 'kernel-state': {
+ header: gettext('Kernel State'),
+ defaultValue: session['kernel-state'],
+ },
+ 'header-digest': { header: gettext('Header Digest') },
+ 'data-digest': { header: gettext('Data Digest') },
+ 'max-recv-data-segment-length': {
+ header: gettext('Max Receive Segment'),
+ renderer: Proxmox.Utils.render_size,
+ },
+ 'max-xmit-data-segment-length': {
+ header: gettext('Max Transmit Segment'),
+ renderer: Proxmox.Utils.render_size,
+ },
+ 'first-burst-length': {
+ header: gettext('First Burst Length'),
+ renderer: Proxmox.Utils.render_size,
+ },
+ 'max-burst-length': {
+ header: gettext('Max Burst Length'),
+ renderer: Proxmox.Utils.render_size,
+ },
+ 'max-outstanding-r2t': { header: gettext('Max Outstanding R2T') },
+ 'immediate-data': { header: gettext('Immediate Data') },
+ 'initial-r2t': { header: gettext('Initial R2T') },
+ 'recovery-timeout': { header: gettext('Recovery Timeout'), renderer: seconds },
+ 'abort-timeout': { header: gettext('Abort Timeout'), renderer: seconds },
+ 'lu-reset-timeout': { header: gettext('LUN Reset Timeout'), renderer: seconds },
+ 'tgt-reset-timeout': { header: gettext('Target Reset Timeout'), renderer: seconds },
+ };
+
+ me.callParent();
+
+ me.rstore.load();
+ },
+});
+
+Ext.define('PVE.node.ISCSITargetTree', {
+ extend: 'Ext.tree.Panel',
+ xtype: 'pveISCSITargetTree',
+
+ stateful: true,
+ stateId: 'grid-node-iscsi-target',
+
+ onlineHelp: 'storage_open_iscsi',
+
+ rootVisible: false,
+
+ viewConfig: {
+ emptyText: gettext('No iSCSI targets found'),
+ },
+
+ selModel: { allowDeselect: true },
+
+ viewModel: {
+ data: {
+ initiatorName: '',
+ initiatorText: '',
+ },
+ },
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ buildChildren: function (rows, collapsedTargets) {
+ let targets = {};
+ for (const row of rows) {
+ targets[row.target] ??= [];
+ targets[row.target].push(row);
+ }
+ return Object.keys(targets)
+ .sort()
+ .map((target) => {
+ let portals = targets[target];
+ // Failed and free sessions still have IDs, so derive connectivity from health.
+ let countHealth = (health) => portals.filter((p) => p.health === health).length;
+ let connected = countHealth('logged-in');
+ let sessions = portals.filter((p) => p['session-id'] !== undefined).length;
+ return {
+ text: target,
+ target: target,
+ isTarget: true,
+ iconCls: 'fa fa-bullseye',
+ expanded: !collapsedTargets[target],
+ connectedCount: connected,
+ recoveringCount: countHealth('recovering'),
+ unknownCount: countHealth('unknown'),
+ sessionCount: sessions,
+ totalPortals: portals.length,
+ 'used-by-storage': portals[0]['used-by-storage'],
+ children: portals.map((p) => ({
+ text: p.portal,
+ isTarget: false,
+ leaf: true,
+ iconCls: 'fa fa-link',
+ target: p.target,
+ portal: p.portal,
+ 'session-id': p['session-id'],
+ transport: p.transport,
+ health: p.health,
+ 'kernel-state': p['kernel-state'],
+ 'used-by-storage': p['used-by-storage'],
+ })),
+ };
+ });
+ },
+
+ // the selection key survives the tree rebuild on reload
+ nodeKey: function (node) {
+ return node.get('isTarget')
+ ? `target:${node.get('target')}`
+ : `portal:${node.get('target')}\0${node.get('portal')}`;
+ },
+
+ // until the first listing, keep the load mask instead of the empty text
+ dataPending: true,
+
+ reloadData: function () {
+ let me = this;
+ let view = me.getView();
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${view.nodename}/iscsi/target`,
+ method: 'GET',
+ waitMsgTarget: view,
+ failure: function (response) {
+ me.dataPending = false;
+ Proxmox.Utils.setErrorMask(view, response.htmlStatus);
+ },
+ success: function (response) {
+ me.dataPending = false;
+ Proxmox.Utils.setErrorMask(view, false);
+ let prev = view.getSelection()[0];
+ let key = prev ? me.nodeKey(prev) : undefined;
+ let collapsedTargets = {};
+ view.getRootNode().eachChild((n) => {
+ if (!n.isExpanded()) {
+ collapsedTargets[n.get('target')] = true;
+ }
+ });
+ view.setRootNode({
+ expanded: true,
+ children: me.buildChildren(response.result.data, collapsedTargets),
+ });
+ if (key) {
+ let node = view
+ .getRootNode()
+ .findChildBy((n) => me.nodeKey(n) === key, me, true);
+ if (node) {
+ view.getSelectionModel().select(node);
+ }
+ }
+ },
+ });
+ },
+
+ onReload: function () {
+ this.getView().reload();
+ },
+
+ onRescan: function () {
+ let me = this;
+ let view = me.getView();
+ let rec = view.getSelection()[0];
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${view.nodename}/iscsi/rescan`,
+ method: 'POST',
+ params: { 'session-id': rec.get('session-id') },
+ waitMsgTarget: view,
+ failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
+ success: () => view.reload(),
+ });
+ },
+
+ // the storage that logs in again on its next activation; unknown if the list cannot be
+ // read
+ withReloginStorage: function (target, callback) {
+ let view = this.getView();
+ Proxmox.Utils.API2Request({
+ url: '/storage',
+ method: 'GET',
+ params: { type: 'iscsi' },
+ failure: () => callback(undefined, true),
+ success: function (response) {
+ let entry = (response.result.data || []).find(
+ (scfg) =>
+ scfg.target === target &&
+ !scfg.disable &&
+ (!scfg.nodes || `${scfg.nodes}`.split(',').includes(view.nodename)),
+ );
+ callback(entry?.storage, false);
+ },
+ });
+ },
+
+ onLogout: function () {
+ let me = this;
+ let view = me.getView();
+ let rec = view.getSelection()[0];
+ me.withReloginStorage(rec.get('target'), function (storage, unknown) {
+ let msg = Ext.htmlEncode(
+ Ext.String.format(
+ gettext('Log out of the session on portal "{0}"?'),
+ rec.get('portal'),
+ ),
+ );
+ if (storage) {
+ msg += `<br><br>${Ext.htmlEncode(
+ Ext.String.format(
+ gettext('Storage "{0}" logs in again within seconds.'),
+ storage,
+ ),
+ )}`;
+ } else if (!unknown) {
+ msg += `<br><br>${Ext.htmlEncode(gettext('No storage you can access will log in to this target again.'))}`;
+ }
+ Ext.Msg.confirm(gettext('Confirm'), msg, function (btn) {
+ if (btn !== 'yes') {
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${view.nodename}/iscsi/session-logout`,
+ method: 'POST',
+ params: { 'session-id': rec.get('session-id') },
+ waitMsgTarget: view,
+ failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
+ success: () => view.reload(),
+ });
+ });
+ });
+ },
+
+ onRemove: function () {
+ let me = this;
+ let view = me.getView();
+ let rec = view.getSelection()[0];
+ let params = { target: rec.get('target') };
+ let msg;
+ if (rec.get('isTarget')) {
+ msg = Ext.String.format(
+ gettext('Remove all records of target "{0}"?'),
+ rec.get('target'),
+ );
+ } else {
+ params.portal = rec.get('portal');
+ msg = Ext.String.format(
+ gettext('Remove the record of target "{0}", portal "{1}"?'),
+ rec.get('target'),
+ rec.get('portal'),
+ );
+ }
+ let target = rec.get('isTarget') ? rec : rec.parentNode;
+ let sessionsLeft = !!target?.get('sessionCount');
+ me.withReloginStorage(rec.get('target'), function (storage) {
+ let text = Ext.htmlEncode(msg);
+ if (storage) {
+ let note = sessionsLeft
+ ? gettext(
+ 'Storage "{0}" uses this target. The record comes back with its next discovery, for example after a reboot.',
+ )
+ : gettext(
+ 'Storage "{0}" uses this target and discovers it again within seconds.',
+ );
+ text += `<br><br>${Ext.htmlEncode(Ext.String.format(note, storage))}`;
+ }
+ Ext.Msg.confirm(gettext('Confirm'), text, function (btn) {
+ if (btn !== 'yes') {
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${view.nodename}/iscsi/target?${Ext.Object.toQueryString(params)}`,
+ method: 'DELETE',
+ waitMsgTarget: view,
+ failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
+ success: () => view.reload(),
+ });
+ });
+ });
+ },
+
+ showInfo: function (rec) {
+ let view = this.getView();
+ if (rec.get('session-id') === undefined) {
+ return;
+ }
+ Ext.create('Ext.window.Window', {
+ title: gettext('Session Parameters'),
+ modal: true,
+ width: 500,
+ height: 600,
+ resizable: true,
+ layout: 'fit',
+ items: [
+ {
+ xtype: 'pveISCSISessionParams',
+ nodename: view.nodename,
+ sessionData: rec.data,
+ },
+ ],
+ }).show();
+ },
+
+ onInfo: function () {
+ this.showInfo(this.getView().getSelection()[0]);
+ },
+
+ onItemDblClick: function (tree, rec) {
+ this.showInfo(rec);
+ },
+
+ onCopyInitiator: function () {
+ let name = this.getViewModel().get('initiatorName');
+ if (name) {
+ PVE.Utils.copyToClipboard(name);
+ }
+ },
+
+ setInitiator: function (status) {
+ let me = this;
+ let view = me.getView();
+ if (!status.supported) {
+ me.dataPending = false;
+ view.setLoading(false);
+ Proxmox.Utils.setErrorMask(
+ view,
+ gettext('open-iscsi is not installed on this node.'),
+ );
+ return;
+ }
+ Proxmox.Utils.setErrorMask(view, false);
+ let name = status['initiator-name'];
+ let text = name || Proxmox.Utils.unknownText;
+ if (status['initiator-alias']) {
+ text += ` (${status['initiator-alias']})`;
+ }
+ me.getViewModel().set({
+ initiatorName: name || '',
+ initiatorText: text,
+ });
+ view.iscsiSupported = true;
+ view.reload();
+ },
+
+ loadStatus: function () {
+ let me = this;
+ let view = me.getView();
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${view.nodename}/iscsi/status`,
+ method: 'GET',
+ failure: function (response) {
+ me.dataPending = false;
+ view.setLoading(false);
+ Proxmox.Utils.setErrorMask(view, response.htmlStatus);
+ },
+ success: (response) => me.setInitiator(response.result.data),
+ });
+ },
+
+ init: function (view) {
+ let me = this;
+ if (!view.nodename) {
+ throw 'no node name specified';
+ }
+ view.on(
+ 'afterrender',
+ function () {
+ if (me.dataPending) {
+ view.setLoading(true);
+ }
+ },
+ me,
+ { single: true },
+ );
+ me.loadStatus();
+ },
+ },
+
+ store: {
+ type: 'tree',
+ fields: [
+ 'text',
+ 'isTarget',
+ 'connectedCount',
+ 'recoveringCount',
+ 'unknownCount',
+ 'sessionCount',
+ 'totalPortals',
+ 'target',
+ 'portal',
+ 'session-id',
+ 'transport',
+ 'health',
+ 'kernel-state',
+ 'used-by-storage',
+ ],
+ root: { expanded: true, children: [] },
+ },
+
+ // list only once the status confirms open-iscsi, until then ask for the status again
+ reload: function () {
+ if (this.iscsiSupported) {
+ this.getController().reloadData();
+ } else {
+ this.getController().loadStatus();
+ }
+ },
+
+ listeners: {
+ activate: 'onReload',
+ itemdblclick: 'onItemDblClick',
+ },
+
+ columns: [
+ {
+ xtype: 'treecolumn',
+ text: gettext('Target'),
+ dataIndex: 'text',
+ flex: 3,
+ renderer: Ext.htmlEncode,
+ },
+ {
+ text: gettext('Session ID'),
+ dataIndex: 'session-id',
+ width: 110,
+ hidden: true,
+ },
+ {
+ text: gettext('Transport'),
+ dataIndex: 'transport',
+ width: 100,
+ renderer: PVE.SANUtils.render_san_transport,
+ },
+ {
+ text: gettext('Status'),
+ dataIndex: 'health',
+ width: 150,
+ // rank by what it means for the LUNs: reconnecting, partly connected, unknown, not
+ // connected
+ renderer: function (value, metaData, rec) {
+ if (!rec.get('isTarget')) {
+ return PVE.node.ISCSISessionParams.renderSessionState(value);
+ }
+ let connected = rec.get('connectedCount');
+ let total = rec.get('totalPortals');
+ let icon = (cls) => `<i class="fa fa-fw ${cls}"></i> `;
+ if (rec.get('recoveringCount')) {
+ let label = connected
+ ? Ext.String.format(
+ gettext('{0}/{1} connected, reconnecting'),
+ connected,
+ total,
+ )
+ : gettext('Reconnecting');
+ return icon('warning fa-exclamation-circle') + label;
+ }
+ if (connected === total) {
+ return PVE.SANUtils.render_san_connection_state('connected');
+ }
+ if (connected > 0) {
+ return (
+ icon('warning fa-exclamation-circle') +
+ Ext.String.format(gettext('{0}/{1} connected'), connected, total)
+ );
+ }
+ if (rec.get('unknownCount')) {
+ return PVE.SANUtils.render_san_connection_state('unknown');
+ }
+ return PVE.SANUtils.render_san_connection_state('not-connected');
+ },
+ },
+ {
+ text: gettext('Storage'),
+ dataIndex: 'used-by-storage',
+ flex: 1,
+ renderer: (value, metaData, rec) =>
+ rec.get('isTarget') ? Ext.htmlEncode(value ?? '') : '',
+ },
+ ],
+
+ initComponent: function () {
+ let me = this;
+
+ me.nodename ??= me.pveSelNode?.data?.node;
+
+ let caps = Ext.state.Manager.get('GuiCap');
+ let hasSession = (rec) => !rec.get('isTarget') && rec.get('session-id') !== undefined;
+
+ let items = [
+ {
+ text: gettext('Reload'),
+ iconCls: 'fa fa-refresh',
+ handler: 'onReload',
+ },
+ ];
+ if (caps.dc['Sys.Modify']) {
+ items.push(
+ {
+ xtype: 'proxmoxButton',
+ text: gettext('Rescan'),
+ iconCls: 'fa fa-search',
+ disabled: true,
+ enableFn: hasSession,
+ handler: 'onRescan',
+ },
+ {
+ xtype: 'proxmoxButton',
+ text: gettext('Logout'),
+ iconCls: 'fa fa-sign-out',
+ disabled: true,
+ dangerous: true,
+ enableFn: hasSession,
+ handler: 'onLogout',
+ },
+ {
+ xtype: 'proxmoxButton',
+ text: gettext('Remove'),
+ iconCls: 'fa fa-trash-o',
+ disabled: true,
+ dangerous: true,
+ enableFn: (rec) =>
+ rec.get('isTarget') ? rec.get('sessionCount') === 0 : !hasSession(rec),
+ handler: 'onRemove',
+ },
+ );
+ }
+ items.push(
+ {
+ xtype: 'proxmoxButton',
+ text: gettext('Details'),
+ iconCls: 'fa fa-info-circle',
+ disabled: true,
+ enableFn: hasSession,
+ handler: 'onInfo',
+ },
+ '->',
+ {
+ xtype: 'displayfield',
+ fieldLabel: gettext('Initiator Name'),
+ labelWidth: 100,
+ htmlEncode: true,
+ bind: {
+ value: '{initiatorText}',
+ },
+ },
+ {
+ xtype: 'button',
+ iconCls: 'fa fa-clipboard',
+ tooltip: gettext('Copy to Clipboard'),
+ handler: 'onCopyInitiator',
+ bind: {
+ disabled: '{!initiatorName}',
+ },
+ },
+ );
+ me.tbar = {
+ defaults: { parentXType: 'treepanel' },
+ items: items,
+ };
+
+ me.callParent();
+ },
+});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 62/76] pvestatd: apply the multipath configuration and report the node status
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (60 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 61/76] ui: node: san: add iSCSI target and session management Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 63/76] api: cluster: add the multipath endpoints Thomas Lamprecht
` (13 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Use pvestatd's periodic loop to apply multipath configuration and
publish node health, including after reboots and storage changes. Run
this before node, guest, and storage status updates so a slow update
does not delay the first report of a path outage.
Nodes without multipath-tools send only a small report, resent once a
minute while unchanged. With the tools, daemon availability is checked
on every pass; without managed LUNs, a successful map listing is
reused for a minute. Log configuration errors when they first occur,
not on every pass.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
PVE/Service/pvestatd.pm | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/PVE/Service/pvestatd.pm b/PVE/Service/pvestatd.pm
index f682c2f9d..4218db2e7 100755
--- a/PVE/Service/pvestatd.pm
+++ b/PVE/Service/pvestatd.pm
@@ -21,6 +21,8 @@ use PVE::NodeConfig;
use PVE::Cluster qw(cfs_read_file);
use PVE::Storage;
use PVE::Storage::StatusCommand;
+use PVE::Multipath::Generator;
+use PVE::Multipath::Status;
use PVE::QemuServer;
use PVE::QemuServer::Monitor;
use PVE::LXC;
@@ -808,6 +810,17 @@ my sub broadcast_version_info : prototype() {
}
}
+my $last_multipath_sync_error = '';
+my sub update_multipath_status : prototype() {
+ my $multipath = eval { PVE::Multipath::Generator::sync(); };
+ $multipath = { error => $@ } if $@;
+ my $sync_error = $multipath->{error} // '';
+ syslog('err', "multipath config sync error: $sync_error")
+ if $sync_error ne '' && $sync_error ne $last_multipath_sync_error;
+ $last_multipath_sync_error = $sync_error;
+ PVE::Multipath::Status::broadcast_status($multipath);
+}
+
sub update_status {
# update worker list. This is not really required, but we want to make sure that we also have a
# correct list in case of an unexpected crash.
@@ -824,6 +837,12 @@ sub update_status {
my $status_cfg = PVE::Cluster::cfs_read_file('status.cfg');
+ # apply the multipath configuration before the other updates: the storage status update can
+ # block in LVM while all paths of a LUN are lost
+ eval { update_multipath_status(); };
+ $err = $@;
+ syslog('err', "multipath status broadcast error: $err") if $err;
+
eval { update_node_status($status_cfg, $pull_txn); };
$err = $@;
syslog('err', "node status update error: $err") if $err;
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 63/76] api: cluster: add the multipath endpoints
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (61 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 62/76] pvestatd: apply the multipath configuration and report the node status Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 64/76] ui: dc: add a cluster-wide multipath view Thomas Lamprecht
` (12 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Mount the cluster-wide multipath configuration and health status under
/cluster/multipath.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
PVE/API2/Cluster.pm | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index 4e5efbfd9..576840181 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -35,6 +35,7 @@ use PVE::API2::Cluster::Qemu;
use PVE::API2::ClusterConfig;
use PVE::API2::Firewall::Cluster;
use PVE::API2::HAConfig;
+use PVE::API2::Multipath;
use PVE::API2::ReplicationConfig;
my $have_sdn;
@@ -100,6 +101,11 @@ __PACKAGE__->register_method({
path => 'ceph',
});
+__PACKAGE__->register_method({
+ subclass => "PVE::API2::Multipath",
+ path => 'multipath',
+});
+
__PACKAGE__->register_method({
subclass => "PVE::API2::Cluster::Jobs",
path => 'jobs',
@@ -169,6 +175,7 @@ __PACKAGE__->register_method({
{ name => 'log' },
{ name => 'mapping' },
{ name => 'metrics' },
+ { name => 'multipath' },
{ name => 'notifications' },
{ name => 'nextid' },
{ name => 'options' },
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 64/76] ui: dc: add a cluster-wide multipath view
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (62 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 63/76] api: cluster: add the multipath endpoints Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 65/76] ui: storage: iscsi: add the multipath option Thomas Lamprecht
` (11 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Administrators need to see which nodes manage each LUN and whether its
configured paths are usable. Add the cluster-wide multipath overview,
entry editor and node details, including the time of each node report.
Keep health counts visible with faults first and distinguish the
multipath service from the health of the maps it manages. Show
configuration failures and missing reports rather than presenting them
as healthy paths. An optimal LUN with a node in an unknown state is
marked incomplete, as the aggregate leaves that node out.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
test/Makefile | 6 +
test/SANUtils_test.js | 54 +++
www/manager6/Makefile | 4 +
www/manager6/SANUtils.js | 165 ++++++++
www/manager6/dc/Config.js | 28 +-
www/manager6/dc/Multipath.js | 547 +++++++++++++++++++++++++++
www/manager6/dc/MultipathWwidEdit.js | 183 +++++++++
www/manager6/dc/SAN.js | 326 ++++++++++++++++
www/manager6/panel/SANNotice.js | 26 ++
9 files changed, 1333 insertions(+), 6 deletions(-)
create mode 100644 test/SANUtils_test.js
create mode 100644 www/manager6/dc/Multipath.js
create mode 100644 www/manager6/dc/MultipathWwidEdit.js
create mode 100644 www/manager6/dc/SAN.js
create mode 100644 www/manager6/panel/SANNotice.js
diff --git a/test/Makefile b/test/Makefile
index 2dfd20a41..d7521c0b3 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -7,6 +7,12 @@ export PERLLIB=..
.PHONY: check
check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-pvesh
+check: test-san-utils
+
+.PHONY: test-san-utils
+test-san-utils:
+ node --test SANUtils_test.js
+
.PHONY: test-balloon
test-balloon:
./balloontest.pl
diff --git a/test/SANUtils_test.js b/test/SANUtils_test.js
new file mode 100644
index 000000000..d22307664
--- /dev/null
+++ b/test/SANUtils_test.js
@@ -0,0 +1,54 @@
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const path = require('node:path');
+const vm = require('node:vm');
+const { test } = require('node:test');
+
+// load PVE.SANUtils with stubs of the ExtJS and translation helpers it uses
+function sanUtils(translations = {}) {
+ const sandbox = {
+ PVE: {},
+ gettext: (text) => translations[text] ?? text,
+ Ext: {
+ define: (name, config) => {
+ sandbox.PVE.SANUtils = config;
+ },
+ htmlEncode: (text) =>
+ String(text).replace(/&/g, '&').replace(/"/g, '"').replace(/</g, '<'),
+ String: {
+ format: (text, ...args) => text.replace(/\{(\d+)\}/g, (_, index) => args[index]),
+ },
+ },
+ };
+ vm.createContext(sandbox);
+ const source = path.join(__dirname, '../www/manager6/SANUtils.js');
+ vm.runInContext(fs.readFileSync(source, 'utf8'), sandbox, { filename: source });
+ return sandbox.PVE.SANUtils;
+}
+
+test('cluster health of a LUN', () => {
+ const utils = sanUtils({ 'Optimal (incomplete)': 'Optimal (unvollständig)' });
+ const render = (value, nodes, extra = {}) => {
+ const metaData = {};
+ const html = utils.render_multipath_cluster_health(value, metaData, { nodes, ...extra });
+ return { html, tip: metaData.tdAttr };
+ };
+ const optimal = { state: 'optimal' };
+
+ let res = render('optimal', { n1: optimal, n2: optimal });
+ assert.match(res.html, /> Optimal$/);
+ assert.equal(res.tip, undefined, 'no tooltip while every node is known');
+
+ res = render('optimal', { n2: { state: 'unknown', reason: 'stale' }, n1: optimal, n3: {} });
+ assert.match(res.html, /check-circle good.*> Optimal \(unvollständig\)$/);
+ assert.equal(res.tip, 'data-qtip="State unknown on: n2, n3"', 'unknown nodes, sorted');
+
+ res = render('degraded', { n1: { state: 'degraded' }, n2: { state: 'unknown' } });
+ assert.match(res.html, /> Degraded$/, 'a worse known state needs no qualifier');
+
+ res = render('optimal', { n1: optimal, n2: { reason: 'not-derived' } });
+ assert.match(res.html, /> Optimal$/, 'a node that does not observe the LUN is not unknown');
+
+ res = render('missing', { n1: { state: 'unknown' } }, { 'not-observed': 1 });
+ assert.match(res.html, /> Not currently observed$/);
+});
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 3b8780790..252c5761a 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -119,6 +119,7 @@ JSSRC= \
panel/TemplateStatusView.js \
panel/MultiDiskEdit.js \
panel/TagConfig.js \
+ panel/SANNotice.js \
tree/ResourceTree.js \
tree/SnapshotTree.js \
tree/ResourceMapTree.js \
@@ -185,6 +186,8 @@ JSSRC= \
dc/Guests.js \
dc/Health.js \
dc/Log.js \
+ dc/Multipath.js \
+ dc/MultipathWwidEdit.js \
dc/NodeView.js \
dc/OptionView.js \
dc/PermissionView.js \
@@ -192,6 +195,7 @@ JSSRC= \
dc/PoolView.js \
dc/RoleEdit.js \
dc/RoleView.js \
+ dc/SAN.js \
dc/SecurityGroups.js \
dc/StorageView.js \
dc/Summary.js \
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index b48ad969d..c101ebb75 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -16,6 +16,138 @@ Ext.define('PVE.SANUtils', {
'by-id',
],
+ multipath_health_display: function (value) {
+ return (
+ {
+ optimal: { iconCls: 'check-circle good', label: gettext('Optimal') },
+ degraded: { iconCls: 'exclamation-circle warning', label: gettext('Degraded') },
+ missing: { iconCls: 'times-circle critical', label: gettext('Missing') },
+ failed: { iconCls: 'times-circle critical', label: gettext('Failed') },
+ unknown: { iconCls: 'question-circle faded', label: gettext('Unknown') },
+ 'not-observed': {
+ iconCls: 'eye-slash faded',
+ label: gettext('Not currently observed'),
+ },
+ }[value] ?? { iconCls: 'question-circle faded', label: Ext.htmlEncode(value ?? '') }
+ );
+ },
+
+ render_multipath_health: function (value) {
+ if (typeof value === 'undefined') {
+ return '';
+ }
+ let { iconCls, label } = PVE.SANUtils.multipath_health_display(value);
+ return `<i class="fa fa-${iconCls}"></i> ${label}`;
+ },
+
+ // The aggregate leaves out nodes without a current observation, so an optimal LUN is marked
+ // incomplete while its state on one of them is unknown.
+ render_multipath_cluster_health: function (value, metaData, data) {
+ if (data['not-observed']) {
+ return PVE.SANUtils.render_multipath_health('not-observed');
+ }
+ let nodes = data.nodes || {};
+ let unknown = Object.keys(nodes)
+ .filter((node) => PVE.SANUtils.multipath_count_state(nodes[node]) === 'unknown')
+ .sort();
+ if (value !== 'optimal' || !unknown.length) {
+ return PVE.SANUtils.render_multipath_health(value);
+ }
+ let tip = Ext.String.format(gettext('State unknown on: {0}'), unknown.join(', '));
+ if (metaData) {
+ metaData.tdAttr = `data-qtip="${Ext.htmlEncode(Ext.htmlEncode(tip))}"`;
+ }
+ let { iconCls } = PVE.SANUtils.multipath_health_display(value);
+ return `<i class="fa fa-${iconCls}"></i> ${gettext('Optimal (incomplete)')}`;
+ },
+
+ render_multipath_reason: function (value) {
+ if (!value) {
+ return '';
+ }
+ return (
+ {
+ offline: gettext('Node is offline'),
+ stale: gettext('Report is stale'),
+ 'not-running': gettext('multipathd is not running'),
+ unsupported: gettext('multipath-tools is not installed'),
+ 'query-error': gettext('Querying the multipath maps failed'),
+ 'summary-only': gettext('Too many LUNs to list, see the SAN view of the node'),
+ 'no-report': gettext('Not reporting, the node may run an older version'),
+ stalled: gettext('Status updates of the node are stalled'),
+ 'not-derived': gettext('The storage no longer finds the LUN on the target'),
+ }[value] ?? Ext.htmlEncode(value)
+ );
+ },
+
+ multipath_count_state: function (entry) {
+ if (entry?.reason === 'not-derived') {
+ return 'not-observed';
+ }
+ return entry?.state ?? 'unknown';
+ },
+
+ // the nodes on which missing multipath tools or a stopped multipathd are a problem
+ multipath_required_nodes: function (status) {
+ let required = {};
+ for (const node of status?.['required-nodes'] || []) {
+ required[node] = true;
+ }
+ for (const lun of status?.luns || []) {
+ Object.keys(lun.nodes || {}).forEach((node) => (required[node] = true));
+ }
+ Ext.Object.each(status?.nodes || {}, function (node, report) {
+ if (report?.summary?.luns || report?.['unresolved-luns']) {
+ required[node] = true;
+ }
+ });
+ return required;
+ },
+
+ render_san_state_counts: function (value) {
+ let counts = value?.counts;
+ if (!counts) {
+ return '';
+ }
+ let order = ['failed', 'missing', 'degraded', 'unknown', 'not-observed', 'optimal'];
+ let states = [
+ ...order.filter((state) => counts[state]),
+ ...Object.keys(counts).filter((state) => !order.includes(state) && counts[state]),
+ ];
+ if (!states.length) {
+ return gettext('None');
+ }
+ let text = states
+ .map(
+ (state) => `${counts[state]} ${PVE.SANUtils.multipath_health_display(state).label}`,
+ )
+ .join(', ');
+ if (value.summary) {
+ let qtip = Ext.htmlEncode(
+ gettext(
+ 'The node has too many LUNs to list and reports only the number of LUNs in each state.',
+ ),
+ );
+ text += ` <i class="fa fa-info-circle" data-qtip="${qtip}"></i>`;
+ }
+ return text;
+ },
+
+ count_states: function (entries, stateFn) {
+ let counts = {};
+ for (const entry of entries) {
+ let state = stateFn(entry);
+ counts[state] = (counts[state] ?? 0) + 1;
+ }
+ return counts;
+ },
+
+ // empty without nodes, where render_san_state_counts() says None
+ render_node_states: function (nodes, stateFn) {
+ let counts = PVE.SANUtils.count_states(Object.values(nodes || {}), stateFn);
+ return Object.keys(counts).length ? PVE.SANUtils.render_san_state_counts({ counts }) : '';
+ },
+
// headers and cells are HTML
render_detail_table: function (headers, rows) {
let header = headers.map((text) => `<th style="text-align: left;">${text}</th>`).join('');
@@ -23,6 +155,19 @@ Ext.define('PVE.SANUtils', {
return `<table style="border-spacing: 10px 2px;"><tr>${header}</tr>${body}</table>`;
},
+ // a stalled report keeps its problems but not its optimal LUNs
+ multipath_summary_counts: function (report) {
+ let { luns, ...counts } = report.summary;
+ if (report.reason) {
+ return { counts: { unknown: luns }, summary: true };
+ }
+ if (report.stalled && counts.optimal) {
+ counts.unknown = (counts.unknown ?? 0) + counts.optimal;
+ delete counts.optimal;
+ }
+ return { counts, summary: true };
+ },
+
render_san_lun_device: function (value, metaData, rec) {
let text = Ext.htmlEncode(value);
// only multipath mapped devices report their path count
@@ -98,10 +243,30 @@ Ext.define('PVE.SANUtils', {
}
},
+ // { text, degraded } of the usable paths, undefined without counts; an absent path counts
+ // against the expected count, without one (FC) only the found paths are known
+ format_san_paths: function (active, found, expected) {
+ if (!Ext.isNumeric(found) && !Ext.isNumeric(expected)) {
+ return undefined;
+ }
+ let total = Math.max(found || 0, expected || 0);
+ return {
+ text: `${Ext.isNumeric(active) ? active : '?'}/${total}`,
+ degraded:
+ (Ext.isNumeric(found) && found < total) ||
+ (Ext.isNumeric(active) && active < total),
+ };
+ },
+
request_uncertain: function (response) {
return response.timedout || response.status === 0 || response.result?.success === undefined;
},
+ // the extjs API format reports the HTTP status of a refused request as a string
+ request_denied: function (response) {
+ return Number(response?.result?.status) === 403;
+ },
+
render_san_lun_usage: function (value, metaData, rec) {
let text = PVE.SANUtils.san_lun_usage_text(rec.data);
metaData.tdAttr = `data-qtip="${Ext.htmlEncode(text)}"`;
diff --git a/www/manager6/dc/Config.js b/www/manager6/dc/Config.js
index 83c48c3d6..8a1e25205 100644
--- a/www/manager6/dc/Config.js
+++ b/www/manager6/dc/Config.js
@@ -124,12 +124,28 @@ Ext.define('PVE.dc.Config', {
}
if (caps.storage['Datastore.Allocate'] || caps.dc['Sys.Audit']) {
- me.items.push({
- xtype: 'pveStorageView',
- title: gettext('Storage'),
- iconCls: 'fa fa-database',
- itemId: 'storage',
- });
+ me.items.push(
+ {
+ xtype: 'pveStorageView',
+ title: gettext('Storage'),
+ iconCls: 'fa fa-database',
+ itemId: 'storage',
+ },
+ // storage admins need the node identities for the target ACLs too
+ {
+ xtype: 'pveDcSAN',
+ title: 'SAN',
+ iconCls: 'fa fa-cubes',
+ itemId: 'san',
+ },
+ {
+ xtype: 'pveDcMultipath',
+ title: gettext('Multipath'),
+ iconCls: 'fa fa-code-fork',
+ itemId: 'multipath',
+ groups: ['san'],
+ },
+ );
}
if (caps.dc['Sys.Audit']) {
diff --git a/www/manager6/dc/Multipath.js b/www/manager6/dc/Multipath.js
new file mode 100644
index 000000000..9646b9d72
--- /dev/null
+++ b/www/manager6/dc/Multipath.js
@@ -0,0 +1,547 @@
+Ext.define('pve-dc-multipath-status', {
+ extend: 'Ext.data.Model',
+ fields: [
+ 'wwid',
+ 'source',
+ 'multipath-storage',
+ 'alias',
+ 'used-by',
+ 'size',
+ 'cluster-state',
+ 'not-observed',
+ 'nodes',
+ ],
+ idProperty: 'wwid',
+});
+
+Ext.define('PVE.dc.MultipathOverridesEdit', {
+ extend: 'Proxmox.window.Edit',
+ xtype: 'pveDcMultipathOverridesEdit',
+
+ subject: gettext('Multipath Overrides'),
+ onlineHelp: 'storage_multipath',
+ width: 600,
+ autoLoad: true,
+ method: 'PUT',
+ url: '/cluster/multipath/overrides',
+
+ items: [
+ {
+ xtype: 'textarea',
+ name: 'overrides',
+ fieldLabel: gettext('Overrides'),
+ height: 320,
+ fieldStyle: 'font-family: monospace;',
+ emptyText:
+ 'devices {\n\tdevice {\n\t\tvendor "..."\n\t\tproduct "..."\n' +
+ '\t\tpath_grouping_policy ...\n\t}\n}',
+ },
+ ],
+});
+
+Ext.define('PVE.dc.MultipathView', {
+ extend: 'Ext.grid.Panel',
+ xtype: 'pveDcMultipath',
+
+ onlineHelp: 'storage_multipath',
+
+ stateful: true,
+ stateId: 'grid-dc-multipath',
+
+ emptyText: gettext(
+ 'No managed LUNs. To add some, create an iSCSI storage with the Multipath option under Datacenter > Storage > Add > Remote Storage, or enable multipath for a LUN under Node > SAN.',
+ ),
+
+ viewModel: {
+ data: {
+ wwid: '',
+ manual: false,
+ },
+ },
+
+ controller: {
+ xclass: 'Ext.app.ViewController',
+
+ reload: function () {
+ this.getView().rstore.load();
+ },
+
+ selectedRecord: function () {
+ let sel = this.getView().getSelection();
+ return sel.length ? sel[0] : undefined;
+ },
+
+ editWwid: function () {
+ let me = this;
+ let rec = me.selectedRecord();
+ if (!rec) {
+ return;
+ }
+
+ let isCreate = rec.data.source !== 'manual';
+ let hint;
+ if (rec.data.source === 'kept') {
+ // nodes without a current report cannot say whether they keep the map, offer all
+ // listed ones
+ let listed = Object.keys(rec.data.nodes || {});
+ let keeping = listed.filter((node) => rec.data.nodes[node]?.kept);
+ if (!keeping.length) {
+ keeping = listed;
+ }
+ Ext.create('PVE.dc.MultipathWwidEdit', {
+ wwid: rec.data.wwid,
+ isCreate: true,
+ nodes: keeping.length ? keeping : undefined,
+ hint: gettext(
+ 'No storage or entry requires this LUN anymore, so nodes keep its map only while it is in use. Saving creates an explicit entry that keeps the LUN configured on the selected nodes.',
+ ),
+ apiCallDone: (success) => success && me.reload(),
+ autoShow: true,
+ });
+ return;
+ }
+ if (isCreate) {
+ hint = gettext(
+ 'Saving creates a separate explicit entry with these settings, which stays even when no storage requires the LUN anymore.',
+ );
+ if (rec.data['multipath-storage']) {
+ let storage = Ext.String.format(
+ gettext('Storage "{0}" enables multipath for this LUN.'),
+ Ext.htmlEncode(rec.data['multipath-storage']),
+ );
+ hint = `${storage} ${hint}`;
+ }
+ }
+
+ let open = (nodes) =>
+ Ext.create('PVE.dc.MultipathWwidEdit', {
+ wwid: rec.data.wwid,
+ isCreate: isCreate,
+ nodes: nodes,
+ hint: hint,
+ apiCallDone: (success) => success && me.reload(),
+ autoShow: true,
+ });
+
+ // start on the nodes of the storage, others never see the LUN and would only count as
+ // missing
+ if (!isCreate || !rec.data['multipath-storage']) {
+ open(undefined);
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: `/storage/${encodeURIComponent(rec.data['multipath-storage'])}`,
+ method: 'GET',
+ waitMsgTarget: me.getView(),
+ failure: () => open(undefined),
+ success: (response) => open(response.result.data?.nodes?.split(',')),
+ });
+ },
+
+ removeWwid: function () {
+ let me = this;
+ let rec = me.selectedRecord();
+ if (!rec || rec.data.source !== 'manual') {
+ return;
+ }
+ let wwid = rec.data.wwid;
+
+ let msg =
+ Ext.String.format(
+ gettext('Remove the multipath entry of LUN {0}?'),
+ Ext.htmlEncode(wwid),
+ ) +
+ '<br><br>' +
+ gettext('Nodes that still use the map keep it until it is no longer in use.');
+
+ Ext.Msg.confirm(gettext('Confirm'), msg, function (btn) {
+ if (btn !== 'yes') {
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: `/cluster/multipath/wwid/${encodeURIComponent(wwid)}`,
+ method: 'DELETE',
+ waitMsgTarget: me.getView(),
+ failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
+ success: () => me.reload(),
+ });
+ });
+ },
+
+ editOverrides: function () {
+ let me = this;
+ Ext.create('PVE.dc.MultipathOverridesEdit', {
+ apiCallDone: (success) => success && me.reload(),
+ autoShow: true,
+ });
+ },
+
+ collectWarnings: function (records) {
+ let nodeStatus = this.getView().nodeStatus || {};
+ let enc = Ext.htmlEncode;
+ let lines = [];
+ let notes = [];
+ // the apply error of these nodes already says that multipath-tools is missing
+ let toolsMissing = {};
+ for (const [node, status] of Object.entries(nodeStatus)) {
+ toolsMissing[node] =
+ status?.['apply-error']?.trim() === 'multipath-tools is not installed';
+ }
+
+ for (const node of Object.keys(nodeStatus).sort()) {
+ let status = nodeStatus[node];
+ if (status['apply-error']) {
+ lines.push(
+ Ext.String.format(
+ gettext('Node "{0}" could not apply the multipath configuration: {1}'),
+ enc(node),
+ enc(status['apply-error']),
+ ),
+ );
+ }
+ if (status['report-error']) {
+ lines.push(
+ Ext.String.format(
+ gettext('Node "{0}" could not query its multipath maps: {1}'),
+ enc(node),
+ enc(status['report-error']),
+ ),
+ );
+ }
+ if (status.unmanaged) {
+ notes.push(
+ Ext.String.format(
+ ngettext(
+ 'Node "{0}" has {1} multipath map outside the cluster-wide configuration, see its SAN view.',
+ 'Node "{0}" has {1} multipath maps outside the cluster-wide configuration, see its SAN view.',
+ status.unmanaged,
+ ),
+ enc(node),
+ status.unmanaged,
+ ),
+ );
+ }
+ if (status['unresolved-luns']) {
+ lines.push(
+ Ext.String.format(
+ ngettext(
+ 'Node "{0}" cannot identify {1} LUN of its multipath-enabled storages because the identifiers of its paths are unavailable or could not be verified.',
+ 'Node "{0}" cannot identify {1} LUNs of its multipath-enabled storages because the identifiers of their paths are unavailable or could not be verified.',
+ status['unresolved-luns'],
+ ),
+ enc(node),
+ status['unresolved-luns'],
+ ),
+ );
+ }
+ if (status.summary) {
+ let { counts } = PVE.SANUtils.multipath_summary_counts(status);
+ let line = Ext.String.format(
+ gettext(
+ 'Node "{0}" has too many LUNs to list here ({1}), see its SAN view.',
+ ),
+ enc(node),
+ PVE.SANUtils.render_san_state_counts({ counts }),
+ );
+ let reason = status.reason ?? (status.stalled ? 'stalled' : undefined);
+ if (reason && !(reason === 'unsupported' && toolsMissing[node])) {
+ line += ` ${PVE.SANUtils.render_multipath_reason(reason)}.`;
+ }
+ let healthy = Object.keys(counts).every(
+ (state) => state === 'optimal' || !counts[state],
+ );
+ (healthy ? notes : lines).push(line);
+ }
+ }
+
+ let unknown = {};
+ for (const rec of records || []) {
+ Ext.Object.each(rec.data.nodes || {}, function (node, entry) {
+ // a node with too many LUNs to list, or whose apply error already says that
+ // multipath-tools is missing, has its own line above
+ if (
+ (entry?.state === 'unknown' || entry?.reason === 'stalled') &&
+ entry.reason &&
+ entry.reason !== 'summary-only' &&
+ !(entry.reason === 'unsupported' && toolsMissing[node])
+ ) {
+ (unknown[entry.reason] ??= {})[node] = true;
+ }
+ });
+ }
+ for (const reason of Object.keys(unknown).sort()) {
+ lines.push(
+ `${enc(Object.keys(unknown[reason]).sort().join(', '))}: ` +
+ PVE.SANUtils.render_multipath_reason(reason),
+ );
+ }
+
+ return { faults: lines, notes };
+ },
+
+ onStoreLoad: function (store, records, success) {
+ let me = this;
+
+ if (!success) {
+ return;
+ }
+
+ // the DiffStore updates a selected row in place, without a selectionchange event
+ me.syncSelection();
+
+ let { faults, notes } = me.collectWarnings(records);
+ let view = me.getView();
+ view.down('#nodeWarnings').setLines(faults);
+ view.down('#nodeNotes').setLines(notes);
+ },
+
+ syncSelection: function () {
+ let vm = this.getViewModel();
+ let rec = this.selectedRecord();
+ vm.set('wwid', rec ? rec.data.wwid : '');
+ vm.set('manual', !!rec && rec.data.source === 'manual');
+ },
+
+ control: {
+ '#': {
+ selectionchange: 'syncSelection',
+ },
+ },
+ },
+
+ plugins: [
+ {
+ ptype: 'rowexpander',
+ expandOnDblClick: false,
+ scrollIntoViewOnExpand: false,
+ rowBodyTpl: [
+ '{[this.renderNodes(values)]}',
+ {
+ renderNodes: function (values) {
+ let enc = Ext.htmlEncode;
+ let nodes = values.nodes || {};
+ let names = Object.keys(nodes).sort();
+ if (!names.length) {
+ return gettext('No node reports this LUN.');
+ }
+
+ let rows = names.map((node) => {
+ let entry = nodes[node] || {};
+ let expected = entry['paths-expected'];
+ let paths = PVE.SANUtils.format_san_paths(
+ entry['paths-active'],
+ entry['paths-total'],
+ expected,
+ );
+ let pathsCell = paths ? enc(paths.text) : '';
+ if (paths && !Ext.isNumeric(expected)) {
+ let qtip = gettext(
+ 'Only the paths found are counted; the expected number of paths is not known.',
+ );
+ pathsCell = `<span data-qtip="${enc(qtip)}">${pathsCell}</span>`;
+ }
+ return [
+ enc(node),
+ PVE.SANUtils.render_multipath_health(
+ PVE.SANUtils.multipath_count_state(entry),
+ ),
+ pathsCell,
+ PVE.SANUtils.render_san_transport(entry.transport),
+ [
+ entry.kept
+ ? Ext.String.format(
+ gettext('Kept while in use ({0})'),
+ enc(entry.kept),
+ )
+ : '',
+ PVE.SANUtils.render_multipath_reason(entry.reason),
+ entry['report-time'] !== undefined
+ ? Ext.String.format(
+ gettext('Last report: {0}'),
+ enc(
+ Proxmox.Utils.render_timestamp(
+ entry['report-time'],
+ ),
+ ),
+ )
+ : '',
+ ]
+ .filter((note) => note !== '')
+ .join('; '),
+ ];
+ });
+
+ return PVE.SANUtils.render_detail_table(
+ [
+ gettext('Node'),
+ gettext('State'),
+ gettext('Paths'),
+ gettext('Transport'),
+ gettext('Notes'),
+ ],
+ rows,
+ );
+ },
+ },
+ ],
+ },
+ ],
+
+ columns: [
+ {
+ text: 'WWID',
+ dataIndex: 'wwid',
+ stateId: 'wwid',
+ flex: 2,
+ renderer: Ext.htmlEncode,
+ },
+ {
+ text: gettext('Alias'),
+ dataIndex: 'alias',
+ stateId: 'alias',
+ width: 140,
+ renderer: (v) => Ext.htmlEncode(v || ''),
+ },
+ {
+ text: gettext('Source'),
+ dataIndex: 'source',
+ stateId: 'source',
+ width: 160,
+ renderer: function (value, metaData, rec) {
+ if (value === 'manual') {
+ return gettext('Explicit entry');
+ } else if (value === 'kept') {
+ return gettext('Kept while in use');
+ } else if (value !== 'storage') {
+ return '';
+ }
+ let storeid = rec.data['multipath-storage'];
+ return storeid
+ ? Ext.htmlEncode(Ext.String.format(gettext('Storage "{0}"'), storeid))
+ : gettext('Storage');
+ },
+ },
+ {
+ text: gettext('Used By'),
+ dataIndex: 'used-by',
+ stateId: 'used-by',
+ width: 120,
+ renderer: (v) => Ext.htmlEncode(v || ''),
+ },
+ {
+ text: gettext('Size'),
+ dataIndex: 'size',
+ stateId: 'size',
+ width: 90,
+ align: 'right',
+ renderer: (v) => (v ? Proxmox.Utils.format_size(v) : ''),
+ },
+ {
+ text: gettext('Nodes'),
+ dataIndex: 'nodes',
+ stateId: 'nodes-health',
+ minWidth: 180,
+ cellWrap: true,
+ width: 220,
+ sortable: false,
+ renderer: (value) =>
+ PVE.SANUtils.render_node_states(value, PVE.SANUtils.multipath_count_state),
+ },
+ {
+ text: gettext('Cluster Health'),
+ dataIndex: 'cluster-state',
+ stateId: 'cluster-state',
+ width: 170,
+ renderer: (value, metaData, rec) =>
+ PVE.SANUtils.render_multipath_cluster_health(value, metaData, rec.data),
+ },
+ ],
+
+ initComponent: function () {
+ let me = this;
+
+ let caps = Ext.state.Manager.get('GuiCap');
+ // the editors load the entry first, which needs Sys.Audit
+ let canModify = !!(caps?.dc?.['Sys.Modify'] && caps?.dc?.['Sys.Audit']);
+
+ let rstore = Ext.create('Proxmox.data.UpdateStore', {
+ storeid: 'dc-multipath-status',
+ interval: 5000,
+ model: 'pve-dc-multipath-status',
+ proxy: {
+ type: 'proxmox',
+ url: '/api2/json/cluster/multipath/status',
+ reader: {
+ rootProperty: 'data.luns',
+ transform: (raw) => {
+ me.nodeStatus = raw?.data?.nodes || {};
+ // in the row data, so a changed report time refreshes an expanded row
+ for (const lun of raw?.data?.luns || []) {
+ for (const [node, entry] of Object.entries(lun.nodes || {})) {
+ entry['report-time'] = me.nodeStatus[node]?.time;
+ }
+ }
+ return raw;
+ },
+ },
+ },
+ });
+ me.rstore = rstore;
+
+ me.store = Ext.create('Proxmox.data.DiffStore', {
+ rstore: rstore,
+ sorters: [{ property: 'wwid' }],
+ });
+ // data only ever comes from the rstore, so drop loads through the memory proxy: they
+ // would replace the synced records with the proxy's empty content
+ me.store.load = Ext.emptyFn;
+
+ me.dockedItems = [
+ { xtype: 'pveSANNotice', itemId: 'nodeWarnings', kind: 'warning' },
+ { xtype: 'pveSANNotice', itemId: 'nodeNotes' },
+ ];
+
+ // only an own entry can be removed, a derived one goes with its storage
+ let hasSelection = canModify ? { bind: { disabled: '{!wwid}' } } : { disabled: true };
+ let isManual = canModify ? { bind: { disabled: '{!manual}' } } : { disabled: true };
+
+ me.tbar = [
+ {
+ text: gettext('Reload'),
+ iconCls: 'fa fa-refresh',
+ handler: 'reload',
+ },
+ Ext.apply(
+ {
+ text: gettext('Edit'),
+ iconCls: 'fa fa-pencil',
+ handler: 'editWwid',
+ },
+ hasSelection,
+ ),
+ Ext.apply(
+ {
+ text: gettext('Remove'),
+ iconCls: 'fa fa-trash-o',
+ handler: 'removeWwid',
+ },
+ isManual,
+ ),
+ '->',
+ {
+ text: gettext('Edit Overrides'),
+ iconCls: 'fa fa-pencil',
+ disabled: !canModify,
+ handler: 'editOverrides',
+ },
+ ];
+
+ me.callParent();
+
+ Proxmox.Utils.monStoreErrors(me, rstore, true);
+ me.mon(rstore, 'load', me.getController().onStoreLoad, me.getController());
+ me.on('activate', () => rstore.startUpdate());
+ me.on('deactivate', () => rstore.stopUpdate());
+ me.on('destroy', () => rstore.stopUpdate());
+ },
+});
diff --git a/www/manager6/dc/MultipathWwidEdit.js b/www/manager6/dc/MultipathWwidEdit.js
new file mode 100644
index 000000000..68a598596
--- /dev/null
+++ b/www/manager6/dc/MultipathWwidEdit.js
@@ -0,0 +1,183 @@
+Ext.define('PVE.dc.MultipathWwidEdit', {
+ extend: 'Proxmox.window.Edit',
+ xtype: 'pveDcMultipathWwidEdit',
+
+ subject: gettext('Multipath LUN'),
+ onlineHelp: 'storage_multipath',
+ width: 520,
+ fieldDefaults: {
+ labelWidth: 150,
+ },
+
+ wwid: undefined,
+ nodes: undefined,
+
+ hint: undefined,
+
+ initComponent: function () {
+ let me = this;
+
+ if (!me.isCreate && !me.wwid) {
+ throw 'no wwid specified';
+ }
+
+ let isCreate = !!me.isCreate;
+
+ let items = [
+ me.wwid
+ ? {
+ xtype: 'displayfield',
+ name: 'wwid',
+ fieldLabel: 'WWID',
+ value: me.wwid,
+ // updates take the WWID from the URL, only creation passes it as a parameter
+ submitValue: isCreate,
+ }
+ : {
+ xtype: 'proxmoxtextfield',
+ name: 'wwid',
+ fieldLabel: 'WWID',
+ allowBlank: false,
+ emptyText: '3600140500a1b2c3d4e5f6a7b8c9d0e1f',
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'alias',
+ fieldLabel: gettext('Alias'),
+ emptyText: 'san-a-lun0',
+ maxLength: 64,
+ deleteEmpty: !isCreate,
+ regex: /^[a-zA-Z0-9][a-zA-Z0-9._-]*$/,
+ regexText: gettext(
+ 'The alias becomes the name of the map. It must start with a letter or digit and may contain only letters, digits, dots, dashes, and underscores.',
+ ),
+ },
+ {
+ xtype: 'pveNodeSelector',
+ name: 'nodes',
+ fieldLabel: gettext('Nodes'),
+ emptyText: gettext('All') + ' (' + gettext('No restrictions') + ')',
+ multiSelect: true,
+ autoSelect: false,
+ allowBlank: true,
+ skipEmptyText: true,
+ deleteEmpty: !isCreate,
+ value: me.nodes,
+ },
+ ];
+
+ if (me.hint) {
+ items.push({
+ xtype: 'displayfield',
+ value: PVE.Utils.note_text(me.hint),
+ });
+ }
+
+ let advancedItems = [
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'no-path-retry',
+ deleteEmpty: !isCreate,
+ fieldLabel: 'no_path_retry',
+ emptyText: Proxmox.Utils.defaultText,
+ regex: /^(queue|fail|\d+)$/,
+ regexText: gettext(
+ "Either 'queue', 'fail', or the number of polling intervals to retry for.",
+ ),
+ },
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'path-grouping-policy',
+ fieldLabel: 'path_grouping_policy',
+ deleteEmpty: !isCreate,
+ value: '__default__',
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext(
+ 'ALUA detection can override this setting, see the documentation.',
+ ),
+ },
+ comboItems: [
+ ['__default__', Proxmox.Utils.defaultText],
+ ['failover', 'failover'],
+ ['multibus', 'multibus'],
+ ['group_by_serial', 'group_by_serial'],
+ ['group_by_prio', 'group_by_prio'],
+ ['group_by_node_name', 'group_by_node_name'],
+ ['group_by_tpg', 'group_by_tpg'],
+ ],
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'failback',
+ deleteEmpty: !isCreate,
+ fieldLabel: 'failback',
+ emptyText: Proxmox.Utils.defaultText,
+ regex: /^(manual|immediate|followover|\d+)$/,
+ regexText: gettext(
+ "Either 'manual', 'immediate', 'followover', or a number of seconds.",
+ ),
+ },
+ {
+ xtype: 'proxmoxtextfield',
+ name: 'path-selector',
+ deleteEmpty: !isCreate,
+ fieldLabel: 'path_selector',
+ emptyText: 'service-time 0',
+ regex: /^(round-robin|queue-length|service-time|historical-service-time|io-affinity) \d+$/,
+ regexText: gettext(
+ "A path selector as in multipath.conf, for example 'service-time 0'.",
+ ),
+ },
+ {
+ xtype: 'proxmoxintegerfield',
+ name: 'rr-min-io-rq',
+ deleteEmpty: !isCreate,
+ fieldLabel: 'rr_min_io_rq',
+ emptyText: Proxmox.Utils.defaultText,
+ minValue: 1,
+ allowBlank: true,
+ },
+ {
+ xtype: 'proxmoxKVComboBox',
+ name: 'rr-weight',
+ fieldLabel: 'rr_weight',
+ deleteEmpty: !isCreate,
+ value: '__default__',
+ comboItems: [
+ ['__default__', Proxmox.Utils.defaultText],
+ ['priorities', 'priorities'],
+ ['uniform', 'uniform'],
+ ],
+ },
+ ];
+
+ Ext.apply(me, {
+ items: [
+ {
+ xtype: 'inputpanel',
+ items: items,
+ advancedItems: advancedItems,
+ },
+ ],
+ });
+
+ if (isCreate) {
+ me.method = 'POST';
+ me.url = '/cluster/multipath/wwid';
+ } else {
+ me.method = 'PUT';
+ me.url = `/cluster/multipath/wwid/${encodeURIComponent(me.wwid)}`;
+ me.autoLoad = true;
+ me.autoLoadOptions = {
+ success: function (response) {
+ let values = response.result.data;
+ values.nodes = values.nodes ? values.nodes.split(',') : [];
+ me.setValues(values);
+ },
+ };
+ }
+
+ me.callParent();
+ },
+});
diff --git a/www/manager6/dc/SAN.js b/www/manager6/dc/SAN.js
new file mode 100644
index 000000000..586911ef2
--- /dev/null
+++ b/www/manager6/dc/SAN.js
@@ -0,0 +1,326 @@
+Ext.define('pve-dc-san-node', {
+ extend: 'Ext.data.Model',
+ fields: ['node', 'multipath', 'managed', 'initiator', 'initiator-note'],
+ idProperty: 'node',
+});
+
+Ext.define('PVE.dc.SANView', {
+ extend: 'Ext.grid.Panel',
+ xtype: 'pveDcSAN',
+
+ onlineHelp: 'storage_multipath',
+
+ stateful: true,
+ stateId: 'grid-dc-san',
+
+ emptyText: gettext('No node reported its SAN state yet.'),
+
+ sources: undefined,
+
+ columns: [
+ {
+ text: gettext('Node'),
+ dataIndex: 'node',
+ width: 140,
+ renderer: Ext.htmlEncode,
+ },
+ {
+ text: gettext('Multipath Service'),
+ dataIndex: 'multipath',
+ width: 200,
+ renderer: (value) => PVE.dc.SANView.renderNodeMultipath(value),
+ },
+ {
+ text: gettext('Managed LUNs'),
+ dataIndex: 'managed',
+ stateId: 'managed-health',
+ minWidth: 180,
+ cellWrap: true,
+ flex: 1,
+ renderer: (value) => PVE.SANUtils.render_san_state_counts(value),
+ },
+ {
+ text: gettext('iSCSI Initiator Name'),
+ dataIndex: 'initiator',
+ flex: 2,
+ renderer: (value, metaData, rec) =>
+ PVE.dc.SANView.renderIdentity(value, rec, 'initiator-note'),
+ },
+ ],
+
+ statics: {
+ renderIdentity: function (value, rec, noteField) {
+ if (!value) {
+ return '';
+ }
+ let text = Ext.htmlEncode(value);
+ let note = rec.get(noteField);
+ if (note) {
+ let qtip = Ext.htmlEncode(note);
+ text = `<i class="fa fa-clock-o faded" data-qtip="${qtip}"></i> ${text}`;
+ }
+ return text;
+ },
+
+ reportNote: function (entry) {
+ if (entry?.reason === 'offline' || entry?.reason === 'stale') {
+ return PVE.SANUtils.render_multipath_reason(entry.reason);
+ }
+ return entry?.stalled ? PVE.SANUtils.render_multipath_reason('stalled') : undefined;
+ },
+
+ renderNodeMultipath: function (value) {
+ let report = value?.report;
+ let icon = (cls, text, qtip) => {
+ let tip = qtip ? ` data-qtip="${Ext.htmlEncode(qtip)}"` : '';
+ return `<i class="fa fa-fw ${cls}"${tip}></i> ${text}`;
+ };
+ let problem = value?.required
+ ? 'warning fa-exclamation-circle'
+ : 'faded fa-minus-circle';
+ if (!report) {
+ return icon('faded fa-question-circle', gettext('No report'));
+ }
+ if (report.reason === 'offline') {
+ return icon('faded fa-question-circle', gettext('Node is offline'));
+ }
+ if (report.reason === 'stale') {
+ return icon('warning fa-exclamation-circle', gettext('Report is stale'));
+ }
+ if (report['apply-error']) {
+ return icon(
+ 'critical fa-times-circle',
+ gettext('Configuration not applied'),
+ report['apply-error'],
+ );
+ }
+ if (!report.supported) {
+ return icon(problem, gettext('Not installed'));
+ }
+ if (!report.running) {
+ return icon(problem, gettext('Not running'));
+ }
+ if (report['report-error']) {
+ return icon(
+ 'warning fa-exclamation-circle',
+ gettext('Query failed'),
+ report['report-error'],
+ );
+ }
+ if (report.stalled) {
+ return icon('warning fa-exclamation-circle', gettext('Status updates stalled'));
+ }
+ let notes = [];
+ if (report['unresolved-luns']) {
+ notes.push(
+ Ext.String.format(
+ ngettext(
+ '{0} LUN not identified',
+ '{0} LUNs not identified',
+ report['unresolved-luns'],
+ ),
+ report['unresolved-luns'],
+ ),
+ );
+ }
+ if (report.unmanaged) {
+ notes.push(
+ Ext.String.format(
+ ngettext('{0} unmanaged map', '{0} unmanaged maps', report.unmanaged),
+ report.unmanaged,
+ ),
+ );
+ }
+ if (notes.length) {
+ return icon('warning fa-exclamation-circle', notes.join(', '));
+ }
+ return icon('good fa-check-circle', gettext('Running'));
+ },
+ },
+
+ // every field is set, so a failed source empties its cells instead of keeping stale values
+ updateRows: function () {
+ let me = this;
+ let mp = me.sources.multipath;
+
+ let rows = {};
+ let row = (node) =>
+ (rows[node] ??= {
+ node,
+ multipath: undefined,
+ managed: undefined,
+ initiator: undefined,
+ 'initiator-note': undefined,
+ });
+ for (const node of PVE.data.ResourceStore.getNodes()) {
+ row(node.node);
+ }
+
+ let inventory = 0;
+ if (mp.data) {
+ let nodes = mp.data.nodes || {};
+ Object.keys(nodes).forEach((node) => row(node));
+ let required = PVE.SANUtils.multipath_required_nodes(mp.data);
+ let entries = {};
+ for (const lun of mp.data.luns || []) {
+ inventory++;
+ Ext.Object.each(lun.nodes || {}, (node, entry) =>
+ (entries[node] ??= []).push(entry),
+ );
+ }
+ for (const node of Object.keys(rows)) {
+ let report = nodes[node];
+ rows[node].multipath = { report, required: !!required[node] };
+ rows[node].initiator = report?.['iscsi-initiator-name'];
+ rows[node]['initiator-note'] = PVE.dc.SANView.reportNote(report);
+ // a node with too many LUNs lists none of them, its own counts stand for all
+ if (report?.summary) {
+ inventory += report.summary.luns ?? 0;
+ rows[node].managed = PVE.SANUtils.multipath_summary_counts(report);
+ } else {
+ rows[node].managed = {
+ counts: PVE.SANUtils.count_states(
+ entries[node] ?? [],
+ PVE.SANUtils.multipath_count_state,
+ ),
+ };
+ }
+ }
+ }
+
+ // update the records in place, a reload keeps the selection
+ let store = me.getStore();
+ store.suspendEvents();
+ store.remove(store.getRange().filter((rec) => !rows[rec.getId()]));
+ Ext.Object.each(rows, function (node, data) {
+ let rec = store.getById(node);
+ if (rec) {
+ rec.set(data, { dirty: false });
+ } else {
+ store.add(data);
+ }
+ });
+ store.resumeEvents();
+ store.fireEvent('refresh', store);
+ me.getView().refresh();
+ // the copy actions follow what the selected row holds now
+ let sm = me.getSelectionModel();
+ sm.fireEvent('selectionchange', sm, sm.getSelection());
+
+ me.updateHints(inventory);
+ },
+
+ updateHints: function (inventory) {
+ let me = this;
+ let faults = [];
+ let notes = [];
+ Ext.Object.each(me.sources, function (key, source) {
+ let label = Ext.htmlEncode(source.label);
+ if (source.denied) {
+ notes.push(
+ Ext.String.format(
+ gettext(
+ '{0}: reading the status needs the Sys.Audit privilege on "/" or Datastore.Allocate on "/storage".',
+ ),
+ label,
+ ),
+ );
+ } else if (source.error) {
+ faults.push(`${label}: ${source.error}`);
+ }
+ });
+ let loaded = Object.values(me.sources).every((source) => source.data);
+ if (loaded && !inventory) {
+ notes.push(
+ gettext(
+ 'No managed LUNs are listed. Add a storage under Datacenter > Storage > Add > Remote Storage, or enable multipath for a LUN under Node > SAN.',
+ ),
+ );
+ }
+ me.down('#faults').setLines(faults);
+ me.down('#notes').setLines(notes);
+ },
+
+ loadSources: function () {
+ let me = this;
+ Ext.Object.each(me.sources, function (key, source) {
+ Proxmox.Utils.API2Request({
+ url: source.url,
+ method: 'GET',
+ failure: function (response) {
+ if (me.destroyed) {
+ return;
+ }
+ source.denied = PVE.SANUtils.request_denied(response);
+ source.error = response.htmlStatus;
+ source.data = undefined;
+ me.updateRows();
+ },
+ success: function (response) {
+ if (me.destroyed) {
+ return;
+ }
+ source.denied = false;
+ source.error = undefined;
+ source.data = response.result.data || {};
+ me.updateRows();
+ },
+ });
+ });
+ },
+
+ copySelected: function (field) {
+ let me = this;
+ let value = me.getSelection()[0]?.get(field);
+ if (!value) {
+ return;
+ }
+ PVE.Utils.copyToClipboard(value);
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ me.sources = {
+ multipath: { label: gettext('Multipath'), url: '/cluster/multipath/status' },
+ };
+
+ me.store = Ext.create('Ext.data.Store', {
+ model: 'pve-dc-san-node',
+ sorters: [{ property: 'node' }],
+ });
+
+ let copy = (text, field) => ({
+ xtype: 'proxmoxButton',
+ text: text,
+ iconCls: 'fa fa-clipboard',
+ disabled: true,
+ enableFn: (rec) => !!rec.get(field),
+ handler: () => me.copySelected(field),
+ });
+ me.tbar = [
+ {
+ text: gettext('Reload'),
+ iconCls: 'fa fa-refresh',
+ handler: () => me.loadSources(),
+ },
+ copy(gettext('Copy Initiator Name'), 'initiator'),
+ ];
+
+ me.dockedItems = [
+ { xtype: 'pveSANNotice', itemId: 'faults', kind: 'warning' },
+ { xtype: 'pveSANNotice', itemId: 'notes' },
+ ];
+
+ me.callParent();
+
+ let task = Ext.create('Ext.util.TaskRunner').newTask({
+ run: () => me.loadSources(),
+ interval: 5000,
+ fireOnStart: true,
+ });
+ me.on('activate', () => task.start());
+ me.on('deactivate', () => task.stop());
+ me.on('destroy', () => task.destroy());
+ },
+});
diff --git a/www/manager6/panel/SANNotice.js b/www/manager6/panel/SANNotice.js
new file mode 100644
index 000000000..08a26abb4
--- /dev/null
+++ b/www/manager6/panel/SANNotice.js
@@ -0,0 +1,26 @@
+Ext.define('PVE.panel.SANNotice', {
+ extend: 'Ext.Component',
+ alias: 'widget.pveSANNotice',
+
+ dock: 'top',
+ hidden: true,
+ padding: '5 10',
+
+ kind: 'info',
+
+ // lines are safe HTML
+ setLines: function (lines) {
+ let me = this;
+ let icon = me.kind === 'warning' ? 'fa-exclamation-triangle warning' : 'fa-info-circle';
+ me.setHtml(lines.map((line) => `<i class="fa ${icon}"></i> ${line}`).join('<br>'));
+ me.setHidden(!lines.length);
+ },
+
+ initComponent: function () {
+ let me = this;
+ if (me.kind === 'warning') {
+ me.userCls = 'pmx-hint';
+ }
+ me.callParent();
+ },
+});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 65/76] ui: storage: iscsi: add the multipath option
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (63 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 64/76] ui: dc: add a cluster-wide multipath view Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 66/76] ui: san luns: show multipath state and manage LUN entries Thomas Lamprecht
` (10 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Let an iSCSI storage request multipath maps for its LUNs. Existing
path-based LVM base volumes wait for their map when activated, but
enabling the option does not move logical volumes that are already
active on individual paths. Explain that limit in the edit dialog and
link to the device-dependency checks before relying on redundancy.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/SANUtils.js | 18 +++++++++++++++
www/manager6/storage/IScsiEdit.js | 38 +++++++++++++++++++++++++++++++
2 files changed, 56 insertions(+)
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index c101ebb75..ee75b5abb 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -168,6 +168,24 @@ Ext.define('PVE.SANUtils', {
return { counts, summary: true };
},
+ multipath_activation_hint: function (pathUsers = []) {
+ let text = Ext.String.format(
+ gettext(
+ 'Existing users of individual paths, such as active logical volumes or a guest that uses a path device directly, do not switch to the map when you enable multipath. Check their device dependencies before you rely on path redundancy. <a href="{0}" target="_blank" rel="noopener noreferrer">See the documentation.</a>',
+ ),
+ Ext.htmlEncode(Proxmox.Utils.get_help_link('storage_multipath')),
+ );
+ if (pathUsers.length) {
+ let found = pathUsers.map(({ node, users }) => `${node}: ${users}`).join('; ');
+ text +=
+ '<br>' +
+ Ext.htmlEncode(
+ Ext.String.format(gettext('Currently using individual paths: {0}.'), found),
+ );
+ }
+ return text;
+ },
+
render_san_lun_device: function (value, metaData, rec) {
let text = Ext.htmlEncode(value);
// only multipath mapped devices report their path count
diff --git a/www/manager6/storage/IScsiEdit.js b/www/manager6/storage/IScsiEdit.js
index c005eeb15..76ac85714 100644
--- a/www/manager6/storage/IScsiEdit.js
+++ b/www/manager6/storage/IScsiEdit.js
@@ -75,6 +75,16 @@ Ext.define('PVE.storage.IScsiInputPanel', {
onlineHelp: 'storage_open_iscsi',
+ // show the activation warning only when enabling multipath on an existing storage; use the
+ // dirty state, as loading sets the field before its original value
+ updateMultipathActivation: function () {
+ let me = this;
+ let field = me.down('field[name=multipath]');
+ me.lookup('multipathActivation').setHidden(
+ me.isCreate || !field.getValue() || !field.isDirty(),
+ );
+ },
+
onGetValues: function (values) {
let me = this;
@@ -149,5 +159,33 @@ Ext.define('PVE.storage.IScsiInputPanel', {
checked: true,
fieldLabel: gettext('Use LUNs directly'),
},
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'multipath',
+ fieldLabel: gettext('Multipath'),
+ // unset instead of stored as 0, a node with an older release does not know the option
+ cbind: {
+ deleteEmpty: '{!isCreate}',
+ },
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext(
+ 'Combine the paths of each LUN into one device on every node using the storage.',
+ ),
+ },
+ listeners: {
+ change: (field) => field.up('inputpanel').updateMultipathActivation(),
+ dirtychange: (field) => field.up('inputpanel').updateMultipathActivation(),
+ },
+ },
+ {
+ xtype: 'displayfield',
+ reference: 'multipathActivation',
+ userCls: 'pmx-hint',
+ hidden: true,
+ cbind: {
+ value: () => PVE.SANUtils.multipath_activation_hint(),
+ },
+ },
],
});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 66/76] ui: san luns: show multipath state and manage LUN entries
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (64 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 65/76] ui: storage: iscsi: add the multipath option Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 67/76] ui: storage wizard: add iSCSI support Thomas Lamprecht
` (9 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Show each LUN's map state and paths with the appropriate entry or
storage action. Preselect nodes that can see the LUN and those whose
visibility could not be checked. Ask administrators to review the node
restriction. Share source rendering with the cluster view.
A failed entry read does not prove absence: check the list before
offering creation, preserve errors otherwise, and discard obsolete
callbacks. Explain that stopped multipathd means unknown health, not
necessarily stopped I/O, and that enabling multipath does not move
active logical volumes.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
test/SANUtils_test.js | 36 ++
www/manager6/SANUtils.js | 166 ++++++-
www/manager6/dc/Multipath.js | 14 +-
www/manager6/dc/MultipathWwidEdit.js | 50 +++
www/manager6/node/SANLuns.js | 459 ++++++++++++++++++--
www/manager6/storage/wizard/FibreChannel.js | 5 +-
www/manager6/storage/wizard/LunPicker.js | 6 +
7 files changed, 671 insertions(+), 65 deletions(-)
diff --git a/test/SANUtils_test.js b/test/SANUtils_test.js
index d22307664..b13d75166 100644
--- a/test/SANUtils_test.js
+++ b/test/SANUtils_test.js
@@ -10,6 +10,7 @@ function sanUtils(translations = {}) {
PVE: {},
gettext: (text) => translations[text] ?? text,
Ext: {
+ apply: (target, source) => Object.assign(target, source),
define: (name, config) => {
sandbox.PVE.SANUtils = config;
},
@@ -52,3 +53,38 @@ test('cluster health of a LUN', () => {
res = render('missing', { n1: { state: 'unknown' } }, { 'not-observed': 1 });
assert.match(res.html, /> Not currently observed$/);
});
+
+test('paths of a LUN without a map', () => {
+ const utils = sanUtils();
+ const path = (devpath, inUse, users = '') => ({
+ devpath,
+ wwid: '3600a0980383235',
+ multipath: 'none',
+ usage: 'unused',
+ 'in-use': inUse,
+ 'in-use-by': users,
+ });
+ const group = (...rows) => {
+ const res = utils.group_san_lun_paths(rows);
+ assert.equal(res.length, 1, 'one row per LUN');
+ return res[0];
+ };
+
+ let lun = group(path('/dev/sdb', 'no'), path('/dev/sdc', 'no'));
+ assert.equal(lun['in-use'], 'no');
+ assert.equal(lun['path-devices'].join(' '), '/dev/sdb /dev/sdc');
+
+ lun = group(path('/dev/sdb', 'no'), path('/dev/sdc', 'yes', 'dm-5'));
+ assert.equal(lun['in-use'], 'yes', 'a later path in use marks the LUN');
+ assert.equal(lun['in-use-by'], 'dm-5');
+
+ lun = group(path('/dev/sdb', 'yes', 'dm-5'), path('/dev/sdc', 'yes', 'dm-5, /mnt'));
+ assert.equal(lun['in-use-by'], 'dm-5, /mnt', 'users of all paths, each once');
+
+ lun = group(path('/dev/sdb', 'no'), path('/dev/sdc', 'unknown', 'check failed: timeout'));
+ assert.equal(lun['in-use'], 'unknown', 'unused only if every path was checked');
+ assert.equal(lun['in-use-by'], 'check failed: timeout');
+
+ lun = group(path('/dev/sdb', 'unknown', 'check failed'), path('/dev/sdc', 'yes', 'dm-5'));
+ assert.equal(lun['in-use'], 'yes', 'a known user wins over a failed check');
+});
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index ee75b5abb..4893cd562 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -8,11 +8,16 @@ Ext.define('PVE.SANUtils', {
'model',
'paths-total',
'paths-active',
+ 'paths-expected',
+ 'path-devices',
'usage',
'used-by-storage',
'vgname',
'pv-in-use',
'detail',
+ 'multipath',
+ 'multipath-storage',
+ 'duplicate-pv',
'by-id',
],
@@ -186,32 +191,67 @@ Ext.define('PVE.SANUtils', {
return text;
},
- render_san_lun_device: function (value, metaData, rec) {
- let text = Ext.htmlEncode(value);
- // only multipath mapped devices report their path count
- let paths = rec.get('paths-total');
- if (paths) {
- let active = rec.get('paths-active');
- if (Ext.isNumeric(active) && active < paths) {
- let qtip = Ext.htmlEncode(
- gettext('Some paths failed, the multipath device is degraded'),
- );
- text =
- `<i data-qtip="${qtip}" class="fa fa-exclamation-triangle warning"></i> ` +
- `${text} (${active}/${paths} ${gettext('paths')})`;
- } else {
- text += ` (${paths} ${ngettext('path', 'paths', paths)})`;
+ // the scan lists a LUN without a multipath map once per path, fold those rows by WWID
+ group_san_lun_paths: function (rows) {
+ let groups = {};
+ let res = [];
+ for (const row of rows) {
+ let unassembled = row.multipath === 'none' || row.multipath === 'pending';
+ if (!unassembled || !row.wwid) {
+ res.push(row);
+ continue;
}
+ let group = groups[row.wwid];
+ if (!group) {
+ group = Ext.apply({}, row);
+ group['path-devices'] = [];
+ groups[row.wwid] = group;
+ res.push(group);
+ }
+ // Every path must prove orphan status; one path reporting membership wins over
+ // missing or contradictory proof on another path.
+ let proof = [group['pv-in-use'], row['pv-in-use']];
+ group['pv-in-use'] = proof.some((value) => [1, '1', true].includes(value))
+ ? 1
+ : proof.every((value) => [0, '0', false].includes(value))
+ ? 0
+ : undefined;
+ if (row.usage === 'unknown' && group.usage === 'unused') {
+ group.usage = 'unknown';
+ group.detail = row.detail;
+ }
+ // the LUN is in use if one of its paths is, and unused only if all of them are
+ if (row['in-use'] === 'yes') {
+ let users = group['in-use'] === 'yes' ? group['in-use-by'].split(', ') : [];
+ for (const user of row['in-use-by'].split(', ')) {
+ if (!users.includes(user)) {
+ users.push(user);
+ }
+ }
+ group['in-use'] = 'yes';
+ group['in-use-by'] = users.join(', ');
+ } else if (row['in-use'] !== 'no' && group['in-use'] === 'no') {
+ group['in-use'] = row['in-use'];
+ group['in-use-by'] = row['in-use-by'];
+ }
+ group['path-devices'].push(row.devpath);
+ // one allow-listed path means the map is on its way for the whole LUN
+ if (row.multipath === 'pending') {
+ group.multipath = 'pending';
+ }
+ group['paths-total'] = group['path-devices'].length;
}
- return text;
+ return res;
},
- // Unchecked nodes (offline, failed or timed out) may still see a LUN.
+ // Unchecked nodes (offline, failed or timed out) may still see a LUN. pathUsers misses process
+ // opens, so an empty list proves nothing.
scan_nodes_for_luns: function (
callback,
request = (options) => Proxmox.Utils.API2Request(options),
) {
let wwids = {};
+ let pathUsers = {};
let unchecked = [];
let nodes = PVE.data.ResourceStore.getNodes();
let pending = nodes.length;
@@ -221,6 +261,11 @@ Ext.define('PVE.SANUtils', {
Object.keys(wwids)
.filter((node) => wwids[node].has(wwid))
.sort(),
+ pathUsers: (wwid) =>
+ Object.keys(pathUsers)
+ .sort()
+ .filter((node) => pathUsers[node][wwid])
+ .map((node) => ({ node, users: pathUsers[node][wwid].join(', ') })),
unchecked: unchecked.sort(),
});
};
@@ -255,6 +300,17 @@ Ext.define('PVE.SANUtils', {
unchecked.push(node.node);
}
wwids[node.node] = new Set(rows.map((row) => row.wwid).filter((w) => w));
+ for (const row of rows) {
+ let onPath = ['none', 'pending'].includes(row.multipath);
+ if (!row.wwid || !onPath || !['yes', 'unknown'].includes(row['in-use'])) {
+ continue;
+ }
+ let users = ((pathUsers[node.node] ??= {})[row.wwid] ??= []);
+ users.push(
+ row['in-use-by'] ||
+ (row['in-use'] === 'unknown' ? gettext('unknown') : row.devpath),
+ );
+ }
done();
},
});
@@ -276,6 +332,30 @@ Ext.define('PVE.SANUtils', {
};
},
+ render_san_lun_paths: function (value, metaData, rec) {
+ let expected = rec.get('paths-expected');
+ let paths = PVE.SANUtils.format_san_paths(
+ rec.get('paths-active'),
+ rec.get('paths-total'),
+ expected,
+ );
+ if (!paths) {
+ return '';
+ }
+ if (!Ext.isNumeric(expected)) {
+ metaData.tdAttr = `data-qtip="${Ext.htmlEncode(
+ gettext(
+ 'Only the paths found are counted; the expected number of paths is not known.',
+ ),
+ )}"`;
+ }
+ if (paths.degraded) {
+ let qtip = Ext.htmlEncode(gettext('Some paths are missing or have failed.'));
+ return `<i data-qtip="${qtip}" class="fa fa-exclamation-triangle warning"></i> ${paths.text}`;
+ }
+ return paths.text;
+ },
+
request_uncertain: function (response) {
return response.timedout || response.status === 0 || response.result?.success === undefined;
},
@@ -285,9 +365,25 @@ Ext.define('PVE.SANUtils', {
return Number(response?.result?.status) === 403;
},
+ // returns { text, tip } as HTML
+ san_lun_usage: function (row) {
+ let text = PVE.SANUtils.san_lun_usage_text(row);
+ let tooltip = [text];
+ if (row['duplicate-pv']) {
+ let tip = gettext(
+ 'Another LUN has an LVM physical volume with the same UUID. LVM refuses to activate or change its volume group.',
+ );
+ tooltip.push(Ext.htmlEncode(tip));
+ text +=
+ ` <i class="fa fa-exclamation-triangle warning" data-qtip="${Ext.htmlEncode(tip)}"></i> ` +
+ gettext('duplicate PV');
+ }
+ return { text, tip: tooltip.join('<br>') };
+ },
+
render_san_lun_usage: function (value, metaData, rec) {
- let text = PVE.SANUtils.san_lun_usage_text(rec.data);
- metaData.tdAttr = `data-qtip="${Ext.htmlEncode(text)}"`;
+ let { text, tip } = PVE.SANUtils.san_lun_usage(rec.data);
+ metaData.tdAttr = `data-qtip="${Ext.htmlEncode(tip)}"`;
return text;
},
@@ -356,6 +452,38 @@ Ext.define('PVE.SANUtils', {
return Ext.htmlEncode(row.detail || gettext('in use'));
},
+ render_san_lun_multipath: function (value, metaData, rec) {
+ if (!value) {
+ return '';
+ }
+ if (value === 'storage') {
+ let sid = rec.get('multipath-storage');
+ return sid
+ ? Ext.htmlEncode(Ext.String.format(gettext('Storage "{0}"'), sid))
+ : gettext('Storage');
+ }
+ if (value === 'none') {
+ if (!(rec.get('paths-total') > 1)) {
+ return gettext('Single path');
+ }
+ let qtip = Ext.htmlEncode(
+ gettext(
+ 'The paths of this LUN are separate devices until multipath is enabled for it.',
+ ),
+ );
+ return `<i data-qtip="${qtip}" class="fa fa-exclamation-triangle warning"></i> ${gettext('Not enabled')}`;
+ }
+
+ return (
+ {
+ pending: gettext('Not assembled yet'),
+ manual: gettext('Explicit entry'),
+ kept: gettext('Kept while in use'),
+ unmanaged: gettext('Unmanaged'),
+ }[value] ?? Ext.htmlEncode(value)
+ );
+ },
+
render_san_transport: function (value) {
if (!value) {
return '';
diff --git a/www/manager6/dc/Multipath.js b/www/manager6/dc/Multipath.js
index 9646b9d72..75ea8d30d 100644
--- a/www/manager6/dc/Multipath.js
+++ b/www/manager6/dc/Multipath.js
@@ -407,19 +407,7 @@ Ext.define('PVE.dc.MultipathView', {
dataIndex: 'source',
stateId: 'source',
width: 160,
- renderer: function (value, metaData, rec) {
- if (value === 'manual') {
- return gettext('Explicit entry');
- } else if (value === 'kept') {
- return gettext('Kept while in use');
- } else if (value !== 'storage') {
- return '';
- }
- let storeid = rec.data['multipath-storage'];
- return storeid
- ? Ext.htmlEncode(Ext.String.format(gettext('Storage "{0}"'), storeid))
- : gettext('Storage');
- },
+ renderer: PVE.SANUtils.render_san_lun_multipath,
},
{
text: gettext('Used By'),
diff --git a/www/manager6/dc/MultipathWwidEdit.js b/www/manager6/dc/MultipathWwidEdit.js
index 68a598596..3a6491454 100644
--- a/www/manager6/dc/MultipathWwidEdit.js
+++ b/www/manager6/dc/MultipathWwidEdit.js
@@ -12,7 +12,43 @@ Ext.define('PVE.dc.MultipathWwidEdit', {
wwid: undefined,
nodes: undefined,
+ // nodes to add to the restriction of an existing entry once it is loaded
+ addNodes: undefined,
+
hint: undefined,
+ activationWarning: undefined,
+
+ statics: {
+ // The entry list tells a failed entry request apart from a missing entry. If the list
+ // request fails too, failure(original, list) gets both. current() guards each continuation.
+ lookup: function (wwid, callbacks) {
+ let { found, missing, failure } = callbacks;
+ let request = callbacks.request ?? ((options) => Proxmox.Utils.API2Request(options));
+ let current = callbacks.current ?? (() => true);
+ request({
+ url: `/cluster/multipath/wwid/${encodeURIComponent(wwid)}`,
+ method: 'GET',
+ failure: (original) =>
+ current() &&
+ request({
+ url: '/cluster/multipath/wwid',
+ method: 'GET',
+ failure: (list) => current() && failure(original, list),
+ success: function (response) {
+ if (!current()) {
+ return;
+ }
+ if ((response.result.data || []).some((entry) => entry.wwid === wwid)) {
+ failure(original);
+ } else {
+ missing();
+ }
+ },
+ }),
+ success: (response) => current() && found(response.result.data),
+ });
+ },
+ },
initComponent: function () {
let me = this;
@@ -73,6 +109,14 @@ Ext.define('PVE.dc.MultipathWwidEdit', {
});
}
+ if (me.activationWarning) {
+ items.push({
+ xtype: 'displayfield',
+ userCls: 'pmx-hint',
+ value: me.activationWarning,
+ });
+ }
+
let advancedItems = [
{
xtype: 'proxmoxtextfield',
@@ -174,6 +218,12 @@ Ext.define('PVE.dc.MultipathWwidEdit', {
let values = response.result.data;
values.nodes = values.nodes ? values.nodes.split(',') : [];
me.setValues(values);
+ // set after loading so the form sees a change; an unrestricted entry covers
+ // every node
+ let added = (me.addNodes ?? []).filter((node) => !values.nodes.includes(node));
+ if (values.nodes.length && added.length) {
+ me.down('field[name=nodes]').setValue([...values.nodes, ...added]);
+ }
},
};
}
diff --git a/www/manager6/node/SANLuns.js b/www/manager6/node/SANLuns.js
index fe30125d8..01c21f2b7 100644
--- a/www/manager6/node/SANLuns.js
+++ b/www/manager6/node/SANLuns.js
@@ -2,11 +2,11 @@ Ext.define('PVE.node.SANLunList', {
extend: 'Ext.grid.GridPanel',
xtype: 'pveSANLunList',
+ onlineHelp: 'storage_multipath',
+
stateful: true,
stateId: 'grid-node-san-luns',
- disableSelection: true,
-
emptyDeviceText: function (showAll) {
let text = Ext.String.format(
gettext(
@@ -24,10 +24,6 @@ Ext.define('PVE.node.SANLunList', {
return text;
},
- viewConfig: {
- trackOver: false,
- },
-
plugins: [
{
ptype: 'rowexpander',
@@ -39,17 +35,26 @@ Ext.define('PVE.node.SANLunList', {
renderDetails: function (values) {
let enc = Ext.htmlEncode;
let lines = [];
- if (values.wwn) {
- lines.push(`<b>WWN:</b> ${enc(values.wwn)}`);
+ let line = (label, value) => lines.push(`<b>${label}:</b> ${enc(value)}`);
+ if (values.wwid) {
+ line('WWID', values.wwid);
+ }
+ // the WWID of a map is the WWN of the LUN
+ if (values.wwn && values.wwn !== values.wwid) {
+ line('WWN', values.wwn);
}
if (values.serial) {
- lines.push(`<b>${gettext('Serial')}:</b> ${enc(values.serial)}`);
+ line(gettext('Serial'), values.serial);
+ }
+ if (values['iscsi-target']) {
+ line(gettext('Target'), values['iscsi-target']);
+ }
+ if (values['path-devices']) {
+ line(gettext('Devices'), values['path-devices'].join(', '));
}
let faults = values['path-faults'];
if (Ext.isNumeric(faults)) {
- lines.push(
- `<b>${gettext('Path Failures Since Map Creation')}:</b> ${enc(faults)}`,
- );
+ line(gettext('Path Failures Since Map Creation'), faults);
}
let paths = values['path-list'] || [];
if (paths.length) {
@@ -100,7 +105,19 @@ Ext.define('PVE.node.SANLunList', {
header: gettext('Device'),
dataIndex: 'devpath',
flex: 2,
- renderer: PVE.SANUtils.render_san_lun_device,
+ renderer: Ext.htmlEncode,
+ },
+ {
+ header: gettext('Paths'),
+ dataIndex: 'paths-total',
+ width: 80,
+ renderer: PVE.SANUtils.render_san_lun_paths,
+ },
+ {
+ // the number the storage system knows the LUN by
+ header: 'LUN',
+ dataIndex: 'lun',
+ width: 60,
},
{
header: gettext('Transport'),
@@ -125,6 +142,7 @@ Ext.define('PVE.node.SANLunList', {
header: 'WWN',
dataIndex: 'wwn',
width: 200,
+ hidden: true,
},
{
header: gettext('Serial'),
@@ -132,6 +150,18 @@ Ext.define('PVE.node.SANLunList', {
width: 140,
hidden: true,
},
+ {
+ header: 'WWID',
+ dataIndex: 'wwid',
+ width: 200,
+ hidden: true,
+ },
+ {
+ header: gettext('Multipath'),
+ dataIndex: 'multipath',
+ flex: 1,
+ renderer: PVE.SANUtils.render_san_lun_multipath,
+ },
{
header: gettext('Usage'),
dataIndex: 'usage',
@@ -141,7 +171,287 @@ Ext.define('PVE.node.SANLunList', {
],
reload: function () {
- this.getStore().load();
+ let me = this;
+ if (me.destroyed) {
+ return;
+ }
+ me.getSelectionModel().deselectAll();
+ me.getStore().load();
+ me.loadNodeStatus();
+ },
+
+ loadNodeStatus: function () {
+ let me = this;
+ let show = ({ faults, notes = [] }) => {
+ if (me.destroyed) {
+ return;
+ }
+ me.down('#nodeProblems').setLines(faults);
+ me.down('#nodeNotes').setLines(notes);
+ };
+ Proxmox.Utils.API2Request({
+ url: '/cluster/multipath/status',
+ method: 'GET',
+ // an auditor of this node only may not read the cluster-wide status, which is no fault
+ failure: (response) =>
+ show({
+ faults: PVE.SANUtils.request_denied(response)
+ ? []
+ : [
+ `${gettext('The multipath status could not be read')}: ${response.htmlStatus}`,
+ ],
+ }),
+ success: function (response) {
+ let data = response.result.data || {};
+ let report = data.nodes?.[me.nodename];
+ let required = !!PVE.SANUtils.multipath_required_nodes(data)[me.nodename];
+ show(PVE.node.SANLunList.nodeProblems(report, required, me.canModify));
+ },
+ });
+ },
+
+ statics: {
+ nodeProblems: function (report, required, canManage) {
+ let enc = Ext.htmlEncode;
+ let lines = [];
+ let notes = [];
+ if (!report) {
+ return {
+ faults: required
+ ? [gettext('The node does not report its multipath status.')]
+ : [],
+ notes,
+ };
+ }
+ if (report.reason === 'stale') {
+ lines.push(gettext('The multipath status of the node is not current.'));
+ }
+ if (report['apply-error']) {
+ lines.push(
+ `${gettext('The multipath configuration could not be applied')}: ${enc(report['apply-error'])}`,
+ );
+ }
+ // missing multipath-tools can already be the apply error
+ let toolsMissing = report['apply-error']?.trim() === 'multipath-tools is not installed';
+ if (required && !report.supported) {
+ if (!toolsMissing) {
+ lines.push(PVE.SANUtils.render_multipath_reason('unsupported'));
+ }
+ } else if (required && report.supported && !report.running) {
+ lines.push(
+ gettext(
+ 'multipathd is not running. Multipath map health is unknown; existing maps may still carry I/O. Check the service and its log.',
+ ),
+ );
+ }
+ if (report['report-error']) {
+ lines.push(
+ `${gettext('Querying the multipath maps failed')}: ${enc(report['report-error'])}`,
+ );
+ }
+ if (report.stalled) {
+ lines.push(PVE.SANUtils.render_multipath_reason('stalled'));
+ }
+ if (report.unmanaged) {
+ notes.push(
+ Ext.String.format(
+ canManage
+ ? ngettext(
+ '{0} multipath map is outside the cluster-wide configuration. To take it over, select it and press Manage.',
+ '{0} multipath maps are outside the cluster-wide configuration. To take them over, select each map and press Manage.',
+ report.unmanaged,
+ )
+ : ngettext(
+ '{0} multipath map is outside the cluster-wide configuration.',
+ '{0} multipath maps are outside the cluster-wide configuration.',
+ report.unmanaged,
+ ),
+ report.unmanaged,
+ ),
+ );
+ }
+ if (report['unresolved-luns']) {
+ lines.push(
+ Ext.String.format(
+ ngettext(
+ 'The node cannot identify {0} LUN of its multipath-enabled storages because the identifiers of its paths are unavailable or could not be verified.',
+ 'The node cannot identify {0} LUNs of its multipath-enabled storages because the identifiers of their paths are unavailable or could not be verified.',
+ report['unresolved-luns'],
+ ),
+ report['unresolved-luns'],
+ ),
+ );
+ }
+ return { faults: lines, notes };
+ },
+ },
+
+ // guards async editor lookups against newer actions and destruction
+ beginEditOperation: function () {
+ let me = this;
+ let token = (me.editGeneration = (me.editGeneration ?? 0) + 1);
+ return () => !me.destroyed && !me.destroying && token === me.editGeneration;
+ },
+
+ openMultipathEdit: function (rec) {
+ let me = this;
+ let current = me.beginEditOperation();
+ if (!current()) {
+ return;
+ }
+ me.setLoading(false);
+ let state = rec.get('multipath');
+ let pathUsers = [];
+ let open = function (config) {
+ if (!current()) {
+ return;
+ }
+ Ext.create('PVE.dc.MultipathWwidEdit', {
+ wwid: rec.get('wwid'),
+ apiCallDone: (success) => current() && success && me.reload(),
+ autoShow: true,
+ ...config,
+ activationWarning:
+ state === 'none'
+ ? PVE.SANUtils.multipath_activation_hint(pathUsers)
+ : undefined,
+ });
+ };
+ if (state === 'manual') {
+ open({ isCreate: false });
+ return;
+ }
+ if (state === 'none') {
+ // Keep this node included even if the fresh inventory no longer reports the LUN.
+ me.setLoading(gettext('Checking which nodes see the LUN...'));
+ PVE.SANUtils.scan_nodes_for_luns(function (result) {
+ if (!current()) {
+ return;
+ }
+ me.setLoading(false);
+ pathUsers = result.pathUsers(rec.get('wwid'));
+ let nodes = [...new Set([me.nodename, ...result.sees(rec.get('wwid'))])];
+ let hint = gettext(
+ 'Preselected: this node and the nodes that see this LUN. Review which nodes should use it.',
+ );
+ if (result.unchecked.length) {
+ nodes.push(...result.unchecked.filter((node) => !nodes.includes(node)));
+ hint +=
+ ' ' +
+ Ext.String.format(
+ gettext('Also included, as they could not be checked: {0}.'),
+ result.unchecked.join(', '),
+ );
+ }
+ me.openEntryFor(rec, open, nodes.sort(), Ext.htmlEncode(hint), current);
+ });
+ return;
+ }
+ let hint = {
+ unmanaged: gettext(
+ 'This map was set up outside the cluster-wide multipath configuration. Saving adds it to that configuration for the selected nodes.',
+ ),
+ kept: gettext(
+ 'No storage or entry requires multipath for this LUN anymore, so this node keeps the map only while it is in use. Saving creates an explicit entry that keeps the LUN configured on the selected nodes.',
+ ),
+ }[state];
+ me.openEntryFor(rec, open, [me.nodename], hint, current);
+ },
+
+ // an entry restricted to other nodes is extended instead of creating one
+ openEntryFor: function (rec, open, nodes, hint, current) {
+ let me = this;
+ if (!current()) {
+ return;
+ }
+ me.setLoading(true);
+ PVE.dc.MultipathWwidEdit.lookup(rec.get('wwid'), {
+ current,
+ missing: function () {
+ me.setLoading(false);
+ open({ isCreate: true, nodes, hint });
+ },
+ failure: function (original, list) {
+ me.setLoading(false);
+ Ext.Msg.alert(
+ gettext('Error'),
+ list
+ ? `${gettext('The multipath entries could not be read.')}<br><br>${original.htmlStatus}`
+ : original.htmlStatus,
+ );
+ },
+ found: function (entry) {
+ me.setLoading(false);
+ let nodes = entry?.nodes?.split(',') ?? [];
+ if (!nodes.length || nodes.includes(me.nodename)) {
+ open({
+ isCreate: false,
+ hint: gettext(
+ 'The LUN already has an entry for this node. Its map is assembled once the node has applied the configuration.',
+ ),
+ });
+ return;
+ }
+ open({
+ isCreate: false,
+ addNodes: [me.nodename],
+ hint: Ext.String.format(
+ gettext(
+ 'The LUN has an entry restricted to other nodes. Saving adds node "{0}" to it.',
+ ),
+ Ext.htmlEncode(me.nodename),
+ ),
+ });
+ },
+ });
+ },
+
+ editStorage: function (storeid) {
+ let me = this;
+ let current = me.beginEditOperation();
+ if (!current()) {
+ return;
+ }
+ me.setLoading(true);
+ Proxmox.Utils.API2Request({
+ url: `/storage/${encodeURIComponent(storeid)}`,
+ method: 'GET',
+ failure: function (response) {
+ if (!current()) {
+ return;
+ }
+ me.setLoading(false);
+ Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+ },
+ success: function (response) {
+ if (!current()) {
+ return;
+ }
+ me.setLoading(false);
+ let type = response.result.data.type;
+ let schema = PVE.Utils.storageSchema[type];
+ if (!schema || !schema.ipanel) {
+ Ext.Msg.alert(
+ gettext('Error'),
+ Ext.String.format(
+ gettext('There is no editor for storage type "{0}".'),
+ Ext.htmlEncode(type),
+ ),
+ );
+ return;
+ }
+ Ext.create('PVE.storage.BaseEdit', {
+ paneltype: `PVE.storage.${schema.ipanel}`,
+ type: type,
+ storageId: storeid,
+ canDoBackups: schema.backups,
+ autoShow: true,
+ listeners: {
+ destroy: () => current() && me.reload(),
+ },
+ });
+ },
+ });
},
listeners: {
@@ -181,44 +491,131 @@ Ext.define('PVE.node.SANLunList', {
'transport',
'serial',
'wwn',
+ 'wwid',
'path-faults',
'path-list',
+ 'iscsi-target',
+ 'iscsi-lun',
+ {
+ name: 'lun',
+ depends: ['iscsi-lun'],
+ calculate: (data) => data['iscsi-lun'],
+ },
],
proxy: {
type: 'proxmox',
url: `/api2/json/nodes/${me.nodename}/scan/san-luns`,
+ reader: {
+ type: 'json',
+ rootProperty: 'data',
+ transform: (raw) => ({
+ ...raw,
+ data: PVE.SANUtils.group_san_lun_paths(raw?.data ?? []),
+ }),
+ },
},
filters: [sanTransportFilter],
- sorters: 'devpath',
+ sorters: ['devpath'],
});
me.store = store;
- me.tbar = [
+ let sm = Ext.create('Ext.selection.RowModel', {});
+ me.selModel = sm;
+
+ // the editor loads the entry, which needs Sys.Audit
+ let caps = Ext.state.Manager.get('GuiCap');
+ let canModify = !!caps?.dc?.['Sys.Modify'] && !!caps?.dc?.['Sys.Audit'];
+ me.canModify = canModify;
+
+ let items = [
{
text: gettext('Reload'),
iconCls: 'fa fa-refresh',
handler: () => me.reload(),
},
- '->',
{
- xtype: 'proxmoxcheckbox',
- boxLabel: gettext('Show all devices'),
- autoEl: {
- tag: 'div',
- 'data-qtip': gettext('Also list devices with local or unknown transports.'),
- },
+ xtype: 'proxmoxButton',
+ text: gettext('Copy WWID'),
+ iconCls: 'fa fa-clipboard',
+ selModel: sm,
+ disabled: true,
+ enableFn: (rec) => !!rec.get('wwid'),
+ handler: (btn, event, rec) => PVE.Utils.copyToClipboard(rec.get('wwid')),
+ },
+ ];
+
+ if (canModify) {
+ let entryText = {
+ none: gettext('Enable Multipath'),
+ unmanaged: gettext('Manage'),
+ kept: gettext('Manage'),
+ manual: gettext('Edit'),
+ };
+ items.push({
+ xtype: 'proxmoxButton',
+ itemId: 'entryButton',
+ text: entryText.none,
+ iconCls: 'fa fa-code-fork',
+ selModel: sm,
+ disabled: true,
+ enableFn: (rec) => !!rec.get('wwid') && !!entryText[rec.get('multipath')],
+ handler: (btn, event, rec) => me.openMultipathEdit(rec),
listeners: {
- change: function (f, value) {
- me.setEmptyText(me.emptyDeviceText(value));
- let filters = store.getFilters();
- if (value) {
- filters.remove(sanTransportFilter);
- } else {
- filters.add(sanTransportFilter);
- }
+ // keep the toolbar from moving when the label changes
+ render: function (btn) {
+ let width = Math.max(
+ ...Object.values(entryText).map((text) => {
+ btn.setText(text);
+ return btn.getSize().width;
+ }),
+ );
+ btn.setText(entryText.none);
+ btn.setSize({ width });
},
},
+ });
+ sm.on('selectionchange', function (model, selection) {
+ let text = entryText[selection[0]?.get('multipath')];
+ me.down('#entryButton').setText(text ?? entryText.none);
+ });
+ }
+
+ if (caps?.storage?.['Datastore.Allocate']) {
+ items.push({
+ xtype: 'proxmoxButton',
+ text: gettext('Edit Storage'),
+ iconCls: 'fa fa-database',
+ selModel: sm,
+ disabled: true,
+ enableFn: (rec) => !!rec.get('multipath-storage'),
+ handler: (btn, event, rec) => me.editStorage(rec.get('multipath-storage')),
+ });
+ }
+
+ items.push('->', {
+ xtype: 'proxmoxcheckbox',
+ boxLabel: gettext('Show all devices'),
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext('Also list devices with local or unknown transports.'),
},
+ listeners: {
+ change: function (f, value) {
+ me.setEmptyText(me.emptyDeviceText(value));
+ let filters = store.getFilters();
+ if (value) {
+ filters.remove(sanTransportFilter);
+ } else {
+ filters.add(sanTransportFilter);
+ }
+ },
+ },
+ });
+ me.tbar = items;
+
+ me.dockedItems = [
+ { xtype: 'pveSANNotice', itemId: 'nodeProblems', kind: 'warning' },
+ { xtype: 'pveSANNotice', itemId: 'nodeNotes' },
];
me.callParent();
diff --git a/www/manager6/storage/wizard/FibreChannel.js b/www/manager6/storage/wizard/FibreChannel.js
index 4affbd8a7..eec8a7790 100644
--- a/www/manager6/storage/wizard/FibreChannel.js
+++ b/www/manager6/storage/wizard/FibreChannel.js
@@ -171,7 +171,7 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
return;
}
grid.getStore().setData(
- (response.result.data || []).map((row) => ({
+ PVE.SANUtils.group_san_lun_paths(response.result.data || []).map((row) => ({
...row,
scanNode: node,
scanGeneration: generation,
@@ -318,8 +318,9 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
header: gettext('Device'),
dataIndex: 'devpath',
flex: 3,
- renderer: PVE.SANUtils.render_san_lun_device,
+ renderer: Ext.htmlEncode,
},
+ columns.paths,
{
header: gettext('Transport'),
dataIndex: 'transport',
diff --git a/www/manager6/storage/wizard/LunPicker.js b/www/manager6/storage/wizard/LunPicker.js
index cb3eba7b3..7ae6b0564 100644
--- a/www/manager6/storage/wizard/LunPicker.js
+++ b/www/manager6/storage/wizard/LunPicker.js
@@ -7,6 +7,12 @@ Ext.define('PVE.storage.wizard.LunPicker', {
lunColumns: function () {
return {
+ paths: {
+ header: gettext('Paths'),
+ dataIndex: 'paths-total',
+ width: 70,
+ renderer: PVE.SANUtils.render_san_lun_paths,
+ },
size: {
header: gettext('Size'),
dataIndex: 'size',
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 67/76] ui: storage wizard: add iSCSI support
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (65 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 66/76] ui: san luns: show multipath state and manage LUN entries Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 68/76] ui: storage wizard: iscsi: show cluster initiator names Thomas Lamprecht
` (8 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Guide iSCSI connection and shared LVM setup, with direct LUN use as an
alternative. Offer shared LVM only with the required privileges and keep
volume-group creation on the node serving the web UI.
Connect and Select LUN creates and activates the connection storage,
then lists LUNs on Confirm. Bound scans and discard obsolete responses.
If the scan is empty after failed or unconfirmed activation, report that
rather than suggesting target access rules are wrong. Offer unused LUNs
with usable identities and paths. Retain the last inventory after a
scan timeout and preselect a sole candidate only on the first settled
scan.
Create Volume Group is the explicit destructive action. Verify the
selected LUN identity and require the submitted volume name to encode
that identity, rather than joining separate observations by LUN number.
Keep retries on that device and skip completed steps. After a failure,
keep the connection storage and explain what remains. Share LVM
submission across transports through submitSanLvm().
Co-authored-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
debian/control | 1 +
test/Makefile | 6 +-
test/StorageWizard_test.js | 117 ++++++
www/manager6/Makefile | 3 +
www/manager6/SANUtils.js | 10 +-
www/manager6/storage/wizard/IScsi.js | 461 +++++++++++++++++++++++
www/manager6/storage/wizard/LunSelect.js | 446 ++++++++++++++++++++++
www/manager6/storage/wizard/SanLvm.js | 114 ++++++
www/manager6/storage/wizard/Wizard.js | 45 ++-
9 files changed, 1194 insertions(+), 9 deletions(-)
create mode 100644 test/StorageWizard_test.js
create mode 100644 www/manager6/storage/wizard/IScsi.js
create mode 100644 www/manager6/storage/wizard/LunSelect.js
create mode 100644 www/manager6/storage/wizard/SanLvm.js
diff --git a/debian/control b/debian/control
index abffe183b..04addf09a 100644
--- a/debian/control
+++ b/debian/control
@@ -21,6 +21,7 @@ Build-Depends: debhelper-compat (= 13),
libtemplate-perl,
libtest-mockmodule-perl,
lintian,
+ nodejs,
proxmox-biome,
proxmox-widget-toolkit (>= 4.1.4),
pve-cluster,
diff --git a/test/Makefile b/test/Makefile
index d7521c0b3..2854688c8 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -7,12 +7,16 @@ export PERLLIB=..
.PHONY: check
check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-pvesh
-check: test-san-utils
+check: test-san-utils test-storage-wizard
.PHONY: test-san-utils
test-san-utils:
node --test SANUtils_test.js
+.PHONY: test-storage-wizard
+test-storage-wizard:
+ node --test StorageWizard_test.js
+
.PHONY: test-balloon
test-balloon:
./balloontest.pl
diff --git a/test/StorageWizard_test.js b/test/StorageWizard_test.js
new file mode 100644
index 000000000..48305ccb3
--- /dev/null
+++ b/test/StorageWizard_test.js
@@ -0,0 +1,117 @@
+const assert = require('node:assert/strict');
+const fs = require('node:fs');
+const path = require('node:path');
+const vm = require('node:vm');
+const { test } = require('node:test');
+
+const selectedWWID = '36000000000000000000000000000000a';
+const otherWWID = '36000000000000000000000000000000b';
+const target = 'iqn.test';
+
+function submit({ name, multipath = false, freshWWID = selectedWWID, lun = 7 }) {
+ const alerts = [];
+ const submitted = [];
+ const sandbox = {
+ PVE: { storage: { wizard: { types: {}, reconcile: {} } } },
+ Proxmox: { NodeName: 'n1' },
+ gettext: (text) => text,
+ ngettext: (single, plural, count) => (count === 1 ? single : plural),
+ Ext: {
+ define: () => undefined,
+ htmlEncode: (text) => String(text),
+ clone: (value) => structuredClone(value),
+ String: {
+ format: (text, ...args) => text.replace(/\{(\d+)\}/g, (_, index) => args[index]),
+ },
+ Msg: { alert: (...args) => alerts.push(args) },
+ },
+ };
+ vm.createContext(sandbox);
+ for (const file of ['SanLvm.js', 'IScsi.js']) {
+ const source = path.join(__dirname, '../www/manager6/storage/wizard', file);
+ vm.runInContext(fs.readFileSync(source, 'utf8'), sandbox, { filename: source });
+ }
+ const selected = {
+ 'iscsi-target': target,
+ 'iscsi-lun': 7,
+ wwid: selectedWWID,
+ };
+ const wizard = {
+ created: { baseStorage: 'base' },
+ lunView: {
+ scanGeneration: 1,
+ selectionGeneration: 1,
+ getSelection: () => selected,
+ waitForLuns: () => undefined,
+ },
+ getScanNode: () => 'n1',
+ down: (selector) =>
+ selector === 'pveStorageWizardIScsiTarget' ? { targetPortals: 1 } : undefined,
+ actionText: () => 'Create Volume Group',
+ updateSummary: () => undefined,
+ canSubmit: () => true,
+ activateLunView: () => true,
+ runSubmitChain: (steps) => {
+ let index = 0;
+ const next = () => {
+ if (index < steps.length) {
+ steps[index++](next);
+ }
+ };
+ next();
+ },
+ request: (request) => {
+ const data = request.url.endsWith('/content')
+ ? [{ volid: `base:0.0.${lun}.${name}` }]
+ : [{ 'iscsi-target': target, 'iscsi-lun': 7, wwid: freshWWID }];
+ request.success({ result: { data } });
+ },
+ createStorage: (params, next) => {
+ submitted.push(params);
+ next();
+ },
+ };
+ sandbox.PVE.storage.wizard.types.iscsi.submit(wizard, {
+ usage: 'lvm',
+ storage: 'vg',
+ 'base-storage': 'base',
+ target,
+ portal: '192.0.2.1',
+ content: 'images',
+ multipath,
+ });
+ return { submitted, alerts, created: wizard.created };
+}
+
+for (const [label, params, accepted] of [
+ ['plain path with the selected identity', { name: `scsi-${selectedWWID}` }, true],
+ ['unmanaged map with the selected identity', { name: `dm-uuid-mpath-${selectedWWID}` }, true],
+ [
+ 'different content identity despite a matching fresh scan',
+ { name: `scsi-${otherWWID}` },
+ false,
+ ],
+ ['different unmanaged map identity', { name: `dm-uuid-mpath-${otherWWID}` }, false],
+ ['opaque alias', { name: 'dm-name-array-lun' }, false],
+ ['partition of the selected LUN', { name: `scsi-${selectedWWID}-part1` }, false],
+ ['different LUN number', { name: `scsi-${selectedWWID}`, lun: 8 }, false],
+ [
+ 'identity changed in the fresh scan',
+ { name: `scsi-${selectedWWID}`, freshWWID: otherWWID },
+ false,
+ ],
+]) {
+ test(label, () => {
+ const { submitted, alerts, created } = submit(params);
+ if (accepted) {
+ assert.equal(alerts.length, 0);
+ assert.equal(submitted.length, 1);
+ assert.equal(submitted[0].base, `base:0.0.7.${params.name}`);
+ assert.equal(created.lun.wwid, selectedWWID);
+ } else {
+ assert.equal(submitted.length, 0);
+ assert.equal(alerts.length, 1);
+ assert.equal(created.baseVolume, undefined);
+ }
+ });
+}
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 252c5761a..a90d5e0fd 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -388,8 +388,11 @@ JSSRC= \
storage/wizard/Wizard.js \
storage/wizard/CommonSettings.js \
storage/wizard/LunPicker.js \
+ storage/wizard/LunSelect.js \
+ storage/wizard/SanLvm.js \
storage/wizard/NFS.js \
storage/wizard/CIFS.js \
+ storage/wizard/IScsi.js \
storage/wizard/FibreChannel.js \
storage/wizard/ZFSOverISCSI.js \
Workspace.js \
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index 4893cd562..bd8115e31 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -365,9 +365,9 @@ Ext.define('PVE.SANUtils', {
return Number(response?.result?.status) === 403;
},
- // returns { text, tip } as HTML
- san_lun_usage: function (row) {
- let text = PVE.SANUtils.san_lun_usage_text(row);
+ // returns { text, tip } as HTML; an attribution to ownStorage alone does not count as use
+ san_lun_usage: function (row, { ownStorage } = {}) {
+ let text = PVE.SANUtils.san_lun_usage_text(row, ownStorage);
let tooltip = [text];
if (row['duplicate-pv']) {
let tip = gettext(
@@ -396,7 +396,7 @@ Ext.define('PVE.SANUtils', {
return content === '' || content === 'none';
},
- san_lun_usage_text: function (row) {
+ san_lun_usage_text: function (row, ownStorage) {
let value = row.usage;
if (value === 'unknown') {
return Ext.htmlEncode(
@@ -406,7 +406,7 @@ Ext.define('PVE.SANUtils', {
),
);
}
- let sid = row['used-by-storage'];
+ let sid = row['used-by-storage'] !== ownStorage ? row['used-by-storage'] : undefined;
// the scan also attributes LUNs to a connection-only storage, which leaves them free
let connects = sid && PVE.SANUtils.san_storage_only_connects(sid);
if (value === 'unused') {
diff --git a/www/manager6/storage/wizard/IScsi.js b/www/manager6/storage/wizard/IScsi.js
new file mode 100644
index 000000000..fb4decb38
--- /dev/null
+++ b/www/manager6/storage/wizard/IScsi.js
@@ -0,0 +1,461 @@
+Ext.define('PVE.storage.wizard.IScsiTarget', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pveStorageWizardIScsiTarget',
+
+ onlineHelp: 'storage_open_iscsi',
+
+ initComponent: function () {
+ let me = this;
+
+ me.column1 = [
+ {
+ xtype: 'textfield',
+ name: 'portal',
+ fieldLabel: 'Portal',
+ allowBlank: false,
+ listeners: {
+ change: {
+ fn: function (f, value) {
+ let targetField = me.down('field[name=target]');
+ targetField.setDisabled(!value);
+ targetField.setPortal(value);
+ targetField.setValue('');
+ },
+ buffer: 500,
+ },
+ },
+ },
+ {
+ xtype: 'pveIScsiScan',
+ name: 'target',
+ fieldLabel: gettext('Target'),
+ disabled: true,
+ showNodeSelector: false,
+ },
+ ];
+
+ me.column2 = [];
+ if (!PVE.Utils.isStandaloneNode()) {
+ me.column2.push({
+ xtype: 'pveStorageScanNodeSelector',
+ listeners: {
+ change: (f, value) => me.down('field[name=target]').setNodeName(value),
+ },
+ });
+ }
+
+ me.callParent();
+ },
+});
+
+Ext.define('PVE.storage.wizard.IScsiUsage', {
+ extend: 'Proxmox.panel.InputPanel',
+ xtype: 'pveStorageWizardIScsiUsage',
+
+ onlineHelp: 'storage_open_iscsi',
+
+ // returns { text (HTML), actionText, action } when the usage cannot proceed; submit rechecks
+ blockedReason: function () {
+ let me = this;
+ let wizard = me.up('window');
+ if (wizard.getViewModel().get('usage') !== 'lvm') {
+ return undefined;
+ }
+ if (wizard.getScanNode() !== Proxmox.NodeName) {
+ return {
+ text: Ext.String.format(
+ gettext(
+ 'Shared LVM is set up from node "{0}", which this web UI is connected to.',
+ ),
+ Ext.htmlEncode(Proxmox.NodeName),
+ ),
+ actionText: Ext.String.format(gettext('Scan from "{0}"'), Proxmox.NodeName),
+ action: () => wizard.down('pveStorageScanNodeSelector')?.setValue(Proxmox.NodeName),
+ };
+ }
+ return undefined;
+ },
+
+ validator: function () {
+ return !this.blockedReason();
+ },
+
+ updateBlockedReason: function () {
+ let me = this;
+ let reason = me.blockedReason();
+ let hint = me.down('#blockedReason');
+ hint.setValue(reason?.text ?? '');
+ hint.setHidden(!reason);
+ let button = me.down('#blockedAction');
+ button.setText(Ext.htmlEncode(reason?.actionText ?? ''));
+ button.setHidden(!reason?.action);
+ me.blockedAction = reason?.action;
+ // the wizard validates on field changes only, a reason found later must disable Next itself
+ let wizard = me.up('window');
+ if (wizard?.down('#wizcontent')?.getActiveTab() === me) {
+ wizard.down('#next').setDisabled(!!reason);
+ }
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ // the LVM branch reads LUNs and volumes before creating anything, check the privileges up
+ // front
+ let caps = Ext.state.Manager.get('GuiCap');
+ let lvmAllowed = !!(
+ caps.storage['Datastore.Allocate'] &&
+ (caps.storage['Datastore.Audit'] || caps.storage['Datastore.AllocateSpace'])
+ );
+
+ me.items = [
+ {
+ xtype: 'displayfield',
+ value: gettext('How do you want to use the SAN storage?'),
+ },
+ {
+ xtype: 'radiogroup',
+ columns: 1,
+ vertical: true,
+ items: [
+ {
+ boxLabel:
+ `<b>${gettext('Shared LVM on a LUN (recommended)')}</b> - ` +
+ gettext(
+ 'Create an LVM volume group on a LUN. Disk images are logical volumes in it, shared by the nodes of the storage.',
+ ),
+ name: 'usage',
+ inputValue: 'lvm',
+ checked: lvmAllowed,
+ disabled: !lvmAllowed,
+ margin: '0 0 10 0',
+ },
+ {
+ boxLabel:
+ `<b>${gettext('Use LUNs directly')}</b> - ` +
+ gettext(
+ 'Each LUN is used as one disk image. You create, resize, and remove the LUNs on the SAN.',
+ ),
+ name: 'usage',
+ inputValue: 'direct',
+ checked: !lvmAllowed,
+ },
+ ],
+ listeners: {
+ change: function (rg, value) {
+ rg.up('window').getViewModel().set('usage', value.usage);
+ me.updateBlockedReason();
+ },
+ // the view model defaults to lvm, which the privileges may rule out
+ afterrender: function (rg) {
+ rg.up('window').getViewModel().set('usage', rg.getValue().usage);
+ },
+ },
+ },
+ {
+ xtype: 'displayfield',
+ hidden: lvmAllowed,
+ value: PVE.Utils.note_text(
+ gettext(
+ 'Shared LVM requires Datastore.Allocate and either Datastore.Audit or Datastore.AllocateSpace on /storage.',
+ ),
+ ),
+ },
+ {
+ xtype: 'displayfield',
+ itemId: 'blockedReason',
+ userCls: 'pmx-hint',
+ hidden: true,
+ value: '',
+ },
+ {
+ xtype: 'button',
+ itemId: 'blockedAction',
+ hidden: true,
+ handler: function () {
+ me.blockedAction?.();
+ me.updateBlockedReason();
+ },
+ },
+ ];
+
+ me.callParent();
+
+ // the steps before may have changed when this one comes up again
+ me.on('activate', () => me.updateBlockedReason());
+ },
+});
+
+PVE.storage.wizard.reconcile.iscsi = {
+ keys: ['target', 'portal', 'multipath'],
+ booleans: ['multipath'],
+};
+
+PVE.storage.wizard.types.iscsi = {
+ text: 'iSCSI',
+ description: gettext('Block storage (LUNs) on a SAN, accessed over the network.'),
+ group: 'san',
+ apiType: 'iscsi',
+ viewModel: {
+ data: {
+ usage: 'lvm',
+ },
+ formulas: {
+ isLvmUsage: (get) => get('usage') === 'lvm',
+ },
+ },
+ steps: () => [
+ {
+ xtype: 'pveStorageWizardIScsiTarget',
+ title: gettext('Target'),
+ },
+ {
+ xtype: 'pveStorageWizardIScsiUsage',
+ title: gettext('Usage'),
+ },
+ ],
+ settings: {
+ onlineHelp: 'storage_open_iscsi',
+ cts: ['images', 'rootdir'],
+ defaultContent: ['images', 'rootdir'],
+ contentFieldConfig: {
+ bind: {
+ disabled: '{!isLvmUsage}',
+ hidden: '{!isLvmUsage}',
+ },
+ },
+ nodesFieldConfig: {
+ validator: PVE.storage.wizard.lvmNodesValidator,
+ },
+ advancedColumn1: [
+ {
+ xtype: 'textfield',
+ name: 'base-storage',
+ fieldLabel: gettext('iSCSI storage ID'),
+ emptyText: gettext('ID') + '-base',
+ vtype: 'StorageId',
+ allowBlank: true,
+ bind: {
+ disabled: '{!isLvmUsage}',
+ hidden: '{!isLvmUsage}',
+ },
+ },
+ ],
+ },
+ summaryFields: {
+ usage: {
+ label: gettext('Usage'),
+ values: {
+ lvm: gettext('Shared LVM on a LUN'),
+ direct: gettext('Use LUNs directly'),
+ },
+ weight: 10,
+ },
+ },
+
+ confirmCard: true,
+
+ actionText: function (wizard) {
+ if (wizard.getValues().usage !== 'lvm') {
+ return undefined;
+ }
+ return gettext('Connect and Select LUN');
+ },
+
+ summaryRows: function (values, wizard) {
+ if (values.usage !== 'lvm') {
+ return {};
+ }
+ let created = wizard.created ?? {};
+ let base = values['base-storage'] || `${values.storage}-base`;
+ if (created.baseStorage) {
+ base = Ext.String.format(gettext('{0} (created)'), created.baseStorage);
+ }
+ let rows = [
+ { label: gettext('LVM storage'), value: values.storage, weight: 20 },
+ { label: gettext('iSCSI storage'), value: base, weight: 21 },
+ {
+ label: 'LUN',
+ value: created.baseVolume ?? gettext('Selected in the next step'),
+ weight: 22,
+ },
+ ];
+ let skip = ['storage', 'base-storage'];
+ return { rows, skip };
+ },
+
+ summaryNotes: function (values, wizard) {
+ if (values.usage === 'lvm') {
+ let base = values['base-storage'] || `${values.storage}-base`;
+ return [
+ Ext.String.format(
+ gettext(
+ 'Press Connect and Select LUN to add the iSCSI storage "{0}" and list its LUNs. The volume group is created on the LUN you select next.',
+ ),
+ Ext.htmlEncode(base),
+ ),
+ ];
+ }
+ return [];
+ },
+
+ submit: function (wizard, values) {
+ let enc = Ext.htmlEncode;
+ let usage = values.usage;
+ delete values.usage;
+
+ if (usage === 'direct') {
+ delete values['base-storage'];
+ values.type = 'iscsi';
+ values.content = 'images';
+ wizard.runSubmitChain([(next) => wizard.createStorage(values, next)]);
+ return;
+ }
+
+ PVE.storage.wizard.submitSanLvm(wizard, values, {
+ sameIdText: gettext('The iSCSI storage and the LVM storage need different IDs.'),
+ volumeGroupText: gettext(
+ 'Press {2} to retry the remaining steps. If the volume group "{0}" already exists on the LUN, you can also complete the setup under Datacenter > Storage > Add > LVM, with base volume "{1}" and volume group "{0}".',
+ ),
+ sameUnitText: gettext(
+ 'Press {1} to try the same LUN again, for example after wiping it. The iSCSI storage "{0}" remains after closing the wizard. Complete its setup under Datacenter > Storage > Add > LVM.',
+ ),
+ // the LUNs of a target that another storage already logs in to are that storage's, and
+ // may be in use through it
+ checkTarget: (entries) => {
+ let other = entries.find(
+ (entry) => entry.type === 'iscsi' && entry.target === values.target,
+ );
+ return other
+ ? Ext.String.format(
+ gettext(
+ 'The iSCSI storage "{0}" already uses this target. Create the LVM storage on top of it under Datacenter > Storage > Add > LVM.',
+ ),
+ enc(other.storage),
+ )
+ : undefined;
+ },
+ baseParams: (baseId) => ({
+ storage: baseId,
+ type: 'iscsi',
+ portal: values.portal,
+ target: values.target,
+ content: 'none',
+ nodes: values.nodes,
+ }),
+ lunView: (node, baseId) => ({
+ xtype: 'pveStorageWizardLunSelect',
+ footnote: Ext.String.format(
+ gettext(
+ 'Closing the wizard keeps storage "{0}". Complete the setup under Datacenter > Storage > Add > LVM if the wizard is closed.',
+ ),
+ enc(baseId),
+ ),
+ belongs: (row) => row['iscsi-target'] === values.target,
+ connectionUrl: `/nodes/${node}/iscsi/target`,
+ connections: (rows) =>
+ rows
+ .filter(
+ (row) =>
+ row.target === values.target && row['session-id'] !== undefined,
+ )
+ .map((row) => row['session-id']),
+ rescanUrl: `/nodes/${node}/iscsi/rescan`,
+ rescanParam: 'session-id',
+ }),
+ verifyVolume: function ({ node, baseId, lun, pin }) {
+ let view = wizard.lunView;
+ let scan = view.scanGeneration;
+ let selection = view.selectionGeneration;
+ let current = () =>
+ !view.destroyed &&
+ scan === view.scanGeneration &&
+ selection === view.selectionGeneration;
+ let unverified = function (changed) {
+ if (!current()) {
+ return;
+ }
+ view.waitForLuns(false);
+ Ext.Msg.alert(
+ gettext('Error'),
+ changed
+ ? Ext.String.format(
+ gettext(
+ 'LUN {0} of the target is a different LUN now. Select the LUN again.',
+ ),
+ enc(lun['iscsi-lun']),
+ )
+ : gettext('Could not verify the selected LUN. Press Rescan.'),
+ );
+ };
+ if (!lun.wwid || lun.wwid === 'unknown') {
+ unverified(false);
+ return;
+ }
+ wizard.request({
+ url: `/nodes/${node}/storage/${baseId}/content`,
+ method: 'GET',
+ params: { content: 'images' },
+ failure: () => {
+ if (!current()) {
+ return;
+ }
+ unverified(false);
+ },
+ success: function (response) {
+ if (!current()) {
+ return;
+ }
+ // LUN numbers can be reassigned between requests. The submitted by-id
+ // name must encode the selected identity for the backend to check it.
+ let volume = (response.result.data || []).find((entry) => {
+ let [, , , number, name] =
+ entry.volid
+ .replace(/^[^:]+:/, '')
+ .match(/^(\d+)\.(\d+)\.(\d+)\.(.+)$/) ?? [];
+ return (
+ Number(number) === lun['iscsi-lun'] &&
+ (name === `dm-uuid-mpath-${lun.wwid}` ||
+ name === `scsi-${lun.wwid}`)
+ );
+ });
+ if (!volume) {
+ unverified(false);
+ return;
+ }
+ wizard.request({
+ url: `/nodes/${node}/scan/san-luns`,
+ method: 'GET',
+ failure: () => unverified(false),
+ success: function (fresh) {
+ if (!current()) {
+ return;
+ }
+ let rows = (fresh.result.data || []).filter(
+ (row) =>
+ row['iscsi-target'] === values.target &&
+ Number(row['iscsi-lun']) === Number(lun['iscsi-lun']),
+ );
+ if (
+ !rows.length ||
+ rows.some(
+ (row) =>
+ !row.wwid ||
+ row.wwid === 'unknown' ||
+ row['wwid-unknown'],
+ )
+ ) {
+ unverified(false);
+ } else if (rows.some((row) => row.wwid !== lun.wwid)) {
+ unverified(true);
+ } else {
+ pin(volume);
+ }
+ },
+ });
+ },
+ });
+ },
+ });
+ },
+};
diff --git a/www/manager6/storage/wizard/LunSelect.js b/www/manager6/storage/wizard/LunSelect.js
new file mode 100644
index 000000000..270daaa73
--- /dev/null
+++ b/www/manager6/storage/wizard/LunSelect.js
@@ -0,0 +1,446 @@
+// The node re-checks the LUN before creating the volume group, so selecting it is the only
+// confirmation: no double click or Enter submit.
+Ext.define('PVE.storage.wizard.LunSelect', {
+ extend: 'Ext.panel.Panel',
+ xtype: 'pveStorageWizardLunSelect',
+ mixins: ['PVE.storage.wizard.LunPicker'],
+
+ border: false,
+ layout: { type: 'vbox', align: 'stretch' },
+
+ node: undefined,
+ storage: undefined,
+ belongs: undefined,
+ multipath: false,
+ initiator: undefined,
+ recap: undefined,
+ footnote: undefined,
+ connectionUrl: undefined,
+ connections: undefined,
+ rescanUrl: undefined,
+ rescanParam: undefined,
+
+ tabTitle: gettext('LUN'),
+
+ selectText: gettext('Select an unused LUN for the volume group.'),
+ waitText: gettext('Waiting for node "{0}" to list the LUNs of the target...'),
+ errorText: gettext('Could not list the LUNs of the target on node "{0}": {1}'),
+ selectedText: gettext('The volume group will be created on LUN {0} ({1} {2}, {3}).'),
+ pinnedText: gettext(
+ 'Press {4} to retry on LUN {0} ({1} {2}, {3}). The wizard keeps this LUN, as the previous attempt may have written to it.',
+ ),
+ noneText: gettext(
+ 'No unused LUN on the target. Present an empty LUN to initiator "{0}", then press Rescan.',
+ ),
+ noSessionText: gettext(
+ 'Node "{0}" has no session to the target. Check that the target allows initiator "{1}" and that a portal is reachable, then press Rescan.',
+ ),
+ activationErrorText: gettext('Activating storage "{0}" failed: {1}'),
+ activationUnknownText: gettext('The outcome of activating storage "{0}" is unknown: {1}'),
+
+ // not one usable device yet: no identity, or with multipath a path still waiting for its map
+ assembling: function (row) {
+ let me = this;
+ if (!row.wwid || row.wwid === 'unknown') {
+ return true;
+ }
+ if (!me.multipath) {
+ return false;
+ }
+ return (
+ row.multipath === 'pending' ||
+ (row.multipath === 'none' && row['multipath-storage'] === me.storage)
+ );
+ },
+
+ // I/O to a map without a usable path could stay queued indefinitely
+ notReady: function (row) {
+ let me = this;
+ if (me.assembling(row)) {
+ return gettext('Not ready');
+ }
+ let mapped = ['storage', 'manual', 'unmanaged', 'kept'].includes(row.multipath);
+ if (me.multipath && !mapped) {
+ return gettext('Not ready');
+ }
+ if (mapped && !(row['paths-active'] > 0)) {
+ return gettext('No usable path');
+ }
+ return undefined;
+ },
+
+ getSelection: function () {
+ return this.down('#lunGrid').getSelection()[0]?.data;
+ },
+
+ canSubmit: function () {
+ let me = this;
+ return (
+ !me.scanning &&
+ !me.scanError &&
+ (!!me.up('window').created?.baseVolume || !!me.getSelection()?.selectable)
+ );
+ },
+
+ updateButtons: function () {
+ this.up('window').updateActions();
+ },
+
+ updateState: function () {
+ let me = this;
+ let wizard = me.up('window');
+ let pinned = wizard.created?.baseVolume;
+ let grid = me.down('#lunGrid');
+ let rec = grid.getSelection()[0];
+ let row = pinned ? wizard.created.lun : rec?.data;
+ let blocked = wizard.busy;
+ grid.setDisabled(!!pinned || blocked || me.scanning || me.scanError);
+ me.down('#rescan').setHidden(!!pinned);
+ me.down('#rescan').setDisabled(blocked || me.scanning);
+ me.down('#footnote').setHidden(!!pinned);
+ let text;
+ if (wizard.unconfirmedStorage) {
+ text = wizard.storageUnknownText(false);
+ } else if (pinned) {
+ text = me.pinnedText;
+ } else if (me.scanning) {
+ text = Ext.String.format(me.waitText, me.node);
+ } else if (me.scanError) {
+ text = '';
+ } else if (row) {
+ text = me.selectedText;
+ } else if (me.activationOutcome && !grid.getStore().getCount()) {
+ let outcome = me.activationOutcome;
+ text = Ext.String.format(
+ outcome.uncertain ? me.activationUnknownText : me.activationErrorText,
+ me.storage,
+ outcome.message,
+ );
+ } else if (me.noSession) {
+ text = Ext.String.format(
+ me.noSessionText,
+ me.node,
+ me.initiator || Proxmox.Utils.unknownText,
+ );
+ } else if (
+ !grid
+ .getStore()
+ .getRange()
+ .some((r) => r.get('selectable'))
+ ) {
+ text = Ext.String.format(me.noneText, me.initiator || Proxmox.Utils.unknownText);
+ } else {
+ text = me.selectText;
+ }
+ if (row && !wizard.unconfirmedStorage && !me.scanning && !me.scanError) {
+ text = Ext.String.format(
+ text,
+ row['iscsi-lun'],
+ 'WWID',
+ row.wwid,
+ Proxmox.Utils.format_size(row.size),
+ wizard.actionText(),
+ );
+ }
+ me.down('#stateLine').setValue(Ext.htmlEncode(text || ''));
+ },
+
+ showScanError: function (errorHtml) {
+ let me = this;
+ me.scanning = false;
+ me.scanError = true;
+ let grid = me.down('#lunGrid');
+ grid.setLoading(false);
+ Proxmox.Utils.setErrorMask(
+ grid,
+ Ext.String.format(me.errorText, Ext.htmlEncode(me.node), errorHtml),
+ );
+ me.updateButtons();
+ },
+
+ scanGeneration: 0,
+ selectionGeneration: 0,
+
+ invalidateScan: function () {
+ this.scanGeneration++;
+ },
+
+ initComponent: function () {
+ let me = this;
+
+ if (!me.storage || !me.node || !me.belongs) {
+ throw 'no storage, node or filter specified';
+ }
+
+ let columns = me.lunColumns();
+ let store = Ext.create('Ext.data.Store', {
+ fields: [
+ ...PVE.SANUtils.lun_fields,
+ 'wwid',
+ 'iscsi-lun',
+ 'reason',
+ {
+ name: 'selectable',
+ depends: ['reason', 'usage', 'used-by-storage'],
+ calculate: (data) =>
+ !data.reason &&
+ data.usage === 'unused' &&
+ (!data['used-by-storage'] || data['used-by-storage'] === me.storage),
+ },
+ ],
+ sorters: [{ property: 'iscsi-lun' }, { property: 'devpath' }],
+ });
+
+ me.items = [
+ {
+ xtype: 'displayfield',
+ value: me.recap,
+ },
+ {
+ xtype: 'displayfield',
+ itemId: 'stateLine',
+ },
+ {
+ xtype: 'grid',
+ itemId: 'lunGrid',
+ flex: 1,
+ store: store,
+ viewConfig: {
+ trackOver: false,
+ getRowClass: (rec) => (rec.get('selectable') ? '' : 'x-item-disabled'),
+ },
+ listeners: {
+ beforeselect: (grid, rec) => rec.get('selectable'),
+ selectionchange: function () {
+ me.selectionGeneration++;
+ me.updateButtons();
+ },
+ },
+ columns: [
+ {
+ header: 'LUN',
+ dataIndex: 'iscsi-lun',
+ width: 60,
+ },
+ {
+ header: gettext('Device'),
+ dataIndex: 'devpath',
+ flex: 2,
+ renderer: Ext.htmlEncode,
+ },
+ columns.paths,
+ {
+ header: 'WWID',
+ dataIndex: 'wwid',
+ flex: 3,
+ renderer: Ext.String.htmlEncode,
+ },
+ columns.size,
+ columns.model,
+ {
+ header: gettext('Usage'),
+ dataIndex: 'usage',
+ flex: 2,
+ renderer: function (value, metaData, rec) {
+ if (rec.get('reason')) {
+ let text = Ext.htmlEncode(rec.get('reason'));
+ metaData.tdAttr = `data-qtip="${Ext.htmlEncode(text)}"`;
+ return text;
+ }
+ let { text, tip } = PVE.SANUtils.san_lun_usage(rec.data, {
+ // every LUN of the storage carries its attribution
+ ownStorage: me.storage,
+ });
+ metaData.tdAttr = `data-qtip="${Ext.htmlEncode(tip)}"`;
+ return text;
+ },
+ },
+ ],
+ },
+ ];
+
+ let controls = [
+ {
+ xtype: 'button',
+ itemId: 'rescan',
+ text: gettext('Rescan'),
+ iconCls: 'fa fa-search',
+ handler: () => me.doRescan(),
+ },
+ ];
+ me.items.push({ xtype: 'container', layout: 'hbox', margin: '5 0', items: controls });
+ me.items.push({ xtype: 'displayfield', itemId: 'footnote', value: me.footnote });
+
+ me.callParent();
+ me.on('beforedestroy', () => me.invalidateScan());
+ },
+
+ prepareScan: function (deadline, current, done) {
+ let me = this;
+ let timeout = () => Math.max(1, deadline - Date.now());
+ let fail = (response) => current() && me.showScanError(response.htmlStatus);
+ let enumerate = function () {
+ if (!current()) {
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: me.connectionUrl,
+ method: 'GET',
+ timeout: timeout(),
+ failure: fail,
+ success: function (response) {
+ if (!current()) {
+ return;
+ }
+ let connections = me.connections(response.result.data || []);
+ me.noSession = connections.length === 0;
+ let caps = Ext.state.Manager.get('GuiCap');
+ if (!caps.dc['Sys.Modify']) {
+ done();
+ return;
+ }
+ let next = function () {
+ if (!current()) {
+ return;
+ }
+ if (!connections.length || Date.now() >= deadline) {
+ done();
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: me.rescanUrl,
+ method: 'POST',
+ timeout: timeout(),
+ params: { [me.rescanParam]: connections.shift() },
+ failure: fail,
+ success: next,
+ });
+ };
+ next();
+ },
+ });
+ };
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${me.node}/storage/${encodeURIComponent(me.storage)}/content`,
+ method: 'GET',
+ timeout: Math.min(15000, timeout()),
+ // listing the content activates the storage; a failure can explain an empty scan but
+ // does not prove one
+ failure: function (response) {
+ if (current()) {
+ me.activationOutcome = {
+ uncertain: PVE.SANUtils.request_uncertain(response),
+ // htmlStatus is HTML, the state line encodes plain text
+ message:
+ response.result?.message?.trim() ||
+ Ext.htmlDecode(response.htmlStatus ?? '') ||
+ Proxmox.Utils.unknownText,
+ };
+ }
+ enumerate();
+ },
+ success: enumerate,
+ });
+ },
+
+ // a rescan never preselects, a LUN appearing then may not be the intended one
+ waitForLuns: function (preselect) {
+ let me = this;
+ let grid = me.down('#lunGrid');
+ let generation = ++me.scanGeneration;
+ let current = () => !me.destroyed && generation === me.scanGeneration;
+ let deadline = Date.now() + PVE.storage.wizard.scanTimeout;
+ let previous;
+ let lastRows = [];
+ let settle = function (settled) {
+ grid.setLoading(false);
+ me.scanning = false;
+ me.showRows(lastRows, settled && preselect && !me.preselectionConsumed);
+ me.preselectionConsumed = true;
+ };
+ me.noSession = false;
+ me.activationOutcome = undefined;
+ me.scanning = true;
+ me.scanError = false;
+ grid.getSelectionModel().deselectAll();
+ Proxmox.Utils.setErrorMask(grid, false);
+ grid.setLoading(Ext.String.format(me.waitText, Ext.htmlEncode(me.node)));
+ me.updateButtons();
+ let poll = function () {
+ if (!current()) {
+ return;
+ }
+ if (Date.now() >= deadline) {
+ settle();
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${me.node}/scan/san-luns`,
+ method: 'GET',
+ // a scan can block on a map without a usable path, it must not outlast the wait
+ timeout: Math.max(deadline - Date.now(), 1),
+ failure: function (response) {
+ if (!current()) {
+ return;
+ }
+ me.showScanError(response.htmlStatus);
+ },
+ success: function (response) {
+ if (!current()) {
+ return;
+ }
+ let rows = PVE.SANUtils.group_san_lun_paths(response.result.data || []).filter(
+ (row) => me.belongs(row),
+ );
+ lastRows = rows;
+ let signature = JSON.stringify(
+ rows
+ .map((row) => [
+ row.devpath,
+ row.multipath,
+ row['paths-active'],
+ row.usage,
+ ])
+ .sort(),
+ );
+ let settled =
+ rows.length > 0 &&
+ signature === previous &&
+ !rows.some((row) => me.assembling(row));
+ previous = signature;
+ if (settled || Date.now() >= deadline) {
+ settle(settled);
+ } else {
+ Ext.defer(
+ poll,
+ Math.min(
+ PVE.storage.wizard.scanInterval,
+ Math.max(1, deadline - Date.now()),
+ ),
+ );
+ }
+ },
+ });
+ };
+ me.prepareScan(deadline, current, poll);
+ },
+
+ showRows: function (rows, preselect) {
+ let me = this;
+ let grid = me.down('#lunGrid');
+ let store = grid.getStore();
+ store.setData(rows.map((row) => ({ ...row, reason: me.notReady(row) })));
+ let selectable = store.getRange().filter((rec) => rec.get('selectable'));
+ if (preselect && selectable.length === 1) {
+ grid.getSelectionModel().select(selectable[0]);
+ }
+ me.updateButtons();
+ grid.focus();
+ },
+
+ doRescan: function () {
+ let me = this;
+ me.preselectionConsumed = true;
+ me.waitForLuns(false);
+ },
+});
diff --git a/www/manager6/storage/wizard/SanLvm.js b/www/manager6/storage/wizard/SanLvm.js
new file mode 100644
index 000000000..657b75e45
--- /dev/null
+++ b/www/manager6/storage/wizard/SanLvm.js
@@ -0,0 +1,114 @@
+PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
+ let enc = Ext.htmlEncode;
+ let created = (wizard.created ??= {});
+ let baseId = values['base-storage'] || `${values.storage}-base`;
+ if (baseId === values.storage) {
+ Ext.Msg.alert(gettext('Error'), transport.sameIdText);
+ return;
+ }
+
+ // the storage API creates the volume group on the node it runs on
+ let node = Proxmox.NodeName;
+ let nodesList = values.nodes ? [].concat(values.nodes).join(',').split(',') : [];
+ nodesList = nodesList.filter((entry) => entry !== '');
+ if (wizard.getScanNode() !== node || (nodesList.length && !nodesList.includes(node))) {
+ Ext.Msg.alert(
+ gettext('Error'),
+ Ext.String.format(
+ gettext(
+ 'The volume group is created from node "{0}", which this web UI is connected to. Scan the target from that node and include it in the node restriction, or open the web UI on a node that should do the setup.',
+ ),
+ enc(node),
+ ),
+ );
+ return;
+ }
+
+ let volumeGroupHint = () =>
+ Ext.String.format(
+ transport.volumeGroupText,
+ enc(values.storage),
+ enc(created.baseVolume),
+ enc(wizard.actionText()),
+ ) +
+ '<br><br>' +
+ Ext.String.format(transport.sameUnitText, enc(baseId), enc(wizard.actionText()));
+
+ let selected;
+
+ wizard.runSubmitChain([
+ (next) => {
+ if (created.baseStorage) {
+ next();
+ return;
+ }
+ wizard.assertStorageIdsFree([baseId, values.storage], next, transport.checkTarget);
+ },
+ (next) => {
+ if (created.baseStorage) {
+ next();
+ return;
+ }
+ wizard.createStorage(transport.baseParams(baseId), () => {
+ created.baseStorage = baseId;
+ wizard.freeze();
+ wizard.updateSummary();
+ next();
+ });
+ },
+ (next) => {
+ if (created.baseVolume) {
+ next();
+ return;
+ }
+ let active = wizard.activateLunView({
+ ...transport.lunView(node, baseId),
+ recap: Ext.String.format(
+ gettext(
+ 'Connection storage "{1}" created. Setting up volume group and LVM storage "{0}".',
+ ),
+ enc(values.storage),
+ enc(baseId),
+ ),
+ node: node,
+ storage: baseId,
+ });
+ if (active && wizard.canSubmit()) {
+ selected = wizard.lunView.getSelection();
+ next();
+ }
+ },
+ (next) => {
+ if (created.baseVolume) {
+ next();
+ return;
+ }
+ transport.verifyVolume({
+ node,
+ baseId,
+ lun: selected,
+ pin: function (volume) {
+ created.baseVolume = volume.volid;
+ created.lun = Ext.clone(selected);
+ wizard.updateSummary();
+ next();
+ },
+ });
+ },
+ (next) =>
+ wizard.createStorage(
+ {
+ storage: values.storage,
+ type: 'lvm',
+ vgname: values.storage,
+ base: created.baseVolume,
+ shared: 1,
+ content: values.content,
+ nodes: values.nodes,
+ disable: values.disable,
+ },
+ next,
+ volumeGroupHint(),
+ ),
+ ]);
+};
diff --git a/www/manager6/storage/wizard/Wizard.js b/www/manager6/storage/wizard/Wizard.js
index ce0eccb73..c5ffa2b8e 100644
--- a/www/manager6/storage/wizard/Wizard.js
+++ b/www/manager6/storage/wizard/Wizard.js
@@ -23,6 +23,10 @@ PVE.storage.wizard.summaryFields = {
options: { label: gettext('Mount Options') },
};
+// the first scans can be empty until pvestatd logs in and applies the multipath allow-list
+PVE.storage.wizard.scanTimeout = 45000;
+PVE.storage.wizard.scanInterval = 3000;
+
PVE.storage.wizard.lvmNodesValidator = function () {
let wizard = this.up('window');
if (wizard?.getViewModel().get('usage') !== 'lvm') {
@@ -158,7 +162,7 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
if (canSubmit) {
return canSubmit(me);
}
- return true;
+ return !me.lunView || me.lunView.canSubmit();
},
actionText: function () {
@@ -166,7 +170,10 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
if (me.unconfirmedStorage) {
return gettext('Check and Retry');
}
- return gettext('Finish');
+ if (me.lunView) {
+ return me.created?.baseVolume ? gettext('Retry Setup') : gettext('Create Volume Group');
+ }
+ return PVE.storage.wizard.types[me.wizardType].actionText?.(me) ?? gettext('Finish');
},
updateActions: function () {
@@ -175,6 +182,11 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
return;
}
me.down('#submit').setText(me.actionText());
+ if (me.lunView) {
+ me.lunView.updateState();
+ me.down('#submit').setDisabled(!me.canSubmit());
+ me.down('#back').setDisabled(true);
+ }
// a type with its own submit condition decides the state of the confirm buttons
let confirmActive = me.down('#wizcontent').getActiveTab() === me.down('#confirm');
if (PVE.storage.wizard.types[me.wizardType].canSubmit && confirmActive) {
@@ -188,6 +200,20 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
}
},
+ activateLunView: function (config) {
+ let me = this;
+ if (me.lunView) {
+ return true;
+ }
+ let panel = me.down('#confirm');
+ me.lunView = panel.add(config);
+ panel.getLayout().setActiveItem(me.lunView);
+ panel.setTitle(me.lunView.tabTitle);
+ me.lunView.waitForLuns(true);
+ me.updateActions();
+ return false;
+ },
+
// keeps the window busy, so it cannot close with the continuation pending
request: function (options) {
let me = this;
@@ -572,7 +598,7 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
data.push({ key: row.label, value: row.value, weight: row.weight ?? 20 });
}
- let summarystore = panel.down('grid').getStore();
+ let summarystore = panel.down('#summaryGrid').getStore();
summarystore.suspendEvents();
summarystore.removeAll();
summarystore.add(data);
@@ -673,6 +699,7 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
items: [
{
xtype: 'grid',
+ itemId: 'summaryGrid',
flex: 1,
store: {
fields: ['key', 'value', 'weight'],
@@ -730,6 +757,18 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
},
];
+ if (entry.confirmCard) {
+ let confirm = me.items[me.items.length - 1];
+ confirm.items = [
+ {
+ xtype: 'container',
+ layout: confirm.layout,
+ items: confirm.items,
+ },
+ ];
+ confirm.layout = 'card';
+ }
+
me.on('beforeclose', function () {
if (!me.busy) {
return true;
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 68/76] ui: storage wizard: iscsi: show cluster initiator names
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (66 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 67/76] ui: storage wizard: add iSCSI support Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 69/76] ui: storage wizard: iscsi: enable multipath for shared LVM Thomas Lamprecht
` (7 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Show initiator names on the Target step so administrators can configure
access rules without reading /etc/iscsi/initiatorname.iscsi on each
node. Use node status reports and offer copying one name per line.
Include all cluster nodes, marking missing or stale reports, because
storage node restrictions are chosen later.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Makefile | 1 +
www/manager6/storage/wizard/IScsi.js | 7 +
www/manager6/storage/wizard/NodeIdentities.js | 123 ++++++++++++++++++
www/manager6/storage/wizard/SanLvm.js | 1 +
4 files changed, 132 insertions(+)
create mode 100644 www/manager6/storage/wizard/NodeIdentities.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index a90d5e0fd..cd64e2366 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -389,6 +389,7 @@ JSSRC= \
storage/wizard/CommonSettings.js \
storage/wizard/LunPicker.js \
storage/wizard/LunSelect.js \
+ storage/wizard/NodeIdentities.js \
storage/wizard/SanLvm.js \
storage/wizard/NFS.js \
storage/wizard/CIFS.js \
diff --git a/www/manager6/storage/wizard/IScsi.js b/www/manager6/storage/wizard/IScsi.js
index fb4decb38..57cbe3243 100644
--- a/www/manager6/storage/wizard/IScsi.js
+++ b/www/manager6/storage/wizard/IScsi.js
@@ -34,6 +34,13 @@ Ext.define('PVE.storage.wizard.IScsiTarget', {
},
];
+ me.columnB = [
+ {
+ xtype: 'pveStorageWizardNodeIdentities',
+ kind: 'iscsi',
+ },
+ ];
+
me.column2 = [];
if (!PVE.Utils.isStandaloneNode()) {
me.column2.push({
diff --git a/www/manager6/storage/wizard/NodeIdentities.js b/www/manager6/storage/wizard/NodeIdentities.js
new file mode 100644
index 000000000..21b89900a
--- /dev/null
+++ b/www/manager6/storage/wizard/NodeIdentities.js
@@ -0,0 +1,123 @@
+Ext.define('PVE.storage.wizard.NodeIdentities', {
+ extend: 'Ext.grid.Panel',
+ xtype: 'pveStorageWizardNodeIdentities',
+
+ kind: 'iscsi',
+
+ // five nodes fit without scrolling
+ height: 200,
+ disableSelection: true,
+
+ sources: {
+ iscsi: {
+ url: '/cluster/multipath/status',
+ field: 'iscsi-initiator-name',
+ header: gettext('Initiator Name'),
+ allowText: gettext(
+ 'The target must allow the initiator names of the nodes that use this storage.',
+ ),
+ copyTip: gettext('Copy the known initiator names, one per line'),
+ },
+ },
+
+ getIdentity: function (node) {
+ return this.getStore().findRecord('node', node, 0, false, true, true)?.get('identity');
+ },
+
+ load: function () {
+ let me = this;
+ let source = me.sources[me.kind];
+ Proxmox.Utils.API2Request({
+ url: source.url,
+ method: 'GET',
+ failure: (response) =>
+ !me.destroyed && Proxmox.Utils.setErrorMask(me, response.htmlStatus),
+ success: function (response) {
+ if (me.destroyed) {
+ return;
+ }
+ Proxmox.Utils.setErrorMask(me, false);
+ let reports = response.result.data?.nodes || {};
+ // every node, the nodes of the storage are only chosen on a later step
+ let nodes = new Set([
+ ...PVE.data.ResourceStore.getNodes().map((node) => node.node),
+ ...Object.keys(reports),
+ ]);
+ me.getStore().setData(
+ [...nodes].sort().map((node) => {
+ let report = reports[node];
+ let current = report && !['offline', 'stale'].includes(report.reason);
+ return {
+ node,
+ identity: report?.[source.field],
+ note: !report
+ ? gettext('No report')
+ : !current
+ ? PVE.SANUtils.render_multipath_reason(report.reason)
+ : undefined,
+ };
+ }),
+ );
+ },
+ });
+ },
+
+ initComponent: function () {
+ let me = this;
+ let source = me.sources[me.kind];
+
+ me.store = Ext.create('Ext.data.Store', {
+ fields: ['node', 'identity', 'note'],
+ });
+
+ me.columns = [
+ {
+ header: gettext('Node'),
+ dataIndex: 'node',
+ width: 120,
+ },
+ {
+ header: source.header,
+ dataIndex: 'identity',
+ flex: 1,
+ renderer: function (value, metaData, rec) {
+ let note = rec.get('note');
+ if (!value) {
+ return `<span class="faded">${Ext.htmlEncode(note ?? Proxmox.Utils.unknownText)}</span>`;
+ }
+ if (note) {
+ let qtip = Ext.htmlEncode(note);
+ return `<i class="fa fa-exclamation-triangle warning" data-qtip="${qtip}"></i> ${Ext.htmlEncode(value)}`;
+ }
+ return Ext.htmlEncode(value);
+ },
+ },
+ ];
+
+ me.tbar = [
+ {
+ xtype: 'displayfield',
+ value: source.allowText,
+ },
+ '->',
+ {
+ text: gettext('Copy'),
+ iconCls: 'fa fa-clipboard',
+ tooltip: source.copyTip,
+ handler: function () {
+ let identities = me
+ .getStore()
+ .getRange()
+ .map((rec) => rec.get('identity'))
+ .filter((identity) => !!identity);
+ if (identities.length) {
+ PVE.Utils.copyToClipboard(identities.join('\n'));
+ }
+ },
+ },
+ ];
+
+ me.callParent();
+ me.load();
+ },
+});
diff --git a/www/manager6/storage/wizard/SanLvm.js b/www/manager6/storage/wizard/SanLvm.js
index 657b75e45..85939c3b2 100644
--- a/www/manager6/storage/wizard/SanLvm.js
+++ b/www/manager6/storage/wizard/SanLvm.js
@@ -72,6 +72,7 @@ PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
),
node: node,
storage: baseId,
+ initiator: wizard.down('pveStorageWizardNodeIdentities')?.getIdentity(node),
});
if (active && wizard.canSubmit()) {
selected = wizard.lunView.getSelection();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 69/76] ui: storage wizard: iscsi: enable multipath for shared LVM
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (67 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 68/76] ui: storage wizard: iscsi: show cluster initiator names Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 70/76] ui: node: san: wipe a LUN that no storage uses Thomas Lamprecht
` (6 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
A target with several portals exposes the same physical volume on
several devices unless multipath combines them. Offer multipath and
enable it by default for such targets. Refuse shared LVM without it when
several portals are known.
Pass the option to the connection storage and wait for a usable map in
the LUN selector. Pin the map identity when creating the volume group,
and explain single-path use when direct LUN access leaves multipath
disabled.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
test/StorageWizard_test.js | 11 +++
www/manager6/storage/wizard/IScsi.js | 131 ++++++++++++++++++++++++--
www/manager6/storage/wizard/SanLvm.js | 7 ++
3 files changed, 143 insertions(+), 6 deletions(-)
diff --git a/test/StorageWizard_test.js b/test/StorageWizard_test.js
index 48305ccb3..05201e572 100644
--- a/test/StorageWizard_test.js
+++ b/test/StorageWizard_test.js
@@ -86,15 +86,26 @@ function submit({ name, multipath = false, freshWWID = selectedWWID, lun = 7 })
for (const [label, params, accepted] of [
['plain path with the selected identity', { name: `scsi-${selectedWWID}` }, true],
['unmanaged map with the selected identity', { name: `dm-uuid-mpath-${selectedWWID}` }, true],
+ [
+ 'managed map with the selected identity',
+ { name: `dm-uuid-mpath-${selectedWWID}`, multipath: true },
+ true,
+ ],
[
'different content identity despite a matching fresh scan',
{ name: `scsi-${otherWWID}` },
false,
],
['different unmanaged map identity', { name: `dm-uuid-mpath-${otherWWID}` }, false],
+ [
+ 'different managed map identity',
+ { name: `dm-uuid-mpath-${otherWWID}`, multipath: true },
+ false,
+ ],
['opaque alias', { name: 'dm-name-array-lun' }, false],
['partition of the selected LUN', { name: `scsi-${selectedWWID}-part1` }, false],
['different LUN number', { name: `scsi-${selectedWWID}`, lun: 8 }, false],
+ ['path rather than a managed map', { name: `scsi-${selectedWWID}`, multipath: true }, false],
[
'identity changed in the fresh scan',
{ name: `scsi-${selectedWWID}`, freshWWID: otherWWID },
diff --git a/www/manager6/storage/wizard/IScsi.js b/www/manager6/storage/wizard/IScsi.js
index 57cbe3243..9bab76cd0 100644
--- a/www/manager6/storage/wizard/IScsi.js
+++ b/www/manager6/storage/wizard/IScsi.js
@@ -4,6 +4,44 @@ Ext.define('PVE.storage.wizard.IScsiTarget', {
onlineHelp: 'storage_open_iscsi',
+ // set once the user picked a multipath setting, the default follows the discovery until then
+ multipathChosen: false,
+
+ targetPortals: undefined,
+
+ updateMultipathDefault: function (targetField, target) {
+ let me = this;
+ let rec = target
+ ? targetField.getStore().findRecord('target', target, 0, false, true, true)
+ : undefined;
+ let portals = rec
+ ? new Set(
+ `${rec.get('portal') ?? ''}`
+ .split(',')
+ .map((portal) => portal.trim())
+ .filter((portal) => portal !== ''),
+ ).size
+ : 0;
+ me.down('field[name=multipath]').setBoxLabel(
+ portals > 1
+ ? Ext.String.format(
+ ngettext(
+ 'Recommended: the target has {0} portal',
+ 'Recommended: the target has {0} portals',
+ portals,
+ ),
+ portals,
+ )
+ : '',
+ );
+ me.targetPortals = portals;
+ if (!me.multipathChosen) {
+ me.settingMultipathDefault = true;
+ me.down('field[name=multipath]').setValue(portals > 1);
+ me.settingMultipathDefault = false;
+ }
+ },
+
initComponent: function () {
let me = this;
@@ -31,10 +69,32 @@ Ext.define('PVE.storage.wizard.IScsiTarget', {
fieldLabel: gettext('Target'),
disabled: true,
showNodeSelector: false,
+ listeners: {
+ change: (f, value) => me.updateMultipathDefault(f, value),
+ },
},
];
me.columnB = [
+ {
+ xtype: 'proxmoxcheckbox',
+ name: 'multipath',
+ uncheckedValue: 0,
+ fieldLabel: gettext('Multipath'),
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext(
+ 'Combine the paths of each LUN into one device on every node using the storage.',
+ ),
+ },
+ listeners: {
+ change: function () {
+ if (!me.settingMultipathDefault) {
+ me.multipathChosen = true;
+ }
+ },
+ },
+ },
{
xtype: 'pveStorageWizardNodeIdentities',
kind: 'iscsi',
@@ -80,7 +140,17 @@ Ext.define('PVE.storage.wizard.IScsiUsage', {
action: () => wizard.down('pveStorageScanNodeSelector')?.setValue(Proxmox.NodeName),
};
}
- return undefined;
+ // without multipath LVM sees the PV once per portal and ignores the volume group
+ let field = wizard.down('field[name=multipath]');
+ let portals = wizard.down('pveStorageWizardIScsiTarget').targetPortals;
+ if (portals < 2 || field.getValue()) {
+ return undefined;
+ }
+ return {
+ text: gettext('Shared LVM on a target with several portals needs multipath.'),
+ actionText: gettext('Enable Multipath'),
+ action: () => field.setValue(true),
+ };
},
validator: function () {
@@ -303,6 +373,14 @@ PVE.storage.wizard.types.iscsi = {
),
];
}
+ let portals = wizard.down('pveStorageWizardIScsiTarget').targetPortals;
+ if (!values.multipath && portals > 1 && values.usage !== 'lvm') {
+ return [
+ gettext(
+ 'Without multipath, every LUN appears once per portal and a disk image uses only one of its paths.',
+ ),
+ ];
+ }
return [];
},
@@ -313,14 +391,28 @@ PVE.storage.wizard.types.iscsi = {
if (usage === 'direct') {
delete values['base-storage'];
+ if (!values.multipath) {
+ delete values.multipath;
+ }
values.type = 'iscsi';
values.content = 'images';
wizard.runSubmitChain([(next) => wizard.createStorage(values, next)]);
return;
}
+ let multipath = !!values.multipath;
+ let portals = wizard.down('pveStorageWizardIScsiTarget').targetPortals;
PVE.storage.wizard.submitSanLvm(wizard, values, {
sameIdText: gettext('The iSCSI storage and the LVM storage need different IDs.'),
+ refusal:
+ !multipath && portals > 1
+ ? gettext(
+ 'The target has several portals, so shared LVM on it needs multipath.',
+ )
+ : undefined,
+ baseExistsText: gettext(
+ 'The iSCSI storage "{0}" exists. Press {1} to retry the remaining steps, or complete the setup under Datacenter > Storage > Add > LVM.',
+ ),
volumeGroupText: gettext(
'Press {2} to retry the remaining steps. If the volume group "{0}" already exists on the LUN, you can also complete the setup under Datacenter > Storage > Add > LVM, with base volume "{1}" and volume group "{0}".',
),
@@ -349,6 +441,8 @@ PVE.storage.wizard.types.iscsi = {
target: values.target,
content: 'none',
nodes: values.nodes,
+ // unset rather than 0, a node with an older release does not know it
+ multipath: multipath ? 1 : undefined,
}),
lunView: (node, baseId) => ({
xtype: 'pveStorageWizardLunSelect',
@@ -358,6 +452,7 @@ PVE.storage.wizard.types.iscsi = {
),
enc(baseId),
),
+ multipath: multipath,
belongs: (row) => row['iscsi-target'] === values.target,
connectionUrl: `/nodes/${node}/iscsi/target`,
connections: (rows) =>
@@ -370,7 +465,7 @@ PVE.storage.wizard.types.iscsi = {
rescanUrl: `/nodes/${node}/iscsi/rescan`,
rescanParam: 'session-id',
}),
- verifyVolume: function ({ node, baseId, lun, pin }) {
+ verifyVolume: function ({ node, baseId, lun, retryHint, pin }) {
let view = wizard.lunView;
let scan = view.scanGeneration;
let selection = view.selectionGeneration;
@@ -403,11 +498,18 @@ PVE.storage.wizard.types.iscsi = {
url: `/nodes/${node}/storage/${baseId}/content`,
method: 'GET',
params: { content: 'images' },
- failure: () => {
+ failure: (response) => {
if (!current()) {
return;
}
- unverified(false);
+ if (!multipath) {
+ unverified(false);
+ } else {
+ Ext.Msg.alert(
+ gettext('Error'),
+ `${response.htmlStatus}<br><br>${retryHint()}`,
+ );
+ }
},
success: function (response) {
if (!current()) {
@@ -423,11 +525,28 @@ PVE.storage.wizard.types.iscsi = {
return (
Number(number) === lun['iscsi-lun'] &&
(name === `dm-uuid-mpath-${lun.wwid}` ||
- name === `scsi-${lun.wwid}`)
+ (!multipath && name === `scsi-${lun.wwid}`))
);
});
if (!volume) {
- unverified(false);
+ if (!multipath) {
+ unverified(false);
+ return;
+ }
+ Ext.Msg.alert(
+ gettext('Error'),
+ Ext.String.format(
+ gettext(
+ 'The iSCSI storage "{0}" does not provide a volume for the selected LUN ({1}).',
+ ),
+ enc(baseId),
+ enc(lun.wwid),
+ ) + `<br><br>${retryHint()}`,
+ );
+ return;
+ }
+ if (multipath) {
+ pin(volume);
return;
}
wizard.request({
diff --git a/www/manager6/storage/wizard/SanLvm.js b/www/manager6/storage/wizard/SanLvm.js
index 85939c3b2..054725753 100644
--- a/www/manager6/storage/wizard/SanLvm.js
+++ b/www/manager6/storage/wizard/SanLvm.js
@@ -6,6 +6,10 @@ PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
Ext.Msg.alert(gettext('Error'), transport.sameIdText);
return;
}
+ if (transport.refusal) {
+ Ext.Msg.alert(gettext('Error'), transport.refusal);
+ return;
+ }
// the storage API creates the volume group on the node it runs on
let node = Proxmox.NodeName;
@@ -24,6 +28,8 @@ PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
return;
}
+ let retryHint = () =>
+ Ext.String.format(transport.baseExistsText, enc(baseId), enc(wizard.actionText()));
let volumeGroupHint = () =>
Ext.String.format(
transport.volumeGroupText,
@@ -88,6 +94,7 @@ PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
node,
baseId,
lun: selected,
+ retryHint,
pin: function (volume) {
created.baseVolume = volume.volid;
created.lun = Ext.clone(selected);
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 70/76] ui: node: san: wipe a LUN that no storage uses
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (68 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 69/76] ui: storage wizard: iscsi: enable multipath for shared LVM Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 71/76] ui: storage wizard: iscsi: reuse existing target connections Thomas Lamprecht
` (5 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Offer root@pam a wipe only when visible storage references, local usage,
and paths allow it. Missing volume-group metadata does not prove a PV is
orphaned. Show refusal reasons and confirm device identity, data loss,
and the need to check other hosts. The backend repeats the authoritative
checks before writing.
The node view records the operation before submission, tracks its task,
and refreshes after every outcome. An unanswered request blocks another
wipe until the administrator checks task history and dismisses it.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Makefile | 1 +
www/manager6/SANUtils.js | 2 +
www/manager6/node/SANLuns.js | 145 ++++++++++++++++
www/manager6/window/SANLunWipe.js | 273 ++++++++++++++++++++++++++++++
4 files changed, 421 insertions(+)
create mode 100644 www/manager6/window/SANLunWipe.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index cd64e2366..8717d041a 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -138,6 +138,7 @@ JSSRC= \
window/Restore.js \
window/SafeDestroyGuest.js \
window/SafeDestroyStorage.js \
+ window/SANLunWipe.js \
window/Settings.js \
window/Snapshot.js \
window/StartupEdit.js \
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index bd8115e31..7bbb287e2 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -18,6 +18,8 @@ Ext.define('PVE.SANUtils', {
'multipath',
'multipath-storage',
'duplicate-pv',
+ 'in-use',
+ 'in-use-by',
'by-id',
],
diff --git a/www/manager6/node/SANLuns.js b/www/manager6/node/SANLuns.js
index 01c21f2b7..663acdea5 100644
--- a/www/manager6/node/SANLuns.js
+++ b/www/manager6/node/SANLuns.js
@@ -178,6 +178,119 @@ Ext.define('PVE.node.SANLunList', {
me.getSelectionModel().deselectAll();
me.getStore().load();
me.loadNodeStatus();
+ me.loadStorageConfig();
+ },
+
+ loadStorageConfig: function () {
+ let me = this;
+ let token = (me.storageConfigToken = (me.storageConfigToken ?? 0) + 1);
+ let apply = function (storages) {
+ if (me.destroyed || token !== me.storageConfigToken) {
+ return;
+ }
+ me.storageConfig = storages;
+ me.getSelectionModel().fireEvent(
+ 'selectionchange',
+ me.getSelectionModel(),
+ me.getSelection(),
+ );
+ };
+ apply(undefined);
+ Proxmox.Utils.API2Request({
+ url: '/storage',
+ method: 'GET',
+ failure: () => apply(undefined),
+ success: (response) => apply(response.result.data || []),
+ });
+ },
+
+ wipeReason: function (rec) {
+ let me = this;
+ if (me.wipeOperation) {
+ return gettext(
+ 'Wait for the previous wipe to finish. If its outcome is unknown, check the task history of the node, then press Dismiss.',
+ );
+ }
+ if (!rec) {
+ return gettext('Select a device to wipe.');
+ }
+ if (!me.storageConfig) {
+ return gettext('The storage configuration could not be read.');
+ }
+ return PVE.window.SANLunWipe.reason(rec.data, me.storageConfig);
+ },
+
+ updateWipeButton: function () {
+ let button = this.down('#wipeButton');
+ if (!button) {
+ return;
+ }
+ let reason = this.wipeReason(this.getSelection()[0]);
+ button.setDisabled(!!reason);
+ button.setTooltip(reason ? Ext.htmlEncode(reason) : '');
+ },
+
+ wipeLun: function (rec) {
+ let me = this;
+ if (me.wipeReason(rec)) {
+ return;
+ }
+ let refresh = () => !me.destroyed && me.reload();
+ Ext.create('PVE.window.SANLunWipe', {
+ node: me.nodename,
+ lun: rec.data,
+ autoShow: true,
+ onSubmit: function (op) {
+ me.wipeOperation = op;
+ me.updateWipeButton();
+ },
+ onClose: function (op) {
+ if (me.destroyed) {
+ return;
+ }
+ if (op.state === 'unknown') {
+ me.down('#wipeProblem').setLines([
+ Ext.htmlEncode(
+ Ext.String.format(
+ PVE.window.SANLunWipe.texts(op.lun).unknownRequest,
+ op.node,
+ ),
+ ),
+ ]);
+ me.down('#clearWipe').show();
+ } else {
+ delete me.wipeOperation;
+ }
+ refresh();
+ },
+ callback: function (op) {
+ Ext.create('Proxmox.window.TaskProgress', {
+ upid: op.upid,
+ autoShow: true,
+ taskDone: function () {
+ if (!me.destroyed && me.wipeOperation === op) {
+ delete me.wipeOperation;
+ me.down('#wipeProblem').hide();
+ me.down('#clearWipe').hide();
+ }
+ refresh();
+ },
+ listeners: {
+ destroy: function () {
+ if (!me.destroyed && me.wipeOperation === op) {
+ me.down('#wipeProblem').setLines([
+ gettext(
+ 'The wipe task may still be running. Check its task log and reload the list before using the device.',
+ ),
+ ]);
+ me.down('#clearWipe').show();
+ }
+ refresh();
+ },
+ },
+ });
+ },
+ });
},
loadNodeStatus: function () {
@@ -592,6 +705,37 @@ Ext.define('PVE.node.SANLunList', {
});
}
+ if (Proxmox.UserName === 'root@pam') {
+ items.push({
+ xtype: 'proxmoxButton',
+ text: gettext('Wipe'),
+ itemId: 'wipeButton',
+ // Disabled buttons still need hover events to explain why wiping is blocked.
+ style: { pointerEvents: 'auto' },
+ iconCls: 'fa fa-eraser',
+ selModel: sm,
+ disabled: true,
+ dangerous: true,
+ enableFn: (rec) => !me.wipeReason(rec),
+ handler: (btn, event, rec) => me.wipeLun(rec),
+ listeners: {
+ afterrender: () => me.updateWipeButton(),
+ },
+ });
+ sm.on('selectionchange', () => me.updateWipeButton());
+ }
+
+ items.push({
+ text: gettext('Dismiss'),
+ itemId: 'clearWipe',
+ hidden: true,
+ handler: function () {
+ delete me.wipeOperation;
+ me.down('#wipeProblem').hide();
+ me.down('#clearWipe').hide();
+ me.reload();
+ },
+ });
items.push('->', {
xtype: 'proxmoxcheckbox',
boxLabel: gettext('Show all devices'),
@@ -614,6 +758,7 @@ Ext.define('PVE.node.SANLunList', {
me.tbar = items;
me.dockedItems = [
+ { xtype: 'pveSANNotice', itemId: 'wipeProblem', kind: 'warning' },
{ xtype: 'pveSANNotice', itemId: 'nodeProblems', kind: 'warning' },
{ xtype: 'pveSANNotice', itemId: 'nodeNotes' },
];
diff --git a/www/manager6/window/SANLunWipe.js b/www/manager6/window/SANLunWipe.js
new file mode 100644
index 000000000..ec8c8fe77
--- /dev/null
+++ b/www/manager6/window/SANLunWipe.js
@@ -0,0 +1,273 @@
+// The node API (root@pam only) re-checks users, storage references and device identity. The caller
+// keeps the operation record past this window and owns its task.
+Ext.define('PVE.window.SANLunWipe', {
+ extend: 'Ext.window.Window',
+
+ modal: true,
+ width: 500,
+ bodyPadding: 10,
+ layout: { type: 'vbox', align: 'stretch' },
+ // config: node, lun (scan row), and the callbacks, each with the operation record { node, lun,
+ // identity, state, upid }: onSubmit(op) before the PUT, callback(op) once the task runs and
+ // onClose(op) after a submit that got no task, with state unknown or refused
+ node: undefined,
+ lun: undefined,
+ callback: undefined,
+
+ statics: {
+ texts: function (lun) {
+ return {
+ title: gettext('Wipe LUN'),
+ onlyRoot: gettext('Only root@pam can wipe a LUN.'),
+ nothing: gettext('The LUN holds nothing to wipe.'),
+ unknown: gettext('Could not check this LUN for data: {0}'),
+ unknownVG: gettext(
+ 'The LUN is an LVM physical volume, but its volume group is not known on this node.',
+ ),
+ duplicate: gettext('Another LUN has an LVM physical volume with the same UUID.'),
+ unchecked: gettext('Could not check whether this node uses the LUN.'),
+ noPath: gettext('The LUN has no usable path.'),
+ noName: gettext('Wiping needs a stable device name, but the LUN has none.'),
+ usedBy: gettext('Storage "{0}" uses the LUN.'),
+ confirm: gettext('LUN {0} ({1}, {2}) on node "{4}" contains data: {3}.'),
+ consequence: gettext(
+ 'Wiping destroys that data. No configured storage was found to use this LUN for data.',
+ ),
+ ownership: gettext(
+ 'The node checks again for local use before wiping, but other nodes, clusters, or hosts may still use the LUN. Confirm that it is no longer needed anywhere.',
+ ),
+ unknownRequest: gettext(
+ 'The outcome of the wipe request is unknown. Check the task history of node "{0}" before you use this LUN.',
+ ),
+ unknownTask: gettext(
+ 'The outcome of wiping LUN {0} ({1} {2}) is unknown. Do not use this LUN until the outcome is known.',
+ ),
+ wiping: gettext('Wiping LUN "{0}" on node "{1}"...'),
+ };
+ },
+
+ holdsText: function (lun) {
+ if (lun.vgname) {
+ return Ext.String.format(gettext('volume group "{0}"'), lun.vgname);
+ }
+ if (lun.usage === 'lvm') {
+ return gettext('LVM physical volume');
+ }
+ return (
+ {
+ partitions: gettext('partition table'),
+ mounted: gettext('mounted file system'),
+ LVM: gettext('LVM physical volume'),
+ LVM2_member: gettext('LVM physical volume'),
+ ZFS: gettext('ZFS pool'),
+ // a map whose only holders are the partition maps kpartx made of it, holders
+ // of any other kind make it in use and not wipeable
+ 'Device Mapper': gettext('partition table'),
+ }[lun.detail] ??
+ (lun.detail
+ ? Ext.String.format(gettext('signature "{0}"'), lun.detail)
+ : gettext('unknown signature'))
+ );
+ },
+
+ // undefined if wipeable; storages is undefined if unreadable; the node checks again
+ reason: function (lun, storages) {
+ let t = PVE.window.SANLunWipe.texts(lun);
+ if (Proxmox.UserName !== 'root@pam') {
+ return t.onlyRoot;
+ }
+ if (lun.usage === 'unknown') {
+ return Ext.String.format(t.unknown, lun.detail || gettext('unknown'));
+ }
+ if (
+ (lun.usage === 'lvm' || (lun.usage === 'other' && lun.detail === 'LVM2_member')) &&
+ !lun.vgname &&
+ ![0, '0', false].includes(lun['pv-in-use'])
+ ) {
+ return t.unknownVG;
+ }
+ if (!['lvm', 'other'].includes(lun.usage)) {
+ return t.nothing;
+ }
+ if (lun['duplicate-pv']) {
+ return t.duplicate;
+ }
+ if (lun['in-use'] === 'yes') {
+ return Ext.String.format(
+ gettext('In use on this node: {0}'),
+ lun['in-use-by'] ?? gettext('unknown'),
+ );
+ }
+ if (lun['in-use'] !== 'no') {
+ return t.unchecked;
+ }
+ if ((lun['path-devices'] ?? []).length > 1) {
+ return gettext(
+ 'Enable multipath for the LUN first. Until then, its paths are separate devices.',
+ );
+ }
+ let mapped = ['storage', 'manual', 'kept', 'unmanaged'].includes(lun.multipath);
+ if (mapped && !(lun['paths-active'] > 0)) {
+ return t.noPath;
+ }
+ if (!lun['by-id']) {
+ return t.noName;
+ }
+ if (!storages) {
+ return gettext('The storage configuration could not be read.');
+ }
+ for (const scfg of storages) {
+ let lvm = scfg.type === 'lvm' || scfg.type === 'lvmthin';
+ let base = `${scfg.base ?? ''}`;
+ if (
+ lvm &&
+ ((lun.vgname && scfg.vgname === lun.vgname) ||
+ (lun.wwid && base.includes(lun.wwid)))
+ ) {
+ return Ext.String.format(t.usedBy, scfg.storage);
+ }
+ }
+ // a connection-only storage leaves its LUNs free, a serving one may hold guest disks
+ // without a signature
+ let sid = lun['used-by-storage'];
+ if (sid) {
+ let scfg = storages.find((entry) => entry.storage === sid);
+ let connects = scfg?.type === 'iscsi' && `${scfg.content}` === 'none';
+ if (!connects) {
+ return Ext.String.format(t.usedBy, sid);
+ }
+ }
+ return undefined;
+ },
+ },
+
+ submitted: false,
+ pending: false,
+ requestState: 'cancelled',
+
+ initComponent: function () {
+ let me = this;
+ let lun = me.lun;
+ let enc = Ext.htmlEncode;
+ let number = lun['iscsi-lun'] ?? lun.devpath;
+ let t = PVE.window.SANLunWipe.texts(lun);
+ me.title = t.title;
+
+ let text = Ext.String.format(
+ t.confirm,
+ enc(number),
+ enc(Proxmox.Utils.format_size(lun.size)),
+ enc(lun.wwid ?? lun.wwn ?? lun.devpath),
+ enc(PVE.window.SANLunWipe.holdsText(lun)),
+ enc(me.node),
+ );
+
+ me.items = [
+ { xtype: 'displayfield', value: text },
+ { xtype: 'displayfield', value: t.consequence },
+ { xtype: 'displayfield', value: t.ownership },
+ { xtype: 'displayfield', itemId: 'errorLine', hidden: true },
+ {
+ xtype: 'textfield',
+ itemId: 'confirmField',
+ hidden: !lun.vgname,
+ fieldLabel: gettext('Type the volume group name to confirm'),
+ labelAlign: 'top',
+ emptyText: lun.vgname,
+ listeners: {
+ change: (field, value) =>
+ me.down('#wipeButton').setDisabled(me.submitted || value !== lun.vgname),
+ specialkey: function (field, event) {
+ if (
+ event.getKey() === event.ENTER &&
+ !me.down('#wipeButton').isDisabled()
+ ) {
+ me.startWipe();
+ }
+ },
+ },
+ },
+ ];
+
+ me.buttons = [
+ {
+ text: gettext('Wipe'),
+ itemId: 'wipeButton',
+ cls: 'pve-dangerous-button',
+ disabled: !!lun.vgname,
+ handler: () => me.startWipe(),
+ },
+ {
+ text: gettext('Cancel'),
+ itemId: 'cancelButton',
+ handler: () => me.close(),
+ },
+ ];
+
+ me.callParent();
+
+ me.on('beforeclose', () => !me.pending);
+ me.on('close', function () {
+ let op = me.operation;
+ if (op && me.requestState !== 'accepted') {
+ op.state = me.requestState;
+ me.onClose?.(op);
+ }
+ });
+ },
+
+ startWipe: function () {
+ let me = this;
+ if (me.submitted || me.down('#wipeButton').isDisabled()) {
+ return;
+ }
+ me.submitted = true;
+ me.pending = true;
+ let op = (me.operation = {
+ node: me.node,
+ lun: Ext.clone(me.lun),
+ identity: me.lun.wwid || me.lun.wwn,
+ state: 'pending',
+ });
+ me.onSubmit?.(op);
+ me.down('#confirmField').disable();
+ me.down('#wipeButton').setDisabled(true);
+ me.down('#cancelButton').setDisabled(true);
+ me.setLoading(gettext('Wiping...'));
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${me.node}/disks/wipedisk`,
+ method: 'PUT',
+ params: { disk: me.lun['by-id'] },
+ failure: function (response) {
+ me.pending = false;
+ me.setLoading(false);
+ me.down('#cancelButton').setDisabled(false);
+ if (PVE.SANUtils.request_uncertain(response)) {
+ me.requestState = 'unknown';
+ me.close();
+ return;
+ }
+ me.requestState = 'refused';
+ me.down('#errorLine').setValue(response.htmlStatus);
+ me.down('#errorLine').show();
+ },
+ success: function (response) {
+ me.pending = false;
+ let upid = response.result.data;
+ try {
+ Proxmox.Utils.parse_task_upid(upid);
+ } catch (_error) {
+ me.requestState = 'unknown';
+ me.close();
+ return;
+ }
+ me.requestState = 'accepted';
+ op.upid = upid;
+ op.state = 'waiting';
+ let callback = me.callback;
+ me.close();
+ callback?.(op);
+ },
+ });
+ },
+});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 71/76] ui: storage wizard: iscsi: reuse existing target connections
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (69 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 70/76] ui: node: san: wipe a LUN that no storage uses Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 72/76] ui: storage wizard: fibre channel: enable multipath for a LUN Thomas Lamprecht
` (4 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Reuse a target's connection-only storage for additional shared LVM
groups instead of creating another login owner. Inherit its ID, nodes,
and multipath setting, restricting LVM to those nodes. Reject
conflicting configurations, targets serving raw disk images, and
connections excluding the setup node, which must activate the storage to
discover LUNs.
Select LUN identifies the reused connection. Back remains available
until setup creates an object or starts an operation. Going back
invalidates the inventory and restores the summary. Recheck target use
before submission and explain how reopening the wizard continues an
interrupted setup.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/storage/wizard/IScsi.js | 184 +++++++++++++++++++----
www/manager6/storage/wizard/LunSelect.js | 2 +-
www/manager6/storage/wizard/SanLvm.js | 71 ++++++++-
www/manager6/storage/wizard/Wizard.js | 67 ++++++++-
4 files changed, 283 insertions(+), 41 deletions(-)
diff --git a/www/manager6/storage/wizard/IScsi.js b/www/manager6/storage/wizard/IScsi.js
index 9bab76cd0..777d491e7 100644
--- a/www/manager6/storage/wizard/IScsi.js
+++ b/www/manager6/storage/wizard/IScsi.js
@@ -70,7 +70,10 @@ Ext.define('PVE.storage.wizard.IScsiTarget', {
disabled: true,
showNodeSelector: false,
listeners: {
- change: (f, value) => me.updateMultipathDefault(f, value),
+ change: function (f, value) {
+ me.updateMultipathDefault(f, value);
+ me.up('window').refreshTargetUse();
+ },
},
},
];
@@ -125,9 +128,18 @@ Ext.define('PVE.storage.wizard.IScsiUsage', {
blockedReason: function () {
let me = this;
let wizard = me.up('window');
- if (wizard.getViewModel().get('usage') !== 'lvm') {
+ let usage = wizard.getViewModel().get('usage');
+ let use = wizard.targetUse;
+ if (use?.conflict || (usage === 'direct' && use?.reuse)) {
+ return { text: use.conflict ?? use.inUse };
+ }
+ if (usage !== 'lvm') {
return undefined;
}
+ let unavailable = PVE.storage.wizard.reuseNodeRefusal(use?.reuse, Proxmox.NodeName);
+ if (unavailable) {
+ return { text: unavailable };
+ }
if (wizard.getScanNode() !== Proxmox.NodeName) {
return {
text: Ext.String.format(
@@ -143,7 +155,7 @@ Ext.define('PVE.storage.wizard.IScsiUsage', {
// without multipath LVM sees the PV once per portal and ignores the volume group
let field = wizard.down('field[name=multipath]');
let portals = wizard.down('pveStorageWizardIScsiTarget').targetPortals;
- if (portals < 2 || field.getValue()) {
+ if (wizard.targetUse?.reuse || portals < 2 || field.getValue()) {
return undefined;
}
return {
@@ -276,9 +288,11 @@ PVE.storage.wizard.types.iscsi = {
viewModel: {
data: {
usage: 'lvm',
+ reuseBase: false,
},
formulas: {
isLvmUsage: (get) => get('usage') === 'lvm',
+ newBase: (get) => get('usage') === 'lvm' && !get('reuseBase'),
},
},
steps: () => [
@@ -313,8 +327,8 @@ PVE.storage.wizard.types.iscsi = {
vtype: 'StorageId',
allowBlank: true,
bind: {
- disabled: '{!isLvmUsage}',
- hidden: '{!isLvmUsage}',
+ disabled: '{!newBase}',
+ hidden: '{!newBase}',
},
},
],
@@ -336,16 +350,99 @@ PVE.storage.wizard.types.iscsi = {
if (wizard.getValues().usage !== 'lvm') {
return undefined;
}
- return gettext('Connect and Select LUN');
+ return wizard.targetUse?.reuse ? gettext('Select LUN') : gettext('Connect and Select LUN');
+ },
+
+ // a storage serving the LUNs as disk images rules the target out, they may be guest disks
+ // without a signature
+ targetUse: function (entries, values, wizard) {
+ let enc = Ext.htmlEncode;
+ let users = entries.filter((entry) => entry.target && entry.target === values.target);
+ if (!users.length) {
+ return undefined;
+ }
+ let connects = (entry) => entry.type === 'iscsi' && `${entry.content}` === 'none';
+ let serving = users.filter((entry) => !connects(entry));
+ if (serving.length) {
+ return {
+ conflict: Ext.String.format(
+ ngettext(
+ 'Storage {0} already uses the LUNs of this target as disk images, so the wizard cannot set up another storage on it.',
+ 'Storages {0} already use the LUNs of this target as disk images, so the wizard cannot set up another storage on it.',
+ serving.length,
+ ),
+ enc(serving.map((entry) => `"${entry.storage}"`).join(', ')),
+ ),
+ };
+ }
+ if (users.length > 1) {
+ return {
+ conflict: Ext.String.format(
+ gettext('Several storages connect to this target: {0}.'),
+ enc(users.map((entry) => entry.storage).join(', ')),
+ ),
+ };
+ }
+ let base = users[0];
+ let inUse = Ext.String.format(
+ gettext('Storage "{0}" already connects to this target.'),
+ enc(base.storage),
+ );
+ if (base.disable) {
+ return {
+ inUse,
+ conflict: Ext.String.format(
+ gettext(
+ 'Storage "{0}" connects to this target, but it is disabled. Enable that storage first.',
+ ),
+ enc(base.storage),
+ ),
+ };
+ }
+ if (!base.multipath && wizard.down('pveStorageWizardIScsiTarget').targetPortals > 1) {
+ return {
+ inUse,
+ conflict: Ext.String.format(
+ gettext(
+ 'Storage "{0}" connects to this target without multipath, but shared LVM on a target with several portals needs multipath. Enable the Multipath option of that storage first.',
+ ),
+ enc(base.storage),
+ ),
+ };
+ }
+ return {
+ inUse,
+ reuse: {
+ storage: base.storage,
+ portal: base.portal,
+ multipath: !!base.multipath,
+ nodes: base.nodes ? `${base.nodes}`.split(',') : [],
+ },
+ };
+ },
+
+ targetUseChanged: function (wizard, use) {
+ let panel = wizard.down('pveStorageWizardIScsiTarget');
+ let field = panel.down('field[name=multipath]');
+ if (use?.reuse) {
+ panel.settingMultipathDefault = true;
+ field.setValue(use.reuse.multipath);
+ panel.settingMultipathDefault = false;
+ }
+ field.setDisabled(!!use?.reuse);
+ wizard.getViewModel().set('reuseBase', !!use?.reuse);
},
summaryRows: function (values, wizard) {
if (values.usage !== 'lvm') {
return {};
}
+ let reuse = wizard.targetUse?.reuse;
let created = wizard.created ?? {};
- let base = values['base-storage'] || `${values.storage}-base`;
- if (created.baseStorage) {
+ let base = reuse?.storage ?? (values['base-storage'] || `${values.storage}-base`);
+ if (reuse) {
+ base = Ext.String.format(gettext('{0} (existing)'), base);
+ } else if (created.baseStorage) {
base = Ext.String.format(gettext('{0} (created)'), created.baseStorage);
}
let rows = [
@@ -358,17 +455,33 @@ PVE.storage.wizard.types.iscsi = {
},
];
let skip = ['storage', 'base-storage'];
+ if (reuse) {
+ rows.push(
+ { label: 'Portal', value: reuse.portal, weight: 50 },
+ {
+ label: gettext('Multipath'),
+ value: Proxmox.Utils.format_boolean(reuse.multipath),
+ weight: 50,
+ },
+ );
+ skip.push('portal', 'multipath');
+ }
return { rows, skip };
},
summaryNotes: function (values, wizard) {
if (values.usage === 'lvm') {
- let base = values['base-storage'] || `${values.storage}-base`;
+ let reuse = wizard.targetUse?.reuse;
+ let base = reuse?.storage ?? (values['base-storage'] || `${values.storage}-base`);
return [
Ext.String.format(
- gettext(
- 'Press Connect and Select LUN to add the iSCSI storage "{0}" and list its LUNs. The volume group is created on the LUN you select next.',
- ),
+ reuse
+ ? gettext(
+ 'Press Select LUN to list the LUNs of the existing iSCSI storage "{0}". The volume group is created on the LUN you select next.',
+ )
+ : gettext(
+ 'Press Connect and Select LUN to add the iSCSI storage "{0}" and list its LUNs. The volume group is created on the LUN you select next.',
+ ),
Ext.htmlEncode(base),
),
];
@@ -389,6 +502,23 @@ PVE.storage.wizard.types.iscsi = {
let usage = values.usage;
delete values.usage;
+ // the storages of the target as they are now, the steps saw an older state
+ let checkTarget = (entries) => {
+ let use = PVE.storage.wizard.types.iscsi.targetUse(entries, values, wizard);
+ if (use?.conflict || (usage === 'direct' && use?.inUse)) {
+ return use.conflict ?? use.inUse;
+ }
+ if (
+ usage === 'lvm' &&
+ (use?.reuse?.storage ?? '') !== (wizard.targetUse?.reuse?.storage ?? '')
+ ) {
+ return gettext(
+ 'The storages that use the target have changed in the meantime. Go back to the Target step to review them.',
+ );
+ }
+ return undefined;
+ };
+
if (usage === 'direct') {
delete values['base-storage'];
if (!values.multipath) {
@@ -396,11 +526,15 @@ PVE.storage.wizard.types.iscsi = {
}
values.type = 'iscsi';
values.content = 'images';
- wizard.runSubmitChain([(next) => wizard.createStorage(values, next)]);
+ wizard.runSubmitChain([
+ (next) => wizard.assertStorageIdsFree([values.storage], next, checkTarget),
+ (next) => wizard.createStorage(values, next),
+ ]);
return;
}
- let multipath = !!values.multipath;
+ let reuse = wizard.targetUse?.reuse;
+ let multipath = reuse ? reuse.multipath : !!values.multipath;
let portals = wizard.down('pveStorageWizardIScsiTarget').targetPortals;
PVE.storage.wizard.submitSanLvm(wizard, values, {
sameIdText: gettext('The iSCSI storage and the LVM storage need different IDs.'),
@@ -411,29 +545,15 @@ PVE.storage.wizard.types.iscsi = {
)
: undefined,
baseExistsText: gettext(
- 'The iSCSI storage "{0}" exists. Press {1} to retry the remaining steps, or complete the setup under Datacenter > Storage > Add > LVM.',
+ 'The iSCSI storage "{0}" exists. Press {1} to retry the remaining steps. If you close the wizard, running it again on the same target continues the setup.',
),
volumeGroupText: gettext(
'Press {2} to retry the remaining steps. If the volume group "{0}" already exists on the LUN, you can also complete the setup under Datacenter > Storage > Add > LVM, with base volume "{1}" and volume group "{0}".',
),
sameUnitText: gettext(
- 'Press {1} to try the same LUN again, for example after wiping it. The iSCSI storage "{0}" remains after closing the wizard. Complete its setup under Datacenter > Storage > Add > LVM.',
+ 'Press {1} to try the same LUN again, for example after wiping it. To use another LUN or storage ID, close the wizard and run it again on the same target. The wizard reuses the iSCSI storage "{0}".',
),
- // the LUNs of a target that another storage already logs in to are that storage's, and
- // may be in use through it
- checkTarget: (entries) => {
- let other = entries.find(
- (entry) => entry.type === 'iscsi' && entry.target === values.target,
- );
- return other
- ? Ext.String.format(
- gettext(
- 'The iSCSI storage "{0}" already uses this target. Create the LVM storage on top of it under Datacenter > Storage > Add > LVM.',
- ),
- enc(other.storage),
- )
- : undefined;
- },
+ checkTarget,
baseParams: (baseId) => ({
storage: baseId,
type: 'iscsi',
@@ -448,7 +568,7 @@ PVE.storage.wizard.types.iscsi = {
xtype: 'pveStorageWizardLunSelect',
footnote: Ext.String.format(
gettext(
- 'Closing the wizard keeps storage "{0}". Complete the setup under Datacenter > Storage > Add > LVM if the wizard is closed.',
+ 'Closing the wizard keeps storage "{0}". Running the wizard again on the same target continues from there.',
),
enc(baseId),
),
diff --git a/www/manager6/storage/wizard/LunSelect.js b/www/manager6/storage/wizard/LunSelect.js
index 270daaa73..4e138f787 100644
--- a/www/manager6/storage/wizard/LunSelect.js
+++ b/www/manager6/storage/wizard/LunSelect.js
@@ -97,7 +97,7 @@ Ext.define('PVE.storage.wizard.LunSelect', {
grid.setDisabled(!!pinned || blocked || me.scanning || me.scanError);
me.down('#rescan').setHidden(!!pinned);
me.down('#rescan').setDisabled(blocked || me.scanning);
- me.down('#footnote').setHidden(!!pinned);
+ me.down('#footnote').setHidden(!!pinned || !!wizard.targetUse?.reuse);
let text;
if (wizard.unconfirmedStorage) {
text = wizard.storageUnknownText(false);
diff --git a/www/manager6/storage/wizard/SanLvm.js b/www/manager6/storage/wizard/SanLvm.js
index 054725753..7050730e0 100644
--- a/www/manager6/storage/wizard/SanLvm.js
+++ b/www/manager6/storage/wizard/SanLvm.js
@@ -1,7 +1,22 @@
+// listing the LUNs and adding the LVM storage both activate the connection storage on that node
+PVE.storage.wizard.reuseNodeRefusal = function (reuse, node) {
+ if (!reuse?.nodes.length || reuse.nodes.includes(node)) {
+ return undefined;
+ }
+ return Ext.String.format(
+ gettext(
+ 'Storage "{0}" is not available on node "{1}", which sets up the volume group. Add the node to that storage, or open the web UI on one of its nodes.',
+ ),
+ Ext.htmlEncode(reuse.storage),
+ Ext.htmlEncode(node),
+ );
+};
+
PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
let enc = Ext.htmlEncode;
let created = (wizard.created ??= {});
- let baseId = values['base-storage'] || `${values.storage}-base`;
+ let reuse = wizard.targetUse?.reuse;
+ let baseId = reuse?.storage ?? (values['base-storage'] || `${values.storage}-base`);
if (baseId === values.storage) {
Ext.Msg.alert(gettext('Error'), transport.sameIdText);
return;
@@ -13,6 +28,11 @@ PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
// the storage API creates the volume group on the node it runs on
let node = Proxmox.NodeName;
+ let unavailable = PVE.storage.wizard.reuseNodeRefusal(reuse, node);
+ if (unavailable) {
+ Ext.Msg.alert(gettext('Error'), unavailable);
+ return;
+ }
let nodesList = values.nodes ? [].concat(values.nodes).join(',').split(',') : [];
nodesList = nodesList.filter((entry) => entry !== '');
if (wizard.getScanNode() !== node || (nodesList.length && !nodesList.includes(node))) {
@@ -27,9 +47,35 @@ PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
);
return;
}
+ // the LVM storage can only be active where its connection storage is
+ let lvmNodes = values.nodes;
+ if (reuse?.nodes.length) {
+ if (!nodesList.length) {
+ lvmNodes = reuse.nodes.join(',');
+ } else if (nodesList.some((entry) => !reuse.nodes.includes(entry))) {
+ Ext.Msg.alert(
+ gettext('Error'),
+ Ext.String.format(
+ ngettext(
+ 'Storage "{0}" is only available on node {1}.',
+ 'Storage "{0}" is only available on nodes {1}.',
+ reuse.nodes.length,
+ ),
+ enc(baseId),
+ enc(reuse.nodes.join(', ')),
+ ),
+ );
+ return;
+ }
+ }
let retryHint = () =>
- Ext.String.format(transport.baseExistsText, enc(baseId), enc(wizard.actionText()));
+ reuse
+ ? Ext.String.format(
+ gettext('Press {0} to retry the remaining steps.'),
+ enc(wizard.actionText()),
+ )
+ : Ext.String.format(transport.baseExistsText, enc(baseId), enc(wizard.actionText()));
let volumeGroupHint = () =>
Ext.String.format(
transport.volumeGroupText,
@@ -48,13 +94,20 @@ PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
next();
return;
}
- wizard.assertStorageIdsFree([baseId, values.storage], next, transport.checkTarget);
+ let ids = reuse ? [values.storage] : [baseId, values.storage];
+ wizard.assertStorageIdsFree(ids, next, transport.checkTarget);
},
(next) => {
if (created.baseStorage) {
next();
return;
}
+ if (reuse) {
+ created.baseStorage = baseId;
+ wizard.freeze();
+ next();
+ return;
+ }
wizard.createStorage(transport.baseParams(baseId), () => {
created.baseStorage = baseId;
wizard.freeze();
@@ -70,9 +123,13 @@ PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
let active = wizard.activateLunView({
...transport.lunView(node, baseId),
recap: Ext.String.format(
- gettext(
- 'Connection storage "{1}" created. Setting up volume group and LVM storage "{0}".',
- ),
+ reuse
+ ? gettext(
+ 'Using existing connection storage "{1}" to set up volume group and LVM storage "{0}".',
+ )
+ : gettext(
+ 'Connection storage "{1}" created. Setting up volume group and LVM storage "{0}".',
+ ),
enc(values.storage),
enc(baseId),
),
@@ -112,7 +169,7 @@ PVE.storage.wizard.submitSanLvm = function (wizard, values, transport) {
base: created.baseVolume,
shared: 1,
content: values.content,
- nodes: values.nodes,
+ nodes: lvmNodes,
disable: values.disable,
},
next,
diff --git a/www/manager6/storage/wizard/Wizard.js b/www/manager6/storage/wizard/Wizard.js
index c5ffa2b8e..433612b1a 100644
--- a/www/manager6/storage/wizard/Wizard.js
+++ b/www/manager6/storage/wizard/Wizard.js
@@ -165,6 +165,17 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
return !me.lunView || me.lunView.canSubmit();
},
+ canReuseBack: function () {
+ let me = this;
+ return !!(
+ me.lunView &&
+ me.targetUse?.reuse &&
+ !me.busy &&
+ !me.unconfirmedStorage &&
+ Object.keys(me.created ?? {}).every((key) => key === 'baseStorage')
+ );
+ },
+
actionText: function () {
let me = this;
if (me.unconfirmedStorage) {
@@ -185,7 +196,7 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
if (me.lunView) {
me.lunView.updateState();
me.down('#submit').setDisabled(!me.canSubmit());
- me.down('#back').setDisabled(true);
+ me.down('#back').setDisabled(!me.canReuseBack());
}
// a type with its own submit condition decides the state of the confirm buttons
let confirmActive = me.down('#wizcontent').getActiveTab() === me.down('#confirm');
@@ -214,6 +225,24 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
return false;
},
+ reuseBack: function () {
+ let me = this;
+ if (!me.canReuseBack()) {
+ return;
+ }
+ let view = me.lunView;
+ view.invalidateScan();
+ delete me.lunView;
+ delete me.created.baseStorage;
+ let panel = me.down('#confirm');
+ panel.getLayout().setActiveItem(0);
+ panel.remove(view, true);
+ panel.setTitle(gettext('Confirm'));
+ me.unfreeze();
+ me.down('#submit').enable();
+ me.updateSummary();
+ },
+
// keeps the window busy, so it cannot close with the continuation pending
request: function (options) {
let me = this;
@@ -420,6 +449,33 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
});
},
+ // { reuse, conflict, inUse } from targetUse() of the type, only from the storages the user may
+ // see
+ targetUse: undefined,
+
+ refreshTargetUse: function () {
+ let me = this;
+ let typeEntry = PVE.storage.wizard.types[me.wizardType];
+ let request = (me.targetUseRequest = (me.targetUseRequest ?? 0) + 1);
+ let update = function (use) {
+ // a newer selection decides, and the wizard may be gone by now
+ if (me.destroyed || request !== me.targetUseRequest) {
+ return;
+ }
+ me.targetUse = use;
+ me.down('#wizcontent').items.each((tab) => tab.updateBlockedReason?.());
+ typeEntry.targetUseChanged?.(me, use);
+ };
+ update(undefined);
+ Proxmox.Utils.API2Request({
+ url: '/storage',
+ method: 'GET',
+ failure: () => update(undefined),
+ success: (response) =>
+ update(typeEntry.targetUse(response.result.data || [], me.getValues(), me)),
+ });
+ },
+
storageUnknownText: function (html = true) {
let encode = html ? Ext.htmlEncode : (value) => value;
return Ext.String.format(
@@ -787,6 +843,15 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
});
me.callParent();
+ let back = me.down('#back');
+ let backHandler = back.handler;
+ back.setHandler(function () {
+ if (me.lunView) {
+ me.reuseBack();
+ } else {
+ backHandler.call(back);
+ }
+ });
me.query('field').forEach((field) => {
field.on('change', () => me.updateActions());
field.on('validitychange', () => me.updateActions());
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 72/76] ui: storage wizard: fibre channel: enable multipath for a LUN
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (70 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 71/76] ui: storage wizard: iscsi: reuse existing target connections Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 73/76] ui: storage wizard: let root@pam wipe a LUN with data Thomas Lamprecht
` (3 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Group Fibre Channel paths by LUN and enable multipath. Create the volume
group on its map only after every online storage node reports a usable
path. Extend existing entries to cover storage nodes, or add an entry
for a map retained only because it is in use.
Without Sys.Modify on "/", accept only entries already covering all
storage nodes. Show coverage on Confirm. Verify absence through the
entry list before creating an entry after a failed read.
Once setup adds an entry, pin the device and label retries accordingly
so partial setup cannot switch LUNs.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/storage/wizard/FibreChannel.js | 365 +++++++++++++++++++-
1 file changed, 352 insertions(+), 13 deletions(-)
diff --git a/www/manager6/storage/wizard/FibreChannel.js b/www/manager6/storage/wizard/FibreChannel.js
index eec8a7790..b0f5922f4 100644
--- a/www/manager6/storage/wizard/FibreChannel.js
+++ b/www/manager6/storage/wizard/FibreChannel.js
@@ -1,3 +1,54 @@
+// whether a multipath entry covers the nodes of a storage (empty: all nodes): a missing entry
+// covers none, an unrestricted one every node, and a restricted one has to list each of them
+PVE.storage.wizard.multipathEntryCovers = function (entry, wanted) {
+ if (!entry) {
+ return false;
+ }
+ let current = entry.nodes ? entry.nodes.split(',') : [];
+ return !current.length || (wanted.length > 0 && wanted.every((node) => current.includes(node)));
+};
+
+// let the multipath entry of wwid cover nodes (empty: all), creating a restricted one if missing
+PVE.storage.wizard.coverMultipathEntry = function (wizard, wwid, nodes, next) {
+ let url = `/cluster/multipath/wwid/${encodeURIComponent(wwid)}`;
+ let fail = (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+ let wanted = []
+ .concat(nodes ?? [])
+ .join(',')
+ .split(',');
+ wanted = wanted.filter((node) => node !== '');
+ let create = () =>
+ wizard.request({
+ url: '/cluster/multipath/wwid',
+ method: 'POST',
+ params: wanted.length ? { wwid, nodes: wanted.join(',') } : { wwid },
+ failure: fail,
+ success: () => next(),
+ });
+ PVE.dc.MultipathWwidEdit.lookup(wwid, {
+ request: (options) => wizard.request(options),
+ missing: create,
+ // without Sys.Audit the list is refused, which leaves creating the entry
+ failure: (original, list) =>
+ PVE.SANUtils.request_denied(list) ? create() : fail(original),
+ found: function (entry) {
+ let current = entry.nodes ? entry.nodes.split(',') : [];
+ let missing = wanted.filter((node) => !current.includes(node));
+ if (PVE.storage.wizard.multipathEntryCovers(entry, wanted)) {
+ next();
+ return;
+ }
+ let params = { digest: entry.digest };
+ if (wanted.length) {
+ params.nodes = [...current, ...missing].join(',');
+ } else {
+ params.delete = 'nodes';
+ }
+ wizard.request({ url, method: 'PUT', params, failure: fail, success: () => next() });
+ },
+ });
+};
+
Ext.define('PVE.storage.wizard.FCLunSelect', {
extend: 'Proxmox.panel.InputPanel',
xtype: 'pveStorageWizardFCLunSelect',
@@ -5,7 +56,7 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
onlineHelp: 'storage_lvm',
- // { lun } of the selected row
+ // { lun, enableMultipath, multipathEntry } of the selected row
selection: undefined,
lunScan: undefined,
@@ -25,6 +76,51 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
wizard.down('field[name=nodes]').setValue([...nodes].sort());
},
+ // How the multipath entry of the selected LUN covers the storage nodes: pending, covered,
+ // uncovered, missing, denied or failed (with error). A different LUN or node list starts a new
+ // lookup, and replies to older lookups are dropped. With done, a new lookup through request
+ // starts in any case, and done(check) gets its result.
+ checkMultipathEntry: function (nodes, done, request) {
+ let me = this;
+ let wwid = me.selection?.multipathEntry;
+ if (!wwid) {
+ return undefined;
+ }
+ let wanted = []
+ .concat(nodes ?? [])
+ .join(',')
+ .split(',')
+ .filter((node) => node !== '');
+ let key = `${wwid}\0${[...new Set(wanted)].sort().join(',')}`;
+ if (!done && me.entryCheck?.key === key) {
+ return me.entryCheck;
+ }
+ let check = (me.entryCheck = { key, state: 'pending' });
+ let finish = function (result) {
+ Ext.apply(check, result);
+ me.up('window').updateSummary();
+ done?.(check);
+ };
+ PVE.dc.MultipathWwidEdit.lookup(wwid, {
+ request,
+ current: () => !me.destroyed && me.entryCheck === check,
+ missing: () => finish({ state: 'missing' }),
+ found: (entry) =>
+ finish({
+ state: PVE.storage.wizard.multipathEntryCovers(entry, wanted)
+ ? 'covered'
+ : 'uncovered',
+ }),
+ failure: (original, list) =>
+ finish(
+ PVE.SANUtils.request_denied(original) || PVE.SANUtils.request_denied(list)
+ ? { state: 'denied' }
+ : { state: 'failed', error: original.htmlStatus },
+ ),
+ });
+ return check;
+ },
+
validator: function () {
let wizard = this.up('window');
return !wizard.busy && !!wizard.getViewModel().get('vgmode');
@@ -33,9 +129,11 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
updateActions: function () {
let me = this;
let wizard = me.up('window');
- let blocked = wizard.busy;
+ let pinned = !!wizard.created?.entry;
+ let blocked = wizard.busy || pinned;
let grid = me.down('#lunGrid');
grid.setDisabled(blocked || me.scanning);
+ me.down('#rescan').setHidden(pinned);
me.down('#rescan').setDisabled(blocked);
me.down('pveStorageScanNodeSelector')?.setDisabled(blocked);
if (wizard.down('#wizcontent').getActiveTab() === me) {
@@ -224,7 +322,7 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
filterFn: (rec) => !remoteTransports[rec.get('transport')],
});
- // creating the volume group needs Sys.Modify on /
+ // creating the volume group and the multipath entry both need Sys.Modify on /
let caps = Ext.state.Manager.get('GuiCap');
let canModify = !!caps.dc['Sys.Modify'];
me.canModify = canModify;
@@ -260,7 +358,20 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
{
name: 'selectable',
calculate: function (data) {
- if (notOffered(data) || data['used-by-storage']) {
+ if (
+ notOffered(data) ||
+ data['used-by-storage'] ||
+ data.multipath === 'pending'
+ ) {
+ return false;
+ }
+ // the new volume group goes on the map the setup enables
+ if (data.multipath === 'none' && data['paths-total'] > 1) {
+ return data.usage === 'unused' && canModify;
+ }
+ // I/O to a map without a usable path could stay queued indefinitely
+ let mapped = ['storage', 'manual', 'kept', 'unmanaged'];
+ if (mapped.includes(data.multipath) && !(data['paths-active'] > 0)) {
return false;
}
if (data.usage === 'unused') {
@@ -281,9 +392,34 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
],
});
+ // naming one path would suggest the others are different LUNs
+ let renderDevice = function (value, metaData, rec) {
+ let devices = rec.get('path-devices');
+ if (devices?.length > 1) {
+ let text = Ext.htmlEncode(devices.join(', '));
+ metaData.tdAttr = `data-qtip="${text}"`;
+ return text;
+ }
+ return Ext.htmlEncode(value);
+ };
+
let updateSelection = function (rec) {
let wizard = me.up('window');
- me.selection = rec ? { lun: rec.data } : undefined;
+ let multipath = rec?.get('multipath');
+ delete me.entryCheck;
+ me.selection = rec
+ ? {
+ lun: rec.data,
+ enableMultipath:
+ multipath === 'none' && rec.get('paths-total') > 1
+ ? rec.get('wwid')
+ : undefined,
+ // a kept map needs an entry, as nothing else keeps it on the storage nodes
+ multipathEntry: ['manual', 'kept'].includes(multipath)
+ ? rec.get('wwid')
+ : undefined,
+ }
+ : undefined;
let vm = wizard.getViewModel();
if (!rec || !rec.get('selectable')) {
vm.set({ vgmode: '', device: '', vgname: '' });
@@ -293,6 +429,9 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
vm.set({ vgmode: 'existing', device: '', vgname: rec.get('vgname') });
}
me.updateNodes();
+
+ let singlePath = multipath === 'none' && !(rec.get('paths-total') > 1);
+ me.down('#singlePathNote').setHidden(!singlePath);
};
me.items = [
@@ -318,7 +457,7 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
header: gettext('Device'),
dataIndex: 'devpath',
flex: 3,
- renderer: Ext.htmlEncode,
+ renderer: renderDevice,
},
columns.paths,
{
@@ -327,6 +466,12 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
width: 80,
renderer: PVE.SANUtils.render_san_transport,
},
+ {
+ header: gettext('Multipath'),
+ dataIndex: 'multipath',
+ width: 120,
+ renderer: PVE.SANUtils.render_san_lun_multipath,
+ },
columns.size,
columns.model,
{
@@ -404,6 +549,14 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
allowBlank: true,
bind: { value: '{device}' },
},
+ {
+ xtype: 'displayfield',
+ itemId: 'singlePathNote',
+ hidden: true,
+ value:
+ '<i class="fa fa-exclamation-triangle warning"></i> ' +
+ gettext('The LUN has a single path, so there is no redundancy.'),
+ },
];
if (!PVE.Utils.isStandaloneNode()) {
@@ -422,6 +575,12 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
'afterrender',
function () {
me.mon(me.up('window'), 'setupchange', me.updateActions, me);
+ // a fresh entry lookup each time the Confirm page opens
+ me.mon(me.up('window').down('#wizcontent'), 'tabchange', function (tp, tab) {
+ if (tab !== me.up('window').down('#confirm')) {
+ delete me.entryCheck;
+ }
+ });
me.refreshLuns();
},
me,
@@ -481,20 +640,62 @@ PVE.storage.wizard.types.fc = {
},
canSubmit: function (wizard) {
- return !!wizard.getValues().vgmode;
+ let values = wizard.getValues();
+ let step = wizard.down('pveStorageWizardFCLunSelect');
+ let check = step.checkMultipathEntry(values.nodes);
+ if (check && !step.canModify && check.state !== 'covered') {
+ return false;
+ }
+ return !!values.vgmode;
},
+ // the Multipath row of an entry check and why it blocks Finish; without Sys.Modify on / only
+ // an entry that already covers the storage nodes will do
+ entryStatus: function (check, canModify) {
+ if (check.state === 'covered') {
+ return { value: gettext('Entry covers all storage nodes') };
+ }
+ if (canModify) {
+ return { value: gettext('Configured on Finish, for the nodes of the storage') };
+ }
+ let modify = gettext(
+ 'Configuring the multipath entry for these nodes requires Sys.Modify on "/".',
+ );
+ return (
+ {
+ missing: { value: gettext('Entry missing'), problem: modify },
+ uncovered: {
+ value: gettext('Entry does not cover all storage nodes'),
+ problem: modify,
+ },
+ denied: {
+ value: gettext('Could not check the multipath entry'),
+ problem: gettext('Checking the multipath entry requires Sys.Audit on "/".'),
+ },
+ failed: {
+ value: gettext('Could not check the multipath entry'),
+ problem: check.error,
+ },
+ }[check.state] ?? { value: gettext('Checking the multipath entry...') }
+ );
+ },
+
+ actionText: (wizard) => (wizard.created?.entry ? gettext('Retry Setup') : gettext('Finish')),
+
summaryRows: function (values, wizard) {
let step = wizard.down('pveStorageWizardFCLunSelect');
- let { lun = {} } = step.selection ?? {};
+ let { lun = {}, enableMultipath, multipathEntry } = step.selection ?? {};
let rows = [];
if (values.vgmode === 'create') {
+ let device = enableMultipath
+ ? `/dev/disk/by-id/dm-uuid-mpath-${enableMultipath}`
+ : values.device;
rows.push({
label: gettext('Volume group'),
value: Ext.String.format(
gettext('Create "{0}" on {1} ({2})'),
values.vgname || values.storage,
- values.device,
+ device,
Proxmox.Utils.format_size(lun.size),
),
weight: 20,
@@ -506,6 +707,22 @@ PVE.storage.wizard.types.fc = {
weight: 20,
});
}
+ if (enableMultipath) {
+ rows.push({
+ label: gettext('Multipath'),
+ value: gettext('Enabled on Finish, for the nodes of the storage'),
+ weight: 21,
+ });
+ } else if (multipathEntry) {
+ rows.push({
+ label: gettext('Multipath'),
+ value: PVE.storage.wizard.types.fc.entryStatus(
+ step.checkMultipathEntry(values.nodes),
+ step.canModify,
+ ).value,
+ weight: 21,
+ });
+ }
let unchecked = (step.lunScan?.unchecked ?? []).filter((node) =>
[]
.concat(values.nodes ?? [])
@@ -526,6 +743,14 @@ PVE.storage.wizard.types.fc = {
return { rows, skip: ['vgmode', 'vgname', 'device'] };
},
+ summaryNotes: function (values, wizard) {
+ let step = wizard.down('pveStorageWizardFCLunSelect');
+ let check = step.checkMultipathEntry(values.nodes);
+ let problem =
+ check && PVE.storage.wizard.types.fc.entryStatus(check, step.canModify).problem;
+ return problem ? [`<i class="fa fa-exclamation-triangle warning"></i> ${problem}`] : [];
+ },
+
submit: function (wizard, values) {
let step = wizard.down('pveStorageWizardFCLunSelect');
if (!step.currentSelection(values)) {
@@ -550,14 +775,42 @@ PVE.storage.wizard.types.fc = {
saferemove: values.saferemove,
};
+ // with Sys.Modify, which creating a volume group needs anyway; without it, the entry has
+ // to cover the nodes already, checked again as it may have changed since
+ let caps = Ext.state.Manager.get('GuiCap');
+ let coverEntry = (next) => {
+ let entry = step.selection?.multipathEntry;
+ if (!entry) {
+ next();
+ return;
+ }
+ if (caps.dc['Sys.Modify']) {
+ PVE.storage.wizard.coverMultipathEntry(wizard, entry, values.nodes, next);
+ return;
+ }
+ step.checkMultipathEntry(
+ values.nodes,
+ function (check) {
+ if (check.state === 'covered') {
+ next();
+ return;
+ }
+ let status = PVE.storage.wizard.types.fc.entryStatus(check, false);
+ Ext.Msg.alert(gettext('Error'), status.problem ?? status.value);
+ },
+ (options) => wizard.request(options),
+ );
+ };
+
if (mode === 'existing') {
- wizard.runSubmitChain([(next) => wizard.createStorage(params, next)]);
+ wizard.runSubmitChain([coverEntry, (next) => wizard.createStorage(params, next)]);
return;
}
let created = (wizard.created ??= {});
let node = wizard.getScanNode();
- let device = values.device;
+ let wwid = step.selection.enableMultipath;
+ let device = wwid ? `/dev/disk/by-id/dm-uuid-mpath-${wwid}` : values.device;
let vgCreatedHint = () =>
Ext.String.format(
@@ -569,10 +822,83 @@ PVE.storage.wizard.types.fc = {
enc(wizard.down('#submit').text),
);
+ // a node without the new allow-list entry could let LVM take one of the paths
+ let waitForMaps = function (next) {
+ let nodeList = []
+ .concat(values.nodes ?? [])
+ .join(',')
+ .split(',');
+ nodeList = nodeList.filter((entry) => entry !== '');
+ let online = PVE.data.ResourceStore.getNodes()
+ .filter((entry) => entry.status === 'online')
+ .map((entry) => entry.node);
+ let wanted = (nodeList.length ? nodeList : online).filter((entry) =>
+ online.includes(entry),
+ );
+ if (!wanted.includes(node)) {
+ wanted.push(node);
+ }
+ let deadline = Date.now() + 2 * PVE.storage.wizard.scanTimeout;
+ let pending = wanted;
+ let poll = function () {
+ if (wizard.destroyed) {
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: '/cluster/multipath/status',
+ method: 'GET',
+ failure: function (response) {
+ wizard.setBusy(false);
+ Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+ },
+ success: function (response) {
+ if (wizard.destroyed) {
+ return;
+ }
+ let status = response.result.data || {};
+ let lun = (status.luns || []).find((entry) => entry.wwid === wwid);
+ pending = wanted.filter((entry) => {
+ let report = status.nodes?.[entry];
+ let state = lun?.nodes?.[entry];
+ return !(
+ report &&
+ !report.reason &&
+ ['optimal', 'degraded'].includes(state?.state) &&
+ state['paths-active'] > 0
+ );
+ });
+ if (!pending.length) {
+ wizard.setBusy(false);
+ next();
+ } else if (Date.now() >= deadline) {
+ wizard.setBusy(false);
+ Ext.Msg.alert(
+ gettext('Error'),
+ Ext.String.format(
+ gettext(
+ 'The multipath map of the LUN is not ready yet on {0}. Check its state under Datacenter > SAN > Multipath, then press {1} to wait again.',
+ ),
+ enc(pending.join(', ')),
+ enc(wizard.down('#submit').text),
+ ),
+ );
+ } else {
+ Ext.defer(poll, PVE.storage.wizard.scanInterval);
+ }
+ },
+ });
+ };
+ wizard.setBusy(true, gettext('Waiting for the multipath map on the nodes...'));
+ poll();
+ };
+
let volumeGroupFailure = function (msg, final) {
if (final) {
delete created.task;
- wizard.unfreeze();
+ // Only a newly committed entry pins the failed setup to this device.
+ if (!created.entry) {
+ wizard.unfreeze();
+ }
} else {
msg +=
'<br><br>' +
@@ -625,12 +951,25 @@ PVE.storage.wizard.types.fc = {
wizard.runSubmitChain([
(next) => {
- if (created.task || created.volumeGroup) {
+ if (created.entry || created.task || created.volumeGroup) {
next();
return;
}
wizard.assertStorageIdsFree([params.storage], next);
},
+ (next) => {
+ if (!wwid || created.entry) {
+ next();
+ return;
+ }
+ PVE.storage.wizard.coverMultipathEntry(wizard, wwid, values.nodes, () => {
+ created.entry = wwid;
+ wizard.freeze();
+ next();
+ });
+ },
+ (next) => (wwid && !created.volumeGroup && !created.task ? waitForMaps(next) : next()),
+ (next) => (created.task || created.volumeGroup ? next() : coverEntry(next)),
(next) => {
if (created.volumeGroup) {
next();
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 73/76] ui: storage wizard: let root@pam wipe a LUN with data
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (71 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 72/76] ui: storage wizard: fibre channel: enable multipath for a LUN Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 74/76] ui: storage wizard: do not offer LUNs in use for a new volume group Thomas Lamprecht
` (2 subsequent siblings)
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Let root@pam wipe a data-bearing LUN without leaving shared LVM setup,
using the Node > SAN confirmation and refusal checks. Unknown usage or
volume-group membership never makes a device wipeable.
Track the same wipe across scans. Block selection and submission until
its outcome is known: resume checks for known tasks, but require a
task-history check before dismissing an unanswered request.
After failed Fibre Channel volume-group creation, recheck the original
device and storage configuration before offering recovery on Confirm.
Distinguish an untouched LUN, an existing group, and a refused wipe from
probe failures. Discard obsolete recovery attempts and block recovery
while creation is pending. Verify the pinned device is unused after
every wipe outcome before retrying. Display-only devices remain
ineligible for wiping.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
www/manager6/Makefile | 2 +
www/manager6/SANUtils.js | 8 +-
www/manager6/storage/wizard/FCRecovery.js | 242 ++++++++++++++++++++
www/manager6/storage/wizard/FibreChannel.js | 89 ++++++-
www/manager6/storage/wizard/LunPicker.js | 70 ++++++
www/manager6/storage/wizard/LunSelect.js | 70 ++++--
www/manager6/storage/wizard/WipeStatus.js | 34 +++
www/manager6/storage/wizard/Wizard.js | 116 +++++++++-
8 files changed, 597 insertions(+), 34 deletions(-)
create mode 100644 www/manager6/storage/wizard/FCRecovery.js
create mode 100644 www/manager6/storage/wizard/WipeStatus.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index 8717d041a..8b410d91c 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -390,11 +390,13 @@ JSSRC= \
storage/wizard/CommonSettings.js \
storage/wizard/LunPicker.js \
storage/wizard/LunSelect.js \
+ storage/wizard/WipeStatus.js \
storage/wizard/NodeIdentities.js \
storage/wizard/SanLvm.js \
storage/wizard/NFS.js \
storage/wizard/CIFS.js \
storage/wizard/IScsi.js \
+ storage/wizard/FCRecovery.js \
storage/wizard/FibreChannel.js \
storage/wizard/ZFSOverISCSI.js \
Workspace.js \
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index 7bbb287e2..801f77777 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -367,10 +367,14 @@ Ext.define('PVE.SANUtils', {
return Number(response?.result?.status) === 403;
},
- // returns { text, tip } as HTML; an attribution to ownStorage alone does not count as use
- san_lun_usage: function (row, { ownStorage } = {}) {
+ // returns { text, tip } as HTML; an attribution to ownStorage alone does not count as use, and
+ // refusal says why the row cannot be wiped
+ san_lun_usage: function (row, { ownStorage, refusal } = {}) {
let text = PVE.SANUtils.san_lun_usage_text(row, ownStorage);
let tooltip = [text];
+ if (refusal) {
+ tooltip.push(Ext.htmlEncode(refusal));
+ }
if (row['duplicate-pv']) {
let tip = gettext(
'Another LUN has an LVM physical volume with the same UUID. LVM refuses to activate or change its volume group.',
diff --git a/www/manager6/storage/wizard/FCRecovery.js b/www/manager6/storage/wizard/FCRecovery.js
new file mode 100644
index 000000000..d4a337c1c
--- /dev/null
+++ b/www/manager6/storage/wizard/FCRecovery.js
@@ -0,0 +1,242 @@
+Ext.define('PVE.storage.wizard.FCRecovery', {
+ extend: 'Ext.container.Container',
+ xtype: 'pveStorageWizardFCRecovery',
+ layout: { type: 'vbox', align: 'stretch' },
+
+ invalidate: function () {
+ delete this.attempt;
+ this.hide();
+ },
+
+ initComponent: function () {
+ let me = this;
+ me.callParent();
+ // only added to the confirm tab of a constructed wizard
+ me.on(
+ 'added',
+ function () {
+ let wizard = me.up('window');
+ me.mon(wizard, 'setupchange', function () {
+ me.down('#wipe').setDisabled(me.blocked());
+ me.down('#check').setDisabled(me.blocked());
+ });
+ me.mon(wizard.down('#wizcontent'), 'tabchange', function (tp, tab) {
+ if (tab !== wizard.down('#confirm')) {
+ me.invalidate();
+ }
+ });
+ },
+ me,
+ { single: true },
+ );
+ },
+
+ discover: function (node, lun, vgname) {
+ let me = this;
+ let field = 'wwid';
+ me.attempt = { node, identity: lun?.[field], field, vgname };
+ me.check();
+ },
+
+ blocked: function () {
+ let wizard = this.up('window');
+ return !!(
+ wizard.busy ||
+ wizard.wipeOperation ||
+ wizard.created?.task ||
+ wizard.created?.volumeGroup ||
+ wizard.unconfirmedStorage ||
+ wizard.unknownVolumeGroup
+ );
+ },
+
+ check: function () {
+ let me = this;
+ let wizard = me.up('window');
+ let attempt = me.attempt;
+ if (!attempt || me.blocked()) {
+ return;
+ }
+ let token = (me.checkToken = (me.checkToken ?? 0) + 1);
+ let current = () => !me.destroyed && me.attempt === attempt && token === me.checkToken;
+ me.show();
+ me.down('#wipe').hide();
+ me.down('#check').hide();
+ me.down('#note').setValue(gettext('Checking the selected device...'));
+ let failure = function () {
+ if (!current()) {
+ return;
+ }
+ me.down('#note').setValue(
+ Ext.htmlEncode(
+ Ext.String.format(
+ gettext(
+ 'Could not check LUN "{0}" for a leftover physical volume. Press Check Again.',
+ ),
+ attempt.identity || Proxmox.Utils.unknownText,
+ ),
+ ),
+ );
+ me.down('#check').show();
+ };
+ if (!attempt.identity || attempt.identity === 'unknown') {
+ failure();
+ return;
+ }
+ Proxmox.Utils.API2Request({
+ url: `/nodes/${attempt.node}/scan/san-luns`,
+ method: 'GET',
+ failure,
+ success: function (response) {
+ if (!current()) {
+ return;
+ }
+ let rows = PVE.SANUtils.group_san_lun_paths(response.result.data || []).filter(
+ (row) => row[attempt.field] === attempt.identity,
+ );
+ if (rows.length !== 1) {
+ failure();
+ return;
+ }
+ let lun = rows[0];
+ Proxmox.Utils.API2Request({
+ url: '/storage',
+ method: 'GET',
+ failure,
+ success: function (config) {
+ if (!current()) {
+ return;
+ }
+ if (
+ attempt.verifyUnused &&
+ lun.usage === 'unused' &&
+ !lun['used-by-storage'] &&
+ lun['in-use'] === 'no'
+ ) {
+ attempt.ready = true;
+ me.down('#note').setValue(
+ Ext.htmlEncode(
+ Ext.String.format(
+ gettext(
+ 'LUN "{0}" is now unused. Press Retry Setup to create volume group "{1}" on this LUN.',
+ ),
+ attempt.identity,
+ attempt.vgname,
+ ),
+ ),
+ );
+ wizard.updateActions();
+ return;
+ }
+ if (lun.usage === 'unused' && !attempt.verifyUnused) {
+ me.down('#note').setValue(
+ Ext.htmlEncode(
+ Ext.String.format(
+ gettext(
+ 'No leftover physical volume was found on LUN "{0}".',
+ ),
+ attempt.identity,
+ ),
+ ),
+ );
+ return;
+ }
+ if (!['lvm', 'other'].includes(lun.usage)) {
+ failure();
+ return;
+ }
+ // an empty VG name alone does not prove an orphan PV
+ let refusal = PVE.window.SANLunWipe.reason(lun, config.result.data || []);
+ if (refusal) {
+ me.down('#note').setValue(Ext.htmlEncode(refusal));
+ me.down('#check').show();
+ return;
+ }
+ if (lun.vgname) {
+ me.down('#note').setValue(
+ Ext.htmlEncode(
+ Ext.String.format(
+ gettext(
+ 'LUN "{0}" contains volume group "{1}". If this setup created it, complete the setup under Datacenter > Storage > Add > LVM.',
+ ),
+ attempt.identity,
+ lun.vgname,
+ ),
+ ),
+ );
+ return;
+ }
+ if (lun.usage !== 'lvm') {
+ failure();
+ return;
+ }
+ attempt.lun = lun;
+ me.down('#note').setValue(
+ Ext.htmlEncode(
+ Ext.String.format(
+ gettext(
+ 'An LVM physical volume without a volume group remains on LUN "{0}" on node "{1}". Wipe it only if it is no longer needed anywhere.',
+ ),
+ attempt.identity,
+ attempt.node,
+ ),
+ ),
+ );
+ me.down('#wipe').show();
+ wizard.updateActions();
+ },
+ });
+ },
+ });
+ },
+
+ wipe: function () {
+ let me = this;
+ let wizard = me.up('window');
+ let attempt = me.attempt;
+ if (!attempt?.lun || Proxmox.UserName !== 'root@pam' || me.blocked()) {
+ return;
+ }
+ wizard.openWipe(attempt.node, attempt.lun, function (wiped) {
+ if (me.destroyed || me.attempt !== attempt) {
+ return;
+ }
+ if (wizard.created?.entry) {
+ attempt.verifyUnused = true;
+ attempt.ready = false;
+ me.check();
+ } else {
+ wizard.down('pveStorageWizardFCLunSelect').refreshLuns();
+ if (wiped) {
+ me.show();
+ me.down('#note').setValue(
+ gettext(
+ 'Wipe completed. Go back to the SAN LUN step and select a LUN before retrying.',
+ ),
+ );
+ me.down('#wipe').hide();
+ me.down('#check').hide();
+ }
+ }
+ });
+ },
+
+ items: [
+ { xtype: 'displayfield', itemId: 'note' },
+ {
+ xtype: 'button',
+ itemId: 'check',
+ text: gettext('Check Again'),
+ hidden: true,
+ handler: (button) => button.up('pveStorageWizardFCRecovery').check(),
+ },
+ {
+ xtype: 'button',
+ itemId: 'wipe',
+ text: gettext('Wipe...'),
+ hidden: true,
+ cls: 'pve-dangerous-button',
+ handler: (button) => button.up('pveStorageWizardFCRecovery').wipe(),
+ },
+ ],
+});
diff --git a/www/manager6/storage/wizard/FibreChannel.js b/www/manager6/storage/wizard/FibreChannel.js
index b0f5922f4..b44ec393a 100644
--- a/www/manager6/storage/wizard/FibreChannel.js
+++ b/www/manager6/storage/wizard/FibreChannel.js
@@ -121,21 +121,39 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
return check;
},
+ wipeable: function (rec) {
+ let me = this;
+ return (
+ !!rec &&
+ me.allowWipe &&
+ !rec.get('reason') &&
+ !PVE.window.SANLunWipe.reason(rec.data, me.storageConfig)
+ );
+ },
+
validator: function () {
let wizard = this.up('window');
- return !wizard.busy && !!wizard.getViewModel().get('vgmode');
+ return !wizard.busy && !wizard.wipeOperation && !!wizard.getViewModel().get('vgmode');
},
updateActions: function () {
let me = this;
let wizard = me.up('window');
let pinned = !!wizard.created?.entry;
- let blocked = wizard.busy || pinned;
+ let blocked = wizard.busy || !!wizard.wipeOperation || pinned;
let grid = me.down('#lunGrid');
grid.setDisabled(blocked || me.scanning);
+ let optin = me.down('#allowWipe');
+ optin.setDisabled(blocked);
+ optin.setHidden(pinned || Proxmox.UserName !== 'root@pam');
+ if (pinned && optin.getValue()) {
+ optin.setValue(false);
+ }
me.down('#rescan').setHidden(pinned);
+ me.down('#wipeButton').setHidden(pinned || !me.allowWipe);
me.down('#rescan').setDisabled(blocked);
me.down('pveStorageScanNodeSelector')?.setDisabled(blocked);
+ me.down('#wipeButton').setDisabled(blocked || !me.wipeable(grid.getSelection()[0]));
if (wizard.down('#wizcontent').getActiveTab() === me) {
wizard.down('#next').setDisabled(!me.validator());
}
@@ -216,6 +234,7 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
let grid = me.down('#lunGrid');
me.scanning = true;
me.scanDone = false;
+ wizard.down('pveStorageWizardFCRecovery')?.invalidate();
grid.getSelectionModel().deselectAll();
me.selection = undefined;
me.lunScan = undefined;
@@ -406,6 +425,7 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
let updateSelection = function (rec) {
let wizard = me.up('window');
let multipath = rec?.get('multipath');
+ wizard.down('pveStorageWizardFCRecovery')?.invalidate();
delete me.entryCheck;
me.selection = rec
? {
@@ -421,6 +441,8 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
}
: undefined;
let vm = wizard.getViewModel();
+ me.down('#wipeButton')?.setDisabled(!me.wipeable(rec));
+ // a row selected only for wiping is not the storage's LUN, so the step stays incomplete
if (!rec || !rec.get('selectable')) {
vm.set({ vgmode: '', device: '', vgname: '' });
} else if (rec.get('usage') === 'unused') {
@@ -439,6 +461,7 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
xtype: 'displayfield',
value: gettext('Select the SAN LUN for the volume group:'),
},
+ { xtype: 'pveStorageWizardWipeStatus' },
{
xtype: 'grid',
itemId: 'lunGrid',
@@ -446,10 +469,11 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
store: store,
viewConfig: {
trackOver: false,
- getRowClass: (rec) => (rec.get('selectable') ? '' : 'x-item-disabled'),
+ getRowClass: (rec) =>
+ rec.get('selectable') || me.wipeable(rec) ? '' : 'x-item-disabled',
},
listeners: {
- beforeselect: (grid, rec) => rec.get('selectable'),
+ beforeselect: (grid, rec) => rec.get('selectable') || me.wipeable(rec),
selectionchange: (sm, selection) => updateSelection(selection[0]),
},
columns: [
@@ -498,15 +522,21 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
{
xtype: 'container',
layout: 'hbox',
- items: [
- {
- xtype: 'button',
- itemId: 'rescan',
- text: gettext('Rescan'),
- iconCls: 'fa fa-search',
- handler: () => me.refreshLuns(),
+ items: me.wipeControls(() => me.refreshLuns(), {
+ margin: '0 0 0 10',
+ handler: function () {
+ let rec = me.down('#lunGrid').getSelection()[0];
+ if (!me.wipeable(rec)) {
+ return;
+ }
+ let wizard = me.up('window');
+ wizard.openWipe(wizard.getScanNode(), rec.data, () => {
+ if (!me.destroyed) {
+ me.refreshLuns();
+ }
+ });
},
- ],
+ }),
},
{
xtype: 'proxmoxcheckbox',
@@ -564,6 +594,7 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
xtype: 'pveStorageScanNodeSelector',
listeners: {
change: function () {
+ me.up('window').down('pveStorageWizardFCRecovery')?.invalidate();
me.refreshLuns();
},
},
@@ -639,7 +670,16 @@ PVE.storage.wizard.types.fc = {
],
},
+ confirmItems: () => [
+ { xtype: 'pveStorageWizardWipeStatus' },
+ { xtype: 'pveStorageWizardFCRecovery', hidden: true },
+ ],
+
canSubmit: function (wizard) {
+ let attempt = wizard.down('pveStorageWizardFCRecovery')?.attempt;
+ if (attempt?.verifyUnused && !attempt.ready) {
+ return false;
+ }
let values = wizard.getValues();
let step = wizard.down('pveStorageWizardFCLunSelect');
let check = step.checkMultipathEntry(values.nodes);
@@ -760,6 +800,7 @@ PVE.storage.wizard.types.fc = {
);
return;
}
+ wizard.down('pveStorageWizardFCRecovery')?.invalidate();
let enc = Ext.htmlEncode;
let mode = values.vgmode;
delete values.vgmode;
@@ -892,6 +933,13 @@ PVE.storage.wizard.types.fc = {
poll();
};
+ let failedLun = Ext.clone(step.selection.lun);
+ let offerWipe = function () {
+ if (Proxmox.UserName === 'root@pam') {
+ wizard.down('pveStorageWizardFCRecovery').discover(node, failedLun, params.vgname);
+ }
+ };
+
let volumeGroupFailure = function (msg, final) {
if (final) {
delete created.task;
@@ -899,6 +947,22 @@ PVE.storage.wizard.types.fc = {
if (!created.entry) {
wizard.unfreeze();
}
+ offerWipe();
+ if (Proxmox.UserName === 'root@pam') {
+ msg = Ext.String.format(
+ gettext(
+ 'Volume group setup did not complete: {0}. The Confirm page checks this LUN for a leftover physical volume and offers Wipe... if the local checks allow it. If volume group "{1}" already exists, complete the setup under Datacenter > Storage > Add > LVM.',
+ ),
+ msg,
+ enc(params.vgname),
+ );
+ } else {
+ msg +=
+ '<br><br>' +
+ gettext(
+ 'If a leftover physical volume remains, root@pam can wipe it under Node > SAN before retrying. If the volume group already exists, complete the setup under Datacenter > Storage > Add > LVM.',
+ );
+ }
} else {
msg +=
'<br><br>' +
@@ -941,6 +1005,7 @@ PVE.storage.wizard.types.fc = {
),
},
success: function () {
+ wizard.down('pveStorageWizardFCRecovery')?.invalidate();
created.volumeGroup = params.vgname;
delete created.task;
next();
diff --git a/www/manager6/storage/wizard/LunPicker.js b/www/manager6/storage/wizard/LunPicker.js
index 7ae6b0564..5bcaace44 100644
--- a/www/manager6/storage/wizard/LunPicker.js
+++ b/www/manager6/storage/wizard/LunPicker.js
@@ -1,3 +1,4 @@
+// the picker provides wipeable(rec) and a #lunGrid
Ext.define('PVE.storage.wizard.LunPicker', {
extend: 'Ext.Mixin',
@@ -5,6 +6,75 @@ Ext.define('PVE.storage.wizard.LunPicker', {
id: 'lunPicker',
},
+ allowWipe: false,
+ storageConfig: undefined,
+ allowWipeText: gettext('Allow wiping LUNs with data'),
+
+ setAllowWipe: function (allow) {
+ let me = this;
+ let grid = me.down('#lunGrid');
+ let token = (me.wipeConfigToken = (me.wipeConfigToken ?? 0) + 1);
+ let apply = function (storages) {
+ if (me.destroyed || token !== me.wipeConfigToken) {
+ return;
+ }
+ me.allowWipe = allow;
+ me.storageConfig = storages;
+ me.down('#wipeButton').setHidden(!allow);
+ let rec = grid.getSelection()[0];
+ if (rec && !rec.get('selectable') && !me.wipeable(rec)) {
+ grid.getSelectionModel().deselectAll();
+ }
+ grid.getView().refresh();
+ me.up('window').updateActions();
+ };
+ if (!allow) {
+ apply(undefined);
+ return;
+ }
+ // the wipe check needs the storages as they are now, a base storage may be new
+ Proxmox.Utils.API2Request({
+ url: '/storage',
+ method: 'GET',
+ failure: () => apply(undefined),
+ success: (response) => apply(response.result.data || []),
+ });
+ },
+
+ // wipeButton completes the config of the Wipe... button
+ wipeControls: function (rescan, wipeButton) {
+ let me = this;
+ return [
+ {
+ xtype: 'button',
+ itemId: 'rescan',
+ text: gettext('Rescan'),
+ iconCls: 'fa fa-search',
+ handler: rescan,
+ },
+ {
+ xtype: 'proxmoxcheckbox',
+ itemId: 'allowWipe',
+ margin: '0 10',
+ hidden: Proxmox.UserName !== 'root@pam',
+ boxLabel: me.allowWipeText,
+ submitValue: false,
+ listeners: {
+ change: (field, value) => me.setAllowWipe(value),
+ },
+ },
+ {
+ xtype: 'button',
+ itemId: 'wipeButton',
+ text: gettext('Wipe...'),
+ cls: 'pve-dangerous-button',
+ hidden: true,
+ disabled: true,
+ ...wipeButton,
+ },
+ ];
+ },
+
lunColumns: function () {
return {
paths: {
diff --git a/www/manager6/storage/wizard/LunSelect.js b/www/manager6/storage/wizard/LunSelect.js
index 4e138f787..947105f4d 100644
--- a/www/manager6/storage/wizard/LunSelect.js
+++ b/www/manager6/storage/wizard/LunSelect.js
@@ -23,9 +23,14 @@ Ext.define('PVE.storage.wizard.LunSelect', {
tabTitle: gettext('LUN'),
selectText: gettext('Select an unused LUN for the volume group.'),
+ wipeSelectText: gettext(
+ 'Select an unused LUN for the volume group. If a LUN holds data that is no longer needed anywhere and no configured storage uses it for data, you can wipe it first.',
+ ),
waitText: gettext('Waiting for node "{0}" to list the LUNs of the target...'),
errorText: gettext('Could not list the LUNs of the target on node "{0}": {1}'),
selectedText: gettext('The volume group will be created on LUN {0} ({1} {2}, {3}).'),
+ wipeText: gettext('LUN {0} ({1} {2}, {3}) holds data. Press Wipe... to review and wipe it.'),
+ wipedText: gettext('Wipe completed. Select a LUN to continue.'),
pinnedText: gettext(
'Press {4} to retry on LUN {0} ({1} {2}, {3}). The wizard keeps this LUN, as the previous attempt may have written to it.',
),
@@ -69,6 +74,15 @@ Ext.define('PVE.storage.wizard.LunSelect', {
return undefined;
},
+ wipeable: function (rec) {
+ let me = this;
+ return (
+ me.allowWipe &&
+ !rec.get('reason') &&
+ !PVE.window.SANLunWipe.reason(rec.data, me.storageConfig)
+ );
+ },
+
getSelection: function () {
return this.down('#lunGrid').getSelection()[0]?.data;
},
@@ -93,10 +107,20 @@ Ext.define('PVE.storage.wizard.LunSelect', {
let grid = me.down('#lunGrid');
let rec = grid.getSelection()[0];
let row = pinned ? wizard.created.lun : rec?.data;
- let blocked = wizard.busy;
+ let blocked = wizard.busy || !!wizard.wipeOperation;
grid.setDisabled(!!pinned || blocked || me.scanning || me.scanError);
me.down('#rescan').setHidden(!!pinned);
me.down('#rescan').setDisabled(blocked || me.scanning);
+ let optin = me.down('#allowWipe');
+ optin.setHidden(!!pinned || Proxmox.UserName !== 'root@pam');
+ optin.setDisabled(blocked || me.scanning);
+ if (pinned && optin.getValue()) {
+ optin.setValue(false);
+ }
+ me.down('#wipeButton').setHidden(!!pinned || !me.allowWipe);
+ me.down('#wipeButton').setDisabled(
+ blocked || me.scanning || me.scanError || !rec || !me.wipeable(rec),
+ );
me.down('#footnote').setHidden(!!pinned || !!wizard.targetUse?.reuse);
let text;
if (wizard.unconfirmedStorage) {
@@ -108,7 +132,7 @@ Ext.define('PVE.storage.wizard.LunSelect', {
} else if (me.scanError) {
text = '';
} else if (row) {
- text = me.selectedText;
+ text = rec.get('selectable') ? me.selectedText : me.wipeText;
} else if (me.activationOutcome && !grid.getStore().getCount()) {
let outcome = me.activationOutcome;
text = Ext.String.format(
@@ -122,15 +146,17 @@ Ext.define('PVE.storage.wizard.LunSelect', {
me.node,
me.initiator || Proxmox.Utils.unknownText,
);
+ } else if (me.wiped) {
+ text = me.wipedText;
} else if (
!grid
.getStore()
.getRange()
- .some((r) => r.get('selectable'))
+ .some((r) => r.get('selectable') || me.wipeable(r))
) {
text = Ext.String.format(me.noneText, me.initiator || Proxmox.Utils.unknownText);
} else {
- text = me.selectText;
+ text = me.allowWipe ? me.wipeSelectText : me.selectText;
}
if (row && !wizard.unconfirmedStorage && !me.scanning && !me.scanError) {
text = Ext.String.format(
@@ -142,7 +168,7 @@ Ext.define('PVE.storage.wizard.LunSelect', {
wizard.actionText(),
);
}
- me.down('#stateLine').setValue(Ext.htmlEncode(text || ''));
+ me.down('#stateLine').setStatus(text);
},
showScanError: function (errorHtml) {
@@ -197,7 +223,7 @@ Ext.define('PVE.storage.wizard.LunSelect', {
value: me.recap,
},
{
- xtype: 'displayfield',
+ xtype: 'pveStorageWizardWipeStatus',
itemId: 'stateLine',
},
{
@@ -207,10 +233,11 @@ Ext.define('PVE.storage.wizard.LunSelect', {
store: store,
viewConfig: {
trackOver: false,
- getRowClass: (rec) => (rec.get('selectable') ? '' : 'x-item-disabled'),
+ getRowClass: (rec) =>
+ rec.get('selectable') || me.wipeable(rec) ? '' : 'x-item-disabled',
},
listeners: {
- beforeselect: (grid, rec) => rec.get('selectable'),
+ beforeselect: (grid, rec) => rec.get('selectable') || me.wipeable(rec),
selectionchange: function () {
me.selectionGeneration++;
me.updateButtons();
@@ -247,9 +274,14 @@ Ext.define('PVE.storage.wizard.LunSelect', {
metaData.tdAttr = `data-qtip="${Ext.htmlEncode(text)}"`;
return text;
}
+ let refusal =
+ me.allowWipe && !rec.get('selectable')
+ ? PVE.window.SANLunWipe.reason(rec.data, me.storageConfig)
+ : undefined;
let { text, tip } = PVE.SANUtils.san_lun_usage(rec.data, {
// every LUN of the storage carries its attribution
ownStorage: me.storage,
+ refusal,
});
metaData.tdAttr = `data-qtip="${Ext.htmlEncode(tip)}"`;
return text;
@@ -259,15 +291,20 @@ Ext.define('PVE.storage.wizard.LunSelect', {
},
];
- let controls = [
- {
- xtype: 'button',
- itemId: 'rescan',
- text: gettext('Rescan'),
- iconCls: 'fa fa-search',
- handler: () => me.doRescan(),
+ let controls = me.wipeControls(() => me.doRescan(), {
+ handler: function () {
+ let rec = me.down('#lunGrid').getSelection()[0];
+ if (!rec || !me.wipeable(rec) || me.up('window').created?.baseVolume) {
+ return;
+ }
+ me.up('window').openWipe(me.node, rec.data, function (wiped) {
+ if (!me.destroyed) {
+ me.wiped = wiped;
+ me.waitForLuns(false);
+ }
+ });
},
- ];
+ });
me.items.push({ xtype: 'container', layout: 'hbox', margin: '5 0', items: controls });
me.items.push({ xtype: 'displayfield', itemId: 'footnote', value: me.footnote });
@@ -440,6 +477,7 @@ Ext.define('PVE.storage.wizard.LunSelect', {
doRescan: function () {
let me = this;
+ me.wiped = false;
me.preselectionConsumed = true;
me.waitForLuns(false);
},
diff --git a/www/manager6/storage/wizard/WipeStatus.js b/www/manager6/storage/wizard/WipeStatus.js
new file mode 100644
index 000000000..a79c25aec
--- /dev/null
+++ b/www/manager6/storage/wizard/WipeStatus.js
@@ -0,0 +1,34 @@
+Ext.define('PVE.storage.wizard.WipeStatus', {
+ extend: 'Ext.container.Container',
+ xtype: 'pveStorageWizardWipeStatus',
+ layout: { type: 'hbox', align: 'middle' },
+
+ setStatus: function (text) {
+ let me = this;
+ let wizard = me.up('window');
+ let op = wizard.wipeOperation;
+ let warning =
+ op?.state === 'unknown' ? '<i class="fa fa-exclamation-triangle warning"></i> ' : '';
+ me.down('#text').setValue(warning + Ext.htmlEncode(wizard.wipeStatusText() || text || ''));
+ me.down('#resume').setVisible(!!op?.upid && !wizard.busy);
+ me.down('#clear').setVisible(op?.state === 'unknown' && !op.upid && !wizard.busy);
+ },
+
+ items: [
+ { xtype: 'displayfield', itemId: 'text', flex: 1, value: '' },
+ {
+ xtype: 'button',
+ itemId: 'resume',
+ text: gettext('Check Task Status'),
+ hidden: true,
+ handler: (button) => button.up('window').waitForWipe(),
+ },
+ {
+ xtype: 'button',
+ itemId: 'clear',
+ text: gettext('Dismiss'),
+ hidden: true,
+ handler: (button) => button.up('window').clearWipe(),
+ },
+ ],
+});
diff --git a/www/manager6/storage/wizard/Wizard.js b/www/manager6/storage/wizard/Wizard.js
index 433612b1a..d50291584 100644
--- a/www/manager6/storage/wizard/Wizard.js
+++ b/www/manager6/storage/wizard/Wizard.js
@@ -155,7 +155,7 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
canSubmit: function () {
let me = this;
- if (me.busy || me.unknownVolumeGroup) {
+ if (me.busy || me.wipeOperation || me.unknownVolumeGroup) {
return false;
}
let canSubmit = PVE.storage.wizard.types[me.wizardType].canSubmit;
@@ -171,6 +171,7 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
me.lunView &&
me.targetUse?.reuse &&
!me.busy &&
+ !me.wipeOperation &&
!me.unconfirmedStorage &&
Object.keys(me.created ?? {}).every((key) => key === 'baseStorage')
);
@@ -202,15 +203,115 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
let confirmActive = me.down('#wizcontent').getActiveTab() === me.down('#confirm');
if (PVE.storage.wizard.types[me.wizardType].canSubmit && confirmActive) {
me.down('#submit').setDisabled(!me.canSubmit());
- me.down('#back').setDisabled(!!me.frozenValues || me.busy);
+ me.down('#back').setDisabled(!!me.frozenValues || me.busy || !!me.wipeOperation);
}
+ me.query('pveStorageWizardWipeStatus').forEach((status) => {
+ if (status !== me.lunView?.down('#stateLine')) {
+ status.setStatus();
+ }
+ });
me.fireEvent('setupchange', me);
// after the listeners, so no step can enable an action again
- if (me.busy || me.unknownVolumeGroup) {
+ if (me.busy || me.wipeOperation || me.unknownVolumeGroup) {
['#back', '#next', '#submit'].forEach((id) => me.down(id).disable());
}
},
+ wipeStatusText: function () {
+ let op = this.wipeOperation;
+ if (!op) {
+ return '';
+ }
+ let t = PVE.window.SANLunWipe.texts(op.lun);
+ if (op.state === 'pending' || op.state === 'waiting') {
+ return Ext.String.format(t.wiping, op.identity, op.node);
+ }
+ if (!op.upid) {
+ return Ext.String.format(t.unknownRequest, op.node);
+ }
+ return Ext.String.format(
+ t.unknownTask,
+ op.lun['iscsi-lun'] ?? op.lun.devpath,
+ 'WWID',
+ op.identity,
+ );
+ },
+
+ openWipe: function (node, lun, refresh) {
+ let me = this;
+ if (me.busy || me.wipeOperation) {
+ return;
+ }
+ Ext.create('PVE.window.SANLunWipe', {
+ node,
+ lun,
+ autoShow: true,
+ onSubmit: function (op) {
+ op.refresh = refresh;
+ me.wipeOperation = op;
+ me.setBusy(true, gettext('Wiping...'));
+ },
+ onClose: function (op) {
+ if (me.destroyed || me.wipeOperation !== op) {
+ return;
+ }
+ if (op.state !== 'unknown') {
+ delete me.wipeOperation;
+ }
+ me.setBusy(false);
+ refresh(false);
+ me.updateActions();
+ },
+ callback: function (op) {
+ if (me.destroyed || me.wipeOperation !== op) {
+ return;
+ }
+ me.waitForWipe();
+ },
+ });
+ },
+
+ waitForWipe: function () {
+ let me = this;
+ let op = me.wipeOperation;
+ if (!op?.upid) {
+ return;
+ }
+ op.state = 'waiting';
+ let done = function (success, final, msg) {
+ if (me.destroyed || me.wipeOperation !== op) {
+ return;
+ }
+ if (final) {
+ delete me.wipeOperation;
+ } else {
+ op.state = 'unknown';
+ }
+ op.refresh(success);
+ me.updateActions();
+ if (final && !success) {
+ Ext.Msg.alert(gettext('Error'), msg);
+ }
+ };
+ me.waitForTask(op.node, op.upid, {
+ msg: Ext.htmlEncode(me.wipeStatusText()),
+ texts: { unknown: gettext('Could not determine the task outcome') },
+ success: () => done(true, true),
+ failure: (msg, final) => done(false, final, msg),
+ });
+ },
+
+ clearWipe: function () {
+ let me = this;
+ let op = me.wipeOperation;
+ if (me.busy || op?.state !== 'unknown' || op.upid) {
+ return;
+ }
+ delete me.wipeOperation;
+ op.refresh(false);
+ me.updateActions();
+ },
+
activateLunView: function (config) {
let me = this;
if (me.lunView) {
@@ -843,6 +944,10 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
});
me.callParent();
+ let confirmItems = entry.confirmItems?.();
+ if (confirmItems) {
+ me.down('#confirm').add(confirmItems);
+ }
let back = me.down('#back');
let backHandler = back.handler;
back.setHandler(function () {
@@ -857,6 +962,9 @@ Ext.define('PVE.storage.wizard.RemoteStorage', {
field.on('validitychange', () => me.updateActions());
});
me.down('#wizcontent').on('tabchange', () => me.updateActions());
- me.down('#wizcontent').on('beforetabchange', () => !me.busy && !me.frozenValues);
+ me.down('#wizcontent').on(
+ 'beforetabchange',
+ () => !me.busy && !me.wipeOperation && !me.frozenValues,
+ );
},
});
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 74/76] ui: storage wizard: do not offer LUNs in use for a new volume group
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (72 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 73/76] ui: storage wizard: let root@pam wipe a LUN with data Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 75/76] ui: san: show the local users of a LUN in its usage tooltip Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 76/76] ui: san: check the other nodes for users before wiping a LUN Thomas Lamprecht
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
The iSCSI and Fibre Channel setups offered every unused LUN for a new
volume group, including LUNs that the scan node uses, for example
through a device-mapper map built on one of their paths, and LUNs
whose local users could not be checked. The node refuses a new volume
group on such a LUN, so the setup failed at its last step instead.
Show these LUNs as not selectable, naming the user or the failed
check. Existing volume groups can still be reused.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
test/SANUtils_test.js | 17 +++++++++++++++++
www/manager6/SANUtils.js | 16 ++++++++++++++++
www/manager6/storage/wizard/FibreChannel.js | 2 +-
www/manager6/storage/wizard/LunSelect.js | 2 +-
4 files changed, 35 insertions(+), 2 deletions(-)
diff --git a/test/SANUtils_test.js b/test/SANUtils_test.js
index b13d75166..86f97e9bd 100644
--- a/test/SANUtils_test.js
+++ b/test/SANUtils_test.js
@@ -88,3 +88,20 @@ test('paths of a LUN without a map', () => {
lun = group(path('/dev/sdb', 'unknown', 'check failed'), path('/dev/sdc', 'yes', 'dm-5'));
assert.equal(lun['in-use'], 'yes', 'a known user wins over a failed check');
});
+
+test('LUNs not offered for a new volume group', () => {
+ const utils = sanUtils();
+ const reason = (row) => utils.san_lun_in_use_reason({ usage: 'unused', ...row }, 'n1');
+
+ assert.equal(reason({ 'in-use': 'no' }), undefined);
+ assert.equal(reason({ 'in-use': 'yes', 'in-use-by': 'dm-5' }), 'In use on node "n1": dm-5');
+ assert.equal(
+ reason({ 'in-use': 'unknown', 'in-use-by': 'check failed: timeout' }),
+ 'Could not check usage on node "n1"',
+ );
+ assert.equal(
+ utils.san_lun_in_use_reason({ usage: 'lvm', 'in-use': 'yes', 'in-use-by': 'dm-5' }, 'n1'),
+ undefined,
+ 'an existing volume group stays available for reuse',
+ );
+});
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index 801f77777..d3a50c1a9 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -367,6 +367,22 @@ Ext.define('PVE.SANUtils', {
return Number(response?.result?.status) === 403;
},
+ // A node refuses a new volume group on an unused LUN that it uses or could not check, so the
+ // setup should not offer it. Returns the reason, or undefined for any other LUN.
+ san_lun_in_use_reason: function (row, node) {
+ if (row.usage !== 'unused' || row['in-use'] === 'no') {
+ return undefined;
+ }
+ if (row['in-use'] === 'yes') {
+ return Ext.String.format(
+ gettext('In use on node "{0}": {1}'),
+ node,
+ row['in-use-by'] ?? gettext('unknown'),
+ );
+ }
+ return Ext.String.format(gettext('Could not check usage on node "{0}"'), node);
+ },
+
// returns { text, tip } as HTML; an attribution to ownStorage alone does not count as use, and
// refusal says why the row cannot be wiped
san_lun_usage: function (row, { ownStorage, refusal } = {}) {
diff --git a/www/manager6/storage/wizard/FibreChannel.js b/www/manager6/storage/wizard/FibreChannel.js
index b44ec393a..265bc4a78 100644
--- a/www/manager6/storage/wizard/FibreChannel.js
+++ b/www/manager6/storage/wizard/FibreChannel.js
@@ -356,7 +356,7 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
if (data.usage === 'unused' && !data['used-by-storage'] && !data['by-id']) {
return gettext('The LUN has no stable device name for creating a volume group.');
}
- return undefined;
+ return PVE.SANUtils.san_lun_in_use_reason(data, data.scanNode);
};
let columns = me.lunColumns();
diff --git a/www/manager6/storage/wizard/LunSelect.js b/www/manager6/storage/wizard/LunSelect.js
index 947105f4d..f8eba9e0a 100644
--- a/www/manager6/storage/wizard/LunSelect.js
+++ b/www/manager6/storage/wizard/LunSelect.js
@@ -71,7 +71,7 @@ Ext.define('PVE.storage.wizard.LunSelect', {
if (mapped && !(row['paths-active'] > 0)) {
return gettext('No usable path');
}
- return undefined;
+ return PVE.SANUtils.san_lun_in_use_reason(row, me.node);
},
wipeable: function (rec) {
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 75/76] ui: san: show the local users of a LUN in its usage tooltip
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (73 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 74/76] ui: storage wizard: do not offer LUNs in use for a new volume group Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 76/76] ui: san: check the other nodes for users before wiping a LUN Thomas Lamprecht
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
The SAN LUN lists showed whether a LUN holds data, but not what uses it
on the node, for example a logical volume that LVM activated on one
path of a multipath map. Only the refusals named such users.
Add the users, or the failed check for them, to the tooltip of the
usage column in Node > SAN and in the setup wizards.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
test/SANUtils_test.js | 42 ++++++++++++++++++++-
www/manager6/SANUtils.js | 40 +++++++++++++-------
www/manager6/storage/wizard/FibreChannel.js | 6 ++-
www/manager6/storage/wizard/LunSelect.js | 1 +
4 files changed, 73 insertions(+), 16 deletions(-)
diff --git a/test/SANUtils_test.js b/test/SANUtils_test.js
index 86f97e9bd..f60e23b73 100644
--- a/test/SANUtils_test.js
+++ b/test/SANUtils_test.js
@@ -7,7 +7,13 @@ const { test } = require('node:test');
// load PVE.SANUtils with stubs of the ExtJS and translation helpers it uses
function sanUtils(translations = {}) {
const sandbox = {
- PVE: {},
+ PVE: {
+ window: {
+ SANLunWipe: {
+ texts: () => ({ unchecked: 'Could not check whether this node uses the LUN.' }),
+ },
+ },
+ },
gettext: (text) => translations[text] ?? text,
Ext: {
apply: (target, source) => Object.assign(target, source),
@@ -105,3 +111,37 @@ test('LUNs not offered for a new volume group', () => {
'an existing volume group stays available for reuse',
);
});
+
+test('local users of a LUN in its usage tooltip', () => {
+ const utils = sanUtils();
+ const row = (inUse, users = '') => ({
+ usage: 'lvm',
+ vgname: 'vg1',
+ 'in-use': inUse,
+ 'in-use-by': users,
+ });
+ const tip = (lun, options) => utils.san_lun_usage(lun, options).tip;
+
+ assert.doesNotMatch(tip(row('no')), /use/i, 'nothing for a LUN the node does not use');
+ assert.match(
+ tip(row('yes', 'sdb held by vg1-vm--100--disk--0')),
+ /<br>In use on this node: sdb held by vg1-vm--100--disk--0$/,
+ 'the users, in the node view',
+ );
+ assert.match(
+ tip(row('yes', 'dm-5'), { node: 'n1' }),
+ /<br>In use on node "n1": dm-5$/,
+ 'the users and the scan node, in a wizard',
+ );
+ assert.match(
+ tip(row('unknown', 'check failed: timeout')),
+ /<br>Could not check whether this node uses the LUN\.$/,
+ 'a failed check',
+ );
+ const refusal = 'In use on this node: dm-5';
+ assert.equal(
+ tip(row('yes', 'dm-5'), { node: 'n1', refusal }).split('<br>').length,
+ 2,
+ 'not again next to the refusal to wipe that names the users',
+ );
+});
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index d3a50c1a9..dbef08c84 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -367,27 +367,39 @@ Ext.define('PVE.SANUtils', {
return Number(response?.result?.status) === 403;
},
+ // The local users of a LUN or the failed check, undefined if the scan node does not use it.
+ // Without a node, the texts are those of the node's own views, which speak of "this node".
+ san_lun_users_text: function (row, node) {
+ if (row['in-use'] === 'no') {
+ return undefined;
+ }
+ if (row['in-use'] === 'yes') {
+ let users = row['in-use-by'] ?? gettext('unknown');
+ return node === undefined
+ ? Ext.String.format(gettext('In use on this node: {0}'), users)
+ : Ext.String.format(gettext('In use on node "{0}": {1}'), node, users);
+ }
+ return node === undefined
+ ? PVE.window.SANLunWipe.texts(row).unchecked
+ : Ext.String.format(gettext('Could not check usage on node "{0}"'), node);
+ },
+
// A node refuses a new volume group on an unused LUN that it uses or could not check, so the
// setup should not offer it. Returns the reason, or undefined for any other LUN.
san_lun_in_use_reason: function (row, node) {
- if (row.usage !== 'unused' || row['in-use'] === 'no') {
- return undefined;
- }
- if (row['in-use'] === 'yes') {
- return Ext.String.format(
- gettext('In use on node "{0}": {1}'),
- node,
- row['in-use-by'] ?? gettext('unknown'),
- );
- }
- return Ext.String.format(gettext('Could not check usage on node "{0}"'), node);
+ return row.usage === 'unused' ? PVE.SANUtils.san_lun_users_text(row, node) : undefined;
},
- // returns { text, tip } as HTML; an attribution to ownStorage alone does not count as use, and
- // refusal says why the row cannot be wiped
- san_lun_usage: function (row, { ownStorage, refusal } = {}) {
+ // returns { text, tip } as HTML; an attribution to ownStorage alone does not count as use,
+ // refusal says why the row cannot be wiped, and node is the scan node outside the node's views
+ san_lun_usage: function (row, { ownStorage, refusal, node } = {}) {
let text = PVE.SANUtils.san_lun_usage_text(row, ownStorage);
let tooltip = [text];
+ let users = PVE.SANUtils.san_lun_users_text(row, node);
+ // the refusal to wipe a LUN in use names its users already
+ if (users && refusal !== PVE.SANUtils.san_lun_users_text(row)) {
+ tooltip.push(Ext.htmlEncode(users));
+ }
if (refusal) {
tooltip.push(Ext.htmlEncode(refusal));
}
diff --git a/www/manager6/storage/wizard/FibreChannel.js b/www/manager6/storage/wizard/FibreChannel.js
index 265bc4a78..01b8590f4 100644
--- a/www/manager6/storage/wizard/FibreChannel.js
+++ b/www/manager6/storage/wizard/FibreChannel.js
@@ -505,7 +505,11 @@ Ext.define('PVE.storage.wizard.FCLunSelect', {
renderer: function (value, metaData, rec) {
let reason = rec.get('reason');
if (!reason) {
- return PVE.SANUtils.render_san_lun_usage(value, metaData, rec);
+ let { text, tip } = PVE.SANUtils.san_lun_usage(rec.data, {
+ node: rec.get('scanNode'),
+ });
+ metaData.tdAttr = `data-qtip="${Ext.htmlEncode(tip)}"`;
+ return text;
}
let text = Ext.htmlEncode(reason);
metaData.tdAttr = `data-qtip="${Ext.htmlEncode(text)}"`;
diff --git a/www/manager6/storage/wizard/LunSelect.js b/www/manager6/storage/wizard/LunSelect.js
index f8eba9e0a..dc7639803 100644
--- a/www/manager6/storage/wizard/LunSelect.js
+++ b/www/manager6/storage/wizard/LunSelect.js
@@ -282,6 +282,7 @@ Ext.define('PVE.storage.wizard.LunSelect', {
// every LUN of the storage carries its attribution
ownStorage: me.storage,
refusal,
+ node: me.node,
});
metaData.tdAttr = `data-qtip="${Ext.htmlEncode(tip)}"`;
return text;
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
* [PATCH v3 manager 76/76] ui: san: check the other nodes for users before wiping a LUN
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
` (74 preceding siblings ...)
2026-11-05 23:31 ` [PATCH v3 manager 75/76] ui: san: show the local users of a LUN in its usage tooltip Thomas Lamprecht
@ 2026-11-05 23:31 ` Thomas Lamprecht
75 siblings, 0 replies; 77+ messages in thread
From: Thomas Lamprecht @ 2026-11-05 23:31 UTC (permalink / raw)
To: pve-devel
Wiping a LUN checked the storage configuration of the cluster and the
users on the node that wipes, but not what other nodes use, for example
a logical volume of a hand-made volume group active there. One
confirmation could destroy data that another node still uses.
When the wipe dialog opens, ask every other node what uses the LUN.
Keep Wipe disabled until they answered and if another node reports a
user, naming the node and the user. List the nodes that could not be
checked, such as offline ones, without refusing the wipe. Like the
check of the local node, this sees mounts and devices built on the LUN,
not programs that open it directly, which the dialog still asks to rule
out. The check runs in the web UI; the wipe API checks the node that
wipes.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
test/SANUtils_test.js | 57 ++++++++++++++++++++++++++
www/manager6/SANUtils.js | 23 ++++++++++-
www/manager6/window/SANLunWipe.js | 66 +++++++++++++++++++++++++++++--
3 files changed, 140 insertions(+), 6 deletions(-)
diff --git a/test/SANUtils_test.js b/test/SANUtils_test.js
index f60e23b73..1ecc01904 100644
--- a/test/SANUtils_test.js
+++ b/test/SANUtils_test.js
@@ -8,6 +8,16 @@ const { test } = require('node:test');
function sanUtils(translations = {}) {
const sandbox = {
PVE: {
+ data: {
+ ResourceStore: {
+ getNodes: () => [
+ { node: 'n1', status: 'online' },
+ { node: 'n2', status: 'online' },
+ { node: 'n3', status: 'online' },
+ { node: 'n4', status: 'offline' },
+ ],
+ },
+ },
window: {
SANLunWipe: {
texts: () => ({ unchecked: 'Could not check whether this node uses the LUN.' }),
@@ -145,3 +155,50 @@ test('local users of a LUN in its usage tooltip', () => {
'not again next to the refusal to wipe that names the users',
);
});
+
+test('users of a LUN on other nodes', () => {
+ const utils = sanUtils();
+ const id = '3600a0980383235';
+ const scans = {
+ n1: [{ devpath: '/dev/mapper/x', wwid: id, multipath: 'storage', 'in-use': 'no' }],
+ n2: [
+ {
+ devpath: '/dev/sdb',
+ wwid: id,
+ multipath: 'none',
+ 'in-use': 'yes',
+ 'in-use-by': 'sdb held by vg-lv',
+ },
+ { devpath: '/dev/sdc', wwid: id, multipath: 'none', 'in-use': 'no' },
+ {
+ devpath: '/dev/nvme0n1',
+ wwn: 'uuid.1',
+ multipath: 'native',
+ 'in-use': 'yes',
+ 'in-use-by': 'nvme0n1 held by nv-lv',
+ },
+ ],
+ n3: [{ devpath: '/dev/mapper/y', wwid: id, multipath: 'storage', 'in-use': 'unknown' }],
+ };
+ let result;
+ utils.scan_nodes_for_luns.call(
+ utils,
+ (res) => (result = res),
+ (options) => {
+ const node = options.url.split('/')[2];
+ options.success({ result: { data: scans[node] } });
+ },
+ );
+ assert.equal(
+ JSON.stringify(result.users(id)),
+ JSON.stringify([{ node: 'n2', users: 'sdb held by vg-lv' }]),
+ 'a path user on another node',
+ );
+ assert.equal(
+ JSON.stringify(result.users('uuid.1')),
+ JSON.stringify([{ node: 'n2', users: 'nvme0n1 held by nv-lv' }]),
+ 'namespaces by their identity',
+ );
+ assert.equal(result.uncheckedUse(id).join(' '), 'n3', 'a node that could not check');
+ assert.equal(result.unchecked.join(' '), 'n4', 'an offline node');
+});
diff --git a/www/manager6/SANUtils.js b/www/manager6/SANUtils.js
index dbef08c84..cbe0887e8 100644
--- a/www/manager6/SANUtils.js
+++ b/www/manager6/SANUtils.js
@@ -246,14 +246,17 @@ Ext.define('PVE.SANUtils', {
return res;
},
- // Unchecked nodes (offline, failed or timed out) may still see a LUN. pathUsers misses process
- // opens, so an empty list proves nothing.
+ // Unchecked nodes (offline, failed or timed out) may still see a LUN. pathUsers and users miss
+ // process opens, so an empty list proves nothing. users(id) lists what each node reports using
+ // the LUN of that WWID or namespace identity, uncheckedUse(id) the nodes that could not tell.
scan_nodes_for_luns: function (
callback,
request = (options) => Proxmox.Utils.API2Request(options),
) {
let wwids = {};
let pathUsers = {};
+ let lunUsers = {};
+ let unknownUse = {};
let unchecked = [];
let nodes = PVE.data.ResourceStore.getNodes();
let pending = nodes.length;
@@ -268,6 +271,15 @@ Ext.define('PVE.SANUtils', {
.sort()
.filter((node) => pathUsers[node][wwid])
.map((node) => ({ node, users: pathUsers[node][wwid].join(', ') })),
+ users: (id) =>
+ Object.keys(lunUsers)
+ .sort()
+ .filter((node) => lunUsers[node][id])
+ .map((node) => ({ node, users: lunUsers[node][id].join(', ') })),
+ uncheckedUse: (id) =>
+ Object.keys(unknownUse)
+ .filter((node) => unknownUse[node][id])
+ .sort(),
unchecked: unchecked.sort(),
});
};
@@ -303,6 +315,13 @@ Ext.define('PVE.SANUtils', {
}
wwids[node.node] = new Set(rows.map((row) => row.wwid).filter((w) => w));
for (const row of rows) {
+ let id = row.wwid ?? row.wwn;
+ if (id && row['in-use'] === 'yes') {
+ let list = ((lunUsers[node.node] ??= {})[id] ??= []);
+ list.push(row['in-use-by'] || row.devpath);
+ } else if (id && row['in-use'] !== 'no') {
+ (unknownUse[node.node] ??= {})[id] = true;
+ }
let onPath = ['none', 'pending'].includes(row.multipath);
if (!row.wwid || !onPath || !['yes', 'unknown'].includes(row['in-use'])) {
continue;
diff --git a/www/manager6/window/SANLunWipe.js b/www/manager6/window/SANLunWipe.js
index ec8c8fe77..6bd2b58a9 100644
--- a/www/manager6/window/SANLunWipe.js
+++ b/www/manager6/window/SANLunWipe.js
@@ -34,8 +34,12 @@ Ext.define('PVE.window.SANLunWipe', {
'Wiping destroys that data. No configured storage was found to use this LUN for data.',
),
ownership: gettext(
- 'The node checks again for local use before wiping, but other nodes, clusters, or hosts may still use the LUN. Confirm that it is no longer needed anywhere.',
+ 'The node checks again for local use before wiping. Other clusters and hosts are not checked, nor programs on other nodes that open the LUN directly. Confirm that it is no longer needed anywhere.',
),
+ checking: gettext('Checking the other nodes for users of the LUN...'),
+ otherUsers: gettext('Another node uses the LUN. Stop that use before wiping it.'),
+ otherUnchecked: gettext('Could not check these nodes for users of the LUN: {0}'),
+ otherNone: gettext('The other nodes report no users of the LUN.'),
unknownRequest: gettext(
'The outcome of the wipe request is unknown. Check the task history of node "{0}" before you use this LUN.',
),
@@ -166,6 +170,7 @@ Ext.define('PVE.window.SANLunWipe', {
{ xtype: 'displayfield', value: text },
{ xtype: 'displayfield', value: t.consequence },
{ xtype: 'displayfield', value: t.ownership },
+ { xtype: 'displayfield', itemId: 'nodesLine', value: t.checking },
{ xtype: 'displayfield', itemId: 'errorLine', hidden: true },
{
xtype: 'textfield',
@@ -175,8 +180,7 @@ Ext.define('PVE.window.SANLunWipe', {
labelAlign: 'top',
emptyText: lun.vgname,
listeners: {
- change: (field, value) =>
- me.down('#wipeButton').setDisabled(me.submitted || value !== lun.vgname),
+ change: () => me.updateWipeButton(),
specialkey: function (field, event) {
if (
event.getKey() === event.ENTER &&
@@ -194,7 +198,7 @@ Ext.define('PVE.window.SANLunWipe', {
text: gettext('Wipe'),
itemId: 'wipeButton',
cls: 'pve-dangerous-button',
- disabled: !!lun.vgname,
+ disabled: true,
handler: () => me.startWipe(),
},
{
@@ -207,6 +211,7 @@ Ext.define('PVE.window.SANLunWipe', {
me.callParent();
me.on('beforeclose', () => !me.pending);
+ me.checkOtherNodes();
me.on('close', function () {
let op = me.operation;
if (op && me.requestState !== 'accepted') {
@@ -216,6 +221,59 @@ Ext.define('PVE.window.SANLunWipe', {
});
},
+ // The node that wipes checks itself; ask the others what they see using the LUN.
+ checkOtherNodes: function () {
+ let me = this;
+ let enc = Ext.htmlEncode;
+ let t = PVE.window.SANLunWipe.texts(me.lun);
+ let id = me.lun.wwid || me.lun.wwn;
+ PVE.SANUtils.scan_nodes_for_luns(function (result) {
+ if (me.destroyed) {
+ return;
+ }
+ let others = (node) => node !== me.node;
+ let users = id ? result.users(id).filter((entry) => others(entry.node)) : [];
+ let unchecked = [
+ ...new Set([...result.unchecked, ...(id ? result.uncheckedUse(id) : [])]),
+ ].filter(others);
+ if (!id) {
+ unchecked = PVE.data.ResourceStore.getNodes()
+ .map((node) => node.node)
+ .filter(others);
+ }
+ let lines;
+ if (users.length) {
+ lines = users.map((entry) =>
+ enc(
+ Ext.String.format(
+ gettext('In use on node "{0}": {1}'),
+ entry.node,
+ entry.users,
+ ),
+ ),
+ );
+ lines.push(enc(t.otherUsers));
+ } else if (unchecked.length) {
+ lines = [enc(Ext.String.format(t.otherUnchecked, unchecked.join(', ')))];
+ } else {
+ lines = [enc(t.otherNone)];
+ }
+ me.down('#nodesLine').setValue(lines.join('<br>'));
+ me.otherUsers = users.length > 0;
+ me.nodesChecked = true;
+ me.updateWipeButton();
+ });
+ },
+
+ updateWipeButton: function () {
+ let me = this;
+ let vgname = me.lun.vgname;
+ let typed = !vgname || me.down('#confirmField').getValue() === vgname;
+ me.down('#wipeButton').setDisabled(
+ me.submitted || !me.nodesChecked || me.otherUsers || !typed,
+ );
+ },
+
startWipe: function () {
let me = this;
if (me.submitted || me.down('#wipeButton').isDisabled()) {
--
2.47.3
^ permalink raw reply related [flat|nested] 77+ messages in thread
end of thread, other threads:[~2026-11-06 1:27 UTC | newest]
Thread overview: 77+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-05 23:30 [PATCH v3 proxmox,proxmox-perl-rs,cluster,storage,docs,manager 00/76] SAN: cluster-wide multipath, SAN views and a guided storage setup Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 proxmox 01/76] add proxmox-multipath crate Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 proxmox-perl-rs 02/76] pve-rs: add PVE::RS::Multipath bindings for multipathd status parsing Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 cluster 03/76] cfs: observe the multipath configuration files Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 04/76] storage: distinguish status-only activation in plugins Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 05/76] lvm: find a shared volume group created by another node Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 06/76] storage: bound LVM status commands Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 07/76] iscsi: rate-limit session rescans per target Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 08/76] iscsi: list the LUNs of each session, not of its SCSI host Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 09/76] lvm: refuse to create a volume group on a device that holds data Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 10/76] lvm: refuse a base volume of a target that serves disk images Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 11/76] iscsi: refuse virtual devices other than multipath maps as volumes Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 12/76] diskmanage: collect disk transport type from lsblk Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 13/76] diskmanage: add helper to list multipath devices Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 14/76] diskmanage: qualify NVMe over fabrics transport Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 15/76] disks: list: add include-remote parameter Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 16/76] diskmanage: include iSCSI session devices in disk enumeration Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 17/76] diskmanage: link multipath member disks to their map device Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 18/76] iscsi: factor out session device map from device list Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 19/76] api: scan: add a SAN LUN inventory Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 20/76] pvesm: expose the " Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 21/76] disks: lvm: allow volume groups on SAN LUNs and multipath maps Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 22/76] multipath: add a library around multipathd Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 23/76] diskmanage: collect native NVMe multipath path state Thomas Lamprecht
2026-11-05 23:30 ` [PATCH v3 storage 24/76] api: scan: san-luns: report " Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 25/76] diskmanage: collect local block device users Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 26/76] api: add node-level iSCSI status and maintenance Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 27/76] api: iscsi: add session logout with last-path protection Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 28/76] multipath: add a cluster-wide configuration Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 29/76] iscsi: add an option to manage storage LUNs with multipath Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 30/76] lvm: activate a missing base volume before using the volume group Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 31/76] multipath: apply the configuration on every node Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 32/76] iscsi: share the initiator identity reader Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 33/76] multipath: publish per-node status Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 34/76] api: add endpoints for the cluster-wide multipath configuration Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 35/76] api: multipath: add the cluster-wide health status Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 36/76] multipath: collect identities of non-iSCSI SCSI paths Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 37/76] api: scan: san-luns: report the multipath state of each LUN Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 38/76] api: scan: flag duplicate physical volumes on distinct LUNs Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 39/76] api: disks: recheck local users and retain wipe device references Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 40/76] iscsi: name plain LUNs after their identity link Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 41/76] api: disks: guard SAN wipes with stable identities and storage checks Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 storage 42/76] api: scan: refresh SAN usage with bounded probes Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 docs 43/76] storage: lvm: note the rescan after extending a shared volume group Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 docs 44/76] storage: document the cluster-wide multipath management Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 45/76] api: ceph: osd: keep iSCSI LUNs out of OSD creation Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 46/76] pvestatd: track pending storage status commands across re-exec Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 47/76] ui: storage: factor out the scan picker error mask Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 48/76] ui: storage: nfs: show scan errors and empty results Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 49/76] ui: storage: allow changing the node for NFS/CIFS scans Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 50/76] ui: storage: add guided remote storage wizard with NFS support Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 51/76] ui: storage wizard: add SMB/CIFS support Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 52/76] ui: storage wizard: preserve settings after partial setup Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 53/76] ui: storage wizard: add Fibre Channel shared LVM setup Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 54/76] ui: storage wizard: reconcile unanswered storage creation Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 55/76] ui: storage wizard: add ZFS over iSCSI support Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 56/76] ui: dc: storage: add remote storage wizard entry to the add menu Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 57/76] ui: node: add SAN LUNs panel Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 58/76] ui: san luns: show multipath path state Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 59/76] api: nodes: expose iSCSI status and maintenance Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 60/76] pvenode: add iscsi commands Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 61/76] ui: node: san: add iSCSI target and session management Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 62/76] pvestatd: apply the multipath configuration and report the node status Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 63/76] api: cluster: add the multipath endpoints Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 64/76] ui: dc: add a cluster-wide multipath view Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 65/76] ui: storage: iscsi: add the multipath option Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 66/76] ui: san luns: show multipath state and manage LUN entries Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 67/76] ui: storage wizard: add iSCSI support Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 68/76] ui: storage wizard: iscsi: show cluster initiator names Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 69/76] ui: storage wizard: iscsi: enable multipath for shared LVM Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 70/76] ui: node: san: wipe a LUN that no storage uses Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 71/76] ui: storage wizard: iscsi: reuse existing target connections Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 72/76] ui: storage wizard: fibre channel: enable multipath for a LUN Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 73/76] ui: storage wizard: let root@pam wipe a LUN with data Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 74/76] ui: storage wizard: do not offer LUNs in use for a new volume group Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 75/76] ui: san: show the local users of a LUN in its usage tooltip Thomas Lamprecht
2026-11-05 23:31 ` [PATCH v3 manager 76/76] ui: san: check the other nodes for users before wiping a LUN Thomas Lamprecht
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.