all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: Re: [PATCH v8 cluster,storage,docs,manager 00/43] SAN: cluster-wide multipath, SAN views and a guided setup for shared LVM
Date: Fri, 20 Nov 2026 09:45:20 +0100	[thread overview]
Message-ID: <179516247284.2632472.8451368514027474084.b4-ty@b4> (raw)
In-Reply-To: <20261119194655.3163258-1-t.lamprecht@proxmox.com>

On Thu, 19 Nov 2026 20:46:05 +0100, Thomas Lamprecht wrote:
> This is v8 of the combined and reduced series of the cluster-wide
> multipath work [0][1] and Dietmar's RFC for guided remote storage setup
> and SAN visibility [2].
>
> If nothing grave comes up I plan to apply this as is.
>
> v8 is v7 [9] plus what an independent review of v7 turned up and held
> up when checked against the code and the lab, i.e.:
>
> [...]

Applied, big thanks here to Blockbridge [0], which where very reactive
and tested basically every version here, ensuring a much better end
result!

[0]: https://www.blockbridge.com

Compared to v8 as posted, I folded in fixes for quite a few more findings,
but mostly rather targeted edge cases:
- If the multipath map of a LUN was not set up on a node, for example
  because multipathd failed there, LVM on that node hid the PV on the
  LUN, so a wipe from there could destroy the volume group of a shared
  storage. The checks before a wipe or a volume group creation now see
  such PVs, and also read the LVM label directly.
- The wipe and the volume group creation checked the LUN through its
  name, which can lead to another device by the time they write. Both
  now check the device they hold open.
- Creating a volume group through the disk API on a LUN that already
  had one of the same name, for example from another node, just
  succeeded, so two guided setups on one LUN could end with two storages
  on the same volume group. This is refused now; only adding a storage
  with the LUN as base volume takes over an existing volume group.
- A wipe of a multipath map without any working path could hang in its
  checks until the cluster lock timed out. It is now refused right away,
  as the volume group creation already did.
- An entry with the name of a map instead of a WWID was accepted by
  'multipath -a' without an error, but never showed up as added, so the
  node reloaded multipathd on every pass. Such an entry is now reported
  as refused.
- The status report that each node publishes only had room for about
  180 LUNs. It is stored compressed now, which raises that to about 700.
- The LUN list, the checks and the status each worked out on their own
  which LUN an LVM storage uses as base, and did not always agree: a
  base with a partition suffix, for example, did not protect its LUN
  from a wipe. They now use one helper.
- The cluster status still expected the LUN of a disabled LVM storage
  on its nodes, so the LUN showed as missing everywhere once its iSCSI
  sessions were gone. A disabled storage now expects nothing.
- A hand-edited overrides file skipped the syntax check that the API
  does, so it could hold a multipaths section that the API refuses.
  Such a file now stops the apply with an error, instead of being
  installed.
- A node without iscsiadm, but in the scope of an iSCSI storage with
  multipath management, could neither apply its Fibre Channel entries
  nor show Node > SAN. It now only asks iscsiadm about targets that
  have paths on that node.
- Adding a second entry for the same WWID now shows the error at the
  WWID field, instead of as a generic error.
- If the base volume of an LVM storage could not be activated, a status
  pass repeated slow LVM scans for it. It now stops at the first
  failure.
- In the web UI, a LUN that is unknown on all nodes no longer lists
  every node in its state, the Target/Subsystem header is one string
  for translators, and the iSCSI setup says where to allow the
  initiator names.
- The multipath chapter is shorter and simpler, and also covers the
  options of a multipath entry and that nothing notifies about a lost
  path. The web UI and pvestatd chapters mention the SAN panels and
  what pvestatd does for multipath.
- Plus some rewording of comments, test descriptions and commit
  messages.

Still open (just for sake of completness):
- Nothing notifies about a lost path yet, so an admin has to look at
  the SAN view. A notification could build on the status that every
  node already reports.
- Storage plugins cannot tell how many paths a volume should have, so
  a namespace that a plugin connected looks single-path once one of its
  paths is gone. A plugin method for the expected paths could add that.
- NVMe over Fabrics support follows as v3 of its own series, on top of
  this.

[pve-cluster]
[01/43] cfs: observe the multipath configuration files
        commit: f196907143173a904a24a42f7c2a9f7f78c183c4

[pve-storage]
[02/43] diskmanage: collect disk transport type from lsblk
        commit: d5167f815b6344718506e4a5449096ed9518bf4c
[03/43] diskmanage: qualify NVMe over Fabrics transport
        commit: 968f1549f8acc3874d253de8a476664438a18d16
[04/43] diskmanage: optionally include iSCSI session devices in disk enumeration
        commit: 08c04c8d7a077ff03f4a6fbdab466037ed6e85dd
[05/43] diskmanage: collect native NVMe multipath path state
        commit: 6e52aaf06509dc8314030793e1b76f785ee7367a
[06/43] iscsi: factor out session device map from device list
        commit: e9e65e672d19cc5d1f33c13a5175cb1eee90fd34
[07/43] multipath: add a library around multipathd
        commit: dfe34094562cfc5de060ff835b6e75debc2be12a
[08/43] diskmanage: add helper to list multipath devices
        commit: d3e5f28dae60dac121a46a42d63527e5db6f03b5
[09/43] multipath: add a cluster-wide configuration
        commit: 59948403ba368b3b771f4860f359c62d1347bd3e
[10/43] iscsi: add an option to manage storage LUNs with multipath
        commit: 55dd1de9a1d8017b23ffe53c2815ce5fcfc5a217
[11/43] lvm: activate a missing base volume before using the volume group
        commit: 44bc0e3e4e4b261fc4cc76e47dc233f0b4768694
[12/43] multipath: apply the configuration on every node
        commit: 51b8a5c820b9ba68d511022350eab4e9cc614937
[13/43] multipath: publish per-node status
        commit: e72b1be2793c36e6991d87fe7707ba5288d05509
[14/43] api: add endpoints for the cluster-wide multipath configuration
        commit: f0d87e448c207bf166fd000bb880c35c551bfba5
[15/43] api: multipath: add the cluster-wide health status
        commit: c98ecf288ff4423d5064e51d19e28734b99a6040
[16/43] diskmanage: look up the requested disks by name
        commit: 8222e318de382ea22f554998dd876b67e8c08ee0
[17/43] diskmanage: read the udev data of all disks with one call
        commit: 298dcfaec978ae611e6c54d155bd1cdd80e6d7fb
[18/43] api: scan: add a SAN LUN inventory
        commit: 2e5f0afad07ffd1845c7d0940e9662df48c72836
[19/43] pvesm: expose the SAN LUN inventory
        commit: 73091d3621e51ad5fccb316d2efd902c0570212d
[20/43] api: add node-level iSCSI status
        commit: cdf8763f784d2eddadfacaa576d39b64b0c8dea4
[21/43] api: scan: name the portal of an iSCSI path
        commit: ad2ccb2ead41c6f4675d9a8d9e382dd59863d7f6
[22/43] disks: lvm: drop unused imports
        commit: 946c8422ed805512515864fe80b6aaae340315ec
[23/43] disks: lvm: allow volume groups on SAN LUNs and multipath maps
        commit: 6a98c4ba61fb133a6dc82bf20931417f5c0c5ed8
[24/43] disks: lvm: refuse to destroy the volume group of a shared storage
        commit: 679d241b7bb4c016704b2fd6ed7983a4983efd98
[25/43] disks: allow wiping a SAN LUN that no storage uses
        commit: c5529caa6377f03654d121573659e1cfc0235074
[26/43] api: scan: report the process that has a SAN LUN open
        commit: 8e9935f595a664290e6edd4ae8c23b28fe631018

[pve-docs]
[27/43] storage: document the cluster-wide multipath management
        commit: 5704d152297524600925469f2065e66d21d49300
[28/43] storage: multipath: describe the use shown for an open LUN
        commit: 64861a0de0a5e086f3f5557a6e9165cf75ff8996

[pve-manager]
[29/43] pvestatd: apply the multipath configuration and report the node status
        commit: ec55d57a3288630c0690dea4e8d63280e70d58c3
[30/43] report: add the managed multipath configuration
        commit: a87eab65140ac4ae098ee4304830d3b9815565a1
[31/43] api: cluster: add the multipath endpoints
        commit: 0a6f615a5594135103c3f6aaedeaeefb08664114
[32/43] api: nodes: expose the iSCSI status
        commit: faaaadcf6f29c7cd7b1d938d88f0957ceba4d5f8
[33/43] ui: dc: add a SAN view
        commit: a335adb38a9b2b9d0a05a1e3afc720bf9a0897c1
[34/43] ui: storage: iscsi: add the multipath option
        commit: ca0bfb7e4b079e328f60037ba784ec2354c836b4
[35/43] ui: lvm: enable shared when a base storage is selected
        commit: 6cfb3b8ac945ae4b7b0b07ac122da7c5d5f20584
[36/43] ui: node: add SAN LUNs panel
        commit: d079fb4b3bd3126df29684822083922e2469afde
[37/43] ui: node: san: wipe a LUN that no storage uses
        commit: 966f798adacd34d5b726ae9e6265a8a02a7ec5c5
[38/43] ui: node: san: show iSCSI targets and sessions
        commit: 56482f5dd485d37a06f33329216c375e135d5b33
[39/43] ui: dc: san: add a guided setup for shared LVM on iSCSI
        commit: e2094f57fff7b8058060b0fb7d0410c2ab3ec744
[40/43] ui: dc: san: add a guided setup for shared LVM on Fibre Channel
        commit: 39162712816a5fc715d00a07d86f683b9a005bf9
[41/43] ui: storage: point the iSCSI and LVM add dialogs to the SAN view
        commit: ab30edfcf78d6b6017ebb82354a20402dab22d3b
[42/43] ui: san: show the guest or process that has a LUN open
        commit: 188f844baad0ce05b701f0f1a9018ff05f207a25
[43/43] ui: node: san: add a column for the target or subsystem of a LUN
        commit: 215871e61ea8ab696d5a4400707d2ea18fb46fa2




      parent reply	other threads:[~2026-11-20  8:48 UTC|newest]

Thread overview: 45+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 19:46 [PATCH v8 cluster,storage,docs,manager 00/43] SAN: cluster-wide multipath, SAN views and a guided setup for shared LVM Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 cluster 01/43] cfs: observe the multipath configuration files Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 02/43] diskmanage: collect disk transport type from lsblk Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 03/43] diskmanage: qualify NVMe over Fabrics transport Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 04/43] diskmanage: optionally include iSCSI session devices in disk enumeration Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 05/43] diskmanage: collect native NVMe multipath path state Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 06/43] iscsi: factor out session device map from device list Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 07/43] multipath: add a library around multipathd Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 08/43] diskmanage: add helper to list multipath devices Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 09/43] multipath: add a cluster-wide configuration Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 10/43] iscsi: add an option to manage storage LUNs with multipath Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 11/43] lvm: activate a missing base volume before using the volume group Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 12/43] multipath: apply the configuration on every node Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 13/43] multipath: publish per-node status Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 14/43] api: add endpoints for the cluster-wide multipath configuration Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 15/43] api: multipath: add the cluster-wide health status Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 16/43] diskmanage: look up the requested disks by name Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 17/43] diskmanage: read the udev data of all disks with one call Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 18/43] api: scan: add a SAN LUN inventory Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 19/43] pvesm: expose the " Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 20/43] api: add node-level iSCSI status Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 21/43] api: scan: name the portal of an iSCSI path Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 22/43] disks: lvm: drop unused imports Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 23/43] disks: lvm: allow volume groups on SAN LUNs and multipath maps Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 24/43] disks: lvm: refuse to destroy the volume group of a shared storage Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 25/43] disks: allow wiping a SAN LUN that no storage uses Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 storage 26/43] api: scan: report the process that has a SAN LUN open Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 docs 27/43] storage: document the cluster-wide multipath management Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 docs 28/43] storage: multipath: describe the use shown for an open LUN Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 29/43] pvestatd: apply the multipath configuration and report the node status Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 30/43] report: add the managed multipath configuration Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 31/43] api: cluster: add the multipath endpoints Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 32/43] api: nodes: expose the iSCSI status Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 33/43] ui: dc: add a SAN view Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 34/43] ui: storage: iscsi: add the multipath option Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 35/43] ui: lvm: enable shared when a base storage is selected Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 36/43] ui: node: add SAN LUNs panel Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 37/43] ui: node: san: wipe a LUN that no storage uses Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 38/43] ui: node: san: show iSCSI targets and sessions Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 39/43] ui: dc: san: add a guided setup for shared LVM on iSCSI Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 40/43] ui: dc: san: add a guided setup for shared LVM on Fibre Channel Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 41/43] ui: storage: point the iSCSI and LVM add dialogs to the SAN view Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 42/43] ui: san: show the guest or process that has a LUN open Thomas Lamprecht
2026-11-19 19:46 ` [PATCH v8 manager 43/43] ui: node: san: add a column for the target or subsystem of a LUN Thomas Lamprecht
2026-11-20  8:45 ` Thomas Lamprecht [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=179516247284.2632472.8451368514027474084.b4-ty@b4 \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal