all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH v5 storage 03/36] diskmanage: qualify NVMe over fabrics transport
Date: Mon, 16 Nov 2026 02:23:12 +0100	[thread overview]
Message-ID: <20261116012355.3187417-4-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20261116012355.3187417-1-t.lamprecht@proxmox.com>

From: Dietmar Maurer <dietmar@proxmox.com>

lsblk reports 'nvme' for namespaces attached over a fabric too, so
they cannot be told from local PCIe drives. Read the transport from
the controller, or from a controller of the subsystem with native
multipath, and report nvme-fc, nvme-tcp, nvme-rdma or nvme-loop for
such namespaces.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
[TL: add the test case with the two namespaces only, as the multipath
 devices it was part of follow later; mention the new values in the
 transport description]
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 src/PVE/API2/Disks.pm                         |  3 +-
 src/PVE/Diskmanage.pm                         | 27 ++++++++++++++
 src/test/disk_tests/multipath/disklist        |  2 ++
 .../multipath/disklist_expected.json          | 35 +++++++++++++++++++
 src/test/disk_tests/multipath/lsblk           | 18 ++++++++++
 .../disk_tests/multipath/nvme8n1/device/model |  1 +
 .../multipath/nvme8n1/device/transport        |  1 +
 .../multipath/nvme8n1/queue/rotational        |  1 +
 src/test/disk_tests/multipath/nvme8n1/size    |  1 +
 src/test/disk_tests/multipath/nvme8n1_udevadm | 14 ++++++++
 .../disk_tests/multipath/nvme9n1/device/model |  1 +
 .../multipath/nvme9n1/device/transport        |  1 +
 .../multipath/nvme9n1/device/vendor           |  1 +
 .../multipath/nvme9n1/queue/rotational        |  1 +
 src/test/disk_tests/multipath/nvme9n1/size    |  1 +
 src/test/disk_tests/multipath/nvme9n1_udevadm | 12 +++++++
 src/test/disklist_test.pm                     | 12 +++++++
 17 files changed, 131 insertions(+), 1 deletion(-)
 create mode 100644 src/test/disk_tests/multipath/disklist
 create mode 100644 src/test/disk_tests/multipath/disklist_expected.json
 create mode 100644 src/test/disk_tests/multipath/lsblk
 create mode 100644 src/test/disk_tests/multipath/nvme8n1/device/model
 create mode 100644 src/test/disk_tests/multipath/nvme8n1/device/transport
 create mode 100644 src/test/disk_tests/multipath/nvme8n1/queue/rotational
 create mode 100644 src/test/disk_tests/multipath/nvme8n1/size
 create mode 100644 src/test/disk_tests/multipath/nvme8n1_udevadm
 create mode 100644 src/test/disk_tests/multipath/nvme9n1/device/model
 create mode 100644 src/test/disk_tests/multipath/nvme9n1/device/transport
 create mode 100644 src/test/disk_tests/multipath/nvme9n1/device/vendor
 create mode 100644 src/test/disk_tests/multipath/nvme9n1/queue/rotational
 create mode 100644 src/test/disk_tests/multipath/nvme9n1/size
 create mode 100644 src/test/disk_tests/multipath/nvme9n1_udevadm

diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index 1abab820..980ba7ff 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -138,7 +138,8 @@ __PACKAGE__->register_method({
                 transport => {
                     type => 'string',
                     description =>
-                        "The device transport, for example 'sata', 'sas', 'usb', 'fc', or 'nvme'.",
+                        "The device transport, for example 'sata', 'sas', 'usb', 'fc',"
+                        . " 'nvme', 'nvme-fc', or 'nvme-tcp'.",
                     optional => 1,
                 },
                 parent => {
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index 4d66902b..7862bcd4 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -505,6 +505,26 @@ sub is_iscsi {
     return 0;
 }
 
+# Returns the NVMe transport (pcie, fc, tcp, rdma, loop) of a namespace, either directly from
+# its controller, or from any controller of its subsystem for native NVMe multipath setups.
+sub get_nvme_transport {
+    my ($sysdir) = @_;
+
+    my $transport = file_read_firstline("$sysdir/device/transport");
+    return $transport if defined($transport);
+
+    dir_glob_foreach(
+        "$sysdir/device",
+        'nvme\d+',
+        sub {
+            my ($controller) = @_;
+            $transport //= file_read_firstline("$sysdir/device/$controller/transport");
+        },
+    );
+
+    return $transport;
+}
+
 my sub is_ssdlike {
     my ($type) = @_;
     return $type eq 'ssd' || $type eq 'nvme';
@@ -597,6 +617,9 @@ sub get_disks {
             # we do not want iscsi devices
             return if is_iscsi($sysdir);
 
+            my $nvme_transport;
+            $nvme_transport = get_nvme_transport($sysdir) if $dev =~ m/^nvme/;
+
             my $sysdata = get_sysdir_info($sysdir);
             return if !defined($sysdata);
 
@@ -648,7 +671,11 @@ sub get_disks {
                 wearout => $wearout,
             };
 
+            # lsblk reports plain 'nvme' for fabrics attached namespaces
             my $transport = $lsblk_info->{$devpath}->{tran};
+            if (defined($nvme_transport) && $nvme_transport ne 'pcie') {
+                $transport = "nvme-$nvme_transport";
+            }
             $disklist->{$dev}->{transport} = $transport if defined($transport);
             $disklist->{$dev}->{mounted} = 1 if exists $mounted->{$devpath};
 
diff --git a/src/test/disk_tests/multipath/disklist b/src/test/disk_tests/multipath/disklist
new file mode 100644
index 00000000..b5e636a5
--- /dev/null
+++ b/src/test/disk_tests/multipath/disklist
@@ -0,0 +1,2 @@
+nvme8n1
+nvme9n1
diff --git a/src/test/disk_tests/multipath/disklist_expected.json b/src/test/disk_tests/multipath/disklist_expected.json
new file mode 100644
index 00000000..908c2b56
--- /dev/null
+++ b/src/test/disk_tests/multipath/disklist_expected.json
@@ -0,0 +1,35 @@
+{
+    "nvme8n1": {
+        "devpath": "/dev/nvme8n1",
+        "size": 10737418240,
+        "vendor": "unknown",
+        "model": "FCMODEL",
+        "serial": "FCNVME1",
+        "wwn": "uuid.20000000-0000-0000-0000-000000000001",
+        "gpt": 0,
+        "rpm": 0,
+        "type": "nvme",
+        "health": "UNKNOWN",
+        "wearout": "N/A",
+        "osdid": -1,
+        "osdid-list": null,
+        "by_id_link": "/dev/disk/by-id/nvme-FCMODEL_FCNVME1",
+        "transport": "nvme-fc"
+    },
+    "nvme9n1": {
+        "devpath": "/dev/nvme9n1",
+        "size": 10737418240,
+        "vendor": "FABVEND",
+        "model": "FABRICS CTRL",
+        "serial": "FAB1",
+        "wwn": "uuid.10000000-0000-0000-0000-000000000001",
+        "gpt": 0,
+        "rpm": 0,
+        "type": "nvme",
+        "health": "UNKNOWN",
+        "wearout": "N/A",
+        "osdid": -1,
+        "osdid-list": null,
+        "transport": "nvme-tcp"
+    }
+}
diff --git a/src/test/disk_tests/multipath/lsblk b/src/test/disk_tests/multipath/lsblk
new file mode 100644
index 00000000..597adf94
--- /dev/null
+++ b/src/test/disk_tests/multipath/lsblk
@@ -0,0 +1,18 @@
+{
+   "blockdevices": [
+      {
+         "path": "/dev/nvme8n1",
+         "parttype": null,
+         "fstype": null,
+         "tran": "nvme",
+         "pttype": null
+      },
+      {
+         "path": "/dev/nvme9n1",
+         "parttype": null,
+         "fstype": null,
+         "tran": "nvme",
+         "pttype": null
+      }
+   ]
+}
diff --git a/src/test/disk_tests/multipath/nvme8n1/device/model b/src/test/disk_tests/multipath/nvme8n1/device/model
new file mode 100644
index 00000000..1aac689e
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1/device/model
@@ -0,0 +1 @@
+FCMODEL
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme8n1/device/transport b/src/test/disk_tests/multipath/nvme8n1/device/transport
new file mode 100644
index 00000000..8b517d01
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1/device/transport
@@ -0,0 +1 @@
+fc
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme8n1/queue/rotational b/src/test/disk_tests/multipath/nvme8n1/queue/rotational
new file mode 100644
index 00000000..c2270834
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1/queue/rotational
@@ -0,0 +1 @@
+0
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme8n1/size b/src/test/disk_tests/multipath/nvme8n1/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme8n1_udevadm b/src/test/disk_tests/multipath/nvme8n1_udevadm
new file mode 100644
index 00000000..d0cd5ea9
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme8n1_udevadm
@@ -0,0 +1,14 @@
+P: /devices/virtual/nvme-subsystem/nvme-subsys8/nvme8n1
+N: nvme8n1
+S: disk/by-id/nvme-FCMODEL_FCNVME1
+E: DEVLINKS=/dev/disk/by-id/nvme-FCMODEL_FCNVME1
+E: DEVNAME=/dev/nvme8n1
+E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys8/nvme8n1
+E: DEVTYPE=disk
+E: ID_MODEL=FCMODEL
+E: ID_SERIAL=FCNVME_1
+E: ID_SERIAL_SHORT=FCNVME1
+E: ID_WWN=uuid.20000000-0000-0000-0000-000000000001
+E: MAJOR=259
+E: MINOR=1
+E: SUBSYSTEM=block
diff --git a/src/test/disk_tests/multipath/nvme9n1/device/model b/src/test/disk_tests/multipath/nvme9n1/device/model
new file mode 100644
index 00000000..aa0287c7
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/device/model
@@ -0,0 +1 @@
+FABMODEL
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1/device/transport b/src/test/disk_tests/multipath/nvme9n1/device/transport
new file mode 100644
index 00000000..f5483dbf
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/device/transport
@@ -0,0 +1 @@
+tcp
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1/device/vendor b/src/test/disk_tests/multipath/nvme9n1/device/vendor
new file mode 100644
index 00000000..cdc6ddfa
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/device/vendor
@@ -0,0 +1 @@
+FABVEND
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1/queue/rotational b/src/test/disk_tests/multipath/nvme9n1/queue/rotational
new file mode 100644
index 00000000..c2270834
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/queue/rotational
@@ -0,0 +1 @@
+0
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1/size b/src/test/disk_tests/multipath/nvme9n1/size
new file mode 100644
index 00000000..8280959b
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1/size
@@ -0,0 +1 @@
+20971520
\ No newline at end of file
diff --git a/src/test/disk_tests/multipath/nvme9n1_udevadm b/src/test/disk_tests/multipath/nvme9n1_udevadm
new file mode 100644
index 00000000..b6d13ccb
--- /dev/null
+++ b/src/test/disk_tests/multipath/nvme9n1_udevadm
@@ -0,0 +1,12 @@
+P: /devices/virtual/nvme-subsystem/nvme-subsys9/nvme9n1
+N: nvme9n1
+E: DEVNAME=/dev/nvme9n1
+E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys9/nvme9n1
+E: DEVTYPE=disk
+E: ID_MODEL=FABRICS CTRL
+E: ID_SERIAL=FABRICS_1
+E: ID_SERIAL_SHORT=FAB1
+E: ID_WWN=uuid.10000000-0000-0000-0000-000000000001
+E: MAJOR=259
+E: MINOR=0
+E: SUBSYSTEM=block
diff --git a/src/test/disklist_test.pm b/src/test/disklist_test.pm
index a0dc01f6..51b20c53 100644
--- a/src/test/disklist_test.pm
+++ b/src/test/disklist_test.pm
@@ -101,6 +101,16 @@ sub mocked_is_iscsi {
     return 0;
 }
 
+sub mocked_file_read_firstline {
+    my ($path) = @_;
+
+    my $originalsub = $diskmanage_module->original('file_read_firstline');
+
+    $path =~ s|^/sys/block|disk_tests/$testcasedir|;
+
+    return &$originalsub($path);
+}
+
 sub mocked_dir_glob_foreach {
     my ($dir, $regex, $sub) = @_;
 
@@ -248,6 +258,8 @@ $diskmanage_module->mock(
     },
 );
 print("\tMocked dir_is_empty\n");
+$diskmanage_module->mock('file_read_firstline' => \&mocked_file_read_firstline);
+print("\tMocked file_read_firstline\n");
 $diskmanage_module->mock('check_bin' => sub { return 1; });
 print("\tMocked check_bin\n");
 my $tools_module = Test::MockModule->new('PVE::ProcFSTools', no_auto => 1);




  parent reply	other threads:[~2026-11-16  1:24 UTC|newest]

Thread overview: 38+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  1:23 [PATCH v5 cluster,storage,docs,manager 00/36] SAN: cluster-wide multipath, SAN views and a guided setup for shared LVM Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 cluster 01/36] cfs: observe the multipath configuration files Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 02/36] diskmanage: collect disk transport type from lsblk Thomas Lamprecht
2026-11-16  1:23 ` Thomas Lamprecht [this message]
2026-11-16  1:23 ` [PATCH v5 storage 04/36] diskmanage: optionally include iSCSI session devices in disk enumeration Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 05/36] diskmanage: collect native NVMe multipath path state Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 06/36] iscsi: factor out session device map from device list Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 07/36] multipath: add a library around multipathd Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 08/36] diskmanage: add helper to list multipath devices Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 09/36] multipath: add a cluster-wide configuration Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 10/36] iscsi: add an option to manage storage LUNs with multipath Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 11/36] lvm: activate a missing base volume before using the volume group Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 12/36] multipath: apply the configuration on every node Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 13/36] multipath: publish per-node status Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 14/36] api: add endpoints for the cluster-wide multipath configuration Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 15/36] api: multipath: add the cluster-wide health status Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 16/36] api: scan: add a SAN LUN inventory Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 17/36] pvesm: expose the " Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 18/36] api: add node-level iSCSI status Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 19/36] disks: lvm: drop unused imports Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 20/36] disks: lvm: allow volume groups on SAN LUNs and multipath maps Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 21/36] disks: lvm: refuse to destroy the volume group of a shared storage Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 storage 22/36] disks: allow wiping a SAN LUN that no storage uses Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 docs 23/36] storage: document the cluster-wide multipath management Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 24/36] pvestatd: apply the multipath configuration and report the node status Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 25/36] report: add the managed multipath configuration Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 26/36] api: cluster: add the multipath endpoints Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 27/36] api: nodes: expose the iSCSI status Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 28/36] ui: dc: add a SAN view Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 29/36] ui: storage: iscsi: add the multipath option Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 30/36] ui: lvm: enable shared when a base storage is selected Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 31/36] ui: node: add SAN LUNs panel Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 32/36] ui: node: san: wipe a LUN that no storage uses Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 33/36] ui: node: san: show iSCSI targets and sessions Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 34/36] ui: dc: san: add a guided setup for shared LVM on iSCSI Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 35/36] ui: dc: san: add a guided setup for shared LVM on Fibre Channel Thomas Lamprecht
2026-11-16  1:23 ` [PATCH v5 manager 36/36] ui: storage: point the iSCSI and LVM add dialogs to the SAN view Thomas Lamprecht
2026-11-18  8:11 ` superseded: [PATCH v5 cluster,storage,docs,manager 00/36] SAN: cluster-wide multipath, SAN views and a guided setup for shared LVM Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116012355.3187417-4-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal