From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 5FDEA1FF0ED for ; Fri, 31 Jul 2026 12:23:12 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id C8D132166C; Fri, 31 Jul 2026 12:22:05 +0200 (CEST) From: Dietmar Maurer To: pve-devel@lists.proxmox.com Subject: [RFC pve-storage 03/27] diskmanage: qualify NVMe over fabrics transport Date: Fri, 31 Jul 2026 12:21:32 +0200 Message-ID: <20260731102156.3947857-4-dietmar@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260731102156.3947857-1-dietmar@proxmox.com> References: <20260731102156.3947857-1-dietmar@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 1 AWL -0.733 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RDNS_NONE 1.274 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Message-ID-Hash: NUQOJZHEY2QCCHMMKJYTUITUVL22XZXG X-Message-ID-Hash: NUQOJZHEY2QCCHMMKJYTUITUVL22XZXG X-MailFrom: dietmar@zilli.proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: lsblk reports plain 'nvme' for fabrics attached namespaces, so clients cannot tell local PCIe drives from shared SAN namespaces. Read the transport from the namespace controller, or from any controller of the subsystem when the namespace belongs to a subsystem due to native NVMe multipath, and report nvme-fc, nvme-tcp, nvme-rdma or nvme-loop instead. This lets clients like the SAN LUN scan or the local storage creation dialogs decide which namespaces are suitable, instead of hiding fabrics attached devices server side. Signed-off-by: Dietmar Maurer --- src/PVE/Diskmanage.pm | 27 +++++++++++++++ src/test/disk_tests/multipath/disklist | 2 ++ .../multipath/disklist_expected.json | 33 +++++++++++++++++++ src/test/disk_tests/multipath/lsblk | 14 ++++++++ .../disk_tests/multipath/nvme8n1/device/model | 1 + .../multipath/nvme8n1/device/transport | 1 + .../multipath/nvme8n1/queue/rotational | 1 + src/test/disk_tests/multipath/nvme8n1/size | 1 + src/test/disk_tests/multipath/nvme8n1_udevadm | 14 ++++++++ .../disk_tests/multipath/nvme9n1/device/model | 1 + .../multipath/nvme9n1/device/transport | 1 + .../multipath/nvme9n1/device/vendor | 1 + .../multipath/nvme9n1/queue/rotational | 1 + src/test/disk_tests/multipath/nvme9n1/size | 1 + src/test/disk_tests/multipath/nvme9n1_udevadm | 12 +++++++ 15 files changed, 111 insertions(+) create mode 100644 src/test/disk_tests/multipath/nvme8n1/device/model create mode 100644 src/test/disk_tests/multipath/nvme8n1/device/transport create mode 100644 src/test/disk_tests/multipath/nvme8n1/queue/rotational create mode 100644 src/test/disk_tests/multipath/nvme8n1/size create mode 100644 src/test/disk_tests/multipath/nvme8n1_udevadm create mode 100644 src/test/disk_tests/multipath/nvme9n1/device/model create mode 100644 src/test/disk_tests/multipath/nvme9n1/device/transport create mode 100644 src/test/disk_tests/multipath/nvme9n1/device/vendor create mode 100644 src/test/disk_tests/multipath/nvme9n1/queue/rotational create mode 100644 src/test/disk_tests/multipath/nvme9n1/size create mode 100644 src/test/disk_tests/multipath/nvme9n1_udevadm diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm index ed5181f..b006ec3 100644 --- a/src/PVE/Diskmanage.pm +++ b/src/PVE/Diskmanage.pm @@ -485,6 +485,26 @@ sub is_iscsi { return 0; } +# Returns the NVMe transport (pcie, fc, tcp, rdma, loop) of a namespace, either directly from +# its controller, or from any controller of its subsystem for native NVMe multipath setups. +sub get_nvme_transport { + my ($sysdir) = @_; + + my $transport = file_read_firstline("$sysdir/device/transport"); + return $transport if defined($transport); + + dir_glob_foreach( + "$sysdir/device", + 'nvme\d+', + sub { + my ($controller) = @_; + $transport //= file_read_firstline("$sysdir/device/$controller/transport"); + }, + ); + + return $transport; +} + my sub is_ssdlike { my ($type) = @_; return $type eq 'ssd' || $type eq 'nvme'; @@ -577,6 +597,9 @@ sub get_disks { # we do not want iscsi devices return if is_iscsi($sysdir); + my $nvme_transport; + $nvme_transport = get_nvme_transport($sysdir) if $dev =~ m/^nvme/; + my $sysdata = get_sysdir_info($sysdir); return if !defined($sysdata); @@ -628,7 +651,11 @@ sub get_disks { wearout => $wearout, }; + # lsblk reports plain 'nvme' for fabrics attached namespaces my $transport = $lsblk_info->{$devpath}->{tran}; + if (defined($nvme_transport) && $nvme_transport ne 'pcie') { + $transport = "nvme-$nvme_transport"; + } $disklist->{$dev}->{transport} = $transport if defined($transport); $disklist->{$dev}->{mounted} = 1 if exists $mounted->{$devpath}; diff --git a/src/test/disk_tests/multipath/disklist b/src/test/disk_tests/multipath/disklist index 9293589..f759ba9 100644 --- a/src/test/disk_tests/multipath/disklist +++ b/src/test/disk_tests/multipath/disklist @@ -7,3 +7,5 @@ dm-1 dm-2 dm-3 dm-4 +nvme8n1 +nvme9n1 diff --git a/src/test/disk_tests/multipath/disklist_expected.json b/src/test/disk_tests/multipath/disklist_expected.json index a8e74fa..ec44296 100644 --- a/src/test/disk_tests/multipath/disklist_expected.json +++ b/src/test/disk_tests/multipath/disklist_expected.json @@ -70,5 +70,38 @@ "by_id_link": "/dev/disk/by-id/scsi-36001405000000000000000000000000d", "transport": "fc", "used": "Device Mapper" + }, + "nvme8n1": { + "devpath": "/dev/nvme8n1", + "size": 10737418240, + "vendor": "unknown", + "model": "FCMODEL", + "serial": "FCNVME1", + "wwn": "uuid.20000000-0000-0000-0000-000000000001", + "gpt": 0, + "rpm": 0, + "type": "nvme", + "health": "UNKNOWN", + "wearout": "N/A", + "osdid": -1, + "osdid-list": null, + "by_id_link": "/dev/disk/by-id/nvme-FCMODEL_FCNVME1", + "transport": "nvme-fc" + }, + "nvme9n1": { + "devpath": "/dev/nvme9n1", + "size": 10737418240, + "vendor": "FABVEND", + "model": "FABRICS CTRL", + "serial": "FAB1", + "wwn": "uuid.10000000-0000-0000-0000-000000000001", + "gpt": 0, + "rpm": 0, + "type": "nvme", + "health": "UNKNOWN", + "wearout": "N/A", + "osdid": -1, + "osdid-list": null, + "transport": "nvme-tcp" } } diff --git a/src/test/disk_tests/multipath/lsblk b/src/test/disk_tests/multipath/lsblk index 91f7457..dfc1ae1 100644 --- a/src/test/disk_tests/multipath/lsblk +++ b/src/test/disk_tests/multipath/lsblk @@ -58,6 +58,20 @@ "fstype": null, "tran": null, "pttype": null + }, + { + "path": "/dev/nvme8n1", + "parttype": null, + "fstype": null, + "tran": "nvme", + "pttype": null + }, + { + "path": "/dev/nvme9n1", + "parttype": null, + "fstype": null, + "tran": "nvme", + "pttype": null } ] } diff --git a/src/test/disk_tests/multipath/nvme8n1/device/model b/src/test/disk_tests/multipath/nvme8n1/device/model new file mode 100644 index 0000000..1aac689 --- /dev/null +++ b/src/test/disk_tests/multipath/nvme8n1/device/model @@ -0,0 +1 @@ +FCMODEL \ No newline at end of file diff --git a/src/test/disk_tests/multipath/nvme8n1/device/transport b/src/test/disk_tests/multipath/nvme8n1/device/transport new file mode 100644 index 0000000..8b517d0 --- /dev/null +++ b/src/test/disk_tests/multipath/nvme8n1/device/transport @@ -0,0 +1 @@ +fc \ No newline at end of file diff --git a/src/test/disk_tests/multipath/nvme8n1/queue/rotational b/src/test/disk_tests/multipath/nvme8n1/queue/rotational new file mode 100644 index 0000000..c227083 --- /dev/null +++ b/src/test/disk_tests/multipath/nvme8n1/queue/rotational @@ -0,0 +1 @@ +0 \ No newline at end of file diff --git a/src/test/disk_tests/multipath/nvme8n1/size b/src/test/disk_tests/multipath/nvme8n1/size new file mode 100644 index 0000000..8280959 --- /dev/null +++ b/src/test/disk_tests/multipath/nvme8n1/size @@ -0,0 +1 @@ +20971520 \ No newline at end of file diff --git a/src/test/disk_tests/multipath/nvme8n1_udevadm b/src/test/disk_tests/multipath/nvme8n1_udevadm new file mode 100644 index 0000000..d0cd5ea --- /dev/null +++ b/src/test/disk_tests/multipath/nvme8n1_udevadm @@ -0,0 +1,14 @@ +P: /devices/virtual/nvme-subsystem/nvme-subsys8/nvme8n1 +N: nvme8n1 +S: disk/by-id/nvme-FCMODEL_FCNVME1 +E: DEVLINKS=/dev/disk/by-id/nvme-FCMODEL_FCNVME1 +E: DEVNAME=/dev/nvme8n1 +E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys8/nvme8n1 +E: DEVTYPE=disk +E: ID_MODEL=FCMODEL +E: ID_SERIAL=FCNVME_1 +E: ID_SERIAL_SHORT=FCNVME1 +E: ID_WWN=uuid.20000000-0000-0000-0000-000000000001 +E: MAJOR=259 +E: MINOR=1 +E: SUBSYSTEM=block diff --git a/src/test/disk_tests/multipath/nvme9n1/device/model b/src/test/disk_tests/multipath/nvme9n1/device/model new file mode 100644 index 0000000..aa0287c --- /dev/null +++ b/src/test/disk_tests/multipath/nvme9n1/device/model @@ -0,0 +1 @@ +FABMODEL \ No newline at end of file diff --git a/src/test/disk_tests/multipath/nvme9n1/device/transport b/src/test/disk_tests/multipath/nvme9n1/device/transport new file mode 100644 index 0000000..f5483db --- /dev/null +++ b/src/test/disk_tests/multipath/nvme9n1/device/transport @@ -0,0 +1 @@ +tcp \ No newline at end of file diff --git a/src/test/disk_tests/multipath/nvme9n1/device/vendor b/src/test/disk_tests/multipath/nvme9n1/device/vendor new file mode 100644 index 0000000..cdc6ddf --- /dev/null +++ b/src/test/disk_tests/multipath/nvme9n1/device/vendor @@ -0,0 +1 @@ +FABVEND \ No newline at end of file diff --git a/src/test/disk_tests/multipath/nvme9n1/queue/rotational b/src/test/disk_tests/multipath/nvme9n1/queue/rotational new file mode 100644 index 0000000..c227083 --- /dev/null +++ b/src/test/disk_tests/multipath/nvme9n1/queue/rotational @@ -0,0 +1 @@ +0 \ No newline at end of file diff --git a/src/test/disk_tests/multipath/nvme9n1/size b/src/test/disk_tests/multipath/nvme9n1/size new file mode 100644 index 0000000..8280959 --- /dev/null +++ b/src/test/disk_tests/multipath/nvme9n1/size @@ -0,0 +1 @@ +20971520 \ No newline at end of file diff --git a/src/test/disk_tests/multipath/nvme9n1_udevadm b/src/test/disk_tests/multipath/nvme9n1_udevadm new file mode 100644 index 0000000..b6d13cc --- /dev/null +++ b/src/test/disk_tests/multipath/nvme9n1_udevadm @@ -0,0 +1,12 @@ +P: /devices/virtual/nvme-subsystem/nvme-subsys9/nvme9n1 +N: nvme9n1 +E: DEVNAME=/dev/nvme9n1 +E: DEVPATH=/devices/virtual/nvme-subsystem/nvme-subsys9/nvme9n1 +E: DEVTYPE=disk +E: ID_MODEL=FABRICS CTRL +E: ID_SERIAL=FABRICS_1 +E: ID_SERIAL_SHORT=FAB1 +E: ID_WWN=uuid.10000000-0000-0000-0000-000000000001 +E: MAJOR=259 +E: MINOR=0 +E: SUBSYSTEM=block -- 2.47.3