* [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk
@ 2023-08-22 9:04 Aaron Lauterer
2023-08-22 9:04 ` [pve-devel] [PATCH storage 1/2] disks: get: add osdid-list return parameter Aaron Lauterer
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Aaron Lauterer @ 2023-08-22 9:04 UTC (permalink / raw)
To: pve-devel
It is possible to have multiple OSD daemons on a single disk. This is
useful if fast NVME drives are used to utilize their full potential.
For these situations we want to list all OSD daemons that are located on
the disk in the disk panel of the node.
I added a new 'osdid-list' parameter to the Disks GET API endpoint for
this.
We can probably deprecate/remove the 'osdid' the next time we can
introduce breaking API changes.
storage: Aaron Lauterer (1):
disks: get: add osdid-list return parameter
src/PVE/API2/Disks.pm | 6 +++-
src/PVE/Diskmanage.pm | 10 ++++++
.../disk_tests/cciss/disklist_expected.json | 1 +
.../hdd_smart/disklist_expected.json | 2 ++
.../nvme_smart/disklist_expected.json | 1 +
.../disk_tests/sas/disklist_expected.json | 1 +
.../disk_tests/sas_ssd/disklist_expected.json | 1 +
.../ssd_smart/disklist_expected.json | 5 +++
.../disk_tests/usages/disklist_expected.json | 32 +++++++++++++------
9 files changed, 49 insertions(+), 10 deletions(-)
widget-toolkit: Aaron Lauterer (1):
DiskList: render osdid-list if present
src/panel/DiskList.js | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
--
2.39.2
^ permalink raw reply [flat|nested] 5+ messages in thread
* [pve-devel] [PATCH storage 1/2] disks: get: add osdid-list return parameter
2023-08-22 9:04 [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk Aaron Lauterer
@ 2023-08-22 9:04 ` Aaron Lauterer
2023-08-22 9:04 ` [pve-devel] [PATCH widget-toolkit 2/2] DiskList: render osdid-list if present Aaron Lauterer
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Aaron Lauterer @ 2023-08-22 9:04 UTC (permalink / raw)
To: pve-devel
It is possible to run multiple OSD daemons on one disk. The new
'osdid-list' parameter returns an array of all OSD IDs found on the
disk.
The old 'osdid' parameter is kept for compatibility. We might want to
deprecate / remove it in the future.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/PVE/API2/Disks.pm | 6 +++-
src/PVE/Diskmanage.pm | 10 ++++++
.../disk_tests/cciss/disklist_expected.json | 1 +
.../hdd_smart/disklist_expected.json | 2 ++
.../nvme_smart/disklist_expected.json | 1 +
.../disk_tests/sas/disklist_expected.json | 1 +
.../disk_tests/sas_ssd/disklist_expected.json | 1 +
.../ssd_smart/disklist_expected.json | 5 +++
.../disk_tests/usages/disklist_expected.json | 32 +++++++++++++------
9 files changed, 49 insertions(+), 10 deletions(-)
diff --git a/src/PVE/API2/Disks.pm b/src/PVE/API2/Disks.pm
index f0e3fc4..408bdbe 100644
--- a/src/PVE/API2/Disks.pm
+++ b/src/PVE/API2/Disks.pm
@@ -124,7 +124,11 @@ __PACKAGE__->register_method ({
gpt => { type => 'boolean' },
mounted => { type => 'boolean' },
size => { type => 'integer'},
- osdid => { type => 'integer'},
+ osdid => { type => 'integer'}, # TODO: deprecate / remove in PVE 9?
+ 'osdid-list' => {
+ type => 'array',
+ items => { type => 'integer' },
+ },
vendor => { type => 'string', optional => 1 },
model => { type => 'string', optional => 1 },
serial => { type => 'string', optional => 1 },
diff --git a/src/PVE/Diskmanage.pm b/src/PVE/Diskmanage.pm
index a311ffd..0217c75 100644
--- a/src/PVE/Diskmanage.pm
+++ b/src/PVE/Diskmanage.pm
@@ -286,6 +286,10 @@ sub get_ceph_volume_infos {
# $result autovivification is wanted, to not creating empty hashes
if (($type eq 'block' || $type eq 'data') && $fields->[2] =~ m/ceph.osd_id=([^,]+)/) {
$result->{$dev}->{osdid} = $1;
+ if ( !defined($result->{$dev}->{'osdid-list'}) ) {
+ $result->{$dev}->{'osdid-list'} = [];
+ }
+ push($result->{$dev}->{'osdid-list'}->@*, $1);
$result->{$dev}->{bluestore} = ($type eq 'block');
if ($fields->[2] =~ m/ceph\.encrypted=1/) {
$result->{$dev}->{encrypted} = 1;
@@ -584,6 +588,7 @@ sub get_disks {
$disklist->{$dev}->{by_id_link} = $by_id_link if defined($by_id_link);
my ($osdid, $bluestore, $osdencrypted) = (-1, 0, 0);
+ my $osdid_list;
my ($journal_count, $db_count, $wal_count) = (0, 0, 0);
my $partpath = $devpath;
@@ -624,6 +629,7 @@ sub get_disks {
$wal_count += $ceph_volume->{wal} // 0;
if (defined($ceph_volume->{osdid})) {
$osdid = $ceph_volume->{osdid};
+ $osdid_list = $ceph_volume->{'osdid-list'};
$bluestore = 1 if $ceph_volume->{bluestore};
$osdencrypted = 1 if $ceph_volume->{encrypted};
}
@@ -648,6 +654,7 @@ sub get_disks {
$partitions->{$part}->{size} = get_sysdir_size("$sysdir/$part") // 0;
$partitions->{$part}->{used} = $determine_usage->("$partpath/$part", "$sysdir/$part", 1);
$partitions->{$part}->{osdid} //= -1;
+ $partitions->{$part}->{'osdid-list'} //= undef;
# avoid counting twice (e.g. partition with the LVM for the DB OSD is in $journalhash)
return if $lvm_based_osd;
@@ -657,6 +664,8 @@ sub get_disks {
if ($mp =~ m|^/var/lib/ceph/osd/ceph-(\d+)$|) {
$osdid = $1;
$partitions->{$part}->{osdid} = $osdid;
+ $osdid_list = [$1]; # assuming only one OSD per disk
+ $partitions->{$part}->{'osdid-list'} = $osdid_list;
}
}
@@ -693,6 +702,7 @@ sub get_disks {
$collect_ceph_info->($devpath);
$disklist->{$dev}->{osdid} = $osdid;
+ $disklist->{$dev}->{'osdid-list'} = $osdid_list;
$disklist->{$dev}->{journals} = $journal_count if $journal_count;
$disklist->{$dev}->{bluestore} = $bluestore if $osdid != -1;
$disklist->{$dev}->{osdencrypted} = $osdencrypted if $osdid != -1;
diff --git a/src/test/disk_tests/cciss/disklist_expected.json b/src/test/disk_tests/cciss/disklist_expected.json
index eff58db..df6606d 100644
--- a/src/test/disk_tests/cciss/disklist_expected.json
+++ b/src/test/disk_tests/cciss/disklist_expected.json
@@ -6,6 +6,7 @@
"type" : "unknown",
"serial" : "SER111",
"osdid" : -1,
+ "osdid-list" : null,
"health" : "UNKNOWN",
"model" : "LOGICAL_VOLUME",
"size" : 5120,
diff --git a/src/test/disk_tests/hdd_smart/disklist_expected.json b/src/test/disk_tests/hdd_smart/disklist_expected.json
index 02a341e..6c827c8 100644
--- a/src/test/disk_tests/hdd_smart/disklist_expected.json
+++ b/src/test/disk_tests/hdd_smart/disklist_expected.json
@@ -4,6 +4,7 @@
"size" : 1024000,
"gpt" : 1,
"osdid" : -1,
+ "osdid-list" : null,
"rpm" : 7200,
"model" : "ST4000NM0033-9ZM170",
"vendor" : "ATA",
@@ -15,6 +16,7 @@
},
"sda" : {
"osdid" : -1,
+ "osdid-list" : null,
"size" : 1024000,
"gpt" : 1,
"devpath" : "/dev/sda",
diff --git a/src/test/disk_tests/nvme_smart/disklist_expected.json b/src/test/disk_tests/nvme_smart/disklist_expected.json
index 4d1c92f..025d8eb 100644
--- a/src/test/disk_tests/nvme_smart/disklist_expected.json
+++ b/src/test/disk_tests/nvme_smart/disklist_expected.json
@@ -8,6 +8,7 @@
"model" : "NVME MODEL 1",
"rpm" : 0,
"osdid" : -1,
+ "osdid-list" : null,
"devpath" : "/dev/nvme0n1",
"gpt" : 0,
"wwn" : "unknown",
diff --git a/src/test/disk_tests/sas/disklist_expected.json b/src/test/disk_tests/sas/disklist_expected.json
index 39b14a5..a4a1c4c 100644
--- a/src/test/disk_tests/sas/disklist_expected.json
+++ b/src/test/disk_tests/sas/disklist_expected.json
@@ -6,6 +6,7 @@
"model" : "MODEL1",
"health" : "UNKNOWN",
"osdid" : -1,
+ "osdid-list" : null,
"wwn" : "0x0000000000000000",
"vendor" : "VENDOR1",
"rpm" : -1,
diff --git a/src/test/disk_tests/sas_ssd/disklist_expected.json b/src/test/disk_tests/sas_ssd/disklist_expected.json
index dd9b748..84f14fd 100644
--- a/src/test/disk_tests/sas_ssd/disklist_expected.json
+++ b/src/test/disk_tests/sas_ssd/disklist_expected.json
@@ -6,6 +6,7 @@
"model" : "MODEL1",
"health" : "OK",
"osdid" : -1,
+ "osdid-list" : null,
"wwn" : "0x0000000000000000",
"vendor" : "VENDOR1",
"rpm" : 0,
diff --git a/src/test/disk_tests/ssd_smart/disklist_expected.json b/src/test/disk_tests/ssd_smart/disklist_expected.json
index d84b9dc..64d2eac 100644
--- a/src/test/disk_tests/ssd_smart/disklist_expected.json
+++ b/src/test/disk_tests/ssd_smart/disklist_expected.json
@@ -7,6 +7,7 @@
"health" : "PASSED",
"wearout" : "100",
"osdid" : -1,
+ "osdid-list" : null,
"size" : 512000,
"type" : "ssd",
"devpath" : "/dev/sda",
@@ -17,6 +18,7 @@
"model" : "INTEL_SSDSC2BB080G6",
"devpath" : "/dev/sdb",
"osdid" : -1,
+ "osdid-list" : null,
"type" : "ssd",
"size" : 512000,
"wwn" : "0x0000000000000000",
@@ -32,6 +34,7 @@
"devpath" : "/dev/sdc",
"model" : "Samsung SSD 850 PRO 512GB",
"osdid" : -1,
+ "osdid-list" : null,
"type" : "ssd",
"size" : 512000,
"wearout" : "99",
@@ -52,12 +55,14 @@
"model" : "SanDisk SD8SB8U1T001122",
"size" : 512000,
"osdid" : -1,
+ "osdid-list" : null,
"type" : "ssd",
"wwn" : "0x0000000000000000"
},
"sde" : {
"type" : "ssd",
"osdid" : -1,
+ "osdid-list" : null,
"size" : 512000,
"model" : "KINGSTON SHFS37A120G",
"devpath" : "/dev/sde",
diff --git a/src/test/disk_tests/usages/disklist_expected.json b/src/test/disk_tests/usages/disklist_expected.json
index e9c5e5d..6020841 100644
--- a/src/test/disk_tests/usages/disklist_expected.json
+++ b/src/test/disk_tests/usages/disklist_expected.json
@@ -6,6 +6,7 @@
"type" : "hdd",
"osdencrypted": 0,
"osdid" : "444",
+ "osdid-list" : ["444"],
"bluestore" : "0",
"health" : "UNKNOWN",
"model" : "MODEL1",
@@ -26,6 +27,7 @@
"size" : 1536000,
"gpt" : 1,
"osdid" : -1,
+ "osdid-list" : null,
"type" : "hdd",
"model" : "MODEL1",
"used" : "Device Mapper",
@@ -44,7 +46,8 @@
"size" : 1536000,
"rpm" : 0,
"type" : "hdd",
- "osdid" : -1
+ "osdid" : -1,
+ "osdid-list" : null
},
"sda" : {
"model" : "MODEL1",
@@ -57,6 +60,7 @@
"rpm" : 0,
"type" : "hdd",
"osdid" : -1,
+ "osdid-list" : null,
"serial" : "SERIAL1",
"vendor" : "ATA",
"wwn" : "0x0000000000000000",
@@ -73,6 +77,7 @@
"health" : "UNKNOWN",
"type" : "hdd",
"osdid" : -1,
+ "osdid-list" : null,
"gpt" : 1,
"rpm" : 0,
"size" : 1536000
@@ -86,6 +91,7 @@
"gpt" : 1,
"type" : "hdd",
"osdid" : -1,
+ "osdid-list" : null,
"serial" : "SERIAL1",
"used": "ZFS",
"vendor" : "ATA",
@@ -107,7 +113,8 @@
"type" : "hdd",
"bluestore": 1,
"osdencrypted": 0,
- "osdid" : 1
+ "osdid" : 1,
+ "osdid-list" : [1]
},
"sdh" : {
"serial" : "SERIAL1",
@@ -123,7 +130,8 @@
"size" : 1536000,
"rpm" : 0,
"type" : "hdd",
- "osdid" : -1
+ "osdid" : -1,
+ "osdid-list" : null
},
"sdi" : {
"serial" : "SERIAL1",
@@ -139,7 +147,8 @@
"rpm" : 0,
"type" : "hdd",
"db": 1,
- "osdid" : -1
+ "osdid" : -1,
+ "osdid-list" : null
},
"sdj" : {
"serial" : "SERIAL1",
@@ -156,7 +165,8 @@
"bluestore": 0,
"type" : "hdd",
"osdencrypted": 1,
- "osdid" : 0
+ "osdid" : 0,
+ "osdid-list" : [0]
},
"sdk" : {
"serial" : "SERIAL1",
@@ -173,7 +183,8 @@
"bluestore": 0,
"type" : "hdd",
"osdencrypted": 0,
- "osdid" : 230
+ "osdid" : 230,
+ "osdid-list": [230]
},
"sdl" : {
"serial" : "SERIAL1",
@@ -188,7 +199,8 @@
"size" : 1536000,
"rpm" : 0,
"type" : "hdd",
- "osdid" : -1
+ "osdid" : -1,
+ "osdid-list" : null
},
"sdm" : {
"serial" : "SERIAL1",
@@ -203,7 +215,8 @@
"size" : 1536000,
"rpm" : 0,
"type" : "hdd",
- "osdid" : -1
+ "osdid" : -1,
+ "osdid-list" : null
},
"sdn" : {
"serial" : "SERIAL1",
@@ -218,6 +231,7 @@
"size" : 1536000,
"rpm" : 0,
"type" : "hdd",
- "osdid" : -1
+ "osdid" : -1,
+ "osdid-list" : null
}
}
--
2.39.2
^ permalink raw reply [flat|nested] 5+ messages in thread
* [pve-devel] [PATCH widget-toolkit 2/2] DiskList: render osdid-list if present
2023-08-22 9:04 [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk Aaron Lauterer
2023-08-22 9:04 ` [pve-devel] [PATCH storage 1/2] disks: get: add osdid-list return parameter Aaron Lauterer
@ 2023-08-22 9:04 ` Aaron Lauterer
2023-09-28 13:21 ` [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk Aaron Lauterer
2023-11-13 16:01 ` [pve-devel] applied.series: " Thomas Lamprecht
3 siblings, 0 replies; 5+ messages in thread
From: Aaron Lauterer @ 2023-08-22 9:04 UTC (permalink / raw)
To: pve-devel
Render all OSD IDs in 'osdid-list' if the parameter is present.
It is possible to have multiple OSD daemons on a disk. We want to list
them all in the UI.
Fall back to the 'osdid' parameter if 'osdid-list' is not available.
We check rec.data['osdid-list'] against its general truthiness as it
might not be present at all or null.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/panel/DiskList.js | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/src/panel/DiskList.js b/src/panel/DiskList.js
index c5a0050..4808f5c 100644
--- a/src/panel/DiskList.js
+++ b/src/panel/DiskList.js
@@ -220,7 +220,11 @@ Ext.define('Proxmox.DiskList', {
let extendedInfo = '';
if (rec) {
let types = [];
- if (rec.data.osdid !== undefined && rec.data.osdid >= 0) {
+ if (rec.data['osdid-list'] && rec.data['osdid-list'].length > 0) {
+ for (const id of rec.data['osdid-list'].sort()) {
+ types.push(`OSD.${id.toString()}`);
+ }
+ } else if (rec.data.osdid !== undefined && rec.data.osdid >= 0) {
types.push(`OSD.${rec.data.osdid.toString()}`);
}
if (rec.data.journals > 0) {
--
2.39.2
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk
2023-08-22 9:04 [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk Aaron Lauterer
2023-08-22 9:04 ` [pve-devel] [PATCH storage 1/2] disks: get: add osdid-list return parameter Aaron Lauterer
2023-08-22 9:04 ` [pve-devel] [PATCH widget-toolkit 2/2] DiskList: render osdid-list if present Aaron Lauterer
@ 2023-09-28 13:21 ` Aaron Lauterer
2023-11-13 16:01 ` [pve-devel] applied.series: " Thomas Lamprecht
3 siblings, 0 replies; 5+ messages in thread
From: Aaron Lauterer @ 2023-09-28 13:21 UTC (permalink / raw)
To: pve-devel
ping? patches still apply cleanly
On 8/22/23 11:04, Aaron Lauterer wrote:
> It is possible to have multiple OSD daemons on a single disk. This is
> useful if fast NVME drives are used to utilize their full potential.
>
> For these situations we want to list all OSD daemons that are located on
> the disk in the disk panel of the node.
>
> I added a new 'osdid-list' parameter to the Disks GET API endpoint for
> this.
> We can probably deprecate/remove the 'osdid' the next time we can
> introduce breaking API changes.
>
> storage: Aaron Lauterer (1):
> disks: get: add osdid-list return parameter
>
> src/PVE/API2/Disks.pm | 6 +++-
> src/PVE/Diskmanage.pm | 10 ++++++
> .../disk_tests/cciss/disklist_expected.json | 1 +
> .../hdd_smart/disklist_expected.json | 2 ++
> .../nvme_smart/disklist_expected.json | 1 +
> .../disk_tests/sas/disklist_expected.json | 1 +
> .../disk_tests/sas_ssd/disklist_expected.json | 1 +
> .../ssd_smart/disklist_expected.json | 5 +++
> .../disk_tests/usages/disklist_expected.json | 32 +++++++++++++------
> 9 files changed, 49 insertions(+), 10 deletions(-)
>
>
> widget-toolkit: Aaron Lauterer (1):
> DiskList: render osdid-list if present
>
> src/panel/DiskList.js | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
^ permalink raw reply [flat|nested] 5+ messages in thread
* [pve-devel] applied.series: [PATCH storage/widget-toolkit 0/2] List all OSDs on disk
2023-08-22 9:04 [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk Aaron Lauterer
` (2 preceding siblings ...)
2023-09-28 13:21 ` [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk Aaron Lauterer
@ 2023-11-13 16:01 ` Thomas Lamprecht
3 siblings, 0 replies; 5+ messages in thread
From: Thomas Lamprecht @ 2023-11-13 16:01 UTC (permalink / raw)
To: Proxmox VE development discussion, Aaron Lauterer
Am 22/08/2023 um 11:04 schrieb Aaron Lauterer:
> It is possible to have multiple OSD daemons on a single disk. This is
> useful if fast NVME drives are used to utilize their full potential.
>
> For these situations we want to list all OSD daemons that are located on
> the disk in the disk panel of the node.
>
> I added a new 'osdid-list' parameter to the Disks GET API endpoint for
> this.
> We can probably deprecate/remove the 'osdid' the next time we can
> introduce breaking API changes.
>
> storage: Aaron Lauterer (1):
> disks: get: add osdid-list return parameter
>
> src/PVE/API2/Disks.pm | 6 +++-
> src/PVE/Diskmanage.pm | 10 ++++++
> .../disk_tests/cciss/disklist_expected.json | 1 +
> .../hdd_smart/disklist_expected.json | 2 ++
> .../nvme_smart/disklist_expected.json | 1 +
> .../disk_tests/sas/disklist_expected.json | 1 +
> .../disk_tests/sas_ssd/disklist_expected.json | 1 +
> .../ssd_smart/disklist_expected.json | 5 +++
> .../disk_tests/usages/disklist_expected.json | 32 +++++++++++++------
> 9 files changed, 49 insertions(+), 10 deletions(-)
>
>
> widget-toolkit: Aaron Lauterer (1):
> DiskList: render osdid-list if present
>
> src/panel/DiskList.js | 6 +++++-
> 1 file changed, 5 insertions(+), 1 deletion(-)
>
applied series, thanks!
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2023-11-13 16:17 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-08-22 9:04 [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk Aaron Lauterer
2023-08-22 9:04 ` [pve-devel] [PATCH storage 1/2] disks: get: add osdid-list return parameter Aaron Lauterer
2023-08-22 9:04 ` [pve-devel] [PATCH widget-toolkit 2/2] DiskList: render osdid-list if present Aaron Lauterer
2023-09-28 13:21 ` [pve-devel] [PATCH storage/widget-toolkit 0/2] List all OSDs on disk Aaron Lauterer
2023-11-13 16:01 ` [pve-devel] applied.series: " Thomas Lamprecht
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox