[pve-devel] [PATCH v3 pve-manager 0/5] Fix #5366: Ceph Build Commit in UI

[pve-devel] [PATCH v3 pve-manager 0/5] Fix #5366: Ceph Build Commit in UI

[Max Carrara]

Ceph Build Commit in UI - Version 3
===================================

Notable Changes Since v2
------------------------

  * Rebase on master branch as v2 was partially applied (thanks!)
  * Factor duplicate build commit rendering code into separate helper
    as suggested [1] (thanks Thomas!)
  * Make variable names more clear / less ambiguous (thanks Thomas!)
  * Increase default width of version column in OSD tree view ever so
    slightlymore
  * Reword commit titles and messages to reflect the changes made

For a detailed list of changes please see the comments in the individual
patches.

NOTE: Any T-b and R-b trailers on patches that received changes are
considered outdated and are thus removed.

Older Versions
--------------

v1: https://lists.proxmox.com/pipermail/pve-devel/2024-April/063772.html
v2: https://lists.proxmox.com/pipermail/pve-devel/2024-July/064349.html

References
----------

[1]: https://lists.proxmox.com/pipermail/pve-devel/2024-July/064789.html

Summary of Changes
------------------

Max Carrara (5):
  fix #5366: ui: ceph: services: parse and display build commit
  fix #5366: api: ceph: add host build commit to Ceph OSD index data
  fix #5366: ui: ceph: osd: rework version field rendering
  ui: ceph: osd: increase width of version column
  fix #5366: api: ceph: change version format in OSD metadata endpoint

 PVE/API2/Ceph/OSD.pm             |   9 ++-
 www/manager6/Utils.js            | 107 +++++++++++++++++++++++++++++++
 www/manager6/ceph/OSD.js         |  54 ++++++++++++----
 www/manager6/ceph/ServiceList.js |  33 +++++++---
 www/manager6/ceph/Services.js    |  19 +++++-
 5 files changed, 198 insertions(+), 24 deletions(-)

-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v3 pve-manager 1/5] fix #5366: ui: ceph: services: parse and display build commit

[Max Carrara]

The build commit is displayed and taken into account when comparing
monitor and manager versions in the client. Specifically, the
shortened build commit is now displayed in parentheses next to the
version for both monitors and managers like so:

  18.2.2 (build: abcd1234)

Should the build commit of the running service differ from the one
that's installed on the host, the newer build commit will also be
shown in parentheses:

  18.2.2 (build: abcd1234 -> 5678fedc)

The icon displayed for running a service with an outdated build is the
same as for running an outdated version. The conditional display of
icons related to cluster health remains the same otherwise.

The Ceph summary view also displays the hash and will show a warning
if a service is running with a build commit that doesn't match the one
that's advertised by the host.

This requires the introduction of two helper functions:

1. `PVE.Utils.parseCephBuildCommit`, which can be used to get the full
  hash "eccf199d..." in parentheses from a string like the following:

  ceph version 17.2.7 (eccf199d63457659c09677399928203b7903c888) quincy (stable)

2. `PVE.Utils.renderCephBuildCommit`, which is used to determine how
   to render a service's current build commit and which accompanying
   icon to choose.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Fixes: https://bugzilla.proxmox.com/show_bug.cgi?id=5366
---
Changes v2 --> v3:
  * add new `renderCephBuildCommit` helper function (thanks Thomas!)
  * add docstrings for helpers
  * use less ambiguous variable names (thanks Thomas!)
  * put 'build: ' in front of build commit when rendering (thanks Thomas!)
  * handle no rendered build commit being available in MGR / MON lists,
    returning the icon + version without the commit instead
  * make the modified logic in Services.js more readable
  * reword message about differing builds in the overview
  * reword commit title & message
  * add 'Fixes' trailer
  * remove outdated R-b and T-b trailers

Changes v1 --> v2:
  * use camelCase instead of snake_case (thanks Lukas!)
  * use more descriptive variable names (thanks Lukas!)
  * use `let` instead of `const` for variables where applicable (thanks Lukas!)

 www/manager6/Utils.js            | 107 +++++++++++++++++++++++++++++++
 www/manager6/ceph/ServiceList.js |  33 +++++++---
 www/manager6/ceph/Services.js    |  19 +++++-
 3 files changed, 148 insertions(+), 11 deletions(-)

diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index db86fa9a..1d42be34 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -128,6 +128,113 @@ Ext.define('PVE.Utils', {
 	return undefined;
     },
 
+    /**
+     * Parses a Ceph build commit from its version string.
+     *
+     * @param {object} service
+     * @param {string} service.ceph_version - The long form of a Ceph version string
+     * @returns {(undefined|string)} The matched build commit or `undefined`
+     */
+    parseCephBuildCommit: function(service) {
+	if (service.ceph_version) {
+	    // See PVE/Ceph/Tools.pm - get_local_version
+	    const match = service.ceph_version.match(
+		/^ceph.*\sv?(?:\d+(?:\.\d+)+)\s+(?:\(([a-zA-Z0-9]+)\))/,
+	    );
+	    if (match) {
+		return match[1];
+	    }
+	}
+
+	return undefined;
+    },
+
+    /**
+     * Determines how Ceph build commits should be rendered and which icon
+     * should be chosen depending on the Ceph versions and build commits given.
+     *
+     * Should the two build commits be identical, the commit is shortened.
+     * Should they differ, they're rendered as e.g.:
+     *
+     *	    0558a7146 -> b72f4fd04
+     *
+     *	Note that the arrow `->` is rendered as a Font Awesome icon.
+     *
+     * @param {Object} versionInfo
+     * @param {string} versionInfo.runningVersion - The version of the currently
+     *	    running Ceph service
+     * @param {string} versionInfo.installedVersion - The version of the Ceph
+     *	    service (or its executable) as reported by the node
+     * @param {string} versionInfo.runningBuildCommit - The build commit of the
+     *	    currently running Ceph service
+     * @param {string} versionInfo.installedBuildCommit - The build commit of the
+     *	    Ceph service (or its executable) as reported by the node
+     * @param {string} versionInfo.maxVersion - The highest version of the
+     *	    Ceph service in the cluster
+     * @param {boolean} versionInfo.hasMixedVersions - Whether different
+     *	    versions of the service are currently running in the cluster
+     * @param {number} [buildCommitLength] - (Optional) The length to which to
+     *	    shorten the rendered build commit(s), `9` by default
+     * @returns {Object} - An object containing the selected `icon`, whether the
+     *	    current build commit has `changed` and the build commit (or build
+     *	    commit changes) as `renderedBuildCommit`
+     */
+    renderCephBuildCommit: function(versionInfo, buildCommitLength) {
+	buildCommitLength ??= 9;
+
+	versionInfo = versionInfo ?? {};
+
+	let runningVersion = versionInfo.runningVersion ?? '';
+	let installedVersion = versionInfo.installedVersion ?? '';
+
+	let runningBuildCommit = versionInfo.runningBuildCommit ?? '';
+	let installedBuildCommit = versionInfo.installedBuildCommit ?? '';
+
+	let maxVersion = versionInfo.maxVersion ?? '';
+	let hasMixedVersions = versionInfo.hasMixedVersions ?? false;
+
+	let hasBuildCommits = runningBuildCommit !== '' && installedBuildCommit !== '';
+	let hasRunningBuildCommitChanged = hasBuildCommits &&
+	    runningBuildCommit !== installedBuildCommit;
+
+	let icon = '';
+
+	if (PVE.Utils.compare_ceph_versions(maxVersion, installedVersion) > 0) {
+	    icon = PVE.Utils.get_ceph_icon_html('HEALTH_UPGRADE');
+	} else if (PVE.Utils.compare_ceph_versions(installedVersion, runningVersion) > 0) {
+	    icon = PVE.Utils.get_ceph_icon_html('HEALTH_OLD');
+	} else if (hasMixedVersions && !hasRunningBuildCommitChanged) {
+	    icon = PVE.Utils.get_ceph_icon_html('HEALTH_OK');
+	}
+
+	if (!hasBuildCommits) {
+	    return {
+		icon: icon,
+		changed: hasRunningBuildCommitChanged,
+		renderedBuildCommit: (runningBuildCommit || installedBuildCommit)
+		    .substring(0, buildCommitLength),
+	    };
+	}
+
+	let renderedBuildCommit = runningBuildCommit.substring(0, buildCommitLength);
+
+	if (hasRunningBuildCommitChanged) {
+	    const arrow = '<i class="fa fa-fw fa-long-arrow-right"></i>';
+	    icon ||= PVE.Utils.get_ceph_icon_html('HEALTH_OLD');
+
+	    let buildCommitFrom = runningBuildCommit.substring(0, buildCommitLength);
+	    let buildCommitTo = installedBuildCommit.substring(0, buildCommitLength);
+
+	    renderedBuildCommit = `${buildCommitFrom}${arrow}${buildCommitTo}`;
+	}
+
+	return {
+	    icon: icon,
+	    changed: hasRunningBuildCommitChanged,
+	    renderedBuildCommit: renderedBuildCommit,
+	};
+    },
+
     compare_ceph_versions: function(a, b) {
 	let avers = [];
 	let bvers = [];
diff --git a/www/manager6/ceph/ServiceList.js b/www/manager6/ceph/ServiceList.js
index 2dd80c14..20dfb5d0 100644
--- a/www/manager6/ceph/ServiceList.js
+++ b/www/manager6/ceph/ServiceList.js
@@ -103,21 +103,34 @@ Ext.define('PVE.node.CephServiceController', {
 	if (value === undefined) {
 	    return '';
 	}
+
 	let view = this.getView();
-	let host = rec.data.host, nodev = [0];
+	let host = rec.data.host;
+
+	let nodeVersion = [0];
+	let nodeBuildCommit = '';
+
 	if (view.nodeversions[host] !== undefined) {
-	    nodev = view.nodeversions[host].version.parts;
+	    nodeVersion = view.nodeversions[host].version.parts;
+	    nodeBuildCommit = view.nodeversions[host].buildcommit;
 	}
 
-	let icon = '';
-	if (PVE.Utils.compare_ceph_versions(view.maxversion, nodev) > 0) {
-	    icon = PVE.Utils.get_ceph_icon_html('HEALTH_UPGRADE');
-	} else if (PVE.Utils.compare_ceph_versions(nodev, value) > 0) {
-	    icon = PVE.Utils.get_ceph_icon_html('HEALTH_OLD');
-	} else if (view.mixedversions) {
-	    icon = PVE.Utils.get_ceph_icon_html('HEALTH_OK');
+	let versionInfo = {
+	    runningVersion: value,
+	    installedVersion: nodeVersion,
+	    runningBuildCommit: PVE.Utils.parseCephBuildCommit(rec.data) ?? '',
+	    installedBuildCommit: nodeBuildCommit,
+	    maxVersion: view.maxversion,
+	    hasMixedVersions: view.mixedversions,
+	};
+
+	let { icon, renderedBuildCommit } = PVE.Utils.renderCephBuildCommit(versionInfo, 9);
+
+	if (renderedBuildCommit) {
+	    return `${icon}${value} (build: ${renderedBuildCommit})`;
 	}
-	return icon + value;
+
+	return `${icon}${value}`;
     },
 
     getMaxVersions: function(store, records, success) {
diff --git a/www/manager6/ceph/Services.js b/www/manager6/ceph/Services.js
index dfafee43..ff6f80e9 100644
--- a/www/manager6/ceph/Services.js
+++ b/www/manager6/ceph/Services.js
@@ -155,6 +155,7 @@ Ext.define('PVE.ceph.Services', {
 		    title: metadata[type][id].name || name,
 		    host: host,
 		    version: PVE.Utils.parse_ceph_version(metadata[type][id]),
+		    buildcommit: PVE.Utils.parseCephBuildCommit(metadata[type][id]),
 		    service: metadata[type][id].service,
 		    addr: metadata[type][id].addr || metadata[type][id].addrs || Proxmox.Utils.unknownText,
 		};
@@ -181,7 +182,15 @@ Ext.define('PVE.ceph.Services', {
 		}
 
 		if (result.version) {
-		    result.statuses.push(gettext('Version') + ": " + result.version);
+		    let installedBuildCommit = metadata.node[host]?.buildcommit ?? '';
+		    let runningBuildCommit = result.buildcommit ?? '';
+
+		    let statusLine = gettext('Version') + `: ${result.version}`;
+		    if (runningBuildCommit) {
+			statusLine += ` (build: ${runningBuildCommit.substring(0, 9)})`;
+		    }
+
+		    result.statuses.push(statusLine);
 
 		    if (PVE.Utils.compare_ceph_versions(result.version, maxversion) !== 0) {
 			let host_version = metadata.node[host]?.version?.parts || metadata.version?.[host] || "";
@@ -202,6 +211,14 @@ Ext.define('PVE.ceph.Services', {
 				gettext('Other cluster members use a newer version of this service, please upgrade and restart'),
 			    );
 			}
+		    } else if (installedBuildCommit !== "" && runningBuildCommit !== installedBuildCommit) {
+			if (result.health > healthstates.HEALTH_OLD) {
+			    result.health = healthstates.HEALTH_OLD;
+			}
+			result.messages.push(
+			    PVE.Utils.get_ceph_icon_html('HEALTH_OLD', true) +
+			    gettext('A newer build of this release was installed but old build still running, please restart'),
+			);
 		    }
 		}
 
-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v3 pve-manager 2/5] fix #5366: api: ceph: add host build commit to Ceph OSD index data

[Max Carrara]

This is required in order to avoid making multiple API calls in the
following commit.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Fixes: https://bugzilla.proxmox.com/show_bug.cgi?id=5366
Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>
---
Changes v2 --> v3:
  * reword commit title
  * add 'Fixes' trailer

Changes v1 --> v2:
  * none

 PVE/API2/Ceph/OSD.pm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/PVE/API2/Ceph/OSD.pm b/PVE/API2/Ceph/OSD.pm
index 5e39eed7..1ff51fbd 100644
--- a/PVE/API2/Ceph/OSD.pm
+++ b/PVE/API2/Ceph/OSD.pm
@@ -206,6 +206,7 @@ __PACKAGE__->register_method ({
 
 	    if ($name && $e->{type} eq 'host') {
 		$new->{version} = $hostversions->{$name}->{version}->{str};
+		$new->{buildcommit} = $hostversions->{$name}->{buildcommit};
 	    }
 	}
 
-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v3 pve-manager 3/5] fix #5366: ui: ceph: osd: rework version field rendering

[Max Carrara]

.. and show the build commit next to the OSD version.

The logic of the `render_version` function is split up in order to
handle how the version is displayed depending on the type of the row.

If the parsed version is `undefined` or the row marks the beginning of
the tree, an empty string is now returned. This behaviour is
equivalent to before, it just makes the overall logic easier.

If the row belongs to a node, the build commit is now additionally
displayed in parentheses next to the installed Ceph version:

  18.2.2 (build: abcd1234)

If the row belongs to an OSD, the build commit is also additionally
displayed in parentheses next to the installed Ceph version.
Furthermore, should the build commit of the running OSD differ from
the one that's installed on the host, the new hash will also be shown
in parentheses:

  18.2.2 (build: abcd1234 -> 5678fedc)

Additionally, the icon displayed for running an OSD with an outdated
build is now the same as for running an outdated version. The
conditional display of cluster health-related icons remains the same
otherwise.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Fixes: https://bugzilla.proxmox.com/show_bug.cgi?id=5366
---
Changes v2 --> v3:
  * use camelCase instead of nocaseatall for certain variables
  * use less ambiguous variable names (thanks Thomas!)
  * use new `renderCephBuildCommit` helper instead of duplicating logic
    (thanks Thomas!)
  * put 'build: ' in front of build commit when rendering (thanks Thomas!)
  * reword commit title
  * add 'Fixes' trailer
  * remove outdated R-b and T-b trailers

Changes v1 --> v2:
  * use camelCase instead of snake_case (thanks Lukas!)
  * use more descriptive variable names (thanks Lukas!)
  * use `let` instead of `const` for variables where applicable (thanks Lukas!)

 www/manager6/ceph/OSD.js | 53 +++++++++++++++++++++++++++++++---------
 1 file changed, 41 insertions(+), 12 deletions(-)

diff --git a/www/manager6/ceph/OSD.js b/www/manager6/ceph/OSD.js
index d2caafa4..11ec7670 100644
--- a/www/manager6/ceph/OSD.js
+++ b/www/manager6/ceph/OSD.js
@@ -642,23 +642,52 @@ Ext.define('PVE.node.CephOsdTree', {
 	},
 
 	render_version: function(value, metadata, rec) {
+	    if (value === undefined || rec.data.type === 'root') {
+		return '';
+	    }
+
 	    let vm = this.getViewModel();
-	    let versions = vm.get('versions');
-	    let icon = "";
-	    let version = value || "";
-	    let maxversion = vm.get('maxversion');
-	    if (value && PVE.Utils.compare_ceph_versions(value, maxversion) !== 0) {
-		let host_version = rec.parentNode?.data?.version || versions[rec.data.host] || "";
-		if (rec.data.type === 'host' || PVE.Utils.compare_ceph_versions(host_version, maxversion) !== 0) {
+	    let maxVersion = vm.get('maxversion');
+	    let hasMixedVersions = vm.get('mixedversions');
+
+	    if (rec.data.type === 'host') {
+		let icon = "";
+		let installedBuildCommit = rec.data.buildcommit ?? '';
+
+		if (PVE.Utils.compare_ceph_versions(maxVersion, value) > 0) {
 		    icon = PVE.Utils.get_ceph_icon_html('HEALTH_UPGRADE');
-		} else {
-		    icon = PVE.Utils.get_ceph_icon_html('HEALTH_OLD');
+		} else if (hasMixedVersions) {
+		    icon = PVE.Utils.get_ceph_icon_html('HEALTH_OK');
+		}
+
+		if (installedBuildCommit === '') {
+		    return `${icon}${value}`;
 		}
-	    } else if (value && vm.get('mixedversions')) {
-		icon = PVE.Utils.get_ceph_icon_html('HEALTH_OK');
+
+		return `${icon}${value} (build: ${installedBuildCommit.substring(0, 9)})`;
+	    }
+
+	    let installedVersion = rec.parentNode?.data?.version ?? '';
+
+	    let runningBuildCommit = PVE.Utils.parseCephBuildCommit(rec.data) ?? '';
+	    let installedBuildCommit = rec.parentNode?.data?.buildcommit ?? '';
+
+	    let versionInfo = {
+		runningVersion: value,
+		installedVersion: installedVersion,
+		runningBuildCommit: runningBuildCommit,
+		installedBuildCommit: installedBuildCommit,
+		maxVersion: maxVersion,
+		hasMixedVersions: hasMixedVersions,
+	    };
+
+	    let { icon, renderedBuildCommit } = PVE.Utils.renderCephBuildCommit(versionInfo, 9);
+
+	    if (renderedBuildCommit) {
+		return `${icon}${value} (build: ${renderedBuildCommit})`;
 	    }
 
-	    return icon + version;
+	    return `${icon}${value}`;
 	},
 
 	render_osd_val: function(value, metaData, rec) {
-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v3 pve-manager 4/5] ui: ceph: osd: increase width of version column

[Max Carrara]

.. so that the Ceph build commit as well as differing build commits
are shown properly.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
---
Changes v2 --> v3:
  * increase the width even further to account for new changes

Changes v1 --> v2:
  * NEW

 www/manager6/ceph/OSD.js | 1 +
 1 file changed, 1 insertion(+)

diff --git a/www/manager6/ceph/OSD.js b/www/manager6/ceph/OSD.js
index 11ec7670..72fea489 100644
--- a/www/manager6/ceph/OSD.js
+++ b/www/manager6/ceph/OSD.js
@@ -802,6 +802,7 @@ Ext.define('PVE.node.CephOsdTree', {
 	    dataIndex: 'version',
 	    align: 'right',
 	    renderer: 'render_version',
+	    width: 270,
 	},
 	{
 	    text: 'weight',
-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH v3 pve-manager 5/5] fix #5366: api: ceph: change version format in OSD metadata endpoint

[Max Carrara]

.. in order to include Ceph's build commit. Instead of e.g.

  18.2.2 (reef)

the string will now contain:

  18.2.2 (build: e9fe820e7) reef

This format is used in the OSD detail view; the build commit will
therefore also be shown there.

Signed-off-by: Max Carrara <m.carrara@proxmox.com>
Fixes: https://bugzilla.proxmox.com/show_bug.cgi?id=5366
---
Changes v2 --> v3:
  * put 'build: ' in front of build commit (thanks Thomas!)
  * reword commit title
  * add 'Fixes' trailer

Changes v1 --> v2:
  * NEW

 PVE/API2/Ceph/OSD.pm | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/PVE/API2/Ceph/OSD.pm b/PVE/API2/Ceph/OSD.pm
index 1ff51fbd..88b7142b 100644
--- a/PVE/API2/Ceph/OSD.pm
+++ b/PVE/API2/Ceph/OSD.pm
@@ -747,6 +747,12 @@ __PACKAGE__->register_method ({
 	my $osd_pss_memory = eval { get_proc_pss_from_pid($pid) } // 0;
 	warn $@ if $@;
 
+	my ($ceph_version, $ceph_buildcommit) = PVE::Ceph::Tools::parse_ceph_version(
+	    $metadata->{ceph_version}
+	);
+
+	$ceph_buildcommit = substr($ceph_buildcommit, 0, 9);
+
 	my $data = {
 	    osd => {
 		hostname => $metadata->{hostname},
@@ -755,7 +761,7 @@ __PACKAGE__->register_method ({
 		osd_data => $metadata->{osd_data},
 		osd_objectstore => $metadata->{osd_objectstore},
 		pid => $pid,
-		version => "$metadata->{ceph_version_short} ($metadata->{ceph_release})",
+		version => "$ceph_version (build: $ceph_buildcommit) $metadata->{ceph_release}",
 		front_addr => $metadata->{front_addr},
 		back_addr => $metadata->{back_addr},
 		hb_front_addr => $metadata->{hb_front_addr},
-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel