all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH ha-manager 05/15] env: pve2: delegate cluster stat gathering to scheduling library
Date: Wed, 18 Nov 2026 12:05:12 +0100	[thread overview]
Message-ID: <20261118110705.595989-6-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20261118110705.595989-1-t.lamprecht@proxmox.com>

The cluster node and guest stat gathering now lives in the
PVE::ResourceScheduling::Stats module of libpve-guest-common-perl.
Consume it here instead of keeping a second copy, so the HA manager
and the resource scheduling API share a single implementation and
cannot drift apart. The library's dynamic guest stats carry the host
memory accounting from fix #7974, and the stats the HA manager sees
stay the same.

The library has no dependency on the HA resource plugins, so build
the static per-guest-type usage resolvers from those plugins and pass
them in. As the library reports undecodable static node info as a
plain warning, log that at error level like before.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
NOTE: the new libpve-guest-common-perl dependency needs a version
requirement for the first release with PVE::ResourceScheduling.

 debian/control         |   2 +
 src/PVE/HA/Env/PVE2.pm | 155 ++++++++---------------------------------
 2 files changed, 32 insertions(+), 125 deletions(-)

diff --git a/debian/control b/debian/control
index 107e367..2577d43 100644
--- a/debian/control
+++ b/debian/control
@@ -8,6 +8,7 @@ Build-Depends: debhelper-compat (= 13),
                libpve-access-control,
                libpve-cluster-perl,
                libpve-common-perl,
+               libpve-guest-common-perl,
                libpve-notify-perl,
                libpve-rs-perl (>= 0.14.0),
                libtest-mockmodule-perl <!nocheck>,
@@ -23,6 +24,7 @@ Architecture: any
 Depends: libjson-perl,
          libpve-cluster-perl (>= 9.1.5),
          libpve-common-perl,
+         libpve-guest-common-perl,
          libpve-notify-perl (>= 8.1.0),
          libpve-rs-perl (>= 0.14.0),
          pve-cluster (>= 3.0-17),
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 5f195da..de56e20 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -14,6 +14,7 @@ use PVE::DataCenterConfig;
 use PVE::INotify;
 use PVE::RPCEnvironment;
 use PVE::Notify;
+use PVE::ResourceScheduling::Stats;
 
 use PVE::HA::Tools ':exit_codes';
 use PVE::HA::Env;
@@ -36,30 +37,19 @@ PVE::HA::Rules::ResourceAffinity->register();
 
 PVE::HA::Rules->init(property_isolation => 1);
 
+# The stats library has no dependency on the HA resource plugins, so build the per-guest-type static
+# usage resolvers from those plugins and pass them in.
+my sub static_stats_resolvers {
+    return {
+        map {
+            my $plugin = PVE::HA::Resources->lookup($_);
+            ($_ => sub { $plugin->get_static_stats_from_config($_[0]) });
+        } qw(vm ct)
+    };
+}
+
 my $lockdir = "/etc/pve/priv/lock";
 
-# rrd entry indices for VM and CT guests
-# taken from PVE::Service::pvestatd::update_{lxc,qemu}_status()
-use constant {
-    RRD_VM_INDEX_STATUS => 2,
-    RRD_VM_INDEX_MAXCPU => 5,
-    RRD_VM_INDEX_CPU => 6,
-    RRD_VM_INDEX_MAXMEM => 7,
-    RRD_VM_INDEX_MEM => 8,
-    RRD_VM_INDEX_MEMHOST => 15,
-};
-
-# rrd entry indices for PVE nodes
-# taken from PVE::Service::pvestatd::update_node_status()
-use constant {
-    RRD_NODE_INDEX_MAXCPU => 4,
-    RRD_NODE_INDEX_CPU => 5,
-    RRD_NODE_INDEX_MAXMEM => 7,
-    RRD_NODE_INDEX_MEM => 8,
-};
-
-my $HOSTNAME_RE = qr/(?:[a-zA-Z0-9](?:[a-zA-Z0-9\-]{,61}?[a-zA-Z0-9])?)/;
-
 sub new {
     my ($this, $nodename) = @_;
 
@@ -529,133 +519,48 @@ sub get_datacenter_settings {
     };
 }
 
-my sub get_cluster_service_stats {
-    my $vmlist = PVE::Cluster::get_vmlist();
-    my $idlist = $vmlist->{ids} // {};
-
-    my $rrd = PVE::Cluster::rrd_dump();
-
-    my $stats = {};
-    for my $id (keys %$idlist) {
-        my $type = eval { PVE::HA::Tools::get_ha_resource_type($idlist->{$id}->{type}) };
-        next if $@; # silently ignore unknown pve types
-
-        my $sid = "$type:$id";
-        my $nodename = $idlist->{$id}->{node};
-
-        my $rrdentry = $rrd->{"pve-vm-9.0/$id"} // [];
-        # can be any QMP RunState, but 'running' is the only active VM state
-        my $status = $rrdentry->[RRD_VM_INDEX_STATUS] // "stopped";
-        my $state = $status eq "running" ? "started" : "stopped";
-
-        $stats->{$sid} = {
-            id => $id,
-            node => $nodename,
-            state => $state,
-            running => $state eq 'started',
-            type => $type,
-            usage => {},
-        };
-    }
-
-    return $stats;
-}
-
 sub get_basic_service_stats {
     my ($self) = @_;
 
-    return get_cluster_service_stats();
+    return PVE::ResourceScheduling::Stats::get_cluster_guest_stats();
 }
 
 sub get_static_service_stats {
     my ($self) = @_;
 
-    my $properties = ['cores', 'cpulimit', 'memory', 'sockets', 'vcpus'];
-    my $stats = get_cluster_service_stats();
-    my $confs = PVE::Cluster::get_guest_config_properties($properties);
-
-    for my $sid (keys %$stats) {
-        my ($id, $type) = $stats->{$sid}->@{qw(id type)};
-
-        my $conf = $confs->{$id} // {};
-        my $plugin = PVE::HA::Resources->lookup($type);
-
-        $stats->{$sid}->{usage} = $plugin->get_static_stats_from_config($conf);
-    }
-
-    return $stats;
+    return PVE::ResourceScheduling::Stats::get_static_guest_stats(static_stats_resolvers());
 }
 
 sub get_dynamic_service_stats {
     my ($self) = @_;
 
-    my $rrd = PVE::Cluster::rrd_dump();
-
-    my $stats = get_cluster_service_stats();
-    for my $sid (keys %$stats) {
-        my $id = $stats->{$sid}->{id};
-        my $rrdentry = $rrd->{"pve-vm-9.0/$id"} // [];
-
-        # NOTE the guests' broadcasted vmstatus() caps maxcpu at the node's maxcpu
-        my $maxcpu = ($rrdentry->[RRD_VM_INDEX_MAXCPU] || 0.0) + 0.0;
-
-        # designated fallback for containers
-        my $mem = $rrdentry->[RRD_VM_INDEX_MEMHOST] // $rrdentry->[RRD_VM_INDEX_MEM];
-
-        $stats->{$sid}->{usage} = {
-            maxcpu => $maxcpu,
-            cpu => (($rrdentry->[RRD_VM_INDEX_CPU] || 0.0) + 0.0) * $maxcpu,
-            maxmem => int($rrdentry->[RRD_VM_INDEX_MAXMEM] || 0),
-            mem => int($mem || 0),
-        };
-    }
-
-    return $stats;
+    return PVE::ResourceScheduling::Stats::get_dynamic_guest_stats();
 }
 
 sub get_static_node_stats {
     my ($self) = @_;
 
-    my $stats = PVE::Cluster::get_node_kv('static-info');
-    for my $node (keys $stats->%*) {
-        $stats->{$node} = eval {
-            my $node_stats = decode_json($stats->{$node});
+    # The library reports undecodable static node info with warn(). Log that at 'err' level, as an
+    # online node without static info makes the CRM fall back to basic scheduling.
+    my $previous = $SIG{__WARN__};
+    local $SIG{__WARN__} = sub {
+        my ($msg) = @_;
+        if ($msg =~ m/^unable to decode static node info /) {
+            $self->log('err', $msg);
+        } elsif (ref($previous) eq 'CODE') {
+            $previous->($msg);
+        } else {
+            warn $msg;
+        }
+    };
 
-            return {
-                maxcpu => $node_stats->{cpus},
-                maxmem => $node_stats->{memory},
-            };
-        };
-        $self->log('err', "unable to decode static node info for '$node' - $@") if $@;
-    }
-
-    return $stats;
+    return PVE::ResourceScheduling::Stats::get_static_node_stats();
 }
 
 sub get_dynamic_node_stats {
     my ($self) = @_;
 
-    my $rrd = PVE::Cluster::rrd_dump();
-
-    my $stats = {};
-    for my $key (keys %$rrd) {
-        my ($nodename) = $key =~ m/^pve-node-9.0\/($HOSTNAME_RE)$/;
-
-        next if !$nodename;
-
-        my $rrdentry = $rrd->{$key} // [];
-
-        my $maxcpu = int($rrdentry->[RRD_NODE_INDEX_MAXCPU] || 0);
-
-        $stats->{$nodename} = {
-            maxcpu => $maxcpu,
-            cpu => (($rrdentry->[RRD_NODE_INDEX_CPU] || 0.0) + 0.0) * $maxcpu,
-            maxmem => int($rrdentry->[RRD_NODE_INDEX_MAXMEM] || 0),
-            mem => int($rrdentry->[RRD_NODE_INDEX_MEM] || 0),
-        };
-    }
-
-    return $stats;
+    return PVE::ResourceScheduling::Stats::get_dynamic_node_stats();
 }
 
 sub get_node_version {
-- 
2.47.3





  parent reply	other threads:[~2026-11-18 11:08 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
2026-11-18 11:05 ` [PATCH guest-common 01/15] scheduling: add cluster resource scheduling library Thomas Lamprecht
2026-11-18 11:05 ` [PATCH ha-manager 02/15] helpers: block both locations of a migrating negative affinity partner Thomas Lamprecht
2026-11-18 11:05 ` [PATCH ha-manager 03/15] usage: static: count services per node again for scoring fallback Thomas Lamprecht
2026-11-18 11:05 ` [PATCH ha-manager 04/15] config: allow getting motion info of many resources at once Thomas Lamprecht
2026-11-18 11:05 ` Thomas Lamprecht [this message]
2026-11-18 11:05 ` [PATCH ha-manager 06/15] usage: share one scheduler base between static and dynamic Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 07/15] ui: bulk action: translate error title and CT restart warning Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 08/15] ui: bulk migrate: show CT restart warning for cluster-wide migration Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 09/15] ui: migrate: recheck preconditions if the target changed meanwhile Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 10/15] api: cluster: add resource scheduling endpoints Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 11/15] ui: migrate: sort target nodes by scheduler recommendation Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 12/15] api: bulk migrate: allow a per-guest target node mapping Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 13/15] ui: bulk migrate: support per-guest target nodes Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 14/15] ui: dc: add a Rebalance Guests bulk action Thomas Lamprecht
2026-11-18 11:05 ` [PATCH docs 15/15] pvecm: document balancing the cluster load with migrations Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118110705.595989-6-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal