* [PATCH ha-manager v2 01/10] rename static node stats to be consistent with similar interfaces
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
@ 2026-03-19 16:33 ` Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 02/10] resources: remove redundant load_config fallback for static config Daniel Kral
` (9 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:33 UTC (permalink / raw)
To: pve-devel
The names `maxcpu` and `maxmem` are used in the static load scheduler
itself and is more telling that these properties provide the maximum
configured amount of CPU cores and memory.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Env/PVE2.pm | 9 ++++++++-
src/PVE/HA/Sim/Hardware.pm | 8 ++++----
src/PVE/HA/Usage/Static.pm | 6 +++---
.../hardware_status | 6 +++---
.../hardware_status | 6 +++---
.../hardware_status | 10 +++++-----
src/test/test-crs-static-rebalance1/hardware_status | 6 +++---
src/test/test-crs-static-rebalance2/hardware_status | 6 +++---
src/test/test-crs-static1/hardware_status | 6 +++---
src/test/test-crs-static2/hardware_status | 10 +++++-----
src/test/test-crs-static3/hardware_status | 6 +++---
src/test/test-crs-static4/hardware_status | 6 +++---
src/test/test-crs-static5/hardware_status | 6 +++---
13 files changed, 49 insertions(+), 42 deletions(-)
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 4a38cc3b..5b8aaae6 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -548,7 +548,14 @@ sub get_static_node_stats {
my $stats = PVE::Cluster::get_node_kv('static-info');
for my $node (keys $stats->%*) {
- $stats->{$node} = eval { decode_json($stats->{$node}) };
+ $stats->{$node} = eval {
+ my $node_stats = decode_json($stats->{$node});
+
+ return {
+ maxcpu => $node_stats->{cpus},
+ maxmem => $node_stats->{memory},
+ };
+ };
$self->log('err', "unable to decode static node info for '$node' - $@") if $@;
}
diff --git a/src/PVE/HA/Sim/Hardware.pm b/src/PVE/HA/Sim/Hardware.pm
index 8cbf48df..60eb9867 100644
--- a/src/PVE/HA/Sim/Hardware.pm
+++ b/src/PVE/HA/Sim/Hardware.pm
@@ -488,9 +488,9 @@ sub new {
|| die "Copy failed: $!\n";
} else {
my $cstatus = {
- node1 => { power => 'off', network => 'off', cpus => 24, memory => 131072 },
- node2 => { power => 'off', network => 'off', cpus => 24, memory => 131072 },
- node3 => { power => 'off', network => 'off', cpus => 24, memory => 131072 },
+ node1 => { power => 'off', network => 'off', maxcpu => 24, maxmem => 131072 },
+ node2 => { power => 'off', network => 'off', maxcpu => 24, maxmem => 131072 },
+ node3 => { power => 'off', network => 'off', maxcpu => 24, maxmem => 131072 },
};
$self->write_hardware_status_nolock($cstatus);
}
@@ -1088,7 +1088,7 @@ sub get_static_node_stats {
my $stats = {};
for my $node (keys $cstatus->%*) {
- $stats->{$node} = { $cstatus->{$node}->%{qw(cpus memory)} };
+ $stats->{$node} = { $cstatus->{$node}->%{qw(maxcpu maxmem)} };
}
return $stats;
diff --git a/src/PVE/HA/Usage/Static.pm b/src/PVE/HA/Usage/Static.pm
index d586b603..395be871 100644
--- a/src/PVE/HA/Usage/Static.pm
+++ b/src/PVE/HA/Usage/Static.pm
@@ -33,10 +33,10 @@ sub add_node {
my $stats = $self->{'node-stats'}->{$nodename}
or die "did not get static node usage information for '$nodename'\n";
- die "static node usage information for '$nodename' missing cpu count\n" if !$stats->{cpus};
- die "static node usage information for '$nodename' missing memory\n" if !$stats->{memory};
+ die "static node usage information for '$nodename' missing cpu count\n" if !$stats->{maxcpu};
+ die "static node usage information for '$nodename' missing memory\n" if !$stats->{maxmem};
- eval { $self->{scheduler}->add_node($nodename, int($stats->{cpus}), int($stats->{memory})); };
+ eval { $self->{scheduler}->add_node($nodename, int($stats->{maxcpu}), int($stats->{maxmem})); };
die "initializing static node usage for '$nodename' failed - $@" if $@;
}
diff --git a/src/test/test-crs-static-rebalance-resource-affinity1/hardware_status b/src/test/test-crs-static-rebalance-resource-affinity1/hardware_status
index 84484af1..3d4cf91f 100644
--- a/src/test/test-crs-static-rebalance-resource-affinity1/hardware_status
+++ b/src/test/test-crs-static-rebalance-resource-affinity1/hardware_status
@@ -1,5 +1,5 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 8, "memory": 112000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 8, "memory": 112000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 8, "memory": 112000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 8, "maxmem": 112000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 8, "maxmem": 112000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 8, "maxmem": 112000000000 }
}
diff --git a/src/test/test-crs-static-rebalance-resource-affinity2/hardware_status b/src/test/test-crs-static-rebalance-resource-affinity2/hardware_status
index 84484af1..3d4cf91f 100644
--- a/src/test/test-crs-static-rebalance-resource-affinity2/hardware_status
+++ b/src/test/test-crs-static-rebalance-resource-affinity2/hardware_status
@@ -1,5 +1,5 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 8, "memory": 112000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 8, "memory": 112000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 8, "memory": 112000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 8, "maxmem": 112000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 8, "maxmem": 112000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 8, "maxmem": 112000000000 }
}
diff --git a/src/test/test-crs-static-rebalance-resource-affinity3/hardware_status b/src/test/test-crs-static-rebalance-resource-affinity3/hardware_status
index b6dcb1a5..7bc741f1 100644
--- a/src/test/test-crs-static-rebalance-resource-affinity3/hardware_status
+++ b/src/test/test-crs-static-rebalance-resource-affinity3/hardware_status
@@ -1,7 +1,7 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 8, "memory": 48000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 32, "memory": 36000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 16, "memory": 24000000000 },
- "node4": { "power": "off", "network": "off", "cpus": 32, "memory": 36000000000 },
- "node5": { "power": "off", "network": "off", "cpus": 8, "memory": 48000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 8, "maxmem": 48000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 36000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 24000000000 },
+ "node4": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 36000000000 },
+ "node5": { "power": "off", "network": "off", "maxcpu": 8, "maxmem": 48000000000 }
}
diff --git a/src/test/test-crs-static-rebalance1/hardware_status b/src/test/test-crs-static-rebalance1/hardware_status
index 651ad792..bfdbbf7b 100644
--- a/src/test/test-crs-static-rebalance1/hardware_status
+++ b/src/test/test-crs-static-rebalance1/hardware_status
@@ -1,5 +1,5 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 32, "memory": 256000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 32, "memory": 256000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 32, "memory": 256000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 256000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 256000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 256000000000 }
}
diff --git a/src/test/test-crs-static-rebalance2/hardware_status b/src/test/test-crs-static-rebalance2/hardware_status
index 9be70a40..c5cbde3d 100644
--- a/src/test/test-crs-static-rebalance2/hardware_status
+++ b/src/test/test-crs-static-rebalance2/hardware_status
@@ -1,5 +1,5 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 40, "memory": 384000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 32, "memory": 256000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 32, "memory": 256000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 40, "maxmem": 384000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 256000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 256000000000 }
}
diff --git a/src/test/test-crs-static1/hardware_status b/src/test/test-crs-static1/hardware_status
index 0fa8c265..bbe44a96 100644
--- a/src/test/test-crs-static1/hardware_status
+++ b/src/test/test-crs-static1/hardware_status
@@ -1,5 +1,5 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 32, "memory": 200000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 32, "memory": 300000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 200000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 300000000000 }
}
diff --git a/src/test/test-crs-static2/hardware_status b/src/test/test-crs-static2/hardware_status
index d426023a..815436ef 100644
--- a/src/test/test-crs-static2/hardware_status
+++ b/src/test/test-crs-static2/hardware_status
@@ -1,7 +1,7 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 32, "memory": 200000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 32, "memory": 300000000000 },
- "node4": { "power": "off", "network": "off", "cpus": 64, "memory": 300000000000 },
- "node5": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 200000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 300000000000 },
+ "node4": { "power": "off", "network": "off", "maxcpu": 64, "maxmem": 300000000000 },
+ "node5": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 }
}
diff --git a/src/test/test-crs-static3/hardware_status b/src/test/test-crs-static3/hardware_status
index dfbf496e..ed84b8bd 100644
--- a/src/test/test-crs-static3/hardware_status
+++ b/src/test/test-crs-static3/hardware_status
@@ -1,5 +1,5 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 64, "memory": 200000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 64, "maxmem": 200000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 }
}
diff --git a/src/test/test-crs-static4/hardware_status b/src/test/test-crs-static4/hardware_status
index a83a2dcc..b08ba7f9 100644
--- a/src/test/test-crs-static4/hardware_status
+++ b/src/test/test-crs-static4/hardware_status
@@ -1,5 +1,5 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 }
}
diff --git a/src/test/test-crs-static5/hardware_status b/src/test/test-crs-static5/hardware_status
index 3eb9e735..edfd6db2 100644
--- a/src/test/test-crs-static5/hardware_status
+++ b/src/test/test-crs-static5/hardware_status
@@ -1,5 +1,5 @@
{
- "node1": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 },
- "node2": { "power": "off", "network": "off", "cpus": 32, "memory": 100000000000 },
- "node3": { "power": "off", "network": "off", "cpus": 128, "memory": 100000000000 }
+ "node1": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 32, "maxmem": 100000000000 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 128, "maxmem": 100000000000 }
}
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH ha-manager v2 02/10] resources: remove redundant load_config fallback for static config
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 01/10] rename static node stats to be consistent with similar interfaces Daniel Kral
@ 2026-03-19 16:33 ` Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 03/10] remove redundant service_node and migration_target parameter Daniel Kral
` (8 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:33 UTC (permalink / raw)
To: pve-devel
The return value of get_static_service_stats(...) is fetched through
PVE::Cluster::get_guest_config_properties(...), which in turn reads all
guest configuration files with a memdb_read_nolock(...) in the pmxcfs.
As PVE::AbstractConfig::load_config(...) internally gets the content of
the guest configuration file through cfs_read_file(...), which in turn
receives the return value of the equivalent memdb_read(...) from a
CFS_IPC_GET_CONFIG message, the fallback is likely to fail as well.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Resources/PVECT.pm | 1 -
src/PVE/HA/Resources/PVEVM.pm | 1 -
2 files changed, 2 deletions(-)
diff --git a/src/PVE/HA/Resources/PVECT.pm b/src/PVE/HA/Resources/PVECT.pm
index 4cbf6db3..b9ce2ac3 100644
--- a/src/PVE/HA/Resources/PVECT.pm
+++ b/src/PVE/HA/Resources/PVECT.pm
@@ -163,7 +163,6 @@ sub get_static_stats {
my ($class, $haenv, $id, $service_node) = @_;
my $conf = $haenv->get_static_service_stats($id);
- $conf = PVE::LXC::Config->load_config($id, $service_node) if !defined($conf);
return {
maxcpu => PVE::LXC::Config->get_derived_property($conf, 'max-cpu'),
diff --git a/src/PVE/HA/Resources/PVEVM.pm b/src/PVE/HA/Resources/PVEVM.pm
index 7586da84..303334ba 100644
--- a/src/PVE/HA/Resources/PVEVM.pm
+++ b/src/PVE/HA/Resources/PVEVM.pm
@@ -184,7 +184,6 @@ sub get_static_stats {
my ($class, $haenv, $id, $service_node) = @_;
my $conf = $haenv->get_static_service_stats($id);
- $conf = PVE::QemuConfig->load_config($id, $service_node) if !defined($conf);
return {
maxcpu => PVE::QemuConfig->get_derived_property($conf, 'max-cpu'),
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH ha-manager v2 03/10] remove redundant service_node and migration_target parameter
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 01/10] rename static node stats to be consistent with similar interfaces Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 02/10] resources: remove redundant load_config fallback for static config Daniel Kral
@ 2026-03-19 16:33 ` Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 04/10] factor out common pve to ha resource type mapping Daniel Kral
` (7 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:33 UTC (permalink / raw)
To: pve-devel
As the retrieval of the static service stats are not dependent on the
location of the guest's config file, there is no need for providing the
$service_node and $migration_target arguments anymore.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Resources.pm | 2 +-
src/PVE/HA/Resources/PVECT.pm | 2 +-
src/PVE/HA/Resources/PVEVM.pm | 2 +-
src/PVE/HA/Sim/Resources.pm | 2 +-
src/PVE/HA/Usage.pm | 10 ++++------
src/PVE/HA/Usage/Basic.pm | 4 ++--
src/PVE/HA/Usage/Static.pm | 19 +++++++------------
7 files changed, 17 insertions(+), 24 deletions(-)
diff --git a/src/PVE/HA/Resources.pm b/src/PVE/HA/Resources.pm
index b6ccd44a..3b20d44c 100644
--- a/src/PVE/HA/Resources.pm
+++ b/src/PVE/HA/Resources.pm
@@ -178,7 +178,7 @@ sub remove_locks {
}
sub get_static_stats {
- my ($class, $haenv, $id, $service_node) = @_;
+ my ($class, $haenv, $id) = @_;
die "implement in subclass";
}
diff --git a/src/PVE/HA/Resources/PVECT.pm b/src/PVE/HA/Resources/PVECT.pm
index b9ce2ac3..1f4eb2e9 100644
--- a/src/PVE/HA/Resources/PVECT.pm
+++ b/src/PVE/HA/Resources/PVECT.pm
@@ -160,7 +160,7 @@ sub remove_locks {
}
sub get_static_stats {
- my ($class, $haenv, $id, $service_node) = @_;
+ my ($class, $haenv, $id) = @_;
my $conf = $haenv->get_static_service_stats($id);
diff --git a/src/PVE/HA/Resources/PVEVM.pm b/src/PVE/HA/Resources/PVEVM.pm
index 303334ba..760259e4 100644
--- a/src/PVE/HA/Resources/PVEVM.pm
+++ b/src/PVE/HA/Resources/PVEVM.pm
@@ -181,7 +181,7 @@ sub remove_locks {
}
sub get_static_stats {
- my ($class, $haenv, $id, $service_node) = @_;
+ my ($class, $haenv, $id) = @_;
my $conf = $haenv->get_static_service_stats($id);
diff --git a/src/PVE/HA/Sim/Resources.pm b/src/PVE/HA/Sim/Resources.pm
index 34462522..48c15dfe 100644
--- a/src/PVE/HA/Sim/Resources.pm
+++ b/src/PVE/HA/Sim/Resources.pm
@@ -138,7 +138,7 @@ sub remove_locks {
}
sub get_static_stats {
- my ($class, $haenv, $id, $service_node) = @_;
+ my ($class, $haenv, $id) = @_;
my $sid = $class->type() . ":$id";
my $hardware = $haenv->hardware();
diff --git a/src/PVE/HA/Usage.pm b/src/PVE/HA/Usage.pm
index 92e575cb..1be5fa09 100644
--- a/src/PVE/HA/Usage.pm
+++ b/src/PVE/HA/Usage.pm
@@ -35,7 +35,7 @@ sub contains_node {
# Logs a warning to $haenv upon failure, but does not die.
sub add_service_usage_to_node {
- my ($self, $nodename, $sid, $service_node, $migration_target) = @_;
+ my ($self, $nodename, $sid) = @_;
die "implement in subclass";
}
@@ -49,10 +49,8 @@ sub add_service_usage {
my ($current_node, $target_node) =
get_used_service_nodes($online_nodes, $service_state, $service_node, $migration_target);
- $self->add_service_usage_to_node($current_node, $sid, $service_node, $migration_target)
- if $current_node;
- $self->add_service_usage_to_node($target_node, $sid, $service_node, $migration_target)
- if $target_node;
+ $self->add_service_usage_to_node($current_node, $sid) if $current_node;
+ $self->add_service_usage_to_node($target_node, $sid) if $target_node;
}
sub remove_service_usage {
@@ -63,7 +61,7 @@ sub remove_service_usage {
# Returns a hash with $nodename => $score pairs. A lower $score is better.
sub score_nodes_to_start_service {
- my ($self, $sid, $service_node) = @_;
+ my ($self, $sid) = @_;
die "implement in subclass";
}
diff --git a/src/PVE/HA/Usage/Basic.pm b/src/PVE/HA/Usage/Basic.pm
index 43817bf6..ef9ae3d6 100644
--- a/src/PVE/HA/Usage/Basic.pm
+++ b/src/PVE/HA/Usage/Basic.pm
@@ -39,7 +39,7 @@ sub contains_node {
}
sub add_service_usage_to_node {
- my ($self, $nodename, $sid, $service_node, $migration_target) = @_;
+ my ($self, $nodename, $sid) = @_;
if ($self->contains_node($nodename)) {
$self->{nodes}->{$nodename}->{$sid} = 1;
@@ -60,7 +60,7 @@ sub remove_service_usage {
}
sub score_nodes_to_start_service {
- my ($self, $sid, $service_node) = @_;
+ my ($self, $sid) = @_;
my $nodes = $self->{nodes};
diff --git a/src/PVE/HA/Usage/Static.pm b/src/PVE/HA/Usage/Static.pm
index 395be871..2304139c 100644
--- a/src/PVE/HA/Usage/Static.pm
+++ b/src/PVE/HA/Usage/Static.pm
@@ -61,20 +61,15 @@ sub contains_node {
}
my sub get_service_usage {
- my ($self, $sid, $service_node, $migration_target) = @_;
+ my ($self, $sid) = @_;
return $self->{'service-stats'}->{$sid} if $self->{'service-stats'}->{$sid};
my (undef, $type, $id) = $self->{haenv}->parse_sid($sid);
my $plugin = PVE::HA::Resources->lookup($type);
- my $stats = eval { $plugin->get_static_stats($self->{haenv}, $id, $service_node) };
- if (my $err = $@) {
- # config might've already moved during a migration
- $stats = eval { $plugin->get_static_stats($self->{haenv}, $id, $migration_target); }
- if $migration_target;
- die "did not get static service usage information for '$sid' - $err\n" if !$stats;
- }
+ my $stats = eval { $plugin->get_static_stats($self->{haenv}, $id) };
+ die "did not get static service usage information for '$sid'\n" if !$stats;
my $service_stats = {
maxcpu => $stats->{maxcpu} + 0.0, # containers allow non-integer cpulimit
@@ -87,12 +82,12 @@ my sub get_service_usage {
}
sub add_service_usage_to_node {
- my ($self, $nodename, $sid, $service_node, $migration_target) = @_;
+ my ($self, $nodename, $sid) = @_;
$self->{'node-services'}->{$nodename}->{$sid} = 1;
eval {
- my $service_usage = get_service_usage($self, $sid, $service_node, $migration_target);
+ my $service_usage = get_service_usage($self, $sid);
$self->{scheduler}->add_service_usage_to_node($nodename, $sid, $service_usage);
};
$self->{haenv}->log('warning', "unable to add service '$sid' usage to node '$nodename' - $@")
@@ -111,10 +106,10 @@ sub remove_service_usage {
}
sub score_nodes_to_start_service {
- my ($self, $sid, $service_node) = @_;
+ my ($self, $sid) = @_;
my $score_list = eval {
- my $service_usage = get_service_usage($self, $sid, $service_node);
+ my $service_usage = get_service_usage($self, $sid);
$self->{scheduler}->score_nodes_to_start_service($service_usage);
};
if (my $err = $@) {
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH ha-manager v2 04/10] factor out common pve to ha resource type mapping
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
` (2 preceding siblings ...)
2026-03-19 16:33 ` [PATCH ha-manager v2 03/10] remove redundant service_node and migration_target parameter Daniel Kral
@ 2026-03-19 16:33 ` Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 05/10] derive static service stats while filling the service stats repository Daniel Kral
` (6 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:33 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Config.pm | 8 +-------
src/PVE/HA/Tools.pm | 23 +++++++++++++++--------
2 files changed, 16 insertions(+), 15 deletions(-)
diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index 19eec2a4..cafbae8c 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -197,13 +197,7 @@ sub parse_sid {
my $vmlist = PVE::Cluster::get_vmlist();
if (defined($vmlist->{ids}->{$name})) {
my $vm_type = $vmlist->{ids}->{$name}->{type};
- if ($vm_type eq 'lxc') {
- $type = 'ct';
- } elsif ($vm_type eq 'qemu') {
- $type = 'vm';
- } else {
- die "internal error";
- }
+ $type = PVE::HA::Tools::get_ha_resource_type($vm_type);
$sid = "$type:$name";
} else {
die "unable to detect SID from VMID - VM/CT $1 does not exist\n";
diff --git a/src/PVE/HA/Tools.pm b/src/PVE/HA/Tools.pm
index 1fa53df8..26629fb5 100644
--- a/src/PVE/HA/Tools.pm
+++ b/src/PVE/HA/Tools.pm
@@ -289,6 +289,18 @@ sub has_min_version {
return 1;
}
+sub get_ha_resource_type {
+ my ($pve_resource_type) = @_;
+
+ if ($pve_resource_type eq 'lxc') {
+ return 'ct';
+ } elsif ($pve_resource_type eq 'qemu') {
+ return 'vm';
+ } else {
+ die "unknown PVE resource type '$pve_resource_type'";
+ }
+}
+
# bash auto completion helper
# NOTE: we use PVE::HA::Config here without declaring an 'use' clause above as
@@ -309,15 +321,10 @@ sub complete_sid {
while (my ($vmid, $info) = each %{ $vmlist->{ids} }) {
- my $sid;
+ my $type = eval { get_ha_resource_type($info->{type}) };
+ next if $@; # silently ignore unknown pve types
- if ($info->{type} eq 'lxc') {
- $sid = "ct:$vmid";
- } elsif ($info->{type} eq 'qemu') {
- $sid = "vm:$vmid";
- } else {
- next; # should not happen
- }
+ my $sid = "$type:$vmid";
next if $cfg->{ids}->{$sid};
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH ha-manager v2 05/10] derive static service stats while filling the service stats repository
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
` (3 preceding siblings ...)
2026-03-19 16:33 ` [PATCH ha-manager v2 04/10] factor out common pve to ha resource type mapping Daniel Kral
@ 2026-03-19 16:33 ` Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 06/10] test: make static service usage explicit for all resources Daniel Kral
` (5 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:33 UTC (permalink / raw)
To: pve-devel
This is needed to be able to derive the proper static service stats for
non-HA resources in a following patch as well.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Env/PVE2.pm | 14 ++++++++------
src/PVE/HA/Resources.pm | 6 ++++++
src/PVE/HA/Resources/PVECT.pm | 12 ++++++++----
src/PVE/HA/Resources/PVEVM.pm | 12 ++++++++----
4 files changed, 30 insertions(+), 14 deletions(-)
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 5b8aaae6..94f30d73 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -523,17 +523,19 @@ sub update_static_service_stats {
my $properties = ['cores', 'cpulimit', 'memory', 'sockets', 'vcpus'];
my $service_stats = eval {
- my $stats = PVE::Cluster::get_guest_config_properties($properties);
+ my $stats = {};
+ my $confs = PVE::Cluster::get_guest_config_properties($properties);
- # get_guest_config_properties(...) doesn't add guests which do not
- # specify any of the given properties, but we need to make a distinction
- # between "not cached" and "not specified" here
my $vmlist = PVE::Cluster::get_vmlist();
my $idlist = $vmlist->{ids} // {};
for my $id (keys %$idlist) {
- next if defined($stats->{$id});
+ my $type = eval { PVE::HA::Tools::get_ha_resource_type($idlist->{$id}->{type}) };
+ next if $@; # silently ignore unknown pve types
- $stats->{$id} = {};
+ my $conf = $confs->{$id} // {};
+ my $plugin = PVE::HA::Resources->lookup($type);
+
+ $stats->{$id} = $plugin->get_static_stats_from_config($conf);
}
return $stats;
diff --git a/src/PVE/HA/Resources.pm b/src/PVE/HA/Resources.pm
index 3b20d44c..cba2d4c2 100644
--- a/src/PVE/HA/Resources.pm
+++ b/src/PVE/HA/Resources.pm
@@ -177,6 +177,12 @@ sub remove_locks {
die "implement in subclass";
}
+sub get_static_stats_from_config {
+ my ($class, $conf) = @_;
+
+ die "implement in subclass";
+}
+
sub get_static_stats {
my ($class, $haenv, $id) = @_;
diff --git a/src/PVE/HA/Resources/PVECT.pm b/src/PVE/HA/Resources/PVECT.pm
index 1f4eb2e9..0dc500d1 100644
--- a/src/PVE/HA/Resources/PVECT.pm
+++ b/src/PVE/HA/Resources/PVECT.pm
@@ -159,10 +159,8 @@ sub remove_locks {
return undef;
}
-sub get_static_stats {
- my ($class, $haenv, $id) = @_;
-
- my $conf = $haenv->get_static_service_stats($id);
+sub get_static_stats_from_config {
+ my ($class, $conf) = @_;
return {
maxcpu => PVE::LXC::Config->get_derived_property($conf, 'max-cpu'),
@@ -170,4 +168,10 @@ sub get_static_stats {
};
}
+sub get_static_stats {
+ my ($class, $haenv, $id) = @_;
+
+ return $haenv->get_static_service_stats($id);
+}
+
1;
diff --git a/src/PVE/HA/Resources/PVEVM.pm b/src/PVE/HA/Resources/PVEVM.pm
index 760259e4..579a7fca 100644
--- a/src/PVE/HA/Resources/PVEVM.pm
+++ b/src/PVE/HA/Resources/PVEVM.pm
@@ -180,10 +180,8 @@ sub remove_locks {
return undef;
}
-sub get_static_stats {
- my ($class, $haenv, $id) = @_;
-
- my $conf = $haenv->get_static_service_stats($id);
+sub get_static_stats_from_config {
+ my ($class, $conf) = @_;
return {
maxcpu => PVE::QemuConfig->get_derived_property($conf, 'max-cpu'),
@@ -191,4 +189,10 @@ sub get_static_stats {
};
}
+sub get_static_stats {
+ my ($class, $haenv, $id) = @_;
+
+ return $haenv->get_static_service_stats($id);
+}
+
1;
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH ha-manager v2 06/10] test: make static service usage explicit for all resources
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
` (4 preceding siblings ...)
2026-03-19 16:33 ` [PATCH ha-manager v2 05/10] derive static service stats while filling the service stats repository Daniel Kral
@ 2026-03-19 16:33 ` Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 07/10] make static service stats indexable by sid Daniel Kral
` (4 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:33 UTC (permalink / raw)
To: pve-devel
Even though deriving these from the HA resource sid is convenient, this
is needed to get the proper static service stats for all HA resources at
once.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Sim/Resources.pm | 10 +---
.../static_service_stats | 52 ++++++++++++++++++-
.../static_service_stats | 9 +++-
3 files changed, 60 insertions(+), 11 deletions(-)
diff --git a/src/PVE/HA/Sim/Resources.pm b/src/PVE/HA/Sim/Resources.pm
index 48c15dfe..ad52437d 100644
--- a/src/PVE/HA/Sim/Resources.pm
+++ b/src/PVE/HA/Sim/Resources.pm
@@ -143,15 +143,7 @@ sub get_static_stats {
my $sid = $class->type() . ":$id";
my $hardware = $haenv->hardware();
- if (my $service_stats = $hardware->get_static_service_stats($sid)) {
- return $service_stats;
- } elsif ($id =~ /^(\d)(\d\d)/) {
- # auto assign usage calculated from ID for convenience
- my ($maxcpu, $maxmeory) = (int($1) + 1, (int($2) + 1) * 1 << 29);
- return { maxcpu => $maxcpu, maxmem => $maxmeory };
- } else {
- return {};
- }
+ return $hardware->get_static_service_stats($sid);
}
1;
diff --git a/src/test/test-crs-static-rebalance1/static_service_stats b/src/test/test-crs-static-rebalance1/static_service_stats
index 7fb992dd..666af861 100644
--- a/src/test/test-crs-static-rebalance1/static_service_stats
+++ b/src/test/test-crs-static-rebalance1/static_service_stats
@@ -1,3 +1,53 @@
{
- "vm:102": { "maxcpu": 2, "maxmem": 4000000000 }
+ "vm:101": { "maxcpu": 2, "maxmem": 1073741824 },
+ "vm:102": { "maxcpu": 2, "maxmem": 4000000000 },
+ "vm:103": { "maxcpu": 2, "maxmem": 2147483648 },
+ "vm:104": { "maxcpu": 2, "maxmem": 2684354560 },
+ "vm:105": { "maxcpu": 2, "maxmem": 3221225472 },
+ "vm:106": { "maxcpu": 2, "maxmem": 3758096384 },
+ "vm:107": { "maxcpu": 2, "maxmem": 4294967296 },
+ "vm:108": { "maxcpu": 2, "maxmem": 4831838208 },
+ "vm:109": { "maxcpu": 2, "maxmem": 5368709120 },
+ "vm:110": { "maxcpu": 2, "maxmem": 5905580032 },
+ "vm:111": { "maxcpu": 2, "maxmem": 6442450944 },
+ "vm:112": { "maxcpu": 2, "maxmem": 6979321856 },
+ "vm:113": { "maxcpu": 2, "maxmem": 7516192768 },
+ "vm:114": { "maxcpu": 2, "maxmem": 8053063680 },
+ "vm:115": { "maxcpu": 2, "maxmem": 8589934592 },
+ "vm:116": { "maxcpu": 2, "maxmem": 9126805504 },
+ "vm:117": { "maxcpu": 2, "maxmem": 9663676416 },
+ "vm:118": { "maxcpu": 2, "maxmem": 10200547328 },
+ "vm:119": { "maxcpu": 2, "maxmem": 10737418240 },
+ "vm:120": { "maxcpu": 2, "maxmem": 11274289152 },
+ "vm:121": { "maxcpu": 2, "maxmem": 11811160064 },
+ "vm:122": { "maxcpu": 2, "maxmem": 12348030976 },
+ "vm:123": { "maxcpu": 2, "maxmem": 12884901888 },
+ "vm:124": { "maxcpu": 2, "maxmem": 13421772800 },
+ "vm:125": { "maxcpu": 2, "maxmem": 13958643712 },
+ "vm:126": { "maxcpu": 2, "maxmem": 14495514624 },
+ "vm:127": { "maxcpu": 2, "maxmem": 15032385536 },
+ "vm:128": { "maxcpu": 2, "maxmem": 15569256448 },
+ "vm:129": { "maxcpu": 2, "maxmem": 16106127360 },
+ "vm:130": { "maxcpu": 2, "maxmem": 16642998272 },
+ "vm:131": { "maxcpu": 2, "maxmem": 17179869184 },
+ "vm:132": { "maxcpu": 2, "maxmem": 17716740096 },
+ "vm:133": { "maxcpu": 2, "maxmem": 18253611008 },
+ "vm:134": { "maxcpu": 2, "maxmem": 18790481920 },
+ "vm:135": { "maxcpu": 2, "maxmem": 19327352832 },
+ "vm:136": { "maxcpu": 2, "maxmem": 19864223744 },
+ "vm:137": { "maxcpu": 2, "maxmem": 20401094656 },
+ "vm:138": { "maxcpu": 2, "maxmem": 20937965568 },
+ "vm:139": { "maxcpu": 2, "maxmem": 21474836480 },
+ "vm:140": { "maxcpu": 2, "maxmem": 22011707392 },
+ "vm:141": { "maxcpu": 2, "maxmem": 22548578304 },
+ "vm:142": { "maxcpu": 2, "maxmem": 23085449216 },
+ "vm:143": { "maxcpu": 2, "maxmem": 23622320128 },
+ "vm:144": { "maxcpu": 2, "maxmem": 24159191040 },
+ "vm:145": { "maxcpu": 2, "maxmem": 24696061952 },
+ "vm:146": { "maxcpu": 2, "maxmem": 25232932864 },
+ "vm:147": { "maxcpu": 2, "maxmem": 25769803776 },
+ "vm:148": { "maxcpu": 2, "maxmem": 26306674688 },
+ "vm:149": { "maxcpu": 2, "maxmem": 26843545600 },
+ "vm:150": { "maxcpu": 2, "maxmem": 27380416512 },
+ "vm:151": { "maxcpu": 2, "maxmem": 27917287424 }
}
diff --git a/src/test/test-crs-static-rebalance2/static_service_stats b/src/test/test-crs-static-rebalance2/static_service_stats
index 0967ef42..a4049f80 100644
--- a/src/test/test-crs-static-rebalance2/static_service_stats
+++ b/src/test/test-crs-static-rebalance2/static_service_stats
@@ -1 +1,8 @@
-{}
+{
+ "vm:100": { "maxcpu": 2, "maxmem": 536870912 },
+ "vm:101": { "maxcpu": 2, "maxmem": 1073741824 },
+ "vm:102": { "maxcpu": 2, "maxmem": 1610612736 },
+ "vm:103": { "maxcpu": 2, "maxmem": 2147483648 },
+ "vm:104": { "maxcpu": 2, "maxmem": 2684354560 },
+ "vm:105": { "maxcpu": 2, "maxmem": 3221225472 }
+}
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH ha-manager v2 07/10] make static service stats indexable by sid
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
` (5 preceding siblings ...)
2026-03-19 16:33 ` [PATCH ha-manager v2 06/10] test: make static service usage explicit for all resources Daniel Kral
@ 2026-03-19 16:33 ` Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 08/10] move static service stats repository to PVE::HA::Usage::Static Daniel Kral
` (3 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:33 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Env/PVE2.pm | 3 ++-
src/PVE/HA/Resources.pm | 2 +-
src/PVE/HA/Resources/PVECT.pm | 4 ++--
src/PVE/HA/Resources/PVEVM.pm | 4 ++--
src/PVE/HA/Sim/Resources.pm | 3 +--
src/PVE/HA/Usage/Static.pm | 4 ++--
6 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 94f30d73..54270a40 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -532,10 +532,11 @@ sub update_static_service_stats {
my $type = eval { PVE::HA::Tools::get_ha_resource_type($idlist->{$id}->{type}) };
next if $@; # silently ignore unknown pve types
+ my $sid = "$type:$id";
my $conf = $confs->{$id} // {};
my $plugin = PVE::HA::Resources->lookup($type);
- $stats->{$id} = $plugin->get_static_stats_from_config($conf);
+ $stats->{$sid} = $plugin->get_static_stats_from_config($conf);
}
return $stats;
diff --git a/src/PVE/HA/Resources.pm b/src/PVE/HA/Resources.pm
index cba2d4c2..01facac8 100644
--- a/src/PVE/HA/Resources.pm
+++ b/src/PVE/HA/Resources.pm
@@ -184,7 +184,7 @@ sub get_static_stats_from_config {
}
sub get_static_stats {
- my ($class, $haenv, $id) = @_;
+ my ($class, $haenv, $sid) = @_;
die "implement in subclass";
}
diff --git a/src/PVE/HA/Resources/PVECT.pm b/src/PVE/HA/Resources/PVECT.pm
index 0dc500d1..bda33717 100644
--- a/src/PVE/HA/Resources/PVECT.pm
+++ b/src/PVE/HA/Resources/PVECT.pm
@@ -169,9 +169,9 @@ sub get_static_stats_from_config {
}
sub get_static_stats {
- my ($class, $haenv, $id) = @_;
+ my ($class, $haenv, $sid) = @_;
- return $haenv->get_static_service_stats($id);
+ return $haenv->get_static_service_stats($sid);
}
1;
diff --git a/src/PVE/HA/Resources/PVEVM.pm b/src/PVE/HA/Resources/PVEVM.pm
index 579a7fca..786c5130 100644
--- a/src/PVE/HA/Resources/PVEVM.pm
+++ b/src/PVE/HA/Resources/PVEVM.pm
@@ -190,9 +190,9 @@ sub get_static_stats_from_config {
}
sub get_static_stats {
- my ($class, $haenv, $id) = @_;
+ my ($class, $haenv, $sid) = @_;
- return $haenv->get_static_service_stats($id);
+ return $haenv->get_static_service_stats($sid);
}
1;
diff --git a/src/PVE/HA/Sim/Resources.pm b/src/PVE/HA/Sim/Resources.pm
index ad52437d..ed7d9c5a 100644
--- a/src/PVE/HA/Sim/Resources.pm
+++ b/src/PVE/HA/Sim/Resources.pm
@@ -138,9 +138,8 @@ sub remove_locks {
}
sub get_static_stats {
- my ($class, $haenv, $id) = @_;
+ my ($class, $haenv, $sid) = @_;
- my $sid = $class->type() . ":$id";
my $hardware = $haenv->hardware();
return $hardware->get_static_service_stats($sid);
diff --git a/src/PVE/HA/Usage/Static.pm b/src/PVE/HA/Usage/Static.pm
index 2304139c..acc3533c 100644
--- a/src/PVE/HA/Usage/Static.pm
+++ b/src/PVE/HA/Usage/Static.pm
@@ -65,10 +65,10 @@ my sub get_service_usage {
return $self->{'service-stats'}->{$sid} if $self->{'service-stats'}->{$sid};
- my (undef, $type, $id) = $self->{haenv}->parse_sid($sid);
+ my (undef, $type, undef) = $self->{haenv}->parse_sid($sid);
my $plugin = PVE::HA::Resources->lookup($type);
- my $stats = eval { $plugin->get_static_stats($self->{haenv}, $id) };
+ my $stats = eval { $plugin->get_static_stats($self->{haenv}, $sid) };
die "did not get static service usage information for '$sid'\n" if !$stats;
my $service_stats = {
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH ha-manager v2 08/10] move static service stats repository to PVE::HA::Usage::Static
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
` (6 preceding siblings ...)
2026-03-19 16:33 ` [PATCH ha-manager v2 07/10] make static service stats indexable by sid Daniel Kral
@ 2026-03-19 16:33 ` Daniel Kral
2026-03-19 16:33 ` [PATCH ha-manager v2 09/10] usage: augment service stats with node and state information Daniel Kral
` (2 subsequent siblings)
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:33 UTC (permalink / raw)
To: pve-devel
Since the static service stats are already processed by their individual
plugin's implementation of get_static_stats_from_config(...), the static
service stats repository can be used by the static load scheduler
directly.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Env.pm | 8 +------
src/PVE/HA/Env/PVE2.pm | 42 +++++++++++------------------------
src/PVE/HA/Manager.pm | 1 -
src/PVE/HA/Resources.pm | 6 -----
src/PVE/HA/Resources/PVECT.pm | 6 -----
src/PVE/HA/Resources/PVEVM.pm | 6 -----
src/PVE/HA/Sim/Env.pm | 8 +------
src/PVE/HA/Sim/Hardware.pm | 13 +----------
src/PVE/HA/Sim/Resources.pm | 8 -------
src/PVE/HA/Usage/Static.pm | 23 +++++--------------
10 files changed, 22 insertions(+), 99 deletions(-)
diff --git a/src/PVE/HA/Env.pm b/src/PVE/HA/Env.pm
index 39ed68cb..76216893 100644
--- a/src/PVE/HA/Env.pm
+++ b/src/PVE/HA/Env.pm
@@ -301,15 +301,9 @@ sub get_datacenter_settings {
}
sub get_static_service_stats {
- my ($self, $id) = @_;
-
- return $self->{plug}->get_static_service_stats($id);
-}
-
-sub update_static_service_stats {
my ($self) = @_;
- return $self->{plug}->update_static_service_stats();
+ return $self->{plug}->get_static_service_stats();
}
sub get_static_node_stats {
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 54270a40..9c64d955 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -49,8 +49,6 @@ sub new {
$self->{nodename} = $nodename;
- $self->{static_service_stats} = undef;
-
return $self;
}
@@ -510,40 +508,26 @@ sub get_datacenter_settings {
}
sub get_static_service_stats {
- my ($self, $id) = @_;
-
- # undef if update_static_service_stats(...) failed before
- return undef if !defined($self->{static_service_stats});
-
- return $self->{static_service_stats}->{$id};
-}
-
-sub update_static_service_stats {
my ($self) = @_;
my $properties = ['cores', 'cpulimit', 'memory', 'sockets', 'vcpus'];
- my $service_stats = eval {
- my $stats = {};
- my $confs = PVE::Cluster::get_guest_config_properties($properties);
+ my $stats = {};
+ my $confs = PVE::Cluster::get_guest_config_properties($properties);
- my $vmlist = PVE::Cluster::get_vmlist();
- my $idlist = $vmlist->{ids} // {};
- for my $id (keys %$idlist) {
- my $type = eval { PVE::HA::Tools::get_ha_resource_type($idlist->{$id}->{type}) };
- next if $@; # silently ignore unknown pve types
+ my $vmlist = PVE::Cluster::get_vmlist();
+ my $idlist = $vmlist->{ids} // {};
+ for my $id (keys %$idlist) {
+ my $type = eval { PVE::HA::Tools::get_ha_resource_type($idlist->{$id}->{type}) };
+ next if $@; # silently ignore unknown pve types
- my $sid = "$type:$id";
- my $conf = $confs->{$id} // {};
- my $plugin = PVE::HA::Resources->lookup($type);
+ my $sid = "$type:$id";
+ my $conf = $confs->{$id} // {};
+ my $plugin = PVE::HA::Resources->lookup($type);
- $stats->{$sid} = $plugin->get_static_stats_from_config($conf);
- }
+ $stats->{$sid} = $plugin->get_static_stats_from_config($conf);
+ }
- return $stats;
- };
- $self->log('warning', "unable to update static service stats cache - $@") if $@;
-
- $self->{static_service_stats} = $service_stats;
+ return $stats;
}
sub get_static_node_stats {
diff --git a/src/PVE/HA/Manager.pm b/src/PVE/HA/Manager.pm
index b1dbe6a8..90c5be50 100644
--- a/src/PVE/HA/Manager.pm
+++ b/src/PVE/HA/Manager.pm
@@ -253,7 +253,6 @@ sub recompute_online_node_usage {
$online_node_usage = eval {
my $scheduler = PVE::HA::Usage::Static->new($haenv);
$scheduler->add_node($_) for $online_nodes->@*;
- $haenv->update_static_service_stats();
return $scheduler;
};
} else {
diff --git a/src/PVE/HA/Resources.pm b/src/PVE/HA/Resources.pm
index 01facac8..4238d9be 100644
--- a/src/PVE/HA/Resources.pm
+++ b/src/PVE/HA/Resources.pm
@@ -183,12 +183,6 @@ sub get_static_stats_from_config {
die "implement in subclass";
}
-sub get_static_stats {
- my ($class, $haenv, $sid) = @_;
-
- die "implement in subclass";
-}
-
# package PVE::HA::Resources::IPAddr;
# use strict;
diff --git a/src/PVE/HA/Resources/PVECT.pm b/src/PVE/HA/Resources/PVECT.pm
index bda33717..79bb7c83 100644
--- a/src/PVE/HA/Resources/PVECT.pm
+++ b/src/PVE/HA/Resources/PVECT.pm
@@ -168,10 +168,4 @@ sub get_static_stats_from_config {
};
}
-sub get_static_stats {
- my ($class, $haenv, $sid) = @_;
-
- return $haenv->get_static_service_stats($sid);
-}
-
1;
diff --git a/src/PVE/HA/Resources/PVEVM.pm b/src/PVE/HA/Resources/PVEVM.pm
index 786c5130..5a0ac348 100644
--- a/src/PVE/HA/Resources/PVEVM.pm
+++ b/src/PVE/HA/Resources/PVEVM.pm
@@ -189,10 +189,4 @@ sub get_static_stats_from_config {
};
}
-sub get_static_stats {
- my ($class, $haenv, $sid) = @_;
-
- return $haenv->get_static_service_stats($sid);
-}
-
1;
diff --git a/src/PVE/HA/Sim/Env.pm b/src/PVE/HA/Sim/Env.pm
index 32b5224c..0e3b02e5 100644
--- a/src/PVE/HA/Sim/Env.pm
+++ b/src/PVE/HA/Sim/Env.pm
@@ -489,15 +489,9 @@ sub get_datacenter_settings {
}
sub get_static_service_stats {
- my ($self, $id) = @_;
-
- return $self->{hardware}->get_static_service_stats($id);
-}
-
-sub update_static_service_stats {
my ($self) = @_;
- return $self->{hardware}->update_static_service_stats();
+ return $self->{hardware}->get_static_service_stats();
}
sub get_static_node_stats {
diff --git a/src/PVE/HA/Sim/Hardware.pm b/src/PVE/HA/Sim/Hardware.pm
index 60eb9867..062d9846 100644
--- a/src/PVE/HA/Sim/Hardware.pm
+++ b/src/PVE/HA/Sim/Hardware.pm
@@ -525,8 +525,6 @@ sub new {
$self->{service_config} = $self->read_service_config();
- $self->{static_service_stats} = undef;
-
return $self;
}
@@ -1063,22 +1061,13 @@ sub watchdog_update {
}
sub get_static_service_stats {
- my ($self, $id) = @_;
-
- # undef if update_static_service_stats(...) failed before
- return undef if !defined($self->{static_service_stats});
-
- return $self->{static_service_stats}->{$id};
-}
-
-sub update_static_service_stats {
my ($self) = @_;
my $filename = "$self->{statusdir}/static_service_stats";
my $stats = eval { PVE::HA::Tools::read_json_from_file($filename) };
$self->log('warning', "unable to update static service stats cache - $@") if $@;
- $self->{static_service_stats} = $stats;
+ return $stats;
}
sub get_static_node_stats {
diff --git a/src/PVE/HA/Sim/Resources.pm b/src/PVE/HA/Sim/Resources.pm
index ed7d9c5a..9b2f3b60 100644
--- a/src/PVE/HA/Sim/Resources.pm
+++ b/src/PVE/HA/Sim/Resources.pm
@@ -137,12 +137,4 @@ sub remove_locks {
return undef;
}
-sub get_static_stats {
- my ($class, $haenv, $sid) = @_;
-
- my $hardware = $haenv->hardware();
-
- return $hardware->get_static_service_stats($sid);
-}
-
1;
diff --git a/src/PVE/HA/Usage/Static.pm b/src/PVE/HA/Usage/Static.pm
index acc3533c..48622d62 100644
--- a/src/PVE/HA/Usage/Static.pm
+++ b/src/PVE/HA/Usage/Static.pm
@@ -14,12 +14,15 @@ sub new {
my $node_stats = eval { $haenv->get_static_node_stats() };
die "did not get static node usage information - $@" if $@;
+ my $service_stats = eval { $haenv->get_static_service_stats() };
+ die "did not get static service usage information - $@" if $@;
+
my $scheduler = eval { PVE::RS::ResourceScheduling::Static->new(); };
die "unable to initialize static scheduling - $@" if $@;
return bless {
'node-stats' => $node_stats,
- 'service-stats' => {},
+ 'service-stats' => $service_stats,
haenv => $haenv,
scheduler => $scheduler,
'node-services' => {}, # Services on each node. Fallback if scoring calculation fails.
@@ -63,20 +66,8 @@ sub contains_node {
my sub get_service_usage {
my ($self, $sid) = @_;
- return $self->{'service-stats'}->{$sid} if $self->{'service-stats'}->{$sid};
-
- my (undef, $type, undef) = $self->{haenv}->parse_sid($sid);
- my $plugin = PVE::HA::Resources->lookup($type);
-
- my $stats = eval { $plugin->get_static_stats($self->{haenv}, $sid) };
- die "did not get static service usage information for '$sid'\n" if !$stats;
-
- my $service_stats = {
- maxcpu => $stats->{maxcpu} + 0.0, # containers allow non-integer cpulimit
- maxmem => int($stats->{maxmem}),
- };
-
- $self->{'service-stats'}->{$sid} = $service_stats;
+ my $service_stats = $self->{'service-stats'}->{$sid}
+ or die "did not get static service usage information for '$sid'\n";
return $service_stats;
}
@@ -101,8 +92,6 @@ sub remove_service_usage {
eval { $self->{scheduler}->remove_service_usage($sid) };
$self->{haenv}->log('warning', "unable to remove service '$sid' usage - $@") if $@;
-
- delete $self->{'service-stats'}->{$sid}; # Invalidate old service stats
}
sub score_nodes_to_start_service {
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH ha-manager v2 09/10] usage: augment service stats with node and state information
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
` (7 preceding siblings ...)
2026-03-19 16:33 ` [PATCH ha-manager v2 08/10] move static service stats repository to PVE::HA::Usage::Static Daniel Kral
@ 2026-03-19 16:33 ` Daniel Kral
2026-03-19 16:34 ` [PATCH ha-manager v2 10/10] include running non-HA resources in the scheduler's accounting Daniel Kral
2026-03-19 16:56 ` [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:33 UTC (permalink / raw)
To: pve-devel
Augment the service stats with the node and state information, which is
necessary for non-HA resources to be added to the accounting done by the
implementations of PVE::HA::Usage in an upcoming patch.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Env/PVE2.pm | 49 +++++++++++++++++++++++++++++++-------
src/PVE/HA/Sim/Hardware.pm | 27 ++++++++++++++++++++-
src/PVE/HA/Usage/Static.pm | 2 +-
3 files changed, 68 insertions(+), 10 deletions(-)
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 9c64d955..b544de4c 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -38,6 +38,12 @@ PVE::HA::Rules->init(property_isolation => 1);
my $lockdir = "/etc/pve/priv/lock";
+# rrd entry indices for VM and CT guests
+# taken from PVE::Service::pvestatd::update_{lxc,qemu}_status()
+use constant {
+ RRD_VM_INDEX_STATUS => 2,
+};
+
sub new {
my ($this, $nodename) = @_;
@@ -507,24 +513,51 @@ sub get_datacenter_settings {
};
}
-sub get_static_service_stats {
- my ($self) = @_;
-
- my $properties = ['cores', 'cpulimit', 'memory', 'sockets', 'vcpus'];
- my $stats = {};
- my $confs = PVE::Cluster::get_guest_config_properties($properties);
-
+my sub get_cluster_service_stats {
my $vmlist = PVE::Cluster::get_vmlist();
my $idlist = $vmlist->{ids} // {};
+
+ my $rrd = PVE::Cluster::rrd_dump();
+
+ my $stats = {};
for my $id (keys %$idlist) {
my $type = eval { PVE::HA::Tools::get_ha_resource_type($idlist->{$id}->{type}) };
next if $@; # silently ignore unknown pve types
my $sid = "$type:$id";
+ my $nodename = $idlist->{$id}->{node};
+
+ my $rrdentry = $rrd->{"pve-vm-9.0/$id"} // [];
+ # can be any QMP RunState, but 'running' is the only active VM state
+ my $status = $rrdentry->[RRD_VM_INDEX_STATUS] // "stopped";
+ my $state = $status eq "running" ? "started" : "stopped";
+
+ $stats->{$sid} = {
+ id => $id,
+ node => $nodename,
+ state => $state,
+ type => $type,
+ usage => {},
+ };
+ }
+
+ return $stats;
+}
+
+sub get_static_service_stats {
+ my ($self) = @_;
+
+ my $properties = ['cores', 'cpulimit', 'memory', 'sockets', 'vcpus'];
+ my $stats = get_cluster_service_stats();
+ my $confs = PVE::Cluster::get_guest_config_properties($properties);
+
+ for my $sid (keys %$stats) {
+ my ($id, $type) = $stats->{$sid}->@{qw(id type)};
+
my $conf = $confs->{$id} // {};
my $plugin = PVE::HA::Resources->lookup($type);
- $stats->{$sid} = $plugin->get_static_stats_from_config($conf);
+ $stats->{$sid}->{usage} = $plugin->get_static_stats_from_config($conf);
}
return $stats;
diff --git a/src/PVE/HA/Sim/Hardware.pm b/src/PVE/HA/Sim/Hardware.pm
index 062d9846..dd7d277c 100644
--- a/src/PVE/HA/Sim/Hardware.pm
+++ b/src/PVE/HA/Sim/Hardware.pm
@@ -1060,13 +1060,38 @@ sub watchdog_update {
return &$modify_watchog($self, $code);
}
+my sub get_cluster_service_stats {
+ my ($self) = @_;
+
+ my $stats = {};
+ for my $sid (keys $self->{service_config}->%*) {
+ my $cfg = $self->{service_config}->{$sid};
+
+ $stats->{$sid} = {
+ node => $cfg->{node},
+ state => $cfg->{state},
+ usage => {},
+ };
+ }
+
+ return $stats;
+}
+
sub get_static_service_stats {
my ($self) = @_;
+ my $stats = get_cluster_service_stats($self);
+
my $filename = "$self->{statusdir}/static_service_stats";
- my $stats = eval { PVE::HA::Tools::read_json_from_file($filename) };
+ my $usage_stats = eval { PVE::HA::Tools::read_json_from_file($filename) };
$self->log('warning', "unable to update static service stats cache - $@") if $@;
+ for my $sid (keys %$stats) {
+ next if !defined($usage_stats->{$sid});
+
+ $stats->{$sid}->{usage} = $usage_stats->{$sid};
+ }
+
return $stats;
}
diff --git a/src/PVE/HA/Usage/Static.pm b/src/PVE/HA/Usage/Static.pm
index 48622d62..98752691 100644
--- a/src/PVE/HA/Usage/Static.pm
+++ b/src/PVE/HA/Usage/Static.pm
@@ -66,7 +66,7 @@ sub contains_node {
my sub get_service_usage {
my ($self, $sid) = @_;
- my $service_stats = $self->{'service-stats'}->{$sid}
+ my $service_stats = $self->{'service-stats'}->{$sid}->{usage}
or die "did not get static service usage information for '$sid'\n";
return $service_stats;
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* [PATCH ha-manager v2 10/10] include running non-HA resources in the scheduler's accounting
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
` (8 preceding siblings ...)
2026-03-19 16:33 ` [PATCH ha-manager v2 09/10] usage: augment service stats with node and state information Daniel Kral
@ 2026-03-19 16:34 ` Daniel Kral
2026-03-19 16:56 ` [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:34 UTC (permalink / raw)
To: pve-devel
As the service stats repository includes non-HA resources as well, use
it to add the static usage stats of these running, non-HA resources to
the scheduler.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
src/PVE/HA/Env.pm | 6 ++++++
src/PVE/HA/Env/PVE2.pm | 6 ++++++
src/PVE/HA/Manager.pm | 15 ++++++++++++++-
src/PVE/HA/Sim/Env.pm | 6 ++++++
src/PVE/HA/Sim/Hardware.pm | 6 ++++++
src/PVE/HA/Usage.pm | 2 +-
src/PVE/HA/Usage/Basic.pm | 2 +-
src/PVE/HA/Usage/Static.pm | 5 +----
8 files changed, 41 insertions(+), 7 deletions(-)
diff --git a/src/PVE/HA/Env.pm b/src/PVE/HA/Env.pm
index 76216893..3643292e 100644
--- a/src/PVE/HA/Env.pm
+++ b/src/PVE/HA/Env.pm
@@ -300,6 +300,12 @@ sub get_datacenter_settings {
return $self->{plug}->get_datacenter_settings();
}
+sub get_basic_service_stats {
+ my ($self) = @_;
+
+ return $self->{plug}->get_basic_service_stats();
+}
+
sub get_static_service_stats {
my ($self) = @_;
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index b544de4c..04cd1bfe 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -544,6 +544,12 @@ my sub get_cluster_service_stats {
return $stats;
}
+sub get_basic_service_stats {
+ my ($self) = @_;
+
+ return get_cluster_service_stats();
+}
+
sub get_static_service_stats {
my ($self) = @_;
diff --git a/src/PVE/HA/Manager.pm b/src/PVE/HA/Manager.pm
index 90c5be50..421c17da 100644
--- a/src/PVE/HA/Manager.pm
+++ b/src/PVE/HA/Manager.pm
@@ -245,13 +245,15 @@ sub recompute_online_node_usage {
my $online_nodes = $self->{ns}->list_online_nodes();
+ my $service_stats;
my $online_node_usage;
if (my $mode = $self->{crs}->{scheduler}) {
if ($mode eq 'static') {
if ($have_static_scheduling) {
$online_node_usage = eval {
- my $scheduler = PVE::HA::Usage::Static->new($haenv);
+ $service_stats = $haenv->get_static_service_stats();
+ my $scheduler = PVE::HA::Usage::Static->new($haenv, $service_stats);
$scheduler->add_node($_) for $online_nodes->@*;
return $scheduler;
};
@@ -271,6 +273,7 @@ sub recompute_online_node_usage {
# fallback to the basic algorithm in any case
if (!$online_node_usage) {
+ $service_stats = $haenv->get_basic_service_stats();
$online_node_usage = PVE::HA::Usage::Basic->new($haenv);
$online_node_usage->add_node($_) for $online_nodes->@*;
}
@@ -281,6 +284,16 @@ sub recompute_online_node_usage {
$online_node_usage->add_service_usage($sid, $sd->{state}, $sd->{node}, $sd->{target});
}
+ # add remaining non-HA resources to online node usage
+ for my $sid (sort keys %$service_stats) {
+ next if $self->{ss}->{$sid};
+
+ my ($node, $state) = $service_stats->{$sid}->@{qw(node state)};
+
+ # the migration target is not known for non-HA resources
+ $online_node_usage->add_service_usage($sid, $state, $node, undef);
+ }
+
$self->{online_node_usage} = $online_node_usage;
}
diff --git a/src/PVE/HA/Sim/Env.pm b/src/PVE/HA/Sim/Env.pm
index 0e3b02e5..ad51245c 100644
--- a/src/PVE/HA/Sim/Env.pm
+++ b/src/PVE/HA/Sim/Env.pm
@@ -488,6 +488,12 @@ sub get_datacenter_settings {
};
}
+sub get_basic_service_stats {
+ my ($self) = @_;
+
+ return $self->{hardware}->get_basic_service_stats();
+}
+
sub get_static_service_stats {
my ($self) = @_;
diff --git a/src/PVE/HA/Sim/Hardware.pm b/src/PVE/HA/Sim/Hardware.pm
index dd7d277c..59afb44a 100644
--- a/src/PVE/HA/Sim/Hardware.pm
+++ b/src/PVE/HA/Sim/Hardware.pm
@@ -1077,6 +1077,12 @@ my sub get_cluster_service_stats {
return $stats;
}
+sub get_basic_service_stats {
+ my ($self) = @_;
+
+ return get_cluster_service_stats($self);
+}
+
sub get_static_service_stats {
my ($self) = @_;
diff --git a/src/PVE/HA/Usage.pm b/src/PVE/HA/Usage.pm
index 1be5fa09..9f19a82b 100644
--- a/src/PVE/HA/Usage.pm
+++ b/src/PVE/HA/Usage.pm
@@ -4,7 +4,7 @@ use strict;
use warnings;
sub new {
- my ($class, $haenv) = @_;
+ my ($class, $haenv, $service_stats) = @_;
die "implement in subclass";
}
diff --git a/src/PVE/HA/Usage/Basic.pm b/src/PVE/HA/Usage/Basic.pm
index ef9ae3d6..2584727b 100644
--- a/src/PVE/HA/Usage/Basic.pm
+++ b/src/PVE/HA/Usage/Basic.pm
@@ -6,7 +6,7 @@ use warnings;
use base qw(PVE::HA::Usage);
sub new {
- my ($class, $haenv) = @_;
+ my ($class, $haenv, $service_stats) = @_;
return bless {
nodes => {},
diff --git a/src/PVE/HA/Usage/Static.pm b/src/PVE/HA/Usage/Static.pm
index 98752691..c8460fd7 100644
--- a/src/PVE/HA/Usage/Static.pm
+++ b/src/PVE/HA/Usage/Static.pm
@@ -9,14 +9,11 @@ use PVE::RS::ResourceScheduling::Static;
use base qw(PVE::HA::Usage);
sub new {
- my ($class, $haenv) = @_;
+ my ($class, $haenv, $service_stats) = @_;
my $node_stats = eval { $haenv->get_static_node_stats() };
die "did not get static node usage information - $@" if $@;
- my $service_stats = eval { $haenv->get_static_service_stats() };
- die "did not get static service usage information - $@" if $@;
-
my $scheduler = eval { PVE::RS::ResourceScheduling::Static->new(); };
die "unable to initialize static scheduling - $@" if $@;
--
2.47.3
^ permalink raw reply [flat|nested] 12+ messages in thread* Re: [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting
2026-03-19 16:33 [PATCH-SERIES ha-manager v2 00/10] include non-HA resources in usage accounting Daniel Kral
` (9 preceding siblings ...)
2026-03-19 16:34 ` [PATCH ha-manager v2 10/10] include running non-HA resources in the scheduler's accounting Daniel Kral
@ 2026-03-19 16:56 ` Daniel Kral
10 siblings, 0 replies; 12+ messages in thread
From: Daniel Kral @ 2026-03-19 16:56 UTC (permalink / raw)
To: Daniel Kral, pve-devel
On Thu Mar 19, 2026 at 5:33 PM CET, Daniel Kral wrote:
> This series is split out from [0] to make the static/dynamic load
> balancer series a little more straight forward to reason through.
>
> This series adds any running non-HA resource into the existing basic and
> static usage scheduler accounting to base decisions on the usage of the
> whole cluster instead of just the HA resources.
>
> For clusters where every guest is configured as a HA resource nothing
> should change.
>
> I tested a few simple cases in a 3 node cluster for both the basic and
> static usage which are variations of:
>
> - 1 running HA resource on each node, starting them on any node (with
> homogeneous nodes and guests) will rebalance them on the
> alphabetically first node as before
> - 1 running HA resource on node1, running 3 non-HA resources on node2,
> starting another HA resource on node1 will rebalance it to node3
> - 3 non-HA resources on node1, starting a HA resource on node1 will
> rebalance it to node2
> - non-running HA and non-HA resources are excluded from the usage
> accounting
> - starting non-HA resources doesn't have any effect obviously
>
Sorry for the noise, there are only two changes in the series in patch
#9:
- removing the duplicate $sid as suggested by @Thomas, and
- adding constants for indexing the rrd status indirectly suggested by
@Thomas in a later patch in the series.
Otherwise the series is identical to the patches from [0].
> [0] https://lore.proxmox.com/pve-devel/20260217141437.584852-1-d.kral@proxmox.com/
^ permalink raw reply [flat|nested] 12+ messages in thread