all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [PATCH pve-ha-manager 1/2] fix #7974: run make tidy
@ 2026-08-28 12:32 Dominik Rusovac
  2026-08-28 12:32 ` [PATCH pve-ha-manager 2/2] fix #7974: env: pve2: set dynamic service mem value to host memory usage Dominik Rusovac
  0 siblings, 1 reply; 3+ messages in thread
From: Dominik Rusovac @ 2026-08-28 12:32 UTC (permalink / raw)
  To: pve-devel

Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---
 src/PVE/HA/Manager.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/PVE/HA/Manager.pm b/src/PVE/HA/Manager.pm
index 9a4f70c..5840a76 100644
--- a/src/PVE/HA/Manager.pm
+++ b/src/PVE/HA/Manager.pm
@@ -146,7 +146,7 @@ sub get_active_stationary_movable_resource_bundles {
     my ($ss, $sc, $resource_affinity) = @_;
 
     my $resource_bundles = {};
-OUTER: for my $sid (sort keys %$ss) {
+    OUTER: for my $sid (sort keys %$ss) {
         # do not consider non-started resource as 'active' leading resource
         next if $ss->{$sid}->{state} ne 'started';
         # do not consider resource if it is not movable
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 3+ messages in thread

* [PATCH pve-ha-manager 2/2] fix #7974: env: pve2: set dynamic service mem value to host memory usage
  2026-08-28 12:32 [PATCH pve-ha-manager 1/2] fix #7974: run make tidy Dominik Rusovac
@ 2026-08-28 12:32 ` Dominik Rusovac
  2026-09-20 15:24   ` Michael Ryom
  0 siblings, 1 reply; 3+ messages in thread
From: Dominik Rusovac @ 2026-08-28 12:32 UTC (permalink / raw)
  To: pve-devel

Make memory accounting in dynamic CRS consistent.

Atm the imbalance value that triggers rebalancing is derived from the
guests' host memory usage, which corresponds to the respective host-side
cgroup footprints of the guests. However, the scheduler's decision
procedure predicts the changes in imbalance caused by migration
candidates based on memory usage (inside the guests). This discrepancy
can lead to oscillating migrations, as described in detail in [0].

For VMs, host memory usage is the relevant stat, as it better describes
the actual cost of running on a node than memory usage. Host memory
usage is not available for containers, so fall back to memory usage for
containers.

[0] https://bugzilla.proxmox.com/show_bug.cgi?id=7974

Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---
 src/PVE/HA/Env/PVE2.pm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 782d19d..5f195da 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -46,6 +46,7 @@ use constant {
     RRD_VM_INDEX_CPU => 6,
     RRD_VM_INDEX_MAXMEM => 7,
     RRD_VM_INDEX_MEM => 8,
+    RRD_VM_INDEX_MEMHOST => 15,
 };
 
 # rrd entry indices for PVE nodes
@@ -598,11 +599,14 @@ sub get_dynamic_service_stats {
         # NOTE the guests' broadcasted vmstatus() caps maxcpu at the node's maxcpu
         my $maxcpu = ($rrdentry->[RRD_VM_INDEX_MAXCPU] || 0.0) + 0.0;
 
+        # designated fallback for containers
+        my $mem = $rrdentry->[RRD_VM_INDEX_MEMHOST] // $rrdentry->[RRD_VM_INDEX_MEM];
+
         $stats->{$sid}->{usage} = {
             maxcpu => $maxcpu,
             cpu => (($rrdentry->[RRD_VM_INDEX_CPU] || 0.0) + 0.0) * $maxcpu,
             maxmem => int($rrdentry->[RRD_VM_INDEX_MAXMEM] || 0),
-            mem => int($rrdentry->[RRD_VM_INDEX_MEM] || 0),
+            mem => int($mem || 0),
         };
     }
 
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 3+ messages in thread

* Re: [PATCH pve-ha-manager 2/2] fix #7974: env: pve2: set dynamic service mem value to host memory usage
  2026-08-28 12:32 ` [PATCH pve-ha-manager 2/2] fix #7974: env: pve2: set dynamic service mem value to host memory usage Dominik Rusovac
@ 2026-09-20 15:24   ` Michael Ryom
  0 siblings, 0 replies; 3+ messages in thread
From: Michael Ryom @ 2026-09-20 15:24 UTC (permalink / raw)
  To: pve-devel; +Cc: Michael Ryom

On 28/08/2026 14:32, Dominik Rusovac wrote:
> Make memory accounting in dynamic CRS consistent.
>
> Atm the imbalance value that triggers rebalancing is derived from the
> guests' host memory usage, which corresponds to the respective host-side
> cgroup footprints of the guests. However, the scheduler's decision
> procedure predicts the changes in imbalance caused by migration
> candidates based on memory usage (inside the guests). This discrepancy
> can lead to oscillating migrations, as described in detail in [0].
>
> For VMs, host memory usage is the relevant stat, as it better describes
> the actual cost of running on a node than memory usage. Host memory
> usage is not available for containers, so fall back to memory usage for
> containers.

Ran into the same issue independently while deliberately stress-testing
CRS with ha=dynamic on a two-node PVE 9.2.20 cluster (pve-ha-manager
5.2.5): a single dominant VM with an active balloon driver was accounted
with mem = 6.94 GiB while its host-side footprint was ~8.15 GiB (QEMU
RSS 7,950,016 KiB). The ~1.2 GiB difference is the same order of
magnitude as the balancer's trigger condition on a small cluster, and
contributed to the VM being migrated back and forth - five moves in 27
minutes on an otherwise idle cluster, matching the oscillation described
in the bug report.

Tested an equivalent change (memhost with fallback to mem) on the same
cluster: the predicted post-migration imbalance now matches the measured
state, where it was systematically off by ~1-2 percentage points before.

Tested-by: Michael Ryom <Michael@RyomHerold.dk>

One note: with the memory accounting fixed, the oscillation shrinks but
does not fully disappear here. With a dominant resource on two nodes,
sub-percent fluctuations of the node base loads (ZFS ARC, Ceph daemons,
the IO of a rebalance migration itself) are still enough to qualify a
motion - and, once carried out, its reverse - since the relative margin
provides almost no hysteresis in that regime. I am about to send a
series addressing that (plus the endless retry of failing rebalance
migrations); one patch in it overlaps with this one, which I will note
there so whichever lands first wins.



^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2026-09-20 15:26 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-08-28 12:32 [PATCH pve-ha-manager 1/2] fix #7974: run make tidy Dominik Rusovac
2026-08-28 12:32 ` [PATCH pve-ha-manager 2/2] fix #7974: env: pve2: set dynamic service mem value to host memory usage Dominik Rusovac
2026-09-20 15:24   ` Michael Ryom

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal