From: "Dominik Rusovac" <d.rusovac@proxmox.com>
To: "Michael Ryom" <Michael@RyomHerold.dk>, <pve-devel@lists.proxmox.com>
Subject: Re: [PATCH ha-manager 2/7] env: dynamic service stats: use host-side memory footprint of guests
Date: Mon, 21 Sep 2026 11:18:09 +0200 [thread overview]
Message-ID: <DLKVRJQZH2XZ.1POHFGT09YFXT@proxmox.com> (raw)
In-Reply-To: <20260920162220.574802-3-Michael@RyomHerold.dk>
thank you for sending this patch!
one nit inline, otherwise lgtm
NOTE: as mentioned in the cover letter, this overlaps with
https://lore.proxmox.com/all/20260828123239.1110595-2-d.rusovac@proxmox.com/
Reviewed-by: Dominik Rusovac <d.rusovac@proxmox.com>
On Sun Sep 20, 2026 at 6:22 PM CEST, Michael Ryom wrote:
> The dynamic service stats used the 'mem' RRD column of the guests, which
> is the guest-reported memory usage (total_mem - free_mem) whenever the
> balloon driver is active, and can be considerably lower than the actual
> host-side footprint of the guest process (QEMU overhead, page cache
> still mapped, etc.). The node stats, on the other hand, always contain
> the full host view.
>
> The scheduler simulates a migration by subtracting exactly the
> service's stats from the source node and adding them to the target
> node. With the guest-reported value, the predicted post-migration state
> is systematically off by the difference between guest-reported and
> host-side usage (over 1 GiB for a mid-size VM). This bias is of the
> same order of magnitude as the trigger condition of the automatic load
> balancer on a two-node cluster with a single dominant resource, and
> contributes to a resource being moved back and forth.
>
> Use the 'memhost' column (host-side cgroup usage, without KSM) instead
> and fall back to 'mem' if it is unset: containers do not broadcast
> 'memhost', because their 'mem' already is the host-side cgroup value.
>
> Signed-off-by: Michael Ryom <Michael@RyomHerold.dk>
> ---
> src/PVE/HA/Env/PVE2.pm | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
> index 782d19d..8c2b03d 100644
> --- a/src/PVE/HA/Env/PVE2.pm
> +++ b/src/PVE/HA/Env/PVE2.pm
> @@ -46,6 +46,7 @@ use constant {
> RRD_VM_INDEX_CPU => 6,
> RRD_VM_INDEX_MAXMEM => 7,
> RRD_VM_INDEX_MEM => 8,
> + RRD_VM_INDEX_MEMHOST => 15,
> };
>
> # rrd entry indices for PVE nodes
> @@ -598,11 +599,22 @@ sub get_dynamic_service_stats {
> # NOTE the guests' broadcasted vmstatus() caps maxcpu at the node's maxcpu
> my $maxcpu = ($rrdentry->[RRD_VM_INDEX_MAXCPU] || 0.0) + 0.0;
>
> + # prefer the host-side memory footprint over 'mem', which is the
> + # guest-reported usage (total_mem - free_mem) when the balloon driver
> + # is active and can be considerably lower than what running the guest
> + # actually consumes on the node; the node stats contain the full host
> + # view, so mixing in the guest view makes the scheduler's prediction
> + # of the post-migration state systematically off
> + #
> + # 'memhost' is only broadcasted for VMs, for CTs 'mem' already is the
> + # host-side (cgroup) usage
> + my $mem = int($rrdentry->[RRD_VM_INDEX_MEMHOST] || $rrdentry->[RRD_VM_INDEX_MEM] || 0);
nit: while this kind of declaration shouldn't cause problems, since it
can be assumed that
$rrdentry->[RRD_VM_INDEX_MEMHOST] >= $rrdentry->[RRD_VM_INDEX_MEM],
technically, we aim to fall back to 'mem' only if 'memhost' is undef and
not if 'memhost' is 0.
[snip]
next prev parent reply other threads:[~2026-09-21 9:18 UTC|newest]
Thread overview: 9+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-20 16:22 [PATCH ha-manager 0/7] auto rebalance: fix failure retry loop, oscillation and idle-cluster churn Michael Ryom
2026-09-20 16:22 ` [PATCH ha-manager 1/7] fix #8059: manager: auto rebalance: back off failed motions and add per-resource cooldown Michael Ryom
2026-09-20 16:22 ` [PATCH ha-manager 2/7] env: dynamic service stats: use host-side memory footprint of guests Michael Ryom
2026-09-21 9:18 ` Dominik Rusovac [this message]
2026-09-20 16:22 ` [PATCH ha-manager 3/7] manager: auto rebalance: require a minimum absolute imbalance improvement Michael Ryom
2026-09-20 16:22 ` [PATCH ha-manager 4/7] sim: hardware: report actual running state in cluster service stats Michael Ryom
2026-09-20 16:22 ` [PATCH ha-manager 5/7] sim: hardware: allow setting a base load for nodes Michael Ryom
2026-09-20 16:22 ` [PATCH ha-manager 6/7] usage: dynamic: smooth the unaccounted node load Michael Ryom
2026-09-20 16:22 ` [PATCH ha-manager 7/7] manager: auto rebalance: only balance under actual node resource pressure Michael Ryom
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DLKVRJQZH2XZ.1POHFGT09YFXT@proxmox.com \
--to=d.rusovac@proxmox.com \
--cc=Michael@RyomHerold.dk \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.