all lists on lists.proxmox.com
 help / color / mirror / Atom feed
* [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests
@ 2026-11-18 11:05 Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH guest-common 01/15] scheduling: add cluster resource scheduling library Thomas Lamprecht
                   ` (14 more replies)
  0 siblings, 15 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

So far only the HA stack can use the balancing of the cluster resource
scheduler (CRS), this series makes it more easily available for all guests on
demand:

 - The migrate dialog sorts the target nodes by how well a migration
   there would balance the cluster load, and preselects the best one.
 - A new "Rebalance Guests" bulk action opens the bulk migrate dialog
   with a rebalance proposal, whose targets can be adjusted per guest
   before migrating.

The cluster stat gathering and the scheduler wrapper move into a new
library in pve-guest-common, with the HA manager switched to using that too.

The new API endpoints only rank nodes and estimate the resulting imbalance.
Whether a migration is possible at all is still decided by the existing
migrate preconditions. HA-managed guests are only proposed for nodes their
HA rules allow. Guests in a positive resource affinity are not proposed, as
HA always moves such a group as a whole.

The first two ha-manager patches are independent fixes found along the
way. A related fix for static mode mispredicting the migration of
containers without a CPU limit touches proxmox-resource-scheduling and
pve-rs, and will follow as a separate series.

Besides the new unit and API tests, I tested this on an actual three-node
cluster with HA, including running proposed migrations through bulk migrate,
but some more testing would be certainly good to have.

Repo (build-)dependencies:
 pve-manager -> pve-ha-manager -> pve-guest-common

pve-guest-common:

Thomas Lamprecht (1):
      scheduling: add cluster resource scheduling library

 debian/control                                |    2 +
 src/Makefile                                  |    4 +
 src/PVE/ResourceScheduling/Planner.pm         |  716 +++++++
 src/PVE/ResourceScheduling/Scheduler.pm       |  164 ++
 src/PVE/ResourceScheduling/Stats.pm           |  194 ++
 src/tests/Makefile                            |   21 +-
 .../resource-scheduling-planner-tests.pl      | 1775 +++++++++++++++++
 .../resource-scheduling-scheduler-tests.pl    |  230 +++
 src/tests/resource-scheduling-stats-tests.pl  |  260 +++
 9 files changed, 3364 insertions(+), 2 deletions(-)

pve-ha-manager:

Thomas Lamprecht (5):
      helpers: block both locations of a migrating negative affinity partner
      usage: static: count services per node again for scoring fallback
      config: allow getting motion info of many resources at once
      env: pve2: delegate cluster stat gathering to scheduling library
      usage: share one scheduler base between static and dynamic

 debian/control                        |   2 +
 debian/pve-ha-manager.install         |   1 +
 src/PVE/HA/Config.pm                  |  83 +++++++---
 src/PVE/HA/Env/PVE2.pm                | 155 ++++-------------
 src/PVE/HA/Helpers.pm                 |   4 +-
 src/PVE/HA/Usage/Dynamic.pm           | 145 +---------------
 src/PVE/HA/Usage/Makefile             |   2 +-
 src/PVE/HA/Usage/Scheduler.pm         | 175 ++++++++++++++++++++
 src/PVE/HA/Usage/Static.pm            | 157 +-----------------
 src/test/Makefile                     |   2 +
 src/test/test_resource_motion_info.pl | 228 ++++++++++++++++++++++++++
 src/test/test_usage_fallback.pl       |  77 +++++++++
 12 files changed, 585 insertions(+), 446 deletions(-)

pve-manager:

Thomas Lamprecht (8):
      ui: bulk action: translate error title and CT restart warning
      ui: bulk migrate: show CT restart warning for cluster-wide migration
      ui: migrate: recheck preconditions if the target changed meanwhile
      api: cluster: add resource scheduling endpoints
      ui: migrate: sort target nodes by scheduler recommendation
      api: bulk migrate: allow a per-guest target node mapping
      ui: bulk migrate: support per-guest target nodes
      ui: dc: add a Rebalance Guests bulk action

 PVE/API2/Cluster.pm                       |   7 +
 PVE/API2/Cluster/BulkAction/Guest.pm      | 100 ++++-
 PVE/API2/Cluster/Makefile                 |   3 +-
 PVE/API2/Cluster/ResourceScheduling.pm    | 413 ++++++++++++++++++++
 test/BulkMigrate_test.pl                  | 204 ++++++++++
 test/Makefile                             |  10 +-
 test/ResourceScheduling_test.pl           | 454 ++++++++++++++++++++++
 www/manager6/Makefile                     |   1 +
 www/manager6/dc/CmdMenu.js                |  21 +
 www/manager6/dc/Config.js                 |  16 +
 www/manager6/form/VMSelectorWithTarget.js | 202 ++++++++++
 www/manager6/window/BulkAction.js         | 173 ++++++++-
 www/manager6/window/Migrate.js            | 164 ++++++++
 13 files changed, 1752 insertions(+), 16 deletions(-)

pve-docs:

Thomas Lamprecht (1):
      pvecm: document balancing the cluster load with migrations

 ha-manager.adoc |  4 ++++
 pvecm.adoc      | 51 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

--
2.47.3




^ permalink raw reply	[flat|nested] 16+ messages in thread

* [PATCH guest-common 01/15] scheduling: add cluster resource scheduling library
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH ha-manager 02/15] helpers: block both locations of a migrating negative affinity partner Thomas Lamprecht
                   ` (13 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

The CRS node-scheduling algorithms are currently only reachable from
inside the HA manager. Provide them as a library that both the HA stack
and plain API calls can build on, so ranking migration targets for a
guest or proposing a cluster rebalance does not need the HA manager,
and both sides share one implementation.

PVE::ResourceScheduling::Stats gathers the node and guest stats the HA
manager uses today, including the host memory accounting for VMs from
pve-ha-manager's fix #7974. Static guest usage comes from resolvers the
caller passes in, keeping the library free of guest type and HA
dependencies. PVE::ResourceScheduling::Scheduler is a plain forwarder
over the static and dynamic scheduler bindings, so the HA usage drivers
can build on it without any change in their decisions.

PVE::ResourceScheduling::Planner works on a snapshot of the whole
cluster. In dynamic mode it counts a node's load as at least the sum of
its running guests' usage, so a simulated migration removes from the
source what it adds to the target, while host overhead such as the ZFS
ARC stays counted. Nodes without usable stats are skipped instead of
failing the whole request, and the automatic mode only uses dynamic
stats if every node has them, so one node with stale data does not skew
the view of the others.

Rebalance proposals take the best migration step by step, for at most
ten steps by default, to keep a synchronous API call fast and the
proposal reviewable. A step must also improve the imbalance once
applied, as the static backend mispredicts migrating a container
without CPU limit. Otherwise it is undone, and the step tries the next
best guest instead. Steps gaining next to nothing, like moving an idle
guest, or little compared to the earlier ones, and proposals barely
improving an already balanced cluster, are dropped, as every migration
disrupts its guest. The result lists one net migration per guest,
since callers carry them out concurrently.
Guests that must be kept apart are checked against the placement
reached by the earlier steps, as per-guest target lists alone cannot
keep apart guests that move in the same proposal.

To keep a synchronous API call fast on large clusters, where scoring
every guest against every node takes about a second per step with 100
nodes and 10000 guests, a step only considers the guests on the
busiest nodes, each with the least loaded node of every node size.

Depend on libpve-rs-perl 0.14.0 or newer, the first version reporting
the imbalance normalized to the unit interval, which the improvement
thresholds are expressed in.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 debian/control                                |    2 +
 src/Makefile                                  |    4 +
 src/PVE/ResourceScheduling/Planner.pm         |  716 +++++++
 src/PVE/ResourceScheduling/Scheduler.pm       |  164 ++
 src/PVE/ResourceScheduling/Stats.pm           |  194 ++
 src/tests/Makefile                            |   21 +-
 .../resource-scheduling-planner-tests.pl      | 1775 +++++++++++++++++
 .../resource-scheduling-scheduler-tests.pl    |  230 +++
 src/tests/resource-scheduling-stats-tests.pl  |  260 +++
 9 files changed, 3364 insertions(+), 2 deletions(-)
 create mode 100644 src/PVE/ResourceScheduling/Planner.pm
 create mode 100644 src/PVE/ResourceScheduling/Scheduler.pm
 create mode 100644 src/PVE/ResourceScheduling/Stats.pm
 create mode 100644 src/tests/resource-scheduling-planner-tests.pl
 create mode 100644 src/tests/resource-scheduling-scheduler-tests.pl
 create mode 100644 src/tests/resource-scheduling-stats-tests.pl

diff --git a/debian/control b/debian/control
index 0adb2b6..de35dfe 100644
--- a/debian/control
+++ b/debian/control
@@ -5,6 +5,7 @@ Maintainer: Proxmox Support Team <support@proxmox.com>
 Build-Depends: debhelper-compat (= 13),
                libpve-cluster-perl,
                libpve-common-perl (>= 9.2.6),
+               libpve-rs-perl (>= 0.14.0),
                libpve-storage-perl (>= 8.3.4),
                libtest-mockmodule-perl,
                pve-cluster (>= 9.1.10),
@@ -16,6 +17,7 @@ Architecture: all
 Depends: libpve-access-control,
          libpve-cluster-perl (>= 8.1.0),
          libpve-common-perl (>= 9.2.6),
+         libpve-rs-perl (>= 0.14.0),
          libpve-storage-perl (>= 8.3.4),
          proxmox-websocket-tunnel,
          pve-cluster (>= 9.1.10),
diff --git a/src/Makefile b/src/Makefile
index 161dbe5..f44009b 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -19,6 +19,10 @@ install: PVE
 	install -m 0644 PVE/Mapping/Dir.pm ${PERL5DIR}/PVE/Mapping/
 	install -m 0644 PVE/Mapping/PCI.pm ${PERL5DIR}/PVE/Mapping/
 	install -m 0644 PVE/Mapping/USB.pm ${PERL5DIR}/PVE/Mapping/
+	install -d ${PERL5DIR}/PVE/ResourceScheduling
+	install -m 0644 PVE/ResourceScheduling/Planner.pm ${PERL5DIR}/PVE/ResourceScheduling/
+	install -m 0644 PVE/ResourceScheduling/Scheduler.pm ${PERL5DIR}/PVE/ResourceScheduling/
+	install -m 0644 PVE/ResourceScheduling/Stats.pm ${PERL5DIR}/PVE/ResourceScheduling/
 	install -d ${PERL5DIR}/PVE/VZDump
 	install -m 0644 PVE/VZDump/Common.pm ${PERL5DIR}/PVE/VZDump/
 	install -m 0644 PVE/VZDump/JobBase.pm ${PERL5DIR}/PVE/VZDump/
diff --git a/src/PVE/ResourceScheduling/Planner.pm b/src/PVE/ResourceScheduling/Planner.pm
new file mode 100644
index 0000000..c557cbc
--- /dev/null
+++ b/src/PVE/ResourceScheduling/Planner.pm
@@ -0,0 +1,716 @@
+package PVE::ResourceScheduling::Planner;
+
+use v5.36;
+
+use List::Util qw(max min);
+use Scalar::Util qw(looks_like_number);
+
+use PVE::ResourceScheduling::Scheduler;
+use PVE::ResourceScheduling::Stats;
+
+# A cluster-wide view of node and guest usage that drives the resource scheduler for two queries:
+# ranking the best node(s) to place or migrate a single guest, and proposing a set of migrations to
+# rebalance the cluster.
+#
+# This is the HA-independent counterpart to the HA stack's PVE::HA::Usage::* drivers; it works on
+# plain guest stats (node, running, usage) instead of HA service state, so it can be used from a
+# normal API call without the HA manager.
+
+=head3 DEFAULT_MAX_MOVES
+
+The number of greedy steps C<suggest_rebalance()> takes at most unless told otherwise. Every step
+scores movable guests against target nodes, so the cost of a proposal grows with its length; the
+default keeps a synchronous API call fast and the proposal short enough to review. Applying a
+proposal and asking again continues from the new state.
+
+=cut
+
+use constant DEFAULT_MAX_MOVES => 10;
+
+=head3 DEFAULT_BUSIEST_NODES
+
+The default C<busiest_nodes> of C<suggest_rebalance()>, 10. Scoring every movable guest against
+every other node makes a step expensive on large clusters, as the number of pairs grows with both
+the guests and the nodes, while the best migration practically always moves load off one of the
+busiest nodes. Limiting a step to the guests on those keeps a synchronous API call fast.
+
+=cut
+
+use constant DEFAULT_BUSIEST_NODES => 10;
+
+=head3 DEFAULT_MIN_STEP_RATIO
+
+The default C<min_step_ratio> of C<suggest_rebalance()>, 0.1. A greedy proposal takes the largest
+improvements first, so once a step gains less than a tenth of the best step taken, the remaining
+steps mostly shuffle guests around for a marginal gain, while each migration disrupts its guest, and
+a container even gets restarted.
+
+=cut
+
+use constant DEFAULT_MIN_STEP_RATIO => 0.1;
+
+=head3 DEFAULT_BALANCED_IMBALANCE
+
+The default C<balanced_imbalance> of C<suggest_rebalance()>, 0.1. Below that imbalance, moves are
+only proposed if together they lower it by at least C<min_total_ratio> of its value. Only such
+clusters need that guard: on a clearly imbalanced cluster with many small guests, the few steps
+of one proposal may not gain much in total, but each of them is a real improvement.
+
+=cut
+
+use constant DEFAULT_BALANCED_IMBALANCE => 0.1;
+
+=head3 DEFAULT_MIN_TOTAL_RATIO
+
+The default C<min_total_ratio> of C<suggest_rebalance()>, 0.1. For a cluster that counts as
+balanced, a proposal lowering the imbalance by less than this fraction of its value is not worth
+the disruption of its migrations. The threshold is relative, as the imbalance caused by a single
+overloaded node shrinks with the number of nodes, so an absolute one would hide such a node in a
+larger cluster.
+
+=cut
+
+use constant DEFAULT_MIN_TOTAL_RATIO => 0.1;
+
+# The scorer truncates the imbalance it predicts for a migration to 15 significant digits, so a move
+# that leaves the load distribution unchanged can look like an improvement of about 1e-16. Requiring
+# at least this much improvement per step keeps such no-op migrations out of a proposal.
+use constant MIN_IMPROVEMENT_FLOOR => 1e-9;
+
+# Evening out two nodes lowers the imbalance by about its value divided by the node count. A step
+# gaining less than this fraction of that, like the move of an idle guest, changes nothing
+# noticeable and is not worth a migration.
+use constant MIN_STEP_NODE_SHARE => 0.05;
+
+# How many moves a step may undo before it ends the proposal. Each retry rescores all candidates.
+use constant MAX_UNDONE_PER_STEP => 5;
+
+# Whether the scheduler can work with a node's stats. The backend divides by the capacity, so a
+# missing or zero capacity would turn every imbalance value into NaN.
+my sub usable_node_stats($stats) {
+    return
+        ref($stats) eq 'HASH'
+        && ($stats->{maxcpu} // 0) > 0
+        && ($stats->{maxmem} // 0) > 0;
+}
+
+# Copy the stats the given mode uses, with absent dynamic load counted as idle.
+my sub copy_node_stats($mode, $stats) {
+    my $copy = { maxcpu => $stats->{maxcpu}, maxmem => $stats->{maxmem} };
+    if ($mode eq 'dynamic') {
+        $copy->{cpu} = $stats->{cpu} // 0;
+        $copy->{mem} = $stats->{mem} // 0;
+    }
+    return $copy;
+}
+
+my sub copy_guest_stats($guest_stats) {
+    my $copy = {};
+    for my $sid (keys %$guest_stats) {
+        my $guest = $guest_stats->{$sid};
+        $copy->{$sid} = { $guest->%* };
+        $copy->{$sid}->{usage} = { $guest->{usage}->%* } if defined($guest->{usage});
+    }
+    return $copy;
+}
+
+# The running guests on each of the usable nodes in $node_stats, as { $node => { $sid => 1 } }. Only
+# these put load on a node; guests on other nodes are treated like guests on an offline node.
+my sub running_guests_by_node($node_stats, $guest_stats) {
+    my $by_node = { map { $_ => {} } keys %$node_stats };
+    for my $sid (keys %$guest_stats) {
+        my $guest = $guest_stats->{$sid};
+        next if !$guest->{running};
+
+        my $node = $guest->{node};
+        next if !defined($node) || !$node_stats->{$node};
+
+        $by_node->{$node}->{$sid} = 1;
+    }
+
+    return $by_node;
+}
+
+# Add $node with its running guests to $scheduler. Returns the node's load as the scheduler sees it,
+# as fractions of its capacity: { cpu, mem }.
+my sub add_node_with_guests($scheduler, $mode, $node, $stats, $sids, $guest_stats) {
+    my @sids = sort keys %$sids;
+
+    my ($cpu, $mem) = (0, 0);
+    for my $sid (@sids) {
+        my $usage = $guest_stats->{$sid}->{usage} // {};
+        if ($mode eq 'static') {
+            # the static backend counts a guest without CPU limit with all cores of its node
+            $cpu += ($usage->{maxcpu} || $stats->{maxcpu});
+            $mem += $usage->{maxmem} // 0;
+        } else {
+            $cpu += $usage->{cpu} // 0;
+            $mem += $usage->{mem} // 0;
+        }
+    }
+
+    if ($mode eq 'static') {
+        $scheduler->add_node($node, $stats);
+    } else {
+        # Reconcile node and guest stats: host overhead not attributable to any guest (e.g. ZFS
+        # ARC, page cache) stays counted as a fixed per-node base, but the node load is never
+        # reported below the sum of its running guests, so removing a guest in a simulated
+        # migration cannot drive a node below the load of its remaining guests. Only the planner
+        # works on a snapshot, so this lives here and not in the scheduler, which stays a pure
+        # forwarder for the HA stack.
+        $cpu = max($stats->{cpu}, $cpu);
+        $mem = max($stats->{mem}, $mem);
+        $scheduler->add_node(
+            $node,
+            {
+                maxcpu => $stats->{maxcpu},
+                cpu => $cpu,
+                maxmem => $stats->{maxmem},
+                mem => $mem,
+            },
+        );
+    }
+
+    for my $sid (@sids) {
+        # always add guests as stationary on their current node: this is a planning view of the
+        # committed placement, and the scheduler only accepts stationary migration candidates
+        $scheduler->add_resource(
+            $sid,
+            { stats => $guest_stats->{$sid}->{usage}, running => 1, 'current-node' => $node },
+        );
+    }
+
+    return { cpu => $cpu / $stats->{maxcpu}, mem => $mem / $stats->{maxmem} };
+}
+
+# Build and populate a fresh scheduler from a snapshot whose $node_stats hold exactly the usable
+# nodes. Returns the scheduler, the running guests by node and the node loads, see
+# running_guests_by_node() and add_node_with_guests().
+my sub build_scheduler($mode, $node_stats, $guest_stats) {
+    my $scheduler = PVE::ResourceScheduling::Scheduler->new($mode);
+    my $by_node = running_guests_by_node($node_stats, $guest_stats);
+
+    my $loads = {};
+    for my $node (sort keys %$node_stats) {
+        $loads->{$node} = add_node_with_guests(
+            $scheduler, $mode, $node, $node_stats->{$node}, $by_node->{$node}, $guest_stats,
+        );
+    }
+
+    return ($scheduler, $by_node, $loads);
+}
+
+# Move guest $sid from $source to $target in the working state of a rebalance proposal and update
+# the scheduler. Only the two nodes involved are re-added, which is much cheaper than a rebuild on
+# large clusters. In dynamic mode the guest's measured usage moves with it, matching how the
+# scheduler predicts a migration; a reverted move restores the saved measured load instead.
+my sub move_guest($work, $sid, $source, $target, $saved = undef) {
+    my ($mode, $node_stats, $guest_stats) = $work->@{qw(mode node_stats guest_stats)};
+
+    if ($saved) {
+        $node_stats->@{ keys %$saved } = values %$saved;
+    } elsif ($mode eq 'dynamic') {
+        my $usage = $guest_stats->{$sid}->{usage} // {};
+        my ($from, $to) = $node_stats->@{ $source, $target };
+
+        $from->{cpu} = max(0, $from->{cpu} - ($usage->{cpu} // 0));
+        $from->{mem} = max(0, $from->{mem} - ($usage->{mem} // 0));
+        $to->{cpu} += $usage->{cpu} // 0;
+        $to->{mem} += $usage->{mem} // 0;
+    }
+
+    $guest_stats->{$sid}->{node} = $target;
+    for my $index ($work->@{qw(by_node movable_by_node)}) {
+        delete $index->{$source}->{$sid};
+        $index->{$target}->{$sid} = 1;
+    }
+
+    for my $node ($source, $target) {
+        $work->{scheduler}->remove_node($node);
+        $work->{loads}->{$node} = add_node_with_guests(
+            $work->{scheduler},
+            $mode,
+            $node,
+            $node_stats->{$node},
+            $work->{by_node}->{$node},
+            $guest_stats,
+        );
+    }
+
+    return;
+}
+
+# The load of $node as the scheduler sees it, as a fraction of its capacity: per dimension ('cpu' or
+# 'mem') or combined ('load'), the latter being what the imbalance is computed from.
+my sub node_load($loads, $node, $dim) {
+    my $load = $loads->{$node};
+    return $dim eq 'load' ? ($load->{cpu} + $load->{mem}) / 2 : $load->{$dim};
+}
+
+# The migration candidates for one step of a rebalance proposal: each movable guest with the target
+# nodes it may go to. Scoring all pairs of guests and nodes is what makes a step expensive on large
+# clusters, so with $limit, only the guests on the $limit most loaded nodes per load dimension are
+# considered, as the best migration practically always moves load off a busy node.
+#
+# Their targets are pruned without losing a better move: a guest adds the same fraction of capacity
+# to any node of the same size, that is with the same CPU and memory capacity, so among these the
+# one with the least combined load is the target with the lowest resulting imbalance. Only nodes of
+# different size compete, so each guest is offered the least loaded node of every size.
+my sub rebalance_candidates($work, $limit) {
+    my ($loads, $movable_by_node, $allowed) = $work->@{qw(loads movable_by_node allowed)};
+
+    my @sources = grep { $movable_by_node->{$_}->%* } sort keys %$movable_by_node;
+    my $orders;
+    if ($limit) {
+        my $pick = {};
+        for my $dim (qw(load cpu mem)) {
+            my @busiest = sort {
+                node_load($loads, $b, $dim) <=> node_load($loads, $a, $dim) || $a cmp $b
+            } @sources;
+            $pick->{$_} = 1 for @busiest[0 .. min($limit, scalar(@busiest)) - 1];
+        }
+        @sources = sort keys %$pick;
+
+        my $sizes = {};
+        for my $node ($work->{target_nodes}->@*) {
+            my $stats = $work->{node_stats}->{$node};
+            push $sizes->{"$stats->{maxcpu}/$stats->{maxmem}"}->@*, $node;
+        }
+
+        $orders = [];
+        for my $size (sort keys %$sizes) {
+            push @$orders, [
+                sort {
+                    node_load($loads, $a, 'load') <=> node_load($loads, $b, 'load') || $a cmp $b
+                } $sizes->{$size}->@*
+            ];
+        }
+    }
+
+    my $targets_of = sub($sid, $source) {
+        my $ok = $allowed->{$sid};
+
+        # never its current node, nor one its partners are on after the earlier steps
+        my $forbidden = { $source => 1 };
+        for my $partner (keys(($work->{partners}->{$sid} // {})->%*)) {
+            my $node = $work->{guest_stats}->{$partner}->{node};
+            $forbidden->{$node} = 1 if defined($node);
+        }
+
+        if (!$orders) {
+            return [grep { !$forbidden->{$_} && (!$ok || $ok->{$_}) }
+                $work->{target_nodes}->@*];
+        }
+        my $pick = {};
+        for my $order (@$orders) {
+            for my $node (@$order) {
+                next if $forbidden->{$node} || ($ok && !$ok->{$node});
+                $pick->{$node} = 1;
+                last;
+            }
+        }
+        return [sort keys %$pick];
+    };
+
+    my @candidates;
+    for my $source (@sources) {
+        my $shared; # target nodes shared by all guests without a restriction of their own
+        for my $sid (sort keys $movable_by_node->{$source}->%*) {
+            my $nodes =
+                $allowed->{$sid} || $work->{partners}->{$sid}
+                ? $targets_of->($sid, $source)
+                : ($shared //= $targets_of->($sid, $source));
+            push @candidates, { leader => $sid, resources => [$sid], nodes => $nodes } if @$nodes;
+        }
+    }
+
+    return [sort { $a->{leader} cmp $b->{leader} } @candidates];
+}
+
+=head3 PVE::ResourceScheduling::Planner->new(%args)
+
+Construct from an explicit stats snapshot given in C<%args>: C<mode> ('dynamic' (default) or
+'static'), C<nodes> (arrayref of node names), C<node_stats> (C<< { $node => { maxcpu, maxmem, cpu?,
+mem? } } >>) and C<guest_stats> (C<< { $sid => { node, running, usage } } >>, as returned by
+C<PVE::ResourceScheduling::Stats>). The snapshot is copied, so later changes to the passed hashes
+do not affect the planner.
+
+Only the nodes from C<nodes> with usable stats take part, that is a stats hash with C<maxcpu> and
+C<maxmem> greater than zero; in dynamic mode a missing C<cpu> or C<mem> counts as idle. The other
+nodes are skipped: they are neither ranking candidates nor rebalance sources or targets, and their
+guests are treated like guests on an offline node. Mostly for tests and internal use; production
+callers usually want C<build()>.
+
+=cut
+
+sub new($class, %args) {
+    my $mode = $args{mode} // 'dynamic';
+    die "invalid scheduling mode '$mode'\n" if $mode ne 'dynamic' && $mode ne 'static';
+
+    my $nodes = $args{nodes} or die "missing required 'nodes' argument\n";
+    my $node_stats = $args{node_stats} or die "missing required 'node_stats' argument\n";
+    my $guest_stats = $args{guest_stats} // {};
+
+    my $usable = {};
+    for my $node (@$nodes) {
+        my $stats = $node_stats->{$node};
+        $usable->{$node} = copy_node_stats($mode, $stats) if usable_node_stats($stats);
+    }
+
+    my $self = bless {
+        mode => $mode,
+        node_stats => $usable,
+        guest_stats => copy_guest_stats($guest_stats),
+    }, $class;
+
+    ($self->{scheduler}) = build_scheduler($mode, $self->{node_stats}, $self->{guest_stats});
+
+    return $self;
+}
+
+=head3 PVE::ResourceScheduling::Planner->build(%args)
+
+Gather the cluster stats and construct. C<%args> takes C<nodes> (arrayref of node names, required),
+C<mode> ('auto' (default), 'dynamic' or 'static') and C<resolvers> (a guest-type to code-ref map,
+see C<PVE::ResourceScheduling::Stats::get_static_guest_stats>), which is required for 'auto' and
+'static'.
+
+'auto' uses dynamic load if every requested node has usable dynamic stats, and static usage
+otherwise, so one node without fresh load data does not distort the view of the others. An explicit
+'dynamic' or 'static' uses that mode and skips the nodes without usable stats. Errors while
+gathering the stats are not caught.
+
+=cut
+
+sub build($class, %args) {
+    my $mode = $args{mode} // 'auto';
+    die "invalid scheduling mode '$mode'\n"
+        if $mode ne 'auto' && $mode ne 'dynamic' && $mode ne 'static';
+
+    my $nodes = $args{nodes} or die "missing required 'nodes' argument\n";
+
+    my $resolvers = $args{resolvers};
+    die "scheduling mode '$mode' needs a 'resolvers' argument\n"
+        if $mode ne 'dynamic' && ref($resolvers) ne 'HASH';
+
+    if ($mode ne 'static') {
+        my $node_stats = PVE::ResourceScheduling::Stats::get_dynamic_node_stats();
+
+        my $complete = !grep { !usable_node_stats($node_stats->{$_}) } @$nodes;
+        if ($mode eq 'dynamic' || $complete) {
+            return $class->new(
+                mode => 'dynamic',
+                nodes => $nodes,
+                node_stats => $node_stats,
+                guest_stats => PVE::ResourceScheduling::Stats::get_dynamic_guest_stats(),
+            );
+        }
+    }
+
+    return $class->new(
+        mode => 'static',
+        nodes => $nodes,
+        node_stats => PVE::ResourceScheduling::Stats::get_static_node_stats(),
+        guest_stats => PVE::ResourceScheduling::Stats::get_static_guest_stats($resolvers),
+    );
+}
+
+=head3 $planner->mode()
+
+The scheduling mode actually in use, 'dynamic' or 'static'.
+
+=cut
+
+sub mode($self) {
+    return $self->{mode};
+}
+
+=head3 $planner->node_imbalance()
+
+The current cluster load imbalance in C<[0, 1]>; lower is more balanced.
+
+=cut
+
+sub node_imbalance($self) {
+    return $self->{scheduler}->calculate_node_imbalance();
+}
+
+=head3 $planner->rank_target_nodes($sid, $candidate_nodes)
+
+Rank the target nodes for guest C<$sid> of the snapshot, best first, as an arrayref of
+C<< { node, imbalance? } >>. The candidates are the snapshot nodes, restricted to the optional
+C<$candidate_nodes> arrayref, never including the guest's current node.
+
+A guest running on a snapshot node is ranked as a migration off that node, and each entry carries
+the resulting cluster C<imbalance> (lower is better). Any other guest, stopped or on a skipped or
+offline node, is ranked as a placement by its usage, and the entries carry no C<imbalance>. Dies if
+C<$sid> is not in the snapshot.
+
+=cut
+
+sub rank_target_nodes($self, $sid, $candidate_nodes = undef) {
+    my $guest = $self->{guest_stats}->{$sid}
+        or die "no such guest '$sid' in the cluster snapshot\n";
+
+    my $nodes = $self->{node_stats};
+    my $current = $guest->{node} // '';
+    my $wanted = $candidate_nodes ? { map { $_ => 1 } @$candidate_nodes } : undef;
+
+    my @candidates = grep { $_ ne $current && (!$wanted || $wanted->{$_}) } sort keys %$nodes;
+    return [] if !@candidates;
+
+    if ($guest->{running} && $nodes->{$current}) {
+        my $candidate = { leader => $sid, resources => [$sid], nodes => [@candidates] };
+        my $scored = $self->{scheduler}
+            ->score_best_balancing_migration_candidates([$candidate], scalar(@candidates));
+
+        my @ranked =
+            map { { node => $_->{migration}->{'target-node'}, imbalance => $_->{imbalance} } }
+            @$scored;
+
+        # lower resulting imbalance is better, with the node name as a stable tie-break
+        return [sort { $a->{imbalance} <=> $b->{imbalance} || $a->{node} cmp $b->{node} } @ranked];
+    }
+
+    my $usage = $guest->{usage} or die "missing usage of guest '$sid' for placement ranking\n";
+    my $allowed = { map { $_ => 1 } @candidates };
+
+    # higher start score is better; the score itself is not exposed, only the resulting order
+    my @ranked =
+        sort { $b->[1] <=> $a->[1] || $a->[0] cmp $b->[0] }
+        grep { $allowed->{ $_->[0] } } $self->{scheduler}->score_nodes_to_start($usage)->@*;
+
+    return [map { { node => $_->[0] } } @ranked];
+}
+
+=head3 $planner->suggest_rebalance($movable, %opts)
+
+Propose migrations that reduce the cluster imbalance, moving only the guests in the C<$movable>
+arrayref of sids. Only guests running on a snapshot node are moved, and only between snapshot nodes.
+
+The optional C<$opts{targets}> arrayref of node names restricts the nodes guests may be moved to,
+for example to fill newly added nodes or to keep guests off a node. Names that are not snapshot
+nodes are ignored, and without any remaining target node no moves are proposed. Guests are still
+moved off any snapshot node, including one outside the targets, and the imbalance is still
+computed over all snapshot nodes.
+
+The optional C<$opts{allowed_targets}> hash maps a sid to an arrayref of the node names that guest
+may be moved to, for example to honor the node rules of an HA resource. For a guest in it, these
+nodes further restrict C<$opts{targets}>; a guest without remaining target node is not moved. Guests
+not in it are not restricted further.
+
+The optional C<$opts{separate}> hash maps a sid to an arrayref of sids of the guests it must be
+kept apart from, for example for a negative resource affinity rule; the relation is mutual, so
+listing it in one direction suffices. A step never moves a guest onto a node that, in the placement
+reached by the earlier steps, holds one of its partners, whether these are movable or not. So no
+reported move puts guests that must be kept apart on one node, while guests that already share a
+node are left for the caller to resolve.
+
+With the default method, each step considers only the guests on the C<$opts{busiest_nodes}>
+(default C<DEFAULT_BUSIEST_NODES>, 0 disables) most loaded nodes by combined, CPU and memory load,
+and offers each guest only its least loaded target node of every node size, which cannot miss a
+better move for these guests. If such a step finds no move passing the checks below, it is retried
+with all guests and target nodes, so the limit never ends a proposal early.
+
+Works greedily on a copy of the snapshot: each step picks the best migration according to
+C<$opts{method}>, 'bruteforce' (default, lowest resulting imbalance) or 'topsis'. The proposal ends
+after at most C<$opts{max_moves}> steps (default C<DEFAULT_MAX_MOVES>), or before the first step
+that falls short, that is, whose improvement of the imbalance
+
+=over
+
+=item * is not larger than C<$opts{min_improvement}> (default 0), or than a small floor that keeps
+no-op migrations out, or
+
+=item * is less than a twentieth of the current imbalance divided by the node count, which keeps
+moves of nearly idle guests out, or
+
+=item * is less than C<$opts{min_step_ratio}> (default C<DEFAULT_MIN_STEP_RATIO>, 0 disables) times
+the largest improvement of a step taken so far.
+
+=back
+
+The improvement is checked both as predicted by the scheduler and as measured on the state after
+the move. If the measured improvement falls short, the move is undone and the step tries the next
+best guest instead, up to a few times. With 'topsis', the proposal thus ends as soon as the
+method's best pick is not predicted to improve the imbalance enough.
+
+Below an imbalance of C<$opts{balanced_imbalance}> (default C<DEFAULT_BALANCED_IMBALANCE>, 0
+disables), moves are only proposed if together they lower the imbalance by at least
+C<$opts{min_total_ratio}> (default C<DEFAULT_MIN_TOTAL_RATIO>, 0 disables) of its value.
+
+Returns C<< { mode, imbalance => { before, after }, moves => [ { sid, source, target } ] } >>.
+C<after> is the imbalance once all moves are done. A guest moved more than once is reported with a
+single move from its original to its final node, and one that ends up on its original node is left
+out. As every step moves a guest to one of its target nodes, so does every reported move. The moves
+are ordered by the step in which each guest first moved. As each guest appears at most once,
+carrying out the moves in any order, or concurrently, leads to the same placement.
+
+=cut
+
+sub suggest_rebalance($self, $movable, %opts) {
+    my $method = $opts{method} // 'bruteforce';
+    die "invalid method '$method'\n" if $method ne 'bruteforce' && $method ne 'topsis';
+
+    my $max_moves = $opts{max_moves} // DEFAULT_MAX_MOVES;
+    die "invalid max_moves '$max_moves', must be a positive integer\n"
+        if $max_moves !~ m/^[0-9]+$/ || $max_moves < 1;
+
+    my $min_improvement = $opts{min_improvement} // 0;
+    die "invalid min_improvement '$min_improvement', must be a non-negative number\n"
+        if !looks_like_number($min_improvement) || !($min_improvement >= 0);
+
+    my $min_step_ratio = $opts{min_step_ratio} // DEFAULT_MIN_STEP_RATIO;
+    die "invalid min_step_ratio '$min_step_ratio', must be a number from 0 to 1\n"
+        if !looks_like_number($min_step_ratio) || !($min_step_ratio >= 0 && $min_step_ratio <= 1);
+
+    my $min_total_ratio = $opts{min_total_ratio} // DEFAULT_MIN_TOTAL_RATIO;
+    die "invalid min_total_ratio '$min_total_ratio', must be a number from 0 to 1\n"
+        if !looks_like_number($min_total_ratio)
+        || !($min_total_ratio >= 0 && $min_total_ratio <= 1);
+
+    my $balanced_imbalance = $opts{balanced_imbalance} // DEFAULT_BALANCED_IMBALANCE;
+    die "invalid balanced_imbalance '$balanced_imbalance', must be a number from 0 to 1\n"
+        if !looks_like_number($balanced_imbalance)
+        || !($balanced_imbalance >= 0 && $balanced_imbalance <= 1);
+
+    my $targets = $opts{targets};
+    die "invalid targets, must be an array reference of node names\n"
+        if defined($targets) && ref($targets) ne 'ARRAY';
+
+    my $allowed_targets = $opts{allowed_targets} // {};
+    die "invalid allowed_targets, must be a hash reference of node name array references\n"
+        if ref($allowed_targets) ne 'HASH' || grep { ref($_) ne 'ARRAY' } values %$allowed_targets;
+
+    my $separate = $opts{separate} // {};
+    die "invalid separate, must be a hash reference of guest id array references\n"
+        if ref($separate) ne 'HASH' || grep { ref($_) ne 'ARRAY' } values %$separate;
+
+    my $busiest_nodes = $opts{busiest_nodes} // DEFAULT_BUSIEST_NODES;
+    die "invalid busiest_nodes '$busiest_nodes', must be a non-negative integer\n"
+        if $busiest_nodes !~ m/^[0-9]+$/;
+
+    my $mode = $self->{mode};
+    my $work = {
+        mode => $mode,
+        node_stats =>
+            { map { $_ => { $self->{node_stats}->{$_}->%* } } keys $self->{node_stats}->%* },
+        guest_stats => copy_guest_stats($self->{guest_stats}),
+    };
+    ($work->{scheduler}, $work->{by_node}, $work->{loads}) =
+        build_scheduler($mode, $work->{node_stats}, $work->{guest_stats});
+
+    my $wanted = $targets ? { map { $_ => 1 } @$targets } : undef;
+    $work->{target_nodes} = [grep { !$wanted || $wanted->{$_} } sort keys $work->{node_stats}->%*];
+
+    for my $sid (keys %$allowed_targets) {
+        $work->{allowed}->{$sid} = { map { $_ => 1 } $allowed_targets->{$sid}->@* };
+    }
+
+    # keeping guests apart is mutual, whichever of them the caller listed
+    for my $sid (keys %$separate) {
+        for my $partner ($separate->{$sid}->@*) {
+            next if $partner eq $sid;
+            $work->{partners}->{$sid}->{$partner} = 1;
+            $work->{partners}->{$partner}->{$sid} = 1;
+        }
+    }
+
+    my $movable_set = { map { $_ => 1 } @$movable };
+    for my $node (keys $work->{by_node}->%*) {
+        $work->{movable_by_node}->{$node} =
+            { map { $_ => 1 } grep { $movable_set->{$_} } keys $work->{by_node}->{$node}->%* };
+    }
+
+    # Pruning only pays off and only makes a difference with more nodes than its limit. TOPSIS ranks
+    # the candidates relative to each other, so leaving some out could change its pick.
+    my $prune =
+        $method eq 'bruteforce'
+        && $busiest_nodes
+        && keys($work->{node_stats}->%*) > $busiest_nodes;
+
+    my $before = $work->{scheduler}->calculate_node_imbalance();
+    my $current = $before;
+
+    my $threshold = max($min_improvement, MIN_IMPROVEMENT_FLOOR);
+    my $node_count = keys $work->{node_stats}->%*;
+    my $largest_gain = 0; # of the steps taken so far
+
+    my $falls_short = sub($gain) {
+        return
+            $gain <= $threshold
+            || $gain < MIN_STEP_NODE_SHARE * $current / $node_count
+            || $gain < $min_step_ratio * $largest_gain;
+    };
+
+    my ($origin, $first_step) = ({}, {});
+    for my $step (1 .. $max_moves) {
+        my $taken;
+        my $undone = {}; # guests whose move this step undid
+
+        # A pruned step can miss the best move, so when it finds nothing worthwhile, retry the
+        # step with all candidates before ending the proposal.
+        LIMIT: for my $limit ($prune ? ($busiest_nodes, 0) : (0)) {
+            my $candidates = rebalance_candidates($work, $limit);
+
+            while (keys %$undone < MAX_UNDONE_PER_STEP) {
+                my @left = grep { !$undone->{ $_->{leader} } } @$candidates;
+                last if !@left;
+
+                my $best = $work->{scheduler}
+                    ->score_best_balancing_migration_candidates(\@left, 1, $method)->[0];
+                last if !$best || $falls_short->($current - $best->{imbalance});
+
+                my ($sid, $source, $target) =
+                    $best->{migration}->@{qw(sid source-node target-node)};
+                my $saved = { map { $_ => { $work->{node_stats}->{$_}->%* } } $source, $target };
+
+                move_guest($work, $sid, $source, $target);
+                my $actual = $work->{scheduler}->calculate_node_imbalance();
+
+                # The prediction can be off where the backend counts a guest differently in the
+                # node load than in a migration, e.g. a static-mode container without a CPU limit.
+                # Trust the updated state, undo a move that does not actually pay off and try the
+                # next best guest.
+                if ($falls_short->($current - $actual)) {
+                    move_guest($work, $sid, $target, $source, $saved);
+                    $undone->{$sid} = 1;
+                    next;
+                }
+
+                $taken = { sid => $sid, source => $source, imbalance => $actual };
+                last LIMIT;
+            }
+        }
+        last if !$taken;
+
+        my $sid = $taken->{sid};
+        $origin->{$sid} //= $taken->{source};
+        $first_step->{$sid} //= $step;
+        $largest_gain = max($largest_gain, $current - $taken->{imbalance});
+        $current = $taken->{imbalance};
+    }
+
+    my $guest_stats = $work->{guest_stats};
+
+    if ($before < $balanced_imbalance && $before - $current < $min_total_ratio * $before) {
+        return {
+            mode => $mode,
+            imbalance => { before => $before, after => $before },
+            moves => [],
+        };
+    }
+
+    my @moved = sort { $first_step->{$a} <=> $first_step->{$b} } keys %$first_step;
+    my @moves =
+        map { { sid => $_, source => $origin->{$_}, target => $guest_stats->{$_}->{node} } }
+        grep { $guest_stats->{$_}->{node} ne $origin->{$_} } @moved;
+
+    return {
+        mode => $mode,
+        imbalance => { before => $before, after => $current },
+        moves => \@moves,
+    };
+}
+
+1;
diff --git a/src/PVE/ResourceScheduling/Scheduler.pm b/src/PVE/ResourceScheduling/Scheduler.pm
new file mode 100644
index 0000000..00ccc03
--- /dev/null
+++ b/src/PVE/ResourceScheduling/Scheduler.pm
@@ -0,0 +1,164 @@
+package PVE::ResourceScheduling::Scheduler;
+
+use v5.36;
+
+use PVE::RS::ResourceScheduling::Static;
+use PVE::RS::ResourceScheduling::Dynamic;
+
+# A thin mode-aware wrapper over PVE::RS::ResourceScheduling::Static / ::Dynamic that exposes the
+# incremental interface the HA stack drives directly (add a node, add/remove a resource one at a
+# time, then query). It hides that the two Rust backends name some methods differently and take
+# different arguments, so callers drive either mode through the same calls.
+#
+# This is deliberately a pure forwarder: it does NOT reconcile dynamic node load against the running
+# guests' usage; PVE::ResourceScheduling::Planner does that on its snapshot. Every call makes the
+# same scheduling decisions as calling the Rust backend directly.
+
+=head3 PVE::ResourceScheduling::Scheduler->new($mode)
+
+Construct a scheduler for C<$mode>, either 'static' or 'dynamic', building the matching
+C<PVE::RS::ResourceScheduling> backend. Dies on an unknown mode.
+
+=cut
+
+sub new($class, $mode) {
+    die "invalid scheduling mode '$mode'\n" if $mode ne 'static' && $mode ne 'dynamic';
+
+    my $backend =
+        $mode eq 'static'
+        ? PVE::RS::ResourceScheduling::Static->new()
+        : PVE::RS::ResourceScheduling::Dynamic->new();
+
+    return bless { mode => $mode, backend => $backend }, $class;
+}
+
+=head3 $scheduler->mode()
+
+The scheduling mode in use, 'static' or 'dynamic'.
+
+=cut
+
+sub mode($self) {
+    return $self->{mode};
+}
+
+=head3 $scheduler->add_node($node, $stats)
+
+Register C<$node> with the C<< { maxcpu, maxmem, cpu, mem } >> stats hash. Static mode uses only
+C<maxcpu>/C<maxmem> (coerced to integers); dynamic mode passes the full hash through. The shape is
+the same in both modes, so callers stay mode-agnostic.
+
+=cut
+
+sub add_node($self, $node, $stats) {
+    if ($self->{mode} eq 'static') {
+        $self->{backend}->add_node($node, int($stats->{maxcpu}), int($stats->{maxmem}));
+    } else {
+        $self->{backend}->add_node($node, $stats);
+    }
+
+    return;
+}
+
+=head3 $scheduler->remove_node($node)
+
+Remove C<$node> from the scheduler.
+
+=cut
+
+sub remove_node($self, $node) {
+    return $self->{backend}->remove_node($node);
+}
+
+=head3 $scheduler->list_nodes()
+
+The registered node names as a list.
+
+=cut
+
+sub list_nodes($self) {
+    return $self->{backend}->list_nodes()->@*;
+}
+
+=head3 $scheduler->contains_node($node)
+
+Whether C<$node> is registered.
+
+=cut
+
+sub contains_node($self, $node) {
+    return $self->{backend}->contains_node($node);
+}
+
+=head3 $scheduler->add_resource($sid, $resource)
+
+Add resource C<$sid> with the C<< { stats, running, 'current-node', 'target-node' } >> description
+C<$resource>.
+
+=cut
+
+sub add_resource($self, $sid, $resource) {
+    if ($self->{mode} eq 'static') {
+        $self->{backend}->add_service($sid, $resource);
+    } else {
+        $self->{backend}->add_resource($sid, $resource);
+    }
+
+    return;
+}
+
+=head3 $scheduler->remove_resource($sid)
+
+Remove resource C<$sid>'s usage from the nodes it was placed on.
+
+=cut
+
+sub remove_resource($self, $sid) {
+    if ($self->{mode} eq 'static') {
+        $self->{backend}->remove_service_usage($sid);
+    } else {
+        $self->{backend}->remove_resource($sid);
+    }
+
+    return;
+}
+
+=head3 $scheduler->calculate_node_imbalance()
+
+The current cluster load imbalance, as reported by the backend.
+
+=cut
+
+sub calculate_node_imbalance($self) {
+    return $self->{backend}->calculate_node_imbalance();
+}
+
+=head3 $scheduler->score_nodes_to_start($usage)
+
+Score every node for placing a resource of C<$usage>. Returns the raw backend result: an arrayref of
+C<< [$node, $score] >> pairs (higher score is better).
+
+=cut
+
+sub score_nodes_to_start($self, $usage) {
+    return $self->{mode} eq 'static'
+        ? $self->{backend}->score_nodes_to_start_service($usage)
+        : $self->{backend}->score_nodes_to_start_resource($usage);
+}
+
+=head3 $scheduler->score_best_balancing_migration_candidates($candidates, $limit, $method)
+
+Rank the migration C<$candidates> (each C<< { leader, resources, nodes } >>) and return up to
+C<$limit> best. The optional C<$method> selects the scorer: 'bruteforce' (default) or 'topsis'.
+
+=cut
+
+sub score_best_balancing_migration_candidates($self, $candidates, $limit, $method = 'bruteforce') {
+    die "invalid scoring method '$method'\n" if $method ne 'bruteforce' && $method ne 'topsis';
+
+    return $method eq 'topsis'
+        ? $self->{backend}->score_best_balancing_migration_candidates_topsis($candidates, $limit)
+        : $self->{backend}->score_best_balancing_migration_candidates($candidates, $limit);
+}
+
+1;
diff --git a/src/PVE/ResourceScheduling/Stats.pm b/src/PVE/ResourceScheduling/Stats.pm
new file mode 100644
index 0000000..1725407
--- /dev/null
+++ b/src/PVE/ResourceScheduling/Stats.pm
@@ -0,0 +1,194 @@
+package PVE::ResourceScheduling::Stats;
+
+use v5.36;
+
+use JSON qw(decode_json);
+
+use PVE::Cluster;
+
+# RRD entry indices for VM and CT guests, taken from
+# PVE::Service::pvestatd::update_{lxc,qemu}_status()
+use constant {
+    RRD_VM_INDEX_STATUS => 2,
+    RRD_VM_INDEX_MAXCPU => 5,
+    RRD_VM_INDEX_CPU => 6,
+    RRD_VM_INDEX_MAXMEM => 7,
+    RRD_VM_INDEX_MEM => 8,
+    RRD_VM_INDEX_MEMHOST => 15,
+};
+
+# RRD entry indices for PVE nodes, taken from
+# PVE::Service::pvestatd::update_node_status()
+use constant {
+    RRD_NODE_INDEX_MAXCPU => 4,
+    RRD_NODE_INDEX_CPU => 5,
+    RRD_NODE_INDEX_MAXMEM => 7,
+    RRD_NODE_INDEX_MEM => 8,
+};
+
+my $HOSTNAME_RE = qr/(?:[a-zA-Z0-9](?:[a-zA-Z0-9\-]{,61}?[a-zA-Z0-9])?)/;
+
+my $guest_type_map = {
+    qemu => 'vm',
+    lxc => 'ct',
+};
+
+# Enumerate the cluster guests with their run state taken from $rrd, a PVE::Cluster::rrd_dump()
+# result. Callers that also read usage from the dump pass in the same one, so run state and usage
+# always come from one consistent snapshot.
+my sub cluster_guest_stats($rrd) {
+    my $vmlist = PVE::Cluster::get_vmlist();
+    my $idlist = $vmlist->{ids} // {};
+
+    my $stats = {};
+    for my $id (keys %$idlist) {
+        my $type = $guest_type_map->{ $idlist->{$id}->{type} // '' };
+        next if !defined($type); # silently ignore unknown guest types
+
+        my $sid = "$type:$id";
+
+        my $rrdentry = $rrd->{"pve-vm-9.0/$id"} // [];
+        # can be any QMP RunState, but 'running' is the only active guest state
+        my $status = $rrdentry->[RRD_VM_INDEX_STATUS] // 'stopped';
+        my $state = $status eq 'running' ? 'started' : 'stopped';
+
+        $stats->{$sid} = {
+            id => $id,
+            node => $idlist->{$id}->{node},
+            type => $type,
+            state => $state,
+            running => $state eq 'started' ? 1 : 0,
+            usage => {},
+        };
+    }
+
+    return $stats;
+}
+
+=head3 get_cluster_guest_stats()
+
+Return all cluster guests as C<< { "$type:$id" => { id, node, type, state, running, usage } } >>
+with empty usage. Useful on its own for callers that only need the guest list and run state; the
+gatherers below fill in the usage.
+
+=cut
+
+sub get_cluster_guest_stats() {
+    return cluster_guest_stats(PVE::Cluster::rrd_dump());
+}
+
+=head3 get_dynamic_guest_stats()
+
+Cluster guest stats with each guest's measured (live, RRD-based) usage filled in, as
+C<< { cpu, maxcpu, mem, maxmem } >>. For VMs, C<mem> is the host-side memory usage.
+
+=cut
+
+sub get_dynamic_guest_stats() {
+    my $rrd = PVE::Cluster::rrd_dump();
+
+    my $stats = cluster_guest_stats($rrd);
+    for my $sid (keys %$stats) {
+        my $id = $stats->{$sid}->{id};
+        my $rrdentry = $rrd->{"pve-vm-9.0/$id"} // [];
+
+        # NOTE the guests' broadcasted vmstatus() caps maxcpu at the node's maxcpu
+        my $maxcpu = ($rrdentry->[RRD_VM_INDEX_MAXCPU] || 0.0) + 0.0;
+
+        # host memory usage is the real cost of a VM on its node and matches what the node stats
+        # account for; containers do not report it, so fall back to their memory usage
+        my $mem = $rrdentry->[RRD_VM_INDEX_MEMHOST] // $rrdentry->[RRD_VM_INDEX_MEM];
+
+        $stats->{$sid}->{usage} = {
+            maxcpu => $maxcpu,
+            cpu => (($rrdentry->[RRD_VM_INDEX_CPU] || 0.0) + 0.0) * $maxcpu,
+            maxmem => int($rrdentry->[RRD_VM_INDEX_MAXMEM] || 0),
+            mem => int($mem || 0),
+        };
+    }
+
+    return $stats;
+}
+
+=head3 get_static_guest_stats($resolvers)
+
+Cluster guest stats with each guest's configured C<< { maxcpu, maxmem } >> usage filled in.
+C<$resolvers> maps a guest type ('vm' or 'ct') to a code ref returning that usage for a guest config
+hash; the caller passes its own so this module stays free of guest-type dependencies. Dies if a
+guest's type has no resolver.
+
+=cut
+
+sub get_static_guest_stats($resolvers) {
+    my $properties = ['cores', 'cpulimit', 'memory', 'sockets', 'vcpus'];
+
+    my $stats = get_cluster_guest_stats();
+    my $confs = PVE::Cluster::get_guest_config_properties($properties);
+
+    for my $sid (keys %$stats) {
+        my ($id, $type) = $stats->{$sid}->@{qw(id type)};
+
+        my $resolver = $resolvers->{$type}
+            or die "no static stats resolver for guest type '$type'\n";
+
+        $stats->{$sid}->{usage} = $resolver->($confs->{$id} // {});
+    }
+
+    return $stats;
+}
+
+=head3 get_dynamic_node_stats()
+
+Return measured (live, RRD-based) per-node usage as
+C<< { $node => { cpu, maxcpu, mem, maxmem } } >>.
+
+=cut
+
+sub get_dynamic_node_stats() {
+    my $rrd = PVE::Cluster::rrd_dump();
+
+    my $stats = {};
+    for my $key (keys %$rrd) {
+        my ($nodename) = $key =~ m/^pve-node-9\.0\/($HOSTNAME_RE)$/;
+        next if !$nodename;
+
+        my $rrdentry = $rrd->{$key} // [];
+
+        my $maxcpu = int($rrdentry->[RRD_NODE_INDEX_MAXCPU] || 0);
+
+        $stats->{$nodename} = {
+            maxcpu => $maxcpu,
+            cpu => (($rrdentry->[RRD_NODE_INDEX_CPU] || 0.0) + 0.0) * $maxcpu,
+            maxmem => int($rrdentry->[RRD_NODE_INDEX_MAXMEM] || 0),
+            mem => int($rrdentry->[RRD_NODE_INDEX_MEM] || 0),
+        };
+    }
+
+    return $stats;
+}
+
+=head3 get_static_node_stats()
+
+Return configured per-node capacity as C<< { $node => { maxcpu, maxmem } } >>, decoded from the
+'static-info' node key-value broadcast.
+
+=cut
+
+sub get_static_node_stats() {
+    my $stats = PVE::Cluster::get_node_kv('static-info');
+
+    for my $node (keys %$stats) {
+        $stats->{$node} = eval {
+            my $node_stats = decode_json($stats->{$node});
+            return {
+                maxcpu => $node_stats->{cpus},
+                maxmem => $node_stats->{memory},
+            };
+        };
+        warn "unable to decode static node info for '$node' - $@" if $@;
+    }
+
+    return $stats;
+}
+
+1;
diff --git a/src/tests/Makefile b/src/tests/Makefile
index 8eb38a8..5d1bde9 100644
--- a/src/tests/Makefile
+++ b/src/tests/Makefile
@@ -1,8 +1,16 @@
 
 all:
 
-.PHONY: check abstract-config guest-id hookscript
-check: abstract-config guest-id hookscript
+TESTS = \
+	abstract-config \
+	guest-id \
+	hookscript \
+	resource-scheduling-planner \
+	resource-scheduling-scheduler \
+	resource-scheduling-stats
+
+.PHONY: check $(TESTS)
+check: $(TESTS)
 
 abstract-config: abstract-config-tests.pl
 	perl -I.. ./abstract-config-tests.pl
@@ -12,3 +20,12 @@ guest-id: guest-id-tests.pl
 
 hookscript: hookscript-tests.pl
 	perl -I.. ./hookscript-tests.pl
+
+resource-scheduling-planner: resource-scheduling-planner-tests.pl
+	perl -I.. ./resource-scheduling-planner-tests.pl
+
+resource-scheduling-scheduler: resource-scheduling-scheduler-tests.pl
+	perl -I.. ./resource-scheduling-scheduler-tests.pl
+
+resource-scheduling-stats: resource-scheduling-stats-tests.pl
+	perl -I.. ./resource-scheduling-stats-tests.pl
diff --git a/src/tests/resource-scheduling-planner-tests.pl b/src/tests/resource-scheduling-planner-tests.pl
new file mode 100644
index 0000000..60ed048
--- /dev/null
+++ b/src/tests/resource-scheduling-planner-tests.pl
@@ -0,0 +1,1775 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib qw(..);
+
+use List::Util qw(max min);
+use Test::More;
+
+use PVE::ResourceScheduling::Planner;
+use PVE::ResourceScheduling::Scheduler;
+
+my $GB = 1024 * 1024 * 1024;
+my $EPS = 1e-9;
+
+# helper: a homogeneous set of nodes with no base load
+my sub nodes {
+    my ($count, %stats) = @_;
+    my $node_stats = {};
+    for my $i (1 .. $count) {
+        $node_stats->{"node$i"} = {
+            maxcpu => 8,
+            cpu => 0,
+            maxmem => 16 * $GB,
+            mem => 0,
+            %stats,
+        };
+    }
+    return $node_stats;
+}
+
+my sub guest {
+    my ($node, %usage) = @_;
+    return {
+        node => $node,
+        running => 1,
+        usage => { maxcpu => 8, cpu => 0, maxmem => 16 * $GB, mem => 0, %usage },
+    };
+}
+
+# build a snapshot Planner over exactly the nodes named by the node-stats keys
+my sub planner {
+    my ($mode, $node_stats, $guest_stats) = @_;
+    return PVE::ResourceScheduling::Planner->new(
+        mode => $mode,
+        nodes => [sort keys %$node_stats],
+        node_stats => $node_stats,
+        guest_stats => $guest_stats // {},
+    );
+}
+
+# Apply a proposal's moves to a copy of the snapshot and return a fresh Planner over the result. In
+# dynamic mode a guest's measured usage moves with it.
+my sub replay {
+    my ($mode, $node_stats, $guest_stats, $moves) = @_;
+
+    my $ns = { map { $_ => { $node_stats->{$_}->%* } } keys %$node_stats };
+    my $gs = { map { $_ => { $guest_stats->{$_}->%* } } keys %$guest_stats };
+
+    for my $move (@$moves) {
+        my ($sid, $source, $target) = $move->@{qw(sid source target)};
+        is($gs->{$sid}->{node}, $source, "replayed move of $sid starts on its source");
+
+        if ($mode eq 'dynamic') {
+            my $usage = $gs->{$sid}->{usage};
+            $ns->{$source}->{cpu} = max(0, $ns->{$source}->{cpu} - $usage->{cpu});
+            $ns->{$source}->{mem} = max(0, $ns->{$source}->{mem} - $usage->{mem});
+            $ns->{$target}->{cpu} += $usage->{cpu};
+            $ns->{$target}->{mem} += $usage->{mem};
+        }
+        $gs->{$sid}->{node} = $target;
+    }
+
+    return planner($mode, $ns, $gs);
+}
+
+# Compare two rebalance proposals: the moves must match exactly, the imbalance values up to the
+# rounding noise of the backend, which sums up the node loads in no fixed order.
+my sub plan_is {
+    my ($got, $expected, $name) = @_;
+    my $close = sub {
+        my ($key) = @_;
+        return abs($got->{imbalance}->{$key} - $expected->{imbalance}->{$key}) < $EPS ? 1 : 0;
+    };
+    is_deeply(
+        [$got->{mode}, $got->{moves}, $close->('before'), $close->('after')],
+        [$expected->{mode}, $expected->{moves}, 1, 1],
+        $name,
+    );
+}
+
+# The pairs of guests from $separate (in either direction) that share a node once the proposal's
+# moves are applied, as "sid/sid" strings.
+my sub shared_nodes {
+    my ($guest_stats, $moves, $separate) = @_;
+    my $node = { map { $_ => $guest_stats->{$_}->{node} } keys %$guest_stats };
+    $node->{ $_->{sid} } = $_->{target} for @$moves;
+
+    my @together;
+    for my $sid (sort keys %$separate) {
+        for my $partner ($separate->{$sid}->@*) {
+            push @together, "$sid/$partner" if $node->{$sid} eq $node->{$partner};
+        }
+    }
+    return \@together;
+}
+
+# A deterministic cluster with an overloaded first node, some unattributed node overhead and some
+# nodes measuring less than their guests report, so the dynamic reconcile matters.
+my sub synthetic_cluster {
+    my ($node_count, $guest_count) = @_;
+
+    my @nodes = map { "node$_" } 1 .. $node_count;
+    my $node_stats = {
+        map {
+            my $i = $_;
+            (
+                $nodes[$i] => {
+                    maxcpu => (16, 32, 64)[$i % 3],
+                    maxmem => (64, 128, 256)[($i + 1) % 3] * $GB,
+                    cpu => 0,
+                    mem => 0,
+                },
+            );
+        } 0 .. $#nodes
+    };
+
+    my $guest_stats = {};
+    for my $i (1 .. $guest_count) {
+        my $node = $i % 3 == 0 ? $nodes[$i % $node_count] : $nodes[0];
+        my $running = $i % 7 != 0;
+        my $usage = {
+            maxcpu => 0.0 + (1, 2, 4)[$i % 3],
+            cpu => $running ? (($i * 7) % 10) / 4 : 0,
+            maxmem => (2, 4, 8, 16)[$i % 4] * $GB,
+            mem => $running ? (($i * 5) % 8 + 1) * $GB / 2 : 0,
+        };
+        $guest_stats->{ "vm:" . (100 + $i) } =
+            { node => $node, running => $running ? 1 : 0, usage => $usage };
+
+        if ($running) {
+            $node_stats->{$node}->{cpu} += $usage->{cpu};
+            $node_stats->{$node}->{mem} += $usage->{mem};
+        }
+    }
+
+    for my $i (0 .. $#nodes) {
+        my $stats = $node_stats->{ $nodes[$i] };
+        if ($i % 3 == 1) { # measures less than its guests report
+            $stats->{cpu} *= 0.6;
+            $stats->{mem} = int($stats->{mem} * 0.7);
+        } else { # host overhead on top of the guests
+            $stats->{cpu} += 0.5 * $i;
+            $stats->{mem} += $i * $GB;
+        }
+    }
+
+    return ($node_stats, $guest_stats);
+}
+
+subtest 'node_imbalance reflects the load distribution' => sub {
+    my $stacked = planner(
+        'dynamic',
+        {
+            node1 => { maxcpu => 8, cpu => 6, maxmem => 16 * $GB, mem => 12 * $GB },
+            node2 => { maxcpu => 8, cpu => 0, maxmem => 16 * $GB, mem => 0 },
+            node3 => { maxcpu => 8, cpu => 0, maxmem => 16 * $GB, mem => 0 },
+        },
+        {
+            'vm:100' => guest('node1', cpu => 3, mem => 6 * $GB),
+            'vm:101' => guest('node1', cpu => 3, mem => 6 * $GB),
+        },
+    );
+
+    my $balanced = planner(
+        'dynamic',
+        {
+            node1 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB },
+            node2 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB },
+            node3 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB },
+        },
+    );
+
+    cmp_ok($stacked->node_imbalance(), '>', 0.5, 'stacked load yields high imbalance');
+    cmp_ok($balanced->node_imbalance(), '<', 0.01, 'even load yields near-zero imbalance');
+};
+
+subtest 'reconcile: host overhead (ARC) stays counted as load' => sub {
+    my $without_base = planner(
+        'dynamic',
+        {
+            node1 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB },
+            node2 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB },
+        },
+        {
+            'vm:100' => guest('node1', cpu => 2, mem => 4 * $GB),
+            'vm:101' => guest('node2', cpu => 2, mem => 4 * $GB),
+        },
+    );
+
+    # node1 additionally carries 6 GiB of non-guest memory (e.g. ZFS ARC)
+    my $with_base = planner(
+        'dynamic',
+        {
+            node1 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 10 * $GB },
+            node2 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB },
+        },
+        {
+            'vm:100' => guest('node1', cpu => 2, mem => 4 * $GB),
+            'vm:101' => guest('node2', cpu => 2, mem => 4 * $GB),
+        },
+    );
+
+    cmp_ok(
+        $with_base->node_imbalance(),
+        '>',
+        $without_base->node_imbalance(),
+        'unattributed host overhead is kept as real node load',
+    );
+};
+
+subtest 'reconcile: node load never drops below the sum of its running guests' => sub {
+    # all nodes carry load, so the imbalance reacts to every node's value
+    my $peers = {
+        node2 => { maxcpu => 8, cpu => 3, maxmem => 16 * $GB, mem => 6 * $GB },
+        node3 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 5 * $GB },
+    };
+    my $guests = { 'vm:100' => guest('node1', cpu => 2, mem => 4 * $GB) };
+
+    # node1 measures less cpu and memory than its running guest reports
+    my $under = { maxcpu => 8, cpu => 1, maxmem => 16 * $GB, mem => 2 * $GB };
+    my $under_reported = planner('dynamic', { %$peers, node1 => $under }, $guests);
+
+    my $consistent = planner(
+        'dynamic',
+        { %$peers, node1 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB } },
+        $guests,
+    );
+
+    # the same measured load without any guest: what the raw values alone amount to
+    my $raw = planner('dynamic', { %$peers, node1 => $under });
+
+    cmp_ok(
+        abs($raw->node_imbalance() - $consistent->node_imbalance()),
+        '>',
+        0.01,
+        'the raw node load is distinguishable from the guest sum',
+    );
+    ok(
+        abs($under_reported->node_imbalance() - $consistent->node_imbalance()) < $EPS,
+        'an under-reported node is lifted to the sum of its running guests',
+    );
+
+    # a stopped guest puts no load on its node, whatever usage it last reported
+    my $with_stopped = planner(
+        'dynamic',
+        { %$peers, node1 => $under },
+        {
+            %$guests,
+            'vm:101' => { guest('node1', cpu => 4, mem => 8 * $GB)->%*, running => 0 },
+        },
+    );
+    ok(
+        abs($with_stopped->node_imbalance() - $consistent->node_imbalance()) < $EPS,
+        'stopped guests do not count toward the guest sum',
+    );
+};
+
+subtest 'rank_target_nodes: running guest is ranked as a migration' => sub {
+    my $node_stats = {
+        node1 => { maxcpu => 8, cpu => 4, maxmem => 16 * $GB, mem => 8 * $GB },
+        node2 => { maxcpu => 8, cpu => 3, maxmem => 16 * $GB, mem => 6 * $GB },
+        node3 => { maxcpu => 8, cpu => 0, maxmem => 16 * $GB, mem => 0 },
+    };
+    my $guest_stats = {
+        'vm:100' => guest('node1', cpu => 4, mem => 8 * $GB),
+        'vm:200' => guest('node2', cpu => 3, mem => 6 * $GB),
+    };
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+
+    my $ranked = $planner->rank_target_nodes('vm:100');
+
+    is_deeply([map { $_->{node} } @$ranked], ['node3', 'node2'], 'emptier node ranks first');
+    for my $entry (@$ranked) {
+        my $moved = [{ sid => 'vm:100', source => 'node1', target => $entry->{node} }];
+        my $expected = replay('dynamic', $node_stats, $guest_stats, $moved)->node_imbalance();
+        ok(
+            abs($entry->{imbalance} - $expected) < $EPS,
+            "imbalance for $entry->{node} is the imbalance after migrating there",
+        ) or diag("ranked $entry->{imbalance}, replayed $expected");
+    }
+};
+
+subtest 'rank_target_nodes: stopped guest is ranked as a placement' => sub {
+    my $planner = planner(
+        'dynamic',
+        {
+            node1 => { maxcpu => 8, cpu => 1, maxmem => 16 * $GB, mem => 2 * $GB },
+            node2 => { maxcpu => 8, cpu => 6, maxmem => 16 * $GB, mem => 12 * $GB },
+            node3 => { maxcpu => 8, cpu => 0, maxmem => 16 * $GB, mem => 0 },
+        },
+        {
+            'vm:100' => guest('node2', cpu => 6, mem => 12 * $GB),
+            'vm:200' => {
+                node => 'node1',
+                running => 0,
+                usage => { maxcpu => 8, cpu => 0, maxmem => 16 * $GB, mem => 0 },
+            },
+        },
+    );
+
+    my $ranked = $planner->rank_target_nodes('vm:200');
+
+    is_deeply(
+        [map { $_->{node} } @$ranked],
+        ['node3', 'node2'],
+        'placement ranking excludes the current node and prefers the emptier node',
+    );
+    ok(!(grep { defined($_->{imbalance}) } @$ranked), 'placement entries carry no imbalance');
+};
+
+subtest 'rank_target_nodes: candidate restriction' => sub {
+    my $planner = PVE::ResourceScheduling::Planner->new(
+        mode => 'dynamic',
+        nodes => [qw(node1 node2 node3 node4 node5)],
+        node_stats => {
+            nodes(4, cpu => 1, mem => 2 * $GB)->%*,
+            node5 => { maxcpu => 0, maxmem => 16 * $GB }, # unusable, skipped
+        },
+        guest_stats => {
+            'vm:100' => guest('node1', cpu => 2, mem => 4 * $GB),
+            'vm:101' => { guest('node2')->%*, running => 0 },
+        },
+    );
+
+    my $restriction = [qw(node3 node3 node2 node1 node5 nosuchnode)];
+
+    is_deeply(
+        [map { $_->{node} } $planner->rank_target_nodes('vm:100', $restriction)->@*],
+        ['node2', 'node3'],
+        'migration ranking keeps only the listed snapshot nodes other than the current one, once',
+    );
+    is_deeply(
+        [sort map { $_->{node} } $planner->rank_target_nodes('vm:101', $restriction)->@*],
+        ['node1', 'node3'],
+        'placement ranking keeps only the listed snapshot nodes other than the current one, once',
+    );
+    is_deeply($planner->rank_target_nodes('vm:100', []), [], 'empty restriction ranks nothing');
+    is_deeply(
+        $planner->rank_target_nodes('vm:101', ['node2']),
+        [],
+        'restriction to the current node ranks nothing',
+    );
+};
+
+subtest 'rank_target_nodes: a running guest on an offline node falls back to placement' => sub {
+    # the guest's node is offline (absent from the snapshot), so it must rank as a placement
+    my $planner = planner(
+        'dynamic',
+        {
+            node1 => { maxcpu => 8, cpu => 6, maxmem => 16 * $GB, mem => 12 * $GB },
+            node2 => { maxcpu => 8, cpu => 0, maxmem => 16 * $GB, mem => 0 },
+        },
+        {
+            'vm:100' => guest('offline-node', cpu => 2, mem => 4 * $GB),
+        },
+    );
+
+    my $ranked = eval { $planner->rank_target_nodes('vm:100') };
+    ok(!$@, 'ranking a guest on an offline node does not die') or diag $@;
+    is_deeply(
+        [map { $_->{node} } @$ranked],
+        ['node2', 'node1'],
+        'placement ranking covers all snapshot nodes, emptier first',
+    );
+    ok(!(grep { defined($_->{imbalance}) } @$ranked), 'placement entries carry no imbalance');
+};
+
+subtest 'rank_target_nodes: dies for an unknown sid' => sub {
+    my $planner = planner('dynamic', nodes(1));
+    eval { $planner->rank_target_nodes('vm:999') };
+    like($@, qr/no such guest 'vm:999'/, 'an unknown sid is rejected');
+};
+
+subtest 'nodes without usable stats are skipped' => sub {
+    my $planner = PVE::ResourceScheduling::Planner->new(
+        mode => 'dynamic',
+        nodes => [qw(node1 node2 node3 node4 node5 node6)],
+        node_stats => {
+            node1 => { maxcpu => 8, cpu => 6, maxmem => 16 * $GB, mem => 12 * $GB },
+            node2 => { maxcpu => 8, maxmem => 16 * $GB }, # no measured load: idle
+            # node3 has no stats at all
+            node4 => { maxcpu => 0, cpu => 0, maxmem => 16 * $GB, mem => 0 },
+            node5 => { maxcpu => 8, cpu => 0, maxmem => 0, mem => 0 },
+            node6 => { cpu => 0, maxmem => 16 * $GB, mem => 0 },
+            offline => undef, # not requested, e.g. undecodable info of an offline node
+        },
+        guest_stats => {
+            'vm:100' => guest('node1', cpu => 3, mem => 6 * $GB),
+            'vm:101' => guest('node1', cpu => 3, mem => 6 * $GB),
+            'vm:300' => guest('node3', cpu => 2, mem => 4 * $GB),
+        },
+    );
+
+    is_deeply(
+        [map { $_->{node} } $planner->rank_target_nodes('vm:100')->@*],
+        ['node2'],
+        'skipped nodes are no migration candidates',
+    );
+
+    my $ranked = $planner->rank_target_nodes('vm:300');
+    ok(
+        !(grep { defined($_->{imbalance}) } @$ranked),
+        'a guest on a skipped node is ranked as a placement',
+    );
+    is_deeply(
+        [sort map { $_->{node} } @$ranked],
+        [qw(node1 node2)],
+        'placement covers the usable nodes only',
+    );
+
+    my $plan = eval { $planner->suggest_rebalance([qw(vm:100 vm:101 vm:300)]) };
+    ok(!$@, 'rebalancing with an undef stats entry for an unrequested node works') or diag $@;
+    is_deeply(
+        $plan->{moves},
+        [{ sid => 'vm:100', source => 'node1', target => 'node2' }],
+        'rebalance only moves between usable nodes and leaves the skipped node\'s guest alone',
+    );
+};
+
+subtest 'the snapshot is copied on construction' => sub {
+    my $node_stats = nodes(3);
+    $node_stats->{node1}->@{qw(cpu mem)} = (6, 12 * $GB);
+    my $guest_stats = {};
+    $guest_stats->{"vm:10$_"} = guest('node1', cpu => 2, mem => 4 * $GB) for 0 .. 2;
+
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $imbalance = $planner->node_imbalance();
+    my $plan = $planner->suggest_rebalance([sort keys %$guest_stats]);
+    is(scalar($plan->{moves}->@*), 2, 'the stacked guests are spread out');
+
+    # changes the planner must not see, neither in its initial scheduler nor in rebuilt ones
+    $node_stats->{node2}->{mem} = 15 * $GB;
+    for my $guest (values %$guest_stats) {
+        $guest->{node} = 'node3';
+        $guest->{usage}->@{qw(cpu mem)} = (0, 0);
+    }
+
+    is($planner->node_imbalance(), $imbalance, 'imbalance unaffected by caller changes');
+    plan_is(
+        $planner->suggest_rebalance([sort keys %$guest_stats]),
+        $plan,
+        'proposal unaffected by caller changes',
+    );
+};
+
+subtest 'suggest_rebalance: greedily descends toward balance, state-free' => sub {
+    my $guest_stats = {};
+    $guest_stats->{"vm:10$_"} = guest('node1', cpu => 1, mem => 2 * $GB) for (0 .. 5);
+
+    my $node_stats = nodes(3);
+    $node_stats->{node1}->{cpu} = 6;
+    $node_stats->{node1}->{mem} = 12 * $GB;
+
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $imbalance = $planner->node_imbalance();
+
+    my $plan = $planner->suggest_rebalance([sort keys %$guest_stats]);
+
+    is($plan->{mode}, 'dynamic', 'the mode is reported');
+    is(scalar($plan->{moves}->@*), 4, 'two guests move to each of the idle nodes');
+    ok(abs($plan->{imbalance}->{before} - $imbalance) < $EPS,
+        'before is the current imbalance');
+    ok($plan->{imbalance}->{after} < $EPS, 'the proposal balances the cluster');
+    ok(
+        !(grep { $_->{source} ne 'node1' || $_->{target} eq 'node1' } $plan->{moves}->@*),
+        'all moves go from the stacked node to another node',
+    );
+
+    # proposing works on a copy: the planner's own state is untouched, so asking again yields the
+    # same proposal
+    is($planner->node_imbalance(), $imbalance, 'planner imbalance unchanged by the proposal');
+    plan_is(
+        $planner->suggest_rebalance([sort keys %$guest_stats]),
+        $plan,
+        'a second proposal is identical',
+    );
+
+    my $strict = $planner->suggest_rebalance([sort keys %$guest_stats], min_improvement => 0.9);
+    is_deeply($strict->{moves}, [], 'no move taken when none clears a high min_improvement');
+    is(
+        $strict->{imbalance}->{after},
+        $strict->{imbalance}->{before},
+        'after equals before without moves',
+    );
+
+    my $restricted = $planner->suggest_rebalance(['vm:100', 'vm:999']);
+    is_deeply(
+        [map { $_->{sid} } $restricted->{moves}->@*],
+        ['vm:100'],
+        'only the movable guests are moved, unknown sids are ignored',
+    );
+};
+
+subtest 'suggest_rebalance: proposals replay to the reported imbalance' => sub {
+    for my $mode (qw(dynamic static)) {
+        for my $shape ([3, 30], [5, 60]) {
+            my ($node_stats, $guest_stats) = synthetic_cluster(@$shape);
+            if ($mode eq 'static') {
+                $node_stats =
+                    { map { $_ => { $node_stats->{$_}->%{qw(maxcpu maxmem)} } } keys %$node_stats };
+                $guest_stats->{$_}->{usage} =
+                    { $guest_stats->{$_}->{usage}->%{qw(maxcpu maxmem)} }
+                    for keys %$guest_stats;
+            }
+            my $planner = planner($mode, $node_stats, $guest_stats);
+
+            for my $method (qw(bruteforce topsis)) {
+                my $plan = $planner->suggest_rebalance(
+                    [sort keys %$guest_stats],
+                    max_moves => 50,
+                    method => $method,
+                );
+                my $name = "$mode/$method, $shape->[0] nodes, $shape->[1] guests";
+
+                my $moves = $plan->{moves};
+                my %sids = map { $_->{sid} => 1 } @$moves;
+                is(scalar(keys %sids), scalar(@$moves), "$name: one move per guest");
+                ok(
+                    !(grep { $_->{source} eq $_->{target} } @$moves),
+                    "$name: no move ends on its source",
+                );
+                ok(
+                    !(grep { !$guest_stats->{ $_->{sid} }->{running} } @$moves),
+                    "$name: only running guests are moved",
+                );
+
+                my $replayed =
+                    replay($mode, $node_stats, $guest_stats, $moves)->node_imbalance();
+                ok(
+                    abs($replayed - $plan->{imbalance}->{after}) < $EPS,
+                    "$name: after matches the replayed proposal",
+                ) or diag("after $plan->{imbalance}->{after}, replayed $replayed");
+                cmp_ok(
+                    $plan->{imbalance}->{after},
+                    '<',
+                    $plan->{imbalance}->{before},
+                    "$name: the proposal improves the imbalance",
+                );
+            }
+        }
+    }
+};
+
+subtest 'suggest_rebalance: a guest moved repeatedly is reported once' => sub {
+    # without overhead, the greedy steps are vm:103 node1->node2, vm:104 node1->node3, vm:101
+    # node4->node3 and vm:104 node3->node2
+    my $usage =
+        sub { return (maxcpu => 4, maxmem => 32 * $GB, cpu => $_[0], mem => $_[1] * $GB) };
+    my $guest_stats = {
+        'vm:101' => guest('node4', $usage->(7, 7)),
+        'vm:102' => guest('node4', $usage->(6, 13)),
+        'vm:103' => guest('node1', $usage->(6, 6)),
+        'vm:104' => guest('node1', $usage->(6, 4)),
+        'vm:105' => guest('node1', $usage->(2, 15)),
+        'vm:106' => guest('node1', $usage->(3, 12)),
+    };
+    my $node_stats = nodes(4, maxcpu => 16, maxmem => 64 * $GB);
+    for my $guest (values %$guest_stats) {
+        $node_stats->{ $guest->{node} }->{cpu} += $guest->{usage}->{cpu};
+        $node_stats->{ $guest->{node} }->{mem} += $guest->{usage}->{mem};
+    }
+
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $plan = $planner->suggest_rebalance([sort keys %$guest_stats]);
+
+    is_deeply(
+        $plan->{moves},
+        [
+            { sid => 'vm:103', source => 'node1', target => 'node2' },
+            { sid => 'vm:104', source => 'node1', target => 'node2' },
+            { sid => 'vm:101', source => 'node4', target => 'node3' },
+        ],
+        'the double hop collapses into one move from the original to the final node',
+    );
+
+    my $replayed =
+        replay('dynamic', $node_stats, $guest_stats, $plan->{moves})->node_imbalance();
+    ok(abs($replayed - $plan->{imbalance}->{after}) < $EPS,
+        'after matches the collapsed moves');
+
+    # the step cap counts greedy steps, collapsed moves can be fewer
+    is(
+        scalar(
+            $planner->suggest_rebalance([sort keys %$guest_stats], max_moves => 4)->{moves}->@*
+        ),
+        3,
+        'four steps yield three moves',
+    );
+};
+
+subtest 'suggest_rebalance: no move in an already balanced cluster' => sub {
+    # moving any guest only permutes the node loads, which a truncated prediction could make look
+    # like a tiny improvement
+    my $node_stats = { map { ("node$_" => { maxcpu => 16, maxmem => 64 * $GB }) } 1 .. 3 };
+    my $guest_stats = {
+        map { (
+            "vm:"
+                . (100 + $_) => {
+                    node => 'node' . (1 + $_ % 3),
+                    running => 1,
+                    usage => { maxcpu => 2, maxmem => 3 * $GB },
+                },
+        ) } 0 .. 6
+    };
+    my $plan =
+        planner('static', $node_stats, $guest_stats)->suggest_rebalance([sort keys %$guest_stats]);
+    is_deeply($plan->{moves}, [], 'no move for a load permutation');
+
+    # moving a guest without any usage changes nothing
+    my $idle = planner(
+        'dynamic',
+        {
+            node1 => { maxcpu => 8, cpu => 1.7, maxmem => 16 * $GB, mem => 5 * $GB },
+            node2 => { maxcpu => 8, cpu => 4.3, maxmem => 16 * $GB, mem => 3 * $GB },
+            node3 => { maxcpu => 8, cpu => 2.9, maxmem => 16 * $GB, mem => 9 * $GB },
+        },
+        { 'vm:100' => guest('node1') },
+    );
+    is_deeply($idle->suggest_rebalance(['vm:100'])->{moves}, [], 'no move for an idle guest');
+};
+
+subtest 'suggest_rebalance: a mispredicted move is not taken' => sub {
+    # The static backend accounts a container without CPU limit with the full core count of its
+    # node, but migrates it as if it used no CPU. The second greedy step here is predicted to
+    # improve the imbalance, but actually worsens it. Its predicted gain is small compared to the
+    # first step, so disable the step ratio guard to see the actual-imbalance check at work.
+    my $node_stats = {
+        node1 => { maxcpu => 16, maxmem => 64 * $GB },
+        node2 => { maxcpu => 16, maxmem => 64 * $GB },
+        node3 => { maxcpu => 64, maxmem => 64 * $GB },
+    };
+    my $guest_stats = {
+        'ct:100' =>
+            { node => 'node1', running => 1, usage => { maxcpu => 0, maxmem => 4 * $GB } },
+        'ct:101' =>
+            { node => 'node1', running => 1, usage => { maxcpu => 0, maxmem => 4 * $GB } },
+        'vm:102' =>
+            { node => 'node2', running => 1, usage => { maxcpu => 4, maxmem => 8 * $GB } },
+    };
+
+    my $plan = planner('static', $node_stats, $guest_stats)
+        ->suggest_rebalance([sort keys %$guest_stats], min_step_ratio => 0);
+
+    is_deeply(
+        $plan->{moves},
+        [{ sid => 'ct:100', source => 'node1', target => 'node3' }],
+        'the proposal stops before the worsening step',
+    );
+    my $replayed =
+        replay('static', $node_stats, $guest_stats, $plan->{moves})->node_imbalance();
+    ok(abs($replayed - $plan->{imbalance}->{after}) < $EPS, 'after is the actual imbalance');
+
+    # The container is the best pick here, but moving it changes nothing, while a VM move pays off.
+    $node_stats = { map { ("node$_" => { maxcpu => 16, maxmem => 64 * $GB }) } 1 .. 5 };
+    $guest_stats = {
+        'ct:100' =>
+            { node => 'node5', running => 1, usage => { maxcpu => 0, maxmem => 10 * $GB } },
+        'vm:101' =>
+            { node => 'node3', running => 1, usage => { maxcpu => 1, maxmem => 11 * $GB } },
+        'vm:102' =>
+            { node => 'node3', running => 1, usage => { maxcpu => 4, maxmem => 4 * $GB } },
+    };
+    $plan =
+        planner('static', $node_stats, $guest_stats)->suggest_rebalance([sort keys %$guest_stats]);
+    is_deeply(
+        $plan->{moves},
+        [{ sid => 'vm:101', source => 'node3', target => 'node1' }],
+        'a step tries the next best guest after an undone move',
+    );
+};
+
+subtest 'suggest_rebalance: step limit and option validation' => sub {
+    my $guest_stats = {};
+    $guest_stats->{ "vm:" . (100 + $_) } = guest('node1', cpu => 1, mem => 1 * $GB) for 1 .. 30;
+    my $node_stats = nodes(3, maxcpu => 64, maxmem => 128 * $GB);
+    $node_stats->{node1}->@{qw(cpu mem)} = (30, 30 * $GB);
+
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+
+    is(PVE::ResourceScheduling::Planner::DEFAULT_MAX_MOVES, 10, 'the default step limit is 10');
+    is(PVE::ResourceScheduling::Planner::DEFAULT_MIN_STEP_RATIO, 0.1, 'the default step ratio');
+    is(
+        PVE::ResourceScheduling::Planner::DEFAULT_MIN_TOTAL_RATIO,
+        0.1,
+        'the default total ratio',
+    );
+    is(
+        PVE::ResourceScheduling::Planner::DEFAULT_BALANCED_IMBALANCE,
+        0.1,
+        'the default balanced imbalance',
+    );
+    is(
+        PVE::ResourceScheduling::Planner::DEFAULT_BUSIEST_NODES,
+        10,
+        'the default candidate nodes',
+    );
+    is(
+        scalar($planner->suggest_rebalance($movable)->{moves}->@*),
+        PVE::ResourceScheduling::Planner::DEFAULT_MAX_MOVES,
+        'the default step limit applies',
+    );
+
+    my $full = $planner->suggest_rebalance($movable, max_moves => 25);
+    is(scalar($full->{moves}->@*), 20, 'a higher limit balances the cluster completely');
+    ok($full->{imbalance}->{after} < $EPS, 'the full proposal reaches balance');
+
+    for my $max_moves (0, -1, 1.5, 'abc') {
+        eval { $planner->suggest_rebalance($movable, max_moves => $max_moves) };
+        like($@, qr/invalid max_moves/, "max_moves '$max_moves' is rejected");
+    }
+    for my $min_improvement (-0.1, 'abc', 'nan') {
+        eval { $planner->suggest_rebalance($movable, min_improvement => $min_improvement) };
+        like($@, qr/invalid min_improvement/, "min_improvement '$min_improvement' is rejected");
+    }
+    for my $ratio (-0.1, 1.5, 'abc', 'nan') {
+        eval { $planner->suggest_rebalance($movable, min_step_ratio => $ratio) };
+        like($@, qr/invalid min_step_ratio/, "min_step_ratio '$ratio' is rejected");
+    }
+    for my $total (-0.1, 1.5, 'abc', 'nan') {
+        eval { $planner->suggest_rebalance($movable, min_total_ratio => $total) };
+        like($@, qr/invalid min_total_ratio/, "min_total_ratio '$total' is rejected");
+    }
+    for my $balanced (-0.1, 1.5, 'abc', 'nan') {
+        eval { $planner->suggest_rebalance($movable, balanced_imbalance => $balanced) };
+        like($@, qr/invalid balanced_imbalance/, "balanced_imbalance '$balanced' is rejected");
+    }
+    for my $limit (-1, 1.5, 'abc') {
+        eval { $planner->suggest_rebalance($movable, busiest_nodes => $limit) };
+        like($@, qr/invalid busiest_nodes/, "busiest_nodes '$limit' is rejected");
+    }
+    for my $allowed ('node2', ['node2'], { 'vm:101' => 'node2' }) {
+        eval { $planner->suggest_rebalance($movable, allowed_targets => $allowed) };
+        like($@, qr/invalid allowed_targets/, 'malformed allowed_targets are rejected');
+    }
+    eval { $planner->suggest_rebalance($movable, method => 'bogus') };
+    like($@, qr/invalid method/, 'an unknown method is rejected');
+};
+
+subtest 'suggest_rebalance: no proposal for a balanced cluster with a negligible gain' => sub {
+    # a nearly balanced cluster whose slight imbalance comes from host overhead on node1, where
+    # moving its two tiny guests would only lower the imbalance a bit
+    my $usage =
+        sub { return { maxcpu => 2, maxmem => 4 * $GB, cpu => 0.02, mem => int(0.02 * $GB) } };
+    my $guest_stats = {
+        'ct:101' => { node => 'node1', running => 1, usage => $usage->() },
+        'ct:102' => { node => 'node1', running => 1, usage => $usage->() },
+    };
+    my $node_stats = {
+        node1 => { maxcpu => 8, cpu => 2.54, maxmem => 16 * $GB, mem => int(5.04 * $GB) },
+        node2 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB },
+        node3 => { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB },
+    };
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+
+    my $unguarded = $planner->suggest_rebalance($movable, min_total_ratio => 0);
+    cmp_ok(scalar($unguarded->{moves}->@*), '>', 0, 'improving migrations exist');
+    my $gain = $unguarded->{imbalance}->{before} - $unguarded->{imbalance}->{after};
+    my $ratio = $gain / $unguarded->{imbalance}->{before};
+    cmp_ok(
+        $ratio,
+        '<',
+        PVE::ResourceScheduling::Planner::DEFAULT_MIN_TOTAL_RATIO,
+        'but they lower the imbalance by less than the default ratio',
+    );
+
+    my $plan = $planner->suggest_rebalance($movable);
+    cmp_ok(
+        $plan->{imbalance}->{before},
+        '<',
+        PVE::ResourceScheduling::Planner::DEFAULT_BALANCED_IMBALANCE,
+        'the cluster counts as balanced',
+    );
+    is_deeply($plan->{moves}, [], 'no migration is proposed');
+    is($plan->{imbalance}->{after}, $plan->{imbalance}->{before}, 'after equals before');
+
+    my $lower = $planner->suggest_rebalance($movable, min_total_ratio => $ratio / 2);
+    plan_is($lower, $unguarded, 'a lower ratio keeps the proposal');
+    plan_is(
+        $planner->suggest_rebalance($movable, balanced_imbalance => 0),
+        $unguarded,
+        'a zero balanced imbalance disables the guard',
+    );
+};
+
+subtest 'suggest_rebalance: no proposal for moving a nearly idle guest' => sub {
+    # node1 is loaded by a single guest, which would overload any other node just as much, so
+    # only moving the nearly idle guest improves the imbalance at all
+    my $guest_stats = {
+        'ct:100' => guest('node1', cpu => 2, mem => $GB),
+        'ct:101' => guest('node2', cpu => 0.001, mem => int(0.01 * $GB)),
+    };
+    my $node_stats = {
+        node1 => { maxcpu => 4, cpu => 2.1, maxmem => 4 * $GB, mem => 3 * $GB },
+        node2 => { maxcpu => 4, cpu => 0.4, maxmem => 4 * $GB, mem => 3 * $GB },
+        node3 => { maxcpu => 4, cpu => 0.2, maxmem => 4 * $GB, mem => 3 * $GB },
+    };
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+
+    my $plan = $planner->suggest_rebalance($movable);
+    cmp_ok(
+        $plan->{imbalance}->{before},
+        '>=',
+        PVE::ResourceScheduling::Planner::DEFAULT_BALANCED_IMBALANCE,
+        'the cluster does not count as balanced',
+    );
+    is_deeply($plan->{moves}, [], 'no migration is proposed');
+    is($plan->{imbalance}->{after}, $plan->{imbalance}->{before}, 'after equals before');
+};
+
+subtest 'suggest_rebalance: a small gain is kept for an imbalanced cluster' => sub {
+    # node1 is overloaded by a guest that may not be moved, only two small guests may
+    my $small =
+        sub { return { maxcpu => 2, maxmem => 4 * $GB, cpu => 0.1, mem => int(0.1 * $GB) } };
+    my $guest_stats = {
+        'vm:100' => guest('node1', cpu => 4, mem => 8 * $GB),
+        'ct:101' => { node => 'node1', running => 1, usage => $small->() },
+        'ct:102' => { node => 'node1', running => 1, usage => $small->() },
+    };
+    my $node_stats = {
+        node1 => { maxcpu => 8, cpu => 5.2, maxmem => 16 * $GB, mem => int(10.2 * $GB) },
+        node2 => { maxcpu => 8, cpu => 1, maxmem => 16 * $GB, mem => 2 * $GB },
+        node3 => { maxcpu => 8, cpu => 1, maxmem => 16 * $GB, mem => 2 * $GB },
+    };
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = ['ct:101', 'ct:102'];
+
+    my $plan = $planner->suggest_rebalance($movable);
+    cmp_ok(
+        $plan->{imbalance}->{before},
+        '>=',
+        PVE::ResourceScheduling::Planner::DEFAULT_BALANCED_IMBALANCE,
+        'the cluster does not count as balanced',
+    );
+    cmp_ok(scalar($plan->{moves}->@*), '>', 0, 'the small guests are moved');
+    my ($before, $after) = $plan->{imbalance}->@{qw(before after)};
+    cmp_ok(
+        ($before - $after) / $before,
+        '<',
+        PVE::ResourceScheduling::Planner::DEFAULT_MIN_TOTAL_RATIO,
+        'although the gain is small',
+    );
+    is_deeply(
+        $planner->suggest_rebalance($movable, balanced_imbalance => 1)->{moves},
+        [],
+        'the same gain is dropped where the cluster counts as balanced',
+    );
+};
+
+subtest 'suggest_rebalance: the tail after a big step is cut' => sub {
+    # without overhead, the first step gains about 0.36 and the next two about 0.03 each
+    my $usage =
+        sub { return (maxcpu => 4, maxmem => 8 * $GB, cpu => $_[0], mem => $_[1] * $GB) };
+    my $guest_stats = {
+        'vm:101' => guest('node3', $usage->(0.1, 1)),
+        'vm:102' => guest('node1', $usage->(1, 0.25)),
+        'vm:103' => guest('node1', $usage->(1, 2)),
+        'vm:104' => guest('node3', $usage->(0.2, 2)),
+        'vm:105' => guest('node3', $usage->(0.5, 6)),
+    };
+    my $node_stats = nodes(3);
+    for my $guest (values %$guest_stats) {
+        $node_stats->{ $guest->{node} }->{cpu} += $guest->{usage}->{cpu};
+        $node_stats->{ $guest->{node} }->{mem} += $guest->{usage}->{mem};
+    }
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+
+    my $unguarded = $planner->suggest_rebalance($movable, min_step_ratio => 0);
+    my $first = $planner->suggest_rebalance($movable, min_step_ratio => 0, max_moves => 1);
+    my $first_gain = $first->{imbalance}->{before} - $first->{imbalance}->{after};
+    my $tail_gain = $first->{imbalance}->{after} - $unguarded->{imbalance}->{after};
+    cmp_ok(scalar($unguarded->{moves}->@*), '>', 1,
+        'more improving steps follow the first one');
+    cmp_ok($tail_gain, '<', 0.2 * $first_gain, 'and gain little compared to it');
+
+    plan_is(
+        $planner->suggest_rebalance($movable),
+        $first,
+        'the default stops after the big step',
+    );
+    plan_is(
+        $planner->suggest_rebalance($movable, min_step_ratio => 0.05),
+        $unguarded,
+        'a lower step ratio keeps the tail',
+    );
+};
+
+subtest 'suggest_rebalance: many similar steps are kept' => sub {
+    my ($node_stats, $guest_stats) = synthetic_cluster(16, 1000);
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+
+    my $plan = $planner->suggest_rebalance($movable);
+    is(
+        scalar($plan->{moves}->@*),
+        PVE::ResourceScheduling::Planner::DEFAULT_MAX_MOVES,
+        'all steps up to the limit are taken',
+    );
+    cmp_ok(
+        $plan->{imbalance}->{after},
+        '<',
+        $plan->{imbalance}->{before},
+        'the proposal improves',
+    );
+    plan_is(
+        $plan,
+        $planner->suggest_rebalance($movable, min_step_ratio => 0, min_total_ratio => 0),
+        'the guards do not change the proposal',
+    );
+};
+
+subtest 'suggest_rebalance: relief for a single overloaded node in a larger cluster' => sub {
+    # One saturated node among half-loaded peers. Its share of the imbalance shrinks with the
+    # number of nodes, so the cluster counts as balanced, and with many small guests one proposal
+    # lowers the imbalance only a little in absolute terms, but the overloaded node needs relief.
+    for my $peers (8, 16, 32) {
+        my ($node_stats, $guest_stats) = ({}, {});
+        my $vmid = 100;
+        for my $i (0 .. $peers) {
+            my $node = sprintf("node%02d", $i + 1);
+            my $guests = $i == 0 ? 128 : 64;
+            $node_stats->{$node} =
+                { maxcpu => 128, cpu => $guests, maxmem => 512 * $GB, mem => 4 * $guests * $GB };
+            $guest_stats->{ 'vm:' . $vmid++ } = guest($node, cpu => 1, mem => 4 * $GB)
+                for 1 .. $guests;
+        }
+
+        my $plan = planner('dynamic', $node_stats, $guest_stats)
+            ->suggest_rebalance([sort keys %$guest_stats]);
+        my ($before, $after) = $plan->{imbalance}->@{qw(before after)};
+
+        is(scalar($plan->{moves}->@*), 10, "$peers peers: the default request takes all steps");
+        ok(
+            !(grep { $_->{source} ne 'node01' } $plan->{moves}->@*),
+            "$peers peers: all moves relieve the overloaded node",
+        );
+        cmp_ok(
+            $before - $after,
+            '>=',
+            PVE::ResourceScheduling::Planner::DEFAULT_MIN_TOTAL_RATIO * $before,
+            "$peers peers: the proposal lowers the imbalance noticeably",
+        );
+    }
+};
+
+subtest 'suggest_rebalance: restricted target nodes' => sub {
+    my $guest_stats = {};
+    $guest_stats->{"vm:10$_"} = guest('node1', cpu => 1, mem => 2 * $GB) for 0 .. 5;
+    my $node_stats = nodes(4);
+    $node_stats->{node1}->@{qw(cpu mem)} = (6, 12 * $GB);
+
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+
+    for my $targets (['node4'], ['node1', 'node4'], ['node3', 'node4', 'nosuchnode']) {
+        my $name = join(',', @$targets);
+        my $wanted = { map { $_ => 1 } @$targets };
+        my $plan = $planner->suggest_rebalance($movable, targets => $targets);
+
+        cmp_ok(scalar($plan->{moves}->@*), '>', 0, "$name: guests are moved");
+        ok(
+            !(grep { !$wanted->{ $_->{target} } } $plan->{moves}->@*),
+            "$name: all moves go to a target node",
+        );
+        ok(
+            !(grep { $_->{source} ne 'node1' } $plan->{moves}->@*),
+            "$name: the overloaded node is drained, whether it is a target or not",
+        );
+        is(
+            $plan->{imbalance}->{before},
+            $planner->node_imbalance(),
+            "$name: before is the imbalance over all nodes",
+        );
+        my $replayed =
+            replay('dynamic', $node_stats, $guest_stats, $plan->{moves})->node_imbalance();
+        ok(
+            abs($replayed - $plan->{imbalance}->{after}) < $EPS,
+            "$name: after is the imbalance over all nodes",
+        );
+    }
+
+    # with only one target node, the idle third node stays idle and the cluster less balanced
+    cmp_ok(
+        $planner->suggest_rebalance($movable, targets => ['node4'])->{imbalance}->{after},
+        '>',
+        $planner->suggest_rebalance($movable)->{imbalance}->{after},
+        'a single target node balances less than all nodes',
+    );
+
+    my $skipping = PVE::ResourceScheduling::Planner->new(
+        mode => 'dynamic',
+        nodes => [qw(node1 node2 node3 node4 node5)],
+        node_stats => { %$node_stats, node5 => { maxcpu => 0, maxmem => 0 } },
+        guest_stats => $guest_stats,
+    );
+    for my $targets ([], ['nosuchnode'], ['node5']) {
+        my $plan = $skipping->suggest_rebalance($movable, targets => $targets);
+        my $name = '[' . join(',', @$targets) . ']';
+        is_deeply($plan->{moves}, [], "$name: no usable target node, no moves");
+        is(
+            $plan->{imbalance}->{after},
+            $plan->{imbalance}->{before},
+            "$name: after equals before",
+        );
+    }
+
+    eval { $planner->suggest_rebalance($movable, targets => 'node4') };
+    like($@, qr/invalid targets/, 'targets must be an array reference');
+};
+
+subtest 'suggest_rebalance: a guest moved twice ends on a target node' => sub {
+    # without overhead and with node2 and node3 as targets, vm:104 first moves to node3 and in the
+    # third step on to node2, which gains about 0.02
+    my $usage =
+        sub { return (maxcpu => 4, maxmem => 32 * $GB, cpu => $_[0], mem => $_[1] * $GB) };
+    my $guest_stats = {
+        'vm:101' => guest('node4', $usage->(4, 4)),
+        'vm:102' => guest('node2', $usage->(2, 6)),
+        'vm:103' => guest('node3', $usage->(1, 11)),
+        'vm:104' => guest('node1', $usage->(2, 3)),
+        'vm:105' => guest('node1', $usage->(7, 13)),
+        'vm:106' => guest('node2', $usage->(1, 14)),
+        'vm:107' => guest('node4', $usage->(5, 9)),
+    };
+    my $node_stats = nodes(4, maxcpu => 16, maxmem => 64 * $GB);
+    for my $guest (values %$guest_stats) {
+        $node_stats->{ $guest->{node} }->{cpu} += $guest->{usage}->{cpu};
+        $node_stats->{ $guest->{node} }->{mem} += $guest->{usage}->{mem};
+    }
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+
+    my $plan = sub {
+        my ($steps) = @_;
+        my $plan = $planner->suggest_rebalance(
+            [sort keys %$guest_stats],
+            targets => ['node2', 'node3'],
+            max_moves => $steps,
+        );
+        return { map { $_->{sid} => $_ } $plan->{moves}->@* }, $plan;
+    };
+
+    my ($two) = $plan->(2);
+    my ($three) = $plan->(3);
+    is_deeply(
+        $two->{'vm:104'},
+        { sid => 'vm:104', source => 'node1', target => 'node3' },
+        'vm:104 first moves to node3',
+    );
+    is_deeply(
+        $three->{'vm:104'},
+        { sid => 'vm:104', source => 'node1', target => 'node2' },
+        'and is then reported with a single move to node2',
+    );
+
+    my (undef, $full) = $plan->(PVE::ResourceScheduling::Planner::DEFAULT_MAX_MOVES);
+    ok(
+        !(grep { $_->{target} ne 'node2' && $_->{target} ne 'node3' } $full->{moves}->@*),
+        'all collapsed moves end on a target node',
+    );
+    my $replayed =
+        replay('dynamic', $node_stats, $guest_stats, $full->{moves})->node_imbalance();
+    ok(abs($replayed - $full->{imbalance}->{after}) < $EPS,
+        'after matches the collapsed moves');
+
+    # the same restriction given per guest yields the same proposals, double hop included
+    my $movable = [sort keys %$guest_stats];
+    my $per_guest = { map { $_ => ['node2', 'node3'] } @$movable };
+    for my $steps (2, 3, PVE::ResourceScheduling::Planner::DEFAULT_MAX_MOVES) {
+        plan_is(
+            $planner->suggest_rebalance(
+                $movable,
+                allowed_targets => $per_guest,
+                max_moves => $steps,
+            ),
+            $planner->suggest_rebalance(
+                $movable,
+                targets => ['node2', 'node3'],
+                max_moves => $steps,
+            ),
+            "$steps steps: a per-guest restriction acts like the same global one",
+        );
+    }
+};
+
+subtest 'suggest_rebalance: per-guest target nodes' => sub {
+    my $guest_stats = {};
+    $guest_stats->{"vm:10$_"} = guest('node1', cpu => 1, mem => 2 * $GB) for 0 .. 5;
+    my $node_stats = nodes(4);
+    $node_stats->{node1}->@{qw(cpu mem)} = (6, 12 * $GB);
+
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+
+    my $allowed = {
+        'vm:100' => ['node3'],
+        'vm:101' => [], # unmovable
+        'vm:102' => ['node1', 'nosuchnode'], # only its current node, unmovable
+        'vm:103' => ['node2', 'node4'],
+    };
+    my $check = sub {
+        my ($name, $plan, $may_go) = @_;
+        my $moved = { map { $_->{sid} => $_->{target} } $plan->{moves}->@* };
+        ok(
+            !(grep { $may_go->{$_} && !$may_go->{$_}->{ $moved->{$_} } } keys %$moved),
+            "$name: restricted guests only move to their allowed nodes",
+        );
+        ok(!$moved->{'vm:101'} && !$moved->{'vm:102'}, "$name: guests without target stay");
+        my $replayed =
+            replay('dynamic', $node_stats, $guest_stats, $plan->{moves})->node_imbalance();
+        ok(
+            abs($replayed - $plan->{imbalance}->{after}) < $EPS,
+            "$name: after matches the moves",
+        );
+        return $moved;
+    };
+
+    my $plan = $planner->suggest_rebalance($movable, allowed_targets => $allowed);
+    my $moved = $check->(
+        'unrestricted',
+        $plan,
+        {
+            'vm:100' => { node3 => 1 },
+            'vm:101' => {},
+            'vm:102' => {},
+            'vm:103' => { node2 => 1, node4 => 1 },
+        },
+    );
+    is($moved->{'vm:100'}, 'node3', 'a guest with one allowed node moves there');
+    cmp_ok(scalar(keys %$moved), '==', 4, 'the other guests balance the cluster');
+
+    my $both = $planner->suggest_rebalance(
+        $movable,
+        allowed_targets => $allowed,
+        targets => ['node4'],
+    );
+    $moved = $check->(
+        'with global targets',
+        $both,
+        {
+            'vm:100' => {},
+            'vm:101' => {},
+            'vm:102' => {},
+            'vm:103' => { node4 => 1 },
+            map { ("vm:10$_" => { node4 => 1 }) } 4 .. 5,
+        },
+    );
+    ok(!$moved->{'vm:100'}, 'per-guest and global restrictions intersect');
+};
+
+subtest 'suggest_rebalance: candidate pruning' => sub {
+    # stacked guests, so the busiest node and the least loaded ones are the only sensible choices
+    my $guest_stats = {};
+    $guest_stats->{"vm:10$_"} = guest('node1', cpu => 1, mem => 2 * $GB) for 0 .. 5;
+    my $node_stats = nodes(4);
+    $node_stats->{node1}->@{qw(cpu mem)} = (6, 12 * $GB);
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+
+    plan_is(
+        $planner->suggest_rebalance($movable, busiest_nodes => 1),
+        $planner->suggest_rebalance($movable, busiest_nodes => 0),
+        'pruning to the busiest node yields the same proposal',
+    );
+
+    # a restricted guest gets the least loaded of its own allowed nodes, not none
+    is_deeply(
+        $planner->suggest_rebalance(
+            ['vm:100'],
+            busiest_nodes => 1,
+            allowed_targets => { 'vm:100' => ['node4'] },
+        )->{moves},
+        [{ sid => 'vm:100', source => 'node1', target => 'node4' }],
+        'pruning picks among the allowed nodes of a restricted guest',
+    );
+
+    # The busiest node only holds a guest too big to move anywhere, so a step pruned to it finds
+    # nothing, while moving the small guests of the second busiest node helps.
+    my $fallback_guests = {
+        'vm:100' => guest('node1', cpu => 6, mem => 12 * $GB),
+        map { ("ct:10$_" => guest('node2', cpu => 1, mem => 2 * $GB)) } 1 .. 4,
+    };
+    my $fallback_nodes = {
+        node1 => { maxcpu => 8, cpu => 6, maxmem => 16 * $GB, mem => 12 * $GB },
+        node2 => { maxcpu => 8, cpu => 4, maxmem => 16 * $GB, mem => 8 * $GB },
+        node3 => { maxcpu => 8, cpu => 0.8, maxmem => 16 * $GB, mem => int(1.6 * $GB) },
+    };
+    my $fallback = planner('dynamic', $fallback_nodes, $fallback_guests);
+    my $fallback_movable = [sort keys %$fallback_guests];
+    my $pruned = $fallback->suggest_rebalance($fallback_movable, busiest_nodes => 1);
+    cmp_ok(
+        scalar($pruned->{moves}->@*),
+        '>',
+        0,
+        'a pruned step without result is retried in full',
+    );
+    is($pruned->{moves}->[0]->{source}, 'node2', 'the retry finds the small guests');
+    plan_is(
+        $pruned,
+        $fallback->suggest_rebalance($fallback_movable, busiest_nodes => 0),
+        'and yields the same proposal as without pruning',
+    );
+
+    # On nodes of different size, the best target need not be the least loaded node: moving a
+    # guest from the small, busy node1 to the small node3 beats the large but idle node2 here.
+    my $sized_guests = {};
+    $sized_guests->{"vm:10$_"} = guest('node1', cpu => 1, mem => 2 * $GB) for 0 .. 5;
+    my $sized_nodes = {
+        node1 => { maxcpu => 8, cpu => 6, maxmem => 16 * $GB, mem => 12 * $GB },
+        node2 => { maxcpu => 64, cpu => 3.2, maxmem => 128 * $GB, mem => int(6.4 * $GB) },
+        node3 => { maxcpu => 8, cpu => 2.4, maxmem => 16 * $GB, mem => int(4.8 * $GB) },
+    };
+    my $sized = planner('dynamic', $sized_nodes, $sized_guests);
+    my $sized_movable = [sort keys %$sized_guests];
+    my $sized_step =
+        $sized->suggest_rebalance($sized_movable, busiest_nodes => 1, max_moves => 1);
+    is(
+        $sized_step->{moves}->[0]->{target},
+        'node3',
+        'the busier node of another size is chosen',
+    );
+    my $sized_plan = $sized->suggest_rebalance($sized_movable, busiest_nodes => 1);
+    plan_is(
+        $sized_plan,
+        $sized->suggest_rebalance($sized_movable, busiest_nodes => 0),
+        'pruning offers the least loaded node of every size and loses nothing',
+    );
+
+    # pruning takes effect with more nodes than its limit
+    my ($synth_nodes, $synth_guests) = synthetic_cluster(16, 300);
+    my $synth = planner('dynamic', $synth_nodes, $synth_guests);
+    my $plan = $synth->suggest_rebalance([sort keys %$synth_guests], busiest_nodes => 3);
+    cmp_ok(scalar($plan->{moves}->@*), '>', 0, 'a pruned proposal on a larger cluster');
+    my $replayed =
+        replay('dynamic', $synth_nodes, $synth_guests, $plan->{moves})->node_imbalance();
+    ok(abs($replayed - $plan->{imbalance}->{after}) < $EPS, 'after matches the pruned moves');
+    plan_is(
+        $plan,
+        $synth->suggest_rebalance([sort keys %$synth_guests], busiest_nodes => 0),
+        'and is the same as without pruning',
+    );
+
+    # TOPSIS ranks the candidates relative to each other, so it always gets all of them
+    plan_is(
+        $synth->suggest_rebalance(
+            [sort keys %$synth_guests],
+            method => 'topsis',
+            busiest_nodes => 1,
+        ),
+        $synth->suggest_rebalance(
+            [sort keys %$synth_guests],
+            method => 'topsis',
+            busiest_nodes => 0,
+        ),
+        'topsis proposals are not pruned',
+    );
+};
+
+subtest 'suggest_rebalance: guests kept apart' => sub {
+    # two busy nodes and an idle one; each of the two guests may move to node3 on its own, but
+    # they must not end up there together
+    my $node_stats = {
+        node1 => { maxcpu => 16, cpu => 14, maxmem => 64 * $GB, mem => 56 * $GB },
+        node2 => { maxcpu => 16, cpu => 14, maxmem => 64 * $GB, mem => 56 * $GB },
+        node3 => { maxcpu => 16, cpu => 0, maxmem => 64 * $GB, mem => 0 },
+    };
+    my $guest_stats = {
+        'vm:101' => guest('node1', cpu => 4, mem => 16 * $GB),
+        'vm:102' => guest('node2', cpu => 4, mem => 16 * $GB),
+    };
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+    my $allowed = { 'vm:101' => ['node3'], 'vm:102' => ['node3'] };
+    my $separate = { 'vm:101' => ['vm:102'], 'vm:102' => ['vm:101'] };
+
+    my $unseparated = $planner->suggest_rebalance($movable, allowed_targets => $allowed);
+    is_deeply(
+        [sort map { $_->{target} } $unseparated->{moves}->@*],
+        ['node3', 'node3'],
+        'without separation, both guests move to the idle node',
+    );
+
+    for my $map ($separate, { 'vm:101' => ['vm:102'] }, { 'vm:102' => ['vm:101'] }) {
+        my $name = join(',', sort keys %$map) . ' listed';
+        my $plan = $planner->suggest_rebalance(
+            $movable,
+            allowed_targets => $allowed,
+            separate => $map,
+        );
+        is(scalar($plan->{moves}->@*), 1, "$name: only one of them moves to node3");
+        is_deeply(
+            shared_nodes($guest_stats, $plan->{moves}, $separate),
+            [],
+            "$name: kept apart",
+        );
+    }
+
+    my $free = $planner->suggest_rebalance($movable, separate => $separate);
+    cmp_ok(scalar($free->{moves}->@*), '>', 0, 'without node restrictions guests still move');
+    is_deeply(
+        shared_nodes($guest_stats, $free->{moves}, $separate),
+        [],
+        'and are kept apart on any node',
+    );
+    my $replayed =
+        replay('dynamic', $node_stats, $guest_stats, $free->{moves})->node_imbalance();
+    ok(abs($replayed - $free->{imbalance}->{after}) < $EPS, 'after matches the moves');
+
+    # a partner that may not move itself still blocks its node
+    my $pinned = $planner->suggest_rebalance(
+        ['vm:101'],
+        separate => { 'vm:101' => ['vm:102'], 'vm:102' => ['vm:999'] },
+        targets => ['node2'],
+    );
+    is_deeply($pinned->{moves}, [], 'the node of an unmovable partner is never a target');
+
+    eval { $planner->suggest_rebalance($movable, separate => 'vm:102') };
+    like($@, qr/invalid separate/, 'separate must be a hash reference');
+    for my $bad (['vm:102'], { 'vm:101' => 'vm:102' }) {
+        eval { $planner->suggest_rebalance($movable, separate => $bad) };
+        like($@, qr/invalid separate/, 'separate must map to array references');
+    }
+};
+
+subtest 'suggest_rebalance: three guests kept apart from each other' => sub {
+    my $node_stats = nodes(4, maxcpu => 16, maxmem => 64 * $GB);
+    my $guest_stats = {};
+    for my $i (1 .. 3) {
+        my $node = "node$i";
+        $guest_stats->{"vm:10$i"} = guest($node, cpu => 4, mem => 16 * $GB);
+        $guest_stats->{"vm:20$i"} = guest($node, cpu => 10, mem => 40 * $GB); # unmovable load
+        $node_stats->{$node}->@{qw(cpu mem)} = (14, 56 * $GB);
+    }
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = ['vm:101', 'vm:102', 'vm:103'];
+
+    my $unseparated = $planner->suggest_rebalance($movable);
+    cmp_ok(
+        scalar(grep { $_->{target} eq 'node4' } $unseparated->{moves}->@*),
+        '>',
+        1,
+        'without separation, several guests move to the idle node',
+    );
+
+    my $mutual = {
+        'vm:101' => ['vm:102', 'vm:103'],
+        'vm:102' => ['vm:101', 'vm:103'],
+        'vm:103' => ['vm:101', 'vm:102'],
+    };
+    my $one_sided = { 'vm:101' => ['vm:102', 'vm:103'], 'vm:102' => ['vm:103'] };
+    for my $map ($mutual, $one_sided) {
+        my $name = $map == $mutual ? 'mutual' : 'one-sided';
+        my $plan = $planner->suggest_rebalance($movable, separate => $map);
+        is(
+            scalar(grep { $_->{target} eq 'node4' } $plan->{moves}->@*),
+            1,
+            "$name: only one of them moves to the idle node",
+        );
+        is_deeply(shared_nodes($guest_stats, $plan->{moves}, $mutual), [], "$name: kept apart");
+    }
+};
+
+subtest 'suggest_rebalance: separation with double hops, pruning and undone moves' => sub {
+    # the double hop scenario: vm:104 moves to node3 and then on to node2, where vm:102 is
+    my $usage =
+        sub { return (maxcpu => 4, maxmem => 32 * $GB, cpu => $_[0], mem => $_[1] * $GB) };
+    my $guest_stats = {
+        'vm:101' => guest('node4', $usage->(4, 4)),
+        'vm:102' => guest('node2', $usage->(2, 6)),
+        'vm:103' => guest('node3', $usage->(1, 11)),
+        'vm:104' => guest('node1', $usage->(2, 3)),
+        'vm:105' => guest('node1', $usage->(7, 13)),
+        'vm:106' => guest('node2', $usage->(1, 14)),
+        'vm:107' => guest('node4', $usage->(5, 9)),
+    };
+    my $node_stats = nodes(4, maxcpu => 16, maxmem => 64 * $GB);
+    for my $guest (values %$guest_stats) {
+        $node_stats->{ $guest->{node} }->{cpu} += $guest->{usage}->{cpu};
+        $node_stats->{ $guest->{node} }->{mem} += $guest->{usage}->{mem};
+    }
+    my $planner = planner('dynamic', $node_stats, $guest_stats);
+    my $movable = [sort keys %$guest_stats];
+
+    my $hops = $planner->suggest_rebalance($movable, targets => ['node2', 'node3']);
+    my ($hop) = grep { $_->{sid} eq 'vm:104' } $hops->{moves}->@*;
+    is($hop->{target}, 'node2', 'unseparated, vm:104 ends next to vm:102');
+
+    my $separate = { 'vm:104' => ['vm:102'] };
+    my $plan = $planner->suggest_rebalance(
+        $movable,
+        targets => ['node2', 'node3'],
+        separate => $separate,
+    );
+    is_deeply(
+        shared_nodes($guest_stats, $plan->{moves}, $separate),
+        [],
+        'separated, it does not',
+    );
+    my $replayed =
+        replay('dynamic', $node_stats, $guest_stats, $plan->{moves})->node_imbalance();
+    ok(abs($replayed - $plan->{imbalance}->{after}) < $EPS, 'after matches the moves');
+
+    # pruned and full scans honor the separation alike
+    my ($synth_nodes, $synth_guests) = synthetic_cluster(16, 300);
+    # pair guests of the busy node1 with guests elsewhere, which are apart to begin with
+    my @busy = grep { $synth_guests->{$_}->{node} eq 'node1' } sort keys %$synth_guests;
+    my @other = grep { $synth_guests->{$_}->{node} ne 'node1' } sort keys %$synth_guests;
+    my $pairs = { map { $busy[$_] => [$other[$_]] } 0 .. min($#busy, $#other) };
+    my $synth = planner('dynamic', $synth_nodes, $synth_guests);
+    my $pruned = $synth->suggest_rebalance(
+        [sort keys %$synth_guests],
+        separate => $pairs,
+        busiest_nodes => 3,
+        max_moves => 30,
+    );
+    cmp_ok(scalar($pruned->{moves}->@*), '>', 0, 'a pruned proposal with separated guests');
+    is_deeply(shared_nodes($synth_guests, $pruned->{moves}, $pairs), [], 'keeps them apart');
+    plan_is(
+        $pruned,
+        $synth->suggest_rebalance(
+            [sort keys %$synth_guests],
+            separate => $pairs,
+            busiest_nodes => 0,
+            max_moves => 30,
+        ),
+        'and equals the unpruned proposal',
+    );
+
+    # an undone move restores the placement the separation is checked against
+    my $static_nodes = {
+        node1 => { maxcpu => 16, maxmem => 64 * $GB },
+        node2 => { maxcpu => 16, maxmem => 64 * $GB },
+        node3 => { maxcpu => 64, maxmem => 64 * $GB },
+    };
+    my $static_guests = {
+        'ct:100' =>
+            { node => 'node1', running => 1, usage => { maxcpu => 0, maxmem => 4 * $GB } },
+        'ct:101' =>
+            { node => 'node1', running => 1, usage => { maxcpu => 0, maxmem => 4 * $GB } },
+        'vm:102' =>
+            { node => 'node2', running => 1, usage => { maxcpu => 4, maxmem => 8 * $GB } },
+    };
+    my $static_separate = { 'ct:100' => ['ct:101'] };
+    my $static = planner('static', $static_nodes, $static_guests)->suggest_rebalance(
+        [sort keys %$static_guests],
+        separate => $static_separate,
+        min_step_ratio => 0,
+    );
+    is_deeply(
+        shared_nodes($static_guests, $static->{moves}, $static_separate),
+        [],
+        'separated guests stay apart around an undone move',
+    );
+    $replayed =
+        replay('static', $static_nodes, $static_guests, $static->{moves})->node_imbalance();
+    ok(abs($replayed - $static->{imbalance}->{after}) < $EPS, 'after matches the moves');
+};
+
+subtest 'static mode works end to end' => sub {
+    my $planner = planner(
+        'static',
+        {
+            node1 => { maxcpu => 8, maxmem => 16 * $GB },
+            node2 => { maxcpu => 8, maxmem => 16 * $GB },
+            node3 => { maxcpu => 8, maxmem => 16 * $GB },
+        },
+        {
+            'vm:100' =>
+                { node => 'node1', running => 1, usage => { maxcpu => 2, maxmem => 4 * $GB } },
+            'vm:101' =>
+                { node => 'node1', running => 1, usage => { maxcpu => 2, maxmem => 4 * $GB } },
+            'vm:102' =>
+                { node => 'node1', running => 1, usage => { maxcpu => 2, maxmem => 4 * $GB } },
+        },
+    );
+
+    is($planner->mode(), 'static', 'static mode is reported');
+    cmp_ok($planner->node_imbalance(), '>', 0, 'static imbalance from configured usage');
+
+    my $plan = $planner->suggest_rebalance([qw(vm:100 vm:101 vm:102)]);
+    is(scalar($plan->{moves}->@*), 2, 'one guest moves to each of the other nodes');
+    ok($plan->{imbalance}->{after} < $EPS, 'static rebalance balances the cluster');
+};
+
+subtest 'a single-node cluster is balanced and has nothing to move' => sub {
+    my $planner = planner(
+        'dynamic',
+        { node1 => { maxcpu => 8, cpu => 4, maxmem => 16 * $GB, mem => 8 * $GB } },
+        { 'vm:100' => guest('node1', cpu => 4, mem => 8 * $GB) },
+    );
+
+    is($planner->node_imbalance(), 0, 'single-node cluster has zero imbalance');
+    is_deeply(
+        $planner->rank_target_nodes('vm:100'),
+        [],
+        'single-node cluster offers no migration targets',
+    );
+    is_deeply(
+        $planner->suggest_rebalance(['vm:100'])->{moves},
+        [],
+        'single-node cluster has nothing to rebalance',
+    );
+};
+
+# Parity with driving PVE::ResourceScheduling::Scheduler by hand, the way the HA usage drivers do:
+# add every node, then every running guest on its node. Static mode must match exactly, and so must
+# dynamic mode as long as no node measures less than its guests, where the reconcile is a no-op.
+my sub hand_scheduler {
+    my ($mode, $node_stats, $guest_stats) = @_;
+
+    my $scheduler = PVE::ResourceScheduling::Scheduler->new($mode);
+    $scheduler->add_node($_, $node_stats->{$_}) for sort keys %$node_stats;
+    for my $sid (sort keys %$guest_stats) {
+        my $guest = $guest_stats->{$sid};
+        next if !$guest->{running};
+        $scheduler->add_resource(
+            $sid,
+            { stats => $guest->{usage}, running => 1, 'current-node' => $guest->{node} },
+        );
+    }
+
+    return $scheduler;
+}
+
+my @parity_scenarios = (
+    {
+        name => 'static, equal capacity, guests stacked on node1',
+        mode => 'static',
+        nodes => { map { ("node$_" => { maxcpu => 8, maxmem => 32 * $GB }) } 1 .. 3 },
+        guests => {
+            'vm:101' =>
+                { node => 'node1', running => 1, usage => { maxcpu => 2, maxmem => 4 * $GB } },
+            'vm:102' =>
+                { node => 'node1', running => 1, usage => { maxcpu => 2, maxmem => 4 * $GB } },
+            'vm:103' =>
+                { node => 'node1', running => 1, usage => { maxcpu => 4, maxmem => 8 * $GB } },
+            'vm:104' =>
+                { node => 'node2', running => 1, usage => { maxcpu => 1, maxmem => 2 * $GB } },
+            'vm:105' =>
+                { node => 'node2', running => 0, usage => { maxcpu => 2, maxmem => 4 * $GB } },
+        },
+    },
+    {
+        name => 'static, heterogeneous capacity, uneven spread',
+        mode => 'static',
+        nodes => {
+            node1 => { maxcpu => 16, maxmem => 64 * $GB },
+            node2 => { maxcpu => 8, maxmem => 32 * $GB },
+            node3 => { maxcpu => 4, maxmem => 16 * $GB },
+        },
+        guests => {
+            'ct:201' =>
+                { node => 'node1', running => 1, usage => { maxcpu => 4, maxmem => 8 * $GB } },
+            'ct:202' =>
+                { node => 'node1', running => 1, usage => { maxcpu => 2, maxmem => 8 * $GB } },
+            'vm:203' =>
+                { node => 'node2', running => 1, usage => { maxcpu => 4, maxmem => 16 * $GB } },
+            'vm:204' =>
+                { node => 'node3', running => 1, usage => { maxcpu => 1, maxmem => 2 * $GB } },
+            'vm:205' =>
+                { node => 'node3', running => 0, usage => { maxcpu => 8, maxmem => 32 * $GB } },
+        },
+    },
+    {
+        name => 'dynamic, no node measures less than its guests',
+        mode => 'dynamic',
+        nodes => {
+            node1 => { maxcpu => 16, cpu => 4.0, maxmem => 64 * $GB, mem => 28 * $GB },
+            node2 => { maxcpu => 16, cpu => 1.5, maxmem => 64 * $GB, mem => 6 * $GB },
+            node3 => { maxcpu => 16, cpu => 0.5, maxmem => 64 * $GB, mem => 2 * $GB },
+        },
+        guests => {
+            'vm:401' =>
+                guest('node1', maxcpu => 8, cpu => 2.0, maxmem => 32 * $GB, mem => 8 * $GB),
+            'vm:402' =>
+                guest('node1', maxcpu => 8, cpu => 1.0, maxmem => 32 * $GB, mem => 4 * $GB),
+            'vm:403' =>
+                guest('node2', maxcpu => 8, cpu => 1.5, maxmem => 32 * $GB, mem => 6 * $GB),
+            'vm:404' =>
+                guest('node3', maxcpu => 8, cpu => 0.5, maxmem => 32 * $GB, mem => 2 * $GB),
+            'vm:405' => { guest('node3', maxcpu => 4, maxmem => 8 * $GB)->%*, running => 0 },
+        },
+    },
+);
+
+for my $scenario (@parity_scenarios) {
+    subtest "parity with a hand-driven scheduler: $scenario->{name}" => sub {
+        my ($mode, $node_stats, $guest_stats) = $scenario->@{qw(mode nodes guests)};
+
+        my $hand = hand_scheduler($mode, $node_stats, $guest_stats);
+        my $planner = planner($mode, $node_stats, $guest_stats);
+
+        ok(
+            abs($planner->node_imbalance() - $hand->calculate_node_imbalance()) < $EPS,
+            'node imbalance matches',
+        );
+
+        for my $sid (sort keys %$guest_stats) {
+            my $guest = $guest_stats->{$sid};
+            my @others = grep { $_ ne $guest->{node} } sort keys %$node_stats;
+
+            my $ranked = $planner->rank_target_nodes($sid);
+            if ($guest->{running}) {
+                my $candidate = { leader => $sid, resources => [$sid], nodes => [@others] };
+                my $expected = {
+                    map { $_->{migration}->{'target-node'} => $_->{imbalance} }
+                        $hand->score_best_balancing_migration_candidates(
+                            [$candidate], scalar(@others),
+                        )->@*
+                };
+                is_deeply(
+                    [sort map { $_->{node} } @$ranked],
+                    [sort keys %$expected],
+                    "$sid: migration targets match",
+                );
+                ok(
+                    !(
+                        grep { abs($_->{imbalance} - $expected->{ $_->{node} }) >= $EPS }
+                        @$ranked
+                    ),
+                    "$sid: migration imbalances match",
+                );
+            } else {
+                my $allowed = { map { $_ => 1 } @others };
+                my $expected = [
+                    map { $_->[0] }
+                    sort { $b->[1] <=> $a->[1] || $a->[0] cmp $b->[0] }
+                    grep { $allowed->{ $_->[0] } }
+                        $hand->score_nodes_to_start($guest->{usage})->@*
+                ];
+                is_deeply(
+                    [map { $_->{node} } @$ranked],
+                    $expected,
+                    "$sid: placement order matches",
+                );
+            }
+        }
+
+        my @candidates;
+        for my $sid (sort keys %$guest_stats) {
+            my $guest = $guest_stats->{$sid};
+            next if !$guest->{running};
+            my @targets = grep { $_ ne $guest->{node} } sort keys %$node_stats;
+            push @candidates, { leader => $sid, resources => [$sid], nodes => [@targets] };
+        }
+        my $best = $hand->score_best_balancing_migration_candidates(\@candidates, 1)->[0];
+        my $improves = $hand->calculate_node_imbalance() - $best->{imbalance} > $EPS;
+
+        # only the scheduler decision matters here, not whether a single step is worth proposing
+        my $plan = $planner->suggest_rebalance(
+            [sort keys %$guest_stats],
+            max_moves => 1,
+            min_total_ratio => 0,
+        );
+        if ($improves) {
+            my ($sid, $source, $target) =
+                $best->{migration}->@{qw(sid source-node target-node)};
+            is_deeply(
+                $plan->{moves},
+                [{ sid => $sid, source => $source, target => $target }],
+                'the first step is the best migration',
+            );
+            ok(
+                abs($plan->{imbalance}->{after} - $best->{imbalance}) < $EPS,
+                'the imbalance after the step is the predicted one',
+            );
+        } else {
+            is_deeply($plan->{moves}, [], 'no step when the best migration does not improve');
+        }
+    };
+}
+
+subtest 'build(): mode selection and argument checks' => sub {
+    my $calls = {};
+    my $dynamic_nodes = {
+        node1 => { maxcpu => 8, cpu => 1, maxmem => 16 * $GB, mem => 2 * $GB },
+        node2 => { maxcpu => 8, cpu => 1, maxmem => 16 * $GB, mem => 2 * $GB },
+    };
+    my $static_nodes = {
+        node1 => { maxcpu => 8, maxmem => 16 * $GB },
+        node2 => { maxcpu => 8, maxmem => 16 * $GB },
+        node3 => { maxcpu => 8, maxmem => 16 * $GB },
+    };
+    my $passed_resolvers;
+
+    # a stopped guest, whose placement ranking lists the other nodes a planner covers
+    my $stopped =
+        sub { return { 'vm:100' => { node => 'node1', running => 0, usage => $_[0] } } };
+    my $covered = sub {
+        return [map { $_->{node} } $_[0]->rank_target_nodes('vm:100')->@*];
+    };
+
+    no warnings 'redefine';
+    local *PVE::ResourceScheduling::Stats::get_dynamic_node_stats = sub {
+        $calls->{dynamic_nodes}++;
+        return { map { $_ => { $dynamic_nodes->{$_}->%* } } keys %$dynamic_nodes };
+    };
+    local *PVE::ResourceScheduling::Stats::get_dynamic_guest_stats = sub {
+        $calls->{dynamic_guests}++;
+        return $stopped->({ maxcpu => 1, cpu => 0, maxmem => $GB, mem => 0 });
+    };
+    local *PVE::ResourceScheduling::Stats::get_static_node_stats = sub {
+        $calls->{static_nodes}++;
+        return { map { $_ => { $static_nodes->{$_}->%* } } keys %$static_nodes };
+    };
+    local *PVE::ResourceScheduling::Stats::get_static_guest_stats = sub {
+        ($passed_resolvers) = @_;
+        $calls->{static_guests}++;
+        return $stopped->({ maxcpu => 1, maxmem => $GB });
+    };
+
+    my $resolvers = { vm => sub { {} }, ct => sub { {} } };
+    my $build = sub { PVE::ResourceScheduling::Planner->build(@_) };
+
+    eval { $build->(mode => 'bogus', nodes => ['node1'], resolvers => $resolvers) };
+    like($@, qr/invalid scheduling mode 'bogus'/, 'an unknown mode is rejected');
+
+    for my $mode (undef, 'auto', 'static') {
+        $calls = {};
+        eval { $build->(defined($mode) ? (mode => $mode) : (), nodes => ['node1']) };
+        my $name = $mode // 'default';
+        like($@, qr/needs a 'resolvers' argument/, "$name mode requires resolvers");
+        is_deeply($calls, {}, "$name mode checks resolvers before gathering stats");
+    }
+
+    eval { $build->(mode => 'dynamic', resolvers => $resolvers) };
+    like($@, qr/missing required 'nodes'/, 'nodes are required');
+
+    $calls = {};
+    my $planner = $build->(nodes => ['node1', 'node2'], resolvers => $resolvers);
+    is($planner->mode(), 'dynamic', 'auto uses dynamic stats when all nodes have them');
+    ok(!$calls->{static_nodes} && !$calls->{static_guests},
+        'auto does not gather static stats');
+
+    $calls = {};
+    $planner = $build->(nodes => [qw(node1 node2 node3)], resolvers => $resolvers);
+    is($planner->mode(), 'static', 'auto falls back to static if a node lacks dynamic stats');
+    is_deeply($covered->($planner), [qw(node2 node3)], 'the static fallback covers all nodes');
+    is($passed_resolvers, $resolvers, 'the resolvers are passed to the static guest stats');
+
+    $dynamic_nodes->{node2}->{maxmem} = 0;
+    $planner = $build->(nodes => [qw(node1 node2)], resolvers => $resolvers);
+    is($planner->mode(), 'static', 'auto falls back to static if a node has unusable stats');
+    $dynamic_nodes->{node2}->{maxmem} = 16 * $GB;
+
+    $planner = $build->(mode => 'dynamic', nodes => [qw(node1 node2 node3)]);
+    is($planner->mode(), 'dynamic', 'explicit dynamic mode does not fall back');
+    is_deeply($covered->($planner), ['node2'], 'and skips the node without stats');
+
+    delete $static_nodes->{node3};
+    $planner =
+        $build->(mode => 'static', nodes => [qw(node1 node2 node3)], resolvers => $resolvers);
+    is($planner->mode(), 'static', 'explicit static mode');
+    is_deeply($covered->($planner), ['node2'], 'skips the node without static stats');
+
+    local *PVE::ResourceScheduling::Stats::get_dynamic_node_stats = sub { die "ipc failure\n" };
+    eval { $build->(nodes => ['node1'], resolvers => $resolvers) };
+    is($@, "ipc failure\n", 'errors while gathering stats propagate');
+};
+
+done_testing();
diff --git a/src/tests/resource-scheduling-scheduler-tests.pl b/src/tests/resource-scheduling-scheduler-tests.pl
new file mode 100644
index 0000000..951677b
--- /dev/null
+++ b/src/tests/resource-scheduling-scheduler-tests.pl
@@ -0,0 +1,230 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib qw(..);
+
+use Test::More;
+
+use PVE::RS::ResourceScheduling::Static;
+use PVE::RS::ResourceScheduling::Dynamic;
+
+use PVE::ResourceScheduling::Scheduler;
+
+my $GB = 1024 * 1024 * 1024;
+
+# The Scheduler is a thin mode-dispatching forwarder, so test it against the raw PVE::RS backend:
+# for the same operations it must return exactly what the underlying scheduler returns. That pins
+# the static/dynamic method dispatch and the argument forwarding without re-testing the backend's
+# own placement math. Placement scores come back unordered, so compare them keyed by node.
+my sub scores_by_node {
+    my ($score_list) = @_;
+    return { map { $_->[0] => $_->[1] } @$score_list };
+}
+
+subtest 'new() validates the mode' => sub {
+    is(
+        PVE::ResourceScheduling::Scheduler->new('static')->mode(),
+        'static',
+        'static scheduler reports its mode',
+    );
+    is(
+        PVE::ResourceScheduling::Scheduler->new('dynamic')->mode(),
+        'dynamic',
+        'dynamic scheduler reports its mode',
+    );
+    eval { PVE::ResourceScheduling::Scheduler->new('bogus') };
+    like($@, qr/invalid scheduling mode/, 'an unknown mode dies');
+};
+
+subtest 'static scheduler forwards verbatim to the raw backend' => sub {
+    my $raw = PVE::RS::ResourceScheduling::Static->new();
+    my $sched = PVE::ResourceScheduling::Scheduler->new('static');
+
+    $raw->add_node('node1', 8, int(32 * $GB));
+    $raw->add_node('node2', 4, int(16 * $GB));
+    $sched->add_node('node1', { maxcpu => 8, maxmem => int(32 * $GB) });
+    $sched->add_node('node2', { maxcpu => 4, maxmem => int(16 * $GB) });
+
+    is_deeply(
+        [sort $sched->list_nodes()],
+        [qw(node1 node2)],
+        'list_nodes returns a flat sorted list',
+    );
+    ok($sched->contains_node('node1'), 'contains_node forwards');
+
+    my $service = {
+        stats => { maxcpu => 4, maxmem => 16 * $GB },
+        running => 1,
+        'current-node' => 'node1',
+    };
+    $raw->add_service('vm:100', $service);
+    $sched->add_resource('vm:100', $service);
+
+    my $usage = { maxcpu => 2, maxmem => 4 * $GB };
+    is_deeply(
+        scores_by_node($sched->score_nodes_to_start($usage)),
+        scores_by_node($raw->score_nodes_to_start_service($usage)),
+        'score_nodes_to_start dispatches to the raw score_nodes_to_start_service',
+    );
+    is(
+        $sched->calculate_node_imbalance(),
+        $raw->calculate_node_imbalance(),
+        'calculate_node_imbalance forwards',
+    );
+
+    my $candidate = { leader => 'vm:100', resources => ['vm:100'], nodes => ['node2'] };
+    is_deeply(
+        $sched->score_best_balancing_migration_candidates([$candidate], 1),
+        $raw->score_best_balancing_migration_candidates([$candidate], 1),
+        'score_best_balancing_migration_candidates forwards (bruteforce, static)',
+    );
+
+    $raw->remove_service_usage('vm:100');
+    $sched->remove_resource('vm:100');
+    is(
+        $sched->calculate_node_imbalance(),
+        $raw->calculate_node_imbalance(),
+        'remove_resource forwards to the raw remove_service_usage (static)',
+    );
+
+    $sched->remove_node('node2');
+    ok(!$sched->contains_node('node2'), 'remove_node forwards');
+};
+
+subtest 'dynamic scheduler forwards verbatim to the raw backend' => sub {
+    my $node = sub { return { maxcpu => 8, cpu => $_[0], maxmem => 16 * $GB, mem => $_[1] } };
+
+    my $raw = PVE::RS::ResourceScheduling::Dynamic->new();
+    my $sched = PVE::ResourceScheduling::Scheduler->new('dynamic');
+    $raw->add_node('node1', $node->(6, 12 * $GB));
+    $raw->add_node('node2', $node->(0, 0));
+    $sched->add_node('node1', $node->(6, 12 * $GB));
+    $sched->add_node('node2', $node->(0, 0));
+
+    my $resource = {
+        stats => { maxcpu => 8, cpu => 3, maxmem => 16 * $GB, mem => 6 * $GB },
+        running => 1,
+        'current-node' => 'node1',
+    };
+    $raw->add_resource('vm:200', $resource);
+    $sched->add_resource('vm:200', $resource);
+
+    my $usage = { maxcpu => 8, cpu => 2, maxmem => 16 * $GB, mem => 4 * $GB };
+    is_deeply(
+        scores_by_node($sched->score_nodes_to_start($usage)),
+        scores_by_node($raw->score_nodes_to_start_resource($usage)),
+        'score_nodes_to_start dispatches to the raw score_nodes_to_start_resource',
+    );
+
+    my $candidate = { leader => 'vm:200', resources => ['vm:200'], nodes => ['node2'] };
+    is_deeply(
+        $sched->score_best_balancing_migration_candidates([$candidate], 1),
+        $raw->score_best_balancing_migration_candidates([$candidate], 1),
+        'bruteforce migration scorer forwards',
+    );
+
+    $raw->remove_resource('vm:200');
+    $sched->remove_resource('vm:200');
+    is(
+        $sched->calculate_node_imbalance(),
+        $raw->calculate_node_imbalance(),
+        'remove_resource forwards to the raw remove_resource (dynamic)',
+    );
+};
+
+subtest "the 'topsis' method arg selects the raw topsis scorer" => sub {
+    # three guests on a loaded node where the two scorers prefer different migrations, so a wrong
+    # dispatch cannot go unnoticed
+    my $raw = PVE::RS::ResourceScheduling::Dynamic->new();
+    my $sched = PVE::ResourceScheduling::Scheduler->new('dynamic');
+
+    my $nodes = { node1 => [8, 12], node2 => [6, 5], node3 => [4, 0] };
+    for my $node (sort keys %$nodes) {
+        my ($cpu, $mem) = $nodes->{$node}->@*;
+        my $stats = { maxcpu => 8, cpu => $cpu, maxmem => 16 * $GB, mem => $mem * $GB };
+        $raw->add_node($node, $stats);
+        $sched->add_node($node, $stats);
+    }
+
+    my $guests = { 'vm:101' => [3, 1], 'vm:102' => [0, 6], 'vm:103' => [3, 3] };
+    my @candidates;
+    for my $sid (sort keys %$guests) {
+        my ($cpu, $mem) = $guests->{$sid}->@*;
+        my $resource = {
+            stats => { maxcpu => 4, cpu => $cpu, maxmem => 8 * $GB, mem => $mem * $GB },
+            running => 1,
+            'current-node' => 'node1',
+        };
+        $raw->add_resource($sid, $resource);
+        $sched->add_resource($sid, $resource);
+        push @candidates, { leader => $sid, resources => [$sid], nodes => ['node2', 'node3'] };
+    }
+
+    my $topsis = $raw->score_best_balancing_migration_candidates_topsis(\@candidates, 1);
+    my $bruteforce = $raw->score_best_balancing_migration_candidates(\@candidates, 1);
+    isnt(
+        $topsis->[0]->{migration}->{sid},
+        $bruteforce->[0]->{migration}->{sid},
+        'the scorers disagree on this cluster',
+    );
+
+    is_deeply(
+        $sched->score_best_balancing_migration_candidates(\@candidates, 1, 'topsis'),
+        $topsis,
+        "'topsis' dispatches to the raw topsis scorer",
+    );
+    is_deeply(
+        $sched->score_best_balancing_migration_candidates(\@candidates, 1),
+        $bruteforce,
+        'the default dispatches to the raw bruteforce scorer',
+    );
+};
+
+subtest 'static add_node accepts fractional capacities like the raw backend given int()' => sub {
+    # Static capacities can come from derived values. The wrapper coerces them to integers itself
+    # instead of relying on how the binding converts a fractional value; the result must match
+    # passing explicit integers to the raw backend.
+    my $raw = PVE::RS::ResourceScheduling::Static->new();
+    my $sched = PVE::ResourceScheduling::Scheduler->new('static');
+
+    eval {
+        $sched->add_node('node1', { maxcpu => 8.7, maxmem => 16.9 * $GB });
+        $sched->add_node('node2', { maxcpu => 8, maxmem => 16 * $GB });
+    };
+    ok(!$@, 'static add_node accepts fractional capacities') or diag $@;
+
+    $raw->add_node('node1', int(8.7), int(16.9 * $GB));
+    $raw->add_node('node2', 8, int(16 * $GB));
+
+    my %resources = (
+        'vm:100' => { maxcpu => 4, maxmem => 4 * $GB, node => 'node1' },
+        'vm:101' => { maxcpu => 2, maxmem => 2 * $GB, node => 'node2' },
+    );
+    for my $sid (sort keys %resources) {
+        my $r = $resources{$sid};
+        my $service = {
+            stats => { maxcpu => $r->{maxcpu}, maxmem => $r->{maxmem} },
+            running => 1,
+            'current-node' => $r->{node},
+        };
+        $raw->add_service($sid, $service);
+        $sched->add_resource($sid, $service);
+    }
+
+    cmp_ok($raw->calculate_node_imbalance(), '>', 0, 'both nodes carry load');
+    is(
+        $sched->calculate_node_imbalance(),
+        $raw->calculate_node_imbalance(),
+        'imbalance matches the raw backend given integer capacities',
+    );
+};
+
+subtest 'an unknown scoring method is rejected' => sub {
+    my $sched = PVE::ResourceScheduling::Scheduler->new('dynamic');
+    eval { $sched->score_best_balancing_migration_candidates([], 1, 'bogus') };
+    like($@, qr/invalid scoring method/, 'an unknown scoring method dies');
+};
+
+done_testing();
diff --git a/src/tests/resource-scheduling-stats-tests.pl b/src/tests/resource-scheduling-stats-tests.pl
new file mode 100644
index 0000000..48fbcac
--- /dev/null
+++ b/src/tests/resource-scheduling-stats-tests.pl
@@ -0,0 +1,260 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use lib qw(..);
+
+use Test::More;
+
+use PVE::Cluster;
+use PVE::ResourceScheduling::Stats;
+
+subtest 'dynamic guest stats: RRD-derived usage and guest-type mapping' => sub {
+    no warnings 'redefine';
+    local *PVE::Cluster::get_vmlist = sub {
+        return {
+            ids => {
+                100 => { node => 'node1', type => 'qemu' },
+                200 => { node => 'node2', type => 'lxc' },
+                300 => { node => 'node1', type => 'storage' }, # not a guest
+            },
+        };
+    };
+    local *PVE::Cluster::rrd_dump = sub {
+        return {
+            # index: 2 status, 5 maxcpu, 6 cpu-fraction, 7 maxmem, 8 mem
+            'pve-vm-9.0/100' =>
+                [undef, undef, 'running', undef, undef, 4, 0.5, 8_000_000_000, 4_000_000_000],
+            'pve-vm-9.0/200' =>
+                [undef, undef, 'stopped', undef, undef, 2, 0.0, 4_000_000_000, 1_000_000_000],
+        };
+    };
+
+    my $stats = PVE::ResourceScheduling::Stats::get_dynamic_guest_stats();
+
+    is_deeply(
+        [sort keys %$stats],
+        ['ct:200', 'vm:100'],
+        'maps guest types and skips non-guests',
+    );
+    is($stats->{'vm:100'}->{running}, 1, 'running guest flagged as running');
+    is($stats->{'ct:200'}->{running}, 0, 'stopped guest not flagged as running');
+
+    my $usage = $stats->{'vm:100'}->{usage};
+    is($usage->{maxcpu}, 4, 'dynamic maxcpu taken from RRD');
+    is($usage->{cpu}, 2, 'dynamic cpu is fraction times maxcpu');
+    is($usage->{maxmem}, 8_000_000_000, 'dynamic maxmem taken from RRD');
+    is($usage->{mem}, 4_000_000_000, 'dynamic mem taken from RRD');
+};
+
+subtest 'dynamic guest stats: run state and usage come from a single RRD dump' => sub {
+    my $dumps = 0;
+
+    no warnings 'redefine';
+    local *PVE::Cluster::get_vmlist = sub {
+        return { ids => { 100 => { node => 'node1', type => 'qemu' } } };
+    };
+    local *PVE::Cluster::rrd_dump = sub {
+        $dumps++;
+        return {
+            'pve-vm-9.0/100' =>
+                [undef, undef, 'running', undef, undef, 4, 0.5, 8_000_000_000, 4_000_000_000],
+        };
+    };
+
+    my $stats = PVE::ResourceScheduling::Stats::get_dynamic_guest_stats();
+
+    is($dumps, 1, 'the RRD data is dumped once');
+    is($stats->{'vm:100'}->{running}, 1, 'run state taken from the dump');
+    is($stats->{'vm:100'}->{usage}->{cpu}, 2, 'usage taken from the same dump');
+};
+
+subtest 'guest run state: only the QMP running state counts as running' => sub {
+    no warnings 'redefine';
+    local *PVE::Cluster::get_vmlist = sub {
+        return {
+            ids => {
+                100 => { node => 'node1', type => 'qemu' },
+                101 => { node => 'node1', type => 'qemu' },
+                102 => { node => 'node1', type => 'qemu' },
+            },
+        };
+    };
+    local *PVE::Cluster::rrd_dump = sub {
+        return {
+            'pve-vm-9.0/100' => [undef, undef, 'running'],
+            'pve-vm-9.0/101' => [undef, undef, 'paused'],
+            # 102 has no RRD entry at all, e.g. never started since its creation
+        };
+    };
+
+    my $stats = PVE::ResourceScheduling::Stats::get_cluster_guest_stats();
+
+    is($stats->{'vm:100'}->{state}, 'started', 'running guest is started');
+    is($stats->{'vm:101'}->{running}, 0, 'paused guest is not running');
+    is($stats->{'vm:101'}->{state}, 'stopped', 'paused guest is treated as stopped');
+    is($stats->{'vm:102'}->{running}, 0, 'guest without RRD entry is not running');
+};
+
+subtest 'dynamic guest stats: VMs account host memory, containers fall back to mem' => sub {
+    no warnings 'redefine';
+    local *PVE::Cluster::get_vmlist = sub {
+        return {
+            ids => {
+                100 => { node => 'node1', type => 'qemu' },
+                200 => { node => 'node1', type => 'lxc' },
+            },
+        };
+    };
+    local *PVE::Cluster::rrd_dump = sub {
+        my @vm = (undef, undef, 'running', undef, undef, 4, 0.5, 8_000_000_000, 4_000_000_000);
+        my @ct = (undef, undef, 'running', undef, undef, 2, 0.5, 4_000_000_000, 1_000_000_000);
+        # index 15 is memhost, which pvestatd records as unknown for containers
+        return {
+            'pve-vm-9.0/100' => [@vm, (undef) x 6, 5_000_000_000],
+            'pve-vm-9.0/200' => [@ct, (undef) x 7],
+        };
+    };
+
+    my $stats = PVE::ResourceScheduling::Stats::get_dynamic_guest_stats();
+
+    is($stats->{'vm:100'}->{usage}->{mem}, 5_000_000_000, 'VM mem is its host memory usage');
+    is($stats->{'ct:200'}->{usage}->{mem}, 1_000_000_000, 'CT mem falls back to memory usage');
+};
+
+subtest 'dynamic guest stats: a reported zero host memory usage is kept' => sub {
+    no warnings 'redefine';
+    local *PVE::Cluster::get_vmlist = sub {
+        return { ids => { 100 => { node => 'node1', type => 'qemu' } } };
+    };
+    local *PVE::Cluster::rrd_dump = sub {
+        my @vm = (undef, undef, 'running', undef, undef, 4, 0.5, 8_000_000_000, 4_000_000_000);
+        return { 'pve-vm-9.0/100' => [@vm, (undef) x 6, 0] };
+    };
+
+    my $stats = PVE::ResourceScheduling::Stats::get_dynamic_guest_stats();
+
+    # only an unknown host memory usage falls back, a known zero is a value like any other
+    is($stats->{'vm:100'}->{usage}->{mem}, 0, 'zero host memory usage does not fall back');
+};
+
+subtest 'cluster guest enumeration returns a skeleton with empty usage' => sub {
+    no warnings 'redefine';
+    local *PVE::Cluster::get_vmlist = sub {
+        return { ids => { 100 => { node => 'node1', type => 'qemu' } } };
+    };
+    local *PVE::Cluster::rrd_dump = sub {
+        return { 'pve-vm-9.0/100' => [undef, undef, 'running'] };
+    };
+
+    my $stats = PVE::ResourceScheduling::Stats::get_cluster_guest_stats();
+
+    is_deeply(
+        $stats->{'vm:100'},
+        {
+            id => 100,
+            node => 'node1',
+            type => 'vm',
+            state => 'started',
+            running => 1,
+            usage => {},
+        },
+        'enumeration returns guest skeleton with empty usage',
+    );
+};
+
+subtest 'dynamic node stats: only node RRD keys, cpu scaled by core count' => sub {
+    no warnings 'redefine';
+    local *PVE::Cluster::rrd_dump = sub {
+        return {
+            'pve-node-9.0/node1' =>
+                [undef, undef, undef, undef, 8, 0.25, undef, 16_000_000_000, 8_000_000_000],
+            'pve-node-9.0/node2' =>
+                [undef, undef, undef, undef, 8, 0.0, undef, 16_000_000_000, 2_000_000_000],
+            'pve-vm-9.0/100' => [undef, undef, 'running'], # must be ignored
+            'some-unrelated-key' => [1, 2, 3],
+        };
+    };
+
+    my $stats = PVE::ResourceScheduling::Stats::get_dynamic_node_stats();
+
+    is_deeply([sort keys %$stats], ['node1', 'node2'], 'only node RRD entries are returned');
+    is($stats->{node1}->{maxcpu}, 8, 'node maxcpu from RRD');
+    is($stats->{node1}->{cpu}, 2, 'node cpu is fraction times core count');
+    is($stats->{node1}->{maxmem}, 16_000_000_000, 'node maxmem from RRD');
+    is($stats->{node1}->{mem}, 8_000_000_000, 'node mem from RRD');
+};
+
+subtest 'static guest stats: usage comes from the given type resolver' => sub {
+    my $resolvers = {
+        vm => sub {
+            my ($conf) = @_;
+            return {
+                maxcpu => 0.0 + ($conf->{cores} // 1),
+                maxmem => int(($conf->{memory} // 512) * 1024 * 1024),
+            };
+        },
+    };
+
+    no warnings 'redefine';
+    local *PVE::Cluster::get_vmlist = sub {
+        return { ids => { 100 => { node => 'node1', type => 'qemu' } } };
+    };
+    local *PVE::Cluster::rrd_dump = sub {
+        return { 'pve-vm-9.0/100' => [undef, undef, 'running'] };
+    };
+    local *PVE::Cluster::get_guest_config_properties = sub {
+        return { 100 => { cores => 4, memory => 2048 } };
+    };
+
+    my $stats = PVE::ResourceScheduling::Stats::get_static_guest_stats($resolvers);
+
+    is($stats->{'vm:100'}->{usage}->{maxcpu}, 4, 'static maxcpu from resolver');
+    is($stats->{'vm:100'}->{usage}->{maxmem}, 2048 * 1024 * 1024,
+        'static maxmem from resolver');
+};
+
+subtest 'static guest stats: a guest type without a resolver is an error' => sub {
+    no warnings 'redefine';
+    local *PVE::Cluster::get_vmlist = sub {
+        return { ids => { 200 => { node => 'node1', type => 'lxc' } } };
+    };
+    local *PVE::Cluster::rrd_dump = sub {
+        return { 'pve-vm-9.0/200' => [undef, undef, 'running'] };
+    };
+    local *PVE::Cluster::get_guest_config_properties = sub { return {} };
+
+    # a resolvers map that lacks the 'ct' the guest needs
+    eval {
+        PVE::ResourceScheduling::Stats::get_static_guest_stats({ vm => sub { {} } });
+    };
+    like(
+        $@,
+        qr/no static stats resolver for guest type 'ct'/,
+        'static stats without a matching resolver is rejected',
+    );
+};
+
+subtest 'static node stats: decoded from the static-info node kv' => sub {
+    no warnings 'redefine';
+    local *PVE::Cluster::get_node_kv = sub {
+        return { node1 => '{"cpus":8,"memory":16000000000}', node2 => 'garbage' };
+    };
+
+    my @warnings;
+    local $SIG{__WARN__} = sub { push @warnings, $_[0] };
+
+    my $stats = PVE::ResourceScheduling::Stats::get_static_node_stats();
+
+    is_deeply(
+        $stats->{node1},
+        { maxcpu => 8, maxmem => 16000000000 },
+        'static node stats decoded from static-info kv',
+    );
+    ok(exists($stats->{node2}) && !defined($stats->{node2}), 'undecodable node info is undef');
+    like($warnings[0], qr/unable to decode static node info for 'node2'/,
+        'decode error warned');
+};
+
+done_testing();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH ha-manager 02/15] helpers: block both locations of a migrating negative affinity partner
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH guest-common 01/15] scheduling: add cluster resource scheduling library Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH ha-manager 03/15] usage: static: count services per node again for scoring fallback Thomas Lamprecht
                   ` (12 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

The motion info of a resource lists, per node, the resources in
negative affinity with it that block the node. A partner was skipped
if its current node did not match, and also if its migration target
did not match, so it only blocked a node that matched both. A
migrating partner, whose source and target differ, thus blocked
neither of them.

As a result, the manager queued a migration to the node a negative
affinity partner was migrating to or away from instead of rejecting
it, the HA migrate and relocate API reported no blocker for it, and
the migrate preconditions of qemu-server and pve-container listed
such a node as allowed. The HA placement itself already treats a
migrating resource as occupying both nodes.

Block a node if it is either the current node or the migration target
of the partner. Add a test for the motion info, which also covers a
migrating partner.

Fixes: b926020 ("manager: handle resource affinity rules in manual migrations")
Fixes: 47340b1 ("api: resources: add check for resource affinity in resource migrations")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 src/PVE/HA/Helpers.pm                 |   4 +-
 src/test/Makefile                     |   1 +
 src/test/test_resource_motion_info.pl | 185 ++++++++++++++++++++++++++
 3 files changed, 188 insertions(+), 2 deletions(-)
 create mode 100755 src/test/test_resource_motion_info.pl

diff --git a/src/PVE/HA/Helpers.pm b/src/PVE/HA/Helpers.pm
index a58b1e1..cdc66a1 100644
--- a/src/PVE/HA/Helpers.pm
+++ b/src/PVE/HA/Helpers.pm
@@ -46,8 +46,8 @@ sub get_resource_motion_info($ss, $sid, $cd, $online_nodes, $compiled_rules) {
         for my $csid (sort keys %$separate) {
             next if !defined($ss->{$csid});
             next if $ss->{$csid}->{state} eq 'ignored';
-            next if $ss->{$csid}->{node} && $ss->{$csid}->{node} ne $node;
-            next if $ss->{$csid}->{target} && $ss->{$csid}->{target} ne $node;
+            # a migrating resource occupies both its source and its target node
+            next if !grep { defined($_) && $_ eq $node } $ss->{$csid}->@{qw(node target)};
 
             push $blocking_resources_by_node->{$node}->@*,
                 {
diff --git a/src/test/Makefile b/src/test/Makefile
index f72b755..a26c34e 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -7,6 +7,7 @@ test:
 	./test_failover1.pl
 	./test_rules_config.pl
 	./test_resource_bundles.pl
+	./test_resource_motion_info.pl
 	./ha-tester.pl
 	./test_fence_config.pl
 	@echo "-- end regression tests (success) --"
diff --git a/src/test/test_resource_motion_info.pl b/src/test/test_resource_motion_info.pl
new file mode 100755
index 0000000..2f7c3cd
--- /dev/null
+++ b/src/test/test_resource_motion_info.pl
@@ -0,0 +1,185 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib qw(..);
+
+use Test::More;
+use Test::MockModule;
+
+use PVE::HA::Config;
+use PVE::HA::Groups;
+use PVE::HA::Resources;
+use PVE::HA::Rules;
+use PVE::HA::Rules::NodeAffinity;
+use PVE::HA::Rules::ResourceAffinity;
+use PVE::HA::Sim::Resources::VirtCT;
+use PVE::HA::Sim::Resources::VirtVM;
+
+PVE::HA::Sim::Resources::VirtVM->register();
+PVE::HA::Sim::Resources::VirtCT->register();
+
+PVE::HA::Resources->init();
+
+PVE::HA::Rules::NodeAffinity->register();
+PVE::HA::Rules::ResourceAffinity->register();
+
+PVE::HA::Rules->init(property_isolation => 1);
+
+my $resources_cfg = <<EOF;
+vm: 101
+	state started
+
+vm: 102
+	state started
+
+vm: 103
+	state ignored
+
+ct: 104
+	state started
+EOF
+
+my $rules_cfg = <<EOF;
+node-affinity: vm101-strict
+	resources vm:101
+	nodes node1,node2
+	strict 1
+
+resource-affinity: vm101-vm102-apart
+	resources vm:101,vm:102
+	affinity negative
+
+resource-affinity: vm101-ct104-together
+	resources vm:101,ct:104
+	affinity positive
+EOF
+
+my $manager_status = {
+    node_status => { node1 => 'online', node2 => 'online', node3 => 'online' },
+    service_status => {
+        'vm:101' => { state => 'started', node => 'node1' },
+        'vm:102' => { state => 'started', node => 'node3' },
+        'vm:103' => { state => 'ignored', node => 'node2' },
+        'ct:104' => { state => 'started', node => 'node1' },
+    },
+};
+
+my $vmlist = {
+    ids => {
+        101 => { node => 'node1', type => 'qemu' },
+        102 => { node => 'node3', type => 'qemu' },
+        103 => { node => 'node2', type => 'qemu' },
+        104 => { node => 'node1', type => 'lxc' },
+    },
+};
+
+my $config_files = {
+    'ha/groups.cfg' => sub { PVE::HA::Groups->parse_config($_[0], undef) },
+    'ha/manager_status' => sub { $manager_status },
+    'ha/resources.cfg' => sub { PVE::HA::Resources->parse_config($_[0], $resources_cfg) },
+    'ha/rules.cfg' => sub { PVE::HA::Rules->parse_config($_[0], $rules_cfg) },
+};
+
+my $config_module = Test::MockModule->new('PVE::HA::Config');
+$config_module->redefine(
+    cfs_read_file => sub($filename) {
+        my $parser = $config_files->{$filename} or die "unexpected config read '$filename'\n";
+        return $parser->($filename);
+    },
+);
+
+my $cluster_module = Test::MockModule->new('PVE::Cluster');
+$cluster_module->redefine(get_vmlist => sub { return $vmlist });
+
+my $expected = {
+    'vm:101' => {
+        'dependent-resources' => ['ct:104'],
+        'blocking-resources-by-node' => {
+            node3 => [
+                { sid => 'vm:101', cause => 'node-affinity' },
+                { sid => 'vm:102', cause => 'resource-affinity' },
+            ],
+        },
+    },
+    'vm:102' => {
+        'dependent-resources' => [],
+        'blocking-resources-by-node' => {
+            node1 => [
+                { sid => 'ct:104', cause => 'resource-affinity' },
+                { sid => 'vm:101', cause => 'resource-affinity' },
+            ],
+        },
+    },
+    'ct:104' => {
+        'dependent-resources' => ['vm:101'],
+        'blocking-resources-by-node' => {
+            node3 => [
+                { sid => 'ct:104', cause => 'node-affinity' },
+                { sid => 'vm:102', cause => 'resource-affinity' },
+            ],
+        },
+    },
+};
+
+my $test_single_motion_info = sub($expected, $description) {
+    for my $sid (sort keys %$expected) {
+        my ($dependent_resources, $blocking_resources_by_node) =
+            PVE::HA::Config::get_resource_motion_info($sid);
+
+        is_deeply(
+            {
+                'dependent-resources' => $dependent_resources,
+                'blocking-resources-by-node' => $blocking_resources_by_node,
+            },
+            $expected->{$sid},
+            "$description: single motion info for $sid",
+        );
+    }
+};
+
+$test_single_motion_info->($expected, 'stationary partners');
+
+for my $sid (qw(vm:103 vm:999)) {
+    is_deeply(
+        [PVE::HA::Config::get_resource_motion_info($sid)],
+        [[], {}],
+        "no single motion info for ignored or unmanaged $sid",
+    );
+}
+
+# a migrating negative affinity partner occupies both its source and its target node
+$manager_status->{service_status}->{'vm:102'} =
+    { state => 'migrate', node => 'node3', target => 'node2' };
+
+my $expected_migrating = {
+    'vm:101' => {
+        'dependent-resources' => ['ct:104'],
+        'blocking-resources-by-node' => {
+            node2 => [
+                { sid => 'vm:102', cause => 'resource-affinity' },
+            ],
+            node3 => [
+                { sid => 'vm:101', cause => 'node-affinity' },
+                { sid => 'vm:102', cause => 'resource-affinity' },
+            ],
+        },
+    },
+    'vm:102' => $expected->{'vm:102'},
+    'ct:104' => {
+        'dependent-resources' => ['vm:101'],
+        'blocking-resources-by-node' => {
+            node2 => [
+                { sid => 'vm:102', cause => 'resource-affinity' },
+            ],
+            node3 => [
+                { sid => 'ct:104', cause => 'node-affinity' },
+                { sid => 'vm:102', cause => 'resource-affinity' },
+            ],
+        },
+    },
+};
+
+$test_single_motion_info->($expected_migrating, 'migrating partner');
+
+done_testing();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH ha-manager 03/15] usage: static: count services per node again for scoring fallback
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH guest-common 01/15] scheduling: add cluster resource scheduling library Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH ha-manager 02/15] helpers: block both locations of a migrating negative affinity partner Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH ha-manager 04/15] config: allow getting motion info of many resources at once Thomas Lamprecht
                   ` (11 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

If scoring the nodes for a service fails, for example because its
usage stats are missing, the static usage driver falls back to ranking
the nodes by the number of services they hold, so the service still
ends up on a less busy node.

Since adding a service moved to a single add_service call, nothing
recorded which services were on which node anymore, so every node got
a fallback score of zero and the service was placed on the first node
by name. Record the service on its current and target node again, as
before.

Add a test for the fallback, which fails without this fix.

Fixes: 9780600e3539 ("usage: use add_service to add service usage to nodes")
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 src/PVE/HA/Usage/Static.pm      |  3 ++
 src/test/Makefile               |  1 +
 src/test/test_usage_fallback.pl | 72 +++++++++++++++++++++++++++++++++
 3 files changed, 76 insertions(+)
 create mode 100755 src/test/test_usage_fallback.pl

diff --git a/src/PVE/HA/Usage/Static.pm b/src/PVE/HA/Usage/Static.pm
index e67d5f5..570a801 100644
--- a/src/PVE/HA/Usage/Static.pm
+++ b/src/PVE/HA/Usage/Static.pm
@@ -87,6 +87,9 @@ sub add_service {
         undef $target_node;
     }
 
+    $self->{'node-services'}->{$current_node}->{$sid} = 1;
+    $self->{'node-services'}->{$target_node}->{$sid} = 1 if defined($target_node);
+
     eval {
         my $service_usage = get_service_usage($self, $sid);
 
diff --git a/src/test/Makefile b/src/test/Makefile
index a26c34e..4b327ca 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -8,6 +8,7 @@ test:
 	./test_rules_config.pl
 	./test_resource_bundles.pl
 	./test_resource_motion_info.pl
+	./test_usage_fallback.pl
 	./ha-tester.pl
 	./test_fence_config.pl
 	@echo "-- end regression tests (success) --"
diff --git a/src/test/test_usage_fallback.pl b/src/test/test_usage_fallback.pl
new file mode 100755
index 0000000..b9bb6dd
--- /dev/null
+++ b/src/test/test_usage_fallback.pl
@@ -0,0 +1,72 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib qw(..);
+
+use Test::More;
+
+use PVE::HA::Usage::Static;
+
+my $GiB = 1024**3;
+
+package MockEnv {
+
+    sub new($class, $node_stats) {
+        return bless { node_stats => $node_stats, logs => [] }, $class;
+    }
+
+    sub get_static_node_stats($self) {
+        return $self->{node_stats};
+    }
+
+    sub log($self, $level, $msg) {
+        push $self->{logs}->@*, "$level: $msg";
+    }
+}
+
+my $node_usage = { maxcpu => 8, cpu => 0.0, maxmem => 16 * $GiB, mem => 0 };
+my $node_stats = { map { $_ => { $node_usage->%* } } qw(node1 node2 node3) };
+
+my $service_usage = { maxcpu => 1, cpu => 0.0, maxmem => $GiB, mem => 0 };
+my $service_stats = { map { ("vm:$_" => { usage => { $service_usage->%* } }) } 101 .. 105 };
+
+my $services = {
+    'vm:101' => { state => 'started', node => 'node1', running => 1 },
+    'vm:102' => { state => 'started', node => 'node1', running => 1 },
+    'vm:103' => { state => 'migrate', node => 'node2', target => 'node3' },
+    'vm:104' => { state => 'request_start_balance', node => 'node1', target => 'node2' },
+    'vm:105' => { state => 'stopped', node => 'node3' },
+};
+
+my $test_fallback = sub($class) {
+    my $haenv = MockEnv->new($node_stats);
+    my $usage = $class->new($haenv, $service_stats);
+    $usage->add_node($_) for sort keys %$node_stats;
+    $usage->add_service_usage($_, $services->{$_}) for sort keys %$services;
+
+    # scoring fails for a service without usage stats, so fall back to the service count
+    is_deeply(
+        $usage->score_nodes_to_start_service('vm:200'),
+        { node1 => 2, node2 => 2, node3 => 1 },
+        "$class: fall back to the service count per node",
+    );
+    like(
+        $haenv->{logs}->[-1],
+        qr/^err: unable to score nodes .* for service 'vm:200'/,
+        "$class: log the scoring failure",
+    );
+
+    $usage->remove_service_usage('vm:101');
+    $usage->remove_service_usage('vm:103');
+
+    is_deeply(
+        $usage->score_nodes_to_start_service('vm:200'),
+        { node1 => 1, node2 => 1, node3 => 0 },
+        "$class: service count fallback drops removed services",
+    );
+};
+
+$test_fallback->('PVE::HA::Usage::Static');
+
+done_testing();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH ha-manager 04/15] config: allow getting motion info of many resources at once
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (2 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH ha-manager 03/15] usage: static: count services per node again for scoring fallback Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH ha-manager 05/15] env: pve2: delegate cluster stat gathering to scheduling library Thomas Lamprecht
                   ` (10 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

The cluster rebalance proposal of the resource scheduling API needs
to know, for possibly every guest in the cluster, which nodes its HA
rules allow and which HA resources move together with it. Getting
that through get_resource_motion_info reads and compiles the whole
rule set again for each resource, which scales poorly with the number
of resources in a request.

Add get_resources_motion_info, which reads the configs and compiles
the rules once and returns the motion info for each HA-managed
resource of the given list. Turn get_resource_motion_info into a thin
wrapper around it, so both share one implementation and its existing
callers, such as the migrate preconditions of qemu-server and
pve-container, see no change.

Each read of the manager status returns a fresh copy, which gets
costly for many resources in a large cluster. So pass the manager
status that was already read to the HA state check instead of reading
it again for every resource.

For the same reason, add get_resources_requested_state, which tells
the HA-managed resources of a list and their requested state with one
read of the resources config, while vm_is_ha_managed copies the whole
config for each guest.

Extend the motion info test to cover the new helpers.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 src/PVE/HA/Config.pm                  | 83 ++++++++++++++++++++-------
 src/test/test_resource_motion_info.pl | 43 ++++++++++++++
 2 files changed, 106 insertions(+), 20 deletions(-)

diff --git a/src/PVE/HA/Config.pm b/src/PVE/HA/Config.pm
index bee1ac7..4e4e8b2 100644
--- a/src/PVE/HA/Config.pm
+++ b/src/PVE/HA/Config.pm
@@ -353,14 +353,14 @@ sub read_crm_commands {
 }
 
 my $service_check_ha_state = sub {
-    my ($conf, $sid, $has_state) = @_;
+    my ($conf, $sid, $has_state, $manager_status) = @_;
 
     if (my $d = $conf->{ids}->{$sid}) {
         if (!defined($has_state)) {
             # ignored service behave as if they were not managed by HA
             return 0 if defined($d->{state}) && $d->{state} eq 'ignored';
             # cluster-wide disarm with ignore mode - resources can be managed directly
-            my $ms = cfs_read_file($manager_status_filename);
+            my $ms = $manager_status // cfs_read_file($manager_status_filename);
             if (my $disarm = $ms->{disarm}) {
                 return 0 if $disarm->{mode} eq 'ignore';
             }
@@ -391,36 +391,79 @@ sub service_is_configured {
     return 0;
 }
 
-sub get_resource_motion_info {
-    my ($sid) = @_;
+# Returns the motion info of each HA-managed resource in $sids as a hash of
+#   $sid => { 'dependent-resources' => [...], 'blocking-resources-by-node' => { $node => [...] } }
+# as described for PVE::HA::Helpers::get_resource_motion_info(). Resources that are not
+# HA-managed are omitted. The rules are read and compiled once, independent of the number of
+# resources.
+sub get_resources_motion_info {
+    my ($sids) = @_;
 
     my $cfg = read_resources_config();
+    # read once, as each copy of a large manager status is costly
+    my $manager_status = read_manager_status();
 
-    my $dependent_resources = [];
-    my $blocking_resources_by_node = {};
+    my $managed_sids =
+        [grep { $service_check_ha_state->($cfg, $_, undef, $manager_status) } @$sids];
+    return {} if !@$managed_sids;
 
-    if (&$service_check_ha_state($cfg, $sid)) {
-        my $manager_status = read_manager_status();
-        my $ss = $manager_status->{service_status};
-        my $ns = $manager_status->{node_status};
-        # get_resource_motion_info expects a hashset of all nodes with status 'online'
-        my $online_nodes = { map { $ns->{$_} eq 'online' ? ($_ => 1) : () } keys %$ns };
-        # get_resource_motion_info expects a resource config with defaults set
-        my $resources = checked_resources_config($cfg);
+    my $ss = $manager_status->{service_status};
+    my $ns = $manager_status->{node_status};
+    # get_resource_motion_info expects a hashset of all nodes with status 'online'
+    my $online_nodes = { map { $ns->{$_} eq 'online' ? ($_ => 1) : () } keys %$ns };
+    # get_resource_motion_info expects a resource config with defaults set
+    my $resources = checked_resources_config($cfg);
 
-        my $rules = read_and_check_rules_config();
-        # TODO PVE 10: Remove group migration when HA groups have been fully migrated to rules
-        my $groups = read_group_config();
-        my $compiled_rules = compile_rules_config($rules, $groups, $resources, $manager_status);
+    my $rules = read_and_check_rules_config();
+    # TODO PVE 10: Remove group migration when HA groups have been fully migrated to rules
+    my $groups = read_group_config();
+    my $compiled_rules = compile_rules_config($rules, $groups, $resources, $manager_status);
 
+    my $motion_info = {};
+    for my $sid (@$managed_sids) {
         my $cd = $resources->{$sid} // {};
-        ($dependent_resources, $blocking_resources_by_node) =
+        my ($dependent_resources, $blocking_resources_by_node) =
             PVE::HA::Helpers::get_resource_motion_info(
                 $ss, $sid, $cd, $online_nodes, $compiled_rules,
             );
+
+        $motion_info->{$sid} = {
+            'dependent-resources' => $dependent_resources,
+            'blocking-resources-by-node' => $blocking_resources_by_node,
+        };
     }
 
-    return ($dependent_resources, $blocking_resources_by_node);
+    return $motion_info;
+}
+
+# Returns the requested state of each HA-managed resource in $sids as a hash of $sid => $state,
+# reading the resources config only once. Resources that are not HA-managed are omitted, as for
+# service_is_ha_managed(). Pass the manager status if it was already read.
+sub get_resources_requested_state {
+    my ($sids, $manager_status) = @_;
+
+    my $cfg = read_resources_config();
+    $manager_status //= read_manager_status();
+
+    my $states = {};
+    for my $sid (@$sids) {
+        next if !$service_check_ha_state->($cfg, $sid, undef, $manager_status);
+        my $state = $cfg->{ids}->{$sid}->{state} // 'started';
+        $states->{$sid} = $state eq 'enabled' ? 'started' : $state;
+    }
+
+    return $states;
+}
+
+sub get_resource_motion_info {
+    my ($sid) = @_;
+
+    my $motion_info = get_resources_motion_info([$sid])->{$sid} // {};
+
+    return (
+        $motion_info->{'dependent-resources'} // [],
+        $motion_info->{'blocking-resources-by-node'} // {},
+    );
 }
 
 # graceful, as long as locking + cfs_write works
diff --git a/src/test/test_resource_motion_info.pl b/src/test/test_resource_motion_info.pl
index 2f7c3cd..9729ca6 100755
--- a/src/test/test_resource_motion_info.pl
+++ b/src/test/test_resource_motion_info.pl
@@ -148,6 +148,30 @@ for my $sid (qw(vm:103 vm:999)) {
     );
 }
 
+my $compile_count = 0;
+my $rules_module = Test::MockModule->new('PVE::HA::Rules');
+$rules_module->redefine(
+    compile => sub {
+        $compile_count++;
+        return $rules_module->original('compile')->(@_);
+    },
+);
+
+is_deeply(
+    PVE::HA::Config::get_resources_motion_info([sort(keys %$expected), 'vm:103', 'vm:999']),
+    $expected,
+    "motion info for multiple resources omits ignored and unmanaged ones",
+);
+is($compile_count, 1, "rules are compiled once for multiple resources");
+
+is_deeply(PVE::HA::Config::get_resources_motion_info([]), {}, "no motion info for no resources");
+
+is_deeply(
+    PVE::HA::Config::get_resources_requested_state([qw(vm:101 vm:103 ct:104 vm:999)]),
+    { 'vm:101' => 'started', 'ct:104' => 'started' },
+    "requested state of the HA-managed resources, without ignored and unmanaged ones",
+);
+
 # a migrating negative affinity partner occupies both its source and its target node
 $manager_status->{service_status}->{'vm:102'} =
     { state => 'migrate', node => 'node3', target => 'node2' };
@@ -182,4 +206,23 @@ my $expected_migrating = {
 
 $test_single_motion_info->($expected_migrating, 'migrating partner');
 
+is_deeply(
+    PVE::HA::Config::get_resources_motion_info([sort keys %$expected_migrating]),
+    $expected_migrating,
+    "migrating partner: motion info for multiple resources",
+);
+
+$manager_status->{disarm} = { mode => 'ignore' };
+
+is_deeply(
+    PVE::HA::Config::get_resources_motion_info([sort keys %$expected]),
+    {},
+    "no motion info while HA is disarmed with ignore mode",
+);
+is_deeply(
+    [PVE::HA::Config::get_resource_motion_info('vm:101')],
+    [[], {}],
+    "no single motion info while HA is disarmed with ignore mode",
+);
+
 done_testing();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH ha-manager 05/15] env: pve2: delegate cluster stat gathering to scheduling library
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (3 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH ha-manager 04/15] config: allow getting motion info of many resources at once Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH ha-manager 06/15] usage: share one scheduler base between static and dynamic Thomas Lamprecht
                   ` (9 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

The cluster node and guest stat gathering now lives in the
PVE::ResourceScheduling::Stats module of libpve-guest-common-perl.
Consume it here instead of keeping a second copy, so the HA manager
and the resource scheduling API share a single implementation and
cannot drift apart. The library's dynamic guest stats carry the host
memory accounting from fix #7974, and the stats the HA manager sees
stay the same.

The library has no dependency on the HA resource plugins, so build
the static per-guest-type usage resolvers from those plugins and pass
them in. As the library reports undecodable static node info as a
plain warning, log that at error level like before.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
NOTE: the new libpve-guest-common-perl dependency needs a version
requirement for the first release with PVE::ResourceScheduling.

 debian/control         |   2 +
 src/PVE/HA/Env/PVE2.pm | 155 ++++++++---------------------------------
 2 files changed, 32 insertions(+), 125 deletions(-)

diff --git a/debian/control b/debian/control
index 107e367..2577d43 100644
--- a/debian/control
+++ b/debian/control
@@ -8,6 +8,7 @@ Build-Depends: debhelper-compat (= 13),
                libpve-access-control,
                libpve-cluster-perl,
                libpve-common-perl,
+               libpve-guest-common-perl,
                libpve-notify-perl,
                libpve-rs-perl (>= 0.14.0),
                libtest-mockmodule-perl <!nocheck>,
@@ -23,6 +24,7 @@ Architecture: any
 Depends: libjson-perl,
          libpve-cluster-perl (>= 9.1.5),
          libpve-common-perl,
+         libpve-guest-common-perl,
          libpve-notify-perl (>= 8.1.0),
          libpve-rs-perl (>= 0.14.0),
          pve-cluster (>= 3.0-17),
diff --git a/src/PVE/HA/Env/PVE2.pm b/src/PVE/HA/Env/PVE2.pm
index 5f195da..de56e20 100644
--- a/src/PVE/HA/Env/PVE2.pm
+++ b/src/PVE/HA/Env/PVE2.pm
@@ -14,6 +14,7 @@ use PVE::DataCenterConfig;
 use PVE::INotify;
 use PVE::RPCEnvironment;
 use PVE::Notify;
+use PVE::ResourceScheduling::Stats;
 
 use PVE::HA::Tools ':exit_codes';
 use PVE::HA::Env;
@@ -36,30 +37,19 @@ PVE::HA::Rules::ResourceAffinity->register();
 
 PVE::HA::Rules->init(property_isolation => 1);
 
+# The stats library has no dependency on the HA resource plugins, so build the per-guest-type static
+# usage resolvers from those plugins and pass them in.
+my sub static_stats_resolvers {
+    return {
+        map {
+            my $plugin = PVE::HA::Resources->lookup($_);
+            ($_ => sub { $plugin->get_static_stats_from_config($_[0]) });
+        } qw(vm ct)
+    };
+}
+
 my $lockdir = "/etc/pve/priv/lock";
 
-# rrd entry indices for VM and CT guests
-# taken from PVE::Service::pvestatd::update_{lxc,qemu}_status()
-use constant {
-    RRD_VM_INDEX_STATUS => 2,
-    RRD_VM_INDEX_MAXCPU => 5,
-    RRD_VM_INDEX_CPU => 6,
-    RRD_VM_INDEX_MAXMEM => 7,
-    RRD_VM_INDEX_MEM => 8,
-    RRD_VM_INDEX_MEMHOST => 15,
-};
-
-# rrd entry indices for PVE nodes
-# taken from PVE::Service::pvestatd::update_node_status()
-use constant {
-    RRD_NODE_INDEX_MAXCPU => 4,
-    RRD_NODE_INDEX_CPU => 5,
-    RRD_NODE_INDEX_MAXMEM => 7,
-    RRD_NODE_INDEX_MEM => 8,
-};
-
-my $HOSTNAME_RE = qr/(?:[a-zA-Z0-9](?:[a-zA-Z0-9\-]{,61}?[a-zA-Z0-9])?)/;
-
 sub new {
     my ($this, $nodename) = @_;
 
@@ -529,133 +519,48 @@ sub get_datacenter_settings {
     };
 }
 
-my sub get_cluster_service_stats {
-    my $vmlist = PVE::Cluster::get_vmlist();
-    my $idlist = $vmlist->{ids} // {};
-
-    my $rrd = PVE::Cluster::rrd_dump();
-
-    my $stats = {};
-    for my $id (keys %$idlist) {
-        my $type = eval { PVE::HA::Tools::get_ha_resource_type($idlist->{$id}->{type}) };
-        next if $@; # silently ignore unknown pve types
-
-        my $sid = "$type:$id";
-        my $nodename = $idlist->{$id}->{node};
-
-        my $rrdentry = $rrd->{"pve-vm-9.0/$id"} // [];
-        # can be any QMP RunState, but 'running' is the only active VM state
-        my $status = $rrdentry->[RRD_VM_INDEX_STATUS] // "stopped";
-        my $state = $status eq "running" ? "started" : "stopped";
-
-        $stats->{$sid} = {
-            id => $id,
-            node => $nodename,
-            state => $state,
-            running => $state eq 'started',
-            type => $type,
-            usage => {},
-        };
-    }
-
-    return $stats;
-}
-
 sub get_basic_service_stats {
     my ($self) = @_;
 
-    return get_cluster_service_stats();
+    return PVE::ResourceScheduling::Stats::get_cluster_guest_stats();
 }
 
 sub get_static_service_stats {
     my ($self) = @_;
 
-    my $properties = ['cores', 'cpulimit', 'memory', 'sockets', 'vcpus'];
-    my $stats = get_cluster_service_stats();
-    my $confs = PVE::Cluster::get_guest_config_properties($properties);
-
-    for my $sid (keys %$stats) {
-        my ($id, $type) = $stats->{$sid}->@{qw(id type)};
-
-        my $conf = $confs->{$id} // {};
-        my $plugin = PVE::HA::Resources->lookup($type);
-
-        $stats->{$sid}->{usage} = $plugin->get_static_stats_from_config($conf);
-    }
-
-    return $stats;
+    return PVE::ResourceScheduling::Stats::get_static_guest_stats(static_stats_resolvers());
 }
 
 sub get_dynamic_service_stats {
     my ($self) = @_;
 
-    my $rrd = PVE::Cluster::rrd_dump();
-
-    my $stats = get_cluster_service_stats();
-    for my $sid (keys %$stats) {
-        my $id = $stats->{$sid}->{id};
-        my $rrdentry = $rrd->{"pve-vm-9.0/$id"} // [];
-
-        # NOTE the guests' broadcasted vmstatus() caps maxcpu at the node's maxcpu
-        my $maxcpu = ($rrdentry->[RRD_VM_INDEX_MAXCPU] || 0.0) + 0.0;
-
-        # designated fallback for containers
-        my $mem = $rrdentry->[RRD_VM_INDEX_MEMHOST] // $rrdentry->[RRD_VM_INDEX_MEM];
-
-        $stats->{$sid}->{usage} = {
-            maxcpu => $maxcpu,
-            cpu => (($rrdentry->[RRD_VM_INDEX_CPU] || 0.0) + 0.0) * $maxcpu,
-            maxmem => int($rrdentry->[RRD_VM_INDEX_MAXMEM] || 0),
-            mem => int($mem || 0),
-        };
-    }
-
-    return $stats;
+    return PVE::ResourceScheduling::Stats::get_dynamic_guest_stats();
 }
 
 sub get_static_node_stats {
     my ($self) = @_;
 
-    my $stats = PVE::Cluster::get_node_kv('static-info');
-    for my $node (keys $stats->%*) {
-        $stats->{$node} = eval {
-            my $node_stats = decode_json($stats->{$node});
+    # The library reports undecodable static node info with warn(). Log that at 'err' level, as an
+    # online node without static info makes the CRM fall back to basic scheduling.
+    my $previous = $SIG{__WARN__};
+    local $SIG{__WARN__} = sub {
+        my ($msg) = @_;
+        if ($msg =~ m/^unable to decode static node info /) {
+            $self->log('err', $msg);
+        } elsif (ref($previous) eq 'CODE') {
+            $previous->($msg);
+        } else {
+            warn $msg;
+        }
+    };
 
-            return {
-                maxcpu => $node_stats->{cpus},
-                maxmem => $node_stats->{memory},
-            };
-        };
-        $self->log('err', "unable to decode static node info for '$node' - $@") if $@;
-    }
-
-    return $stats;
+    return PVE::ResourceScheduling::Stats::get_static_node_stats();
 }
 
 sub get_dynamic_node_stats {
     my ($self) = @_;
 
-    my $rrd = PVE::Cluster::rrd_dump();
-
-    my $stats = {};
-    for my $key (keys %$rrd) {
-        my ($nodename) = $key =~ m/^pve-node-9.0\/($HOSTNAME_RE)$/;
-
-        next if !$nodename;
-
-        my $rrdentry = $rrd->{$key} // [];
-
-        my $maxcpu = int($rrdentry->[RRD_NODE_INDEX_MAXCPU] || 0);
-
-        $stats->{$nodename} = {
-            maxcpu => $maxcpu,
-            cpu => (($rrdentry->[RRD_NODE_INDEX_CPU] || 0.0) + 0.0) * $maxcpu,
-            maxmem => int($rrdentry->[RRD_NODE_INDEX_MAXMEM] || 0),
-            mem => int($rrdentry->[RRD_NODE_INDEX_MEM] || 0),
-        };
-    }
-
-    return $stats;
+    return PVE::ResourceScheduling::Stats::get_dynamic_node_stats();
 }
 
 sub get_node_version {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH ha-manager 06/15] usage: share one scheduler base between static and dynamic
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (4 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH ha-manager 05/15] env: pve2: delegate cluster stat gathering to scheduling library Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH manager 07/15] ui: bulk action: translate error title and CT restart warning Thomas Lamprecht
                   ` (8 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

The static and dynamic usage drivers each wrapped the resource
scheduling bindings directly, in two near-identical copies. Route them
through PVE::ResourceScheduling::Scheduler from the scheduling library
and fold their shared logic into a common base, so static and dynamic
differ only in the mode they select. The HA stack and the non-HA
resource scheduling API then share one scheduler wrapper rather than
copies that can drift.

The wrapper forwards every call to the same bindings with the same
arguments. The HA simulation tests produce byte-identical logs before
and after this change, and their static and dynamic CRS cases fail
when the library is missing, so they do exercise the new code path.

The one intended difference is the handling of a failure to score
nodes for a service: dynamic mode now also falls back to ranking the
nodes by their service count, like static mode does. Before, it
returned no scores, so the caller placed the service on the first
node by name and logged uninitialized value warnings. Extend the
fallback test to cover dynamic mode.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 debian/pve-ha-manager.install   |   1 +
 src/PVE/HA/Usage/Dynamic.pm     | 145 +-------------------------
 src/PVE/HA/Usage/Makefile       |   2 +-
 src/PVE/HA/Usage/Scheduler.pm   | 175 ++++++++++++++++++++++++++++++++
 src/PVE/HA/Usage/Static.pm      | 160 +----------------------------
 src/test/test_usage_fallback.pl |   7 +-
 6 files changed, 187 insertions(+), 303 deletions(-)
 create mode 100644 src/PVE/HA/Usage/Scheduler.pm

diff --git a/debian/pve-ha-manager.install b/debian/pve-ha-manager.install
index 301e9fc..4451dd3 100644
--- a/debian/pve-ha-manager.install
+++ b/debian/pve-ha-manager.install
@@ -41,6 +41,7 @@
 /usr/share/perl5/PVE/HA/Tools.pm
 /usr/share/perl5/PVE/HA/Usage.pm
 /usr/share/perl5/PVE/HA/Usage/Basic.pm
+/usr/share/perl5/PVE/HA/Usage/Scheduler.pm
 /usr/share/perl5/PVE/HA/Usage/Static.pm
 /usr/share/perl5/PVE/HA/Usage/Dynamic.pm
 /usr/share/perl5/PVE/Service/pve_ha_crm.pm
diff --git a/src/PVE/HA/Usage/Dynamic.pm b/src/PVE/HA/Usage/Dynamic.pm
index 76d0fea..05be1a5 100644
--- a/src/PVE/HA/Usage/Dynamic.pm
+++ b/src/PVE/HA/Usage/Dynamic.pm
@@ -3,153 +3,12 @@ package PVE::HA::Usage::Dynamic;
 use strict;
 use warnings;
 
-use PVE::HA::Resources;
-use PVE::RS::ResourceScheduling::Dynamic;
-
-use base qw(PVE::HA::Usage);
+use base qw(PVE::HA::Usage::Scheduler);
 
 sub new {
     my ($class, $haenv, $service_stats) = @_;
 
-    my $node_stats = eval { $haenv->get_dynamic_node_stats() };
-    die "did not get dynamic node usage information - $@" if $@;
-
-    my $scheduler = eval { PVE::RS::ResourceScheduling::Dynamic->new() };
-    die "unable to initialize dynamic scheduling - $@" if $@;
-
-    return bless {
-        'node-stats' => $node_stats,
-        'service-stats' => $service_stats,
-        haenv => $haenv,
-        scheduler => $scheduler,
-    }, $class;
-}
-
-sub add_node {
-    my ($self, $nodename) = @_;
-
-    my $stats = $self->{'node-stats'}->{$nodename}
-        or die "did not get dynamic node usage information for '$nodename'\n";
-    die "dynamic node usage information for '$nodename' missing cpu count\n"
-        if !defined($stats->{maxcpu});
-    die "dynamic node usage information for '$nodename' missing memory\n"
-        if !defined($stats->{maxmem});
-
-    eval { $self->{scheduler}->add_node($nodename, $stats); };
-    die "initializing dynamic node usage for '$nodename' failed - $@" if $@;
-}
-
-sub remove_node {
-    my ($self, $nodename) = @_;
-
-    $self->{scheduler}->remove_node($nodename);
-}
-
-sub list_nodes {
-    my ($self) = @_;
-
-    return $self->{scheduler}->list_nodes()->@*;
-}
-
-sub contains_node {
-    my ($self, $nodename) = @_;
-
-    return $self->{scheduler}->contains_node($nodename);
-}
-
-my sub get_service_usage {
-    my ($self, $sid) = @_;
-
-    my $service_stats = $self->{'service-stats'}->{$sid}->{usage}
-        or die "did not get dynamic service usage information for '$sid'\n";
-
-    return $service_stats;
-}
-
-sub add_service {
-    my ($self, $sid, $current_node, $target_node, $running) = @_;
-
-    # do not add service, which does not put any usage on the nodes
-    return if !defined($current_node) && !defined($target_node);
-
-    # PVE::RS::ResourceScheduling::Dynamic::add_resource() expects $current_node
-    # to be set, so consider $target_node as $current_node for unset $current_node;
-    #
-    # currently, this happens for the request_start_balance service state and if
-    # node maintenance causes services to migrate to other nodes
-    if (!defined($current_node)) {
-        $current_node = $target_node;
-        undef $target_node;
-    }
-
-    eval {
-        my $service_usage = get_service_usage($self, $sid);
-
-        my $service = {
-            stats => $service_usage,
-            running => $running,
-            'current-node' => $current_node,
-            'target-node' => $target_node,
-        };
-
-        $self->{scheduler}->add_resource($sid, $service);
-    };
-    $self->{haenv}->log('warning', "unable to add service '$sid' - $@") if $@;
-}
-
-sub remove_service_usage {
-    my ($self, $sid) = @_;
-
-    eval { $self->{scheduler}->remove_resource($sid) };
-    $self->{haenv}->log('warning', "unable to remove service '$sid' usage - $@") if $@;
-}
-
-sub calculate_node_imbalance {
-    my ($self) = @_;
-
-    my $node_imbalance = eval { $self->{scheduler}->calculate_node_imbalance() };
-    $self->{haenv}->log('warning', "unable to calculate dynamic node imbalance - $@") if $@;
-
-    return $node_imbalance // 0.0;
-}
-
-sub score_best_balancing_migrations {
-    my ($self, $migration_candidates, $limit) = @_;
-
-    my $migrations = eval {
-        $self->{scheduler}
-            ->score_best_balancing_migration_candidates($migration_candidates, $limit);
-    };
-    $self->{haenv}->log('warning', "unable to score best balancing migration - $@") if $@;
-
-    return $migrations;
-}
-
-sub score_best_balancing_migrations_topsis {
-    my ($self, $migration_candidates, $limit) = @_;
-
-    my $migrations = eval {
-        $self->{scheduler}
-            ->score_best_balancing_migration_candidates_topsis($migration_candidates, $limit);
-    };
-    $self->{haenv}->log('warning', "unable to score best balancing migration - $@") if $@;
-
-    return $migrations;
-}
-
-sub score_nodes_to_start_service {
-    my ($self, $sid) = @_;
-
-    my $score_list = eval {
-        my $service_usage = get_service_usage($self, $sid);
-        $self->{scheduler}->score_nodes_to_start_resource($service_usage);
-    };
-    $self->{haenv}
-        ->log('err', "unable to score nodes according to dynamic usage for service '$sid' - $@")
-        if $@;
-
-    # Take minus the value, so that a lower score is better, which our caller(s) expect(s).
-    return { map { $_->[0] => -$_->[1] } $score_list->@* };
+    return $class->SUPER::new($haenv, $service_stats, 'dynamic');
 }
 
 1;
diff --git a/src/PVE/HA/Usage/Makefile b/src/PVE/HA/Usage/Makefile
index 5d51a9c..619e0a3 100644
--- a/src/PVE/HA/Usage/Makefile
+++ b/src/PVE/HA/Usage/Makefile
@@ -1,5 +1,5 @@
 SIM_SOURCES=Basic.pm
-SOURCES=${SIM_SOURCES} Static.pm Dynamic.pm
+SOURCES=${SIM_SOURCES} Scheduler.pm Static.pm Dynamic.pm
 
 .PHONY: install
 install:
diff --git a/src/PVE/HA/Usage/Scheduler.pm b/src/PVE/HA/Usage/Scheduler.pm
new file mode 100644
index 0000000..777ac5a
--- /dev/null
+++ b/src/PVE/HA/Usage/Scheduler.pm
@@ -0,0 +1,175 @@
+package PVE::HA::Usage::Scheduler;
+
+use strict;
+use warnings;
+
+use PVE::ResourceScheduling::Scheduler;
+
+use base qw(PVE::HA::Usage);
+
+# Shared base for the resource-scheduling-backed HA usage drivers; PVE::HA::Usage::Static and
+# ::Dynamic only pick the mode. The mode selects the matching PVE::ResourceScheduling::Scheduler
+# backend and the haenv node-stats getter; everything else is identical.
+
+sub new {
+    my ($class, $haenv, $service_stats, $mode) = @_;
+
+    my $node_stats = eval {
+        $mode eq 'static' ? $haenv->get_static_node_stats() : $haenv->get_dynamic_node_stats();
+    };
+    die "did not get $mode node usage information - $@" if $@;
+
+    my $scheduler = eval { PVE::ResourceScheduling::Scheduler->new($mode) };
+    die "unable to initialize $mode scheduling - $@" if $@;
+
+    return bless {
+        mode => $mode,
+        'node-stats' => $node_stats,
+        'service-stats' => $service_stats,
+        haenv => $haenv,
+        scheduler => $scheduler,
+        'node-services' => {}, # services on each node, fallback if scoring calculation fails
+    }, $class;
+}
+
+sub add_node {
+    my ($self, $nodename) = @_;
+
+    $self->{'node-services'}->{$nodename} = {};
+
+    my $stats = $self->{'node-stats'}->{$nodename}
+        or die "did not get $self->{mode} node usage information for '$nodename'\n";
+    die "$self->{mode} node usage information for '$nodename' missing cpu count\n"
+        if !defined($stats->{maxcpu});
+    die "$self->{mode} node usage information for '$nodename' missing memory\n"
+        if !defined($stats->{maxmem});
+
+    eval { $self->{scheduler}->add_node($nodename, $stats); };
+    die "initializing $self->{mode} node usage for '$nodename' failed - $@" if $@;
+}
+
+sub remove_node {
+    my ($self, $nodename) = @_;
+
+    delete $self->{'node-services'}->{$nodename};
+
+    $self->{scheduler}->remove_node($nodename);
+}
+
+sub list_nodes {
+    my ($self) = @_;
+
+    return $self->{scheduler}->list_nodes();
+}
+
+sub contains_node {
+    my ($self, $nodename) = @_;
+
+    return $self->{scheduler}->contains_node($nodename);
+}
+
+my sub get_service_usage {
+    my ($self, $sid) = @_;
+
+    my $service_stats = $self->{'service-stats'}->{$sid}->{usage}
+        or die "did not get $self->{mode} service usage information for '$sid'\n";
+
+    return $service_stats;
+}
+
+sub add_service {
+    my ($self, $sid, $current_node, $target_node, $running) = @_;
+
+    # do not add services which do not put any usage on the nodes
+    return if !defined($current_node) && !defined($target_node);
+
+    # the scheduler backend expects $current_node to be set, so consider $target_node as
+    # $current_node for unset $current_node; currently, this happens for the request_start_balance
+    # service state and if node maintenance causes services to migrate to other nodes
+    if (!defined($current_node)) {
+        $current_node = $target_node;
+        undef $target_node;
+    }
+
+    $self->{'node-services'}->{$current_node}->{$sid} = 1;
+    $self->{'node-services'}->{$target_node}->{$sid} = 1 if defined($target_node);
+
+    eval {
+        my $service_usage = get_service_usage($self, $sid);
+
+        my $service = {
+            stats => $service_usage,
+            running => $running,
+            'current-node' => $current_node,
+            'target-node' => $target_node,
+        };
+
+        $self->{scheduler}->add_resource($sid, $service);
+    };
+    $self->{haenv}->log('warning', "unable to add service '$sid' - $@") if $@;
+}
+
+sub remove_service_usage {
+    my ($self, $sid) = @_;
+
+    delete($self->{'node-services'}->{$_}->{$sid}) for $self->list_nodes();
+
+    eval { $self->{scheduler}->remove_resource($sid) };
+    $self->{haenv}->log('warning', "unable to remove service '$sid' usage - $@") if $@;
+}
+
+sub calculate_node_imbalance {
+    my ($self) = @_;
+
+    my $node_imbalance = eval { $self->{scheduler}->calculate_node_imbalance() };
+    $self->{haenv}->log('warning', "unable to calculate $self->{mode} node imbalance - $@") if $@;
+
+    return $node_imbalance // 0.0;
+}
+
+sub score_best_balancing_migrations {
+    my ($self, $migration_candidates, $limit) = @_;
+
+    my $migrations = eval {
+        $self->{scheduler}
+            ->score_best_balancing_migration_candidates($migration_candidates, $limit);
+    };
+    $self->{haenv}->log('warning', "unable to score best balancing migration - $@") if $@;
+
+    return $migrations;
+}
+
+sub score_best_balancing_migrations_topsis {
+    my ($self, $migration_candidates, $limit) = @_;
+
+    my $migrations = eval {
+        $self->{scheduler}->score_best_balancing_migration_candidates(
+            $migration_candidates, $limit, 'topsis',
+        );
+    };
+    $self->{haenv}->log('warning', "unable to score best balancing migration - $@") if $@;
+
+    return $migrations;
+}
+
+sub score_nodes_to_start_service {
+    my ($self, $sid) = @_;
+
+    my $score_list = eval {
+        my $service_usage = get_service_usage($self, $sid);
+        $self->{scheduler}->score_nodes_to_start($service_usage);
+    };
+    if (my $err = $@) {
+        $self->{haenv}->log(
+            'err',
+            "unable to score nodes according to $self->{mode} usage for service '$sid' - $err",
+        );
+        my $node_services = $self->{'node-services'};
+        return { map { $_ => scalar(keys $node_services->{$_}->%*) } keys $node_services->%* };
+    }
+
+    # Take minus the value, so that a lower score is better, which our caller(s) expect(s).
+    return { map { $_->[0] => -$_->[1] } $score_list->@* };
+}
+
+1;
diff --git a/src/PVE/HA/Usage/Static.pm b/src/PVE/HA/Usage/Static.pm
index 570a801..a7ee0a7 100644
--- a/src/PVE/HA/Usage/Static.pm
+++ b/src/PVE/HA/Usage/Static.pm
@@ -3,168 +3,12 @@ package PVE::HA::Usage::Static;
 use strict;
 use warnings;
 
-use PVE::HA::Resources;
-use PVE::RS::ResourceScheduling::Static;
-
-use base qw(PVE::HA::Usage);
+use base qw(PVE::HA::Usage::Scheduler);
 
 sub new {
     my ($class, $haenv, $service_stats) = @_;
 
-    my $node_stats = eval { $haenv->get_static_node_stats() };
-    die "did not get static node usage information - $@" if $@;
-
-    my $scheduler = eval { PVE::RS::ResourceScheduling::Static->new(); };
-    die "unable to initialize static scheduling - $@" if $@;
-
-    return bless {
-        'node-stats' => $node_stats,
-        'service-stats' => $service_stats,
-        haenv => $haenv,
-        scheduler => $scheduler,
-        'node-services' => {}, # Services on each node. Fallback if scoring calculation fails.
-    }, $class;
-}
-
-sub add_node {
-    my ($self, $nodename) = @_;
-
-    $self->{'node-services'}->{$nodename} = {};
-
-    my $stats = $self->{'node-stats'}->{$nodename}
-        or die "did not get static node usage information for '$nodename'\n";
-    die "static node usage information for '$nodename' missing cpu count\n"
-        if !defined($stats->{maxcpu});
-    die "static node usage information for '$nodename' missing memory\n"
-        if !defined($stats->{maxmem});
-
-    eval { $self->{scheduler}->add_node($nodename, int($stats->{maxcpu}), int($stats->{maxmem})); };
-    die "initializing static node usage for '$nodename' failed - $@" if $@;
-}
-
-sub remove_node {
-    my ($self, $nodename) = @_;
-
-    delete $self->{'node-services'}->{$nodename};
-
-    $self->{scheduler}->remove_node($nodename);
-}
-
-sub list_nodes {
-    my ($self) = @_;
-
-    return $self->{scheduler}->list_nodes()->@*;
-}
-
-sub contains_node {
-    my ($self, $nodename) = @_;
-
-    return $self->{scheduler}->contains_node($nodename);
-}
-
-my sub get_service_usage {
-    my ($self, $sid) = @_;
-
-    my $service_stats = $self->{'service-stats'}->{$sid}->{usage}
-        or die "did not get static service usage information for '$sid'\n";
-
-    return $service_stats;
-}
-
-sub add_service {
-    my ($self, $sid, $current_node, $target_node, $running) = @_;
-
-    # do not add service which do not put any usage on the nodes
-    return if !defined($current_node) && !defined($target_node);
-
-    # PVE::RS::ResourceScheduling::Static::add_service() expects $current_node
-    # to be set, so consider $target_node as $current_node for unset $current_node;
-    #
-    # currently, this happens for the request_start_balance service state and if
-    # node maintenance causes services to migrate to other nodes
-    if (!defined($current_node)) {
-        $current_node = $target_node;
-        undef $target_node;
-    }
-
-    $self->{'node-services'}->{$current_node}->{$sid} = 1;
-    $self->{'node-services'}->{$target_node}->{$sid} = 1 if defined($target_node);
-
-    eval {
-        my $service_usage = get_service_usage($self, $sid);
-
-        my $service = {
-            stats => $service_usage,
-            running => $running,
-            'current-node' => $current_node,
-            'target-node' => $target_node,
-        };
-
-        $self->{scheduler}->add_service($sid, $service);
-    };
-    $self->{haenv}->log('warning', "unable to add service '$sid' - $@") if $@;
-}
-
-sub remove_service_usage {
-    my ($self, $sid) = @_;
-
-    delete($self->{'node-services'}->{$_}->{$sid}) for $self->list_nodes();
-
-    eval { $self->{scheduler}->remove_service_usage($sid) };
-    $self->{haenv}->log('warning', "unable to remove service '$sid' usage - $@") if $@;
-}
-
-sub calculate_node_imbalance {
-    my ($self) = @_;
-
-    my $node_imbalance = eval { $self->{scheduler}->calculate_node_imbalance() };
-    $self->{haenv}->log('warning', "unable to calculate static node imbalance - $@") if $@;
-
-    return $node_imbalance // 0.0;
-}
-
-sub score_best_balancing_migrations {
-    my ($self, $migration_candidates, $limit) = @_;
-
-    my $migrations = eval {
-        $self->{scheduler}
-            ->score_best_balancing_migration_candidates($migration_candidates, $limit);
-    };
-    $self->{haenv}->log('warning', "unable to score best balancing migration - $@") if $@;
-
-    return $migrations;
-}
-
-sub score_best_balancing_migrations_topsis {
-    my ($self, $migration_candidates, $limit) = @_;
-
-    my $migrations = eval {
-        $self->{scheduler}
-            ->score_best_balancing_migration_candidates_topsis($migration_candidates, $limit);
-    };
-    $self->{haenv}->log('warning', "unable to score best balancing migration - $@") if $@;
-
-    return $migrations;
-}
-
-sub score_nodes_to_start_service {
-    my ($self, $sid) = @_;
-
-    my $score_list = eval {
-        my $service_usage = get_service_usage($self, $sid);
-        $self->{scheduler}->score_nodes_to_start_service($service_usage);
-    };
-    if (my $err = $@) {
-        $self->{haenv}->log(
-            'err',
-            "unable to score nodes according to static usage for service '$sid' - $err",
-        );
-        my $node_services = $self->{'node-services'};
-        return { map { $_ => scalar(keys $node_services->{$_}->%*) } keys $node_services->%* };
-    }
-
-    # Take minus the value, so that a lower score is better, which our caller(s) expect(s).
-    return { map { $_->[0] => -$_->[1] } $score_list->@* };
+    return $class->SUPER::new($haenv, $service_stats, 'static');
 }
 
 1;
diff --git a/src/test/test_usage_fallback.pl b/src/test/test_usage_fallback.pl
index b9bb6dd..e1d1d46 100755
--- a/src/test/test_usage_fallback.pl
+++ b/src/test/test_usage_fallback.pl
@@ -6,6 +6,7 @@ use lib qw(..);
 
 use Test::More;
 
+use PVE::HA::Usage::Dynamic;
 use PVE::HA::Usage::Static;
 
 my $GiB = 1024**3;
@@ -20,6 +21,10 @@ package MockEnv {
         return $self->{node_stats};
     }
 
+    sub get_dynamic_node_stats($self) {
+        return $self->{node_stats};
+    }
+
     sub log($self, $level, $msg) {
         push $self->{logs}->@*, "$level: $msg";
     }
@@ -67,6 +72,6 @@ my $test_fallback = sub($class) {
     );
 };
 
-$test_fallback->('PVE::HA::Usage::Static');
+$test_fallback->($_) for qw(PVE::HA::Usage::Static PVE::HA::Usage::Dynamic);
 
 done_testing();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH manager 07/15] ui: bulk action: translate error title and CT restart warning
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (5 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH ha-manager 06/15] usage: share one scheduler base between static and dynamic Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH manager 08/15] ui: bulk migrate: show CT restart warning for cluster-wide migration Thomas Lamprecht
                   ` (7 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

Neither string was marked for translation, so they showed up in
English in otherwise localized dialogs.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 www/manager6/window/BulkAction.js | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/www/manager6/window/BulkAction.js b/www/manager6/window/BulkAction.js
index 7e58fb3fb..139f7a425 100644
--- a/www/manager6/window/BulkAction.js
+++ b/www/manager6/window/BulkAction.js
@@ -36,7 +36,7 @@ Ext.define('PVE.window.BulkAction', {
             url,
             waitMsgTarget: me,
             method: 'POST',
-            failure: (response) => Ext.Msg.alert('Error', response.htmlStatus),
+            failure: (response) => Ext.Msg.alert(gettext('Error'), response.htmlStatus),
             success: function ({ result }, options) {
                 Ext.create('Proxmox.window.TaskViewer', {
                     autoShow: true,
@@ -116,7 +116,9 @@ Ext.define('PVE.window.BulkAction', {
                             itemId: 'lxcwarning',
                             xtype: 'displayfield',
                             userCls: 'pmx-hint',
-                            value: 'Warning: Running CTs will be migrated in Restart Mode.',
+                            value: gettext(
+                                'Warning: Running CTs will be migrated in Restart Mode.',
+                            ),
                             hidden: true, // only visible if running container chosen
                             flex: 1,
                         },
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH manager 08/15] ui: bulk migrate: show CT restart warning for cluster-wide migration
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (6 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH manager 07/15] ui: bulk action: translate error title and CT restart warning Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH manager 09/15] ui: migrate: recheck preconditions if the target changed meanwhile Thomas Lamprecht
                   ` (6 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

The Datacenter bulk migrate dialog contains the warning that running
containers are migrated in restart mode, but only the node-level
dialog ever updated its visibility, so it was never shown there.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 www/manager6/window/BulkAction.js | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/www/manager6/window/BulkAction.js b/www/manager6/window/BulkAction.js
index 139f7a425..3f8ad5674 100644
--- a/www/manager6/window/BulkAction.js
+++ b/www/manager6/window/BulkAction.js
@@ -321,7 +321,7 @@ Ext.define('PVE.window.BulkAction', {
                 filterFn,
             });
             vmselector.checkChange();
-            if (me.action === 'migrateall') {
+            if (me.action === 'migrateall' || me.action === 'migrate') {
                 let records = vmselector.getSelection();
                 refreshLxcWarning(vmselector.getValue(), records);
             }
@@ -518,7 +518,7 @@ Ext.define('PVE.window.BulkAction', {
             nodename: me.nodename,
             listeners: {
                 selectionchange: function (vmselector, records) {
-                    if (me.action === 'migrateall') {
+                    if (me.action === 'migrateall' || me.action === 'migrate') {
                         let vmids = me.down('#vms').getValue();
                         refreshLxcWarning(vmids, records);
                     }
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH manager 09/15] ui: migrate: recheck preconditions if the target changed meanwhile
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (7 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH manager 08/15] ui: bulk migrate: show CT restart warning for cluster-wide migration Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH manager 10/15] api: cluster: add resource scheduling endpoints Thomas Lamprecht
                   ` (5 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

While the migration preconditions of a VM are being checked, a check
requested for a newly selected target node is skipped. If the target
changed after the running check read it, the dialog showed the result
for the previous target, which can block the migration to a suitable
node or allow it to an unsuitable one. Redo the check in that case.

Also end a check whose dialog was closed meanwhile, instead of failing
on the fields that are gone.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 www/manager6/window/Migrate.js | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/www/manager6/window/Migrate.js b/www/manager6/window/Migrate.js
index 23090b7aa..40bca4228 100644
--- a/www/manager6/window/Migrate.js
+++ b/www/manager6/window/Migrate.js
@@ -206,6 +206,9 @@ Ext.define('PVE.window.Migrate', {
             } else {
                 await me.checkLxcPreconditions(resetMigrationPossible);
             }
+            if (me.destroyed) {
+                return;
+            }
 
             // Only allow nodes where the local storage is available in case of offline migration
             // where storage migration is not possible
@@ -243,6 +246,9 @@ Ext.define('PVE.window.Migrate', {
                 me.fetchingNodeMigrateInfo = false;
                 return;
             }
+            if (me.destroyed) {
+                return;
+            }
 
             const target = me.lookup('pveNodeSelector').value;
             let targetCapabilities = {};
@@ -261,6 +267,15 @@ Ext.define('PVE.window.Migrate', {
 
             me.fetchingNodeMigrateInfo = false;
 
+            if (me.destroyed) {
+                return;
+            }
+            // a check for a target selected meanwhile was skipped, so redo it for the current one
+            if (me.lookup('pveNodeSelector').value !== target) {
+                await me.checkQemuPreconditions(resetMigrationPossible);
+                return;
+            }
+
             if (migrateStats.running) {
                 vm.set('running', true);
             }
@@ -503,6 +518,9 @@ Ext.define('PVE.window.Migrate', {
                 me.fetchingNodeMigrateInfo = false;
                 return;
             }
+            if (me.destroyed) {
+                return;
+            }
 
             if (migrateStats.running) {
                 vm.set('running', true);
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH manager 10/15] api: cluster: add resource scheduling endpoints
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (8 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH manager 09/15] ui: migrate: recheck preconditions if the target changed meanwhile Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH manager 11/15] ui: migrate: sort target nodes by scheduler recommendation Thomas Lamprecht
                   ` (4 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

The cluster resource scheduler, which HA uses to place and rebalance
its resources, is not available for guests that HA does not manage.
Add two endpoints based on the PVE::ResourceScheduling library from
libpve-guest-common-perl, so the web interface can use it for all
guests.

recommend-target-nodes ranks the nodes as migration targets for a
guest by the cluster load imbalance the migration would result in.

rebalance-plan proposes up to a configurable number of migrations that
lower the cluster load imbalance, optionally only onto a set of target
nodes, for example to fill newly added nodes. Locked guests cannot be
migrated right now and are not proposed. It is a POST request, as the
list of guests to consider can exceed the request line limit of GET.

Guests managed by HA are only proposed if HA would carry out their
migration, that is, if their HA resource should be started and currently
is, and HA is not disarmed. They are only proposed for nodes that HA
considers online and their HA rules allow, as HA does not carry out
other migrations. Guests in a negative resource affinity are kept apart
across the whole proposal, as checking each move against the current
placement alone could still put them onto the same node. An HA guest
that HA would migrate together with other resources due to a positive
resource affinity is not proposed at all, as a per-guest proposal cannot
represent such a group move. Guests that are not proposed still count
towards the load of their node.

Both use the measured load and fall back to the configured guest
resources if a node did not report its load yet. Nodes in HA
maintenance mode are not considered, as they are about to be drained.
Only the load is considered; whether a migration is possible, for
example with regard to local resources, is still determined by the
existing migration precondition checks.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
NOTE: needs bumped version requirements for libpve-guest-common-perl
and pve-ha-manager, for their first releases with this series.

 PVE/API2/Cluster.pm                    |   7 +
 PVE/API2/Cluster/Makefile              |   3 +-
 PVE/API2/Cluster/ResourceScheduling.pm | 413 ++++++++++++++++++++++
 test/Makefile                          |   6 +-
 test/ResourceScheduling_test.pl        | 454 +++++++++++++++++++++++++
 5 files changed, 881 insertions(+), 2 deletions(-)
 create mode 100644 PVE/API2/Cluster/ResourceScheduling.pm
 create mode 100755 test/ResourceScheduling_test.pl

diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index c487980a4..39b0eaf24 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -33,6 +33,7 @@ use PVE::API2::Cluster::Mapping;
 use PVE::API2::Cluster::MetricServer;
 use PVE::API2::Cluster::Notifications;
 use PVE::API2::Cluster::Qemu;
+use PVE::API2::Cluster::ResourceScheduling;
 use PVE::API2::ClusterConfig;
 use PVE::API2::Firewall::Cluster;
 use PVE::API2::HAConfig;
@@ -116,6 +117,11 @@ __PACKAGE__->register_method({
     path => 'bulk-action',
 });
 
+__PACKAGE__->register_method({
+    subclass => "PVE::API2::Cluster::ResourceScheduling",
+    path => 'resource-scheduling',
+});
+
 if ($have_sdn) {
     __PACKAGE__->register_method({
         subclass => "PVE::API2::Network::SDN",
@@ -175,6 +181,7 @@ __PACKAGE__->register_method({
             { name => 'options' },
             { name => 'qemu' },
             { name => 'replication' },
+            { name => 'resource-scheduling' },
             { name => 'resources' },
             { name => 'status' },
             { name => 'tasks' },
diff --git a/PVE/API2/Cluster/Makefile b/PVE/API2/Cluster/Makefile
index d3a56830c..83a407931 100644
--- a/PVE/API2/Cluster/Makefile
+++ b/PVE/API2/Cluster/Makefile
@@ -14,7 +14,8 @@ PERLSOURCE= 			\
 	Mapping.pm			\
 	MetricServer.pm		\
 	Notifications.pm	\
-	Qemu.pm
+	Qemu.pm				\
+	ResourceScheduling.pm
 
 all:
 
diff --git a/PVE/API2/Cluster/ResourceScheduling.pm b/PVE/API2/Cluster/ResourceScheduling.pm
new file mode 100644
index 000000000..428b9770a
--- /dev/null
+++ b/PVE/API2/Cluster/ResourceScheduling.pm
@@ -0,0 +1,413 @@
+package PVE::API2::Cluster::ResourceScheduling;
+
+use v5.36;
+
+use PVE::Cluster;
+use PVE::Exception qw(raise_param_exc raise_perm_exc);
+use PVE::HA::Config;
+use PVE::INotify;
+use PVE::JSONSchema qw(get_standard_option);
+use PVE::LXC::Config;
+use PVE::QemuConfig;
+use PVE::RPCEnvironment;
+use PVE::Tools;
+
+use PVE::ResourceScheduling::Planner;
+
+use base qw(PVE::RESTHandler);
+
+# guest types as reported by the cluster guest list, mapped to the scheduler's resource id prefix
+my $guest_type_map = {
+    qemu => 'vm',
+    lxc => 'ct',
+};
+
+my $guest_type_by_prefix = { reverse %$guest_type_map };
+
+# The static (config-derived) per-guest-type usage resolvers the planner needs in static mode; the
+# HA daemons build their own from the HA resource plugins.
+my sub static_stats_resolvers {
+    return {
+        vm => sub($conf) {
+            return {
+                maxcpu => 0.0 + PVE::QemuConfig->get_derived_property($conf, 'max-cpu'),
+                maxmem => int(PVE::QemuConfig->get_derived_property($conf, 'max-memory')),
+            };
+        },
+        ct => sub($conf) {
+            return {
+                maxcpu => 0.0 + PVE::LXC::Config->get_derived_property($conf, 'max-cpu'),
+                maxmem => int(PVE::LXC::Config->get_derived_property($conf, 'max-memory')),
+            };
+        },
+    };
+}
+
+# The nodes that can currently take guests. A node in HA maintenance mode is about to be drained,
+# and as it is usually the least loaded node it would otherwise be the top recommendation.
+my sub schedulable_nodes($ha_status = PVE::HA::Config::read_manager_status()) {
+    my $members = PVE::Cluster::get_members() // {};
+    my $online = { map { $_ => 1 } grep { $members->{$_}->{online} } keys %$members };
+    # the local node serving this request is always up, even if pmxcfs has not (yet) broadcast its
+    # own membership, e.g. right after a restart; this also covers a standalone, non-clustered host
+    $online->{ PVE::INotify::nodename() } = 1;
+
+    my $ha_node_status = $ha_status->{node_status} // {};
+    delete $online->{$_} for grep { ($ha_node_status->{$_} // '') eq 'maintenance' } keys %$online;
+
+    return [sort keys %$online];
+}
+
+my sub guest_sid($vmid, $param_name) {
+    my $entry = (PVE::Cluster::get_vmlist()->{ids} // {})->{$vmid}
+        or raise_param_exc({ $param_name => "guest '$vmid' does not exist" });
+
+    my $type = $guest_type_map->{ $entry->{type} // '' }
+        or raise_param_exc(
+            { $param_name => "guest '$vmid' has unhandled type '" . ($entry->{type} // '') . "'" });
+
+    return "$type:$vmid";
+}
+
+my $mode_property = {
+    type => 'string',
+    enum => ['auto', 'dynamic', 'static'],
+    default => 'auto',
+    optional => 1,
+    description => "Usage information to base the result on. 'dynamic' uses the measured load,"
+        . " 'static' the configured guest resources and node capacity, and 'auto' uses 'dynamic'"
+        . " if the measured load of every node is available, else 'static'.",
+};
+
+my $mode_return_property = {
+    type => 'string',
+    enum => ['dynamic', 'static'],
+    description => "The usage information actually used.",
+};
+
+__PACKAGE__->register_method({
+    name => 'index',
+    path => '',
+    method => 'GET',
+    description => "Resource scheduling index.",
+    permissions => { user => 'all' },
+    parameters => {
+        additionalProperties => 0,
+        properties => {},
+    },
+    returns => {
+        type => 'array',
+        items => {
+            type => 'object',
+            properties => {},
+        },
+        links => [{ rel => 'child', href => "{name}" }],
+    },
+    code => sub($param) {
+        return [
+            { name => 'rebalance-plan' }, { name => 'recommend-target-nodes' },
+        ];
+    },
+});
+
+__PACKAGE__->register_method({
+    name => 'recommend_target_nodes',
+    path => 'recommend-target-nodes',
+    method => 'GET',
+    description => "Rank the cluster nodes as migration targets for a guest, best first. Only"
+        . " the load is considered; whether a migration to a node is possible, for example with"
+        . " regard to local resources or HA rules, must be checked separately.",
+    permissions => {
+        check => ['perm', '/vms/{vmid}', ['VM.Migrate']],
+    },
+    parameters => {
+        additionalProperties => 0,
+        properties => {
+            vmid => get_standard_option('pve-vmid'),
+            mode => $mode_property,
+            'target-nodes' => get_standard_option(
+                'pve-node-list',
+                {
+                    optional => 1,
+                    description => "Restrict the ranking to these candidate nodes.",
+                },
+            ),
+        },
+    },
+    returns => {
+        type => 'object',
+        properties => {
+            mode => $mode_return_property,
+            nodes => {
+                type => 'array',
+                description => "The candidate nodes, best first. The guest's current node and"
+                    . " nodes that are offline, in HA maintenance mode or without usage"
+                    . " information are omitted.",
+                items => {
+                    type => 'object',
+                    properties => {
+                        node => get_standard_option('pve-node'),
+                        imbalance => {
+                            type => 'number',
+                            optional => 1,
+                            description => "The cluster load imbalance, from 0 to 1, that"
+                                . " migrating the guest to this node would result in. Only"
+                                . " present for a running guest.",
+                        },
+                    },
+                },
+            },
+        },
+    },
+    code => sub($param) {
+        my $sid = guest_sid($param->{vmid}, 'vmid');
+
+        my $candidate_nodes =
+            $param->{'target-nodes'}
+            ? [PVE::Tools::split_list($param->{'target-nodes'})]
+            : undef;
+
+        my $planner = PVE::ResourceScheduling::Planner->build(
+            mode => $param->{mode} // 'auto',
+            nodes => schedulable_nodes(),
+            resolvers => static_stats_resolvers(),
+        );
+
+        return {
+            mode => $planner->mode(),
+            nodes => $planner->rank_target_nodes($sid, $candidate_nodes),
+        };
+    },
+});
+
+__PACKAGE__->register_method({
+    name => 'rebalance_plan',
+    path => 'rebalance-plan',
+    method => 'POST',
+    description =>
+        "Propose guest migrations that reduce the cluster load imbalance. Only running"
+        . " guests are considered, and locked guests are not proposed. Guests managed by HA are"
+        . " only proposed if HA would migrate them, that is, if their HA resource should be"
+        . " started and currently is, and no other resource has to move along. They are only"
+        . " proposed for nodes that HA considers online and their HA rules allow, taking the"
+        . " other proposed migrations into account. Guests that are not proposed still count"
+        . " towards the load of their node. Migrations that would barely improve the balance"
+        . " are not proposed.",
+    permissions => {
+        description => "The 'VM.Migrate' permission is required on '/vms/<ID>' for each ID"
+            . " passed via 'vms'. Without 'vms', only guests with 'VM.Migrate' are proposed, and"
+            . " the user needs that privilege on at least one guest.",
+        user => 'all',
+    },
+    parameters => {
+        additionalProperties => 0,
+        properties => {
+            vms => {
+                type => 'string',
+                format => 'pve-vmid-list',
+                optional => 1,
+                description =>
+                    "Only propose migrations of these guests (default: all guests the"
+                    . " user may migrate).",
+            },
+            'target-nodes' => get_standard_option(
+                'pve-node-list',
+                {
+                    optional => 1,
+                    description => "Only propose migrations to these nodes, for example to fill"
+                        . " newly added nodes. Guests can still be moved away from any node.",
+                },
+            ),
+            mode => $mode_property,
+            method => {
+                type => 'string',
+                enum => ['bruteforce', 'topsis'],
+                default => 'bruteforce',
+                optional => 1,
+                description => "The method used to score the candidate migrations.",
+            },
+            'max-moves' => {
+                type => 'integer',
+                minimum => 1,
+                maximum => 50,
+                default => 10,
+                optional => 1,
+                description => "The maximum number of migrations to propose.",
+            },
+            'min-improvement' => {
+                type => 'number',
+                minimum => 0,
+                maximum => 1,
+                default => 0,
+                optional => 1,
+                description =>
+                    "Only propose a migration if it lowers the imbalance by more than"
+                    . " this value, on the same 0 to 1 scale as the imbalance.",
+            },
+        },
+    },
+    returns => {
+        type => 'object',
+        properties => {
+            mode => $mode_return_property,
+            imbalance => {
+                type => 'object',
+                description => "The cluster load imbalance, from 0 (balanced) to 1.",
+                properties => {
+                    before => {
+                        type => 'number',
+                        description => "The current imbalance.",
+                    },
+                    after => {
+                        type => 'number',
+                        description => "The expected imbalance after all proposed migrations.",
+                    },
+                },
+            },
+            moves => {
+                type => 'array',
+                description => "The proposed migrations, at most one per guest.",
+                items => {
+                    type => 'object',
+                    properties => {
+                        vmid => get_standard_option('pve-vmid'),
+                        type => {
+                            type => 'string',
+                            enum => ['qemu', 'lxc'],
+                            description => "The guest type.",
+                        },
+                        source => get_standard_option(
+                            'pve-node',
+                            { description => "The node the guest currently runs on." },
+                        ),
+                        target => get_standard_option(
+                            'pve-node',
+                            { description => "The proposed target node." },
+                        ),
+                    },
+                },
+            },
+        },
+    },
+    code => sub($param) {
+        my $rpcenv = PVE::RPCEnvironment::get();
+        my $authuser = $rpcenv->get_user();
+
+        my $ids = PVE::Cluster::get_vmlist()->{ids} // {};
+
+        my @sids;
+        # an empty list is treated like an omitted one, so it cannot bypass the permission check
+        if (my @vmids = PVE::Tools::split_list($param->{vms} // '')) {
+            for my $vmid (@vmids) {
+                $rpcenv->check($authuser, "/vms/$vmid", ['VM.Migrate']);
+                push @sids, guest_sid($vmid, 'vms');
+            }
+        } else {
+            for my $vmid (sort keys %$ids) {
+                my $type = $guest_type_map->{ $ids->{$vmid}->{type} // '' } or next;
+                next if !$rpcenv->check($authuser, "/vms/$vmid", ['VM.Migrate'], 1);
+                push @sids, "$type:$vmid";
+            }
+            raise_perm_exc("VM.Migrate on at least one guest") if !@sids;
+        }
+
+        my $ha_status = PVE::HA::Config::read_manager_status();
+        my $nodes = schedulable_nodes($ha_status);
+
+        # a locked guest cannot be migrated right now, but still counts as load on its node
+        my $locks = PVE::Cluster::get_guest_config_properties(['lock']);
+        my $ha_states = PVE::HA::Config::get_resources_requested_state(\@sids, $ha_status);
+        my ($movable, $ha_sids) = ([], []);
+        for my $sid (@sids) {
+            my (undef, $vmid) = split(/:/, $sid, 2);
+            next if defined($locks->{$vmid}->{lock});
+            if (my $state = $ha_states->{$sid}) {
+                # HA stops a service configured to be stopped instead of migrating it, even if its
+                # current state is still started
+                push $ha_sids->@*, $sid if $state eq 'started';
+            } else {
+                push $movable->@*, $sid;
+            }
+        }
+
+        # HA only migrates a started service and none while HA is disarmed, only to a node it sees
+        # online, refuses a migration that its rules block and moves resources in positive affinity
+        # along. So propose an HA guest only if it is started and moves alone, and only to the
+        # online nodes its rules allow.
+        my $ha_nodes = $ha_status->{node_status} // {};
+        my $ha_services = $ha_status->{service_status} // {};
+        my $ha_started =
+            defined($ha_status->{disarm})
+            ? []
+            : [grep { ($ha_services->{$_}->{state} // '') eq 'started' } $ha_sids->@*];
+
+        my $snapshot_sids = {};
+        for my $vmid (keys %$ids) {
+            my $type = $guest_type_map->{ $ids->{$vmid}->{type} // '' } or next;
+            $snapshot_sids->{"$type:$vmid"} = 1;
+        }
+
+        my ($allowed_targets, $partners) = ({}, {});
+        if (scalar($ha_started->@*)) {
+            my $motion_info = PVE::HA::Config::get_resources_motion_info($ha_started);
+            for my $sid ($ha_started->@*) {
+                my $info = $motion_info->{$sid} // {};
+                next if scalar(($info->{'dependent-resources'} // [])->@*);
+
+                my $blocking = $info->{'blocking-resources-by-node'} // {};
+                $allowed_targets->{$sid} =
+                    [grep { ($ha_nodes->{$_} // '') eq 'online' && !$blocking->{$_} } $nodes->@*];
+                push $movable->@*, $sid;
+
+                # the blockers only reflect the current placement, so also keep guests in negative
+                # resource affinity apart from each other while the proposal moves them
+                for my $blocker (map { $_->@* } values %$blocking) {
+                    my $partner = $blocker->{sid};
+                    next if $blocker->{cause} ne 'resource-affinity';
+                    next if !$snapshot_sids->{$partner};
+                    $partners->{$sid}->{$partner} = 1;
+                    $partners->{$partner}->{$sid} = 1;
+                }
+            }
+        }
+        my $separate = { map { $_ => [sort keys $partners->{$_}->%*] } keys %$partners };
+
+        my $planner = PVE::ResourceScheduling::Planner->build(
+            mode => $param->{mode} // 'auto',
+            nodes => $nodes,
+            resolvers => static_stats_resolvers(),
+        );
+
+        my $plan = $planner->suggest_rebalance(
+            $movable,
+            method => $param->{method} // 'bruteforce',
+            max_moves => $param->{'max-moves'} // 10,
+            min_improvement => $param->{'min-improvement'} // 0,
+            targets => $param->{'target-nodes'}
+            ? [PVE::Tools::split_list($param->{'target-nodes'})]
+            : undef,
+            allowed_targets => $allowed_targets,
+            separate => $separate,
+        );
+
+        my $moves = [];
+        for my $move ($plan->{moves}->@*) {
+            my ($prefix, $vmid) = split(/:/, $move->{sid}, 2);
+            push $moves->@*,
+                {
+                    vmid => int($vmid),
+                    type => $guest_type_by_prefix->{$prefix},
+                    source => $move->{source},
+                    target => $move->{target},
+                };
+        }
+
+        return {
+            mode => $plan->{mode},
+            imbalance => $plan->{imbalance},
+            moves => $moves,
+        };
+    },
+});
+
+1;
diff --git a/test/Makefile b/test/Makefile
index 91761579d..5073c1bfc 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -5,7 +5,7 @@ all:
 export PERLLIB=..
 
 .PHONY: check
-check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-pvesh test-report
+check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-resource-scheduling test-pvesh test-report
 
 .PHONY: test-balloon
 test-balloon:
@@ -57,6 +57,10 @@ test-ceph-lockbox-migration:
 test-custom-cpu-models:
 	./CustomCPUModels_test.pl
 
+.PHONY: test-resource-scheduling
+test-resource-scheduling:
+	./ResourceScheduling_test.pl
+
 .PHONY: test-pvesh
 test-pvesh:
 	./pvesh_test.pl
diff --git a/test/ResourceScheduling_test.pl b/test/ResourceScheduling_test.pl
new file mode 100755
index 000000000..ecc1f8b25
--- /dev/null
+++ b/test/ResourceScheduling_test.pl
@@ -0,0 +1,454 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib ('.', '..');
+
+use JSON qw(encode_json);
+use Test::MockModule;
+use Test::More;
+
+use PVE::JSONSchema;
+use PVE::RPCEnvironment;
+
+use PVE::API2::Cluster::ResourceScheduling;
+
+my $GiB = 1024 * 1024 * 1024;
+
+# The cluster the mocked PVE::Cluster and HA getters below report, set up by new_cluster(). The
+# endpoints only read cluster state, so mocking these getters keeps the tests independent of a
+# running pmxcfs.
+my $cluster;
+
+# Per user, the ACL paths with 'VM.Migrate', the only privilege the endpoints check.
+my $migrate_acl = {};
+
+my $cluster_mock = Test::MockModule->new('PVE::Cluster');
+$cluster_mock->redefine(
+    get_members => sub() { return $cluster->{members} },
+    get_vmlist => sub() { return { ids => $cluster->{vmlist} } },
+    rrd_dump => sub() { return $cluster->{rrd} },
+    # callers decode the returned values in place, so hand out a copy
+    get_node_kv => sub($key, $node = undef) { return { ($cluster->{kv}->{$key} // {})->%* } },
+    get_guest_config_properties => sub($properties) {
+        my $res = {};
+        for my $vmid (keys $cluster->{confs}->%*) {
+            for my $prop (@$properties) {
+                my $value = $cluster->{confs}->{$vmid}->{$prop} // next;
+                $res->{$vmid}->{$prop} = $value;
+            }
+        }
+        return $res;
+    },
+);
+
+my $inotify_mock = Test::MockModule->new('PVE::INotify');
+$inotify_mock->redefine(nodename => sub() { return 'node1' });
+
+my $ha_config_mock = Test::MockModule->new('PVE::HA::Config');
+$ha_config_mock->redefine(read_manager_status => sub() { return $cluster->{ha_status} });
+# mock() instead of redefine(), as the batch helpers are newer than the pve-ha-manager a build may
+# have installed
+$ha_config_mock->mock(
+    # $cluster->{ha_managed} holds the configured HA state of each HA-managed guest
+    get_resources_requested_state => sub($sids, $manager_status = undef) {
+        my $res = {};
+        for my $sid (@$sids) {
+            my (undef, $vmid) = split(/:/, $sid, 2);
+            my $state = $cluster->{ha_managed}->{$vmid} // 'ignored';
+            $res->{$sid} = $state if $state ne 'ignored';
+        }
+        return $res;
+    },
+    get_resources_motion_info => sub($sids) {
+        push $cluster->{ha_motion_calls}->@*, [sort @$sids];
+        my $res = {};
+        for my $sid (@$sids) {
+            my (undef, $vmid) = split(/:/, $sid, 2);
+            next if !$cluster->{ha_managed}->{$vmid};
+            $res->{$sid} = $cluster->{ha_motion}->{$sid}
+                // { 'dependent-resources' => [], 'blocking-resources-by-node' => {} };
+        }
+        return $res;
+    },
+);
+
+my $rpcenv_mock = Test::MockModule->new('PVE::RPCEnvironment');
+$rpcenv_mock->redefine(
+    permissions => sub($self, $user, $path) {
+        return { 'VM.Migrate' => 1 }
+            if $user eq 'root@pam' || grep { $_ eq $path } ($migrate_acl->{$user} // [])->@*;
+        return {};
+    },
+);
+
+my $rpcenv = PVE::RPCEnvironment->init('cli');
+
+# Three online, idle nodes with 16 cores and 64 GiB of memory each, and no guests.
+sub new_cluster() {
+    $cluster = {
+        members => {},
+        vmlist => {},
+        rrd => {},
+        kv => { 'static-info' => {} },
+        confs => {},
+        ha_managed => {},
+        ha_motion => {},
+        ha_motion_calls => [],
+        ha_status => { node_status => {}, service_status => {} },
+    };
+
+    for my $node (qw(node1 node2 node3)) {
+        $cluster->{members}->{$node} = { online => 1 };
+        $cluster->{ha_status}->{node_status}->{$node} = 'online';
+        # RRD layout as broadcast by pvestatd: maxcpu, cpu, maxmem and mem at index 4, 5, 7 and 8
+        $cluster->{rrd}->{"pve-node-9.0/$node"} = [(undef) x 4, 16, 0, undef, 64 * $GiB, 0];
+        $cluster->{kv}->{'static-info'}->{$node} = encode_json({ cpus => 16, memory => 64 * $GiB });
+    }
+}
+
+# Add a guest with $cores cores and $mem_gib GiB of memory, by default a running VM on node1 that
+# uses 80% of its CPUs and 90% of its memory.
+sub add_guest($vmid, %opts) {
+    my $type = $opts{type} // 'qemu';
+    my $node = $opts{node} // 'node1';
+    my $running = $opts{running} // 1;
+    my ($cores, $mem) = ($opts{cores} // 2, ($opts{mem_gib} // 4) * $GiB);
+
+    $cluster->{vmlist}->{$vmid} = { type => $type, node => $node, version => 1 };
+
+    # RRD layout as broadcast by pvestatd: status, maxcpu, cpu, maxmem, mem at index 2, 5, 6, 7, 8
+    # and, for VMs, the host memory usage at index 15
+    my $used_mem = $running ? int(0.9 * $mem) : 0;
+    my $entry = [undef, undef, $running ? 'running' : 'stopped'];
+    $entry->@[5 .. 8] = ($cores, $running ? 0.8 : 0, $mem, $used_mem);
+    $entry->[15] = $used_mem if $type eq 'qemu';
+    $cluster->{rrd}->{"pve-vm-9.0/$vmid"} = $entry;
+
+    $cluster->{confs}->{$vmid} = { cores => $cores, memory => $mem / 1024 / 1024 };
+    $cluster->{confs}->{$vmid}->{lock} = $opts{lock} if defined($opts{lock});
+    if ($opts{ha}) {
+        $cluster->{ha_managed}->{$vmid} = 'started';
+        my $sid = ($type eq 'lxc' ? 'ct' : 'vm') . ":$vmid";
+        $cluster->{ha_status}->{service_status}->{$sid} = { state => 'started', node => $node };
+    }
+}
+
+# Let the HA rules of the HA-managed VM $vmid block migrations to the nodes in $opts{blocked}, keep
+# it apart from the VMs in $opts{apart} and move the resources in $opts{dependent} along with it.
+sub set_ha_motion($vmid, %opts) {
+    my $blocking = {};
+    for my $node (($opts{blocked} // [])->@*) {
+        push $blocking->{$node}->@*, { sid => "vm:$vmid", cause => 'node-affinity' };
+    }
+    # like HA, report a resource in negative affinity as blocking the node it currently runs on
+    for my $partner (($opts{apart} // [])->@*) {
+        my $node = $cluster->{vmlist}->{$partner}->{node};
+        push $blocking->{$node}->@*, { sid => "vm:$partner", cause => 'resource-affinity' };
+    }
+    $cluster->{ha_motion}->{"vm:$vmid"} = {
+        'dependent-resources' => $opts{dependent} // [],
+        'blocking-resources-by-node' => $blocking,
+    };
+}
+
+# Call an endpoint the way the API server does: validate the parameters, run the method's declared
+# permission check, then the handler, and check the result against the declared return schema.
+my $api_method = { 'recommend-target-nodes' => 'GET', 'rebalance-plan' => 'POST' };
+
+sub api_call($path, $user, $param = {}) {
+    my ($handler, $info) =
+        PVE::API2::Cluster::ResourceScheduling->find_handler($api_method->{$path}, $path, {});
+    die "no handler for '$path'\n" if !$info;
+
+    my $call_param = {%$param};
+    PVE::JSONSchema::validate($call_param, $info->{parameters});
+
+    $rpcenv->set_user($user);
+    $rpcenv->check_api2_permissions($info->{permissions}, $user, $call_param);
+
+    my $res = $handler->handle($info, $call_param);
+    PVE::JSONSchema::validate($res, $info->{returns}, "result does not match the return schema\n");
+
+    return $res;
+}
+
+# The exception an endpoint call raises, or undef if it succeeds.
+sub api_error($path, $user, $param = {}) {
+    eval { api_call($path, $user, $param) };
+    return $@ || undef;
+}
+
+# The ranked nodes, sorted by name: the tests check which nodes are candidates, not their order.
+sub candidate_nodes($ranking) {
+    return [sort map { $_->{node} } $ranking->{nodes}->@*];
+}
+
+sub moved_vmids($plan) {
+    return [sort { $a <=> $b } map { $_->{vmid} } $plan->{moves}->@*];
+}
+
+sub move_target($plan, $vmid) {
+    my ($move) = grep { $_->{vmid} == $vmid } $plan->{moves}->@*;
+    return $move ? $move->{target} : undef;
+}
+
+sub move_targets($plan) {
+    my $targets = { map { $_->{target} => 1 } $plan->{moves}->@* };
+    return [sort keys %$targets];
+}
+
+subtest 'recommend-target-nodes: permissions' => sub {
+    new_cluster();
+    add_guest($_) for 100, 101;
+    $migrate_acl = { 'op@pve' => ['/vms/101'] };
+
+    my $err = api_error('recommend-target-nodes', 'op@pve', { vmid => 100 });
+    is($err->{code}, 403, "guest without 'VM.Migrate' is rejected");
+
+    ok(api_call('recommend-target-nodes', 'op@pve', { vmid => 101 }),
+        "guest with 'VM.Migrate'");
+
+    $err = api_error('recommend-target-nodes', 'root@pam', { vmid => 999 });
+    is($err->{code}, 400, 'unknown guest is a parameter error');
+    like($err->{errors}->{vmid}, qr/does not exist/, 'error names the unknown guest');
+};
+
+subtest 'recommend-target-nodes: candidate nodes' => sub {
+    new_cluster();
+    add_guest(100);
+    add_guest(101, node => 'node2', running => 0);
+
+    my $ranking = api_call('recommend-target-nodes', 'root@pam', { vmid => 100 });
+    is($ranking->{mode}, 'dynamic', 'measured load is used if every node reports it');
+    is_deeply(candidate_nodes($ranking), [qw(node2 node3)], 'current node is excluded');
+    ok(!grep({ !defined($_->{imbalance}) } $ranking->{nodes}->@*), 'running guest: imbalance');
+
+    # a stopped guest is ranked as a placement, which has no resulting imbalance to report
+    $ranking = api_call('recommend-target-nodes', 'root@pam', { vmid => 101 });
+    is_deeply(candidate_nodes($ranking), [qw(node1 node3)], 'stopped guest: current excluded');
+    ok(!grep({ defined($_->{imbalance}) } $ranking->{nodes}->@*), 'stopped guest: none');
+
+    my $param = { vmid => 100, 'target-nodes' => 'node3' };
+    $ranking = api_call('recommend-target-nodes', 'root@pam', $param);
+    is_deeply(candidate_nodes($ranking), ['node3'], "ranking is limited to 'target-nodes'");
+
+    $cluster->{members}->{node2}->{online} = 0;
+    $ranking = api_call('recommend-target-nodes', 'root@pam', { vmid => 100 });
+    is_deeply(candidate_nodes($ranking), ['node3'], 'offline node is excluded');
+
+    # a node in maintenance is about to be drained, yet it would rank first as the least loaded one
+    $cluster->{members}->{node2}->{online} = 1;
+    $cluster->{ha_status}->{node_status}->{node3} = 'maintenance';
+    $ranking = api_call('recommend-target-nodes', 'root@pam', { vmid => 100 });
+    is_deeply(candidate_nodes($ranking), ['node2'], 'node in HA maintenance mode is excluded');
+
+    # a node without any usage information must not fail the request, it is left out instead
+    $cluster->{ha_status}->{node_status} = {};
+    delete $cluster->{rrd}->{'pve-node-9.0/node3'};
+    delete $cluster->{kv}->{'static-info'}->{node3};
+    $ranking = api_call('recommend-target-nodes', 'root@pam', { vmid => 100 });
+    is($ranking->{mode}, 'static', 'missing load of a node falls back to static usage');
+    is_deeply(candidate_nodes($ranking), ['node2'], 'node without usage data is excluded');
+};
+
+subtest 'rebalance-plan: permissions' => sub {
+    new_cluster();
+    add_guest($_) for 100 .. 103;
+    $migrate_acl = { 'op@pve' => ['/vms/101'] };
+
+    # an empty or separator-only list must behave like an omitted one, so it cannot bypass the check
+    my $no_vms = { omitted => {}, empty => { vms => '' }, 'separator-only' => { vms => ',;' } };
+
+    # without any migratable guest the user could only learn the cluster load, so refuse
+    for my $case (sort keys %$no_vms) {
+        my $err = api_error('rebalance-plan', 'nobody@pve', $no_vms->{$case});
+        is($err->{code}, 403, "no privileges, 'vms' $case: rejected");
+    }
+
+    for my $case (sort keys %$no_vms) {
+        my $plan = api_call('rebalance-plan', 'op@pve', $no_vms->{$case});
+        is_deeply(moved_vmids($plan), [101], "'vms' $case: only the permitted guest moves");
+    }
+
+    my $err = api_error('rebalance-plan', 'op@pve', { vms => '101,102' });
+    is($err->{code}, 403, "explicit guest without 'VM.Migrate' is rejected");
+
+    # the permission check comes first, so the error does not reveal whether a guest exists
+    $err = api_error('rebalance-plan', 'op@pve', { vms => '101,999' });
+    is($err->{code}, 403, 'unknown guest without privileges is a permission error');
+
+    $err = api_error('rebalance-plan', 'root@pam', { vms => '999' });
+    is($err->{code}, 400, 'unknown guest is a parameter error');
+    like($err->{errors}->{vms}, qr/does not exist/, 'error names the unknown guest');
+};
+
+subtest 'rebalance-plan: locked guests stay put' => sub {
+    # the two big guests are the most effective ones to move off the overloaded node1
+    new_cluster();
+    add_guest(100, cores => 8, mem_gib => 16);
+    add_guest(101, cores => 8, mem_gib => 16);
+    add_guest(102);
+    add_guest(103, type => 'lxc');
+
+    my $unlocked = api_call('rebalance-plan', 'root@pam');
+    ok(defined(move_target($unlocked, 101)), 'without a lock, the big guest is proposed');
+
+    add_guest(101, cores => 8, mem_gib => 16, lock => 'backup');
+
+    my $plan = api_call('rebalance-plan', 'root@pam');
+    ok(!defined(move_target($plan, 101)), 'locked guest is not moved');
+    is(
+        $plan->{imbalance}->{before},
+        $unlocked->{imbalance}->{before},
+        'locked guest still counts as load on its node',
+    );
+};
+
+subtest 'rebalance-plan: HA-managed guests follow their HA rules' => sub {
+    # as above, the big guests 100 and 101 are the most effective ones to move
+    new_cluster();
+    add_guest(100, cores => 8, mem_gib => 16, ha => 1);
+    add_guest(101, cores => 8, mem_gib => 16);
+    add_guest(102);
+    add_guest(103, type => 'lxc');
+
+    my $free = api_call('rebalance-plan', 'root@pam');
+    ok(defined(move_target($free, 100)), 'HA-managed guest is proposed');
+    my $calls = $cluster->{ha_motion_calls};
+    is_deeply($calls, [['vm:100']], 'HA rules are queried once, only for HA-managed guests');
+
+    set_ha_motion(100, blocked => ['node2']);
+    my $plan = api_call('rebalance-plan', 'root@pam');
+    is(move_target($plan, 100), 'node3', 'HA-managed guest only goes to an allowed node');
+    is(move_target($plan, 101), 'node2', 'node blocked for HA is still a target for others');
+
+    set_ha_motion(100, blocked => ['node2', 'node3']);
+    $plan = api_call('rebalance-plan', 'root@pam');
+    ok(!defined(move_target($plan, 100)), 'HA-managed guest without an allowed node stays');
+    ok(defined(move_target($plan, 101)), 'other guests are still proposed');
+    is(
+        $plan->{imbalance}->{before},
+        $free->{imbalance}->{before},
+        'HA-managed guest that stays still counts as load on its node',
+    );
+
+    # HA would migrate the dependent resources along, which a per-guest proposal cannot express
+    set_ha_motion(100, dependent => ['vm:104']);
+    $plan = api_call('rebalance-plan', 'root@pam');
+    ok(!defined(move_target($plan, 100)), 'HA-managed guest with dependent resources stays');
+
+    # HA drops a migration to a node it does not see online, even if the cluster does
+    set_ha_motion(100, blocked => ['node2']);
+    $cluster->{ha_status}->{node_status}->{node3} = 'unknown';
+    $plan = api_call('rebalance-plan', 'root@pam');
+    ok(!defined(move_target($plan, 100)),
+        'HA-managed guest only goes to a node HA sees online');
+};
+
+subtest 'rebalance-plan: HA-managed guests only if HA would migrate them' => sub {
+    # as above, the big guests 100 and 101 are the most effective ones to move
+    new_cluster();
+    add_guest(100, cores => 8, mem_gib => 16, ha => 1);
+    add_guest(101, cores => 8, mem_gib => 16);
+    add_guest(102);
+    add_guest(103, type => 'lxc');
+
+    my $started = api_call('rebalance-plan', 'root@pam');
+    ok(defined(move_target($started, 100)), 'started HA service is proposed');
+    my $load = $started->{imbalance}->{before};
+
+    # until HA processes a changed configuration, the current state can still read started, but HA
+    # then stops the guest instead of migrating it
+    for my $configured (qw(stopped disabled)) {
+        $cluster->{ha_managed}->{100} = $configured;
+        my $plan = api_call('rebalance-plan', 'root@pam');
+        ok(!defined(move_target($plan, 100)), "HA service configured as $configured stays");
+        is($plan->{imbalance}->{before}, $load, 'and still counts as load');
+    }
+    $cluster->{ha_managed}->{100} = 'started';
+
+    # a service that is migrating, recovering or the like is busy with HA's own plans
+    $cluster->{ha_status}->{service_status}->{'vm:100'}->{state} = 'migrate';
+    my $plan = api_call('rebalance-plan', 'root@pam');
+    ok(!defined(move_target($plan, 100)), 'HA service in another state stays');
+    is($plan->{imbalance}->{before}, $load, 'and still counts as load');
+
+    # a frozen HA stack still owns its guests, but does not carry out any migration request
+    $cluster->{ha_status}->{service_status}->{'vm:100'}->{state} = 'freeze';
+    $cluster->{ha_status}->{disarm} = { mode => 'freeze', state => 'disarmed' };
+    $plan = api_call('rebalance-plan', 'root@pam');
+    ok(!defined(move_target($plan, 100)), 'HA-managed guest stays while HA is frozen');
+    ok(defined(move_target($plan, 101)), 'other guests are still proposed');
+    is($plan->{imbalance}->{before}, $load, 'and still counts as load');
+
+    # while disarming, the services may still be marked started for a moment
+    $cluster->{ha_status}->{service_status}->{'vm:100'}->{state} = 'started';
+    $cluster->{ha_status}->{disarm} = { mode => 'freeze', state => 'disarming' };
+    $plan = api_call('rebalance-plan', 'root@pam');
+    ok(!defined(move_target($plan, 100)), 'HA-managed guest stays while HA is disarming');
+};
+
+subtest 'rebalance-plan: negative resource affinity holds across the proposal' => sub {
+    # node1 and node2 are busy and node3 is idle, so it is the best target for 101 and 102 alike,
+    # which their negative resource affinity forbids
+    new_cluster();
+    $cluster->{rrd}->{"pve-node-9.0/$_"}->@[5, 8] = (0.875, 56 * $GiB) for qw(node1 node2);
+    add_guest(101, cores => 4, mem_gib => 16, ha => 1);
+    add_guest(102, cores => 4, mem_gib => 16, ha => 1, node => 'node2');
+    set_ha_motion(101, apart => [102]);
+    set_ha_motion(102, apart => [101]);
+
+    my $plan = api_call('rebalance-plan', 'root@pam');
+    my @targets = grep { defined } map { move_target($plan, $_) } 101, 102;
+    is_deeply(\@targets, ['node3'], 'only one of the separated guests moves to the idle node');
+
+    # knowing the affinity from one side suffices to keep the two apart
+    set_ha_motion(102);
+    $plan = api_call('rebalance-plan', 'root@pam');
+    @targets = grep { defined } map { move_target($plan, $_) } 101, 102;
+    is_deeply(\@targets, ['node3'], 'one-sided affinity also keeps them apart');
+};
+
+subtest 'rebalance-plan: proposal' => sub {
+    # the big container is the most effective guest to move, so the proposal includes a container
+    new_cluster();
+    add_guest($_) for 100 .. 102;
+    add_guest(103, type => 'lxc', cores => 8, mem_gib => 16);
+    add_guest(104, running => 0);
+
+    my $plan = api_call('rebalance-plan', 'root@pam');
+    ok(scalar($plan->{moves}->@*), 'overloaded node gets a proposal');
+    my $imbalance = $plan->{imbalance};
+    ok($imbalance->{after} < $imbalance->{before}, 'proposal lowers the imbalance');
+    ok(!grep({ $_ == 104 } moved_vmids($plan)->@*), 'stopped guest is not moved');
+
+    # the result carries plain guest ids and types, not the scheduler's internal resource ids
+    for my $move ($plan->{moves}->@*) {
+        my ($vmid, $guest) = ($move->{vmid}, $cluster->{vmlist}->{ $move->{vmid} });
+        is_deeply([sort keys %$move], [qw(source target type vmid)], "move of $vmid: keys");
+        is($move->{type}, $guest->{type}, "move of $vmid: type");
+        is($move->{source}, $guest->{node}, "move of $vmid: source");
+    }
+    ok((grep { $_->{type} eq 'lxc' } $plan->{moves}->@*), 'container is reported as lxc');
+
+    $plan = api_call('rebalance-plan', 'root@pam', { 'max-moves' => 1 });
+    is(scalar($plan->{moves}->@*), 1, "'max-moves' caps the proposal");
+
+    for my $max_moves (0, 51) {
+        my $err = api_error('rebalance-plan', 'root@pam', { 'max-moves' => $max_moves });
+        ok($err->{errors}->{'max-moves'}, "'max-moves' $max_moves is out of range");
+    }
+
+    # restricting the targets, e.g. to fill a new node, still moves guests off any other node
+    $plan = api_call('rebalance-plan', 'root@pam', { 'target-nodes' => 'node2' });
+    ok(scalar($plan->{moves}->@*), "proposal with 'target-nodes'");
+    is_deeply(move_targets($plan), ['node2'], "only 'target-nodes' are targets");
+
+    $cluster->{ha_status}->{node_status}->{node2} = 'maintenance';
+    $plan = api_call('rebalance-plan', 'root@pam');
+    ok(scalar($plan->{moves}->@*), 'proposal with a node in HA maintenance mode');
+    is_deeply(move_targets($plan), ['node3'], 'HA maintenance node is no target');
+};
+
+done_testing();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH manager 11/15] ui: migrate: sort target nodes by scheduler recommendation
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (9 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH manager 10/15] api: cluster: add resource scheduling endpoints Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH manager 12/15] api: bulk migrate: allow a per-guest target node mapping Thomas Lamprecht
                   ` (3 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

Order the target node list of the migrate dialog by the recommendation
of the cluster resource scheduler and preselect the best node, instead
of the alphabetically first one. For a running guest, also show the
cluster load imbalance that the migration to each node would result
in.

The scheduler only considers the load, so offline nodes and nodes that
the migration preconditions rule out, for example due to HA rules, are
sorted last and never preselected. The latter are marked as not allowed
instead of showing their imbalance.

The recommendation is loaded after the dialog opened and never
overrides a target node the user already looked at or picked. If it is
not available, the node list keeps its previous order.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 www/manager6/window/Migrate.js | 146 +++++++++++++++++++++++++++++++++
 1 file changed, 146 insertions(+)

diff --git a/www/manager6/window/Migrate.js b/www/manager6/window/Migrate.js
index 40bca4228..b27d679df 100644
--- a/www/manager6/window/Migrate.js
+++ b/www/manager6/window/Migrate.js
@@ -112,6 +112,112 @@ Ext.define('PVE.window.Migrate', {
                 onlineHelp: vm.get(view.vmtype).onlineHelp,
             });
             me.lookup('formPanel').isValid();
+
+            me.sortTargetsByRecommendation();
+        },
+
+        // Order the target nodes by the cluster resource scheduler's recommendation and preselect
+        // the best one. Best-effort: without a recommendation the default order stays.
+        sortTargetsByRecommendation: async function () {
+            let me = this;
+            let selector = me.lookup('pveNodeSelector');
+
+            let nodes;
+            try {
+                let { result } = await Proxmox.Async.api2({
+                    url: '/cluster/resource-scheduling/recommend-target-nodes',
+                    method: 'GET',
+                    params: { vmid: me.getViewModel().get('vmid') },
+                });
+                nodes = result.data.nodes ?? [];
+            } catch {
+                return;
+            }
+            if (me.destroyed || !nodes.length) {
+                return;
+            }
+
+            let store = selector.getStore();
+            // read by the imbalance column, which may render before or after the store loads
+            store.resultingImbalance = Object.fromEntries(
+                nodes.map((entry) => [entry.node, entry.imbalance]),
+            );
+            let picker = selector.getPicker();
+            let column = picker.down('#imbalance');
+            if (column?.isHidden() && nodes.some((entry) => entry.imbalance !== undefined)) {
+                column.setHidden(false);
+                picker.setWidth(picker.width + column.width);
+            }
+
+            me.recommendedNodes = nodes.map((entry) => entry.node);
+            me.applyRecommendation();
+        },
+
+        // The scheduler only considers the load, so sort the nodes that the precondition check rules
+        // out, for example due to HA rules, last. Missing storage only blocks an offline migration,
+        // as a running VM can migrate its disks.
+        noteBlockedTargets: function (notAllowedNodes, storageBlocks = false) {
+            let me = this;
+            let blocked = Object.entries(notAllowedNodes ?? {})
+                .filter(
+                    ([_node, reasons]) =>
+                        reasons['blocking-ha-resources']?.length ||
+                        reasons['unavailable-resources']?.length ||
+                        (storageBlocks && reasons.unavailable_storages?.length),
+                )
+                .map(([node]) => node)
+                .sort();
+            if (blocked.join() !== (me.blockedTargets ?? []).join()) {
+                me.blockedTargets = blocked;
+                me.applyRecommendation();
+            }
+        },
+
+        // Sort the target nodes by recommendation, with blocked and offline ones last, and make the
+        // best usable node the default, unless the user already chose a target.
+        applyRecommendation: function () {
+            let me = this;
+            let recommended = me.recommendedNodes;
+            if (me.destroyed || !recommended) {
+                return;
+            }
+            let selector = me.lookup('pveNodeSelector');
+            let store = selector.getStore();
+            let blocked = me.blockedTargets ?? [];
+            store.blockedTargets = blocked; // read by the imbalance column
+
+            let rankOf = (node) => {
+                let rank = recommended.indexOf(node);
+                return rank === -1 ? Number.POSITIVE_INFINITY : rank;
+            };
+            let unusable = (rec) =>
+                blocked.includes(rec.data.node) || rec.data.status !== 'online' ? 1 : 0;
+            // a single sorter without a property, so no column header claims to define the order
+            store.getSorters().clear();
+            store.sort(
+                new Ext.util.Sorter({
+                    sorterFn: (a, b) =>
+                        unusable(a) - unusable(b) ||
+                        rankOf(a.data.node) - rankOf(b.data.node) ||
+                        a.data.node.localeCompare(b.data.node),
+                }),
+            );
+
+            // While its store loads, the selector auto-selects the first node of the new order.
+            // Once loaded, it already picked the first node of the old order, so correct that.
+            if (me.targetTouched || !store.isLoaded()) {
+                return;
+            }
+            let allowed = selector.allowedNodes;
+            let best = recommended.find(
+                (node) =>
+                    store.findExact('node', node) !== -1 &&
+                    !blocked.includes(node) &&
+                    (!allowed || allowed.includes(node)),
+            );
+            if (best && best !== selector.getValue()) {
+                selector.setValue(best);
+            }
         },
 
         onTargetChange: function (nodeSelector) {
@@ -120,6 +226,11 @@ Ext.define('PVE.window.Migrate', {
             this.checkMigratePreconditions(true);
         },
 
+        // the recommendation arrives asynchronously and must not override the user's choice
+        onTargetTouched: function () {
+            this.targetTouched = true;
+        },
+
         startMigration: function () {
             var me = this,
                 view = me.getView(),
@@ -286,6 +397,7 @@ Ext.define('PVE.window.Migrate', {
             }
             migration.preconditions = [];
             let disallowed = migrateStats.not_allowed_nodes?.[target] ?? {};
+            me.noteBlockedTargets(migrateStats.not_allowed_nodes, !vm.get('running'));
 
             if (migrateStats.allowed_nodes && !vm.get('running')) {
                 migration.allowedNodes = migrateStats.allowed_nodes;
@@ -534,6 +646,7 @@ Ext.define('PVE.window.Migrate', {
             migration.preconditions = [];
             let targetNode = me.lookup('pveNodeSelector').value;
             let disallowed = migrateStats['not-allowed-nodes']?.[targetNode] ?? {};
+            me.noteBlockedTargets(migrateStats['not-allowed-nodes']);
 
             let blockingHAResources = disallowed['blocking-ha-resources'] ?? [];
             if (blockingHAResources.length) {
@@ -637,8 +750,41 @@ Ext.define('PVE.window.Migrate', {
                             allowBlank: false,
                             disallowedNodes: undefined,
                             onlineValidator: true,
+                            // replaces the class default, so extend its columns
+                            listConfig: {
+                                columns: [
+                                    ...PVE.form.NodeSelector.prototype.listConfig.columns.map(
+                                        (column) => ({ ...column }),
+                                    ),
+                                    {
+                                        header: gettext('Resulting imbalance'),
+                                        itemId: 'imbalance',
+                                        dataIndex: 'node',
+                                        hidden: true, // shown once a recommendation provides it
+                                        sortable: false,
+                                        width: 140,
+                                        tooltip: gettext(
+                                            'Estimated cluster load imbalance after migrating the guest to this node (lower is better)',
+                                        ),
+                                        renderer: function (node, meta, record) {
+                                            let store = record.store;
+                                            // a better score of a blocked node would mislead
+                                            if (store?.blockedTargets?.includes(node)) {
+                                                return gettext('Not allowed');
+                                            }
+                                            let value = store?.resultingImbalance?.[node];
+                                            if (value === undefined) {
+                                                return '';
+                                            }
+                                            return `${(value * 100).toFixed(1)}%`;
+                                        },
+                                    },
+                                ],
+                            },
                             listeners: {
                                 change: 'onTargetChange',
+                                expand: 'onTargetTouched',
+                                select: 'onTargetTouched',
                             },
                         },
                         {
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH manager 12/15] api: bulk migrate: allow a per-guest target node mapping
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (10 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH manager 11/15] ui: migrate: sort target nodes by scheduler recommendation Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH manager 13/15] ui: bulk migrate: support per-guest target nodes Thomas Lamprecht
                   ` (2 subsequent siblings)
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

The bulk migration can only send all guests to the same node, which
does not fit spreading guests over the cluster, for example to apply a
rebalance proposal. Add a 'target-map' parameter with per-guest target
nodes that take precedence over 'target'. Without 'vms' and 'target',
only the mapped guests are migrated.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 PVE/API2/Cluster/BulkAction/Guest.pm | 100 ++++++++++++-
 test/BulkMigrate_test.pl             | 204 +++++++++++++++++++++++++++
 test/Makefile                        |   6 +-
 3 files changed, 303 insertions(+), 7 deletions(-)
 create mode 100755 test/BulkMigrate_test.pl

diff --git a/PVE/API2/Cluster/BulkAction/Guest.pm b/PVE/API2/Cluster/BulkAction/Guest.pm
index a55ea2ea6..ae6186324 100644
--- a/PVE/API2/Cluster/BulkAction/Guest.pm
+++ b/PVE/API2/Cluster/BulkAction/Guest.pm
@@ -11,12 +11,52 @@ use PVE::JSONSchema qw(get_standard_option);
 use PVE::RESTEnvironment qw(log_warn);
 use PVE::RPCEnvironment;
 use PVE::Storage;
+use PVE::Tools;
 use PVE::UPID;
 
 use PVE::API2::Nodes;
 
 use base qw(PVE::RESTHandler);
 
+PVE::JSONSchema::register_format('pve-vmid-node-pair', \&pve_verify_vmid_node_pair);
+
+sub pve_verify_vmid_node_pair {
+    my ($pair, $noerr) = @_;
+
+    if ($pair =~ m/^(\d+):(.+)$/) {
+        my ($vmid, $node) = ($1, $2);
+        return $pair
+            if eval {
+                PVE::JSONSchema::check_format('pve-vmid', $vmid);
+                PVE::JSONSchema::check_format('pve-node', $node);
+                1;
+            };
+    }
+
+    return undef if $noerr;
+    die "value '$pair' is not a 'vmid:node' pair\n";
+}
+
+# Parse a 'pve-vmid-node-pair-list' into a { vmid => node } hash.
+sub parse_target_map {
+    my ($target_map) = @_;
+
+    my $map = {};
+    return $map if !defined($target_map);
+
+    for my $entry (PVE::Tools::split_list($target_map)) {
+        my ($vmid, $node) = $entry =~ m/^(\d+):(.+)$/
+            or raise_param_exc({ 'target-map' => "value '$entry' is not a 'vmid:node' pair" });
+        raise_param_exc({ 'target-map' => "duplicate entry for guest '$vmid'" })
+            if defined($map->{$vmid});
+        raise_param_exc({ 'target-map' => "no such cluster node '$node'" })
+            if !PVE::Cluster::check_node_exists($node, 1);
+        $map->{$vmid} = $node;
+    }
+
+    return $map;
+}
+
 __PACKAGE__->register_method({
     name => 'index',
     path => '',
@@ -711,7 +751,23 @@ __PACKAGE__->register_method({
                 items => get_standard_option('pve-vmid'),
                 optional => 1,
             },
-            target => get_standard_option('pve-node', { description => "Target node." }),
+            target => get_standard_option(
+                'pve-node',
+                {
+                    description =>
+                        "Default target node for guests without an entry in 'target-map'.",
+                    optional => 1,
+                },
+            ),
+            'target-map' => {
+                description =>
+                    "Per-guest target nodes, taking precedence over 'target'. If neither"
+                    . " 'vms' nor 'target' is given, only the listed guests are migrated.",
+                type => 'string',
+                format => 'pve-vmid-node-pair-list',
+                typetext => '<vmid>:<node>[,<vmid>:<node>...]',
+                optional => 1,
+            },
             online => {
                 type => 'boolean',
                 description => "Enable live migration for VMs and restart migration for CTs.",
@@ -740,7 +796,6 @@ __PACKAGE__->register_method({
                 maximum => 64,
             },
             # TODO:
-            # Node mappings, not just a single target node!
             # Failure resolution mode (fail, warn, retry?)
             # mode-limits (offline only, suspend only, ?)
             # filter (tags, name, ?)
@@ -756,15 +811,46 @@ __PACKAGE__->register_method({
         my $rpcenv = PVE::RPCEnvironment::get();
         my $authuser = $rpcenv->get_user();
 
+        my $target_map = parse_target_map($param->{'target-map'});
+        my $default_target = $param->{target};
+
+        raise_param_exc({ target => "neither 'target' nor 'target-map' given" })
+            if !defined($default_target) && !%$target_map;
+
+        # without a default target only the mapped guests can be migrated, so limit the action to
+        # them rather than walking, and skipping, every other guest in the cluster
+        $param->{vms} = [sort { $a <=> $b } keys %$target_map]
+            if !defined($default_target) && !($param->{vms} && $param->{vms}->@*);
+
         my ($vmlist, $vmlist_string) = extract_vmlist($param);
 
         check_guest_permissions($rpcenv, $authuser, $vmlist, ['VM.Migrate']);
 
+        my %in_scope = map { $_ => 1 } $vmlist->@*;
+        if (%in_scope) {
+            for my $vmid (sort { $a <=> $b } keys %$target_map) {
+                raise_param_exc({ 'target-map' => "guest '$vmid' is not in the 'vms' list" })
+                    if !$in_scope{$vmid};
+            }
+        }
+
+        if (!defined($default_target)) {
+            my @missing = grep { !defined($target_map->{$_}) } $vmlist->@*;
+            raise_param_exc({
+                'target-map' => "no target node for guest(s) " . join(', ', @missing),
+            })
+                if @missing;
+        }
+
         my $code = sub {
             my $list =
                 PVE::API2::Nodes::Nodeinfo::get_filtered_vmlist(undef, $vmlist_string, 1, 1);
 
-            print_start_action($vmlist, "Migrating", "to $param->{target}");
+            print_start_action(
+                $vmlist,
+                "Migrating",
+                %$target_map ? "to their target nodes" : "to $default_target",
+            );
 
             my $start_task = sub {
                 my ($api_client, $vmid, $guest) = @_;
@@ -773,13 +859,15 @@ __PACKAGE__->register_method({
                 my $type = $guest->{type};
                 my $type_text = get_type_text($type);
 
-                if ($node eq $param->{target}) {
+                my $target = $target_map->{$vmid} // $default_target;
+
+                if ($node eq $target) {
                     log_warn("$type_text $vmid already on $node, skipping.\n");
                     return 1;
                 }
 
                 my $params = {
-                    target => $param->{target},
+                    target => $target,
                 };
 
                 if ($type eq 'lxc') {
@@ -791,7 +879,7 @@ __PACKAGE__->register_method({
                 }
 
                 my $url = "/nodes/$node/$type/$vmid/migrate";
-                print "Migrating $type_text $vmid\n";
+                print "Migrating $type_text $vmid to $target\n";
                 return $api_client->post($url, $params);
             };
 
diff --git a/test/BulkMigrate_test.pl b/test/BulkMigrate_test.pl
new file mode 100755
index 000000000..bee292f9a
--- /dev/null
+++ b/test/BulkMigrate_test.pl
@@ -0,0 +1,204 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib ('.', '..');
+
+use Test::MockModule;
+use Test::More;
+
+use PVE::JSONSchema;
+use PVE::RPCEnvironment;
+use PVE::Tools;
+
+use PVE::API2::Cluster::BulkAction::Guest;
+
+my $nodes = { map { $_ => 1 } qw(node1 node2 node3) };
+
+my $guests = {
+    100 => { type => 'qemu', node => 'node1' },
+    101 => { type => 'qemu', node => 'node1' },
+    102 => { type => 'lxc', node => 'node2' },
+    103 => { type => 'qemu', node => 'node3' },
+};
+
+# Per user, the ACL paths on which the user has 'VM.Migrate', the only privilege the bulk migration
+# checks itself; the per-guest migrate API calls check their own permissions.
+my $migrate_acl = {};
+
+# What the bulk migration worker did in the current test, see bulk_migrate().
+my $worker;
+
+my $cluster_mock = Test::MockModule->new('PVE::Cluster');
+$cluster_mock->redefine(
+    check_node_exists => sub($node, $noerr = undef) {
+        return 1 if $nodes->{$node};
+        return undef if $noerr;
+        die "no such cluster node '$node'\n";
+    },
+);
+
+my $nodeinfo_mock = Test::MockModule->new('PVE::API2::Nodes::Nodeinfo', no_auto => 1);
+$nodeinfo_mock->redefine(
+    get_filtered_vmlist => sub($nodename, $vmfilter, $templates, $ha_managed) {
+        $worker->{filter} = $vmfilter;
+        my @vmids = defined($vmfilter) ? PVE::Tools::split_list($vmfilter) : keys %$guests;
+        return { map { $_ => { $guests->{$_}->%* } } grep { $guests->{$_} } @vmids };
+    },
+);
+
+# Run the per-guest step directly instead of starting and polling tasks through the API client, and
+# record each migration it starts.
+my $guest_mock = Test::MockModule->new('PVE::API2::Cluster::BulkAction::Guest');
+$guest_mock->redefine(
+    get_max_workers => sub($param, $default = undef) { return 1 },
+    log_warn => sub($message) { push $worker->{warnings}->@*, $message },
+    handle_task_foreach_guest => sub($startlist, $max_workers, $start_task, $check_task) {
+        my $client = bless {}, 'FakeAPIClient';
+        for my $order (sort { $a <=> $b } keys %$startlist) {
+            my $list = $startlist->{$order};
+            $start_task->($client, $_, $list->{$_}) for sort { $a <=> $b } keys %$list;
+        }
+        return [];
+    },
+);
+
+package FakeAPIClient {
+
+    sub post($self, $url, $param) {
+        my ($vmid) = $url =~ m!^/nodes/[^/]+/(?:qemu|lxc)/(\d+)/migrate$!
+            or die "unexpected API call '$url'\n";
+        $worker->{migrations}->{$vmid} = $param->{target};
+        return "UPID:migrate:$vmid";
+    }
+}
+
+my $rpcenv_mock = Test::MockModule->new('PVE::RPCEnvironment');
+$rpcenv_mock->redefine(
+    permissions => sub($self, $user, $path) {
+        return { 'VM.Migrate' => 1 }
+            if $user eq 'root@pam' || grep { $_ eq $path } ($migrate_acl->{$user} // [])->@*;
+        return {};
+    },
+    fork_worker => sub($self, $type, $id, $user, $code) {
+        $worker->{code} = $code;
+        return "UPID:$type";
+    },
+);
+
+my $rpcenv = PVE::RPCEnvironment->init('cli');
+
+# Call the bulk migration the way the API server does and run the worker it forks. Returns what the
+# worker did: the guest filter it applied, the started migrations as { vmid => target }, its log
+# output and warnings. Dies like the API call if the request is rejected.
+sub bulk_migrate($user, $param) {
+    $worker = { migrations => {}, warnings => [] };
+
+    my ($handler, $info) =
+        PVE::API2::Cluster::BulkAction::Guest->find_handler('POST', 'migrate', {});
+
+    my $call_param = {%$param};
+    PVE::JSONSchema::validate($call_param, $info->{parameters});
+
+    $rpcenv->set_user($user);
+    $rpcenv->check_api2_permissions($info->{permissions}, $user, $call_param);
+
+    $handler->handle($info, $call_param);
+    die "no worker started\n" if !$worker->{code};
+
+    open(my $log_fh, '>', \$worker->{log}) or die "cannot capture worker log - $!\n";
+    my $previous_fh = select($log_fh);
+    eval { $worker->{code}->() };
+    my $err = $@;
+    select($previous_fh);
+    die "worker failed - $err" if $err;
+
+    return $worker;
+}
+
+# The exception a rejected bulk migration raises, or undef if it is accepted.
+sub bulk_migrate_error($user, $param) {
+    eval { bulk_migrate($user, $param) };
+    return $@ || undef;
+}
+
+subtest 'single target node' => sub {
+    my $run = bulk_migrate('root@pam', { vms => [100, 102], target => 'node3' });
+    is_deeply($run->{migrations}, { 100 => 'node3', 102 => 'node3' }, 'guests go to target');
+    like($run->{log}, qr/^Migrating guests to node3: 100, 102$/m, 'log names the target node');
+};
+
+subtest 'per-guest target nodes' => sub {
+    my $param = { vms => [100, 101, 103], target => 'node2' };
+    $param->{'target-map'} = '101:node3,103:node3';
+
+    my $run = bulk_migrate('root@pam', $param);
+    my $expected = { 100 => 'node2', 101 => 'node3' };
+    is_deeply($run->{migrations}, $expected, "'target-map' takes precedence over 'target'");
+    like(
+        join('', $run->{warnings}->@*),
+        qr/VM 103 already on node3, skipping/,
+        'guest already on its target node is skipped',
+    );
+
+    # whitespace and semicolons separate list entries like everywhere else
+    $param = { vms => [100, 101], 'target-map' => '100:node2; 101:node3' };
+    $run = bulk_migrate('root@pam', $param);
+    $expected = { 100 => 'node2', 101 => 'node3' };
+    is_deeply($run->{migrations}, $expected, 'all list separators are accepted');
+
+    # with a default target, omitting 'vms' still means all guests
+    $run = bulk_migrate('root@pam', { target => 'node3', 'target-map' => '100:node2' });
+    ok(!defined($run->{filter}), "with 'target' and without 'vms' all guests are considered");
+    $expected = { 100 => 'node2', 101 => 'node3', 102 => 'node3' };
+    is_deeply($run->{migrations}, $expected, 'unmapped guests go to the default target');
+};
+
+subtest 'only mapped guests without vms and target' => sub {
+    # without a default target the other guests could only be skipped, one warning each
+    my $run = bulk_migrate('root@pam', { 'target-map' => '102:node3,100:node2' });
+    is($run->{filter}, '100,102', 'action is limited to the mapped guests');
+    is_deeply($run->{migrations}, { 100 => 'node2', 102 => 'node3' }, 'mapped guests migrate');
+    is_deeply($run->{warnings}, [], 'no warnings for the other guests');
+
+    # as the mapped guests are the explicit selection, permissions on them suffice
+    $migrate_acl = { 'op@pve' => ['/vms/100', '/vms/102'] };
+    $run = bulk_migrate('op@pve', { 'target-map' => '100:node2,102:node3' });
+    is_deeply($run->{migrations}, { 100 => 'node2', 102 => 'node3' }, 'no privilege on /');
+
+    my $err = bulk_migrate_error('op@pve', { 'target-map' => '100:node2,101:node3' });
+    is($err->{code}, 403, "mapped guest without 'VM.Migrate' is rejected");
+};
+
+subtest 'invalid target maps are rejected' => sub {
+    my $not_a_pair = qr/not a 'vmid:node' pair/;
+
+    # name, 'target-map', expected error, 'vms' as list (default: 100)
+    my $cases = [
+        ['bare node', 'node2', $not_a_pair],
+        ['bare guest id', '100', $not_a_pair],
+        ['pair without node', '100:', $not_a_pair],
+        ['invalid guest id', '0100:node2', $not_a_pair],
+        ['invalid node name', '100:-x', $not_a_pair],
+        ['duplicate entry', '100:node2,100:node3', qr/duplicate entry for guest '100'/],
+        ['unknown node', '100:nodeX', qr/no such cluster node 'nodeX'/],
+        ['guest not in vms', '100:node2,101:node3', qr/guest '101' is not in the 'vms' list/],
+        ['guest without target', '100:node2', qr/no target node for guest\(s\) 101/, '100,101'],
+    ];
+
+    for my $case (@$cases) {
+        my ($name, $target_map, $message, $vms) = @$case;
+        my $param = { vms => [split(/,/, $vms // '100')], 'target-map' => $target_map };
+
+        my $err = bulk_migrate_error('root@pam', $param);
+        is($err->{code}, 400, "$name: rejected");
+        like($err->{errors}->{'target-map'}, $message, "$name: error message");
+        ok(!$worker->{code}, "$name: no worker started");
+    }
+
+    my $err = bulk_migrate_error('root@pam', { vms => [100] });
+    is($err->{code}, 400, 'neither target nor target map: rejected');
+    like($err->{errors}->{target}, qr/neither 'target' nor 'target-map'/, 'error names both');
+};
+
+done_testing();
diff --git a/test/Makefile b/test/Makefile
index 5073c1bfc..20c00d624 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -5,7 +5,7 @@ all:
 export PERLLIB=..
 
 .PHONY: check
-check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-resource-scheduling test-pvesh test-report
+check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-resource-scheduling test-bulk-migrate test-pvesh test-report
 
 .PHONY: test-balloon
 test-balloon:
@@ -61,6 +61,10 @@ test-custom-cpu-models:
 test-resource-scheduling:
 	./ResourceScheduling_test.pl
 
+.PHONY: test-bulk-migrate
+test-bulk-migrate:
+	./BulkMigrate_test.pl
+
 .PHONY: test-pvesh
 test-pvesh:
 	./pvesh_test.pl
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH manager 13/15] ui: bulk migrate: support per-guest target nodes
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (11 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH manager 12/15] api: bulk migrate: allow a per-guest target node mapping Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH manager 14/15] ui: dc: add a Rebalance Guests bulk action Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH docs 15/15] pvecm: document balancing the cluster load with migrations Thomas Lamprecht
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

Add a Target column to the guest list of the Datacenter bulk migrate
dialog, to migrate guests to different nodes in one go, and a Suggest
Migrations button that fills it from a rebalance proposal for the
selected guests, or all listed guests if none are selected. Only the
guests that the proposal moves stay selected, as the others would
otherwise be migrated to the default target node. They are listed
first, so that a proposal stays together in a long guest list.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 www/manager6/Makefile                     |   1 +
 www/manager6/form/VMSelectorWithTarget.js | 173 ++++++++++++++++++++++
 www/manager6/window/BulkAction.js         | 143 +++++++++++++++++-
 3 files changed, 314 insertions(+), 3 deletions(-)
 create mode 100644 www/manager6/form/VMSelectorWithTarget.js

diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index e8e2bd0b4..299a809e1 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -87,6 +87,7 @@ JSSRC= 							\
 	form/VLanField.js				\
 	form/VMCPUFlagSelector.js			\
 	form/VMSelector.js				\
+	form/VMSelectorWithTarget.js			\
 	form/VNCKeyboardSelector.js			\
 	form/ViewSelector.js				\
 	form/iScsiProviderSelector.js			\
diff --git a/www/manager6/form/VMSelectorWithTarget.js b/www/manager6/form/VMSelectorWithTarget.js
new file mode 100644
index 000000000..ad09221d5
--- /dev/null
+++ b/www/manager6/form/VMSelectorWithTarget.js
@@ -0,0 +1,173 @@
+// A VMSelector with an extra, editable per-guest target node column, for bulk migrations that send
+// different guests to different nodes. The per-guest targets are held in a vmid-keyed map, not on
+// the records, so they survive store filtering and refreshes.
+Ext.define('PVE.form.VMSelectorWithTarget', {
+    extend: 'PVE.form.VMSelector',
+    alias: 'widget.pveVMSelectorWithTarget',
+
+    // the extra Target column would otherwise push the grid past the dialog width
+    forceFit: true,
+
+    // Returns the target node used for guests without a per-guest target.
+    getDefaultTarget: () => undefined,
+
+    targetOverrides: null,
+
+    setTargetOverride: function (vmid, node) {
+        let me = this;
+        if (node) {
+            me.targetOverrides[vmid] = node;
+        } else {
+            delete me.targetOverrides[vmid];
+        }
+        me.updateTargetWidgets();
+        me.validate();
+        me.fireEvent('targetchange', me);
+    },
+
+    // Replace all per-guest targets with the given vmid to node map.
+    setTargetOverrides: function (map) {
+        let me = this;
+        me.targetOverrides = { ...map };
+        me.updateTargetWidgets();
+        me.validate();
+    },
+
+    // Only the rows of selected guests show their target, so the column stays quiet for the
+    // guests that are not migrated.
+    updateTargetWidget: function (combo, record) {
+        let me = this;
+        let override = me.targetOverrides[record.data.vmid];
+        combo.setValue(override ?? null);
+        combo.getTrigger('clear').setVisible(!!override);
+
+        let emptyText = '';
+        if (me.getSelectionModel().isSelected(record)) {
+            let defaultTarget = me.getDefaultTarget();
+            emptyText = defaultTarget
+                ? `${Proxmox.Utils.defaultText} (${defaultTarget})`
+                : Proxmox.Utils.defaultText;
+        }
+        combo.setEmptyText(emptyText);
+    },
+
+    updateTargetWidgets: function () {
+        let me = this;
+        Ext.suspendLayouts();
+        for (const combo of me.query('combobox')) {
+            let record = combo.getWidgetRecord?.();
+            if (record) {
+                me.updateTargetWidget(combo, record);
+            }
+        }
+        Ext.resumeLayouts(true);
+    },
+
+    // The selected guests' per-guest targets as a list of 'vmid:node' pairs.
+    getTargetMap: function () {
+        let me = this;
+        return me
+            .getValue()
+            .filter((vmid) => me.targetOverrides[vmid])
+            .map((vmid) => `${vmid}:${me.targetOverrides[vmid]}`)
+            .join(',');
+    },
+
+    // rows without a per-guest target show the default target
+    onDefaultTargetChange: function () {
+        let me = this;
+        me.updateTargetWidgets();
+    },
+
+    initComponent: function () {
+        let me = this;
+
+        me.targetOverrides = {};
+
+        // one node store shared by all per-row combos; online-only, like the default target field
+        me.nodeStore = Ext.create('Ext.data.Store', {
+            fields: ['node', 'status'],
+            proxy: { type: 'proxmox', url: '/api2/json/nodes' },
+            sorters: 'node',
+            filters: [{ filterFn: (rec) => rec.data.status === 'online' }],
+            autoLoad: true,
+        });
+
+        let targetColumn = {
+            header: gettext('Target'),
+            // No dataIndex: the target is not a model field. Binding one would make the
+            // widgetcolumn reset the combo to the empty field value on every store update, while
+            // onWidgetAttach must stay the only source of the combo's value.
+            width: 160,
+            sortable: false,
+            hideable: false,
+            menuDisabled: true,
+            tooltip: gettext("Overrides the 'Target node' for this guest"),
+            xtype: 'widgetcolumn',
+            // rows are recycled, so restore this row's target on every (re)attach
+            onWidgetAttach: (column, combo, record) => me.updateTargetWidget(combo, record),
+            widget: {
+                xtype: 'combobox',
+                store: me.nodeStore,
+                valueField: 'node',
+                displayField: 'node',
+                editable: false,
+                queryMode: 'local',
+                triggers: {
+                    clear: {
+                        cls: 'pmx-clear-trigger',
+                        hidden: true,
+                        handler: function (combo) {
+                            let record = combo.getWidgetRecord();
+                            if (record) {
+                                me.setTargetOverride(record.data.vmid, null);
+                            }
+                        },
+                    },
+                },
+                listeners: {
+                    // Only an explicit pick updates the map. Row recycling and store updates fire
+                    // 'change' but not 'select', so they cannot overwrite another row's target.
+                    select: function (combo) {
+                        let record = combo.getWidgetRecord();
+                        if (!record) {
+                            return;
+                        }
+                        me.targetOverrides[record.data.vmid] = combo.getValue();
+                        // picking a target for a guest expresses the intent to migrate it
+                        me.getSelectionModel().select(record, true);
+                        me.updateTargetWidget(combo, record);
+                        me.validate();
+                        me.fireEvent('targetchange', me);
+                    },
+                },
+            },
+        };
+
+        // HA state is a short value, so narrow it. Copy the configs, as the declaration is shared.
+        me.columnsDeclaration = me.columnsDeclaration.flatMap((column) => {
+            if (column.dataIndex === 'hastate') {
+                let narrowed = { ...column, width: 90 };
+                delete narrowed.flex;
+                return [narrowed];
+            }
+            if (column.dataIndex === 'node') {
+                return [{ ...column, width: 120 }, targetColumn];
+            }
+            return [column];
+        });
+
+        me.callParent();
+
+        me.on('selectionchange', me.updateTargetWidgets, me);
+    },
+
+    doDestroy: function () {
+        let me = this;
+
+        me.callParent();
+
+        // the node store is shared across rows and not destroyed by the combos
+        me.nodeStore = Ext.destroy(me.nodeStore);
+    },
+});
diff --git a/www/manager6/window/BulkAction.js b/www/manager6/window/BulkAction.js
index 3f8ad5674..c81784948 100644
--- a/www/manager6/window/BulkAction.js
+++ b/www/manager6/window/BulkAction.js
@@ -31,6 +31,13 @@ Ext.define('PVE.window.BulkAction', {
             params.vms = params.vms.split(/[,; ]/);
         }
 
+        if (me.action === 'migrate') {
+            let targetMap = me.down('#vms').getTargetMap();
+            if (targetMap) {
+                params['target-map'] = targetMap;
+            }
+        }
+
         Proxmox.Utils.API2Request({
             params: params,
             url,
@@ -63,6 +70,11 @@ Ext.define('PVE.window.BulkAction', {
             throw 'no title specified';
         }
 
+        // the guest grid of the cluster-wide migration has an extra per-guest Target column
+        if (me.action === 'migrate') {
+            me.width = 1100;
+        }
+
         let items = [];
         if (me.action === 'migrateall' || me.action === 'migrate') {
             let disallowedNodes = [];
@@ -77,11 +89,15 @@ Ext.define('PVE.window.BulkAction', {
                         {
                             flex: 1,
                             xtype: 'pveNodeSelector',
+                            itemId: 'defaultTarget',
                             name: 'target',
                             disallowedNodes,
                             fieldLabel: gettext('Target node'),
                             labelWidth: 200,
                             allowBlank: false,
+                            listeners: {
+                                change: () => me.down('#vms')?.onDefaultTargetChange?.(),
+                            },
                             onlineValidator: true,
                             padding: '0 10 0 0',
                         },
@@ -506,7 +522,20 @@ Ext.define('PVE.window.BulkAction', {
             ],
         });
 
-        items.push({
+        // The status describes the last proposal, so show it only while the selected guests and
+        // their targets match that proposal.
+        let proposedMigrations = null;
+        let updateSummary = function () {
+            let grid = me.down('#vms');
+            let current = grid
+                .getValue()
+                .map((vmid) => `${vmid}:${grid.targetOverrides[vmid] ?? ''}`)
+                .sort()
+                .join();
+            me.down('#rebalanceSummary').setVisible(current === proposedMigrations);
+        };
+
+        let selector = {
             xtype: 'vmselector',
             itemId: 'vms',
             name: 'vms',
@@ -524,7 +553,18 @@ Ext.define('PVE.window.BulkAction', {
                     }
                 },
             },
-        });
+        };
+        if (me.action === 'migrate') {
+            Ext.apply(selector, {
+                xtype: 'pveVMSelectorWithTarget',
+                getDefaultTarget: () => me.down('#defaultTarget')?.getValue(),
+            });
+            Ext.apply(selector.listeners, {
+                change: updateSummary,
+                targetchange: updateSummary,
+            });
+        }
+        items.push(selector);
 
         me.formPanel = Ext.create('Ext.form.Panel', {
             bodyPadding: 10,
@@ -549,9 +589,106 @@ Ext.define('PVE.window.BulkAction', {
             },
         });
 
+        // Fill the per-guest targets from a rebalance proposal for the selected guests, or for all
+        // listed guests if none are selected. Only the guests the proposal moves stay selected, as
+        // the others would otherwise be migrated to the default target node.
+        let suggestTargets = function () {
+            let grid = me.down('#vms');
+            let store = grid.getStore();
+            let guests = grid.getValue();
+            if (!guests.length && store.isLoaded()) {
+                guests = store.getRange().map((record) => record.data.vmid);
+                if (!guests.length) {
+                    return; // the filters hide all guests
+                }
+            }
+            let params = {};
+            // Without a list, the API considers all guests the user may migrate, while a list
+            // needs that permission on every guest in it.
+            let total = (store.getData().getSource() ?? store.getData()).getCount();
+            if (guests.length && guests.length < total) {
+                params.vms = guests.join(',');
+            }
+
+            Proxmox.Utils.API2Request({
+                url: '/cluster/resource-scheduling/rebalance-plan',
+                method: 'POST',
+                params,
+                waitMsgTarget: me,
+                failure: function (response) {
+                    if (!me.destroyed) {
+                        Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+                    }
+                },
+                success: function ({ result }) {
+                    if (me.destroyed) {
+                        return;
+                    }
+                    // apply even an empty proposal, so a previous one does not stay ready to submit
+                    let { moves, imbalance } = result.data;
+                    proposedMigrations = moves
+                        .map((move) => `${move.vmid}:${move.target}`)
+                        .sort()
+                        .join();
+                    let before = `${(imbalance.before * 100).toFixed(1)}%`;
+                    let after = `${(imbalance.after * 100).toFixed(1)}%`;
+                    me.down('#rebalanceSummary').setHtml(
+                        moves.length
+                            ? Ext.String.format(
+                                  gettext(
+                                      'Cluster imbalance: {0} now, {1} expected after migration',
+                                  ),
+                                  before,
+                                  after,
+                              )
+                            : Ext.String.format(
+                                  gettext(
+                                      'Cluster imbalance: {0} now, no migration would noticeably lower it',
+                                  ),
+                                  before,
+                              ),
+                    );
+                    grid.setTargetOverrides(
+                        Object.fromEntries(moves.map((move) => [move.vmid, move.target])),
+                    );
+                    grid.setValue(moves.map((move) => move.vmid));
+                    // show the proposed guests together, not scattered over a long guest list
+                    let proposed = new Set(moves.map((move) => move.vmid));
+                    store.sort([
+                        {
+                            sorterFn: (a, b) =>
+                                proposed.has(b.data.vmid) - proposed.has(a.data.vmid),
+                        },
+                        { property: 'vmid' },
+                    ]);
+                    grid.getView().setScrollY(0);
+                    updateSummary();
+                },
+            });
+        };
+
+        let buttons = [submitBtn];
+        if (me.action === 'migrate') {
+            buttons.unshift(
+                {
+                    text: gettext('Suggest Migrations'),
+                    tooltip: gettext(
+                        'Propose migrations that balance the cluster load, for the selected guests or, if none are selected, all listed guests. The proposal replaces the selection and the per-guest targets.',
+                    ),
+                    handler: suggestTargets,
+                },
+                {
+                    xtype: 'tbtext',
+                    itemId: 'rebalanceSummary',
+                    hidden: true,
+                },
+                '->',
+            );
+        }
+
         Ext.apply(me, {
             items: [me.formPanel],
-            buttons: [submitBtn],
+            buttons,
         });
 
         me.callParent();
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH manager 14/15] ui: dc: add a Rebalance Guests bulk action
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (12 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH manager 13/15] ui: bulk migrate: support per-guest target nodes Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  2026-11-18 11:05 ` [PATCH docs 15/15] pvecm: document balancing the cluster load with migrations Thomas Lamprecht
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

Add a Rebalance Guests entry to the Datacenter bulk actions. It opens
the bulk migrate dialog with the guests and target nodes of a
rebalance proposal for all guests the user may migrate, so the
proposed migrations can be reviewed and adjusted before starting them.
The tag and pool menus do not offer it, as it considers all guests.

In this mode, the dialog starts without a default target node, so a
guest added to the selection needs its own target instead of being
migrated to a node that nobody picked.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 www/manager6/dc/CmdMenu.js                | 21 +++++++++++
 www/manager6/dc/Config.js                 | 16 +++++++++
 www/manager6/form/VMSelectorWithTarget.js | 43 +++++++++++++++++++----
 www/manager6/window/BulkAction.js         | 20 ++++++++++-
 4 files changed, 92 insertions(+), 8 deletions(-)

diff --git a/www/manager6/dc/CmdMenu.js b/www/manager6/dc/CmdMenu.js
index a8df24104..1ca035262 100644
--- a/www/manager6/dc/CmdMenu.js
+++ b/www/manager6/dc/CmdMenu.js
@@ -71,6 +71,21 @@ Ext.define('PVE.dc.CmdMenu', {
                 });
             },
         },
+        {
+            text: gettext('Rebalance Guests'),
+            itemId: 'rebalance',
+            iconCls: 'fa fa-fw fa-balance-scale',
+            handler: function () {
+                Ext.create('PVE.window.BulkAction', {
+                    autoShow: true,
+                    vmsAsArray: true,
+                    title: gettext('Rebalance Guests'),
+                    btnText: gettext('Migrate'),
+                    action: 'migrate',
+                    rebalance: true,
+                });
+            },
+        },
     ],
 
     initComponent: function () {
@@ -90,6 +105,7 @@ Ext.define('PVE.dc.CmdMenu', {
 
         if (!caps.vms['VM.Migrate']) {
             me.getComponent('bulkmigrate').setDisabled(true);
+            me.getComponent('rebalance').setDisabled(true);
         }
         if (!caps.vms['VM.PowerMgmt']) {
             me.getComponent('bulkstart').setDisabled(true);
@@ -98,6 +114,11 @@ Ext.define('PVE.dc.CmdMenu', {
         }
         if (PVE.Utils.isStandaloneNode()) {
             me.getComponent('bulkmigrate').setVisible(false);
+            me.getComponent('rebalance').setVisible(false);
+        }
+        // a rebalance considers all guests, not only those a tag or pool menu preselects
+        if (Object.keys(me.extraHandlerArgs).length) {
+            me.getComponent('rebalance').setVisible(false);
         }
     },
 });
diff --git a/www/manager6/dc/Config.js b/www/manager6/dc/Config.js
index 83c48c3d6..655e22207 100644
--- a/www/manager6/dc/Config.js
+++ b/www/manager6/dc/Config.js
@@ -78,6 +78,22 @@ Ext.define('PVE.dc.Config', {
                             });
                         },
                     },
+                    {
+                        text: gettext('Rebalance Guests'),
+                        iconCls: 'fa fa-fw fa-balance-scale',
+                        disabled: !caps.vms['VM.Migrate'],
+                        hidden: PVE.Utils.isStandaloneNode(),
+                        handler: function () {
+                            Ext.create('PVE.window.BulkAction', {
+                                autoShow: true,
+                                vmsAsArray: true,
+                                title: gettext('Rebalance Guests'),
+                                btnText: gettext('Migrate'),
+                                action: 'migrate',
+                                rebalance: true,
+                            });
+                        },
+                    },
                 ],
             }),
         });
diff --git a/www/manager6/form/VMSelectorWithTarget.js b/www/manager6/form/VMSelectorWithTarget.js
index ad09221d5..a4cc3a3da 100644
--- a/www/manager6/form/VMSelectorWithTarget.js
+++ b/www/manager6/form/VMSelectorWithTarget.js
@@ -8,7 +8,8 @@ Ext.define('PVE.form.VMSelectorWithTarget', {
     // the extra Target column would otherwise push the grid past the dialog width
     forceFit: true,
 
-    // Returns the target node used for guests without a per-guest target.
+    // Returns the target node used for guests without a per-guest target, if any. Without one,
+    // every selected guest needs a per-guest target.
     getDefaultTarget: () => undefined,
 
     targetOverrides: null,
@@ -33,22 +34,28 @@ Ext.define('PVE.form.VMSelectorWithTarget', {
         me.validate();
     },
 
-    // Only the rows of selected guests show their target, so the column stays quiet for the
-    // guests that are not migrated.
+    // Only the rows of selected guests show their target or that it is missing, so the column
+    // stays quiet for the guests that are not migrated.
     updateTargetWidget: function (combo, record) {
         let me = this;
         let override = me.targetOverrides[record.data.vmid];
         combo.setValue(override ?? null);
         combo.getTrigger('clear').setVisible(!!override);
 
+        let selected = me.getSelectionModel().isSelected(record);
+        let defaultTarget = me.getDefaultTarget();
         let emptyText = '';
-        if (me.getSelectionModel().isSelected(record)) {
-            let defaultTarget = me.getDefaultTarget();
+        if (selected) {
             emptyText = defaultTarget
                 ? `${Proxmox.Utils.defaultText} (${defaultTarget})`
-                : Proxmox.Utils.defaultText;
+                : gettext('Select node');
         }
         combo.setEmptyText(emptyText);
+        if (selected && !override && !defaultTarget) {
+            combo.markInvalid(gettext('Select a target node for this guest'));
+        } else {
+            combo.clearInvalid();
+        }
     },
 
     updateTargetWidgets: function () {
@@ -73,10 +80,29 @@ Ext.define('PVE.form.VMSelectorWithTarget', {
             .join(',');
     },
 
-    // rows without a per-guest target show the default target
+    // The default target decides whether a guest without a per-guest target is valid and what
+    // its row shows.
     onDefaultTargetChange: function () {
         let me = this;
         me.updateTargetWidgets();
+        me.validate();
+    },
+
+    getErrors: function (value) {
+        let me = this;
+        let errors = me.callParent([value]);
+        if (errors.length || me.isDisabled() || me.getDefaultTarget()) {
+            return errors;
+        }
+
+        let missing = me.getValue().filter((vmid) => !me.targetOverrides[vmid]);
+        if (missing.length) {
+            me.addBodyCls(['x-form-trigger-wrap-default', 'x-form-trigger-wrap-invalid']);
+            return [
+                Ext.String.format(gettext('No target node for guests: {0}'), missing.join(', ')),
+            ];
+        }
+        return errors;
     },
 
     initComponent: function () {
@@ -113,6 +139,9 @@ Ext.define('PVE.form.VMSelectorWithTarget', {
                 displayField: 'node',
                 editable: false,
                 queryMode: 'local',
+                // the row's validity depends on the selection, see updateTargetWidget
+                validateOnChange: false,
+                validateOnBlur: false,
                 triggers: {
                     clear: {
                         cls: 'pmx-clear-trigger',
diff --git a/www/manager6/window/BulkAction.js b/www/manager6/window/BulkAction.js
index c81784948..44b33179a 100644
--- a/www/manager6/window/BulkAction.js
+++ b/www/manager6/window/BulkAction.js
@@ -17,6 +17,10 @@ Ext.define('PVE.window.BulkAction', {
     // necessary for the cluster-wide api call
     vmsAsArray: false,
 
+    // For the cluster-wide 'migrate' action: start with the guests and per-guest targets of a
+    // rebalance proposal instead of all guests and a single target node.
+    rebalance: false,
+
     submit: function (params) {
         let me = this;
 
@@ -36,6 +40,10 @@ Ext.define('PVE.window.BulkAction', {
             if (targetMap) {
                 params['target-map'] = targetMap;
             }
+            // in rebalance mode the default target is optional
+            if (!params.target) {
+                delete params.target;
+            }
         }
 
         Proxmox.Utils.API2Request({
@@ -94,7 +102,12 @@ Ext.define('PVE.window.BulkAction', {
                             disallowedNodes,
                             fieldLabel: gettext('Target node'),
                             labelWidth: 200,
-                            allowBlank: false,
+                            // in rebalance mode every proposed guest has its own target, and an
+                            // automatically picked default would silently apply to any guest the
+                            // user adds to the selection
+                            allowBlank: me.rebalance,
+                            autoSelect: !me.rebalance,
+                            emptyText: me.rebalance ? gettext('Per-guest targets only') : undefined,
                             listeners: {
                                 change: () => me.down('#vms')?.onDefaultTargetChange?.(),
                             },
@@ -557,6 +570,7 @@ Ext.define('PVE.window.BulkAction', {
         if (me.action === 'migrate') {
             Ext.apply(selector, {
                 xtype: 'pveVMSelectorWithTarget',
+                selectAll: !me.rebalance,
                 getDefaultTarget: () => me.down('#defaultTarget')?.getValue(),
             });
             Ext.apply(selector.listeners, {
@@ -707,5 +721,9 @@ Ext.define('PVE.window.BulkAction', {
         form.isValid();
 
         filterChange();
+
+        if (me.rebalance) {
+            me.on('show', suggestTargets, me, { single: true });
+        }
     },
 });
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

* [PATCH docs 15/15] pvecm: document balancing the cluster load with migrations
  2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
                   ` (13 preceding siblings ...)
  2026-11-18 11:05 ` [PATCH manager 14/15] ui: dc: add a Rebalance Guests bulk action Thomas Lamprecht
@ 2026-11-18 11:05 ` Thomas Lamprecht
  14 siblings, 0 replies; 16+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 11:05 UTC (permalink / raw)
  To: pve-devel

The web interface ranks the target nodes of a migration by the
resulting cluster load imbalance, and proposes rebalancing migrations
for all guests through the Rebalance Guests bulk action. Describe both
next to the other guest migration topics, and point there from the
CRS section of the HA chapter, which otherwise reads as if the
scheduler were available to HA resources only.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 ha-manager.adoc |  4 ++++
 pvecm.adoc      | 51 +++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 55 insertions(+)

diff --git a/ha-manager.adoc b/ha-manager.adoc
index 95ab799..671aab2 100644
--- a/ha-manager.adoc
+++ b/ha-manager.adoc
@@ -1464,6 +1464,10 @@ from a fenced node or a new HA affinity rule. See the section
 xref:ha_manager_crs_scheduling_points[CRS Scheduling Points] for a more thorough
 description of which actions the CRS takes for specific changes in the cluster.
 
+The scheduler can also propose migrations on request, for all guests and
+independent of the HA Manager rounds, see
+xref:pvecm_migration_load_balancing[Balancing the Cluster Load].
+
 CRS Scheduling Mode
 ~~~~~~~~~~~~~~~~~~~
 
diff --git a/pvecm.adoc b/pvecm.adoc
index cd5a0fb..6e017f3 100644
--- a/pvecm.adoc
+++ b/pvecm.adoc
@@ -1712,6 +1712,57 @@ migration: secure,network=10.1.2.0/24
 NOTE: The migration type must always be set when the migration network
 is set in `/etc/pve/datacenter.cfg`.
 
+[[pvecm_migration_load_balancing]]
+Balancing the Cluster Load
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The scheduler that the HA stack uses to place its resources, see
+xref:ha_manager_crs[Cluster Resource Scheduling], can also assist with the
+migrations you start yourself. This works for all guests, whether they are
+managed by HA or not, and requires the `VM.Migrate` privilege on them.
+
+The scheduler rates a migration by the cluster imbalance it would result in.
+The imbalance tells how much the load differs between the nodes, from 0% for an
+evenly loaded cluster up to 100%. It is calculated from the average CPU and
+memory usage of the nodes and their running guests. As long as a node did not
+report its usage yet, the configured CPU and memory limits of the guests are
+used instead. The CRS scheduling mode set for HA has no influence on this.
+
+When you migrate a running guest in the web interface, the target node selector
+lists the nodes by the imbalance that a migration to them would result in, and
+preselects the best one. Whether a migration is possible at all is still
+determined by the usual migration checks. Nodes that these checks rule out, for
+example because of an HA rule, are listed last.
+
+To spread the load of the whole cluster, select `Datacenter` -> `Bulk Actions`
+-> `Rebalance Guests` in the web interface. This opens the bulk migration dialog
+with a proposal of up to ten migrations that lower the cluster imbalance. Before
+you start them with `Migrate`, you can change the target node of each guest or
+deselect guests. `Suggest Migrations` requests a new proposal for the selected
+guests, or for all listed guests if none are selected. The same button is
+available in `Datacenter` -> `Bulk Actions` -> `Bulk Migrate`.
+
+A proposal follows these rules:
+
+* Only running guests are proposed, and no guests with a lock, for example from
+  a running backup.
+* Guests managed by HA are only proposed if their HA resource is started, and
+  only for nodes that their HA rules allow. Guests that HA would migrate
+  together with others because of a positive resource affinity are not proposed.
+* Nodes in HA maintenance mode are left out.
+* Migrations that would barely improve the balance are not proposed, as every
+  migration disrupts its guest.
+
+NOTE: Running containers are restarted when they are migrated, see the
+xref:pct_migration[Container Migration Chapter].
+
+The proposals are also available through the API, for example:
+
+----
+# pvesh get /cluster/resource-scheduling/recommend-target-nodes --vmid 100
+# pvesh create /cluster/resource-scheduling/rebalance-plan
+----
+
 
 ifdef::manvolnum[]
 include::pve-copyright.adoc[]
-- 
2.47.3





^ permalink raw reply related	[flat|nested] 16+ messages in thread

end of thread, other threads:[~2026-11-18 12:10 UTC | newest]

Thread overview: 16+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 11:05 [PATCH guest-common,ha-manager,manager,docs 0/15] cluster resource scheduling for all guests Thomas Lamprecht
2026-11-18 11:05 ` [PATCH guest-common 01/15] scheduling: add cluster resource scheduling library Thomas Lamprecht
2026-11-18 11:05 ` [PATCH ha-manager 02/15] helpers: block both locations of a migrating negative affinity partner Thomas Lamprecht
2026-11-18 11:05 ` [PATCH ha-manager 03/15] usage: static: count services per node again for scoring fallback Thomas Lamprecht
2026-11-18 11:05 ` [PATCH ha-manager 04/15] config: allow getting motion info of many resources at once Thomas Lamprecht
2026-11-18 11:05 ` [PATCH ha-manager 05/15] env: pve2: delegate cluster stat gathering to scheduling library Thomas Lamprecht
2026-11-18 11:05 ` [PATCH ha-manager 06/15] usage: share one scheduler base between static and dynamic Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 07/15] ui: bulk action: translate error title and CT restart warning Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 08/15] ui: bulk migrate: show CT restart warning for cluster-wide migration Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 09/15] ui: migrate: recheck preconditions if the target changed meanwhile Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 10/15] api: cluster: add resource scheduling endpoints Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 11/15] ui: migrate: sort target nodes by scheduler recommendation Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 12/15] api: bulk migrate: allow a per-guest target node mapping Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 13/15] ui: bulk migrate: support per-guest target nodes Thomas Lamprecht
2026-11-18 11:05 ` [PATCH manager 14/15] ui: dc: add a Rebalance Guests bulk action Thomas Lamprecht
2026-11-18 11:05 ` [PATCH docs 15/15] pvecm: document balancing the cluster load with migrations Thomas Lamprecht

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal