From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 04BA61FF0C0 for ; Wed, 18 Nov 2026 12:08:25 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 5BC6621810; Wed, 18 Nov 2026 12:07:35 +0100 (CET) From: Thomas Lamprecht To: pve-devel@lists.proxmox.com Subject: [PATCH docs 15/15] pvecm: document balancing the cluster load with migrations Date: Wed, 18 Nov 2026 12:05:22 +0100 Message-ID: <20261118110705.595989-16-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261118110705.595989-1-t.lamprecht@proxmox.com> References: <20261118110705.595989-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1795000031136 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.025 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) POISEN_SPAM_PILL 0.1 Meta: its spam POISEN_SPAM_PILL_1 0.1 random spam to be learned in bayes POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: PVYJ3SU6QEI5BY2EU2A4OYRPHXIZCZCJ X-Message-ID-Hash: PVYJ3SU6QEI5BY2EU2A4OYRPHXIZCZCJ X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: The web interface ranks the target nodes of a migration by the resulting cluster load imbalance, and proposes rebalancing migrations for all guests through the Rebalance Guests bulk action. Describe both next to the other guest migration topics, and point there from the CRS section of the HA chapter, which otherwise reads as if the scheduler were available to HA resources only. Signed-off-by: Thomas Lamprecht --- ha-manager.adoc | 4 ++++ pvecm.adoc | 51 +++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+) diff --git a/ha-manager.adoc b/ha-manager.adoc index 95ab799..671aab2 100644 --- a/ha-manager.adoc +++ b/ha-manager.adoc @@ -1464,6 +1464,10 @@ from a fenced node or a new HA affinity rule. See the section xref:ha_manager_crs_scheduling_points[CRS Scheduling Points] for a more thorough description of which actions the CRS takes for specific changes in the cluster. +The scheduler can also propose migrations on request, for all guests and +independent of the HA Manager rounds, see +xref:pvecm_migration_load_balancing[Balancing the Cluster Load]. + CRS Scheduling Mode ~~~~~~~~~~~~~~~~~~~ diff --git a/pvecm.adoc b/pvecm.adoc index cd5a0fb..6e017f3 100644 --- a/pvecm.adoc +++ b/pvecm.adoc @@ -1712,6 +1712,57 @@ migration: secure,network=10.1.2.0/24 NOTE: The migration type must always be set when the migration network is set in `/etc/pve/datacenter.cfg`. +[[pvecm_migration_load_balancing]] +Balancing the Cluster Load +~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The scheduler that the HA stack uses to place its resources, see +xref:ha_manager_crs[Cluster Resource Scheduling], can also assist with the +migrations you start yourself. This works for all guests, whether they are +managed by HA or not, and requires the `VM.Migrate` privilege on them. + +The scheduler rates a migration by the cluster imbalance it would result in. +The imbalance tells how much the load differs between the nodes, from 0% for an +evenly loaded cluster up to 100%. It is calculated from the average CPU and +memory usage of the nodes and their running guests. As long as a node did not +report its usage yet, the configured CPU and memory limits of the guests are +used instead. The CRS scheduling mode set for HA has no influence on this. + +When you migrate a running guest in the web interface, the target node selector +lists the nodes by the imbalance that a migration to them would result in, and +preselects the best one. Whether a migration is possible at all is still +determined by the usual migration checks. Nodes that these checks rule out, for +example because of an HA rule, are listed last. + +To spread the load of the whole cluster, select `Datacenter` -> `Bulk Actions` +-> `Rebalance Guests` in the web interface. This opens the bulk migration dialog +with a proposal of up to ten migrations that lower the cluster imbalance. Before +you start them with `Migrate`, you can change the target node of each guest or +deselect guests. `Suggest Migrations` requests a new proposal for the selected +guests, or for all listed guests if none are selected. The same button is +available in `Datacenter` -> `Bulk Actions` -> `Bulk Migrate`. + +A proposal follows these rules: + +* Only running guests are proposed, and no guests with a lock, for example from + a running backup. +* Guests managed by HA are only proposed if their HA resource is started, and + only for nodes that their HA rules allow. Guests that HA would migrate + together with others because of a positive resource affinity are not proposed. +* Nodes in HA maintenance mode are left out. +* Migrations that would barely improve the balance are not proposed, as every + migration disrupts its guest. + +NOTE: Running containers are restarted when they are migrated, see the +xref:pct_migration[Container Migration Chapter]. + +The proposals are also available through the API, for example: + +---- +# pvesh get /cluster/resource-scheduling/recommend-target-nodes --vmid 100 +# pvesh create /cluster/resource-scheduling/rebalance-plan +---- + ifdef::manvolnum[] include::pve-copyright.adoc[] -- 2.47.3