[pve-devel] [PATCH docs v2 1/4] ha: rules: update about mixed usage of node and resource affinity rules

[pve-devel] [PATCH docs v2 1/4] ha: rules: update about mixed usage of node and resource affinity rules

[Daniel Kral]

As HA resources can be part of node and resource affinity rules at the
same time in simpler cases, update the relevant sections about specific
interactions and restrictions.

Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
changes since v1:
- s/rules/rule/
- simplify language in "Note that..." paragraph

 ha-manager.adoc | 30 ++++++++++++++++++++++++++++--
 1 file changed, 28 insertions(+), 2 deletions(-)

diff --git a/ha-manager.adoc b/ha-manager.adoc
index d6ac75d..996543f 100644
--- a/ha-manager.adoc
+++ b/ha-manager.adoc
@@ -837,6 +837,24 @@ Two or more HA resources cannot be kept on the same node and separated on
 different nodes at the same time. For more information on these cases, see the
 section about xref:ha_manager_rule_conflicts[rule conflicts and errors] below.
 
+Interactions between Node and Positive Resource Affinity Rules
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+
+If there are HA resources in a node affinity rule, which are also part of a
+positive resource affinity rule, then all the other HA resources in the
+positive resource affinity rule inherit the node affinity rule as well.
+
+For example, if the HA resources `vm:100`, `vm:101`, and `vm:102` are in a
+positive resource affinity rule, and `vm:102` is in a node affinity rule, which
+restricts `vm:102` to be only on `node3`, then `vm:100` and `vm:101` are
+restricted to be only on `node3` as well.
+
+Please note that if there are two or more HA resources in a positive resource
+affinity rule and in different node affinity rules, then those rules will cause
+an error and will be disabled as this is currently not supported. For more
+information on these cases, see the section about
+xref:ha_manager_rule_conflicts[rule conflicts and errors] below.
+
 Resource Affinity Rule Properties
 +++++++++++++++++++++++++++++++++
 
@@ -866,8 +884,16 @@ Currently, HA rules are checked for the following feasibility tests:
   resources as a negative HA resources affinity rule. That is, two or more HA
   resources cannot be kept together and separate at the same time.
 
-* A HA resource can only be part of either a HA node affinity rule or HA
-  resource affinity rules, but not both at the same time.
+* An HA resource can only be part of a HA node affinity rule and a HA resource
+  affinity rule at the same time, if the HA node affinity rule has a single
+  priority class.
+
+* The HA resources of a positive HA resource affinity rule can only be part of
+  a single HA node affinity rule at most.
+
+* The HA resources of a negative HA resource affinity rule cannot be restricted
+  to less nodes than HA resources by their node affinity rules. Otherwise, the
+  HA resources do not have enough nodes to be separated.
 
 [[ha_manager_fencing]]
 Fencing

-- 
2.47.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs v2 2/4] ha: rules: document crs behavior for split positive resource affinity

[Daniel Kral]

Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
changes since v1:
- changed the latter sentence about alphabetical ordering

 ha-manager.adoc | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/ha-manager.adoc b/ha-manager.adoc
index 996543f..434c33f 100644
--- a/ha-manager.adoc
+++ b/ha-manager.adoc
@@ -787,6 +787,11 @@ HA resources `vm:101` and `vm:102` are each in a positive resource affinity
 rule, then it is the same as if `vm:100`, `vm:101` and `vm:102` would have been
 in a single positive resource affinity rule.
 
+NOTE: If the HA resources of a positive resource affinity rule are currently
+running on different nodes, the CRS will move the HA resources to the node,
+where most of them are running already. If there is a tie in the HA resource
+count, the node whose name appears first in alphabetical order is selected.
+
 However, suppose there are computationally expensive, and/or distributed
 programs running on the HA resources `vm:200` and `ct:300`, for example,
 sharded database instances. In that case, running them on the same node could
-- 
2.47.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs v2 3/4] ha: mark ha groups as deprecated and note migration to node affinity rules

[Daniel Kral]

As the migration is handled automatically by the HA Manager now instead
of manual user migration, the notice should be placed in the HA Groups
section instead.

Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
changes since v1:
- none

 ha-manager.adoc | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ha-manager.adoc b/ha-manager.adoc
index 434c33f..705f522 100644
--- a/ha-manager.adoc
+++ b/ha-manager.adoc
@@ -594,6 +594,9 @@ The above config was generated using the `ha-manager` command-line tool:
 Groups
 ~~~~~~
 
+NOTE: HA Groups are deprecated and migrated to HA Node Affinity rules since
+Proxmox VE 9.0.
+
 [thumbnail="screenshot/gui-ha-manager-groups-view.png"]
 
 The HA group configuration file `/etc/pve/ha/groups.cfg` is used to
@@ -700,9 +703,6 @@ on the same node.
 Node Affinity Rules
 ^^^^^^^^^^^^^^^^^^^
 
-NOTE: HA Node Affinity rules are equivalent to HA Groups and will replace them
-in an upcoming major release.
-
 By default, a HA resource is able to run on any cluster node, but a common
 requirement is that a HA resource should run on a specific node. That can be
 implemented by defining a HA node affinity rule to make the HA resource
-- 
2.47.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs v2 4/4] ha: replace in-text references to ha groups with ha rules

[Daniel Kral]

As HA groups are replaced by HA node affinity rules and HA rules became
more powerful than before now, update texts that reference HA groups
with references to HA rules instead.

Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
changes since v1:
- s/those nodes/these nodes/
- s/fufill/fulfill/
- revert changing "service{,s}" to "{HA ,}resources" in "Recover Fenced
  Services" section as it should rather be done in one sweep in a future
  patch

 ha-manager.adoc | 45 +++++++++++++++++++++++----------------------
 1 file changed, 23 insertions(+), 22 deletions(-)

diff --git a/ha-manager.adoc b/ha-manager.adoc
index 705f522..f16cfbb 100644
--- a/ha-manager.adoc
+++ b/ha-manager.adoc
@@ -314,9 +314,8 @@ recovery state.
 recovery::
 
 Wait for recovery of the service. The HA manager tries to find a new node where
-the service can run on. This search depends not only on the list of online and
-quorate nodes, but also if the service is a group member and how such a group
-is limited.
+the service can run on. This search depends on the list of online and quorate
+nodes as well as the affinity rules the service is part of, if any.
 As soon as a new available node is found, the service will be moved there and
 initially placed into stopped state. If it's configured to run the new node
 will do so.
@@ -980,15 +979,19 @@ Recover Fenced Services
 After a node failed and its fencing was successful, the CRM tries to
 move services from the failed node to nodes which are still online.
 
-The selection of nodes, on which those services gets recovered, is
-influenced by the resource `group` settings, the list of currently active
-nodes, and their respective active service count.
+The selection of the recovery nodes is influenced by the list of
+currently active nodes, their respective loads depending on the used
+scheduler, and the affinity rules the service is part of, if any.
 
-The CRM first builds a set out of the intersection between user selected
-nodes (from `group` setting) and available nodes. It then choose the
-subset of nodes with the highest priority, and finally select the node
-with the lowest active service count. This minimizes the possibility
-of an overloaded node.
+First, the CRM builds a set of nodes available to the service. If the
+service is part of a node affinity rule, the set is reduced to the
+highest priority nodes in the node affinity rule. If the service is part
+of a resource affinity rule, the set is further reduced to fulfill their
+constraints, which is either keeping the service on the same node as
+some other services or keeping the service on a different node than some
+other services. Finally, the CRM selects the node with the lowest load
+according to the used scheduler to minimize the possibility of an
+overloaded node.
 
 CAUTION: On node failure, the CRM distributes services to the
 remaining nodes. This increases the service count on those nodes, and
@@ -1103,7 +1106,7 @@ You can use the manual maintenance mode to mark the node as unavailable for HA
 operation, prompting all services managed by HA to migrate to other nodes.
 
 The target nodes for these migrations are selected from the other currently
-available nodes, and determined by the HA group configuration and the configured
+available nodes, and determined by the HA rules configuration and the configured
 cluster resource scheduler (CRS) mode.
 During each migration, the original node will be recorded in the HA managers'
 state, so that the service can be moved back again automatically once the
@@ -1174,14 +1177,12 @@ This triggers a migration of all HA Services currently located on this node.
 The LRM will try to delay the shutdown process, until all running services get
 moved away. But, this expects that the running services *can* be migrated to
 another node. In other words, the service must not be locally bound, for example
-by using hardware passthrough. As non-group member nodes are considered as
-runnable target if no group member is available, this policy can still be used
-when making use of HA groups with only some nodes selected. But, marking a group
-as 'restricted' tells the HA manager that the service cannot run outside of the
-chosen set of nodes. If all of those nodes are unavailable, the shutdown will
-hang until you manually intervene. Once the shut down node comes back online
-again, the previously displaced services will be moved back, if they were not
-already manually migrated in-between.
+by using hardware passthrough. For example, strict node affinity rules tell the
+HA Manager that the service cannot run outside of the chosen set of nodes. If all
+of these nodes are unavailable, the shutdown will hang until you manually
+intervene. Once the shut down node comes back online again, the previously
+displaced services will be moved back, if they were not already manually migrated
+in-between.
 
 NOTE: The watchdog is still active during the migration process on shutdown.
 If the node loses quorum it will be fenced and the services will be recovered.
@@ -1267,8 +1268,8 @@ The change will be in effect starting with the next manager round (after a few
 seconds).
 
 For each service that needs to be recovered or migrated, the scheduler
-iteratively chooses the best node among the nodes with the highest priority in
-the service's group.
+iteratively chooses the best node among the nodes that are available to
+the service according to their HA rules, if any.
 
 NOTE: There are plans to add modes for (static and dynamic) load-balancing in
 the future.
-- 
2.47.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] applied: [PATCH docs v2 1/4] ha: rules: update about mixed usage of node and resource affinity rules

[Thomas Lamprecht]

On Tue, 05 Aug 2025 09:58:36 +0200, Daniel Kral wrote:
> As HA resources can be part of node and resource affinity rules at the
> same time in simpler cases, update the relevant sections about specific
> interactions and restrictions.
> 
> 

Applied, thanks!

[1/4] ha: rules: update about mixed usage of node and resource affinity rules
      commit: 72f874e505fdfe85fe287ba99ed57f88c1c5bff4
[2/4] ha: rules: document crs behavior for split positive resource affinity
      commit: da333059f54bdc19c3f9a7435e614d209f4930e0
[3/4] ha: mark ha groups as deprecated and note migration to node affinity rules
      commit: aecd8e5497cf156add07910faf561cba8106fa63
[4/4] ha: replace in-text references to ha groups with ha rules
      commit: 62815c23d86215da50a91f11929269ad6d33a436


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel