From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 531B61FF0E1 for ; Mon, 13 Jul 2026 14:45:06 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 2367221446; Mon, 13 Jul 2026 14:45:06 +0200 (CEST) Message-ID: Date: Mon, 13 Jul 2026 14:44:18 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: David Riley Subject: Re: [PATCH docs v2 12/12] ha-manager: rules: add negative node affinity rule descriptions To: Daniel Kral , pve-devel@lists.proxmox.com References: <20260602100226.180071-1-d.kral@proxmox.com> <20260602100226.180071-13-d.kral@proxmox.com> Content-Language: en-US In-Reply-To: <20260602100226.180071-13-d.kral@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1783946643118 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.236 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_LOW -0.7 Sender listed at https://www.dnswl.org/, low trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: M373BYLPKSS4K6JCMSMZFGXHIZANCQWU X-Message-ID-Hash: M373BYLPKSS4K6JCMSMZFGXHIZANCQWU X-MailFrom: d.riley@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On 6/2/26 12:03 PM, Daniel Kral wrote: > Since the node affinity rules can be either 'positive' or 'negative' > now, add some information about the new negative node affinity rules and > why these are interesting to users in larger cluster setups. > > The following description about node priority classes is only slightly > changed to make the reading flow a little smoother. Additionally, the HA > resource identifiers are adapted as each can only be referenced by a > single node affinity rule. might be nicer to write: "Additionally, HA resource identifiers were updated to reflect that each can only be referenced by a single node affinity rule." but no hard feelings. > Signed-off-by: Daniel Kral > --- > changes since v1: > - new > > ha-manager.adoc | 42 +++++++++++++++++++++++++++++++++++------- > 1 file changed, 35 insertions(+), 7 deletions(-) > > diff --git a/ha-manager.adoc b/ha-manager.adoc > index 802e5b9..369f0ec 100644 > --- a/ha-manager.adoc > +++ b/ha-manager.adoc > @@ -709,6 +709,13 @@ include::generated/ha-rules-opts.adoc[] > | HA Rule Type | Description > | `node-affinity` | Places affinity from one or more HA resources to one or > more nodes. > + > +The affinity `positive` specifies that HA resources should/must be kept on one nit: should or must instead of / but no hard feelings on that. > +of the specified nodes, while the affinity `negative` specifes that the HA typo: specifes/specifies > +resources should/must avoid the specified nodes. > + > +The strictness attribute `strict` specifies whether this is a soft constraint > +("should") or whether tis is a hard constraint ("must"). typo: tis should probably be it > | `resource-affinity` | Places affinity between two or more HA resources. > > The affinity `positive` specifies that HA resources must be kept on the same > @@ -741,11 +748,23 @@ resource `vm:100` to be only on `node1`: > # ha-manager rules set node-affinity ha-rule-vm100 --strict 1 > ---- > > -For bigger clusters or specific use cases, it makes sense to define a more > -detailed failover behavior. For example, the resources `vm:200` and `ct:300` > -should run on `node1`. If `node1` becomes unavailable, the resources should be > -distributed on `node2` and `node3`. If `node2` and `node3` are also > -unavailable, the resources should run on `node4`. > +Furthermore, node affinity rules use positive affinity by default, meaning that > +the preferred cluster nodes must be explicitly specified. However, in larger > +clusters, it is often more practical to define node affinity using an opt-out > +approach to avoid long and verbose lists of cluster nodes. For example, the > +following command defines an HA node affinity rule that makes the HA resource typo: plural resource/resources as we have ct:200 and vm:300 > +`ct:200` and `vm:300` avoid the node `node3`: > + > +---- > +# ha-manager rules add node-affinity ha-rule-negative \ > + --affinity negative --resources ct:200,vm:300 --nodes node3 > +---- > + > +Moreover, specific use cases might require to define a more detailed failover might sound better: specific use cases might require defining a more detailed failover behavior. > +behavior. For example, the resources `vm:400` and `ct:500` should run on > +`node1`. If `node1` becomes unavailable, the resources should be distributed on > +`node2` and `node3`. If `node2` and `node3` are also unavailable, the resources > +should run on `node4`. > > To implement this behavior in a node affinity rule, nodes can be paired with > priorities to order the preference for nodes. If two or more nodes have the same > @@ -755,7 +774,7 @@ last `node4` gets the lowest priority, which can be omitted to default to `0`: > > ---- > # ha-manager rules add node-affinity priority-cascade \ > - --resources vm:200,ct:300 --nodes "node1:2,node2:1,node3:1,node4" > + --resources vm:400,ct:500 --nodes "node1:2,node2:1,node3:1,node4" > ---- > > The above commands create the following rules in the rules configuration file: > @@ -767,9 +786,14 @@ node-affinity: ha-rule-vm100 > resources vm:100 > strict 1 > > +node-affinity: ha-rule-negative > + nodes node3 > + resources ct:200,vm:300 > + affinity negative > + > node-affinity: priority-cascade > nodes node1:2,node2:1,node3:1,node4 > - resources vm:200,ct:300 > + resources vm:400,ct:500 > ---- > > Node Affinity Rule Properties > @@ -896,6 +920,10 @@ Currently, HA rules are checked for the following feasibility tests: > > * An HA resource can only be part of a single HA node affinity rule. > > +* Negative HA node affinity rules cannot specify node priorities. > + > +* Negative HA node affinity rules cannot specify all cluster nodes. > + > * An HA resource affinity rule must have at least two HA resources. > > * A negative HA resource affinity rule cannot specify more HA resources than