* [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules
@ 2025-12-19 13:35 Daniel Kral
2025-12-19 13:35 ` [pve-devel] [RFC PATCH ha-manager 1/2] rules: node affinity: add affinity property to node affinity rules Daniel Kral
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Daniel Kral @ 2025-12-19 13:35 UTC (permalink / raw)
To: pve-devel
For larger HA clusters, specifying the nodes in simple* node affinity
rules as opt-out (negative) instead of opt-in (positive) can make the
rule set easier to follow and implement by users.
* simple = without priority groups
There's no web interface integration yet, because I'm not entirely sure
yet how to integrate it with the concept of priority groups for positive
node affinity rules, which do not make sense in this context as the
specified nodes will be removed from the effective node set.
As the conversion is pretty straightforward, we could even allow users
to convert between positive and negative node affinity rules (e.g. when
switching the affinity type in the web interface?).
Negative node affinity rules can be declared manually in the rules.cfg
now with these two small patches:
node-affinity: ha-rule-name
affinity negative
nodes node1,node2
resources vm:100,vm:101
In a cluster with node1,...,node5 this will result in the identical
positive node affinity rule:
node-affinity: ha-rule-name-dual
affinity positive
nodes node3,node4,node5
resources vm:100,vm:101
Daniel Kral (2):
rules: node affinity: add affinity property to node affinity rules
rules: node affinity: implement negative node affinity rules
src/PVE/HA/Groups.pm | 1 +
src/PVE/HA/HashTools.pm | 20 ++++++
src/PVE/HA/Rules.pm | 2 +
src/PVE/HA/Rules/NodeAffinity.pm | 64 +++++++++++++++++++
.../defaults-for-node-affinity-rules.cfg | 23 +++++--
...efaults-for-node-affinity-rules.cfg.expect | 62 +++++++++++++++++-
...nt-node-resource-affinity-rules.cfg.expect | 3 +
...ositive-resource-affinity-rules.cfg.expect | 4 ++
...ty-with-resource-affinity-rules.cfg.expect | 2 +
...rce-refs-in-node-affinity-rules.cfg.expect | 3 +
10 files changed, 179 insertions(+), 5 deletions(-)
--
2.47.3
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 9+ messages in thread* [pve-devel] [RFC PATCH ha-manager 1/2] rules: node affinity: add affinity property to node affinity rules 2025-12-19 13:35 [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules Daniel Kral @ 2025-12-19 13:35 ` Daniel Kral 2026-02-24 12:22 ` Fiona Ebner 2025-12-19 13:35 ` [pve-devel] [RFC PATCH ha-manager 2/2] rules: node affinity: implement negative " Daniel Kral 2026-02-24 12:22 ` [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules Fiona Ebner 2 siblings, 1 reply; 9+ messages in thread From: Daniel Kral @ 2025-12-19 13:35 UTC (permalink / raw) To: pve-devel This is in preparation of the next patch, which adds negative node affinity rules. Signed-off-by: Daniel Kral <d.kral@proxmox.com> --- src/PVE/HA/Groups.pm | 1 + src/PVE/HA/Rules/NodeAffinity.pm | 9 +++++++++ src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg | 8 ++++---- .../defaults-for-node-affinity-rules.cfg.expect | 6 +++++- .../inconsistent-node-resource-affinity-rules.cfg.expect | 3 +++ ...inity-for-positive-resource-affinity-rules.cfg.expect | 4 ++++ ...node-affinity-with-resource-affinity-rules.cfg.expect | 2 ++ ...tiple-resource-refs-in-node-affinity-rules.cfg.expect | 3 +++ 8 files changed, 31 insertions(+), 5 deletions(-) diff --git a/src/PVE/HA/Groups.pm b/src/PVE/HA/Groups.pm index d2802eee..b23e3414 100644 --- a/src/PVE/HA/Groups.pm +++ b/src/PVE/HA/Groups.pm @@ -147,6 +147,7 @@ sub migrate_groups_to_rules { type => 'node-affinity', resources => $resources, nodes => $nodes, + affinity => 'positive', strict => $groups->{ids}->{$group}->{restricted}, comment => $groups->{ids}->{$group}->{comment}, }; diff --git a/src/PVE/HA/Rules/NodeAffinity.pm b/src/PVE/HA/Rules/NodeAffinity.pm index 3fa1fdb4..1f15ae2d 100644 --- a/src/PVE/HA/Rules/NodeAffinity.pm +++ b/src/PVE/HA/Rules/NodeAffinity.pm @@ -64,6 +64,14 @@ sub properties { optional => 0, }, ), + affinity => { + description => "Describes whether the HA resources are supposed to" + . " be placed on the given nodes ('positive').", + type => 'string', + enum => ['positive'], + default => 'positive', + optional => 1, + }, strict => { description => "Describes whether the node affinity rule is strict or non-strict.", verbose_description => <<EODESC, @@ -86,6 +94,7 @@ sub options { return { resources => { optional => 0 }, nodes => { optional => 0 }, + affinity => { optional => 1 }, strict => { optional => 1 }, disable => { optional => 1 }, comment => { optional => 1 }, diff --git a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg index c8b2f2dd..27658562 100644 --- a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg +++ b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg @@ -1,21 +1,21 @@ -# Case 1: Node Affinity rules are enabled and loose by default, so set it so if it isn't yet. +# Case 1: Node Affinity rules are enabled, positive and loose by default, so set it so if it isn't yet. node-affinity: node-affinity-defaults resources vm:101 nodes node1 -# Case 2: Node Affinity rule is disabled, it shouldn't be enabled afterwards. +# Case 2: Positive Node Affinity rule is disabled, it shouldn't be enabled afterwards. node-affinity: node-affinity-disabled resources vm:102 nodes node2 disable -# Case 3: Node Affinity rule is disabled with explicit 1 set, it shouldn't be enabled afterwards. +# Case 3: Positive Node Affinity rule is disabled with explicit 1 set, it shouldn't be enabled afterwards. node-affinity: node-affinity-disabled-explicit resources vm:103 nodes node2 disable 1 -# Case 4: Node Affinity rule is set to strict, so it shouldn't be loose afterwards. +# Case 4: Positive Node Affinity rule is set to strict, so it shouldn't be loose afterwards. node-affinity: node-affinity-strict resources vm:104 nodes node3 diff --git a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect index 35d061bd..0d6e5605 100644 --- a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect +++ b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect @@ -1,9 +1,10 @@ --- Log --- --- Config --- { - "digest" : "c96c9de143221a82e44efa8bb4814b8248a8ea11", + "digest" : "fcad82def12abc4422061b79cfd0399967053d93", "ids" : { "node-affinity-defaults" : { + "affinity" : "positive", "nodes" : { "node1" : { "priority" : 0 @@ -15,6 +16,7 @@ "type" : "node-affinity" }, "node-affinity-disabled" : { + "affinity" : "positive", "disable" : 1, "nodes" : { "node2" : { @@ -27,6 +29,7 @@ "type" : "node-affinity" }, "node-affinity-disabled-explicit" : { + "affinity" : "positive", "disable" : 1, "nodes" : { "node2" : { @@ -39,6 +42,7 @@ "type" : "node-affinity" }, "node-affinity-strict" : { + "affinity" : "positive", "nodes" : { "node3" : { "priority" : 0 diff --git a/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg.expect b/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg.expect index 4317292b..42dbc004 100644 --- a/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg.expect +++ b/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg.expect @@ -22,6 +22,7 @@ Drop rule 'vm503-must-be-on-node2', because at least one resource is in a positi "type" : "resource-affinity" }, "vm101-vm102-must-be-on-node1" : { + "affinity" : "positive", "nodes" : { "node1" : { "priority" : 0 @@ -35,6 +36,7 @@ Drop rule 'vm503-must-be-on-node2', because at least one resource is in a positi "type" : "node-affinity" }, "vm201-must-be-on-node1" : { + "affinity" : "positive", "nodes" : { "node1" : { "priority" : 0 @@ -55,6 +57,7 @@ Drop rule 'vm503-must-be-on-node2', because at least one resource is in a positi "type" : "resource-affinity" }, "vm202-must-be-on-node2" : { + "affinity" : "positive", "nodes" : { "node2" : { "priority" : 0 diff --git a/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg.expect b/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg.expect index ed339777..db505420 100644 --- a/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg.expect +++ b/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg.expect @@ -4,6 +4,7 @@ "digest" : "32ae135ef2f8bd84cd12c18af6910dce9d6bc9fa", "ids" : { "do-not-infer-negative1" : { + "affinity" : "positive", "nodes" : { "node1" : { "priority" : 0 @@ -19,6 +20,7 @@ "type" : "node-affinity" }, "do-not-infer-negative2" : { + "affinity" : "positive", "nodes" : { "node3" : { "priority" : 0 @@ -47,6 +49,7 @@ "type" : "resource-affinity" }, "infer-multi-resources1" : { + "affinity" : "positive", "nodes" : { "node1" : { "priority" : 0 @@ -76,6 +79,7 @@ "type" : "resource-affinity" }, "infer-single-resource1" : { + "affinity" : "positive", "nodes" : { "node3" : { "priority" : 0 diff --git a/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg.expect b/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg.expect index 68a2b75f..cc6595ac 100644 --- a/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg.expect +++ b/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg.expect @@ -8,6 +8,7 @@ Drop rule 'vm201-vm202-must-be-on-node1-or-node2', because resources are in a re "digest" : "722a98914555f296af0916c980a9d6c780f5f072", "ids" : { "vm301-must-be-on-node1-with-prio-1" : { + "affinity" : "positive", "nodes" : { "node1" : { "priority" : 1 @@ -28,6 +29,7 @@ Drop rule 'vm201-vm202-must-be-on-node1-or-node2', because resources are in a re "type" : "resource-affinity" }, "vm302-must-be-on-node2-with-prio-2" : { + "affinity" : "positive", "nodes" : { "node2" : { "priority" : 2 diff --git a/src/test/rules_cfgs/multiple-resource-refs-in-node-affinity-rules.cfg.expect b/src/test/rules_cfgs/multiple-resource-refs-in-node-affinity-rules.cfg.expect index 425de2b1..2776b56b 100644 --- a/src/test/rules_cfgs/multiple-resource-refs-in-node-affinity-rules.cfg.expect +++ b/src/test/rules_cfgs/multiple-resource-refs-in-node-affinity-rules.cfg.expect @@ -7,6 +7,7 @@ Drop rule 'same-resource3', because resource 'vm:201' is already used in another "digest" : "5865d23b1a342e7f8cfa68bd0e1da556ca8d28a6", "ids" : { "no-same-resource1" : { + "affinity" : "positive", "nodes" : { "node1" : { "priority" : 0 @@ -24,6 +25,7 @@ Drop rule 'same-resource3', because resource 'vm:201' is already used in another "type" : "node-affinity" }, "no-same-resource2" : { + "affinity" : "positive", "nodes" : { "node1" : { "priority" : 0 @@ -40,6 +42,7 @@ Drop rule 'same-resource3', because resource 'vm:201' is already used in another "type" : "node-affinity" }, "no-same-resource3" : { + "affinity" : "positive", "nodes" : { "node1" : { "priority" : 0 -- 2.47.3 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [pve-devel] [RFC PATCH ha-manager 1/2] rules: node affinity: add affinity property to node affinity rules 2025-12-19 13:35 ` [pve-devel] [RFC PATCH ha-manager 1/2] rules: node affinity: add affinity property to node affinity rules Daniel Kral @ 2026-02-24 12:22 ` Fiona Ebner 0 siblings, 0 replies; 9+ messages in thread From: Fiona Ebner @ 2026-02-24 12:22 UTC (permalink / raw) To: Proxmox VE development discussion, Daniel Kral Am 19.12.25 um 2:37 PM schrieb Daniel Kral: > This is in preparation of the next patch, which adds negative node > affinity rules. > > Signed-off-by: Daniel Kral <d.kral@proxmox.com> Reviewed-by: Fiona Ebner <f.ebner@proxmox.com> ^ permalink raw reply [flat|nested] 9+ messages in thread
* [pve-devel] [RFC PATCH ha-manager 2/2] rules: node affinity: implement negative node affinity rules 2025-12-19 13:35 [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules Daniel Kral 2025-12-19 13:35 ` [pve-devel] [RFC PATCH ha-manager 1/2] rules: node affinity: add affinity property to node affinity rules Daniel Kral @ 2025-12-19 13:35 ` Daniel Kral 2026-02-24 12:22 ` Fiona Ebner 2026-02-24 12:22 ` [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules Fiona Ebner 2 siblings, 1 reply; 9+ messages in thread From: Daniel Kral @ 2025-12-19 13:35 UTC (permalink / raw) To: pve-devel Extend the existing node affinity rules plugin to allow users to specify negative node affinity constraints, which specify the nodes where HA resources SHOULD NOT/MUST NOT be placed. Negative node affinity rules are internally represented as positive node affinity rules, where the positive node affinity rules' nodes set is the set complement of the negative node affinity rules' node set. As this is semantically equivalent, this allows no change in the apply logic. As node priority groups do only hold semantic value for positive node affinity rules, add all resulting nodes to the default priority group. Signed-off-by: Daniel Kral <d.kral@proxmox.com> --- src/PVE/HA/HashTools.pm | 20 +++++++ src/PVE/HA/Rules.pm | 2 + src/PVE/HA/Rules/NodeAffinity.pm | 59 ++++++++++++++++++- .../defaults-for-node-affinity-rules.cfg | 15 +++++ ...efaults-for-node-affinity-rules.cfg.expect | 58 +++++++++++++++++- 5 files changed, 151 insertions(+), 3 deletions(-) diff --git a/src/PVE/HA/HashTools.pm b/src/PVE/HA/HashTools.pm index ebe47e38..b6e2136b 100644 --- a/src/PVE/HA/HashTools.pm +++ b/src/PVE/HA/HashTools.pm @@ -6,6 +6,7 @@ use warnings; use base qw(Exporter); our @EXPORT_OK = qw( + set_difference set_intersect set_union sets_are_disjoint @@ -29,6 +30,25 @@ more verbose implementation. =cut +=head3 set_difference($hash1, $hash2) + +Returns a hash set of the set difference between the hash sets C<$hash1> and +C<$hash2>, i.e. the elements that are in C<$hash1> without the elements that +are in C<$hash2>. + +The hashes C<$hash1> and C<$hash2> are expected to be hash sets, i.e. +key-value pairs are always set to C<1> or another truthy value. + +=cut + +sub set_difference : prototype($$) { + my ($hash1, $hash2) = @_; + + my $result = { map { $hash2->{$_} ? () : ($_ => 1) } keys %$hash1 }; + + return $result; +} + =head3 set_intersect($hash1, $hash2) Returns a hash set of the intersection of the hash sets C<$hash1> and diff --git a/src/PVE/HA/Rules.pm b/src/PVE/HA/Rules.pm index c4a2ccea..7f9f428d 100644 --- a/src/PVE/HA/Rules.pm +++ b/src/PVE/HA/Rules.pm @@ -459,6 +459,8 @@ sub transform { for my $transform ($transformdef->{$type}->@*) { my $global_args = $class->get_check_arguments($rules); + $global_args->{nodes} = $nodes; + $transform->($rules, $global_args); } } diff --git a/src/PVE/HA/Rules/NodeAffinity.pm b/src/PVE/HA/Rules/NodeAffinity.pm index 1f15ae2d..cdf67a55 100644 --- a/src/PVE/HA/Rules/NodeAffinity.pm +++ b/src/PVE/HA/Rules/NodeAffinity.pm @@ -9,6 +9,7 @@ use PVE::Cluster; use PVE::JSONSchema qw(get_standard_option); use PVE::Tools; +use PVE::HA::HashTools qw(set_difference); use PVE::HA::Rules; use PVE::HA::Tools; @@ -28,6 +29,22 @@ PVE::HA::Rules::NodeAffinity This package provides the capability to specify and apply rules, which put affinity constraints between a set of HA resources and a set of nodes. +HA Node Affinity rules can be one of two types: + +=over + +=item C<'positive'> + +Positive node affinity rules specify the nodes, which SHOULD/MUST be preferred +by the given HA resources. + +=item C<'negative'> + +Positive node affinity rules specify the nodes, which SHOULD NOT/MUST NOT be +preferred by the given HA resources. + +=back + HA Node Affinity rules can be either C<'non-strict'> or C<'strict'>: =over @@ -66,9 +83,10 @@ sub properties { ), affinity => { description => "Describes whether the HA resources are supposed to" - . " be placed on the given nodes ('positive').", + . " be placed on the given nodes ('positive'), or are supposed" + . " to be placed on any but the given nodes ('negative').", type => 'string', - enum => ['positive'], + enum => ['positive', 'negative'], default => 'positive', optional => 1, }, @@ -256,6 +274,43 @@ __PACKAGE__->register_check( }, ); +=head1 NODE AFFINITY RULE TRANSFORMATION HELPERS + +=cut + +=head3 invert_negative_node_affinity_rules($rules, $node_affinity_rules, $nodes) + +Modifies C<$rules> such that all negative node affinity rules, defined in +C<$node_affinity_rules>, are transformed to positive node affinity rules, where +the nodes set is the complement of the negative node affinity rules' nodes set. + +=cut + +sub invert_negative_node_affinity_rules { + my ($rules, $node_affinity_rules, $nodes) = @_; + + my $cluster_nodes = { map { $_ => 1 } @$nodes }; + + while (my ($node_affinity_id, $node_affinity_rule) = each %$node_affinity_rules) { + next if $node_affinity_rule->{affinity} ne 'negative'; + + my $positive_nodes = { map { $_ => 1 } keys $node_affinity_rule->{nodes}->%* }; + my $new_nodes = set_difference($cluster_nodes, $positive_nodes); + $new_nodes->{$_} = { priority => 0 } for keys %$new_nodes; + + $rules->{ids}->{$node_affinity_id}->{affinity} = 'positive'; + $rules->{ids}->{$node_affinity_id}->{nodes} = $new_nodes; + } +} + +__PACKAGE__->register_transform(sub { + my ($rules, $args) = @_; + + invert_negative_node_affinity_rules( + $rules, $args->{node_affinity_rules}, $args->{nodes}, + ); +}); + =head1 NODE AFFINITY RULE HELPERS =cut diff --git a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg index 27658562..b89d725d 100644 --- a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg +++ b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg @@ -20,3 +20,18 @@ node-affinity: node-affinity-strict resources vm:104 nodes node3 strict 1 + +# Case 5: Non-strict negative node affinity rule is transformed to positive node affinity rule with complement node set +# and specified nodes as fallback internally. +node-affinity: negative-node-affinity-defaults + resources vm:105 + nodes node2 + affinity negative + +# Case 6: Strict negative node affinity rule is transformed to positive node affinity rule with complement node set +# internally. +node-affinity: negative-node-affinity-strict + resources vm:106 + nodes node3 + affinity negative + strict 1 diff --git a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect index 0d6e5605..877f1967 100644 --- a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect +++ b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect @@ -1,8 +1,39 @@ --- Log --- --- Config --- { - "digest" : "fcad82def12abc4422061b79cfd0399967053d93", + "digest" : "308a8b704d8b4dc3fb83d8c97c86133caf735755", "ids" : { + "negative-node-affinity-defaults" : { + "affinity" : "positive", + "nodes" : { + "node1" : { + "priority" : 0 + }, + "node3" : { + "priority" : 0 + } + }, + "resources" : { + "vm:105" : 1 + }, + "type" : "node-affinity" + }, + "negative-node-affinity-strict" : { + "affinity" : "positive", + "nodes" : { + "node1" : { + "priority" : 0 + }, + "node2" : { + "priority" : 0 + } + }, + "resources" : { + "vm:106" : 1 + }, + "strict" : 1, + "type" : "node-affinity" + }, "node-affinity-defaults" : { "affinity" : "positive", "nodes" : { @@ -56,6 +87,8 @@ } }, "order" : { + "negative-node-affinity-defaults" : 5, + "negative-node-affinity-strict" : 6, "node-affinity-defaults" : 1, "node-affinity-disabled" : 2, "node-affinity-disabled-explicit" : 3, @@ -84,6 +117,29 @@ "priority" : 0 } } + }, + "vm:105" : { + "nodes" : { + "node1" : { + "priority" : 0 + }, + "node2" : { + "priority" : -1 + }, + "node3" : { + "priority" : 0 + } + } + }, + "vm:106" : { + "nodes" : { + "node1" : { + "priority" : 0 + }, + "node2" : { + "priority" : 0 + } + } } }, "resource-affinity" : { -- 2.47.3 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [pve-devel] [RFC PATCH ha-manager 2/2] rules: node affinity: implement negative node affinity rules 2025-12-19 13:35 ` [pve-devel] [RFC PATCH ha-manager 2/2] rules: node affinity: implement negative " Daniel Kral @ 2026-02-24 12:22 ` Fiona Ebner 2026-02-25 8:32 ` Daniel Kral 0 siblings, 1 reply; 9+ messages in thread From: Fiona Ebner @ 2026-02-24 12:22 UTC (permalink / raw) To: Proxmox VE development discussion, Daniel Kral Am 19.12.25 um 2:36 PM schrieb Daniel Kral: > Extend the existing node affinity rules plugin to allow users to specify > negative node affinity constraints, which specify the nodes where HA > resources SHOULD NOT/MUST NOT be placed. > > Negative node affinity rules are internally represented as positive node > affinity rules, where the positive node affinity rules' nodes set is the > set complement of the negative node affinity rules' node set. As this is > semantically equivalent, this allows no change in the apply logic. Nit: 'allows making no change' > As node priority groups do only hold semantic value for positive node > affinity rules, add all resulting nodes to the default priority group. > > Signed-off-by: Daniel Kral <d.kral@proxmox.com> [I] root@pve9a1 ~# pvesh create /cluster/ha/rules/ --rule ha-rule-new --resources ct:105 --type node-affinity --affinity negative --nodes pve9a1:1,pve9a2:2 This currently goes through, but should be rejected, indicating that node priorities may not be specified with negative affinity. Two other smaller comments below, otherwise it looks good to me. > --- > src/PVE/HA/HashTools.pm | 20 +++++++ > src/PVE/HA/Rules.pm | 2 + > src/PVE/HA/Rules/NodeAffinity.pm | 59 ++++++++++++++++++- > .../defaults-for-node-affinity-rules.cfg | 15 +++++ > ...efaults-for-node-affinity-rules.cfg.expect | 58 +++++++++++++++++- > 5 files changed, 151 insertions(+), 3 deletions(-) > > diff --git a/src/PVE/HA/HashTools.pm b/src/PVE/HA/HashTools.pm > index ebe47e38..b6e2136b 100644 > --- a/src/PVE/HA/HashTools.pm > +++ b/src/PVE/HA/HashTools.pm > @@ -6,6 +6,7 @@ use warnings; > use base qw(Exporter); > > our @EXPORT_OK = qw( > + set_difference > set_intersect > set_union > sets_are_disjoint > @@ -29,6 +30,25 @@ more verbose implementation. > > =cut > > +=head3 set_difference($hash1, $hash2) > + > +Returns a hash set of the set difference between the hash sets C<$hash1> and > +C<$hash2>, i.e. the elements that are in C<$hash1> without the elements that > +are in C<$hash2>. > + > +The hashes C<$hash1> and C<$hash2> are expected to be hash sets, i.e. > +key-value pairs are always set to C<1> or another truthy value. > + > +=cut > + > +sub set_difference : prototype($$) { > + my ($hash1, $hash2) = @_; > + > + my $result = { map { $hash2->{$_} ? () : ($_ => 1) } keys %$hash1 }; > + > + return $result; > +} > + > =head3 set_intersect($hash1, $hash2) > > Returns a hash set of the intersection of the hash sets C<$hash1> and > diff --git a/src/PVE/HA/Rules.pm b/src/PVE/HA/Rules.pm > index c4a2ccea..7f9f428d 100644 > --- a/src/PVE/HA/Rules.pm > +++ b/src/PVE/HA/Rules.pm > @@ -459,6 +459,8 @@ sub transform { > for my $transform ($transformdef->{$type}->@*) { > my $global_args = $class->get_check_arguments($rules); > > + $global_args->{nodes} = $nodes; Maybe 'cluster-nodes' as the key to be more explicit? > + > $transform->($rules, $global_args); > } > } > diff --git a/src/PVE/HA/Rules/NodeAffinity.pm b/src/PVE/HA/Rules/NodeAffinity.pm > index 1f15ae2d..cdf67a55 100644 > --- a/src/PVE/HA/Rules/NodeAffinity.pm > +++ b/src/PVE/HA/Rules/NodeAffinity.pm > @@ -9,6 +9,7 @@ use PVE::Cluster; > use PVE::JSONSchema qw(get_standard_option); > use PVE::Tools; > > +use PVE::HA::HashTools qw(set_difference); > use PVE::HA::Rules; > use PVE::HA::Tools; > > @@ -28,6 +29,22 @@ PVE::HA::Rules::NodeAffinity > This package provides the capability to specify and apply rules, which put > affinity constraints between a set of HA resources and a set of nodes. > > +HA Node Affinity rules can be one of two types: > + > +=over > + > +=item C<'positive'> > + > +Positive node affinity rules specify the nodes, which SHOULD/MUST be preferred > +by the given HA resources. > + > +=item C<'negative'> > + > +Positive node affinity rules specify the nodes, which SHOULD NOT/MUST NOT be > +preferred by the given HA resources. > + > +=back > + > HA Node Affinity rules can be either C<'non-strict'> or C<'strict'>: > > =over > @@ -66,9 +83,10 @@ sub properties { > ), > affinity => { > description => "Describes whether the HA resources are supposed to" > - . " be placed on the given nodes ('positive').", > + . " be placed on the given nodes ('positive'), or are supposed" > + . " to be placed on any but the given nodes ('negative').", > type => 'string', > - enum => ['positive'], > + enum => ['positive', 'negative'], > default => 'positive', > optional => 1, > }, > @@ -256,6 +274,43 @@ __PACKAGE__->register_check( > }, > ); > > +=head1 NODE AFFINITY RULE TRANSFORMATION HELPERS > + > +=cut > + > +=head3 invert_negative_node_affinity_rules($rules, $node_affinity_rules, $nodes) > + > +Modifies C<$rules> such that all negative node affinity rules, defined in > +C<$node_affinity_rules>, are transformed to positive node affinity rules, where > +the nodes set is the complement of the negative node affinity rules' nodes set. > + > +=cut > + > +sub invert_negative_node_affinity_rules { > + my ($rules, $node_affinity_rules, $nodes) = @_; > + > + my $cluster_nodes = { map { $_ => 1 } @$nodes }; > + > + while (my ($node_affinity_id, $node_affinity_rule) = each %$node_affinity_rules) { > + next if $node_affinity_rule->{affinity} ne 'negative'; > + > + my $positive_nodes = { map { $_ => 1 } keys $node_affinity_rule->{nodes}->%* }; I'm confused by the variable name. There is negative affinity towards these nodes, so why $positive_nodes? > + my $new_nodes = set_difference($cluster_nodes, $positive_nodes); > + $new_nodes->{$_} = { priority => 0 } for keys %$new_nodes; > + > + $rules->{ids}->{$node_affinity_id}->{affinity} = 'positive'; > + $rules->{ids}->{$node_affinity_id}->{nodes} = $new_nodes; > + } > +} > + > +__PACKAGE__->register_transform(sub { > + my ($rules, $args) = @_; > + > + invert_negative_node_affinity_rules( > + $rules, $args->{node_affinity_rules}, $args->{nodes}, > + ); > +}); > + > =head1 NODE AFFINITY RULE HELPERS > > =cut ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [pve-devel] [RFC PATCH ha-manager 2/2] rules: node affinity: implement negative node affinity rules 2026-02-24 12:22 ` Fiona Ebner @ 2026-02-25 8:32 ` Daniel Kral 0 siblings, 0 replies; 9+ messages in thread From: Daniel Kral @ 2026-02-25 8:32 UTC (permalink / raw) To: Fiona Ebner, Proxmox VE development discussion Thanks for the review and testing! Will send a v2 with the changes and the ui patches as well. On Tue Feb 24, 2026 at 1:22 PM CET, Fiona Ebner wrote: > Am 19.12.25 um 2:36 PM schrieb Daniel Kral: >> Extend the existing node affinity rules plugin to allow users to specify >> negative node affinity constraints, which specify the nodes where HA >> resources SHOULD NOT/MUST NOT be placed. >> >> Negative node affinity rules are internally represented as positive node >> affinity rules, where the positive node affinity rules' nodes set is the >> set complement of the negative node affinity rules' node set. As this is >> semantically equivalent, this allows no change in the apply logic. > > Nit: 'allows making no change' Ack! > >> As node priority groups do only hold semantic value for positive node >> affinity rules, add all resulting nodes to the default priority group. >> >> Signed-off-by: Daniel Kral <d.kral@proxmox.com> > > [I] root@pve9a1 ~# pvesh create /cluster/ha/rules/ --rule ha-rule-new > --resources ct:105 --type node-affinity --affinity negative --nodes > pve9a1:1,pve9a2:2 > > This currently goes through, but should be rejected, indicating that > node priorities may not be specified with negative affinity. > > Two other smaller comments below, otherwise it looks good to me. Right, those should be rejected entirely, ack! > [ snip ] >> diff --git a/src/PVE/HA/Rules.pm b/src/PVE/HA/Rules.pm >> index c4a2ccea..7f9f428d 100644 >> --- a/src/PVE/HA/Rules.pm >> +++ b/src/PVE/HA/Rules.pm >> @@ -459,6 +459,8 @@ sub transform { >> for my $transform ($transformdef->{$type}->@*) { >> my $global_args = $class->get_check_arguments($rules); >> >> + $global_args->{nodes} = $nodes; > > Maybe 'cluster-nodes' as the key to be more explicit? Good idea, I'll do that for the v2! > >> + >> $transform->($rules, $global_args); >> } >> } >> diff --git a/src/PVE/HA/Rules/NodeAffinity.pm b/src/PVE/HA/Rules/NodeAffinity.pm >> index 1f15ae2d..cdf67a55 100644 [ snip ] >> @@ -256,6 +274,43 @@ __PACKAGE__->register_check( >> }, >> ); >> >> +=head1 NODE AFFINITY RULE TRANSFORMATION HELPERS >> + >> +=cut >> + >> +=head3 invert_negative_node_affinity_rules($rules, $node_affinity_rules, $nodes) >> + >> +Modifies C<$rules> such that all negative node affinity rules, defined in >> +C<$node_affinity_rules>, are transformed to positive node affinity rules, where >> +the nodes set is the complement of the negative node affinity rules' nodes set. >> + >> +=cut >> + >> +sub invert_negative_node_affinity_rules { >> + my ($rules, $node_affinity_rules, $nodes) = @_; >> + >> + my $cluster_nodes = { map { $_ => 1 } @$nodes }; >> + >> + while (my ($node_affinity_id, $node_affinity_rule) = each %$node_affinity_rules) { >> + next if $node_affinity_rule->{affinity} ne 'negative'; >> + >> + my $positive_nodes = { map { $_ => 1 } keys $node_affinity_rule->{nodes}->%* }; > > I'm confused by the variable name. There is negative affinity towards > these nodes, so why $positive_nodes? Right, I'm sorry, this should be $negative_nodes instead, this slipped right past me. Will change the name in a v2! If it makes things more clearer, $new_nodes could then take the name of $positive_nodes. > >> + my $new_nodes = set_difference($cluster_nodes, $positive_nodes); >> + $new_nodes->{$_} = { priority => 0 } for keys %$new_nodes; >> + >> + $rules->{ids}->{$node_affinity_id}->{affinity} = 'positive'; >> + $rules->{ids}->{$node_affinity_id}->{nodes} = $new_nodes; >> + } >> +} >> + >> +__PACKAGE__->register_transform(sub { >> + my ($rules, $args) = @_; >> + >> + invert_negative_node_affinity_rules( >> + $rules, $args->{node_affinity_rules}, $args->{nodes}, >> + ); >> +}); >> + >> =head1 NODE AFFINITY RULE HELPERS >> >> =cut ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules 2025-12-19 13:35 [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules Daniel Kral 2025-12-19 13:35 ` [pve-devel] [RFC PATCH ha-manager 1/2] rules: node affinity: add affinity property to node affinity rules Daniel Kral 2025-12-19 13:35 ` [pve-devel] [RFC PATCH ha-manager 2/2] rules: node affinity: implement negative " Daniel Kral @ 2026-02-24 12:22 ` Fiona Ebner 2026-02-25 8:46 ` Daniel Kral 2 siblings, 1 reply; 9+ messages in thread From: Fiona Ebner @ 2026-02-24 12:22 UTC (permalink / raw) To: Proxmox VE development discussion, Daniel Kral Am 19.12.25 um 2:36 PM schrieb Daniel Kral: > For larger HA clusters, specifying the nodes in simple* node affinity > rules as opt-out (negative) instead of opt-in (positive) can make the > rule set easier to follow and implement by users. > > * simple = without priority groups > > > There's no web interface integration yet, because I'm not entirely sure > yet how to integrate it with the concept of priority groups for positive > node affinity rules, which do not make sense in this context as the > specified nodes will be removed from the effective node set. Wouldn't it be enough to not use/show the priority column when the affinity is negative? If people need both, to exclude certain nodes and to prioritize certain others, they can use two rules: 1. a negative node affinity rule 2. a non-strict positive node affinity rule with priorities > As the conversion is pretty straightforward, we could even allow users > to convert between positive and negative node affinity rules (e.g. when > switching the affinity type in the web interface?). Limited to those without priorities I suppose ;) ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules 2026-02-24 12:22 ` [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules Fiona Ebner @ 2026-02-25 8:46 ` Daniel Kral 2026-02-25 8:58 ` Fiona Ebner 0 siblings, 1 reply; 9+ messages in thread From: Daniel Kral @ 2026-02-25 8:46 UTC (permalink / raw) To: Fiona Ebner, Proxmox VE development discussion On Tue Feb 24, 2026 at 1:22 PM CET, Fiona Ebner wrote: > Am 19.12.25 um 2:36 PM schrieb Daniel Kral: >> For larger HA clusters, specifying the nodes in simple* node affinity >> rules as opt-out (negative) instead of opt-in (positive) can make the >> rule set easier to follow and implement by users. >> >> * simple = without priority groups >> >> >> There's no web interface integration yet, because I'm not entirely sure >> yet how to integrate it with the concept of priority groups for positive >> node affinity rules, which do not make sense in this context as the >> specified nodes will be removed from the effective node set. > > Wouldn't it be enough to not use/show the priority column when the > affinity is negative? > > If people need both, to exclude certain nodes and to prioritize certain > others, they can use two rules: > 1. a negative node affinity rule > 2. a non-strict positive node affinity rule with priorities Yes, this would only need some changes in the checks for node affinity rules to allow both types for a HA resource at once and a check that they do not contradict one another. I'm only contemplating about whether this could bite us later on if we allow those rule sets and should wait for the demand for this. >> As the conversion is pretty straightforward, we could even allow users >> to convert between positive and negative node affinity rules (e.g. when >> switching the affinity type in the web interface?). > > Limited to those without priorities I suppose ;) Right :). I'll see what makes sense in the web interface then, but it would probably be the easiest to restrict changing the type of the node affinity rule after it was created... Other options would be to either just reset the node selection or drop the node priorities and invert the selection if users change from 'positive' to 'negative'. ^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules 2026-02-25 8:46 ` Daniel Kral @ 2026-02-25 8:58 ` Fiona Ebner 0 siblings, 0 replies; 9+ messages in thread From: Fiona Ebner @ 2026-02-25 8:58 UTC (permalink / raw) To: Daniel Kral, Proxmox VE development discussion Am 25.02.26 um 9:45 AM schrieb Daniel Kral: > On Tue Feb 24, 2026 at 1:22 PM CET, Fiona Ebner wrote: >> Am 19.12.25 um 2:36 PM schrieb Daniel Kral: >>> For larger HA clusters, specifying the nodes in simple* node affinity >>> rules as opt-out (negative) instead of opt-in (positive) can make the >>> rule set easier to follow and implement by users. >>> >>> * simple = without priority groups >>> >>> >>> There's no web interface integration yet, because I'm not entirely sure >>> yet how to integrate it with the concept of priority groups for positive >>> node affinity rules, which do not make sense in this context as the >>> specified nodes will be removed from the effective node set. >> >> Wouldn't it be enough to not use/show the priority column when the >> affinity is negative? >> >> If people need both, to exclude certain nodes and to prioritize certain >> others, they can use two rules: >> 1. a negative node affinity rule >> 2. a non-strict positive node affinity rule with priorities > > Yes, this would only need some changes in the checks for node affinity > rules to allow both types for a HA resource at once and a check that > they do not contradict one another. > > I'm only contemplating about whether this could bite us later on if we > allow those rule sets and should wait for the demand for this. Waiting for demand is fine by me :) >>> As the conversion is pretty straightforward, we could even allow users >>> to convert between positive and negative node affinity rules (e.g. when >>> switching the affinity type in the web interface?). >> >> Limited to those without priorities I suppose ;) > > Right :). > > I'll see what makes sense in the web interface then, but it would > probably be the easiest to restrict changing the type of the node > affinity rule after it was created... Other options would be to either > just reset the node selection or drop the node priorities and invert the > selection if users change from 'positive' to 'negative'. Here too. ^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2026-02-25 8:57 UTC | newest] Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2025-12-19 13:35 [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules Daniel Kral 2025-12-19 13:35 ` [pve-devel] [RFC PATCH ha-manager 1/2] rules: node affinity: add affinity property to node affinity rules Daniel Kral 2026-02-24 12:22 ` Fiona Ebner 2025-12-19 13:35 ` [pve-devel] [RFC PATCH ha-manager 2/2] rules: node affinity: implement negative " Daniel Kral 2026-02-24 12:22 ` Fiona Ebner 2026-02-25 8:32 ` Daniel Kral 2026-02-24 12:22 ` [pve-devel] [RFC PATCH-SERIES ha-manager 0/2] Negative Node Affinity Rules Fiona Ebner 2026-02-25 8:46 ` Daniel Kral 2026-02-25 8:58 ` Fiona Ebner
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.