From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 984BE1FF187 for ; Fri, 19 Dec 2025 14:36:13 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 16CD1CA38; Fri, 19 Dec 2025 14:36:51 +0100 (CET) From: Daniel Kral To: pve-devel@lists.proxmox.com Date: Fri, 19 Dec 2025 14:35:47 +0100 Message-ID: <20251219133643.295514-3-d.kral@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20251219133643.295514-1-d.kral@proxmox.com> References: <20251219133643.295514-1-d.kral@proxmox.com> MIME-Version: 1.0 X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1766151393685 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.014 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_VALIDITY_CERTIFIED_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_RPBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. RCVD_IN_VALIDITY_SAFE_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to Validity was blocked. See https://knowledge.validity.com/hc/en-us/articles/20961730681243 for more information. SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [hashtools.pm, nodeaffinity.pm, rules.pm] Subject: [pve-devel] [RFC PATCH ha-manager 2/2] rules: node affinity: implement negative node affinity rules X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" Extend the existing node affinity rules plugin to allow users to specify negative node affinity constraints, which specify the nodes where HA resources SHOULD NOT/MUST NOT be placed. Negative node affinity rules are internally represented as positive node affinity rules, where the positive node affinity rules' nodes set is the set complement of the negative node affinity rules' node set. As this is semantically equivalent, this allows no change in the apply logic. As node priority groups do only hold semantic value for positive node affinity rules, add all resulting nodes to the default priority group. Signed-off-by: Daniel Kral --- src/PVE/HA/HashTools.pm | 20 +++++++ src/PVE/HA/Rules.pm | 2 + src/PVE/HA/Rules/NodeAffinity.pm | 59 ++++++++++++++++++- .../defaults-for-node-affinity-rules.cfg | 15 +++++ ...efaults-for-node-affinity-rules.cfg.expect | 58 +++++++++++++++++- 5 files changed, 151 insertions(+), 3 deletions(-) diff --git a/src/PVE/HA/HashTools.pm b/src/PVE/HA/HashTools.pm index ebe47e38..b6e2136b 100644 --- a/src/PVE/HA/HashTools.pm +++ b/src/PVE/HA/HashTools.pm @@ -6,6 +6,7 @@ use warnings; use base qw(Exporter); our @EXPORT_OK = qw( + set_difference set_intersect set_union sets_are_disjoint @@ -29,6 +30,25 @@ more verbose implementation. =cut +=head3 set_difference($hash1, $hash2) + +Returns a hash set of the set difference between the hash sets C<$hash1> and +C<$hash2>, i.e. the elements that are in C<$hash1> without the elements that +are in C<$hash2>. + +The hashes C<$hash1> and C<$hash2> are expected to be hash sets, i.e. +key-value pairs are always set to C<1> or another truthy value. + +=cut + +sub set_difference : prototype($$) { + my ($hash1, $hash2) = @_; + + my $result = { map { $hash2->{$_} ? () : ($_ => 1) } keys %$hash1 }; + + return $result; +} + =head3 set_intersect($hash1, $hash2) Returns a hash set of the intersection of the hash sets C<$hash1> and diff --git a/src/PVE/HA/Rules.pm b/src/PVE/HA/Rules.pm index c4a2ccea..7f9f428d 100644 --- a/src/PVE/HA/Rules.pm +++ b/src/PVE/HA/Rules.pm @@ -459,6 +459,8 @@ sub transform { for my $transform ($transformdef->{$type}->@*) { my $global_args = $class->get_check_arguments($rules); + $global_args->{nodes} = $nodes; + $transform->($rules, $global_args); } } diff --git a/src/PVE/HA/Rules/NodeAffinity.pm b/src/PVE/HA/Rules/NodeAffinity.pm index 1f15ae2d..cdf67a55 100644 --- a/src/PVE/HA/Rules/NodeAffinity.pm +++ b/src/PVE/HA/Rules/NodeAffinity.pm @@ -9,6 +9,7 @@ use PVE::Cluster; use PVE::JSONSchema qw(get_standard_option); use PVE::Tools; +use PVE::HA::HashTools qw(set_difference); use PVE::HA::Rules; use PVE::HA::Tools; @@ -28,6 +29,22 @@ PVE::HA::Rules::NodeAffinity This package provides the capability to specify and apply rules, which put affinity constraints between a set of HA resources and a set of nodes. +HA Node Affinity rules can be one of two types: + +=over + +=item C<'positive'> + +Positive node affinity rules specify the nodes, which SHOULD/MUST be preferred +by the given HA resources. + +=item C<'negative'> + +Positive node affinity rules specify the nodes, which SHOULD NOT/MUST NOT be +preferred by the given HA resources. + +=back + HA Node Affinity rules can be either C<'non-strict'> or C<'strict'>: =over @@ -66,9 +83,10 @@ sub properties { ), affinity => { description => "Describes whether the HA resources are supposed to" - . " be placed on the given nodes ('positive').", + . " be placed on the given nodes ('positive'), or are supposed" + . " to be placed on any but the given nodes ('negative').", type => 'string', - enum => ['positive'], + enum => ['positive', 'negative'], default => 'positive', optional => 1, }, @@ -256,6 +274,43 @@ __PACKAGE__->register_check( }, ); +=head1 NODE AFFINITY RULE TRANSFORMATION HELPERS + +=cut + +=head3 invert_negative_node_affinity_rules($rules, $node_affinity_rules, $nodes) + +Modifies C<$rules> such that all negative node affinity rules, defined in +C<$node_affinity_rules>, are transformed to positive node affinity rules, where +the nodes set is the complement of the negative node affinity rules' nodes set. + +=cut + +sub invert_negative_node_affinity_rules { + my ($rules, $node_affinity_rules, $nodes) = @_; + + my $cluster_nodes = { map { $_ => 1 } @$nodes }; + + while (my ($node_affinity_id, $node_affinity_rule) = each %$node_affinity_rules) { + next if $node_affinity_rule->{affinity} ne 'negative'; + + my $positive_nodes = { map { $_ => 1 } keys $node_affinity_rule->{nodes}->%* }; + my $new_nodes = set_difference($cluster_nodes, $positive_nodes); + $new_nodes->{$_} = { priority => 0 } for keys %$new_nodes; + + $rules->{ids}->{$node_affinity_id}->{affinity} = 'positive'; + $rules->{ids}->{$node_affinity_id}->{nodes} = $new_nodes; + } +} + +__PACKAGE__->register_transform(sub { + my ($rules, $args) = @_; + + invert_negative_node_affinity_rules( + $rules, $args->{node_affinity_rules}, $args->{nodes}, + ); +}); + =head1 NODE AFFINITY RULE HELPERS =cut diff --git a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg index 27658562..b89d725d 100644 --- a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg +++ b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg @@ -20,3 +20,18 @@ node-affinity: node-affinity-strict resources vm:104 nodes node3 strict 1 + +# Case 5: Non-strict negative node affinity rule is transformed to positive node affinity rule with complement node set +# and specified nodes as fallback internally. +node-affinity: negative-node-affinity-defaults + resources vm:105 + nodes node2 + affinity negative + +# Case 6: Strict negative node affinity rule is transformed to positive node affinity rule with complement node set +# internally. +node-affinity: negative-node-affinity-strict + resources vm:106 + nodes node3 + affinity negative + strict 1 diff --git a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect index 0d6e5605..877f1967 100644 --- a/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect +++ b/src/test/rules_cfgs/defaults-for-node-affinity-rules.cfg.expect @@ -1,8 +1,39 @@ --- Log --- --- Config --- { - "digest" : "fcad82def12abc4422061b79cfd0399967053d93", + "digest" : "308a8b704d8b4dc3fb83d8c97c86133caf735755", "ids" : { + "negative-node-affinity-defaults" : { + "affinity" : "positive", + "nodes" : { + "node1" : { + "priority" : 0 + }, + "node3" : { + "priority" : 0 + } + }, + "resources" : { + "vm:105" : 1 + }, + "type" : "node-affinity" + }, + "negative-node-affinity-strict" : { + "affinity" : "positive", + "nodes" : { + "node1" : { + "priority" : 0 + }, + "node2" : { + "priority" : 0 + } + }, + "resources" : { + "vm:106" : 1 + }, + "strict" : 1, + "type" : "node-affinity" + }, "node-affinity-defaults" : { "affinity" : "positive", "nodes" : { @@ -56,6 +87,8 @@ } }, "order" : { + "negative-node-affinity-defaults" : 5, + "negative-node-affinity-strict" : 6, "node-affinity-defaults" : 1, "node-affinity-disabled" : 2, "node-affinity-disabled-explicit" : 3, @@ -84,6 +117,29 @@ "priority" : 0 } } + }, + "vm:105" : { + "nodes" : { + "node1" : { + "priority" : 0 + }, + "node2" : { + "priority" : -1 + }, + "node3" : { + "priority" : 0 + } + } + }, + "vm:106" : { + "nodes" : { + "node1" : { + "priority" : 0 + }, + "node2" : { + "priority" : 0 + } + } } }, "resource-affinity" : { -- 2.47.3 _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel