From: Daniel Kral <d.kral@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH ha-manager v2 11/12] test: rules: add test cases for inter-plugin checks allowing simple use cases
Date: Fri, 1 Aug 2025 18:22:26 +0200 [thread overview]
Message-ID: <20250801162230.296214-12-d.kral@proxmox.com> (raw)
In-Reply-To: <20250801162230.296214-1-d.kral@proxmox.com>
Add test cases to verify that the rule checkers correctly identify and
remove HA node and resource affinity rules from the rules to make the
rule set feasible. The added test cases verify:
- the resources of a resource affinity rule are not part of any node
affinity rule, which has multiple priority groups. This is because of
the dynamic nature of priority groups.
- the resources of a positive resource affinity rule are part of at most
one node affinity rule, but no more. Otherwise, it is not easily
decidable (yet) what the common node restrictions are.
- the positive resource affinity rules, which have at least one resource
which is part of one node affinity rule, make all the resources part
of the node affinity rule.
- the resources of a negative resource affinity rule are not restricted
by their node affinity rules in such a way that these do not have
enough nodes to be separated on.
Signed-off-by: Daniel Kral <d.kral@proxmox.com>
---
...onsistent-node-resource-affinity-rules.cfg | 54 +++++++++
...nt-node-resource-affinity-rules.cfg.expect | 73 ++++++++++++
...y-for-positive-resource-affinity-rules.cfg | 37 ++++++
...ositive-resource-affinity-rules.cfg.expect | 111 ++++++++++++++++++
...-affinity-with-resource-affinity-rules.cfg | 35 ++++++
...ty-with-resource-affinity-rules.cfg.expect | 48 ++++++++
6 files changed, 358 insertions(+)
create mode 100644 src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg
create mode 100644 src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg.expect
create mode 100644 src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg
create mode 100644 src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg.expect
create mode 100644 src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg
create mode 100644 src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg.expect
diff --git a/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg b/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg
new file mode 100644
index 00000000..88e6dd0e
--- /dev/null
+++ b/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg
@@ -0,0 +1,54 @@
+# Case 1: Do not remove a positive resource affinity rule, where there is exactly one node to keep them together.
+node-affinity: vm101-vm102-must-be-on-node1
+ resources vm:101,vm:102
+ nodes node1
+ strict 1
+
+resource-affinity: vm101-vm102-must-be-kept-together
+ resources vm:101,vm:102
+ affinity positive
+
+# Case 2: Do not remove a negative resource affinity rule, where there are exactly enough nodes available to keep them apart.
+node-affinity: vm201-must-be-on-node1
+ resources vm:201
+ nodes node1
+ strict 1
+
+node-affinity: vm202-must-be-on-node2
+ resources vm:202
+ nodes node2
+ strict 1
+
+resource-affinity: vm201-vm202-must-be-kept-separate
+ resources vm:201,vm:202
+ affinity negative
+
+# Case 3: Remove positive resource affinity rules, where two resources are restricted to different nodes.
+node-affinity: vm301-must-be-on-node1
+ resources vm:301
+ nodes node1
+ strict 1
+
+node-affinity: vm301-must-be-on-node2
+ resources vm:302
+ nodes node2
+ strict 1
+
+resource-affinity: vm301-vm302-must-be-kept-together
+ resources vm:301,vm:302
+ affinity positive
+
+# Case 4: Remove negative resource affinity rules, where two resources are restricted to less nodes than needed to keep them apart.
+node-affinity: vm401-must-be-on-node1
+ resources vm:401
+ nodes node1
+ strict 1
+
+node-affinity: vm402-must-be-on-node1
+ resources vm:402
+ nodes node1
+ strict 1
+
+resource-affinity: vm401-vm402-must-be-kept-separate
+ resources vm:401,vm:402
+ affinity negative
diff --git a/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg.expect b/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg.expect
new file mode 100644
index 00000000..e12242ab
--- /dev/null
+++ b/src/test/rules_cfgs/inconsistent-node-resource-affinity-rules.cfg.expect
@@ -0,0 +1,73 @@
+--- Log ---
+Drop rule 'vm301-must-be-on-node1', because at least one resource is in a positive resource affinity rule and there are other resources in at least one other node affinity rule already.
+Drop rule 'vm301-must-be-on-node2', because at least one resource is in a positive resource affinity rule and there are other resources in at least one other node affinity rule already.
+Drop rule 'vm301-vm302-must-be-kept-together', because resources are in multiple node affinity rules.
+Drop rule 'vm401-must-be-on-node1', because at least one resource is in a negative resource affinity rule and this rule would restrict these to less nodes than available to the resources.
+Drop rule 'vm401-vm402-must-be-kept-separate', because two or more resources are restricted to less nodes than available to the resources.
+Drop rule 'vm402-must-be-on-node1', because at least one resource is in a negative resource affinity rule and this rule would restrict these to less nodes than available to the resources.
+--- Config ---
+$VAR1 = {
+ 'digest' => 'a5d782a442bbe3bf3a4d088db82a575b382a53fe',
+ 'ids' => {
+ 'vm101-vm102-must-be-kept-together' => {
+ 'affinity' => 'positive',
+ 'resources' => {
+ 'vm:101' => 1,
+ 'vm:102' => 1
+ },
+ 'type' => 'resource-affinity'
+ },
+ 'vm101-vm102-must-be-on-node1' => {
+ 'nodes' => {
+ 'node1' => {
+ 'priority' => 0
+ }
+ },
+ 'resources' => {
+ 'vm:101' => 1,
+ 'vm:102' => 1
+ },
+ 'strict' => 1,
+ 'type' => 'node-affinity'
+ },
+ 'vm201-must-be-on-node1' => {
+ 'nodes' => {
+ 'node1' => {
+ 'priority' => 0
+ }
+ },
+ 'resources' => {
+ 'vm:201' => 1
+ },
+ 'strict' => 1,
+ 'type' => 'node-affinity'
+ },
+ 'vm201-vm202-must-be-kept-separate' => {
+ 'affinity' => 'negative',
+ 'resources' => {
+ 'vm:201' => 1,
+ 'vm:202' => 1
+ },
+ 'type' => 'resource-affinity'
+ },
+ 'vm202-must-be-on-node2' => {
+ 'nodes' => {
+ 'node2' => {
+ 'priority' => 0
+ }
+ },
+ 'resources' => {
+ 'vm:202' => 1
+ },
+ 'strict' => 1,
+ 'type' => 'node-affinity'
+ }
+ },
+ 'order' => {
+ 'vm101-vm102-must-be-kept-together' => 2,
+ 'vm101-vm102-must-be-on-node1' => 1,
+ 'vm201-must-be-on-node1' => 3,
+ 'vm201-vm202-must-be-kept-separate' => 5,
+ 'vm202-must-be-on-node2' => 4
+ }
+ };
diff --git a/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg b/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg
new file mode 100644
index 00000000..ebbf5e63
--- /dev/null
+++ b/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg
@@ -0,0 +1,37 @@
+# Case 1: Do not change any node affinity rules, if there are no resources of a positive resource affinity rule in any node affinity rules.
+resource-affinity: do-not-infer-positive1
+ resources vm:101,vm:102,vm:103
+ affinity positive
+
+# Case 2: Do not change any node affinity rules for node affinity rules of resources in a negative resource affinity rule.
+node-affinity: do-not-infer-negative1
+ resources vm:203
+ nodes node1,node2
+ strict 1
+
+node-affinity: do-not-infer-negative2
+ resources vm:201
+ nodes node3
+
+resource-affinity: do-not-infer-negative3
+ resources vm:201,vm:203
+ affinity negative
+
+# Case 2: Add two resources, which are not part of the node affinity rule of another resource in a positive resource affinity rule, to the node affinity rule.
+node-affinity: infer-single-resource1
+ resources vm:302
+ nodes node3
+
+resource-affinity: infer-single-resource2
+ resources vm:301,vm:302,vm:303
+ affinity positive
+
+# Case 3: Add one resource, which is not part of the node affinity rule of the other resources in a positive resource affinity rule, to the node affinity rule.
+node-affinity: infer-multi-resources1
+ resources vm:402,vm:404,vm:405
+ nodes node1,node3
+ strict 1
+
+resource-affinity: infer-multi-resources2
+ resources vm:401,vm:402,vm:403,vm:404
+ affinity positive
diff --git a/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg.expect b/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg.expect
new file mode 100644
index 00000000..33c56c62
--- /dev/null
+++ b/src/test/rules_cfgs/infer-node-affinity-for-positive-resource-affinity-rules.cfg.expect
@@ -0,0 +1,111 @@
+--- Log ---
+--- Config ---
+$VAR1 = {
+ 'digest' => '32ae135ef2f8bd84cd12c18af6910dce9d6bc9fa',
+ 'ids' => {
+ 'do-not-infer-negative1' => {
+ 'nodes' => {
+ 'node1' => {
+ 'priority' => 0
+ },
+ 'node2' => {
+ 'priority' => 0
+ }
+ },
+ 'resources' => {
+ 'vm:203' => 1
+ },
+ 'strict' => 1,
+ 'type' => 'node-affinity'
+ },
+ 'do-not-infer-negative2' => {
+ 'nodes' => {
+ 'node3' => {
+ 'priority' => 0
+ }
+ },
+ 'resources' => {
+ 'vm:201' => 1
+ },
+ 'type' => 'node-affinity'
+ },
+ 'do-not-infer-negative3' => {
+ 'affinity' => 'negative',
+ 'resources' => {
+ 'vm:201' => 1,
+ 'vm:203' => 1
+ },
+ 'type' => 'resource-affinity'
+ },
+ 'do-not-infer-positive1' => {
+ 'affinity' => 'positive',
+ 'resources' => {
+ 'vm:101' => 1,
+ 'vm:102' => 1,
+ 'vm:103' => 1
+ },
+ 'type' => 'resource-affinity'
+ },
+ 'infer-multi-resources1' => {
+ 'nodes' => {
+ 'node1' => {
+ 'priority' => 0
+ },
+ 'node3' => {
+ 'priority' => 0
+ }
+ },
+ 'resources' => {
+ 'vm:401' => 1,
+ 'vm:402' => 1,
+ 'vm:403' => 1,
+ 'vm:404' => 1,
+ 'vm:405' => 1
+ },
+ 'strict' => 1,
+ 'type' => 'node-affinity'
+ },
+ 'infer-multi-resources2' => {
+ 'affinity' => 'positive',
+ 'resources' => {
+ 'vm:401' => 1,
+ 'vm:402' => 1,
+ 'vm:403' => 1,
+ 'vm:404' => 1
+ },
+ 'type' => 'resource-affinity'
+ },
+ 'infer-single-resource1' => {
+ 'nodes' => {
+ 'node3' => {
+ 'priority' => 0
+ }
+ },
+ 'resources' => {
+ 'vm:301' => 1,
+ 'vm:302' => 1,
+ 'vm:303' => 1
+ },
+ 'type' => 'node-affinity'
+ },
+ 'infer-single-resource2' => {
+ 'affinity' => 'positive',
+ 'resources' => {
+ 'vm:301' => 1,
+ 'vm:302' => 1,
+ 'vm:303' => 1
+ },
+ 'type' => 'resource-affinity'
+ }
+ },
+ 'order' => {
+ 'do-not-infer-negative1' => 2,
+ 'do-not-infer-negative2' => 3,
+ 'do-not-infer-negative3' => 4,
+ 'do-not-infer-positive1' => 1,
+ 'infer-multi-resources1' => 7,
+ 'infer-multi-resources2' => 8,
+ 'infer-single-resource1' => 5,
+ 'infer-single-resource2' => 6
+ }
+ };
diff --git a/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg b/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg
new file mode 100644
index 00000000..7fb9cdd3
--- /dev/null
+++ b/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg
@@ -0,0 +1,35 @@
+# Case 1: Remove resource affinity rules, where there is a loose Node Affinity rule with multiple priority groups set for the nodes.
+node-affinity: vm101-vm102-should-be-on-node1-or-node2
+ resources vm:101,vm:102
+ nodes node1:1,node2:2
+ strict 0
+
+resource-affinity: vm101-vm102-must-be-kept-separate
+ resources vm:101,vm:102
+ affinity negative
+
+# Case 2: Remove resource affinity rules, where there is a strict Node Affinity rule with multiple priority groups set for the nodes.
+node-affinity: vm201-vm202-must-be-on-node1-or-node2
+ resources vm:201,vm:202
+ nodes node1:1,node2:2
+ strict 1
+
+resource-affinity: vm201-vm202-must-be-kept-together
+ resources vm:201,vm:202
+ affinity positive
+
+# Case 3: Do not remove the resource affinity rule, if there is only one priority group in each node affinity rule for the ha
+# resource affinity rule's resources.
+node-affinity: vm301-must-be-on-node1-with-prio-1
+ resources vm:301
+ nodes node1:1
+ strict 1
+
+node-affinity: vm302-must-be-on-node2-with-prio-2
+ resources vm:302
+ nodes node2:2
+ strict 1
+
+resource-affinity: vm301-vm302-must-be-kept-together
+ resources vm:301,vm:302
+ affinity negative
diff --git a/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg.expect b/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg.expect
new file mode 100644
index 00000000..92d12929
--- /dev/null
+++ b/src/test/rules_cfgs/multi-priority-node-affinity-with-resource-affinity-rules.cfg.expect
@@ -0,0 +1,48 @@
+--- Log ---
+Drop rule 'vm101-vm102-must-be-kept-separate', because resources are in node affinity rules with multiple priorities.
+Drop rule 'vm101-vm102-should-be-on-node1-or-node2', because resources are in a resource affinity rule and cannot be in a node affinity rule with multiple priorities.
+Drop rule 'vm201-vm202-must-be-kept-together', because resources are in node affinity rules with multiple priorities.
+Drop rule 'vm201-vm202-must-be-on-node1-or-node2', because resources are in a resource affinity rule and cannot be in a node affinity rule with multiple priorities.
+--- Config ---
+$VAR1 = {
+ 'digest' => '722a98914555f296af0916c980a9d6c780f5f072',
+ 'ids' => {
+ 'vm301-must-be-on-node1-with-prio-1' => {
+ 'nodes' => {
+ 'node1' => {
+ 'priority' => 1
+ }
+ },
+ 'resources' => {
+ 'vm:301' => 1
+ },
+ 'strict' => 1,
+ 'type' => 'node-affinity'
+ },
+ 'vm301-vm302-must-be-kept-together' => {
+ 'affinity' => 'negative',
+ 'resources' => {
+ 'vm:301' => 1,
+ 'vm:302' => 1
+ },
+ 'type' => 'resource-affinity'
+ },
+ 'vm302-must-be-on-node2-with-prio-2' => {
+ 'nodes' => {
+ 'node2' => {
+ 'priority' => 2
+ }
+ },
+ 'resources' => {
+ 'vm:302' => 1
+ },
+ 'strict' => 1,
+ 'type' => 'node-affinity'
+ }
+ },
+ 'order' => {
+ 'vm301-must-be-on-node1-with-prio-1' => 5,
+ 'vm301-vm302-must-be-kept-together' => 7,
+ 'vm302-must-be-on-node2-with-prio-2' => 6
+ }
+ };
--
2.47.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
next prev parent reply other threads:[~2025-08-01 16:22 UTC|newest]
Thread overview: 14+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-08-01 16:22 [pve-devel] [PATCH ha-manager v2 00/12] HA rules follow up (part 1) Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 01/12] manager: fix ~revision version check for ha groups migration Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 02/12] test: ha tester: add ha groups migration tests with runtime upgrades Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 03/12] tree-wide: pass optional parameters as hash values for for_each_rule helper Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 04/12] api: rules: add missing return schema for the read_rule api endpoint Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 05/12] api: rules: ignore disable parameter if it is set to a falsy value Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 06/12] rules: resource affinity: make message in inter-consistency check clearer Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 07/12] config, manager: do not check ignored resources with affinity when migrating Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 08/12] rules: make positive affinity resources migrate on single resource fail Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 09/12] rules: allow same resources in node and resource affinity rules Daniel Kral
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 10/12] rules: restrict inter-plugin resource references to simple cases Daniel Kral
2025-08-01 16:22 ` Daniel Kral [this message]
2025-08-01 16:22 ` [pve-devel] [PATCH ha-manager v2 12/12] test: ha tester: add resource affinity test cases mixed with node affinity rules Daniel Kral
2025-08-01 17:36 ` [pve-devel] applied: [PATCH ha-manager v2 00/12] HA rules follow up (part 1) Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250801162230.296214-12-d.kral@proxmox.com \
--to=d.kral@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.