[PATCH ha-manager v2 07/14] test: add auto rebalance test case for a marginal absolute improvement

[Michael Ryom <Michael@RyomHerold.dk>]

When a single resource dominates the cluster load, the relative margin
provides almost no hysteresis, so a rebalance motion with an absolute
improvement of only a few percentage points qualifies.

Add a test case, where migrating vm:101 would improve the imbalance from
35.6% to 31.1%, i.e. by 12.5% relative, but only by 4.4 percentage
points absolute.

The test case already configures an absolute margin of 5 percentage
points with the ha-auto-rebalance-margin-mode option, which is not used
yet, so the margin of 5 is still interpreted as relative and its
expected log records that vm:101 is migrated. It is adapted by the
following patch.

Signed-off-by: Michael Ryom <Michael@RyomHerold.dk>
---
 .../test-crs-dynamic-auto-rebalance8/README   | 12 ++++++
 .../test-crs-dynamic-auto-rebalance8/cmdlist  |  3 ++
 .../datacenter.cfg                            |  8 ++++
 .../dynamic_service_stats                     |  5 +++
 .../hardware_status                           |  4 ++
 .../log.expect                                | 39 +++++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  5 +++
 .../static_service_stats                      |  5 +++
 9 files changed, 82 insertions(+)
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance8/README
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance8/cmdlist
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance8/datacenter.cfg
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance8/dynamic_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance8/hardware_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance8/log.expect
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance8/manager_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance8/service_config
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance8/static_service_stats

diff --git a/src/test/test-crs-dynamic-auto-rebalance8/README b/src/test/test-crs-dynamic-auto-rebalance8/README
new file mode 100644
index 0000000..80e9ac3
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance8/README
@@ -0,0 +1,12 @@
+Test that the auto rebalance system with an absolute margin does not act on a
+rebalance motion whose absolute improvement of the node imbalance is below
+the margin, even though its relative improvement is well above it.
+
+The cluster has two nodes with a HA resource excluded from auto rebalancing
+on each (vm:100 on node1 and vm:200 on node2) acting as node base loads, and
+one movable HA resource vm:101 on node1. The imbalance is 35.6% and stays
+above the default trigger threshold of 30%, and migrating vm:101 to node2
+would improve the imbalance to 31.1%, i.e. by 12.5% relative. With a relative
+margin of 5% the motion would qualify, but with the margin mode set to
+'absolute', the improvement of 4.4 percentage points is below the margin of 5
+percentage points, so the motion must not be issued.
diff --git a/src/test/test-crs-dynamic-auto-rebalance8/cmdlist b/src/test/test-crs-dynamic-auto-rebalance8/cmdlist
new file mode 100644
index 0000000..76f0313
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance8/cmdlist
@@ -0,0 +1,3 @@
+[
+    [ "power node1 on", "power node2 on" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance8/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance8/datacenter.cfg
new file mode 100644
index 0000000..63c1d6b
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance8/datacenter.cfg
@@ -0,0 +1,8 @@
+{
+    "crs": {
+        "ha": "dynamic",
+        "ha-auto-rebalance": 1,
+        "ha-auto-rebalance-margin": 5,
+        "ha-auto-rebalance-margin-mode": "absolute"
+    }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance8/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance8/dynamic_service_stats
new file mode 100644
index 0000000..231af30
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance8/dynamic_service_stats
@@ -0,0 +1,5 @@
+{
+    "vm:100": { "cpu": 7.44, "mem": 0 },
+    "vm:101": { "cpu": 7.2, "mem": 0 },
+    "vm:200": { "cpu": 6.96, "mem": 0 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance8/hardware_status b/src/test/test-crs-dynamic-auto-rebalance8/hardware_status
new file mode 100644
index 0000000..864adb3
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance8/hardware_status
@@ -0,0 +1,4 @@
+{
+  "node1": { "power": "off", "network": "off", "maxcpu": 24, "maxmem": 34359738368 },
+  "node2": { "power": "off", "network": "off", "maxcpu": 24, "maxmem": 34359738368 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance8/log.expect b/src/test/test-crs-dynamic-auto-rebalance8/log.expect
new file mode 100644
index 0000000..df11212
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance8/log.expect
@@ -0,0 +1,39 @@
+info      0     hardware: starting simulation
+info     20      cmdlist: execute power node1 on
+info     20    node1/crm: status change startup => wait_for_quorum
+info     20    node1/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node2 on
+info     20    node2/crm: status change startup => wait_for_quorum
+info     20    node2/lrm: status change startup => wait_for_agent_lock
+info     20    node1/crm: got lock 'ha_manager_lock'
+info     20    node1/crm: status change wait_for_quorum => master
+info     20    node1/crm: using scheduler mode 'dynamic'
+info     20    node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info     20    node1/crm: adding new service 'vm:100' on node 'node1'
+info     20    node1/crm: adding new service 'vm:101' on node 'node1'
+info     20    node1/crm: adding new service 'vm:200' on node 'node2'
+info     20    node1/crm: service 'vm:100': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:101': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:200': state changed from 'request_start' to 'started'  (node = node2)
+info     21    node1/lrm: got lock 'ha_agent_node1_lock'
+info     21    node1/lrm: status change wait_for_agent_lock => active
+info     21    node1/lrm: starting service vm:100
+info     21    node1/lrm: service status vm:100 started
+info     21    node1/lrm: starting service vm:101
+info     21    node1/lrm: service status vm:101 started
+info     22    node2/crm: status change wait_for_quorum => slave
+info     23    node2/lrm: got lock 'ha_agent_node2_lock'
+info     23    node2/lrm: status change wait_for_agent_lock => active
+info     23    node2/lrm: starting service vm:200
+info     23    node2/lrm: service status vm:200 started
+info     80    node1/crm: auto rebalance - migrate vm:101 to node2 (expected change for imbalance from 35.6% to 31.1%)
+info     80    node1/crm: got crm command: migrate vm:101 node2
+info     80    node1/crm: migrate service 'vm:101' to node 'node2'
+info     80    node1/crm: service 'vm:101': state changed from 'started' to 'migrate'  (node = node1, target = node2)
+info     81    node1/lrm: service vm:101 - start migrate to node 'node2'
+info     81    node1/lrm: service vm:101 - end migrate to node 'node2'
+info    100    node1/crm: service 'vm:101': state changed from 'migrate' to 'started'  (node = node2)
+info    103    node2/lrm: starting service vm:101
+info    103    node2/lrm: service status vm:101 started
+info    620     hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance8/manager_status b/src/test/test-crs-dynamic-auto-rebalance8/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance8/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance8/service_config b/src/test/test-crs-dynamic-auto-rebalance8/service_config
new file mode 100644
index 0000000..83bf42e
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance8/service_config
@@ -0,0 +1,5 @@
+{
+    "vm:100": { "node": "node1", "state": "started", "auto-rebalance": 0 },
+    "vm:101": { "node": "node1", "state": "started" },
+    "vm:200": { "node": "node2", "state": "started", "auto-rebalance": 0 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance8/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance8/static_service_stats
new file mode 100644
index 0000000..300fb7a
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance8/static_service_stats
@@ -0,0 +1,5 @@
+{
+    "vm:100": { "maxcpu": 8.0, "maxmem": 8589934592 },
+    "vm:101": { "maxcpu": 8.0, "maxmem": 8589934592 },
+    "vm:200": { "maxcpu": 8.0, "maxmem": 8589934592 }
+}
-- 
2.47.3