all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Dominik Rusovac <d.rusovac@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-ha-manager v2 19/19] test: add automatic rebalancing test cases with weights
Date: Thu, 12 Nov 2026 10:44:37 +0100	[thread overview]
Message-ID: <20261112094437.1951271-20-d.rusovac@proxmox.com> (raw)
In-Reply-To: <20261112094437.1951271-1-d.rusovac@proxmox.com>

add test cases which document that the crs weights steer the automatic
rebalancing towards balancing the CPU load or the memory usage of the
nodes; increasing a weight lets the load imbalance exceed the threshold
for a cluster that is balanced with equal weights; and a weight of 0
balances the other resource only

Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---

Notes:
    changes since v1:
    * reword commit
    * use the new option names
    * drop the redundant commands setting the weights to 1
    * add test cases with one of the weights set to 0

 .../README                                    |  4 ++
 .../cmdlist                                   |  4 ++
 .../datacenter.cfg                            |  6 ++
 .../dynamic_service_stats                     |  8 +++
 .../hardware_status                           |  5 ++
 .../log.expect                                | 61 +++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  8 +++
 .../static_service_stats                      |  8 +++
 .../README                                    |  4 ++
 .../cmdlist                                   |  6 ++
 .../datacenter.cfg                            |  6 ++
 .../dynamic_service_stats                     |  8 +++
 .../hardware_status                           |  5 ++
 .../log.expect                                | 65 +++++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  8 +++
 .../static_service_stats                      |  8 +++
 .../README                                    |  5 ++
 .../cmdlist                                   |  4 ++
 .../datacenter.cfg                            |  6 ++
 .../dynamic_service_stats                     |  8 +++
 .../hardware_status                           |  5 ++
 .../log.expect                                | 61 +++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  8 +++
 .../static_service_stats                      |  8 +++
 .../README                                    |  5 ++
 .../cmdlist                                   |  4 ++
 .../datacenter.cfg                            |  7 ++
 .../dynamic_service_stats                     |  8 +++
 .../hardware_status                           |  5 ++
 .../log.expect                                | 62 ++++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  8 +++
 .../static_service_stats                      |  8 +++
 .../README                                    |  5 ++
 .../cmdlist                                   |  4 ++
 .../datacenter.cfg                            |  6 ++
 .../dynamic_service_stats                     |  8 +++
 .../hardware_status                           |  5 ++
 .../log.expect                                | 61 +++++++++++++++++
 .../manager_status                            |  1 +
 .../service_config                            |  8 +++
 .../static_service_stats                      |  8 +++
 45 files changed, 536 insertions(+)
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/README
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/cmdlist
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/datacenter.cfg
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/dynamic_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/hardware_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/log.expect
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/manager_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/service_config
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/static_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/README
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/cmdlist
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/datacenter.cfg
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/dynamic_service_stats
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/hardware_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/log.expect
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/manager_status
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/service_config
 create mode 100644 src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/static_service_stats

diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README
new file mode 100644
index 0000000..05cb5df
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README
@@ -0,0 +1,4 @@
+With equal weights, the node imbalance is 27.5%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that increasing ha-cpu-weight
+increases the node imbalance and triggers a rebalancing migration from
+the CPU-heavy node3 to node2, which has a low CPU load and memory usage.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist
new file mode 100644
index 0000000..869d8e8
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist
@@ -0,0 +1,4 @@
+[
+    [ "power node1 on", "power node2 on", "power node3 on" ],
+    [ "datacenter crs set ha-cpu-weight 2" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+    "crs": {
+        "ha": "dynamic",
+        "ha-auto-rebalance": 1
+    }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats
new file mode 100644
index 0000000..0d0b3b2
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "cpu": 1.5, "mem": 42949672960 },
+    "vm:102": { "cpu": 0.5, "mem": 21474836480 },
+    "vm:103": { "cpu": 2.0, "mem": 8589934592  },
+    "vm:104": { "cpu": 1.0, "mem": 4294967296  },
+    "vm:105": { "cpu": 9.0, "mem": 5368709120  },
+    "vm:106": { "cpu": 6.0, "mem": 3221225472  }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status
@@ -0,0 +1,5 @@
+{
+  "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect
new file mode 100644
index 0000000..cf770d5
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect
@@ -0,0 +1,61 @@
+info      0     hardware: starting simulation
+info     20      cmdlist: execute power node1 on
+info     20    node1/crm: status change startup => wait_for_quorum
+info     20    node1/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node2 on
+info     20    node2/crm: status change startup => wait_for_quorum
+info     20    node2/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node3 on
+info     20    node3/crm: status change startup => wait_for_quorum
+info     20    node3/lrm: status change startup => wait_for_agent_lock
+info     20    node1/crm: got lock 'ha_manager_lock'
+info     20    node1/crm: status change wait_for_quorum => master
+info     20    node1/crm: using scheduler mode 'dynamic'
+info     20    node1/crm: using ha-auto-rebalance '1'
+info     20    node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info     20    node1/crm: adding new service 'vm:101' on node 'node1'
+info     20    node1/crm: adding new service 'vm:102' on node 'node1'
+info     20    node1/crm: adding new service 'vm:103' on node 'node2'
+info     20    node1/crm: adding new service 'vm:104' on node 'node2'
+info     20    node1/crm: adding new service 'vm:105' on node 'node3'
+info     20    node1/crm: adding new service 'vm:106' on node 'node3'
+info     20    node1/crm: service 'vm:101': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:102': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:103': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:104': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:105': state changed from 'request_start' to 'started'  (node = node3)
+info     20    node1/crm: service 'vm:106': state changed from 'request_start' to 'started'  (node = node3)
+info     21    node1/lrm: got lock 'ha_agent_node1_lock'
+info     21    node1/lrm: status change wait_for_agent_lock => active
+info     21    node1/lrm: starting service vm:101
+info     21    node1/lrm: service status vm:101 started
+info     21    node1/lrm: starting service vm:102
+info     21    node1/lrm: service status vm:102 started
+info     22    node2/crm: status change wait_for_quorum => slave
+info     23    node2/lrm: got lock 'ha_agent_node2_lock'
+info     23    node2/lrm: status change wait_for_agent_lock => active
+info     23    node2/lrm: starting service vm:103
+info     23    node2/lrm: service status vm:103 started
+info     23    node2/lrm: starting service vm:104
+info     23    node2/lrm: service status vm:104 started
+info     24    node3/crm: status change wait_for_quorum => slave
+info     25    node3/lrm: got lock 'ha_agent_node3_lock'
+info     25    node3/lrm: status change wait_for_agent_lock => active
+info     25    node3/lrm: starting service vm:105
+info     25    node3/lrm: service status vm:105 started
+info     25    node3/lrm: starting service vm:106
+info     25    node3/lrm: service status vm:106 started
+info    120      cmdlist: execute datacenter crs set ha-cpu-weight 2
+info    120    node1/crm: switching ha-cpu-weight from '1' to '2'
+info    160    node1/crm: auto rebalance - migrate vm:106 to node2 (expected change for imbalance from 33.3% to 4.4%)
+info    160    node1/crm: got crm command: migrate vm:106 node2
+info    160    node1/crm: migrate service 'vm:106' to node 'node2'
+info    160    node1/crm: service 'vm:106': state changed from 'started' to 'migrate'  (node = node3, target = node2)
+info    165    node3/lrm: service vm:106 - start migrate to node 'node2'
+info    165    node3/lrm: service vm:106 - end migrate to node 'node2'
+info    180    node1/crm: service 'vm:106': state changed from 'migrate' to 'started'  (node = node2)
+info    183    node2/lrm: starting service vm:106
+info    183    node2/lrm: service status vm:106 started
+info    720     hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "node": "node1", "state": "started" },
+    "vm:102": { "node": "node1", "state": "started" },
+    "vm:103": { "node": "node2", "state": "started" },
+    "vm:104": { "node": "node2", "state": "started" },
+    "vm:105": { "node": "node3", "state": "started" },
+    "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README
new file mode 100644
index 0000000..e2568fb
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README
@@ -0,0 +1,4 @@
+With equal weights, the node imbalance is 18.7%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that incrementing
+ha-mem-weight eventually triggers a rebalancing migration from the
+memory-heavy node3.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist
new file mode 100644
index 0000000..375ab2f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist
@@ -0,0 +1,6 @@
+[
+    [ "power node1 on", "power node2 on", "power node3 on" ],
+    [ "datacenter crs set ha-mem-weight 2" ],
+    [ "datacenter crs set ha-mem-weight 3" ],
+    [ "datacenter crs set ha-mem-weight 4" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+    "crs": {
+        "ha": "dynamic",
+        "ha-auto-rebalance": 1
+    }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats
new file mode 100644
index 0000000..14740f1
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "cpu": 10.0, "mem": 3221225472  },
+    "vm:102": { "cpu": 5.0,  "mem": 1073741824  },
+    "vm:103": { "cpu": 5.0,  "mem": 3221225472  },
+    "vm:104": { "cpu": 3.0,  "mem": 1073741824  },
+    "vm:105": { "cpu": 0.5,  "mem": 32212254720 },
+    "vm:106": { "cpu": 0.5,  "mem": 6442450944  }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status
@@ -0,0 +1,5 @@
+{
+  "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect
new file mode 100644
index 0000000..6dd1901
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect
@@ -0,0 +1,65 @@
+info      0     hardware: starting simulation
+info     20      cmdlist: execute power node1 on
+info     20    node1/crm: status change startup => wait_for_quorum
+info     20    node1/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node2 on
+info     20    node2/crm: status change startup => wait_for_quorum
+info     20    node2/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node3 on
+info     20    node3/crm: status change startup => wait_for_quorum
+info     20    node3/lrm: status change startup => wait_for_agent_lock
+info     20    node1/crm: got lock 'ha_manager_lock'
+info     20    node1/crm: status change wait_for_quorum => master
+info     20    node1/crm: using scheduler mode 'dynamic'
+info     20    node1/crm: using ha-auto-rebalance '1'
+info     20    node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info     20    node1/crm: adding new service 'vm:101' on node 'node1'
+info     20    node1/crm: adding new service 'vm:102' on node 'node1'
+info     20    node1/crm: adding new service 'vm:103' on node 'node2'
+info     20    node1/crm: adding new service 'vm:104' on node 'node2'
+info     20    node1/crm: adding new service 'vm:105' on node 'node3'
+info     20    node1/crm: adding new service 'vm:106' on node 'node3'
+info     20    node1/crm: service 'vm:101': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:102': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:103': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:104': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:105': state changed from 'request_start' to 'started'  (node = node3)
+info     20    node1/crm: service 'vm:106': state changed from 'request_start' to 'started'  (node = node3)
+info     21    node1/lrm: got lock 'ha_agent_node1_lock'
+info     21    node1/lrm: status change wait_for_agent_lock => active
+info     21    node1/lrm: starting service vm:101
+info     21    node1/lrm: service status vm:101 started
+info     21    node1/lrm: starting service vm:102
+info     21    node1/lrm: service status vm:102 started
+info     22    node2/crm: status change wait_for_quorum => slave
+info     23    node2/lrm: got lock 'ha_agent_node2_lock'
+info     23    node2/lrm: status change wait_for_agent_lock => active
+info     23    node2/lrm: starting service vm:103
+info     23    node2/lrm: service status vm:103 started
+info     23    node2/lrm: starting service vm:104
+info     23    node2/lrm: service status vm:104 started
+info     24    node3/crm: status change wait_for_quorum => slave
+info     25    node3/lrm: got lock 'ha_agent_node3_lock'
+info     25    node3/lrm: status change wait_for_agent_lock => active
+info     25    node3/lrm: starting service vm:105
+info     25    node3/lrm: service status vm:105 started
+info     25    node3/lrm: starting service vm:106
+info     25    node3/lrm: service status vm:106 started
+info    120      cmdlist: execute datacenter crs set ha-mem-weight 2
+info    120    node1/crm: switching ha-mem-weight from '1' to '2'
+info    220      cmdlist: execute datacenter crs set ha-mem-weight 3
+info    220    node1/crm: switching ha-mem-weight from '2' to '3'
+info    320      cmdlist: execute datacenter crs set ha-mem-weight 4
+info    320    node1/crm: switching ha-mem-weight from '3' to '4'
+info    360    node1/crm: auto rebalance - migrate vm:106 to node2 (expected change for imbalance from 32.9% to 17.3%)
+info    360    node1/crm: got crm command: migrate vm:106 node2
+info    360    node1/crm: migrate service 'vm:106' to node 'node2'
+info    360    node1/crm: service 'vm:106': state changed from 'started' to 'migrate'  (node = node3, target = node2)
+info    365    node3/lrm: service vm:106 - start migrate to node 'node2'
+info    365    node3/lrm: service vm:106 - end migrate to node 'node2'
+info    380    node1/crm: service 'vm:106': state changed from 'migrate' to 'started'  (node = node2)
+info    383    node2/lrm: starting service vm:106
+info    383    node2/lrm: service status vm:106 started
+info    920     hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "node": "node1", "state": "started" },
+    "vm:102": { "node": "node1", "state": "started" },
+    "vm:103": { "node": "node2", "state": "started" },
+    "vm:104": { "node": "node2", "state": "started" },
+    "vm:105": { "node": "node3", "state": "started" },
+    "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README
new file mode 100644
index 0000000..e9ea92c
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README
@@ -0,0 +1,5 @@
+With equal weights, the node imbalance is 27.5%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that increasing ha-mem-weight
+increases the node imbalance and triggers a rebalancing migration from
+the memory-heavy node1 to node2, which has a low CPU load and memory
+usage.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist
new file mode 100644
index 0000000..ed207f5
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist
@@ -0,0 +1,4 @@
+[
+    [ "power node1 on", "power node2 on", "power node3 on" ],
+    [ "datacenter crs set ha-mem-weight 2" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+    "crs": {
+        "ha": "dynamic",
+        "ha-auto-rebalance": 1
+    }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats
new file mode 100644
index 0000000..0d0b3b2
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "cpu": 1.5, "mem": 42949672960 },
+    "vm:102": { "cpu": 0.5, "mem": 21474836480 },
+    "vm:103": { "cpu": 2.0, "mem": 8589934592  },
+    "vm:104": { "cpu": 1.0, "mem": 4294967296  },
+    "vm:105": { "cpu": 9.0, "mem": 5368709120  },
+    "vm:106": { "cpu": 6.0, "mem": 3221225472  }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status
@@ -0,0 +1,5 @@
+{
+  "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect
new file mode 100644
index 0000000..91f6695
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect
@@ -0,0 +1,61 @@
+info      0     hardware: starting simulation
+info     20      cmdlist: execute power node1 on
+info     20    node1/crm: status change startup => wait_for_quorum
+info     20    node1/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node2 on
+info     20    node2/crm: status change startup => wait_for_quorum
+info     20    node2/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node3 on
+info     20    node3/crm: status change startup => wait_for_quorum
+info     20    node3/lrm: status change startup => wait_for_agent_lock
+info     20    node1/crm: got lock 'ha_manager_lock'
+info     20    node1/crm: status change wait_for_quorum => master
+info     20    node1/crm: using scheduler mode 'dynamic'
+info     20    node1/crm: using ha-auto-rebalance '1'
+info     20    node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info     20    node1/crm: adding new service 'vm:101' on node 'node1'
+info     20    node1/crm: adding new service 'vm:102' on node 'node1'
+info     20    node1/crm: adding new service 'vm:103' on node 'node2'
+info     20    node1/crm: adding new service 'vm:104' on node 'node2'
+info     20    node1/crm: adding new service 'vm:105' on node 'node3'
+info     20    node1/crm: adding new service 'vm:106' on node 'node3'
+info     20    node1/crm: service 'vm:101': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:102': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:103': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:104': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:105': state changed from 'request_start' to 'started'  (node = node3)
+info     20    node1/crm: service 'vm:106': state changed from 'request_start' to 'started'  (node = node3)
+info     21    node1/lrm: got lock 'ha_agent_node1_lock'
+info     21    node1/lrm: status change wait_for_agent_lock => active
+info     21    node1/lrm: starting service vm:101
+info     21    node1/lrm: service status vm:101 started
+info     21    node1/lrm: starting service vm:102
+info     21    node1/lrm: service status vm:102 started
+info     22    node2/crm: status change wait_for_quorum => slave
+info     23    node2/lrm: got lock 'ha_agent_node2_lock'
+info     23    node2/lrm: status change wait_for_agent_lock => active
+info     23    node2/lrm: starting service vm:103
+info     23    node2/lrm: service status vm:103 started
+info     23    node2/lrm: starting service vm:104
+info     23    node2/lrm: service status vm:104 started
+info     24    node3/crm: status change wait_for_quorum => slave
+info     25    node3/lrm: got lock 'ha_agent_node3_lock'
+info     25    node3/lrm: status change wait_for_agent_lock => active
+info     25    node3/lrm: starting service vm:105
+info     25    node3/lrm: service status vm:105 started
+info     25    node3/lrm: starting service vm:106
+info     25    node3/lrm: service status vm:106 started
+info    120      cmdlist: execute datacenter crs set ha-mem-weight 2
+info    120    node1/crm: switching ha-mem-weight from '1' to '2'
+info    160    node1/crm: auto rebalance - migrate vm:102 to node2 (expected change for imbalance from 33.3% to 3.8%)
+info    160    node1/crm: got crm command: migrate vm:102 node2
+info    160    node1/crm: migrate service 'vm:102' to node 'node2'
+info    160    node1/crm: service 'vm:102': state changed from 'started' to 'migrate'  (node = node1, target = node2)
+info    161    node1/lrm: service vm:102 - start migrate to node 'node2'
+info    161    node1/lrm: service vm:102 - end migrate to node 'node2'
+info    180    node1/crm: service 'vm:102': state changed from 'migrate' to 'started'  (node = node2)
+info    183    node2/lrm: starting service vm:102
+info    183    node2/lrm: service status vm:102 started
+info    720     hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "node": "node1", "state": "started" },
+    "vm:102": { "node": "node1", "state": "started" },
+    "vm:103": { "node": "node2", "state": "started" },
+    "vm:104": { "node": "node2", "state": "started" },
+    "vm:105": { "node": "node3", "state": "started" },
+    "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/README b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/README
new file mode 100644
index 0000000..82e46f8
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/README
@@ -0,0 +1,5 @@
+With equal weights, the node imbalance is 27.5%, which is below the
+ha-auto-rebalance-threshold of 35%. Test that setting ha-cpu-weight to
+0 balances the memory usage only, which triggers a rebalancing
+migration from the memory-heavy node1 to node3, which has the highest
+CPU load but a low memory usage.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/cmdlist
new file mode 100644
index 0000000..863befe
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/cmdlist
@@ -0,0 +1,4 @@
+[
+    [ "power node1 on", "power node2 on", "power node3 on" ],
+    [ "datacenter crs set ha-cpu-weight 0" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/datacenter.cfg
new file mode 100644
index 0000000..7fcaf00
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/datacenter.cfg
@@ -0,0 +1,7 @@
+{
+    "crs": {
+        "ha": "dynamic",
+        "ha-auto-rebalance": 1,
+        "ha-auto-rebalance-threshold": 35
+    }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/dynamic_service_stats
new file mode 100644
index 0000000..0d0b3b2
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "cpu": 1.5, "mem": 42949672960 },
+    "vm:102": { "cpu": 0.5, "mem": 21474836480 },
+    "vm:103": { "cpu": 2.0, "mem": 8589934592  },
+    "vm:104": { "cpu": 1.0, "mem": 4294967296  },
+    "vm:105": { "cpu": 9.0, "mem": 5368709120  },
+    "vm:106": { "cpu": 6.0, "mem": 3221225472  }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/hardware_status
@@ -0,0 +1,5 @@
+{
+  "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/log.expect b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/log.expect
new file mode 100644
index 0000000..b0cdaaf
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/log.expect
@@ -0,0 +1,62 @@
+info      0     hardware: starting simulation
+info     20      cmdlist: execute power node1 on
+info     20    node1/crm: status change startup => wait_for_quorum
+info     20    node1/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node2 on
+info     20    node2/crm: status change startup => wait_for_quorum
+info     20    node2/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node3 on
+info     20    node3/crm: status change startup => wait_for_quorum
+info     20    node3/lrm: status change startup => wait_for_agent_lock
+info     20    node1/crm: got lock 'ha_manager_lock'
+info     20    node1/crm: status change wait_for_quorum => master
+info     20    node1/crm: using scheduler mode 'dynamic'
+info     20    node1/crm: using ha-auto-rebalance '1'
+info     20    node1/crm: using ha-auto-rebalance-threshold '0.35'
+info     20    node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info     20    node1/crm: adding new service 'vm:101' on node 'node1'
+info     20    node1/crm: adding new service 'vm:102' on node 'node1'
+info     20    node1/crm: adding new service 'vm:103' on node 'node2'
+info     20    node1/crm: adding new service 'vm:104' on node 'node2'
+info     20    node1/crm: adding new service 'vm:105' on node 'node3'
+info     20    node1/crm: adding new service 'vm:106' on node 'node3'
+info     20    node1/crm: service 'vm:101': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:102': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:103': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:104': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:105': state changed from 'request_start' to 'started'  (node = node3)
+info     20    node1/crm: service 'vm:106': state changed from 'request_start' to 'started'  (node = node3)
+info     21    node1/lrm: got lock 'ha_agent_node1_lock'
+info     21    node1/lrm: status change wait_for_agent_lock => active
+info     21    node1/lrm: starting service vm:101
+info     21    node1/lrm: service status vm:101 started
+info     21    node1/lrm: starting service vm:102
+info     21    node1/lrm: service status vm:102 started
+info     22    node2/crm: status change wait_for_quorum => slave
+info     23    node2/lrm: got lock 'ha_agent_node2_lock'
+info     23    node2/lrm: status change wait_for_agent_lock => active
+info     23    node2/lrm: starting service vm:103
+info     23    node2/lrm: service status vm:103 started
+info     23    node2/lrm: starting service vm:104
+info     23    node2/lrm: service status vm:104 started
+info     24    node3/crm: status change wait_for_quorum => slave
+info     25    node3/lrm: got lock 'ha_agent_node3_lock'
+info     25    node3/lrm: status change wait_for_agent_lock => active
+info     25    node3/lrm: starting service vm:105
+info     25    node3/lrm: service status vm:105 started
+info     25    node3/lrm: starting service vm:106
+info     25    node3/lrm: service status vm:106 started
+info    120      cmdlist: execute datacenter crs set ha-cpu-weight 0
+info    120    node1/crm: switching ha-cpu-weight from '1' to '0'
+info    160    node1/crm: auto rebalance - migrate vm:102 to node3 (expected change for imbalance from 62.6% to 30.4%)
+info    160    node1/crm: got crm command: migrate vm:102 node3
+info    160    node1/crm: migrate service 'vm:102' to node 'node3'
+info    160    node1/crm: service 'vm:102': state changed from 'started' to 'migrate'  (node = node1, target = node3)
+info    161    node1/lrm: service vm:102 - start migrate to node 'node3'
+info    161    node1/lrm: service vm:102 - end migrate to node 'node3'
+info    180    node1/crm: service 'vm:102': state changed from 'migrate' to 'started'  (node = node3)
+info    185    node3/lrm: starting service vm:102
+info    185    node3/lrm: service status vm:102 started
+info    720     hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/manager_status b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/service_config b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/service_config
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "node": "node1", "state": "started" },
+    "vm:102": { "node": "node1", "state": "started" },
+    "vm:103": { "node": "node2", "state": "started" },
+    "vm:104": { "node": "node2", "state": "started" },
+    "vm:105": { "node": "node3", "state": "started" },
+    "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-cpu-weight/static_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/README b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/README
new file mode 100644
index 0000000..8ede76c
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/README
@@ -0,0 +1,5 @@
+With equal weights, the node imbalance is 27.5%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that setting ha-mem-weight to
+0 balances the CPU load only, which triggers a rebalancing migration
+from the CPU-heavy node3 to node1, which has the lowest CPU load but
+the highest memory usage.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/cmdlist
new file mode 100644
index 0000000..12b01a7
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/cmdlist
@@ -0,0 +1,4 @@
+[
+    [ "power node1 on", "power node2 on", "power node3 on" ],
+    [ "datacenter crs set ha-mem-weight 0" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+    "crs": {
+        "ha": "dynamic",
+        "ha-auto-rebalance": 1
+    }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/dynamic_service_stats
new file mode 100644
index 0000000..0d0b3b2
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "cpu": 1.5, "mem": 42949672960 },
+    "vm:102": { "cpu": 0.5, "mem": 21474836480 },
+    "vm:103": { "cpu": 2.0, "mem": 8589934592  },
+    "vm:104": { "cpu": 1.0, "mem": 4294967296  },
+    "vm:105": { "cpu": 9.0, "mem": 5368709120  },
+    "vm:106": { "cpu": 6.0, "mem": 3221225472  }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/hardware_status
@@ -0,0 +1,5 @@
+{
+  "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+  "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/log.expect b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/log.expect
new file mode 100644
index 0000000..1a7e746
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/log.expect
@@ -0,0 +1,61 @@
+info      0     hardware: starting simulation
+info     20      cmdlist: execute power node1 on
+info     20    node1/crm: status change startup => wait_for_quorum
+info     20    node1/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node2 on
+info     20    node2/crm: status change startup => wait_for_quorum
+info     20    node2/lrm: status change startup => wait_for_agent_lock
+info     20      cmdlist: execute power node3 on
+info     20    node3/crm: status change startup => wait_for_quorum
+info     20    node3/lrm: status change startup => wait_for_agent_lock
+info     20    node1/crm: got lock 'ha_manager_lock'
+info     20    node1/crm: status change wait_for_quorum => master
+info     20    node1/crm: using scheduler mode 'dynamic'
+info     20    node1/crm: using ha-auto-rebalance '1'
+info     20    node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info     20    node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info     20    node1/crm: adding new service 'vm:101' on node 'node1'
+info     20    node1/crm: adding new service 'vm:102' on node 'node1'
+info     20    node1/crm: adding new service 'vm:103' on node 'node2'
+info     20    node1/crm: adding new service 'vm:104' on node 'node2'
+info     20    node1/crm: adding new service 'vm:105' on node 'node3'
+info     20    node1/crm: adding new service 'vm:106' on node 'node3'
+info     20    node1/crm: service 'vm:101': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:102': state changed from 'request_start' to 'started'  (node = node1)
+info     20    node1/crm: service 'vm:103': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:104': state changed from 'request_start' to 'started'  (node = node2)
+info     20    node1/crm: service 'vm:105': state changed from 'request_start' to 'started'  (node = node3)
+info     20    node1/crm: service 'vm:106': state changed from 'request_start' to 'started'  (node = node3)
+info     21    node1/lrm: got lock 'ha_agent_node1_lock'
+info     21    node1/lrm: status change wait_for_agent_lock => active
+info     21    node1/lrm: starting service vm:101
+info     21    node1/lrm: service status vm:101 started
+info     21    node1/lrm: starting service vm:102
+info     21    node1/lrm: service status vm:102 started
+info     22    node2/crm: status change wait_for_quorum => slave
+info     23    node2/lrm: got lock 'ha_agent_node2_lock'
+info     23    node2/lrm: status change wait_for_agent_lock => active
+info     23    node2/lrm: starting service vm:103
+info     23    node2/lrm: service status vm:103 started
+info     23    node2/lrm: starting service vm:104
+info     23    node2/lrm: service status vm:104 started
+info     24    node3/crm: status change wait_for_quorum => slave
+info     25    node3/lrm: got lock 'ha_agent_node3_lock'
+info     25    node3/lrm: status change wait_for_agent_lock => active
+info     25    node3/lrm: starting service vm:105
+info     25    node3/lrm: service status vm:105 started
+info     25    node3/lrm: starting service vm:106
+info     25    node3/lrm: service status vm:106 started
+info    120      cmdlist: execute datacenter crs set ha-mem-weight 0
+info    120    node1/crm: switching ha-mem-weight from '1' to '0'
+info    160    node1/crm: auto rebalance - migrate vm:106 to node1 (expected change for imbalance from 62.6% to 27.8%)
+info    160    node1/crm: got crm command: migrate vm:106 node1
+info    160    node1/crm: migrate service 'vm:106' to node 'node1'
+info    160    node1/crm: service 'vm:106': state changed from 'started' to 'migrate'  (node = node3, target = node1)
+info    165    node3/lrm: service vm:106 - start migrate to node 'node1'
+info    165    node3/lrm: service vm:106 - end migrate to node 'node1'
+info    180    node1/crm: service 'vm:106': state changed from 'migrate' to 'started'  (node = node1)
+info    181    node1/lrm: starting service vm:106
+info    181    node1/lrm: service status vm:106 started
+info    720     hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/manager_status b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/service_config b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/service_config
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "node": "node1", "state": "started" },
+    "vm:102": { "node": "node1", "state": "started" },
+    "vm:103": { "node": "node2", "state": "started" },
+    "vm:104": { "node": "node2", "state": "started" },
+    "vm:105": { "node": "node3", "state": "started" },
+    "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-zero-mem-weight/static_service_stats
@@ -0,0 +1,8 @@
+{
+    "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+    "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
-- 
2.47.3





      parent reply	other threads:[~2026-11-12  9:47 UTC|newest]

Thread overview: 33+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-12  9:44 [RFC PATCH-SERIES cluster/ha-manager/manager/proxmox{,-perl-rs} v2 00/19] fix #7993: implement weighted crs load balancing Dominik Rusovac
2026-11-12  9:44 ` [PATCH proxmox v2 01/19] resource-scheduling: fix typos Dominik Rusovac
2026-11-19 12:27   ` Daniel Kral
2026-11-12  9:44 ` [PATCH proxmox v2 02/19] resource-scheduling: scheduler: refine score_* docs Dominik Rusovac
2026-11-19 12:27   ` Daniel Kral
2026-11-19 12:47     ` Dominik Rusovac
2026-11-12  9:44 ` [PATCH proxmox v2 03/19] resource-scheduling: scheduler: abstract post-migration stats logic Dominik Rusovac
2026-11-19 12:28   ` Daniel Kral
2026-11-19 12:47     ` Dominik Rusovac
2026-11-12  9:44 ` [PATCH proxmox v2 04/19] fix #7993: resource-scheduling: node: add weighted node load Dominik Rusovac
2026-11-19 12:28   ` Daniel Kral
2026-11-19 12:50     ` Dominik Rusovac
2026-11-12  9:44 ` [PATCH proxmox v2 05/19] resource-scheduling: scheduler: make imbalance with candidate public Dominik Rusovac
2026-11-19 12:41   ` Daniel Kral
2026-11-19 12:52     ` Dominik Rusovac
2026-11-12  9:44 ` [PATCH proxmox v2 06/19] fix #7993: resource-scheduling: scheduler: add weights Dominik Rusovac
2026-11-19 12:40   ` Daniel Kral
2026-11-19 13:52     ` Dominik Rusovac
2026-11-12  9:44 ` [PATCH proxmox v2 07/19] workspace,resource-scheduling: add proptest as dev-dependency Dominik Rusovac
2026-11-12  9:44 ` [PATCH proxmox v2 08/19] resource-scheduling: scheduler: add property tests for weights Dominik Rusovac
2026-11-12  9:44 ` [PATCH proxmox-perl-rs v2 09/19] fix #7993: pve-rs: resource-scheduling: add scheduler options Dominik Rusovac
2026-11-19 12:45   ` Daniel Kral
2026-11-19 13:28     ` Dominik Rusovac
2026-11-12  9:44 ` [PATCH pve-cluster v2 10/19] fix #7993: datacenter config: ad ha-{cpu,mem}-weight crs options Dominik Rusovac
2026-11-12  9:44 ` [PATCH pve-manager v2 11/19] ui: form: crs options: make fine-tuning options advanced Dominik Rusovac
2026-11-12  9:44 ` [PATCH pve-manager v2 12/19] fix #7993: ui: form: crs options: add weights Dominik Rusovac
2026-11-12  9:44 ` [PATCH pve-ha-manager v2 13/19] manager: remove obsolete argument Dominik Rusovac
2026-11-12  9:44 ` [PATCH pve-ha-manager v2 14/19] manager: rename update_crs_scheduler_mod to update_crs_settings Dominik Rusovac
2026-11-12  9:44 ` [PATCH pve-ha-manager v2 15/19] manager: factor out crs option update helper Dominik Rusovac
2026-11-12  9:44 ` [PATCH pve-ha-manager v2 16/19] manager: log changes to crs settings Dominik Rusovac
2026-11-12  9:44 ` [PATCH pve-ha-manager v2 17/19] sim: hardware: allow changing datacenter settings at runtime Dominik Rusovac
2026-11-12  9:44 ` [PATCH pve-ha-manager v2 18/19] fix #7993: manager,usage: use weights for static and dynamic mode Dominik Rusovac
2026-11-12  9:44 ` Dominik Rusovac [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261112094437.1951271-20-d.rusovac@proxmox.com \
    --to=d.rusovac@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal