From: Dominik Rusovac <d.rusovac@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-ha-manager 13/15] test: add auto rebalance with weights test cases
Date: Wed, 21 Oct 2026 10:59:30 +0200 [thread overview]
Message-ID: <20261021085932.1196302-14-d.rusovac@proxmox.com> (raw)
In-Reply-To: <20261021085932.1196302-1-d.rusovac@proxmox.com>
Add tests documenting basic efficacy of auto rebalancing with weights.
Overview:
- test-crs-dynamic-auto-rebalance-with-increasing-bias: no rebalancing,
increment mem-weight until rebalancing is triggered
- test-crs-dynamic-auto-rebalance-with-{cpu,mem}-bias: trigger migration
that rebalances {cpu,mem} by incrementing {cpu,mem}-weight once
Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---
.../README | 5 ++
.../cmdlist | 6 ++
.../datacenter.cfg | 6 ++
.../dynamic_service_stats | 8 +++
.../hardware_status | 5 ++
.../log.expect | 62 +++++++++++++++++
.../manager_status | 1 +
.../service_config | 8 +++
.../static_service_stats | 8 +++
.../README | 4 ++
.../cmdlist | 8 +++
.../datacenter.cfg | 6 ++
.../dynamic_service_stats | 8 +++
.../hardware_status | 5 ++
.../log.expect | 66 +++++++++++++++++++
.../manager_status | 1 +
.../service_config | 8 +++
.../static_service_stats | 8 +++
.../README | 5 ++
.../cmdlist | 6 ++
.../datacenter.cfg | 6 ++
.../dynamic_service_stats | 8 +++
.../hardware_status | 5 ++
.../log.expect | 62 +++++++++++++++++
.../manager_status | 1 +
.../service_config | 8 +++
.../static_service_stats | 8 +++
27 files changed, 332 insertions(+)
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README
new file mode 100644
index 0000000..f67978e
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README
@@ -0,0 +1,5 @@
+At equal weights (=1) load imbalance sits at 27.5%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that increasing the
+ha-auto-rebalance-cpu-weight will increase load imbalance and trigger a
+rebalancing migration from cpu-heavy node3 to node2, which sits low on either
+resource.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist
new file mode 100644
index 0000000..c29b4b6
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist
@@ -0,0 +1,6 @@
+[
+ [ "power node1 on", "power node2 on", "power node3 on" ],
+ [ "datacenter crs set ha-auto-rebalance-cpu-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-cpu-weight 2" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+ "crs": {
+ "ha": "dynamic",
+ "ha-auto-rebalance": 1
+ }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats
new file mode 100644
index 0000000..0d0b3b2
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "cpu": 1.5, "mem": 42949672960 },
+ "vm:102": { "cpu": 0.5, "mem": 21474836480 },
+ "vm:103": { "cpu": 2.0, "mem": 8589934592 },
+ "vm:104": { "cpu": 1.0, "mem": 4294967296 },
+ "vm:105": { "cpu": 9.0, "mem": 5368709120 },
+ "vm:106": { "cpu": 6.0, "mem": 3221225472 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status
@@ -0,0 +1,5 @@
+{
+ "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect
new file mode 100644
index 0000000..352aa1c
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect
@@ -0,0 +1,62 @@
+info 0 hardware: starting simulation
+info 20 cmdlist: execute power node1 on
+info 20 node1/crm: status change startup => wait_for_quorum
+info 20 node1/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node2 on
+info 20 node2/crm: status change startup => wait_for_quorum
+info 20 node2/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node3 on
+info 20 node3/crm: status change startup => wait_for_quorum
+info 20 node3/lrm: status change startup => wait_for_agent_lock
+info 20 node1/crm: got lock 'ha_manager_lock'
+info 20 node1/crm: status change wait_for_quorum => master
+info 20 node1/crm: using scheduler mode 'dynamic'
+info 20 node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info 20 node1/crm: adding new service 'vm:101' on node 'node1'
+info 20 node1/crm: adding new service 'vm:102' on node 'node1'
+info 20 node1/crm: adding new service 'vm:103' on node 'node2'
+info 20 node1/crm: adding new service 'vm:104' on node 'node2'
+info 20 node1/crm: adding new service 'vm:105' on node 'node3'
+info 20 node1/crm: adding new service 'vm:106' on node 'node3'
+info 20 node1/crm: service 'vm:101': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:102': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:103': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:104': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:105': state changed from 'request_start' to 'started' (node = node3)
+info 20 node1/crm: service 'vm:106': state changed from 'request_start' to 'started' (node = node3)
+info 21 node1/lrm: got lock 'ha_agent_node1_lock'
+info 21 node1/lrm: status change wait_for_agent_lock => active
+info 21 node1/lrm: starting service vm:101
+info 21 node1/lrm: service status vm:101 started
+info 21 node1/lrm: starting service vm:102
+info 21 node1/lrm: service status vm:102 started
+info 22 node2/crm: status change wait_for_quorum => slave
+info 23 node2/lrm: got lock 'ha_agent_node2_lock'
+info 23 node2/lrm: status change wait_for_agent_lock => active
+info 23 node2/lrm: starting service vm:103
+info 23 node2/lrm: service status vm:103 started
+info 23 node2/lrm: starting service vm:104
+info 23 node2/lrm: service status vm:104 started
+info 24 node3/crm: status change wait_for_quorum => slave
+info 25 node3/lrm: got lock 'ha_agent_node3_lock'
+info 25 node3/lrm: status change wait_for_agent_lock => active
+info 25 node3/lrm: starting service vm:105
+info 25 node3/lrm: service status vm:105 started
+info 25 node3/lrm: starting service vm:106
+info 25 node3/lrm: service status vm:106 started
+info 120 cmdlist: execute datacenter crs set ha-auto-rebalance-cpu-weight 1
+info 220 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 1
+info 320 cmdlist: execute datacenter crs set ha-auto-rebalance-cpu-weight 2
+info 320 node1/crm: crs settings: 'ha-auto-rebalance-cpu-weight' changed from '1' to '2'
+info 360 node1/crm: auto rebalance - migrate vm:106 to node2 (expected change for imbalance from 33.3% to 4.4%)
+info 360 node1/crm: got crm command: migrate vm:106 node2
+info 360 node1/crm: migrate service 'vm:106' to node 'node2'
+info 360 node1/crm: service 'vm:106': state changed from 'started' to 'migrate' (node = node3, target = node2)
+info 365 node3/lrm: service vm:106 - start migrate to node 'node2'
+info 365 node3/lrm: service vm:106 - end migrate to node 'node2'
+info 380 node1/crm: service 'vm:106': state changed from 'migrate' to 'started' (node = node2)
+info 383 node2/lrm: starting service vm:106
+info 383 node2/lrm: service status vm:106 started
+info 920 hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "node": "node1", "state": "started" },
+ "vm:102": { "node": "node1", "state": "started" },
+ "vm:103": { "node": "node2", "state": "started" },
+ "vm:104": { "node": "node2", "state": "started" },
+ "vm:105": { "node": "node3", "state": "started" },
+ "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README
new file mode 100644
index 0000000..a9974a7
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README
@@ -0,0 +1,4 @@
+At equal weights (=1) load imbalance sits at 18.7%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that incrementing
+ha-auto-rebalance-mem-weight will eventually trigger a rebalancing migration
+from mem-heavy node3.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist
new file mode 100644
index 0000000..643ab0b
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist
@@ -0,0 +1,8 @@
+[
+ [ "power node1 on", "power node2 on", "power node3 on" ],
+ [ "datacenter crs set ha-auto-rebalance-cpu-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 2" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 3" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 4" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+ "crs": {
+ "ha": "dynamic",
+ "ha-auto-rebalance": 1
+ }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats
new file mode 100644
index 0000000..14740f1
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "cpu": 10.0, "mem": 3221225472 },
+ "vm:102": { "cpu": 5.0, "mem": 1073741824 },
+ "vm:103": { "cpu": 5.0, "mem": 3221225472 },
+ "vm:104": { "cpu": 3.0, "mem": 1073741824 },
+ "vm:105": { "cpu": 0.5, "mem": 32212254720 },
+ "vm:106": { "cpu": 0.5, "mem": 6442450944 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status
@@ -0,0 +1,5 @@
+{
+ "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect
new file mode 100644
index 0000000..596ef29
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect
@@ -0,0 +1,66 @@
+info 0 hardware: starting simulation
+info 20 cmdlist: execute power node1 on
+info 20 node1/crm: status change startup => wait_for_quorum
+info 20 node1/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node2 on
+info 20 node2/crm: status change startup => wait_for_quorum
+info 20 node2/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node3 on
+info 20 node3/crm: status change startup => wait_for_quorum
+info 20 node3/lrm: status change startup => wait_for_agent_lock
+info 20 node1/crm: got lock 'ha_manager_lock'
+info 20 node1/crm: status change wait_for_quorum => master
+info 20 node1/crm: using scheduler mode 'dynamic'
+info 20 node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info 20 node1/crm: adding new service 'vm:101' on node 'node1'
+info 20 node1/crm: adding new service 'vm:102' on node 'node1'
+info 20 node1/crm: adding new service 'vm:103' on node 'node2'
+info 20 node1/crm: adding new service 'vm:104' on node 'node2'
+info 20 node1/crm: adding new service 'vm:105' on node 'node3'
+info 20 node1/crm: adding new service 'vm:106' on node 'node3'
+info 20 node1/crm: service 'vm:101': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:102': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:103': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:104': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:105': state changed from 'request_start' to 'started' (node = node3)
+info 20 node1/crm: service 'vm:106': state changed from 'request_start' to 'started' (node = node3)
+info 21 node1/lrm: got lock 'ha_agent_node1_lock'
+info 21 node1/lrm: status change wait_for_agent_lock => active
+info 21 node1/lrm: starting service vm:101
+info 21 node1/lrm: service status vm:101 started
+info 21 node1/lrm: starting service vm:102
+info 21 node1/lrm: service status vm:102 started
+info 22 node2/crm: status change wait_for_quorum => slave
+info 23 node2/lrm: got lock 'ha_agent_node2_lock'
+info 23 node2/lrm: status change wait_for_agent_lock => active
+info 23 node2/lrm: starting service vm:103
+info 23 node2/lrm: service status vm:103 started
+info 23 node2/lrm: starting service vm:104
+info 23 node2/lrm: service status vm:104 started
+info 24 node3/crm: status change wait_for_quorum => slave
+info 25 node3/lrm: got lock 'ha_agent_node3_lock'
+info 25 node3/lrm: status change wait_for_agent_lock => active
+info 25 node3/lrm: starting service vm:105
+info 25 node3/lrm: service status vm:105 started
+info 25 node3/lrm: starting service vm:106
+info 25 node3/lrm: service status vm:106 started
+info 120 cmdlist: execute datacenter crs set ha-auto-rebalance-cpu-weight 1
+info 220 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 1
+info 320 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 2
+info 320 node1/crm: crs settings: 'ha-auto-rebalance-mem-weight' changed from '1' to '2'
+info 420 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 3
+info 420 node1/crm: crs settings: 'ha-auto-rebalance-mem-weight' changed from '2' to '3'
+info 520 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 4
+info 520 node1/crm: crs settings: 'ha-auto-rebalance-mem-weight' changed from '3' to '4'
+info 560 node1/crm: auto rebalance - migrate vm:106 to node2 (expected change for imbalance from 32.9% to 17.3%)
+info 560 node1/crm: got crm command: migrate vm:106 node2
+info 560 node1/crm: migrate service 'vm:106' to node 'node2'
+info 560 node1/crm: service 'vm:106': state changed from 'started' to 'migrate' (node = node3, target = node2)
+info 565 node3/lrm: service vm:106 - start migrate to node 'node2'
+info 565 node3/lrm: service vm:106 - end migrate to node 'node2'
+info 580 node1/crm: service 'vm:106': state changed from 'migrate' to 'started' (node = node2)
+info 583 node2/lrm: starting service vm:106
+info 583 node2/lrm: service status vm:106 started
+info 1120 hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "node": "node1", "state": "started" },
+ "vm:102": { "node": "node1", "state": "started" },
+ "vm:103": { "node": "node2", "state": "started" },
+ "vm:104": { "node": "node2", "state": "started" },
+ "vm:105": { "node": "node3", "state": "started" },
+ "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README
new file mode 100644
index 0000000..19cebf9
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README
@@ -0,0 +1,5 @@
+At equal weights (=1) load imbalance sits at 27.5%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that increasing the
+ha-auto-rebalance-mem-weight will increase load imbalance and trigger a
+rebalancing migration from mem-heavy node1 to node2, which sits low on either
+resource.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist
new file mode 100644
index 0000000..2144ffb
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist
@@ -0,0 +1,6 @@
+[
+ [ "power node1 on", "power node2 on", "power node3 on" ],
+ [ "datacenter crs set ha-auto-rebalance-cpu-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 2" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+ "crs": {
+ "ha": "dynamic",
+ "ha-auto-rebalance": 1
+ }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats
new file mode 100644
index 0000000..0d0b3b2
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "cpu": 1.5, "mem": 42949672960 },
+ "vm:102": { "cpu": 0.5, "mem": 21474836480 },
+ "vm:103": { "cpu": 2.0, "mem": 8589934592 },
+ "vm:104": { "cpu": 1.0, "mem": 4294967296 },
+ "vm:105": { "cpu": 9.0, "mem": 5368709120 },
+ "vm:106": { "cpu": 6.0, "mem": 3221225472 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status
@@ -0,0 +1,5 @@
+{
+ "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect
new file mode 100644
index 0000000..5fc901f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect
@@ -0,0 +1,62 @@
+info 0 hardware: starting simulation
+info 20 cmdlist: execute power node1 on
+info 20 node1/crm: status change startup => wait_for_quorum
+info 20 node1/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node2 on
+info 20 node2/crm: status change startup => wait_for_quorum
+info 20 node2/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node3 on
+info 20 node3/crm: status change startup => wait_for_quorum
+info 20 node3/lrm: status change startup => wait_for_agent_lock
+info 20 node1/crm: got lock 'ha_manager_lock'
+info 20 node1/crm: status change wait_for_quorum => master
+info 20 node1/crm: using scheduler mode 'dynamic'
+info 20 node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info 20 node1/crm: adding new service 'vm:101' on node 'node1'
+info 20 node1/crm: adding new service 'vm:102' on node 'node1'
+info 20 node1/crm: adding new service 'vm:103' on node 'node2'
+info 20 node1/crm: adding new service 'vm:104' on node 'node2'
+info 20 node1/crm: adding new service 'vm:105' on node 'node3'
+info 20 node1/crm: adding new service 'vm:106' on node 'node3'
+info 20 node1/crm: service 'vm:101': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:102': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:103': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:104': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:105': state changed from 'request_start' to 'started' (node = node3)
+info 20 node1/crm: service 'vm:106': state changed from 'request_start' to 'started' (node = node3)
+info 21 node1/lrm: got lock 'ha_agent_node1_lock'
+info 21 node1/lrm: status change wait_for_agent_lock => active
+info 21 node1/lrm: starting service vm:101
+info 21 node1/lrm: service status vm:101 started
+info 21 node1/lrm: starting service vm:102
+info 21 node1/lrm: service status vm:102 started
+info 22 node2/crm: status change wait_for_quorum => slave
+info 23 node2/lrm: got lock 'ha_agent_node2_lock'
+info 23 node2/lrm: status change wait_for_agent_lock => active
+info 23 node2/lrm: starting service vm:103
+info 23 node2/lrm: service status vm:103 started
+info 23 node2/lrm: starting service vm:104
+info 23 node2/lrm: service status vm:104 started
+info 24 node3/crm: status change wait_for_quorum => slave
+info 25 node3/lrm: got lock 'ha_agent_node3_lock'
+info 25 node3/lrm: status change wait_for_agent_lock => active
+info 25 node3/lrm: starting service vm:105
+info 25 node3/lrm: service status vm:105 started
+info 25 node3/lrm: starting service vm:106
+info 25 node3/lrm: service status vm:106 started
+info 120 cmdlist: execute datacenter crs set ha-auto-rebalance-cpu-weight 1
+info 220 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 1
+info 320 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 2
+info 320 node1/crm: crs settings: 'ha-auto-rebalance-mem-weight' changed from '1' to '2'
+info 360 node1/crm: auto rebalance - migrate vm:102 to node2 (expected change for imbalance from 33.3% to 3.8%)
+info 360 node1/crm: got crm command: migrate vm:102 node2
+info 360 node1/crm: migrate service 'vm:102' to node 'node2'
+info 360 node1/crm: service 'vm:102': state changed from 'started' to 'migrate' (node = node1, target = node2)
+info 361 node1/lrm: service vm:102 - start migrate to node 'node2'
+info 361 node1/lrm: service vm:102 - end migrate to node 'node2'
+info 380 node1/crm: service 'vm:102': state changed from 'migrate' to 'started' (node = node2)
+info 383 node2/lrm: starting service vm:102
+info 383 node2/lrm: service status vm:102 started
+info 920 hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "node": "node1", "state": "started" },
+ "vm:102": { "node": "node1", "state": "started" },
+ "vm:103": { "node": "node2", "state": "started" },
+ "vm:104": { "node": "node2", "state": "started" },
+ "vm:105": { "node": "node3", "state": "started" },
+ "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
--
2.47.3
next prev parent reply other threads:[~2026-10-21 9:01 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-21 8:59 [RFC PATCH-SERIES cluster/ha-manager/manager/proxmox{,-perl-rs} 00/15] fix #7993: implement weighted auto rebalancing Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox 01/15] resource-scheduling: scheduler: fix typos Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox 02/15] resource-scheduling: scheduler: abstract post-migration stats logic Dominik Rusovac
2026-10-21 12:49 ` Daniel Kral
2026-10-21 8:59 ` [PATCH proxmox 03/15] fix #7993: resource-scheduling: lift to weighted load balancing Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox 04/15] workspace,resource-scheduling: add proptest crate Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox 05/15] resource-scheduling: scheduler: proptest weights Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox-perl-rs 06/15] fix #7993: pve-rs: resource-scheduling: add weights to pve bindings Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-manager 07/15] fix #7993: ui: form: crs options: add weights Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-manager 08/15] ui: form: crs options: make options advanced Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-ha-manager 09/15] manager: remove obsolete argument Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-ha-manager 10/15] fix #7993: manager,usage: implement weighted auto rebalancing Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-ha-manager 11/15] sim: hardware: enable datacenter config changes at runtime Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-ha-manager 12/15] manager: log changes to crs settings Dominik Rusovac
2026-10-21 8:59 ` Dominik Rusovac [this message]
2026-10-21 8:59 ` [PATCH pve-ha-manager 14/15] manager: rename update_crs_scheduler_mode Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-cluster 15/15] fix #7993: datacenter config: add ha-auto-rebalance-{cpu,mem}-weight Dominik Rusovac
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261021085932.1196302-14-d.rusovac@proxmox.com \
--to=d.rusovac@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox