From: Dominik Rusovac <d.rusovac@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-ha-manager 13/15] test: add auto rebalance with weights test cases
Date: Wed, 21 Oct 2026 10:59:30 +0200 [thread overview]
Message-ID: <20261021085932.1196302-14-d.rusovac@proxmox.com> (raw)
In-Reply-To: <20261021085932.1196302-1-d.rusovac@proxmox.com>
Add tests documenting basic efficacy of auto rebalancing with weights.
Overview:
- test-crs-dynamic-auto-rebalance-with-increasing-bias: no rebalancing,
increment mem-weight until rebalancing is triggered
- test-crs-dynamic-auto-rebalance-with-{cpu,mem}-bias: trigger migration
that rebalances {cpu,mem} by incrementing {cpu,mem}-weight once
Signed-off-by: Dominik Rusovac <d.rusovac@proxmox.com>
---
.../README | 5 ++
.../cmdlist | 6 ++
.../datacenter.cfg | 6 ++
.../dynamic_service_stats | 8 +++
.../hardware_status | 5 ++
.../log.expect | 62 +++++++++++++++++
.../manager_status | 1 +
.../service_config | 8 +++
.../static_service_stats | 8 +++
.../README | 4 ++
.../cmdlist | 8 +++
.../datacenter.cfg | 6 ++
.../dynamic_service_stats | 8 +++
.../hardware_status | 5 ++
.../log.expect | 66 +++++++++++++++++++
.../manager_status | 1 +
.../service_config | 8 +++
.../static_service_stats | 8 +++
.../README | 5 ++
.../cmdlist | 6 ++
.../datacenter.cfg | 6 ++
.../dynamic_service_stats | 8 +++
.../hardware_status | 5 ++
.../log.expect | 62 +++++++++++++++++
.../manager_status | 1 +
.../service_config | 8 +++
.../static_service_stats | 8 +++
27 files changed, 332 insertions(+)
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config
create mode 100644 src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README
new file mode 100644
index 0000000..f67978e
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/README
@@ -0,0 +1,5 @@
+At equal weights (=1) load imbalance sits at 27.5%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that increasing the
+ha-auto-rebalance-cpu-weight will increase load imbalance and trigger a
+rebalancing migration from cpu-heavy node3 to node2, which sits low on either
+resource.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist
new file mode 100644
index 0000000..c29b4b6
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/cmdlist
@@ -0,0 +1,6 @@
+[
+ [ "power node1 on", "power node2 on", "power node3 on" ],
+ [ "datacenter crs set ha-auto-rebalance-cpu-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-cpu-weight 2" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+ "crs": {
+ "ha": "dynamic",
+ "ha-auto-rebalance": 1
+ }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats
new file mode 100644
index 0000000..0d0b3b2
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "cpu": 1.5, "mem": 42949672960 },
+ "vm:102": { "cpu": 0.5, "mem": 21474836480 },
+ "vm:103": { "cpu": 2.0, "mem": 8589934592 },
+ "vm:104": { "cpu": 1.0, "mem": 4294967296 },
+ "vm:105": { "cpu": 9.0, "mem": 5368709120 },
+ "vm:106": { "cpu": 6.0, "mem": 3221225472 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/hardware_status
@@ -0,0 +1,5 @@
+{
+ "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect
new file mode 100644
index 0000000..352aa1c
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/log.expect
@@ -0,0 +1,62 @@
+info 0 hardware: starting simulation
+info 20 cmdlist: execute power node1 on
+info 20 node1/crm: status change startup => wait_for_quorum
+info 20 node1/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node2 on
+info 20 node2/crm: status change startup => wait_for_quorum
+info 20 node2/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node3 on
+info 20 node3/crm: status change startup => wait_for_quorum
+info 20 node3/lrm: status change startup => wait_for_agent_lock
+info 20 node1/crm: got lock 'ha_manager_lock'
+info 20 node1/crm: status change wait_for_quorum => master
+info 20 node1/crm: using scheduler mode 'dynamic'
+info 20 node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info 20 node1/crm: adding new service 'vm:101' on node 'node1'
+info 20 node1/crm: adding new service 'vm:102' on node 'node1'
+info 20 node1/crm: adding new service 'vm:103' on node 'node2'
+info 20 node1/crm: adding new service 'vm:104' on node 'node2'
+info 20 node1/crm: adding new service 'vm:105' on node 'node3'
+info 20 node1/crm: adding new service 'vm:106' on node 'node3'
+info 20 node1/crm: service 'vm:101': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:102': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:103': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:104': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:105': state changed from 'request_start' to 'started' (node = node3)
+info 20 node1/crm: service 'vm:106': state changed from 'request_start' to 'started' (node = node3)
+info 21 node1/lrm: got lock 'ha_agent_node1_lock'
+info 21 node1/lrm: status change wait_for_agent_lock => active
+info 21 node1/lrm: starting service vm:101
+info 21 node1/lrm: service status vm:101 started
+info 21 node1/lrm: starting service vm:102
+info 21 node1/lrm: service status vm:102 started
+info 22 node2/crm: status change wait_for_quorum => slave
+info 23 node2/lrm: got lock 'ha_agent_node2_lock'
+info 23 node2/lrm: status change wait_for_agent_lock => active
+info 23 node2/lrm: starting service vm:103
+info 23 node2/lrm: service status vm:103 started
+info 23 node2/lrm: starting service vm:104
+info 23 node2/lrm: service status vm:104 started
+info 24 node3/crm: status change wait_for_quorum => slave
+info 25 node3/lrm: got lock 'ha_agent_node3_lock'
+info 25 node3/lrm: status change wait_for_agent_lock => active
+info 25 node3/lrm: starting service vm:105
+info 25 node3/lrm: service status vm:105 started
+info 25 node3/lrm: starting service vm:106
+info 25 node3/lrm: service status vm:106 started
+info 120 cmdlist: execute datacenter crs set ha-auto-rebalance-cpu-weight 1
+info 220 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 1
+info 320 cmdlist: execute datacenter crs set ha-auto-rebalance-cpu-weight 2
+info 320 node1/crm: crs settings: 'ha-auto-rebalance-cpu-weight' changed from '1' to '2'
+info 360 node1/crm: auto rebalance - migrate vm:106 to node2 (expected change for imbalance from 33.3% to 4.4%)
+info 360 node1/crm: got crm command: migrate vm:106 node2
+info 360 node1/crm: migrate service 'vm:106' to node 'node2'
+info 360 node1/crm: service 'vm:106': state changed from 'started' to 'migrate' (node = node3, target = node2)
+info 365 node3/lrm: service vm:106 - start migrate to node 'node2'
+info 365 node3/lrm: service vm:106 - end migrate to node 'node2'
+info 380 node1/crm: service 'vm:106': state changed from 'migrate' to 'started' (node = node2)
+info 383 node2/lrm: starting service vm:106
+info 383 node2/lrm: service status vm:106 started
+info 920 hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/service_config
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "node": "node1", "state": "started" },
+ "vm:102": { "node": "node1", "state": "started" },
+ "vm:103": { "node": "node2", "state": "started" },
+ "vm:104": { "node": "node2", "state": "started" },
+ "vm:105": { "node": "node3", "state": "started" },
+ "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-cpu-bias/static_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README
new file mode 100644
index 0000000..a9974a7
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/README
@@ -0,0 +1,4 @@
+At equal weights (=1) load imbalance sits at 18.7%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that incrementing
+ha-auto-rebalance-mem-weight will eventually trigger a rebalancing migration
+from mem-heavy node3.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist
new file mode 100644
index 0000000..643ab0b
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/cmdlist
@@ -0,0 +1,8 @@
+[
+ [ "power node1 on", "power node2 on", "power node3 on" ],
+ [ "datacenter crs set ha-auto-rebalance-cpu-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 2" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 3" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 4" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+ "crs": {
+ "ha": "dynamic",
+ "ha-auto-rebalance": 1
+ }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats
new file mode 100644
index 0000000..14740f1
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "cpu": 10.0, "mem": 3221225472 },
+ "vm:102": { "cpu": 5.0, "mem": 1073741824 },
+ "vm:103": { "cpu": 5.0, "mem": 3221225472 },
+ "vm:104": { "cpu": 3.0, "mem": 1073741824 },
+ "vm:105": { "cpu": 0.5, "mem": 32212254720 },
+ "vm:106": { "cpu": 0.5, "mem": 6442450944 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/hardware_status
@@ -0,0 +1,5 @@
+{
+ "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect
new file mode 100644
index 0000000..596ef29
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/log.expect
@@ -0,0 +1,66 @@
+info 0 hardware: starting simulation
+info 20 cmdlist: execute power node1 on
+info 20 node1/crm: status change startup => wait_for_quorum
+info 20 node1/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node2 on
+info 20 node2/crm: status change startup => wait_for_quorum
+info 20 node2/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node3 on
+info 20 node3/crm: status change startup => wait_for_quorum
+info 20 node3/lrm: status change startup => wait_for_agent_lock
+info 20 node1/crm: got lock 'ha_manager_lock'
+info 20 node1/crm: status change wait_for_quorum => master
+info 20 node1/crm: using scheduler mode 'dynamic'
+info 20 node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info 20 node1/crm: adding new service 'vm:101' on node 'node1'
+info 20 node1/crm: adding new service 'vm:102' on node 'node1'
+info 20 node1/crm: adding new service 'vm:103' on node 'node2'
+info 20 node1/crm: adding new service 'vm:104' on node 'node2'
+info 20 node1/crm: adding new service 'vm:105' on node 'node3'
+info 20 node1/crm: adding new service 'vm:106' on node 'node3'
+info 20 node1/crm: service 'vm:101': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:102': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:103': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:104': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:105': state changed from 'request_start' to 'started' (node = node3)
+info 20 node1/crm: service 'vm:106': state changed from 'request_start' to 'started' (node = node3)
+info 21 node1/lrm: got lock 'ha_agent_node1_lock'
+info 21 node1/lrm: status change wait_for_agent_lock => active
+info 21 node1/lrm: starting service vm:101
+info 21 node1/lrm: service status vm:101 started
+info 21 node1/lrm: starting service vm:102
+info 21 node1/lrm: service status vm:102 started
+info 22 node2/crm: status change wait_for_quorum => slave
+info 23 node2/lrm: got lock 'ha_agent_node2_lock'
+info 23 node2/lrm: status change wait_for_agent_lock => active
+info 23 node2/lrm: starting service vm:103
+info 23 node2/lrm: service status vm:103 started
+info 23 node2/lrm: starting service vm:104
+info 23 node2/lrm: service status vm:104 started
+info 24 node3/crm: status change wait_for_quorum => slave
+info 25 node3/lrm: got lock 'ha_agent_node3_lock'
+info 25 node3/lrm: status change wait_for_agent_lock => active
+info 25 node3/lrm: starting service vm:105
+info 25 node3/lrm: service status vm:105 started
+info 25 node3/lrm: starting service vm:106
+info 25 node3/lrm: service status vm:106 started
+info 120 cmdlist: execute datacenter crs set ha-auto-rebalance-cpu-weight 1
+info 220 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 1
+info 320 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 2
+info 320 node1/crm: crs settings: 'ha-auto-rebalance-mem-weight' changed from '1' to '2'
+info 420 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 3
+info 420 node1/crm: crs settings: 'ha-auto-rebalance-mem-weight' changed from '2' to '3'
+info 520 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 4
+info 520 node1/crm: crs settings: 'ha-auto-rebalance-mem-weight' changed from '3' to '4'
+info 560 node1/crm: auto rebalance - migrate vm:106 to node2 (expected change for imbalance from 32.9% to 17.3%)
+info 560 node1/crm: got crm command: migrate vm:106 node2
+info 560 node1/crm: migrate service 'vm:106' to node 'node2'
+info 560 node1/crm: service 'vm:106': state changed from 'started' to 'migrate' (node = node3, target = node2)
+info 565 node3/lrm: service vm:106 - start migrate to node 'node2'
+info 565 node3/lrm: service vm:106 - end migrate to node 'node2'
+info 580 node1/crm: service 'vm:106': state changed from 'migrate' to 'started' (node = node2)
+info 583 node2/lrm: starting service vm:106
+info 583 node2/lrm: service status vm:106 started
+info 1120 hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/service_config
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "node": "node1", "state": "started" },
+ "vm:102": { "node": "node1", "state": "started" },
+ "vm:103": { "node": "node2", "state": "started" },
+ "vm:104": { "node": "node2", "state": "started" },
+ "vm:105": { "node": "node3", "state": "started" },
+ "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-increasing-bias/static_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README
new file mode 100644
index 0000000..19cebf9
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/README
@@ -0,0 +1,5 @@
+At equal weights (=1) load imbalance sits at 27.5%, which is below the
+ha-auto-rebalance-threshold of 30%. Test that increasing the
+ha-auto-rebalance-mem-weight will increase load imbalance and trigger a
+rebalancing migration from mem-heavy node1 to node2, which sits low on either
+resource.
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist
new file mode 100644
index 0000000..2144ffb
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/cmdlist
@@ -0,0 +1,6 @@
+[
+ [ "power node1 on", "power node2 on", "power node3 on" ],
+ [ "datacenter crs set ha-auto-rebalance-cpu-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 1" ],
+ [ "datacenter crs set ha-auto-rebalance-mem-weight 2" ]
+]
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg
new file mode 100644
index 0000000..01c8114
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/datacenter.cfg
@@ -0,0 +1,6 @@
+{
+ "crs": {
+ "ha": "dynamic",
+ "ha-auto-rebalance": 1
+ }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats
new file mode 100644
index 0000000..0d0b3b2
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/dynamic_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "cpu": 1.5, "mem": 42949672960 },
+ "vm:102": { "cpu": 0.5, "mem": 21474836480 },
+ "vm:103": { "cpu": 2.0, "mem": 8589934592 },
+ "vm:104": { "cpu": 1.0, "mem": 4294967296 },
+ "vm:105": { "cpu": 9.0, "mem": 5368709120 },
+ "vm:106": { "cpu": 6.0, "mem": 3221225472 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status
new file mode 100644
index 0000000..4395b83
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/hardware_status
@@ -0,0 +1,5 @@
+{
+ "node1": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node2": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 },
+ "node3": { "power": "off", "network": "off", "maxcpu": 16, "maxmem": 68719476736 }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect
new file mode 100644
index 0000000..5fc901f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/log.expect
@@ -0,0 +1,62 @@
+info 0 hardware: starting simulation
+info 20 cmdlist: execute power node1 on
+info 20 node1/crm: status change startup => wait_for_quorum
+info 20 node1/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node2 on
+info 20 node2/crm: status change startup => wait_for_quorum
+info 20 node2/lrm: status change startup => wait_for_agent_lock
+info 20 cmdlist: execute power node3 on
+info 20 node3/crm: status change startup => wait_for_quorum
+info 20 node3/lrm: status change startup => wait_for_agent_lock
+info 20 node1/crm: got lock 'ha_manager_lock'
+info 20 node1/crm: status change wait_for_quorum => master
+info 20 node1/crm: using scheduler mode 'dynamic'
+info 20 node1/crm: node 'node1': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node2': state changed from 'unknown' => 'online'
+info 20 node1/crm: node 'node3': state changed from 'unknown' => 'online'
+info 20 node1/crm: adding new service 'vm:101' on node 'node1'
+info 20 node1/crm: adding new service 'vm:102' on node 'node1'
+info 20 node1/crm: adding new service 'vm:103' on node 'node2'
+info 20 node1/crm: adding new service 'vm:104' on node 'node2'
+info 20 node1/crm: adding new service 'vm:105' on node 'node3'
+info 20 node1/crm: adding new service 'vm:106' on node 'node3'
+info 20 node1/crm: service 'vm:101': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:102': state changed from 'request_start' to 'started' (node = node1)
+info 20 node1/crm: service 'vm:103': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:104': state changed from 'request_start' to 'started' (node = node2)
+info 20 node1/crm: service 'vm:105': state changed from 'request_start' to 'started' (node = node3)
+info 20 node1/crm: service 'vm:106': state changed from 'request_start' to 'started' (node = node3)
+info 21 node1/lrm: got lock 'ha_agent_node1_lock'
+info 21 node1/lrm: status change wait_for_agent_lock => active
+info 21 node1/lrm: starting service vm:101
+info 21 node1/lrm: service status vm:101 started
+info 21 node1/lrm: starting service vm:102
+info 21 node1/lrm: service status vm:102 started
+info 22 node2/crm: status change wait_for_quorum => slave
+info 23 node2/lrm: got lock 'ha_agent_node2_lock'
+info 23 node2/lrm: status change wait_for_agent_lock => active
+info 23 node2/lrm: starting service vm:103
+info 23 node2/lrm: service status vm:103 started
+info 23 node2/lrm: starting service vm:104
+info 23 node2/lrm: service status vm:104 started
+info 24 node3/crm: status change wait_for_quorum => slave
+info 25 node3/lrm: got lock 'ha_agent_node3_lock'
+info 25 node3/lrm: status change wait_for_agent_lock => active
+info 25 node3/lrm: starting service vm:105
+info 25 node3/lrm: service status vm:105 started
+info 25 node3/lrm: starting service vm:106
+info 25 node3/lrm: service status vm:106 started
+info 120 cmdlist: execute datacenter crs set ha-auto-rebalance-cpu-weight 1
+info 220 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 1
+info 320 cmdlist: execute datacenter crs set ha-auto-rebalance-mem-weight 2
+info 320 node1/crm: crs settings: 'ha-auto-rebalance-mem-weight' changed from '1' to '2'
+info 360 node1/crm: auto rebalance - migrate vm:102 to node2 (expected change for imbalance from 33.3% to 3.8%)
+info 360 node1/crm: got crm command: migrate vm:102 node2
+info 360 node1/crm: migrate service 'vm:102' to node 'node2'
+info 360 node1/crm: service 'vm:102': state changed from 'started' to 'migrate' (node = node1, target = node2)
+info 361 node1/lrm: service vm:102 - start migrate to node 'node2'
+info 361 node1/lrm: service vm:102 - end migrate to node 'node2'
+info 380 node1/crm: service 'vm:102': state changed from 'migrate' to 'started' (node = node2)
+info 383 node2/lrm: starting service vm:102
+info 383 node2/lrm: service status vm:102 started
+info 920 hardware: exit simulation - done
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status
new file mode 100644
index 0000000..0967ef4
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/manager_status
@@ -0,0 +1 @@
+{}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config
new file mode 100644
index 0000000..60cadca
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/service_config
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "node": "node1", "state": "started" },
+ "vm:102": { "node": "node1", "state": "started" },
+ "vm:103": { "node": "node2", "state": "started" },
+ "vm:104": { "node": "node2", "state": "started" },
+ "vm:105": { "node": "node3", "state": "started" },
+ "vm:106": { "node": "node3", "state": "started" }
+}
diff --git a/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats
new file mode 100644
index 0000000..9fcd31f
--- /dev/null
+++ b/src/test/test-crs-dynamic-auto-rebalance-with-mem-bias/static_service_stats
@@ -0,0 +1,8 @@
+{
+ "vm:101": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:102": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:103": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:104": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:105": { "maxcpu": 16.0, "maxmem": 68719476736 },
+ "vm:106": { "maxcpu": 16.0, "maxmem": 68719476736 }
+}
--
2.47.3
next prev parent reply other threads:[~2026-10-21 9:01 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-10-21 8:59 [RFC PATCH-SERIES cluster/ha-manager/manager/proxmox{,-perl-rs} 00/15] fix #7993: implement weighted auto rebalancing Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox 01/15] resource-scheduling: scheduler: fix typos Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox 02/15] resource-scheduling: scheduler: abstract post-migration stats logic Dominik Rusovac
2026-10-21 12:49 ` Daniel Kral
2026-10-21 8:59 ` [PATCH proxmox 03/15] fix #7993: resource-scheduling: lift to weighted load balancing Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox 04/15] workspace,resource-scheduling: add proptest crate Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox 05/15] resource-scheduling: scheduler: proptest weights Dominik Rusovac
2026-10-21 8:59 ` [PATCH proxmox-perl-rs 06/15] fix #7993: pve-rs: resource-scheduling: add weights to pve bindings Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-manager 07/15] fix #7993: ui: form: crs options: add weights Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-manager 08/15] ui: form: crs options: make options advanced Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-ha-manager 09/15] manager: remove obsolete argument Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-ha-manager 10/15] fix #7993: manager,usage: implement weighted auto rebalancing Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-ha-manager 11/15] sim: hardware: enable datacenter config changes at runtime Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-ha-manager 12/15] manager: log changes to crs settings Dominik Rusovac
2026-10-21 8:59 ` Dominik Rusovac [this message]
2026-10-21 8:59 ` [PATCH pve-ha-manager 14/15] manager: rename update_crs_scheduler_mode Dominik Rusovac
2026-10-21 8:59 ` [PATCH pve-cluster 15/15] fix #7993: datacenter config: add ha-auto-rebalance-{cpu,mem}-weight Dominik Rusovac
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20261021085932.1196302-14-d.rusovac@proxmox.com \
--to=d.rusovac@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.