* [PATCH proxmox-perl-rs/qemu-server/qemu-server-rs v3 0/3] vCPU binding for qemu guests
@ 2026-10-22 9:22 Elias Huhsovitz
2026-10-22 9:22 ` [PATCH qemu-server-rs v3 1/3] cpu: add pve-qemu-server-cpu crate for vCPU binding Elias Huhsovitz
` (2 more replies)
0 siblings, 3 replies; 4+ messages in thread
From: Elias Huhsovitz @ 2026-10-22 9:22 UTC (permalink / raw)
To: pve-devel; +Cc: Elias Huhsovitz
1. Overview
-----------
This series adds a convinient way to achive vCPU:CPU binding in PVE.
A new Rust crate, pve-qemu-server-cpu, exposes binding and validation via
PVE::RS::CPU::Bind, called from the VM start path after QEMU has
spawned its vCPU threads.
Five modes are added:
* none (default): no binding.
* manual: explicit vCPU-to-hostCPU mapping. The config string is a
semicolon-separated list of `<vcpu-set>:<hostcpu-set>` entries.
Example: `0:0;1:1;2:2;3:3` pins vCPU 0 to CPU 0, vCPU 1 to CPU 1,
etc. Multi-value sets use `a-b` ranges or `{a,b,c}` unions.
* one-to-one: pin each vCPU to its own physical core, chosen from a
user-supplied pool. The config string is an upper bound: the list of
host cores the vCPUs may be assigned to. SMT siblings are stripped
from that pool before assignment, so each vCPU gets its own distinct
physical core, not a hyperthread.
If the pool has N cores and the VM has M vCPUs, each vCPU is pinned
to exactly one of the N cores (N must be >= M, otherwise validation
fails). For example, with `0-3` on a 4-vCPU VM, vCPU 0 gets core 0,
vCPU 1 gets core 1, and so on. Each vCPU is fixed to a single core,
it does not float within the pool. With `0;1;4;5` on the same VM,
vCPU 2 is pinned to core 4 and vCPU 3 to core 5, skipping cores 2-3.
An empty string means "use all physical cores on the host" as the
upper bound. On a host with 16 cores, that pins an N-vCPU VM to
cores 0, 1, ..., N-1 — one core per vCPU, SMT stripped.
Unlike `manual`, you do not specify which core each vCPU gets; that
assignment is up to hwloc. Use `manual` if you need a specific
vCPU-to-core mapping.
* numa: bind the set of vCPUs of each guest vNUMA node to the set of
CPUs of a host NUMA node. The config string is a `;`-separated list
of host NUMA node indices, consumed in order: guest vNUMA 0 gets the
first host node, guest vNUMA 1 the second, and so on.
Each guest vNUMA node's vCPUs form a group that may run on any CPU
of the assigned host NUMA node — including its cores and SMT threads.
The host scheduler is free to migrate vCPUs within the node, but not
outside it. This is the intended trade-off: memory allocated by a
vCPU stays local to the host node it runs on (preserving NUMA
memory locality), while the scheduler still has room to balance
load across the node's cores and hyperthreads.
This is a set-to-set binding, not a per-vCPU assignment. All vCPUs
of a guest vNUMA node share the same host CPU pool. If you need
each vCPU pinned to its own distinct core, use one-to-one or manual
instead.
Example: `3;4` lets all vCPUs on guest vNUMA 0 run on any CPU of
host NUMA node 3, and all vCPUs on guest vNUMA 1 run on any CPU of
host NUMA node 4.
An empty string uses host NUMA nodes in ascending index order:
guest vNUMA 0 → host node 0, guest vNUMA 1 → host node 1, etc.
* numa-manual: explicit variant of numa. The config string is a
`;`-separated list of `<guest-vnuma>:<host-numa>` pairs. Each guest
vNUMA node's set of vCPUs is bound to the set of CPUs of the
specified host NUMA node.
Semantics are identical to numa — vCPUs of a guest vNUMA node may
be scheduled on any CPU of the assigned host node, but cannot leave
it. Only the node mapping is explicit here.
Example: `0:2;1:3` binds all vCPUs on guest vNUMA 0 to host NUMA
node 2's CPUs, and all vCPUs on guest vNUMA 1 to host NUMA node 3's
CPUs.
Configuration is via two new VM config keys, cpu-binding-mode
and cpu-binding-cfg. Host CPU and NUMA indices are OS indices
(PU P#N from lstopo -p).
2. Prerequisites
----------------
The series is based on Dominik's PCI series [0].
This patch adds a new build dependency: libhwloc-dev. It is
declared in pve-qemu-server-rs/pve-qemu-server-cpu/debian/
debcargo.toml.
3. Request for feedback
-----------------------
I am looking for feedback, especially on:
* whether adding hwloc as a dependency is acceptable for
qemu-server.
* whether binding belongs in a dedicated crate or should live
closer to qemu-server.
* the config schema and the chosen mode split.
* naming.
* general design decisions (e.g. if rust is the right choice here)
4. Why hwloc
------------
hwloc [4] is widely used, well maintained, small and already available
on Debian (ver 2.12.0-4 at the time of writing this).
It abstracts hardware topology discovery across architectures and
CPU generations, including SMT, cache hierarchy, NUMA nodes, and
CPU kinds.
Without hwloc we would parse topology ourselves, from sysfs or
from arch-specific interfaces. Every new CPU family would
become our maintenance burden, and parser bugs would surface
only as binding failures on customer hardware.
hwloc also unlocks further performance work requiring topology
data:
* NUMA-aware memory placement (bind memory to a node)
* cache-aware vCPU placement (group by shared L2/L3)
* heterogeneous CPU support (Intel P/E cores) via CPU kinds
* distance-aware placement (latency/bandwidth between nodes)
The hwloc FAQ has a section for exactly this question:
"I only need binding, or the number of cores, why should I use
hwloc?" [1].
5. Why not hwlocality
---------------------
hwlocality [2,3] is the idiomatic high-level Rust binding to
hwloc. It is still very new (1.0.0-alpha) and not packaged in
Debian. Packaging it would require packaging its full
transitive dependency tree, several of which are also not yet
in Debian. That scope is too much for this feature IMO.
Our FFI bindings are a minimal hand-rolled subset of hwloc's
public C API, matching what hwlocality-sys exposes (I have
literally taken the code from hwlocality-sys [5]). Migration
to hwlocality later is straightforward: swap the bindings
module for the crate and delete the wrapper types.
6. Affinity settings
--------------------
The existing 'affinity' option is not integrated with this
feature yet. IMO these two are conceptually different: affinity
in PVE is a list of allowed CPUs; binding is a per-vCPU placement.
A single flat list cannot express per-vCPU intent. Also, the affinity
code lives purely in perl.
For now this binding code is experimental and should be used
without 'affinity'. Running taskset alongside the Rust binder
would race on the same threads and is hard to reason about.
Long-term I would like to move affinity handling into Rust as
well, so both features share topology-based validation and a
common binding backend. For this patch, shipping binding
independently and harmonizing with affinity later seems
cleanest.
7. Reproducibility
------------------
The original RFC [6] relied on 'affinity' to limit the CPU set and
kept the vCPU-to-CPU assignment in a Perl hash. Two configs
that looked identical could produce different placement,
depending on hash ordering and how affinity was set.
To make placement reproducible from the config file alone, the
binding logic lives entirely in Rust and the assignment is
read from cpu-binding-cfg. Two VMs with the same config on the same host
should get the same placement. (unless there is some randomness in the
hwloc_bitmap_singlify_per_core function that i am unaware of)
8. Basic usage
--------------
# Explicit per-vCPU binding
qm set <vmid> --cpu-binding-mode manual
qm set <vmid> --cpu-binding-cfg "0:0;1:1;2:2;3:3"
# One physical core per vCPU, let hwloc place them
qm set <vmid> --cpu-binding-mode one-to-one
qm set <vmid> --cpu-binding-cfg "0-3"
# Guest NUMA 0 -> host NUMA 2, guest NUMA 1 -> host NUMA 3
qm set <vmid> --cpu-binding-mode numa-manual
qm set <vmid> --cpu-binding-cfg "0:2;1:3"
# Automatic numa binding:
#Guest NUMA 0 -> host NUMA 2, guest NUMA 1 -> host NUMA 3
qm set <vmid> --cpu-binding-mode numa
qm set <vmid> --cpu-binding-cfg "2-3"
Without cpu-binding-mode the VM behaves as before. Validation
runs at qm set time and at VM start; binding itself runs after
QEMU spawns the vCPU threads.
9. What is not in this PATCH
----------------------------
* memory_binding: IMO this is best handled by qemu directly and already
possible via, e.g.:
numa0: cpus=0-1,hostnodes=0,memory=2048,policy=bind
numa1: cpus=2-3,hostnodes=1,memory=2048,policy=bind
numa2: cpus=4-5,hostnodes=2,memory=2048,policy=bind
numa3: cpus=6-7,hostnodes=3,memory=2048,policy=bind
What we need is a convinient way to automatically apply the correct
memory binding for a given CPU Binding config.
* Qemu I/O Thread control/binding
* CPU Isolation
I plan on adding most of the features requested in the forum post [7].
10. Previous versions
---------------------
v2: https://lore.proxmox.com/pve-devel/20260921095404.61552-1-e.huhsovitz@proxmox.com/
v1: https://lore.proxmox.com/pve-devel/20260217114813.2063770-1-d.csapak@proxmox.com/
(Credits to Dominik)
11. References
-------------
[0] https://lore.proxmox.com/pve-devel/20261015081041.743422-1-d.csapak@proxmox.com
[1] https://hwloc.readthedocs.io/en/hwloc-2.15.0/doxygen/html/faq.html
[2] https://crates.io/crates/hwlocality
[3] https://github.com/HadrienG2/hwlocality
[4] https://github.com/open-mpi/hwloc
[5] https://github.com/HadrienG2/hwlocality/tree/main/hwlocality-sys
[6] https://lore.proxmox.com/pve-devel/20260217114813.2063770-1-d.csapak@proxmox.com/
[7] https://forum.proxmox.com/threads/hey-proxmox-community-lets-talk-about-resources-isolation.124256/
Thank you to whoever reviews this! :)
Summary of changes
-------------
pve-qemu-server-rs:
Elias Huhsovitz (1):
cpu: add pve-qemu-server-cpu crate for vCPU binding
Cargo.toml | 2 +-
build.sh | 2 +-
pve-qemu-server-cpu/Cargo.toml | 16 +
pve-qemu-server-cpu/debian/changelog | 5 +
pve-qemu-server-cpu/debian/control | 41 ++
pve-qemu-server-cpu/debian/copyright | 18 +
pve-qemu-server-cpu/debian/debcargo.toml | 9 +
pve-qemu-server-cpu/src/bind.rs | 478 ++++++++++++++++++++++
pve-qemu-server-cpu/src/error.rs | 38 ++
pve-qemu-server-cpu/src/hwloc/bindings.rs | 171 ++++++++
pve-qemu-server-cpu/src/hwloc/mod.rs | 222 ++++++++++
pve-qemu-server-cpu/src/lib.rs | 92 +++++
pve-qemu-server-cpu/src/qmp_cpu.rs | 99 +++++
13 files changed, 1191 insertions(+), 2 deletions(-)
create mode 100644 pve-qemu-server-cpu/Cargo.toml
create mode 100644 pve-qemu-server-cpu/debian/changelog
create mode 100644 pve-qemu-server-cpu/debian/control
create mode 100644 pve-qemu-server-cpu/debian/copyright
create mode 100644 pve-qemu-server-cpu/debian/debcargo.toml
create mode 100644 pve-qemu-server-cpu/src/bind.rs
create mode 100644 pve-qemu-server-cpu/src/error.rs
create mode 100644 pve-qemu-server-cpu/src/hwloc/bindings.rs
create mode 100644 pve-qemu-server-cpu/src/hwloc/mod.rs
create mode 100644 pve-qemu-server-cpu/src/lib.rs
create mode 100644 pve-qemu-server-cpu/src/qmp_cpu.rs
proxmox-perl-rs:
Elias Huhsovitz (1):
pve: cpu: add PVE::RS::CPU::Bind perl bindings
pve-rs/Cargo.toml | 1 +
pve-rs/Makefile | 1 +
pve-rs/src/bindings/cpu/bind.rs | 40 +++++++++++++++++++++++++++++++++
pve-rs/src/bindings/cpu/mod.rs | 2 ++
pve-rs/src/bindings/mod.rs | 3 +++
5 files changed, 47 insertions(+)
create mode 100644 pve-rs/src/bindings/cpu/bind.rs
create mode 100644 pve-rs/src/bindings/cpu/mod.rs
qemu-server:
Elias Huhsovitz (1):
cpu: add CPU binding
src/PVE/QemuServer.pm | 36 +++++++++
src/PVE/QemuServer/CPUBinding.pm | 133 +++++++++++++++++++++++++++++++
src/PVE/QemuServer/Makefile | 1 +
3 files changed, 170 insertions(+)
create mode 100644 src/PVE/QemuServer/CPUBinding.pm
Summary over all repositories:
21 files changed, 1408 insertions(+), 2 deletions(-)
--
Generated by murpp 0.12.0
^ permalink raw reply [flat|nested] 4+ messages in thread
* [PATCH qemu-server-rs v3 1/3] cpu: add pve-qemu-server-cpu crate for vCPU binding
2026-10-22 9:22 [PATCH proxmox-perl-rs/qemu-server/qemu-server-rs v3 0/3] vCPU binding for qemu guests Elias Huhsovitz
@ 2026-10-22 9:22 ` Elias Huhsovitz
2026-10-22 9:22 ` [PATCH proxmox-perl-rs v3 2/3] pve: cpu: add PVE::RS::CPU::Bind perl bindings Elias Huhsovitz
2026-10-22 9:22 ` [PATCH qemu-server v3 3/3] cpu: add CPU binding Elias Huhsovitz
2 siblings, 0 replies; 4+ messages in thread
From: Elias Huhsovitz @ 2026-10-22 9:22 UTC (permalink / raw)
To: pve-devel; +Cc: Elias Huhsovitz
Add a new library crate that implements CPU binding on top of libhwloc.
Two public entry points are exposed:
* validate_binding(mode, sockets, cores, cfg) - pure pre-start
validation. Checks cfg syntax, host CPU existence, vCPU range, and
NUMA node existence against the local topology. No side effects.
* bind_vcpus(mode, cfg, cpuinfo_json) - applies the binding to a running
QEMU process. Consumes the query-cpus-fast QMP response and binds each
vCPU thread by kernel TID.
Five modes are implemented:
* none: no binding
* manual: explicit vCPU:hostCPU mapping
* one-to-one: one vCPU per physical core, SMT stripped
* numa: per-vNUMA-node set-to-set binding
* numa-manual: explicit vNUMA->host NUMA mapping
hwloc is accessed via a small FFI subset (src/bindings.rs).
Thread binding uses hwloc_linux_set_tid_cpubind, which accepts a kernel
TID directly. The TID used is reported by qemu via query-cpus-fast.
Signed-off-by: Elias Huhsovitz <e.huhsovitz@proxmox.com>
---
Cargo.toml | 2 +-
build.sh | 2 +-
pve-qemu-server-cpu/Cargo.toml | 16 +
pve-qemu-server-cpu/debian/changelog | 5 +
pve-qemu-server-cpu/debian/control | 41 ++
pve-qemu-server-cpu/debian/copyright | 18 +
pve-qemu-server-cpu/debian/debcargo.toml | 9 +
pve-qemu-server-cpu/src/bind.rs | 478 ++++++++++++++++++++++
pve-qemu-server-cpu/src/error.rs | 38 ++
pve-qemu-server-cpu/src/hwloc/bindings.rs | 171 ++++++++
pve-qemu-server-cpu/src/hwloc/mod.rs | 222 ++++++++++
pve-qemu-server-cpu/src/lib.rs | 92 +++++
pve-qemu-server-cpu/src/qmp_cpu.rs | 99 +++++
13 files changed, 1191 insertions(+), 2 deletions(-)
create mode 100644 pve-qemu-server-cpu/Cargo.toml
create mode 100644 pve-qemu-server-cpu/debian/changelog
create mode 100644 pve-qemu-server-cpu/debian/control
create mode 100644 pve-qemu-server-cpu/debian/copyright
create mode 100644 pve-qemu-server-cpu/debian/debcargo.toml
create mode 100644 pve-qemu-server-cpu/src/bind.rs
create mode 100644 pve-qemu-server-cpu/src/error.rs
create mode 100644 pve-qemu-server-cpu/src/hwloc/bindings.rs
create mode 100644 pve-qemu-server-cpu/src/hwloc/mod.rs
create mode 100644 pve-qemu-server-cpu/src/lib.rs
create mode 100644 pve-qemu-server-cpu/src/qmp_cpu.rs
diff --git a/Cargo.toml b/Cargo.toml
index 71bc8e5..c100e5a 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,5 +1,6 @@
[workspace]
members = [
+ "pve-qemu-server-cpu",
"pve-qemu-server-pci"
]
exclude = [
@@ -26,4 +27,3 @@ strum = { version = "0.26", features = ["derive"] }
# proxmox dependencies
pve-api-types = "8.1"
-
diff --git a/build.sh b/build.sh
index 3c2293b..c3ee7b6 100755
--- a/build.sh
+++ b/build.sh
@@ -28,7 +28,7 @@ debcargo package \
--no-overlay-write-back \
--directory "$PWD/"${BUILDDIR}"/${CRATE}" \
"${CRATE}" \
- "$(dpkg-parsechangelog -l "${CRATE}/debian/changelog" -SVersion | sed -e 's/-.*//')"
+ "$(dpkg-parsechangelog -l "${CRATE}/debian/changelog" -SVersion | sed -e 's/-.*//' -e 's/~/-/')"
cd "${BUILDDIR}/${CRATE}"
rm -f debian/source/format.debcargo.hint
diff --git a/pve-qemu-server-cpu/Cargo.toml b/pve-qemu-server-cpu/Cargo.toml
new file mode 100644
index 0000000..b236796
--- /dev/null
+++ b/pve-qemu-server-cpu/Cargo.toml
@@ -0,0 +1,16 @@
+[package]
+name = "pve-qemu-server-cpu"
+version = "0.1.0"
+authors.workspace = true
+edition.workspace = true
+license.workspace = true
+repository.workspace = true
+homepage.workspace = true
+exclude.workspace = true
+rust-version.workspace = true
+
+[dependencies]
+serde = { version = "1", features = ["derive"] }
+serde_json = "1"
+thiserror = "2"
+libc = "0.2"
diff --git a/pve-qemu-server-cpu/debian/changelog b/pve-qemu-server-cpu/debian/changelog
new file mode 100644
index 0000000..3731cfd
--- /dev/null
+++ b/pve-qemu-server-cpu/debian/changelog
@@ -0,0 +1,5 @@
+rust-pve-qemu-server-cpu (0.1.0) trixie; urgency=medium
+
+ * initial release
+
+ -- Proxmox Support Team <support@proxmox.com> Tue, 01 Sep 2026 14:02:25 +0200
diff --git a/pve-qemu-server-cpu/debian/control b/pve-qemu-server-cpu/debian/control
new file mode 100644
index 0000000..1298d61
--- /dev/null
+++ b/pve-qemu-server-cpu/debian/control
@@ -0,0 +1,41 @@
+Source: rust-pve-qemu-server-cpu
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo,
+ libhwloc-dev (>= 2.12)
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.94) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-libc-0.2+default-dev <!nocheck>,
+ librust-serde-1+default-dev <!nocheck>,
+ librust-serde-1+derive-dev <!nocheck>,
+ librust-serde-json-1+default-dev <!nocheck>,
+ librust-thiserror-2+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.2
+Vcs-Git: git://git.proxmox.com/git/pve-qemu-server-rs.git
+Vcs-Browser: https://git.proxmox.com/?p=pve-qemu-server-rs.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: pve-qemu-server-cpu
+
+Package: librust-pve-qemu-server-cpu-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-libc-0.2+default-dev,
+ librust-serde-1+default-dev,
+ librust-serde-1+derive-dev,
+ librust-serde-json-1+default-dev,
+ librust-thiserror-2+default-dev
+Provides:
+ librust-pve-qemu-server-cpu+default-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0+default-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0.1-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0.1+default-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0.1.0-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0.1.0+default-dev (= ${binary:Version})
+Description: Rust crate "pve-qemu-server-cpu" - Rust source code
+ Source code for Debianized Rust crate "pve-qemu-server-cpu"
diff --git a/pve-qemu-server-cpu/debian/copyright b/pve-qemu-server-cpu/debian/copyright
new file mode 100644
index 0000000..01138fa
--- /dev/null
+++ b/pve-qemu-server-cpu/debian/copyright
@@ -0,0 +1,18 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+
+Files:
+ *
+Copyright: 2026 Proxmox Server Solutions GmbH <support@proxmox.com>
+License: AGPL-3.0-or-later
+ This program is free software: you can redistribute it and/or modify it under
+ the terms of the GNU Affero General Public License as published by the Free
+ Software Foundation, either version 3 of the License, or (at your option) any
+ later version.
+ .
+ This program is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+ details.
+ .
+ You should have received a copy of the GNU Affero General Public License along
+ with this program. If not, see <https://www.gnu.org/licenses/>.
diff --git a/pve-qemu-server-cpu/debian/debcargo.toml b/pve-qemu-server-cpu/debian/debcargo.toml
new file mode 100644
index 0000000..23feea4
--- /dev/null
+++ b/pve-qemu-server-cpu/debian/debcargo.toml
@@ -0,0 +1,9 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+allow_prerelease_deps = true
+
+[source]
+vcs_git = "git://git.proxmox.com/git/pve-qemu-server-rs.git"
+vcs_browser = "https://git.proxmox.com/?p=pve-qemu-server-rs.git"
+build_depends = ["libhwloc-dev (>= 2.12)"]
diff --git a/pve-qemu-server-cpu/src/bind.rs b/pve-qemu-server-cpu/src/bind.rs
new file mode 100644
index 0000000..4f2927d
--- /dev/null
+++ b/pve-qemu-server-cpu/src/bind.rs
@@ -0,0 +1,478 @@
+use std::collections::HashSet;
+
+use crate::hwloc::{self, CpuSet, Topology};
+use crate::qmp_cpu::QmpCpu;
+use crate::BindError;
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+struct Cpu(u32);
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+struct NumaNodeId(u32);
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+struct Vcpu(u32);
+
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+struct VnumaNode {
+ id: NumaNodeId,
+ cpus: Vec<Vcpu>,
+}
+
+#[derive(Debug, Clone)]
+struct BindRule {
+ vcpus: Vec<Vcpu>,
+ cpuset: CpuSet,
+}
+
+#[derive(Debug, Clone)]
+struct NumaBindRule {
+ vnode: VnumaNode,
+ host_node: NumaNodeId,
+}
+
+impl From<u32> for Cpu {
+ fn from(v: u32) -> Self {
+ Self(v)
+ }
+}
+impl From<u32> for Vcpu {
+ fn from(v: u32) -> Self {
+ Self(v)
+ }
+}
+impl From<u32> for NumaNodeId {
+ fn from(v: u32) -> Self {
+ Self(v)
+ }
+}
+
+trait BindableRule {
+ fn vcpus(&self) -> &[Vcpu];
+ fn resolve_cpuset(&self, topo: &Topology) -> Result<CpuSet, BindError>;
+}
+
+impl BindableRule for BindRule {
+ fn vcpus(&self) -> &[Vcpu] {
+ &self.vcpus
+ }
+ fn resolve_cpuset(&self, _topo: &Topology) -> Result<CpuSet, BindError> {
+ Ok(self.cpuset.clone())
+ }
+}
+
+impl BindableRule for NumaBindRule {
+ fn vcpus(&self) -> &[Vcpu] {
+ &self.vnode.cpus
+ }
+ fn resolve_cpuset(&self, topo: &Topology) -> Result<CpuSet, BindError> {
+ hwloc::numa_node_cpuset(topo, self.host_node.0).ok_or(BindError::UnknownHostCpu(self.host_node.0))
+ }
+}
+
+fn bind(topo: &Topology, vcpu: &Vcpu, tid: u32, set: &CpuSet) -> Result<(), BindError> {
+ // Note: DO NOT singlify here — we want the scheduler to be free to move
+ // the vCPU within the specified set.
+ hwloc::bind_thread(topo, tid as i32, set).map_err(|message| BindError::Bind {
+ vcpu: vcpu.0,
+ message,
+ })
+}
+
+fn bind_rules<R: BindableRule>(
+ topo: &Topology,
+ rules: &[R],
+ qmp_cpuinfo: &[QmpCpu],
+) -> Result<(), BindError> {
+ for rule in rules {
+ let set = rule.resolve_cpuset(topo)?;
+ for vcpu in rule.vcpus() {
+ let tid = qmp_cpuinfo
+ .get(vcpu.0 as usize)
+ .ok_or(BindError::ThreadNotFound(vcpu.0))?
+ .thread_id;
+ bind(topo, vcpu, tid, &set)?;
+ }
+ }
+ Ok(())
+}
+
+pub enum Mode {
+ None,
+ Manual,
+ OneToOne,
+ Numa,
+ NumaManual,
+}
+
+impl std::str::FromStr for Mode {
+ type Err = BindError;
+ fn from_str(s: &str) -> Result<Self, Self::Err> {
+ match s {
+ "none" => Ok(Mode::None),
+ "manual" => Ok(Mode::Manual),
+ "one-to-one" => Ok(Mode::OneToOne),
+ "numa" => Ok(Mode::Numa),
+ "numa-manual" => Ok(Mode::NumaManual),
+ other => Err(BindError::BadSyntax(format!("unknown cpu-bind-mode '{other}'"))),
+ }
+ }
+}
+
+fn to_cpuset(cpus: &[u32]) -> CpuSet {
+ let mut set = CpuSet::new();
+ for &c in cpus {
+ set.set(c);
+ }
+ set
+}
+
+fn parse_cpu_binding_map(map: &str) -> Result<Vec<BindRule>, BindError> {
+ if map.is_empty() {
+ return Err(BindError::BadSyntax("binding map is empty".into()));
+ }
+
+ let mut rules = Vec::new();
+ for entry in map.split(';') {
+ let (vcpu_side, host_side) = entry
+ .split_once(':')
+ .ok_or_else(|| BindError::MissingColon(entry.to_string()))?;
+
+ let vcpus: Vec<Vcpu> = parse_side(vcpu_side)?.into_iter().map(Vcpu::from).collect();
+ let cpuset = to_cpuset(&parse_side(host_side)?);
+
+ rules.push(BindRule { vcpus, cpuset });
+ }
+ Ok(rules)
+}
+
+fn parse_side(s: &str) -> Result<Vec<u32>, BindError> {
+ if let Some(inner) = s.strip_prefix('{').and_then(|r| r.strip_suffix('}')) {
+ let mut out = Vec::new();
+ for part in inner.split(',') {
+ out.extend(parse_element(part)?);
+ }
+ Ok(out)
+ } else {
+ if s.contains(',') {
+ return Err(BindError::BadSyntax(format!(
+ "unions require braces: write '{{0,1}}', not '{s}'"
+ )));
+ }
+ parse_element(s)
+ }
+}
+
+fn parse_element(s: &str) -> Result<Vec<u32>, BindError> {
+ if let Some((a, b)) = s.split_once('-') {
+ let a: u32 = a
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad index in '{s}'")))?;
+ let b: u32 = b
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad index in '{s}'")))?;
+ if a > b {
+ return Err(BindError::BadSyntax(format!("reversed range '{s}'")));
+ }
+ Ok((a..=b).collect())
+ } else {
+ Ok(vec![s
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad index '{s}'")))?])
+ }
+}
+
+fn parse_index_list<T: From<u32>>(s: &str) -> Result<Vec<T>, BindError> {
+ if s.is_empty() {
+ return Ok(Vec::new());
+ }
+ let mut out = Vec::new();
+ for part in s.split(';') {
+ out.extend(parse_element(part)?.into_iter().map(T::from));
+ }
+ Ok(out)
+}
+
+fn parse_numa_binding_map(map: &str) -> Result<Vec<(NumaNodeId, NumaNodeId)>, BindError> {
+ if map.is_empty() {
+ return Err(BindError::BadSyntax("numa binding map is empty".into()));
+ }
+ let mut pairs = Vec::new();
+ for entry in map.split(';') {
+ let entry = entry.trim();
+ let (v, h) = entry
+ .split_once(':')
+ .ok_or_else(|| BindError::MissingColon(entry.to_string()))?;
+ let vnode: u32 = v
+ .trim()
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad vNUMA node in '{entry}'")))?;
+ let hnode: u32 = h
+ .trim()
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad host NUMA node in '{entry}'")))?;
+ pairs.push((NumaNodeId(vnode), NumaNodeId(hnode)));
+ }
+ Ok(pairs)
+}
+
+fn collect_vnuma_nodes(qmp_cpuinfo: &[QmpCpu]) -> Vec<VnumaNode> {
+ let max_node = qmp_cpuinfo
+ .iter()
+ .map(|c| c.props.node_id)
+ .max()
+ .unwrap_or(0);
+ let mut buckets: Vec<Vec<Vcpu>> = vec![Vec::new(); (max_node + 1) as usize];
+ for cpu in qmp_cpuinfo {
+ buckets[cpu.props.node_id as usize].push(Vcpu::from(cpu.cpu_index));
+ }
+ buckets
+ .into_iter()
+ .enumerate()
+ .map(|(id, cpus)| VnumaNode {
+ id: NumaNodeId(id as u32),
+ cpus,
+ })
+ .collect()
+}
+
+fn build_numa_rules(
+ pairs: &[(NumaNodeId, NumaNodeId)],
+ vnuma_nodes: &[VnumaNode],
+) -> Result<Vec<NumaBindRule>, BindError> {
+ let mut rules = Vec::with_capacity(pairs.len());
+ for (vnode_id, hnode_id) in pairs {
+ let vnode = vnuma_nodes
+ .get(vnode_id.0 as usize)
+ .ok_or_else(|| {
+ BindError::BadSyntax(format!(
+ "vNUMA node {} does not exist in this guest",
+ vnode_id.0
+ ))
+ })?
+ .clone();
+ rules.push(NumaBindRule {
+ vnode,
+ host_node: *hnode_id,
+ });
+ }
+ Ok(rules)
+}
+
+pub fn bind_manual(binding_map: &str, qmp_cpuinfo: &[QmpCpu]) -> Result<(), BindError> {
+ let rules = parse_cpu_binding_map(binding_map)?;
+ let topo = Topology::new()
+ .map_err(|e| BindError::TopologyInit(format!("topology init failed: {e}")))?;
+ bind_rules(&topo, &rules, qmp_cpuinfo)
+}
+
+pub fn bind_one_to_one(binding_list: &str, qmp_cpuinfo: &[QmpCpu]) -> Result<(), BindError> {
+ // TODO: use complex distribution logic like:
+ // https://github.com/open-mpi/hwloc/blob/master/include/hwloc/helper.h::hwloc_distrib()
+ // https://github.com/HadrienG2/hwlocality/blob/main/src/topology/mod.rs:distribute_items()
+ let topo = Topology::new()
+ .map_err(|e| BindError::TopologyInit(format!("topology init failed: {e}")))?;
+
+ let mut allowed: CpuSet = if binding_list.is_empty() {
+ topo.allowed_cpuset()
+ } else {
+ let cpus: Vec<Cpu> = parse_index_list(binding_list)?;
+ let mut s = CpuSet::new();
+ for c in cpus {
+ s.set(c.0);
+ }
+ s
+ };
+
+ // One vCPU per physical core: strip SMT siblings.
+ allowed.singlify_per_core(&topo);
+
+ let available: Vec<u32> = allowed.iter_set().collect();
+
+ if available.len() < qmp_cpuinfo.len() {
+ return Err(BindError::BadSyntax(format!(
+ "one-to-one mode requires {} physical cores but only {} are available",
+ qmp_cpuinfo.len(),
+ available.len(),
+ )));
+ }
+
+ let mut rules: Vec<BindRule> = Vec::with_capacity(qmp_cpuinfo.len());
+ for (i, cpu) in available.iter().take(qmp_cpuinfo.len()).enumerate() {
+ rules.push(BindRule {
+ vcpus: vec![Vcpu::from(qmp_cpuinfo[i].cpu_index)],
+ cpuset: CpuSet::single(*cpu),
+ });
+ }
+
+ bind_rules(&topo, &rules, qmp_cpuinfo)
+}
+
+pub fn bind_numa_manual(binding_map: &str, qmp_cpuinfo: &[QmpCpu]) -> Result<(), BindError> {
+ let topo = Topology::new()
+ .map_err(|e| BindError::TopologyInit(format!("topology init failed: {e}")))?;
+ let pairs = parse_numa_binding_map(binding_map)?;
+ let vnuma_nodes = collect_vnuma_nodes(qmp_cpuinfo);
+ let rules = build_numa_rules(&pairs, &vnuma_nodes)?;
+ bind_rules(&topo, &rules, qmp_cpuinfo)
+}
+
+pub fn bind_numa(binding_list: &str, qmp_cpuinfo: &[QmpCpu]) -> Result<(), BindError> {
+ let topo = Topology::new()
+ .map_err(|e| BindError::TopologyInit(format!("topology init failed: {e}")))?;
+ let vnuma_nodes = collect_vnuma_nodes(qmp_cpuinfo);
+
+ let mut configured_nodes: Vec<NumaNodeId> = parse_index_list(binding_list)?;
+ if configured_nodes.is_empty() {
+ configured_nodes = topo.allowed_nodeset().iter_set().map(NumaNodeId).collect();
+ }
+
+ if configured_nodes.len() < vnuma_nodes.len() {
+ return Err(BindError::BadSyntax(format!(
+ "VM has {} vNUMA nodes but only {} host NUMA nodes configured",
+ vnuma_nodes.len(),
+ configured_nodes.len()
+ )));
+ }
+
+ let pairs: Vec<(NumaNodeId, NumaNodeId)> = (0..vnuma_nodes.len())
+ .map(|i| (NumaNodeId(i as u32), configured_nodes[i]))
+ .collect();
+
+ let rules = build_numa_rules(&pairs, &vnuma_nodes)?;
+ bind_rules(&topo, &rules, qmp_cpuinfo)
+}
+
+pub fn validate_manual(sockets: u32, cores: u32, binding_map: &str) -> Result<(), BindError> {
+ if binding_map.is_empty() {
+ return Err(BindError::BadSyntax("cpu-bind-map is empty".into()));
+ }
+
+ let rules = parse_cpu_binding_map(binding_map)?;
+ let nr_vcpus = sockets.saturating_mul(cores);
+
+ let topo = Topology::new()
+ .map_err(|e| BindError::BadSyntax(format!("topology init failed: {e}")))?;
+ let topo_cpuset = topo.allowed_cpuset();
+
+ let mut seen_vcpu: HashSet<u32> = HashSet::new();
+
+ for rule in &rules {
+ if !rule.cpuset.is_subset_of(&topo_cpuset) {
+ let missing = rule.cpuset.difference(&topo_cpuset);
+ let first = missing.first_set().ok_or_else(|| {
+ BindError::BadSyntax("internal error: subset check failed but no CPU missing".into())
+ })?;
+ return Err(BindError::UnknownHostCpu(first));
+ }
+ for vcpu in &rule.vcpus {
+ if vcpu.0 >= nr_vcpus {
+ return Err(BindError::VcpuOutOfRange {
+ vcpu: vcpu.0,
+ max: nr_vcpus,
+ });
+ }
+ if !seen_vcpu.insert(vcpu.0) {
+ return Err(BindError::VcpuReused(vcpu.0));
+ }
+ }
+ }
+ Ok(())
+}
+
+pub fn validate_one_to_one(sockets: u32, cores: u32, binding_list: &str) -> Result<(), BindError> {
+ let nr_vcpus = sockets.saturating_mul(cores);
+
+ let topo = Topology::new()
+ .map_err(|e| BindError::BadSyntax(format!("topology init failed: {e}")))?;
+ let allowed = topo.allowed_cpuset();
+
+ let mut available: CpuSet = if binding_list.is_empty() {
+ allowed
+ } else {
+ let cpus: Vec<Cpu> = parse_index_list(binding_list)?;
+ let mut set = CpuSet::new();
+ for cpu in &cpus {
+ if !allowed.isset(cpu.0) {
+ return Err(BindError::UnknownHostCpu(cpu.0));
+ }
+ set.set(cpu.0);
+ }
+ set
+ };
+
+ available.singlify_per_core(&topo);
+
+ let core_count = available.weight();
+ if core_count < nr_vcpus as usize {
+ return Err(BindError::BadSyntax(format!(
+ "one-to-one mode requires {nr_vcpus} physical cores but only {core_count} are available",
+ )));
+ }
+ Ok(())
+}
+
+pub fn validate_numa(sockets: u32, binding_list: &str) -> Result<(), BindError> {
+ let topo = Topology::new()
+ .map_err(|e| BindError::BadSyntax(format!("topology init failed: {e}")))?;
+ let topo_nodeset = topo.allowed_nodeset();
+
+ let host_node_ids: Vec<u32> = if binding_list.is_empty() {
+ topo_nodeset.iter_set().collect()
+ } else {
+ let ids: Vec<NumaNodeId> = parse_index_list(binding_list)?;
+ for id in &ids {
+ if !topo_nodeset.isset(id.0) {
+ return Err(BindError::BadSyntax(format!(
+ "host NUMA node {} does not exist on this host",
+ id.0
+ )));
+ }
+ }
+ ids.into_iter().map(|n| n.0).collect()
+ };
+
+ if (sockets as usize) > host_node_ids.len() {
+ return Err(BindError::BadSyntax(format!(
+ "VM has {sockets} vNUMA nodes but only {} host NUMA nodes available",
+ host_node_ids.len(),
+ )));
+ }
+ Ok(())
+}
+
+pub fn validate_numa_manual(sockets: u32, binding_map: &str) -> Result<(), BindError> {
+ if binding_map.is_empty() {
+ return Err(BindError::BadSyntax("numa-binding-map is empty".into()));
+ }
+
+ let pairs = parse_numa_binding_map(binding_map)?;
+
+ let topo = Topology::new()
+ .map_err(|e| BindError::BadSyntax(format!("topology init failed: {e}")))?;
+ let topo_nodeset = topo.allowed_nodeset();
+
+ let mut seen_vnode: HashSet<u32> = HashSet::new();
+
+ for (vnode, hnode) in &pairs {
+ if !topo_nodeset.isset(hnode.0) {
+ return Err(BindError::BadSyntax(format!(
+ "host NUMA node {} does not exist on this host",
+ hnode.0
+ )));
+ }
+ if vnode.0 >= sockets {
+ return Err(BindError::BadSyntax(format!(
+ "guest vNUMA node {} is out of range (VM has {sockets} sockets)",
+ vnode.0
+ )));
+ }
+ if !seen_vnode.insert(vnode.0) {
+ return Err(BindError::BadSyntax(format!(
+ "guest vNUMA node {} appears more than once",
+ vnode.0
+ )));
+ }
+ }
+ Ok(())
+}
diff --git a/pve-qemu-server-cpu/src/error.rs b/pve-qemu-server-cpu/src/error.rs
new file mode 100644
index 0000000..73a569e
--- /dev/null
+++ b/pve-qemu-server-cpu/src/error.rs
@@ -0,0 +1,38 @@
+use thiserror::Error;
+
+#[derive(Debug, Error)]
+pub enum BindError {
+ // Parse errors
+ #[error("map entry '{0}' is missing a ':' separator")]
+ MissingColon(String),
+
+ #[error("invalid syntax in binding map: '{0}'")]
+ BadSyntax(String),
+
+ // Validation errors (pre-start)
+ #[error("host CPU {0} does not exist in this topology")]
+ UnknownHostCpu(u32),
+
+ #[error("vCPU {0} appears in more than one map entry")]
+ VcpuReused(u32),
+
+ #[error("vCPU {vcpu} is out of range (VM has vCPUs 0..{max})")]
+ VcpuOutOfRange { vcpu: u32, max: u32 },
+
+ // Runtime errors (post-start)
+ #[error("no running thread found for vCPU {0}")]
+ ThreadNotFound(u32),
+
+ #[error("invalid kernel TID {tid} for vCPU {vcpu}")]
+ BadThreadId { vcpu: u32, tid: u32 },
+
+ #[error("failed to init host topology: '{0}'")]
+ TopologyInit(String),
+
+ #[error("binding vCPU {vcpu} failed: {message}")]
+ Bind {
+ vcpu: u32,
+ /// Human-readable error from libhwloc (errno-based).
+ message: String,
+ },
+}
diff --git a/pve-qemu-server-cpu/src/hwloc/bindings.rs b/pve-qemu-server-cpu/src/hwloc/bindings.rs
new file mode 100644
index 0000000..7e9c5bc
--- /dev/null
+++ b/pve-qemu-server-cpu/src/hwloc/bindings.rs
@@ -0,0 +1,171 @@
+//! Minimal raw FFI bindings for libhwloc.
+//!
+//! Taken from https://github.com/HadrienG2/hwlocality/blob/main/hwlocality-sys/src/lib.rs
+
+#![allow(non_camel_case_types)]
+
+use std::cell::UnsafeCell;
+use std::marker::{PhantomData, PhantomPinned};
+
+use libc::{c_char, c_int, c_uint, c_void, pid_t};
+
+#[repr(C)]
+struct IncompleteType {
+ _data: [u8; 0],
+ _marker: PhantomData<(*mut u8, PhantomPinned, &'static UnsafeCell<u8>)>,
+}
+
+#[repr(C)]
+pub struct hwloc_topology(IncompleteType);
+#[repr(C)]
+pub struct hwloc_bitmap_s(IncompleteType);
+
+#[repr(C)]
+pub struct hwloc_obj_attr_u(IncompleteType);
+#[repr(C)]
+pub struct hwloc_info_s(IncompleteType);
+
+pub type hwloc_topology_t = *mut hwloc_topology;
+pub type hwloc_const_topology_t = *const hwloc_topology;
+
+pub type hwloc_bitmap_t = *mut hwloc_bitmap_s;
+pub type hwloc_const_bitmap_t = *const hwloc_bitmap_s;
+
+pub type hwloc_cpuset_t = hwloc_bitmap_t;
+pub type hwloc_const_cpuset_t = hwloc_const_bitmap_t;
+pub type hwloc_nodeset_t = hwloc_bitmap_t;
+pub type hwloc_const_nodeset_t = hwloc_const_bitmap_t;
+
+pub type hwloc_obj_type_t = c_uint;
+pub const HWLOC_OBJ_MACHINE: hwloc_obj_type_t = 0;
+pub const HWLOC_OBJ_PACKAGE: hwloc_obj_type_t = 1;
+pub const HWLOC_OBJ_CORE: hwloc_obj_type_t = 2;
+pub const HWLOC_OBJ_PU: hwloc_obj_type_t = 3;
+pub const HWLOC_OBJ_NUMANODE: hwloc_obj_type_t = 13;
+
+pub type hwloc_get_type_depth_e = c_int;
+pub const HWLOC_TYPE_DEPTH_UNKNOWN: hwloc_get_type_depth_e = -1;
+pub const HWLOC_TYPE_DEPTH_MULTIPLE: hwloc_get_type_depth_e = -2;
+pub const HWLOC_TYPE_DEPTH_NUMANODE: hwloc_get_type_depth_e = -3;
+
+pub type hwloc_cpubind_flags_t = c_int;
+pub const HWLOC_CPUBIND_THREAD: hwloc_cpubind_flags_t = 1 << 1;
+pub const HWLOC_CPUBIND_PROCESS: hwloc_cpubind_flags_t = 1 << 0;
+pub const HWLOC_CPUBIND_STRICT: hwloc_cpubind_flags_t = 1 << 2;
+pub const HWLOC_CPUBIND_NOMEMBIND: hwloc_cpubind_flags_t = 1 << 3;
+
+#[repr(C)]
+pub struct hwloc_obj {
+ pub ty: hwloc_obj_type_t,
+ pub subtype: *mut c_char,
+ pub os_index: c_uint,
+ pub name: *mut c_char,
+ pub total_memory: u64,
+ pub attr: *mut hwloc_obj_attr_u,
+ pub depth: hwloc_get_type_depth_e,
+ pub logical_index: c_uint,
+ pub next_cousin: hwloc_obj_t,
+ pub prev_cousin: hwloc_obj_t,
+ pub parent: hwloc_obj_t,
+ pub sibling_rank: c_uint,
+ pub next_sibling: hwloc_obj_t,
+ pub prev_sibling: hwloc_obj_t,
+ pub arity: c_uint,
+ pub children: *mut hwloc_obj_t,
+ pub first_child: hwloc_obj_t,
+ pub last_child: hwloc_obj_t,
+ pub symmetric_subtree: c_int,
+ pub memory_arity: c_uint,
+ pub memory_first_child: hwloc_obj_t,
+ pub io_arity: c_uint,
+ pub io_first_child: hwloc_obj_t,
+ pub misc_arity: c_uint,
+ pub misc_first_child: hwloc_obj_t,
+ pub cpuset: hwloc_cpuset_t,
+ pub complete_cpuset: hwloc_cpuset_t,
+ pub nodeset: hwloc_nodeset_t,
+ pub complete_nodeset: hwloc_nodeset_t,
+ pub infos: *mut hwloc_info_s,
+ pub infos_count: c_uint,
+ pub userdata: *mut c_void,
+ pub gp_index: u64,
+}
+pub type hwloc_obj_t = *mut hwloc_obj;
+
+#[link(name = "hwloc")]
+unsafe extern "C" {
+ pub fn hwloc_topology_init(topology: *mut hwloc_topology_t) -> c_int;
+ pub fn hwloc_topology_load(topology: hwloc_topology_t) -> c_int;
+ pub fn hwloc_topology_destroy(topology: hwloc_topology_t);
+
+ pub fn hwloc_topology_get_topology_cpuset(
+ topology: hwloc_const_topology_t,
+ ) -> hwloc_const_cpuset_t;
+ pub fn hwloc_topology_get_topology_nodeset(
+ topology: hwloc_const_topology_t,
+ ) -> hwloc_const_nodeset_t;
+
+ pub fn hwloc_topology_get_allowed_cpuset(
+ topology: hwloc_const_topology_t,
+ ) -> hwloc_const_cpuset_t;
+ pub fn hwloc_topology_get_allowed_nodeset(
+ topology: hwloc_const_topology_t,
+ ) -> hwloc_const_nodeset_t;
+
+ pub fn hwloc_get_type_depth(
+ topology: hwloc_const_topology_t,
+ object_type: hwloc_obj_type_t,
+ ) -> hwloc_get_type_depth_e;
+ pub fn hwloc_get_nbobjs_by_depth(
+ topology: hwloc_const_topology_t,
+ depth: hwloc_get_type_depth_e,
+ ) -> c_uint;
+ pub fn hwloc_get_obj_by_depth(
+ topology: hwloc_const_topology_t,
+ depth: hwloc_get_type_depth_e,
+ idx: c_uint,
+ ) -> hwloc_obj_t;
+
+ pub fn hwloc_bitmap_alloc() -> hwloc_bitmap_t;
+ pub fn hwloc_bitmap_free(bitmap: hwloc_bitmap_t);
+ pub fn hwloc_bitmap_dup(src: hwloc_const_bitmap_t) -> hwloc_bitmap_t;
+
+ pub fn hwloc_bitmap_set(bitmap: hwloc_bitmap_t, id: c_uint) -> c_int;
+ pub fn hwloc_bitmap_isset(bitmap: hwloc_const_bitmap_t, id: c_uint) -> c_int;
+ pub fn hwloc_bitmap_weight(bitmap: hwloc_const_bitmap_t) -> c_int;
+ pub fn hwloc_bitmap_first(bitmap: hwloc_const_bitmap_t) -> c_int;
+ pub fn hwloc_bitmap_next(bitmap: hwloc_const_bitmap_t, prev: c_int) -> c_int;
+
+ pub fn hwloc_bitmap_isincluded(
+ left: hwloc_const_bitmap_t,
+ right: hwloc_const_bitmap_t,
+ ) -> c_int;
+ pub fn hwloc_bitmap_and(
+ result: hwloc_bitmap_t,
+ bitmap1: hwloc_const_bitmap_t,
+ bitmap2: hwloc_const_bitmap_t,
+ ) -> c_int;
+ pub fn hwloc_bitmap_andnot(
+ result: hwloc_bitmap_t,
+ bitmap1: hwloc_const_bitmap_t,
+ bitmap2: hwloc_const_bitmap_t,
+ ) -> c_int;
+ pub fn hwloc_bitmap_not(result: hwloc_bitmap_t, bitmap: hwloc_const_bitmap_t) -> c_int;
+ pub fn hwloc_bitmap_or(
+ result: hwloc_bitmap_t,
+ bitmap1: hwloc_const_bitmap_t,
+ bitmap2: hwloc_const_bitmap_t,
+ ) -> c_int;
+
+ pub fn hwloc_bitmap_singlify_per_core(
+ topology: hwloc_const_topology_t,
+ cpuset: hwloc_cpuset_t,
+ which: c_uint,
+ ) -> c_int;
+
+ pub fn hwloc_linux_set_tid_cpubind(
+ topology: hwloc_const_topology_t,
+ tid: pid_t,
+ set: hwloc_const_cpuset_t,
+ ) -> c_int;
+}
diff --git a/pve-qemu-server-cpu/src/hwloc/mod.rs b/pve-qemu-server-cpu/src/hwloc/mod.rs
new file mode 100644
index 0000000..2e3e741
--- /dev/null
+++ b/pve-qemu-server-cpu/src/hwloc/mod.rs
@@ -0,0 +1,222 @@
+//! Safe wrappers around the libhwloc FFI bindings.
+//!
+//! Replace with https://github.com/HadrienG2/hwlocality
+//! once the crate is mature enough
+
+mod bindings;
+
+use std::io;
+use bindings::*;
+
+/// Return the current `errno` as a message. hwloc reports errors via `errno`
+/// and a `-1` return value, so this is what you feed into user-facing errors.
+fn last_error(ctx: &str) -> String {
+ format!("{ctx}: {}", io::Error::last_os_error())
+}
+
+fn check(rc: std::ffi::c_int, ctx: &str) -> Result<(), String> {
+ if rc < 0 {
+ Err(last_error(ctx))
+ } else {
+ Ok(())
+ }
+}
+
+pub struct Topology(*mut hwloc_topology);
+
+impl Topology {
+ pub fn new() -> Result<Self, String> {
+ let mut raw: hwloc_topology_t = std::ptr::null_mut();
+ unsafe { check(hwloc_topology_init(&mut raw), "hwloc_topology_init")? };
+ let rc = unsafe { hwloc_topology_load(raw) };
+ if rc < 0 {
+ unsafe { hwloc_topology_destroy(raw) };
+ return Err(last_error("hwloc_topology_load"));
+ }
+ Ok(Topology(raw))
+ }
+
+ pub fn cpuset(&self) -> CpuSet {
+ unsafe {
+ let raw = hwloc_topology_get_topology_cpuset(self.0);
+ CpuSet::dup_from(raw)
+ }
+ }
+
+ pub fn nodeset(&self) -> CpuSet {
+ unsafe {
+ let raw = hwloc_topology_get_topology_nodeset(self.0);
+ CpuSet::dup_from(raw)
+ }
+ }
+
+ pub fn allowed_cpuset(&self) -> CpuSet {
+ unsafe {
+ let raw = hwloc_topology_get_allowed_cpuset(self.0);
+ CpuSet::dup_from(raw)
+ }
+ }
+
+ pub fn allowed_nodeset(&self) -> CpuSet {
+ unsafe {
+ let raw = hwloc_topology_get_allowed_nodeset(self.0);
+ CpuSet::dup_from(raw)
+ }
+ }
+
+ pub(crate) fn as_raw(&self) -> hwloc_const_topology_t {
+ self.0
+ }
+}
+
+impl Drop for Topology {
+ fn drop(&mut self) {
+ unsafe { hwloc_topology_destroy(self.0) };
+ }
+}
+
+pub struct CpuSet(*mut hwloc_bitmap_s);
+
+impl Clone for CpuSet {
+ fn clone(&self) -> Self {
+ unsafe {
+ let owned = hwloc_bitmap_dup(self.0);
+ assert!(!owned.is_null(), "hwloc_bitmap_dup failed");
+ CpuSet(owned)
+ }
+ }
+}
+
+impl std::fmt::Debug for CpuSet {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ f.debug_tuple("CpuSet")
+ .field(&self.iter_set().collect::<Vec<_>>())
+ .finish()
+ }
+}
+
+impl CpuSet {
+ pub fn new() -> Self {
+ let raw = unsafe { hwloc_bitmap_alloc() };
+ assert!(!raw.is_null(), "hwloc_bitmap_alloc failed");
+ CpuSet(raw)
+ }
+
+ /// Allocate a bitmap containing exactly one CPU.
+ pub fn single(idx: u32) -> Self {
+ let mut set = Self::new();
+ set.set(idx);
+ set
+ }
+
+ /// Clone a borrowed bitmap returned by the topology.
+ fn dup_from(raw: hwloc_const_bitmap_t) -> Self {
+ let owned = unsafe { hwloc_bitmap_dup(raw) };
+ assert!(!owned.is_null(), "hwloc_bitmap_dup failed");
+ CpuSet(owned)
+ }
+
+ pub fn difference(&self, other: &CpuSet) -> CpuSet {
+ let out = CpuSet::new();
+ let rc = unsafe { hwloc_bitmap_andnot(out.0, self.0, other.0) };
+ assert!(rc >= 0, "hwloc_bitmap_andnot failed");
+ out
+ }
+
+ pub fn intersect(&mut self, other: &CpuSet) {
+ let rc = unsafe { hwloc_bitmap_and(self.0, self.0, other.0) };
+ assert!(rc >= 0, "hwloc_bitmap_and failed");
+ }
+
+ pub fn set(&mut self, idx: u32) {
+ let rc = unsafe { hwloc_bitmap_set(self.0, idx) };
+ assert!(rc >= 0, "hwloc_bitmap_set failed");
+ }
+
+ pub fn isset(&self, idx: u32) -> bool {
+ unsafe { hwloc_bitmap_isset(self.0, idx) != 0 }
+ }
+
+ pub fn weight(&self) -> usize {
+ let w = unsafe { hwloc_bitmap_weight(self.0) };
+ w.max(0) as usize
+ }
+
+ pub fn is_empty(&self) -> bool {
+ self.first_set().is_none()
+ }
+
+ pub fn first_set(&self) -> Option<u32> {
+ let i = unsafe { hwloc_bitmap_first(self.0) };
+ if i < 0 { None } else { Some(i as u32) }
+ }
+
+ fn next_set_after(&self, prev: u32) -> Option<u32> {
+ let i = unsafe { hwloc_bitmap_next(self.0, prev as std::ffi::c_int) };
+ if i < 0 { None } else { Some(i as u32) }
+ }
+
+ pub fn iter_set(&self) -> impl Iterator<Item = u32> + '_ {
+ std::iter::successors(self.first_set(), move |&cur| self.next_set_after(cur))
+ }
+
+ pub fn is_subset_of(&self, other: &CpuSet) -> bool {
+ unsafe { hwloc_bitmap_isincluded(self.0, other.0) != 0 }
+ }
+
+ /// Remove SMT siblings from each core (requires hwloc 2.2+).
+ pub fn singlify_per_core(&mut self, topo: &Topology) {
+ let rc = unsafe { hwloc_bitmap_singlify_per_core(topo.as_raw(), self.0, 0) };
+ assert!(rc >= 0, "hwloc_bitmap_singlify_per_core failed");
+ }
+}
+
+impl Drop for CpuSet {
+ fn drop(&mut self) {
+ unsafe { hwloc_bitmap_free(self.0) };
+ }
+}
+
+/// Return the CPU set belonging to the host NUMA node with the given OS index.
+///
+/// The returned set is intersected with the topology's allowed CPU set.
+/// Returns `None` if the node does not exist,or if it has no allowed
+/// CPUs (e.g., a CPU-less memory node).
+pub fn numa_node_cpuset(topo: &Topology, node_id: u32) -> Option<CpuSet> {
+ let depth = unsafe { hwloc_get_type_depth(topo.as_raw(), HWLOC_OBJ_NUMANODE) };
+ if depth == HWLOC_TYPE_DEPTH_UNKNOWN {
+ return None;
+ }
+
+ let count = unsafe { hwloc_get_nbobjs_by_depth(topo.as_raw(), depth) };
+ for i in 0..count {
+ let obj = unsafe { hwloc_get_obj_by_depth(topo.as_raw(), depth, i) };
+ if obj.is_null() {
+ continue;
+ }
+
+ if unsafe { (*obj).os_index } == node_id {
+ let raw = unsafe { (*obj).cpuset };
+
+ // hwloc guarantees cpuset != NULL for NUMA nodes.
+ let mut set = CpuSet::dup_from(raw);
+
+ let allowed = topo.allowed_cpuset();
+ set.intersect(&allowed);
+
+ if set.is_empty() {
+ return None;
+ }
+
+ return Some(set);
+ }
+ }
+
+ None
+}
+
+#[cfg(target_os = "linux")]
+pub fn bind_thread(topo: &Topology, tid: i32, set: &CpuSet) -> Result<(), String> {
+ let rc = unsafe { hwloc_linux_set_tid_cpubind(topo.as_raw(), tid, set.0) };
+ check(rc, "hwloc_linux_set_tid_cpubind")
+}
diff --git a/pve-qemu-server-cpu/src/lib.rs b/pve-qemu-server-cpu/src/lib.rs
new file mode 100644
index 0000000..e297bd9
--- /dev/null
+++ b/pve-qemu-server-cpu/src/lib.rs
@@ -0,0 +1,92 @@
+mod bind;
+mod hwloc;
+mod qmp_cpu;
+mod error;
+
+pub use error::BindError;
+
+pub fn validate_binding(
+ mode: &str,
+ sockets: u32,
+ cores: u32,
+ binding_cfg: &str,
+) -> Result<(), BindError> {
+ match mode.parse::<bind::Mode>()? {
+ bind::Mode::None => Ok(()),
+ bind::Mode::Manual => bind::validate_manual(sockets, cores, binding_cfg),
+ bind::Mode::OneToOne => bind::validate_one_to_one(sockets, cores, binding_cfg),
+ bind::Mode::Numa => bind::validate_numa(sockets, binding_cfg),
+ bind::Mode::NumaManual => bind::validate_numa_manual(sockets, binding_cfg),
+ }
+}
+
+/// Apply CPU pinning to a running VM.
+///
+/// The function parses the QMP `query-cpus-fast` response and binds each
+/// vCPU thread according to the selected mode.
+///
+/// # Arguments
+///
+/// * `mode` - Pinning mode: `none`, `manual`, `one-to-one`, `numa`, or
+/// `numa-manual`.
+/// * `binding_info` - Mode-specific CPU or NUMA mapping. Ignored for `none`.
+/// * `cpuinfo_json` - JSON output from QMP `query-cpus-fast`.
+///
+/// # Modes
+///
+/// | Mode | Meaning |
+/// |---|---|
+/// | `none` | Do not pin vCPUs. |
+/// | `manual` | Explicitly map vCPUs to host CPUs. |
+/// | `one-to-one` | Assign each vCPU to its own physical core. |
+/// | `numa` | Map guest NUMA nodes to host NUMA nodes in order. |
+/// | `numa-manual` | Explicitly map guest NUMA nodes to host NUMA nodes. |
+///
+/// # Examples
+///
+/// ```ignore
+///
+/// // Manual mode:
+/// // vCPU 0 -> host CPUs 2-3
+/// // vCPU 1 -> host CPU 4
+/// bind_vcpus("manual", "0:2-3;1:4", &cpuinfo_json)?;
+///
+/// // One-to-one mode:
+/// // Distribute vCPUs over host CPUs 0-7.
+/// bind_vcpus("one-to-one", "0-7", &cpuinfo_json)?;
+///
+/// // One-to-one mode over all allowed host CPUs.
+/// bind_vcpus("one-to-one", "", &cpuinfo_json)?;
+///
+/// // NUMA mode:
+/// // Guest NUMA node 0 -> host NUMA node 0
+/// // Guest NUMA node 1 -> host NUMA node 1
+/// bind_vcpus("numa", "0;1", &cpuinfo_json)?;
+///
+/// // NUMA mode over all allowed host NUMA nodes.
+/// bind_vcpus("numa", "", &cpuinfo_json)?;
+///
+/// // Manual NUMA mode:
+/// // Guest NUMA node 0 -> host NUMA node 1
+/// // Guest NUMA node 1 -> host NUMA node 0
+/// bind_vcpus("numa-manual", "0:1;1:0", &cpuinfo_json)?;
+/// ```
+///
+/// # Errors
+///
+/// Returns [`BindError`] if the mode or binding syntax is invalid, if the
+/// requested host CPUs or NUMA nodes do not exist, or if binding fails.
+pub fn bind_vcpus(
+ mode: &str,
+ binding_cfg: &str,
+ cpuinfo_json: &str,
+) -> Result<(), BindError> {
+ let qmp_cpuinfo = qmp_cpu::parse_cpuinfo(cpuinfo_json)?;
+ match mode.parse::<bind::Mode>()? {
+ bind::Mode::None => Ok(()),
+ bind::Mode::Manual => bind::bind_manual(binding_cfg, &qmp_cpuinfo),
+ bind::Mode::OneToOne => bind::bind_one_to_one(binding_cfg, &qmp_cpuinfo),
+ bind::Mode::Numa => bind::bind_numa(binding_cfg, &qmp_cpuinfo),
+ bind::Mode::NumaManual => bind::bind_numa_manual(binding_cfg, &qmp_cpuinfo),
+ }
+}
diff --git a/pve-qemu-server-cpu/src/qmp_cpu.rs b/pve-qemu-server-cpu/src/qmp_cpu.rs
new file mode 100644
index 0000000..00d88c5
--- /dev/null
+++ b/pve-qemu-server-cpu/src/qmp_cpu.rs
@@ -0,0 +1,99 @@
+//! Parsing of the QEMU `query-cpus-fast` QMP response.
+//!
+//! The response is a JSON array; each element describes one guest vCPU
+//! and carries both its guest-visible topology (socket/core/NUMA node)
+//! and its host kernel TID. Every binding mode consumes this shape, so
+//! the parsing lives here rather than inside any individual mode.
+//!
+//! The full QEMU schema is modeled, including fields not currently used
+//! by any binding mode. Serde ignores fields it doesn't know about, but
+//! keeping them explicit documents the wire format and makes future
+//! modes cheaper to implement.
+
+use serde::Deserialize;
+
+use crate::BindError;
+
+/// One entry of the `query-cpus-fast` array.
+#[derive(Debug, Clone, Deserialize)]
+pub struct QmpCpu {
+ /// Guest-visible vCPU index (0 .. sockets*cores).
+ #[serde(rename = "cpu-index")]
+ pub cpu_index: u32,
+
+ /// Host kernel TID of the QEMU vCPU thread.
+ #[serde(rename = "thread-id")]
+ pub thread_id: u32,
+
+ /// QOM path of the CPU object, e.g. `/machine/unattached/device[0]`.
+ #[expect(dead_code)]
+ #[serde(rename = "qom-path")]
+ pub qom_path: String,
+
+ /// QOM type of the CPU object, e.g. `host-x86_64-cpu`.
+ #[expect(dead_code)]
+ #[serde(rename = "qom-type")]
+ pub qom_type: String,
+
+ /// Target architecture, e.g. `x86_64`.
+ #[expect(dead_code)]
+ pub target: String,
+
+ /// Guest-visible topology properties of the vCPU.
+ pub props: QmpCpuProps,
+}
+
+/// Guest-visible topology properties of a vCPU.
+#[derive(Debug, Clone, Deserialize)]
+pub struct QmpCpuProps {
+ /// Guest socket the vCPU belongs to.
+ #[expect(dead_code)]
+ #[serde(rename = "socket-id")]
+ pub socket_id: u32,
+
+ /// Guest core within that socket.
+ #[expect(dead_code)]
+ #[serde(rename = "core-id")]
+ pub core_id: u32,
+
+ /// Guest NUMA node the vCPU belongs to.
+ ///
+ /// QEMU reports this even when `numa` is disabled in the VM config,
+ /// in which case every vCPU is on node 0.
+ #[serde(rename = "node-id")]
+ pub node_id: u32,
+
+ /// Guest thread within the core.
+ ///
+ /// Always 0 today: QEMU's CPU object model carries this field for
+ /// SMT support in the guest, but PVE does not expose SMT to guests,
+ /// so it is always zero in practice. Modeled for completeness.
+ /// TODO: PVE should expose SMT to the guests
+ #[expect(dead_code)]
+ #[serde(rename = "thread-id")]
+ pub thread_id: u32,
+}
+
+// returned vec is sorted by vCPU id (0,1,2,3...)
+// Guarantees vec[id] == vec[id].cpu_index
+pub fn parse_cpuinfo(cpuinfo_json: &str) -> Result<Vec<QmpCpu>, BindError> {
+ let mut cpus: Vec<QmpCpu> = serde_json::from_str(cpuinfo_json)
+ .map_err(|e| BindError::BadSyntax(format!("invalid query-cpus-fast JSON: {e}")))?;
+
+ // QEMU assigns cpu-index sequentially from 0. Sort by it so
+ // `vec[i]` indexes vCPU i, then verify the values really are
+ // 0..N-1 so that assumption holds.
+ cpus.sort_unstable_by_key(|c| c.cpu_index);
+
+ for (expected, cpu) in cpus.iter().enumerate() {
+ if cpu.cpu_index != expected as u32 {
+ return Err(BindError::BadSyntax(format!(
+ "query-cpus-fast returned non-contiguous cpu-index: \
+ expected {expected}, got {}",
+ cpu.cpu_index
+ )));
+ }
+ }
+
+ Ok(cpus)
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH proxmox-perl-rs v3 2/3] pve: cpu: add PVE::RS::CPU::Bind perl bindings
2026-10-22 9:22 [PATCH proxmox-perl-rs/qemu-server/qemu-server-rs v3 0/3] vCPU binding for qemu guests Elias Huhsovitz
2026-10-22 9:22 ` [PATCH qemu-server-rs v3 1/3] cpu: add pve-qemu-server-cpu crate for vCPU binding Elias Huhsovitz
@ 2026-10-22 9:22 ` Elias Huhsovitz
2026-10-22 9:22 ` [PATCH qemu-server v3 3/3] cpu: add CPU binding Elias Huhsovitz
2 siblings, 0 replies; 4+ messages in thread
From: Elias Huhsovitz @ 2026-10-22 9:22 UTC (permalink / raw)
To: pve-devel; +Cc: Elias Huhsovitz
Expose the pve-qemu-server-cpu crate to Perl as PVE::RS::CPU::Bind. Two
functions are exported, matching the crate's public API:
* validate_binding(mode, sockets, cores, cfg)
* bind_vcpus(mode, cfg, cpuinfo_json)
Add pve-qemu-server-cpu as a path dependency of pve-rs. The crate is
packaged as librust-pve-qemu-server-cpu-dev.
Signed-off-by: Elias Huhsovitz <e.huhsovitz@proxmox.com>
---
pve-rs/Cargo.toml | 1 +
pve-rs/Makefile | 1 +
pve-rs/src/bindings/cpu/bind.rs | 40 +++++++++++++++++++++++++++++++++
pve-rs/src/bindings/cpu/mod.rs | 2 ++
pve-rs/src/bindings/mod.rs | 3 +++
5 files changed, 47 insertions(+)
create mode 100644 pve-rs/src/bindings/cpu/bind.rs
create mode 100644 pve-rs/src/bindings/cpu/mod.rs
diff --git a/pve-rs/Cargo.toml b/pve-rs/Cargo.toml
index 0c4a47e..282266d 100644
--- a/pve-rs/Cargo.toml
+++ b/pve-rs/Cargo.toml
@@ -53,6 +53,7 @@ proxmox-time = "2"
proxmox-ve-config = { version = "0.10.4", features = [ "frr" ] }
proxmox-wireguard = { version = "0.1.2" }
pve-qemu-server-pci = "0.1"
+pve-qemu-server-cpu = "0.1"
# [patch.crates-io]
# pbs-api-types = { path = "../../proxmox/pbs-api-types" }
diff --git a/pve-rs/Makefile b/pve-rs/Makefile
index c239f2a..077ad4a 100644
--- a/pve-rs/Makefile
+++ b/pve-rs/Makefile
@@ -30,6 +30,7 @@ PERLMOD_PACKAGES := \
PVE::RS::NVML \
PVE::RS::OCI \
PVE::RS::PCI::HardwareConfig \
+ PVE::RS::CPU::Bind \
PVE::RS::OpenId \
PVE::RS::ResourceScheduling::Static \
PVE::RS::ResourceScheduling::Dynamic \
diff --git a/pve-rs/src/bindings/cpu/bind.rs b/pve-rs/src/bindings/cpu/bind.rs
new file mode 100644
index 0000000..864c56d
--- /dev/null
+++ b/pve-rs/src/bindings/cpu/bind.rs
@@ -0,0 +1,40 @@
+#[perlmod::package(name = "PVE::RS::CPU::Bind", lib = "pve_rs")]
+pub mod pve_rs_cpu_bind {
+ //! `PVE::RS::CPU::Bind` package.
+ //!
+ //! Exposes the vCPU binding entry points.
+ //! from [`pve_qemu_server_cpu`].
+
+ use anyhow::{Error, format_err};
+
+ /// Validate a `cpu-bind-map` for the given mode before the VM starts.
+ ///
+ /// `binding_info` is the raw config value (`cpu-bind-map`,
+ /// `cpu-binding-list`, `numa-binding-map`, or `numa-binding-list`,
+ /// depending on `mode`). Returns `Ok(())` on success; on error, the
+ /// returned string is suitable for `die` in Perl.
+ #[export]
+ pub fn validate_binding(
+ mode: &str,
+ sockets: u32,
+ cores: u32,
+ binding_cfg: &str,
+ ) -> Result<(), Error> {
+ pve_qemu_server_cpu::validate_binding(mode, sockets, cores, binding_cfg)
+ .map_err(|e| format_err!("{e}"))
+ }
+
+ /// Apply the binding to a running VM.
+ ///
+ /// `cpuinfo_json` must be the JSON-encoded `query-cpus-fast` QMP
+ /// response.
+ #[export]
+ pub fn bind_vcpus(
+ mode: &str,
+ binding_cfg: &str,
+ cpuinfo_json: &str,
+ ) -> Result<(), Error> {
+ pve_qemu_server_cpu::bind_vcpus(mode, binding_cfg, cpuinfo_json)
+ .map_err(|e| format_err!("{e}"))
+ }
+}
diff --git a/pve-rs/src/bindings/cpu/mod.rs b/pve-rs/src/bindings/cpu/mod.rs
new file mode 100644
index 0000000..3dd4427
--- /dev/null
+++ b/pve-rs/src/bindings/cpu/mod.rs
@@ -0,0 +1,2 @@
+mod bind;
+pub use bind::pve_rs_cpu_bind;
diff --git a/pve-rs/src/bindings/mod.rs b/pve-rs/src/bindings/mod.rs
index b26c602..e9473a8 100644
--- a/pve-rs/src/bindings/mod.rs
+++ b/pve-rs/src/bindings/mod.rs
@@ -9,6 +9,9 @@ pub use oci::pve_rs_oci;
mod pci;
pub use pci::pve_rs_pci_hardware_config;
+mod cpu;
+pub use cpu::pve_rs_cpu_bind;
+
pub mod resource_scheduling;
mod tfa;
--
2.47.3
^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH qemu-server v3 3/3] cpu: add CPU binding
2026-10-22 9:22 [PATCH proxmox-perl-rs/qemu-server/qemu-server-rs v3 0/3] vCPU binding for qemu guests Elias Huhsovitz
2026-10-22 9:22 ` [PATCH qemu-server-rs v3 1/3] cpu: add pve-qemu-server-cpu crate for vCPU binding Elias Huhsovitz
2026-10-22 9:22 ` [PATCH proxmox-perl-rs v3 2/3] pve: cpu: add PVE::RS::CPU::Bind perl bindings Elias Huhsovitz
@ 2026-10-22 9:22 ` Elias Huhsovitz
2 siblings, 0 replies; 4+ messages in thread
From: Elias Huhsovitz @ 2026-10-22 9:22 UTC (permalink / raw)
To: pve-devel; +Cc: Elias Huhsovitz
Add two new VM config options:
* cpu-binding-mode none | manual | one-to-one | numa | numa-manual
* cpu-binding-cfg mode-dependent syntax
When cpu-binding-mode is unset or "none", behavior is unchanged.
Otherwise, vCPU threads are pinned to host CPUs after QEMU has spawned
them, driven by the query-cpus-fast QMP response.
Five modes are supported:
* manual - explicit vCPU:hostCPU mapping, e.g. "0:0;1:1;2:2;3:3"
* one-to-one - each vCPU gets its own physical core from a user pool,
SMT stripped, e.g. "0-3"
* numa - per-vNUMA-node set-to-set binding, sequential, e.g. "3;4"
* numa-manual - explicit vNUMA->host NUMA mapping, e.g. "0:2;1:3"
Two entry points are provided in PVE::QemuServer::CPUBinding:
* validate_binding() - called at VM start, before QEMU spawns. Rejects
invalid configs against the current topology.
* bind_vcpus() - called after QEMU is up. Encodes the QMP cpu
information as JSON and hands it to Rust together with the mode and cfg
string.
Signed-off-by: Elias Huhsovitz <e.huhsovitz@proxmox.com>
---
src/PVE/QemuServer.pm | 36 +++++++++
src/PVE/QemuServer/CPUBinding.pm | 133 +++++++++++++++++++++++++++++++
src/PVE/QemuServer/Makefile | 1 +
3 files changed, 170 insertions(+)
create mode 100644 src/PVE/QemuServer/CPUBinding.pm
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 1efeeb66..3ba266be 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -60,6 +60,7 @@ use PVE::QemuServer::Helpers
qw(config_aware_timeout get_iscsi_initiator_name get_host_arch min_version kvm_user_version windows_version);
use PVE::QemuServer::Cloudinit;
use PVE::QemuServer::CGroup;
+use PVE::QemuServer::CPUBinding;
use PVE::QemuServer::CPUConfig qw(
print_cpu_device
get_cpu_options
@@ -739,6 +740,18 @@ EODESCR
optional => 1,
default => 1,
},
+ 'cpu-binding-mode' => {
+ type => 'string',
+ format => $PVE::QemuServer::Bind::bind_mode_fmt,
+ description => "Set CPU binding mode for the guest",
+ optional => 1,
+ },
+ 'cpu-binding-cfg' => {
+ type => 'string',
+ format => $PVE::QemuServer::Bind::bind_cfg_fmt,
+ description => "Configuration for CPU binding for the guest",
+ optional => 1,
+ },
};
my $cicustom_fmt = {
@@ -3218,6 +3231,15 @@ sub config_to_command {
push @$cmd, '/usr/bin/taskset', '--cpu-list', '--all-tasks', $conf->{affinity};
}
+ if ($conf->{'cpu-binding-mode'}) {
+ PVE::QemuServer::CPUBinding::validate_binding(
+ $conf->{'cpu-binding-mode'},
+ $conf->{'sockets'},
+ $conf->{'cores'},
+ $conf->{'cpu-binding-cfg'},
+ );
+ }
+
push @$cmd, $kvm_binary;
push @$cmd, '-id', $vmid;
@@ -5715,6 +5737,20 @@ sub vm_start_nolock {
my $qemu_pkg_string = PVE::QemuServer::QMPHelpers::get_qemu_package_string($vmid);
syslog("info", "VM $vmid started with PID $pid using $qemu_pkg_string.");
+ if ($conf->{'cpu-binding-mode'}) {
+ eval {
+ PVE::QemuServer::CPUBinding::bind_vcpus(
+ $vmid,
+ $conf->{'cpu-binding-mode'},
+ $conf->{'cpu-binding-cfg'},
+ );
+ 1;
+ } or do {
+ my $error = $@ || 'Unknown failure';
+ log_warn("unable to bind vCPU threads - $error");
+ };
+ }
+
PVE::QemuServer::RunState::create_cleanup_flag($vmid);
if (defined(my $migrate = $res->{migrate})) {
diff --git a/src/PVE/QemuServer/CPUBinding.pm b/src/PVE/QemuServer/CPUBinding.pm
new file mode 100644
index 00000000..3ce3653b
--- /dev/null
+++ b/src/PVE/QemuServer/CPUBinding.pm
@@ -0,0 +1,133 @@
+package PVE::QemuServer::CPUBinding;
+
+use v5.36;
+
+use JSON;
+
+use PVE::QemuServer::Monitor;
+use PVE::RS::CPU::Bind;
+
+our $bind_mode_fmt = {
+ mode => {
+ type => 'string',
+ enum => [qw(none manual one-to-one numa numa-manual)],
+ description => "Set the CPU binding mode for the guest.",
+ verbose_description => <<'EODESCR',
+Selects how guest vCPUs are pinned to host CPUs. The mode determines
+how cpu-binding-cfg is interpreted.
+
+* none (default): no binding.
+
+* manual: explicit vCPU-to-hostCPU mapping. The config string is a
+ semicolon-separated list of `<vcpu-set>:<hostcpu-set>` entries.
+ Example: `0:0;1:1;2:2;3:3` pins vCPU 0 to CPU 0, vCPU 1 to CPU 1,
+ etc. Multi-value sets use `a-b` ranges or `{a,b,c}` unions.
+
+* one-to-one: pin each vCPU to its own physical core, chosen from a
+ user-supplied pool. The config string is an upper bound: the list of
+ host cores the vCPUs may be assigned to. SMT siblings are stripped
+ from that pool before assignment, so each vCPU gets its own distinct
+ physical core, not a hyperthread.
+
+ If the pool has N cores and the VM has M vCPUs, each vCPU is pinned
+ to exactly one of the N cores (N must be >= M, otherwise validation
+ fails). For example, with `0-3` on a 4-vCPU VM, vCPU 0 gets core 0,
+ vCPU 1 gets core 1, and so on. Each vCPU is fixed to a single core,
+ it does not float within the pool. With `0;1;4;5` on the same VM,
+ vCPU 2 is pinned to core 4 and vCPU 3 to core 5, skipping cores 2-3.
+
+ An empty string means "use all physical cores on the host" as the
+ upper bound. On a host with 16 cores, that pins an N-vCPU VM to
+ cores 0, 1, ..., N-1 — one core per vCPU, SMT stripped.
+
+ Unlike `manual`, you do not specify which core each vCPU gets; that
+ assignment is up to hwloc. Use `manual` if you need a specific
+ vCPU-to-core mapping.
+
+* numa: bind the set of vCPUs of each guest vNUMA node to the set of
+ CPUs of a host NUMA node. The config string is a `;`-separated list
+ of host NUMA node indices, consumed in order: guest vNUMA 0 gets the
+ first host node, guest vNUMA 1 the second, and so on.
+
+ Each guest vNUMA node's vCPUs form a group that may run on any CPU
+ of the assigned host NUMA node — including its cores and SMT threads.
+ The host scheduler is free to migrate vCPUs within the node, but not
+ outside it. This is the intended trade-off: memory allocated by a
+ vCPU stays local to the host node it runs on (preserving NUMA
+ memory locality), while the scheduler still has room to balance
+ load across the node's cores and hyperthreads.
+
+ This is a set-to-set binding, not a per-vCPU assignment. All vCPUs
+ of a guest vNUMA node share the same host CPU pool. If you need
+ each vCPU pinned to its own distinct core, use one-to-one or manual
+ instead.
+
+ Example: `3;4` lets all vCPUs on guest vNUMA 0 run on any CPU of
+ host NUMA node 3, and all vCPUs on guest vNUMA 1 run on any CPU of
+ host NUMA node 4.
+
+ An empty string uses host NUMA nodes in ascending index order:
+ guest vNUMA 0 → host node 0, guest vNUMA 1 → host node 1, etc.
+
+* numa-manual: explicit variant of numa. The config string is a
+ `;`-separated list of `<guest-vnuma>:<host-numa>` pairs. Each guest
+ vNUMA node's set of vCPUs is bound to the set of CPUs of the
+ specified host NUMA node.
+
+ Semantics are identical to numa — vCPUs of a guest vNUMA node may
+ be scheduled on any CPU of the assigned host node, but cannot leave
+ it. Only the node mapping is explicit here.
+
+ Example: `0:2;1:3` binds all vCPUs on guest vNUMA 0 to host NUMA
+ node 2's CPUs, and all vCPUs on guest vNUMA 1 to host NUMA node 3's
+ CPUs.
+EODESCR
+ default => 'none',
+ default_key => 1,
+ },
+};
+
+our $bind_cfg_fmt = {
+ cfg => {
+ type => 'string',
+ description => "CPU binding configuration (syntax depends on cpu-binding-mode).",
+ verbose_description => <<'EODESCR',
+The exact syntax depends on cpu-binding-mode:
+
+* manual, one-to-one:
+ `;`-separated list of `<vcpu-set>:<hostcpu-set>` entries (manual)
+ or a plain host CPU list (one-to-one). CPU sets accept `a-b` ranges
+ and `{a,b,c}` unions.
+
+* numa:
+ `;`-separated list of host NUMA node indices, e.g. `3;4`.
+
+* numa-manual:
+ `;`-separated list of `<guest-vnuma>:<host-numa>` pairs, e.g. `0:2;1:3`.
+
+Host CPU and NUMA node indices are OS indices as reported by `lstopo -p`.
+EODESCR
+ default => '',
+ default_key => 1,
+ },
+};
+
+sub validate_binding($mode, $sockets, $cores, $binding_cfg) {
+ return if $mode eq 'none';
+
+ $binding_cfg //= '';
+
+ PVE::RS::CPU::Bind::validate_binding($mode, $sockets, $cores, $binding_cfg);
+}
+
+sub bind_vcpus($vmid, $mode, $binding_cfg) {
+ return if $mode eq 'none';
+
+ $binding_cfg //= '';
+
+ my $cpuinfo = PVE::QemuServer::Monitor::mon_cmd($vmid, 'query-cpus-fast');
+ my $cpuinfo_json = JSON::encode_json($cpuinfo);
+ PVE::RS::CPU::Bind::bind_vcpus($mode, $binding_cfg, $cpuinfo_json);
+}
+
+1;
diff --git a/src/PVE/QemuServer/Makefile b/src/PVE/QemuServer/Makefile
index 61ed1a4e..ee123cf5 100644
--- a/src/PVE/QemuServer/Makefile
+++ b/src/PVE/QemuServer/Makefile
@@ -8,6 +8,7 @@ SOURCES=Agent.pm \
Cfg2Cmd.pm \
CGroup.pm \
Cloudinit.pm \
+ CPUBinding.pm \
CPUConfig.pm \
CPUFlags.pm \
DBusVMState.pm \
--
2.47.3
^ permalink raw reply related [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-10-22 9:23 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-10-22 9:22 [PATCH proxmox-perl-rs/qemu-server/qemu-server-rs v3 0/3] vCPU binding for qemu guests Elias Huhsovitz
2026-10-22 9:22 ` [PATCH qemu-server-rs v3 1/3] cpu: add pve-qemu-server-cpu crate for vCPU binding Elias Huhsovitz
2026-10-22 9:22 ` [PATCH proxmox-perl-rs v3 2/3] pve: cpu: add PVE::RS::CPU::Bind perl bindings Elias Huhsovitz
2026-10-22 9:22 ` [PATCH qemu-server v3 3/3] cpu: add CPU binding Elias Huhsovitz
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox