* [PATCH qemu-server-rs v3 1/3] cpu: add pve-qemu-server-cpu crate for vCPU binding
2026-10-22 9:22 [PATCH proxmox-perl-rs/qemu-server/qemu-server-rs v3 0/3] vCPU binding for qemu guests Elias Huhsovitz
@ 2026-10-22 9:22 ` Elias Huhsovitz
2026-10-22 9:22 ` [PATCH proxmox-perl-rs v3 2/3] pve: cpu: add PVE::RS::CPU::Bind perl bindings Elias Huhsovitz
2026-10-22 9:22 ` [PATCH qemu-server v3 3/3] cpu: add CPU binding Elias Huhsovitz
2 siblings, 0 replies; 4+ messages in thread
From: Elias Huhsovitz @ 2026-10-22 9:22 UTC (permalink / raw)
To: pve-devel; +Cc: Elias Huhsovitz
Add a new library crate that implements CPU binding on top of libhwloc.
Two public entry points are exposed:
* validate_binding(mode, sockets, cores, cfg) - pure pre-start
validation. Checks cfg syntax, host CPU existence, vCPU range, and
NUMA node existence against the local topology. No side effects.
* bind_vcpus(mode, cfg, cpuinfo_json) - applies the binding to a running
QEMU process. Consumes the query-cpus-fast QMP response and binds each
vCPU thread by kernel TID.
Five modes are implemented:
* none: no binding
* manual: explicit vCPU:hostCPU mapping
* one-to-one: one vCPU per physical core, SMT stripped
* numa: per-vNUMA-node set-to-set binding
* numa-manual: explicit vNUMA->host NUMA mapping
hwloc is accessed via a small FFI subset (src/bindings.rs).
Thread binding uses hwloc_linux_set_tid_cpubind, which accepts a kernel
TID directly. The TID used is reported by qemu via query-cpus-fast.
Signed-off-by: Elias Huhsovitz <e.huhsovitz@proxmox.com>
---
Cargo.toml | 2 +-
build.sh | 2 +-
pve-qemu-server-cpu/Cargo.toml | 16 +
pve-qemu-server-cpu/debian/changelog | 5 +
pve-qemu-server-cpu/debian/control | 41 ++
pve-qemu-server-cpu/debian/copyright | 18 +
pve-qemu-server-cpu/debian/debcargo.toml | 9 +
pve-qemu-server-cpu/src/bind.rs | 478 ++++++++++++++++++++++
pve-qemu-server-cpu/src/error.rs | 38 ++
pve-qemu-server-cpu/src/hwloc/bindings.rs | 171 ++++++++
pve-qemu-server-cpu/src/hwloc/mod.rs | 222 ++++++++++
pve-qemu-server-cpu/src/lib.rs | 92 +++++
pve-qemu-server-cpu/src/qmp_cpu.rs | 99 +++++
13 files changed, 1191 insertions(+), 2 deletions(-)
create mode 100644 pve-qemu-server-cpu/Cargo.toml
create mode 100644 pve-qemu-server-cpu/debian/changelog
create mode 100644 pve-qemu-server-cpu/debian/control
create mode 100644 pve-qemu-server-cpu/debian/copyright
create mode 100644 pve-qemu-server-cpu/debian/debcargo.toml
create mode 100644 pve-qemu-server-cpu/src/bind.rs
create mode 100644 pve-qemu-server-cpu/src/error.rs
create mode 100644 pve-qemu-server-cpu/src/hwloc/bindings.rs
create mode 100644 pve-qemu-server-cpu/src/hwloc/mod.rs
create mode 100644 pve-qemu-server-cpu/src/lib.rs
create mode 100644 pve-qemu-server-cpu/src/qmp_cpu.rs
diff --git a/Cargo.toml b/Cargo.toml
index 71bc8e5..c100e5a 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -1,5 +1,6 @@
[workspace]
members = [
+ "pve-qemu-server-cpu",
"pve-qemu-server-pci"
]
exclude = [
@@ -26,4 +27,3 @@ strum = { version = "0.26", features = ["derive"] }
# proxmox dependencies
pve-api-types = "8.1"
-
diff --git a/build.sh b/build.sh
index 3c2293b..c3ee7b6 100755
--- a/build.sh
+++ b/build.sh
@@ -28,7 +28,7 @@ debcargo package \
--no-overlay-write-back \
--directory "$PWD/"${BUILDDIR}"/${CRATE}" \
"${CRATE}" \
- "$(dpkg-parsechangelog -l "${CRATE}/debian/changelog" -SVersion | sed -e 's/-.*//')"
+ "$(dpkg-parsechangelog -l "${CRATE}/debian/changelog" -SVersion | sed -e 's/-.*//' -e 's/~/-/')"
cd "${BUILDDIR}/${CRATE}"
rm -f debian/source/format.debcargo.hint
diff --git a/pve-qemu-server-cpu/Cargo.toml b/pve-qemu-server-cpu/Cargo.toml
new file mode 100644
index 0000000..b236796
--- /dev/null
+++ b/pve-qemu-server-cpu/Cargo.toml
@@ -0,0 +1,16 @@
+[package]
+name = "pve-qemu-server-cpu"
+version = "0.1.0"
+authors.workspace = true
+edition.workspace = true
+license.workspace = true
+repository.workspace = true
+homepage.workspace = true
+exclude.workspace = true
+rust-version.workspace = true
+
+[dependencies]
+serde = { version = "1", features = ["derive"] }
+serde_json = "1"
+thiserror = "2"
+libc = "0.2"
diff --git a/pve-qemu-server-cpu/debian/changelog b/pve-qemu-server-cpu/debian/changelog
new file mode 100644
index 0000000..3731cfd
--- /dev/null
+++ b/pve-qemu-server-cpu/debian/changelog
@@ -0,0 +1,5 @@
+rust-pve-qemu-server-cpu (0.1.0) trixie; urgency=medium
+
+ * initial release
+
+ -- Proxmox Support Team <support@proxmox.com> Tue, 01 Sep 2026 14:02:25 +0200
diff --git a/pve-qemu-server-cpu/debian/control b/pve-qemu-server-cpu/debian/control
new file mode 100644
index 0000000..1298d61
--- /dev/null
+++ b/pve-qemu-server-cpu/debian/control
@@ -0,0 +1,41 @@
+Source: rust-pve-qemu-server-cpu
+Section: rust
+Priority: optional
+Build-Depends: debhelper-compat (= 13),
+ dh-sequence-cargo,
+ libhwloc-dev (>= 2.12)
+Build-Depends-Arch: cargo:native <!nocheck>,
+ rustc:native (>= 1.94) <!nocheck>,
+ libstd-rust-dev <!nocheck>,
+ librust-libc-0.2+default-dev <!nocheck>,
+ librust-serde-1+default-dev <!nocheck>,
+ librust-serde-1+derive-dev <!nocheck>,
+ librust-serde-json-1+default-dev <!nocheck>,
+ librust-thiserror-2+default-dev <!nocheck>
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Standards-Version: 4.7.2
+Vcs-Git: git://git.proxmox.com/git/pve-qemu-server-rs.git
+Vcs-Browser: https://git.proxmox.com/?p=pve-qemu-server-rs.git
+Homepage: https://proxmox.com
+X-Cargo-Crate: pve-qemu-server-cpu
+
+Package: librust-pve-qemu-server-cpu-dev
+Architecture: any
+Multi-Arch: same
+Depends:
+ ${misc:Depends},
+ librust-libc-0.2+default-dev,
+ librust-serde-1+default-dev,
+ librust-serde-1+derive-dev,
+ librust-serde-json-1+default-dev,
+ librust-thiserror-2+default-dev
+Provides:
+ librust-pve-qemu-server-cpu+default-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0+default-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0.1-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0.1+default-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0.1.0-dev (= ${binary:Version}),
+ librust-pve-qemu-server-cpu-0.1.0+default-dev (= ${binary:Version})
+Description: Rust crate "pve-qemu-server-cpu" - Rust source code
+ Source code for Debianized Rust crate "pve-qemu-server-cpu"
diff --git a/pve-qemu-server-cpu/debian/copyright b/pve-qemu-server-cpu/debian/copyright
new file mode 100644
index 0000000..01138fa
--- /dev/null
+++ b/pve-qemu-server-cpu/debian/copyright
@@ -0,0 +1,18 @@
+Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1.0/
+
+Files:
+ *
+Copyright: 2026 Proxmox Server Solutions GmbH <support@proxmox.com>
+License: AGPL-3.0-or-later
+ This program is free software: you can redistribute it and/or modify it under
+ the terms of the GNU Affero General Public License as published by the Free
+ Software Foundation, either version 3 of the License, or (at your option) any
+ later version.
+ .
+ This program is distributed in the hope that it will be useful, but WITHOUT
+ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License for more
+ details.
+ .
+ You should have received a copy of the GNU Affero General Public License along
+ with this program. If not, see <https://www.gnu.org/licenses/>.
diff --git a/pve-qemu-server-cpu/debian/debcargo.toml b/pve-qemu-server-cpu/debian/debcargo.toml
new file mode 100644
index 0000000..23feea4
--- /dev/null
+++ b/pve-qemu-server-cpu/debian/debcargo.toml
@@ -0,0 +1,9 @@
+overlay = "."
+crate_src_path = ".."
+maintainer = "Proxmox Support Team <support@proxmox.com>"
+allow_prerelease_deps = true
+
+[source]
+vcs_git = "git://git.proxmox.com/git/pve-qemu-server-rs.git"
+vcs_browser = "https://git.proxmox.com/?p=pve-qemu-server-rs.git"
+build_depends = ["libhwloc-dev (>= 2.12)"]
diff --git a/pve-qemu-server-cpu/src/bind.rs b/pve-qemu-server-cpu/src/bind.rs
new file mode 100644
index 0000000..4f2927d
--- /dev/null
+++ b/pve-qemu-server-cpu/src/bind.rs
@@ -0,0 +1,478 @@
+use std::collections::HashSet;
+
+use crate::hwloc::{self, CpuSet, Topology};
+use crate::qmp_cpu::QmpCpu;
+use crate::BindError;
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+struct Cpu(u32);
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+struct NumaNodeId(u32);
+
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Hash)]
+struct Vcpu(u32);
+
+#[derive(Debug, Clone, PartialEq, Eq, Hash)]
+struct VnumaNode {
+ id: NumaNodeId,
+ cpus: Vec<Vcpu>,
+}
+
+#[derive(Debug, Clone)]
+struct BindRule {
+ vcpus: Vec<Vcpu>,
+ cpuset: CpuSet,
+}
+
+#[derive(Debug, Clone)]
+struct NumaBindRule {
+ vnode: VnumaNode,
+ host_node: NumaNodeId,
+}
+
+impl From<u32> for Cpu {
+ fn from(v: u32) -> Self {
+ Self(v)
+ }
+}
+impl From<u32> for Vcpu {
+ fn from(v: u32) -> Self {
+ Self(v)
+ }
+}
+impl From<u32> for NumaNodeId {
+ fn from(v: u32) -> Self {
+ Self(v)
+ }
+}
+
+trait BindableRule {
+ fn vcpus(&self) -> &[Vcpu];
+ fn resolve_cpuset(&self, topo: &Topology) -> Result<CpuSet, BindError>;
+}
+
+impl BindableRule for BindRule {
+ fn vcpus(&self) -> &[Vcpu] {
+ &self.vcpus
+ }
+ fn resolve_cpuset(&self, _topo: &Topology) -> Result<CpuSet, BindError> {
+ Ok(self.cpuset.clone())
+ }
+}
+
+impl BindableRule for NumaBindRule {
+ fn vcpus(&self) -> &[Vcpu] {
+ &self.vnode.cpus
+ }
+ fn resolve_cpuset(&self, topo: &Topology) -> Result<CpuSet, BindError> {
+ hwloc::numa_node_cpuset(topo, self.host_node.0).ok_or(BindError::UnknownHostCpu(self.host_node.0))
+ }
+}
+
+fn bind(topo: &Topology, vcpu: &Vcpu, tid: u32, set: &CpuSet) -> Result<(), BindError> {
+ // Note: DO NOT singlify here — we want the scheduler to be free to move
+ // the vCPU within the specified set.
+ hwloc::bind_thread(topo, tid as i32, set).map_err(|message| BindError::Bind {
+ vcpu: vcpu.0,
+ message,
+ })
+}
+
+fn bind_rules<R: BindableRule>(
+ topo: &Topology,
+ rules: &[R],
+ qmp_cpuinfo: &[QmpCpu],
+) -> Result<(), BindError> {
+ for rule in rules {
+ let set = rule.resolve_cpuset(topo)?;
+ for vcpu in rule.vcpus() {
+ let tid = qmp_cpuinfo
+ .get(vcpu.0 as usize)
+ .ok_or(BindError::ThreadNotFound(vcpu.0))?
+ .thread_id;
+ bind(topo, vcpu, tid, &set)?;
+ }
+ }
+ Ok(())
+}
+
+pub enum Mode {
+ None,
+ Manual,
+ OneToOne,
+ Numa,
+ NumaManual,
+}
+
+impl std::str::FromStr for Mode {
+ type Err = BindError;
+ fn from_str(s: &str) -> Result<Self, Self::Err> {
+ match s {
+ "none" => Ok(Mode::None),
+ "manual" => Ok(Mode::Manual),
+ "one-to-one" => Ok(Mode::OneToOne),
+ "numa" => Ok(Mode::Numa),
+ "numa-manual" => Ok(Mode::NumaManual),
+ other => Err(BindError::BadSyntax(format!("unknown cpu-bind-mode '{other}'"))),
+ }
+ }
+}
+
+fn to_cpuset(cpus: &[u32]) -> CpuSet {
+ let mut set = CpuSet::new();
+ for &c in cpus {
+ set.set(c);
+ }
+ set
+}
+
+fn parse_cpu_binding_map(map: &str) -> Result<Vec<BindRule>, BindError> {
+ if map.is_empty() {
+ return Err(BindError::BadSyntax("binding map is empty".into()));
+ }
+
+ let mut rules = Vec::new();
+ for entry in map.split(';') {
+ let (vcpu_side, host_side) = entry
+ .split_once(':')
+ .ok_or_else(|| BindError::MissingColon(entry.to_string()))?;
+
+ let vcpus: Vec<Vcpu> = parse_side(vcpu_side)?.into_iter().map(Vcpu::from).collect();
+ let cpuset = to_cpuset(&parse_side(host_side)?);
+
+ rules.push(BindRule { vcpus, cpuset });
+ }
+ Ok(rules)
+}
+
+fn parse_side(s: &str) -> Result<Vec<u32>, BindError> {
+ if let Some(inner) = s.strip_prefix('{').and_then(|r| r.strip_suffix('}')) {
+ let mut out = Vec::new();
+ for part in inner.split(',') {
+ out.extend(parse_element(part)?);
+ }
+ Ok(out)
+ } else {
+ if s.contains(',') {
+ return Err(BindError::BadSyntax(format!(
+ "unions require braces: write '{{0,1}}', not '{s}'"
+ )));
+ }
+ parse_element(s)
+ }
+}
+
+fn parse_element(s: &str) -> Result<Vec<u32>, BindError> {
+ if let Some((a, b)) = s.split_once('-') {
+ let a: u32 = a
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad index in '{s}'")))?;
+ let b: u32 = b
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad index in '{s}'")))?;
+ if a > b {
+ return Err(BindError::BadSyntax(format!("reversed range '{s}'")));
+ }
+ Ok((a..=b).collect())
+ } else {
+ Ok(vec![s
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad index '{s}'")))?])
+ }
+}
+
+fn parse_index_list<T: From<u32>>(s: &str) -> Result<Vec<T>, BindError> {
+ if s.is_empty() {
+ return Ok(Vec::new());
+ }
+ let mut out = Vec::new();
+ for part in s.split(';') {
+ out.extend(parse_element(part)?.into_iter().map(T::from));
+ }
+ Ok(out)
+}
+
+fn parse_numa_binding_map(map: &str) -> Result<Vec<(NumaNodeId, NumaNodeId)>, BindError> {
+ if map.is_empty() {
+ return Err(BindError::BadSyntax("numa binding map is empty".into()));
+ }
+ let mut pairs = Vec::new();
+ for entry in map.split(';') {
+ let entry = entry.trim();
+ let (v, h) = entry
+ .split_once(':')
+ .ok_or_else(|| BindError::MissingColon(entry.to_string()))?;
+ let vnode: u32 = v
+ .trim()
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad vNUMA node in '{entry}'")))?;
+ let hnode: u32 = h
+ .trim()
+ .parse()
+ .map_err(|_| BindError::BadSyntax(format!("bad host NUMA node in '{entry}'")))?;
+ pairs.push((NumaNodeId(vnode), NumaNodeId(hnode)));
+ }
+ Ok(pairs)
+}
+
+fn collect_vnuma_nodes(qmp_cpuinfo: &[QmpCpu]) -> Vec<VnumaNode> {
+ let max_node = qmp_cpuinfo
+ .iter()
+ .map(|c| c.props.node_id)
+ .max()
+ .unwrap_or(0);
+ let mut buckets: Vec<Vec<Vcpu>> = vec![Vec::new(); (max_node + 1) as usize];
+ for cpu in qmp_cpuinfo {
+ buckets[cpu.props.node_id as usize].push(Vcpu::from(cpu.cpu_index));
+ }
+ buckets
+ .into_iter()
+ .enumerate()
+ .map(|(id, cpus)| VnumaNode {
+ id: NumaNodeId(id as u32),
+ cpus,
+ })
+ .collect()
+}
+
+fn build_numa_rules(
+ pairs: &[(NumaNodeId, NumaNodeId)],
+ vnuma_nodes: &[VnumaNode],
+) -> Result<Vec<NumaBindRule>, BindError> {
+ let mut rules = Vec::with_capacity(pairs.len());
+ for (vnode_id, hnode_id) in pairs {
+ let vnode = vnuma_nodes
+ .get(vnode_id.0 as usize)
+ .ok_or_else(|| {
+ BindError::BadSyntax(format!(
+ "vNUMA node {} does not exist in this guest",
+ vnode_id.0
+ ))
+ })?
+ .clone();
+ rules.push(NumaBindRule {
+ vnode,
+ host_node: *hnode_id,
+ });
+ }
+ Ok(rules)
+}
+
+pub fn bind_manual(binding_map: &str, qmp_cpuinfo: &[QmpCpu]) -> Result<(), BindError> {
+ let rules = parse_cpu_binding_map(binding_map)?;
+ let topo = Topology::new()
+ .map_err(|e| BindError::TopologyInit(format!("topology init failed: {e}")))?;
+ bind_rules(&topo, &rules, qmp_cpuinfo)
+}
+
+pub fn bind_one_to_one(binding_list: &str, qmp_cpuinfo: &[QmpCpu]) -> Result<(), BindError> {
+ // TODO: use complex distribution logic like:
+ // https://github.com/open-mpi/hwloc/blob/master/include/hwloc/helper.h::hwloc_distrib()
+ // https://github.com/HadrienG2/hwlocality/blob/main/src/topology/mod.rs:distribute_items()
+ let topo = Topology::new()
+ .map_err(|e| BindError::TopologyInit(format!("topology init failed: {e}")))?;
+
+ let mut allowed: CpuSet = if binding_list.is_empty() {
+ topo.allowed_cpuset()
+ } else {
+ let cpus: Vec<Cpu> = parse_index_list(binding_list)?;
+ let mut s = CpuSet::new();
+ for c in cpus {
+ s.set(c.0);
+ }
+ s
+ };
+
+ // One vCPU per physical core: strip SMT siblings.
+ allowed.singlify_per_core(&topo);
+
+ let available: Vec<u32> = allowed.iter_set().collect();
+
+ if available.len() < qmp_cpuinfo.len() {
+ return Err(BindError::BadSyntax(format!(
+ "one-to-one mode requires {} physical cores but only {} are available",
+ qmp_cpuinfo.len(),
+ available.len(),
+ )));
+ }
+
+ let mut rules: Vec<BindRule> = Vec::with_capacity(qmp_cpuinfo.len());
+ for (i, cpu) in available.iter().take(qmp_cpuinfo.len()).enumerate() {
+ rules.push(BindRule {
+ vcpus: vec![Vcpu::from(qmp_cpuinfo[i].cpu_index)],
+ cpuset: CpuSet::single(*cpu),
+ });
+ }
+
+ bind_rules(&topo, &rules, qmp_cpuinfo)
+}
+
+pub fn bind_numa_manual(binding_map: &str, qmp_cpuinfo: &[QmpCpu]) -> Result<(), BindError> {
+ let topo = Topology::new()
+ .map_err(|e| BindError::TopologyInit(format!("topology init failed: {e}")))?;
+ let pairs = parse_numa_binding_map(binding_map)?;
+ let vnuma_nodes = collect_vnuma_nodes(qmp_cpuinfo);
+ let rules = build_numa_rules(&pairs, &vnuma_nodes)?;
+ bind_rules(&topo, &rules, qmp_cpuinfo)
+}
+
+pub fn bind_numa(binding_list: &str, qmp_cpuinfo: &[QmpCpu]) -> Result<(), BindError> {
+ let topo = Topology::new()
+ .map_err(|e| BindError::TopologyInit(format!("topology init failed: {e}")))?;
+ let vnuma_nodes = collect_vnuma_nodes(qmp_cpuinfo);
+
+ let mut configured_nodes: Vec<NumaNodeId> = parse_index_list(binding_list)?;
+ if configured_nodes.is_empty() {
+ configured_nodes = topo.allowed_nodeset().iter_set().map(NumaNodeId).collect();
+ }
+
+ if configured_nodes.len() < vnuma_nodes.len() {
+ return Err(BindError::BadSyntax(format!(
+ "VM has {} vNUMA nodes but only {} host NUMA nodes configured",
+ vnuma_nodes.len(),
+ configured_nodes.len()
+ )));
+ }
+
+ let pairs: Vec<(NumaNodeId, NumaNodeId)> = (0..vnuma_nodes.len())
+ .map(|i| (NumaNodeId(i as u32), configured_nodes[i]))
+ .collect();
+
+ let rules = build_numa_rules(&pairs, &vnuma_nodes)?;
+ bind_rules(&topo, &rules, qmp_cpuinfo)
+}
+
+pub fn validate_manual(sockets: u32, cores: u32, binding_map: &str) -> Result<(), BindError> {
+ if binding_map.is_empty() {
+ return Err(BindError::BadSyntax("cpu-bind-map is empty".into()));
+ }
+
+ let rules = parse_cpu_binding_map(binding_map)?;
+ let nr_vcpus = sockets.saturating_mul(cores);
+
+ let topo = Topology::new()
+ .map_err(|e| BindError::BadSyntax(format!("topology init failed: {e}")))?;
+ let topo_cpuset = topo.allowed_cpuset();
+
+ let mut seen_vcpu: HashSet<u32> = HashSet::new();
+
+ for rule in &rules {
+ if !rule.cpuset.is_subset_of(&topo_cpuset) {
+ let missing = rule.cpuset.difference(&topo_cpuset);
+ let first = missing.first_set().ok_or_else(|| {
+ BindError::BadSyntax("internal error: subset check failed but no CPU missing".into())
+ })?;
+ return Err(BindError::UnknownHostCpu(first));
+ }
+ for vcpu in &rule.vcpus {
+ if vcpu.0 >= nr_vcpus {
+ return Err(BindError::VcpuOutOfRange {
+ vcpu: vcpu.0,
+ max: nr_vcpus,
+ });
+ }
+ if !seen_vcpu.insert(vcpu.0) {
+ return Err(BindError::VcpuReused(vcpu.0));
+ }
+ }
+ }
+ Ok(())
+}
+
+pub fn validate_one_to_one(sockets: u32, cores: u32, binding_list: &str) -> Result<(), BindError> {
+ let nr_vcpus = sockets.saturating_mul(cores);
+
+ let topo = Topology::new()
+ .map_err(|e| BindError::BadSyntax(format!("topology init failed: {e}")))?;
+ let allowed = topo.allowed_cpuset();
+
+ let mut available: CpuSet = if binding_list.is_empty() {
+ allowed
+ } else {
+ let cpus: Vec<Cpu> = parse_index_list(binding_list)?;
+ let mut set = CpuSet::new();
+ for cpu in &cpus {
+ if !allowed.isset(cpu.0) {
+ return Err(BindError::UnknownHostCpu(cpu.0));
+ }
+ set.set(cpu.0);
+ }
+ set
+ };
+
+ available.singlify_per_core(&topo);
+
+ let core_count = available.weight();
+ if core_count < nr_vcpus as usize {
+ return Err(BindError::BadSyntax(format!(
+ "one-to-one mode requires {nr_vcpus} physical cores but only {core_count} are available",
+ )));
+ }
+ Ok(())
+}
+
+pub fn validate_numa(sockets: u32, binding_list: &str) -> Result<(), BindError> {
+ let topo = Topology::new()
+ .map_err(|e| BindError::BadSyntax(format!("topology init failed: {e}")))?;
+ let topo_nodeset = topo.allowed_nodeset();
+
+ let host_node_ids: Vec<u32> = if binding_list.is_empty() {
+ topo_nodeset.iter_set().collect()
+ } else {
+ let ids: Vec<NumaNodeId> = parse_index_list(binding_list)?;
+ for id in &ids {
+ if !topo_nodeset.isset(id.0) {
+ return Err(BindError::BadSyntax(format!(
+ "host NUMA node {} does not exist on this host",
+ id.0
+ )));
+ }
+ }
+ ids.into_iter().map(|n| n.0).collect()
+ };
+
+ if (sockets as usize) > host_node_ids.len() {
+ return Err(BindError::BadSyntax(format!(
+ "VM has {sockets} vNUMA nodes but only {} host NUMA nodes available",
+ host_node_ids.len(),
+ )));
+ }
+ Ok(())
+}
+
+pub fn validate_numa_manual(sockets: u32, binding_map: &str) -> Result<(), BindError> {
+ if binding_map.is_empty() {
+ return Err(BindError::BadSyntax("numa-binding-map is empty".into()));
+ }
+
+ let pairs = parse_numa_binding_map(binding_map)?;
+
+ let topo = Topology::new()
+ .map_err(|e| BindError::BadSyntax(format!("topology init failed: {e}")))?;
+ let topo_nodeset = topo.allowed_nodeset();
+
+ let mut seen_vnode: HashSet<u32> = HashSet::new();
+
+ for (vnode, hnode) in &pairs {
+ if !topo_nodeset.isset(hnode.0) {
+ return Err(BindError::BadSyntax(format!(
+ "host NUMA node {} does not exist on this host",
+ hnode.0
+ )));
+ }
+ if vnode.0 >= sockets {
+ return Err(BindError::BadSyntax(format!(
+ "guest vNUMA node {} is out of range (VM has {sockets} sockets)",
+ vnode.0
+ )));
+ }
+ if !seen_vnode.insert(vnode.0) {
+ return Err(BindError::BadSyntax(format!(
+ "guest vNUMA node {} appears more than once",
+ vnode.0
+ )));
+ }
+ }
+ Ok(())
+}
diff --git a/pve-qemu-server-cpu/src/error.rs b/pve-qemu-server-cpu/src/error.rs
new file mode 100644
index 0000000..73a569e
--- /dev/null
+++ b/pve-qemu-server-cpu/src/error.rs
@@ -0,0 +1,38 @@
+use thiserror::Error;
+
+#[derive(Debug, Error)]
+pub enum BindError {
+ // Parse errors
+ #[error("map entry '{0}' is missing a ':' separator")]
+ MissingColon(String),
+
+ #[error("invalid syntax in binding map: '{0}'")]
+ BadSyntax(String),
+
+ // Validation errors (pre-start)
+ #[error("host CPU {0} does not exist in this topology")]
+ UnknownHostCpu(u32),
+
+ #[error("vCPU {0} appears in more than one map entry")]
+ VcpuReused(u32),
+
+ #[error("vCPU {vcpu} is out of range (VM has vCPUs 0..{max})")]
+ VcpuOutOfRange { vcpu: u32, max: u32 },
+
+ // Runtime errors (post-start)
+ #[error("no running thread found for vCPU {0}")]
+ ThreadNotFound(u32),
+
+ #[error("invalid kernel TID {tid} for vCPU {vcpu}")]
+ BadThreadId { vcpu: u32, tid: u32 },
+
+ #[error("failed to init host topology: '{0}'")]
+ TopologyInit(String),
+
+ #[error("binding vCPU {vcpu} failed: {message}")]
+ Bind {
+ vcpu: u32,
+ /// Human-readable error from libhwloc (errno-based).
+ message: String,
+ },
+}
diff --git a/pve-qemu-server-cpu/src/hwloc/bindings.rs b/pve-qemu-server-cpu/src/hwloc/bindings.rs
new file mode 100644
index 0000000..7e9c5bc
--- /dev/null
+++ b/pve-qemu-server-cpu/src/hwloc/bindings.rs
@@ -0,0 +1,171 @@
+//! Minimal raw FFI bindings for libhwloc.
+//!
+//! Taken from https://github.com/HadrienG2/hwlocality/blob/main/hwlocality-sys/src/lib.rs
+
+#![allow(non_camel_case_types)]
+
+use std::cell::UnsafeCell;
+use std::marker::{PhantomData, PhantomPinned};
+
+use libc::{c_char, c_int, c_uint, c_void, pid_t};
+
+#[repr(C)]
+struct IncompleteType {
+ _data: [u8; 0],
+ _marker: PhantomData<(*mut u8, PhantomPinned, &'static UnsafeCell<u8>)>,
+}
+
+#[repr(C)]
+pub struct hwloc_topology(IncompleteType);
+#[repr(C)]
+pub struct hwloc_bitmap_s(IncompleteType);
+
+#[repr(C)]
+pub struct hwloc_obj_attr_u(IncompleteType);
+#[repr(C)]
+pub struct hwloc_info_s(IncompleteType);
+
+pub type hwloc_topology_t = *mut hwloc_topology;
+pub type hwloc_const_topology_t = *const hwloc_topology;
+
+pub type hwloc_bitmap_t = *mut hwloc_bitmap_s;
+pub type hwloc_const_bitmap_t = *const hwloc_bitmap_s;
+
+pub type hwloc_cpuset_t = hwloc_bitmap_t;
+pub type hwloc_const_cpuset_t = hwloc_const_bitmap_t;
+pub type hwloc_nodeset_t = hwloc_bitmap_t;
+pub type hwloc_const_nodeset_t = hwloc_const_bitmap_t;
+
+pub type hwloc_obj_type_t = c_uint;
+pub const HWLOC_OBJ_MACHINE: hwloc_obj_type_t = 0;
+pub const HWLOC_OBJ_PACKAGE: hwloc_obj_type_t = 1;
+pub const HWLOC_OBJ_CORE: hwloc_obj_type_t = 2;
+pub const HWLOC_OBJ_PU: hwloc_obj_type_t = 3;
+pub const HWLOC_OBJ_NUMANODE: hwloc_obj_type_t = 13;
+
+pub type hwloc_get_type_depth_e = c_int;
+pub const HWLOC_TYPE_DEPTH_UNKNOWN: hwloc_get_type_depth_e = -1;
+pub const HWLOC_TYPE_DEPTH_MULTIPLE: hwloc_get_type_depth_e = -2;
+pub const HWLOC_TYPE_DEPTH_NUMANODE: hwloc_get_type_depth_e = -3;
+
+pub type hwloc_cpubind_flags_t = c_int;
+pub const HWLOC_CPUBIND_THREAD: hwloc_cpubind_flags_t = 1 << 1;
+pub const HWLOC_CPUBIND_PROCESS: hwloc_cpubind_flags_t = 1 << 0;
+pub const HWLOC_CPUBIND_STRICT: hwloc_cpubind_flags_t = 1 << 2;
+pub const HWLOC_CPUBIND_NOMEMBIND: hwloc_cpubind_flags_t = 1 << 3;
+
+#[repr(C)]
+pub struct hwloc_obj {
+ pub ty: hwloc_obj_type_t,
+ pub subtype: *mut c_char,
+ pub os_index: c_uint,
+ pub name: *mut c_char,
+ pub total_memory: u64,
+ pub attr: *mut hwloc_obj_attr_u,
+ pub depth: hwloc_get_type_depth_e,
+ pub logical_index: c_uint,
+ pub next_cousin: hwloc_obj_t,
+ pub prev_cousin: hwloc_obj_t,
+ pub parent: hwloc_obj_t,
+ pub sibling_rank: c_uint,
+ pub next_sibling: hwloc_obj_t,
+ pub prev_sibling: hwloc_obj_t,
+ pub arity: c_uint,
+ pub children: *mut hwloc_obj_t,
+ pub first_child: hwloc_obj_t,
+ pub last_child: hwloc_obj_t,
+ pub symmetric_subtree: c_int,
+ pub memory_arity: c_uint,
+ pub memory_first_child: hwloc_obj_t,
+ pub io_arity: c_uint,
+ pub io_first_child: hwloc_obj_t,
+ pub misc_arity: c_uint,
+ pub misc_first_child: hwloc_obj_t,
+ pub cpuset: hwloc_cpuset_t,
+ pub complete_cpuset: hwloc_cpuset_t,
+ pub nodeset: hwloc_nodeset_t,
+ pub complete_nodeset: hwloc_nodeset_t,
+ pub infos: *mut hwloc_info_s,
+ pub infos_count: c_uint,
+ pub userdata: *mut c_void,
+ pub gp_index: u64,
+}
+pub type hwloc_obj_t = *mut hwloc_obj;
+
+#[link(name = "hwloc")]
+unsafe extern "C" {
+ pub fn hwloc_topology_init(topology: *mut hwloc_topology_t) -> c_int;
+ pub fn hwloc_topology_load(topology: hwloc_topology_t) -> c_int;
+ pub fn hwloc_topology_destroy(topology: hwloc_topology_t);
+
+ pub fn hwloc_topology_get_topology_cpuset(
+ topology: hwloc_const_topology_t,
+ ) -> hwloc_const_cpuset_t;
+ pub fn hwloc_topology_get_topology_nodeset(
+ topology: hwloc_const_topology_t,
+ ) -> hwloc_const_nodeset_t;
+
+ pub fn hwloc_topology_get_allowed_cpuset(
+ topology: hwloc_const_topology_t,
+ ) -> hwloc_const_cpuset_t;
+ pub fn hwloc_topology_get_allowed_nodeset(
+ topology: hwloc_const_topology_t,
+ ) -> hwloc_const_nodeset_t;
+
+ pub fn hwloc_get_type_depth(
+ topology: hwloc_const_topology_t,
+ object_type: hwloc_obj_type_t,
+ ) -> hwloc_get_type_depth_e;
+ pub fn hwloc_get_nbobjs_by_depth(
+ topology: hwloc_const_topology_t,
+ depth: hwloc_get_type_depth_e,
+ ) -> c_uint;
+ pub fn hwloc_get_obj_by_depth(
+ topology: hwloc_const_topology_t,
+ depth: hwloc_get_type_depth_e,
+ idx: c_uint,
+ ) -> hwloc_obj_t;
+
+ pub fn hwloc_bitmap_alloc() -> hwloc_bitmap_t;
+ pub fn hwloc_bitmap_free(bitmap: hwloc_bitmap_t);
+ pub fn hwloc_bitmap_dup(src: hwloc_const_bitmap_t) -> hwloc_bitmap_t;
+
+ pub fn hwloc_bitmap_set(bitmap: hwloc_bitmap_t, id: c_uint) -> c_int;
+ pub fn hwloc_bitmap_isset(bitmap: hwloc_const_bitmap_t, id: c_uint) -> c_int;
+ pub fn hwloc_bitmap_weight(bitmap: hwloc_const_bitmap_t) -> c_int;
+ pub fn hwloc_bitmap_first(bitmap: hwloc_const_bitmap_t) -> c_int;
+ pub fn hwloc_bitmap_next(bitmap: hwloc_const_bitmap_t, prev: c_int) -> c_int;
+
+ pub fn hwloc_bitmap_isincluded(
+ left: hwloc_const_bitmap_t,
+ right: hwloc_const_bitmap_t,
+ ) -> c_int;
+ pub fn hwloc_bitmap_and(
+ result: hwloc_bitmap_t,
+ bitmap1: hwloc_const_bitmap_t,
+ bitmap2: hwloc_const_bitmap_t,
+ ) -> c_int;
+ pub fn hwloc_bitmap_andnot(
+ result: hwloc_bitmap_t,
+ bitmap1: hwloc_const_bitmap_t,
+ bitmap2: hwloc_const_bitmap_t,
+ ) -> c_int;
+ pub fn hwloc_bitmap_not(result: hwloc_bitmap_t, bitmap: hwloc_const_bitmap_t) -> c_int;
+ pub fn hwloc_bitmap_or(
+ result: hwloc_bitmap_t,
+ bitmap1: hwloc_const_bitmap_t,
+ bitmap2: hwloc_const_bitmap_t,
+ ) -> c_int;
+
+ pub fn hwloc_bitmap_singlify_per_core(
+ topology: hwloc_const_topology_t,
+ cpuset: hwloc_cpuset_t,
+ which: c_uint,
+ ) -> c_int;
+
+ pub fn hwloc_linux_set_tid_cpubind(
+ topology: hwloc_const_topology_t,
+ tid: pid_t,
+ set: hwloc_const_cpuset_t,
+ ) -> c_int;
+}
diff --git a/pve-qemu-server-cpu/src/hwloc/mod.rs b/pve-qemu-server-cpu/src/hwloc/mod.rs
new file mode 100644
index 0000000..2e3e741
--- /dev/null
+++ b/pve-qemu-server-cpu/src/hwloc/mod.rs
@@ -0,0 +1,222 @@
+//! Safe wrappers around the libhwloc FFI bindings.
+//!
+//! Replace with https://github.com/HadrienG2/hwlocality
+//! once the crate is mature enough
+
+mod bindings;
+
+use std::io;
+use bindings::*;
+
+/// Return the current `errno` as a message. hwloc reports errors via `errno`
+/// and a `-1` return value, so this is what you feed into user-facing errors.
+fn last_error(ctx: &str) -> String {
+ format!("{ctx}: {}", io::Error::last_os_error())
+}
+
+fn check(rc: std::ffi::c_int, ctx: &str) -> Result<(), String> {
+ if rc < 0 {
+ Err(last_error(ctx))
+ } else {
+ Ok(())
+ }
+}
+
+pub struct Topology(*mut hwloc_topology);
+
+impl Topology {
+ pub fn new() -> Result<Self, String> {
+ let mut raw: hwloc_topology_t = std::ptr::null_mut();
+ unsafe { check(hwloc_topology_init(&mut raw), "hwloc_topology_init")? };
+ let rc = unsafe { hwloc_topology_load(raw) };
+ if rc < 0 {
+ unsafe { hwloc_topology_destroy(raw) };
+ return Err(last_error("hwloc_topology_load"));
+ }
+ Ok(Topology(raw))
+ }
+
+ pub fn cpuset(&self) -> CpuSet {
+ unsafe {
+ let raw = hwloc_topology_get_topology_cpuset(self.0);
+ CpuSet::dup_from(raw)
+ }
+ }
+
+ pub fn nodeset(&self) -> CpuSet {
+ unsafe {
+ let raw = hwloc_topology_get_topology_nodeset(self.0);
+ CpuSet::dup_from(raw)
+ }
+ }
+
+ pub fn allowed_cpuset(&self) -> CpuSet {
+ unsafe {
+ let raw = hwloc_topology_get_allowed_cpuset(self.0);
+ CpuSet::dup_from(raw)
+ }
+ }
+
+ pub fn allowed_nodeset(&self) -> CpuSet {
+ unsafe {
+ let raw = hwloc_topology_get_allowed_nodeset(self.0);
+ CpuSet::dup_from(raw)
+ }
+ }
+
+ pub(crate) fn as_raw(&self) -> hwloc_const_topology_t {
+ self.0
+ }
+}
+
+impl Drop for Topology {
+ fn drop(&mut self) {
+ unsafe { hwloc_topology_destroy(self.0) };
+ }
+}
+
+pub struct CpuSet(*mut hwloc_bitmap_s);
+
+impl Clone for CpuSet {
+ fn clone(&self) -> Self {
+ unsafe {
+ let owned = hwloc_bitmap_dup(self.0);
+ assert!(!owned.is_null(), "hwloc_bitmap_dup failed");
+ CpuSet(owned)
+ }
+ }
+}
+
+impl std::fmt::Debug for CpuSet {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
+ f.debug_tuple("CpuSet")
+ .field(&self.iter_set().collect::<Vec<_>>())
+ .finish()
+ }
+}
+
+impl CpuSet {
+ pub fn new() -> Self {
+ let raw = unsafe { hwloc_bitmap_alloc() };
+ assert!(!raw.is_null(), "hwloc_bitmap_alloc failed");
+ CpuSet(raw)
+ }
+
+ /// Allocate a bitmap containing exactly one CPU.
+ pub fn single(idx: u32) -> Self {
+ let mut set = Self::new();
+ set.set(idx);
+ set
+ }
+
+ /// Clone a borrowed bitmap returned by the topology.
+ fn dup_from(raw: hwloc_const_bitmap_t) -> Self {
+ let owned = unsafe { hwloc_bitmap_dup(raw) };
+ assert!(!owned.is_null(), "hwloc_bitmap_dup failed");
+ CpuSet(owned)
+ }
+
+ pub fn difference(&self, other: &CpuSet) -> CpuSet {
+ let out = CpuSet::new();
+ let rc = unsafe { hwloc_bitmap_andnot(out.0, self.0, other.0) };
+ assert!(rc >= 0, "hwloc_bitmap_andnot failed");
+ out
+ }
+
+ pub fn intersect(&mut self, other: &CpuSet) {
+ let rc = unsafe { hwloc_bitmap_and(self.0, self.0, other.0) };
+ assert!(rc >= 0, "hwloc_bitmap_and failed");
+ }
+
+ pub fn set(&mut self, idx: u32) {
+ let rc = unsafe { hwloc_bitmap_set(self.0, idx) };
+ assert!(rc >= 0, "hwloc_bitmap_set failed");
+ }
+
+ pub fn isset(&self, idx: u32) -> bool {
+ unsafe { hwloc_bitmap_isset(self.0, idx) != 0 }
+ }
+
+ pub fn weight(&self) -> usize {
+ let w = unsafe { hwloc_bitmap_weight(self.0) };
+ w.max(0) as usize
+ }
+
+ pub fn is_empty(&self) -> bool {
+ self.first_set().is_none()
+ }
+
+ pub fn first_set(&self) -> Option<u32> {
+ let i = unsafe { hwloc_bitmap_first(self.0) };
+ if i < 0 { None } else { Some(i as u32) }
+ }
+
+ fn next_set_after(&self, prev: u32) -> Option<u32> {
+ let i = unsafe { hwloc_bitmap_next(self.0, prev as std::ffi::c_int) };
+ if i < 0 { None } else { Some(i as u32) }
+ }
+
+ pub fn iter_set(&self) -> impl Iterator<Item = u32> + '_ {
+ std::iter::successors(self.first_set(), move |&cur| self.next_set_after(cur))
+ }
+
+ pub fn is_subset_of(&self, other: &CpuSet) -> bool {
+ unsafe { hwloc_bitmap_isincluded(self.0, other.0) != 0 }
+ }
+
+ /// Remove SMT siblings from each core (requires hwloc 2.2+).
+ pub fn singlify_per_core(&mut self, topo: &Topology) {
+ let rc = unsafe { hwloc_bitmap_singlify_per_core(topo.as_raw(), self.0, 0) };
+ assert!(rc >= 0, "hwloc_bitmap_singlify_per_core failed");
+ }
+}
+
+impl Drop for CpuSet {
+ fn drop(&mut self) {
+ unsafe { hwloc_bitmap_free(self.0) };
+ }
+}
+
+/// Return the CPU set belonging to the host NUMA node with the given OS index.
+///
+/// The returned set is intersected with the topology's allowed CPU set.
+/// Returns `None` if the node does not exist,or if it has no allowed
+/// CPUs (e.g., a CPU-less memory node).
+pub fn numa_node_cpuset(topo: &Topology, node_id: u32) -> Option<CpuSet> {
+ let depth = unsafe { hwloc_get_type_depth(topo.as_raw(), HWLOC_OBJ_NUMANODE) };
+ if depth == HWLOC_TYPE_DEPTH_UNKNOWN {
+ return None;
+ }
+
+ let count = unsafe { hwloc_get_nbobjs_by_depth(topo.as_raw(), depth) };
+ for i in 0..count {
+ let obj = unsafe { hwloc_get_obj_by_depth(topo.as_raw(), depth, i) };
+ if obj.is_null() {
+ continue;
+ }
+
+ if unsafe { (*obj).os_index } == node_id {
+ let raw = unsafe { (*obj).cpuset };
+
+ // hwloc guarantees cpuset != NULL for NUMA nodes.
+ let mut set = CpuSet::dup_from(raw);
+
+ let allowed = topo.allowed_cpuset();
+ set.intersect(&allowed);
+
+ if set.is_empty() {
+ return None;
+ }
+
+ return Some(set);
+ }
+ }
+
+ None
+}
+
+#[cfg(target_os = "linux")]
+pub fn bind_thread(topo: &Topology, tid: i32, set: &CpuSet) -> Result<(), String> {
+ let rc = unsafe { hwloc_linux_set_tid_cpubind(topo.as_raw(), tid, set.0) };
+ check(rc, "hwloc_linux_set_tid_cpubind")
+}
diff --git a/pve-qemu-server-cpu/src/lib.rs b/pve-qemu-server-cpu/src/lib.rs
new file mode 100644
index 0000000..e297bd9
--- /dev/null
+++ b/pve-qemu-server-cpu/src/lib.rs
@@ -0,0 +1,92 @@
+mod bind;
+mod hwloc;
+mod qmp_cpu;
+mod error;
+
+pub use error::BindError;
+
+pub fn validate_binding(
+ mode: &str,
+ sockets: u32,
+ cores: u32,
+ binding_cfg: &str,
+) -> Result<(), BindError> {
+ match mode.parse::<bind::Mode>()? {
+ bind::Mode::None => Ok(()),
+ bind::Mode::Manual => bind::validate_manual(sockets, cores, binding_cfg),
+ bind::Mode::OneToOne => bind::validate_one_to_one(sockets, cores, binding_cfg),
+ bind::Mode::Numa => bind::validate_numa(sockets, binding_cfg),
+ bind::Mode::NumaManual => bind::validate_numa_manual(sockets, binding_cfg),
+ }
+}
+
+/// Apply CPU pinning to a running VM.
+///
+/// The function parses the QMP `query-cpus-fast` response and binds each
+/// vCPU thread according to the selected mode.
+///
+/// # Arguments
+///
+/// * `mode` - Pinning mode: `none`, `manual`, `one-to-one`, `numa`, or
+/// `numa-manual`.
+/// * `binding_info` - Mode-specific CPU or NUMA mapping. Ignored for `none`.
+/// * `cpuinfo_json` - JSON output from QMP `query-cpus-fast`.
+///
+/// # Modes
+///
+/// | Mode | Meaning |
+/// |---|---|
+/// | `none` | Do not pin vCPUs. |
+/// | `manual` | Explicitly map vCPUs to host CPUs. |
+/// | `one-to-one` | Assign each vCPU to its own physical core. |
+/// | `numa` | Map guest NUMA nodes to host NUMA nodes in order. |
+/// | `numa-manual` | Explicitly map guest NUMA nodes to host NUMA nodes. |
+///
+/// # Examples
+///
+/// ```ignore
+///
+/// // Manual mode:
+/// // vCPU 0 -> host CPUs 2-3
+/// // vCPU 1 -> host CPU 4
+/// bind_vcpus("manual", "0:2-3;1:4", &cpuinfo_json)?;
+///
+/// // One-to-one mode:
+/// // Distribute vCPUs over host CPUs 0-7.
+/// bind_vcpus("one-to-one", "0-7", &cpuinfo_json)?;
+///
+/// // One-to-one mode over all allowed host CPUs.
+/// bind_vcpus("one-to-one", "", &cpuinfo_json)?;
+///
+/// // NUMA mode:
+/// // Guest NUMA node 0 -> host NUMA node 0
+/// // Guest NUMA node 1 -> host NUMA node 1
+/// bind_vcpus("numa", "0;1", &cpuinfo_json)?;
+///
+/// // NUMA mode over all allowed host NUMA nodes.
+/// bind_vcpus("numa", "", &cpuinfo_json)?;
+///
+/// // Manual NUMA mode:
+/// // Guest NUMA node 0 -> host NUMA node 1
+/// // Guest NUMA node 1 -> host NUMA node 0
+/// bind_vcpus("numa-manual", "0:1;1:0", &cpuinfo_json)?;
+/// ```
+///
+/// # Errors
+///
+/// Returns [`BindError`] if the mode or binding syntax is invalid, if the
+/// requested host CPUs or NUMA nodes do not exist, or if binding fails.
+pub fn bind_vcpus(
+ mode: &str,
+ binding_cfg: &str,
+ cpuinfo_json: &str,
+) -> Result<(), BindError> {
+ let qmp_cpuinfo = qmp_cpu::parse_cpuinfo(cpuinfo_json)?;
+ match mode.parse::<bind::Mode>()? {
+ bind::Mode::None => Ok(()),
+ bind::Mode::Manual => bind::bind_manual(binding_cfg, &qmp_cpuinfo),
+ bind::Mode::OneToOne => bind::bind_one_to_one(binding_cfg, &qmp_cpuinfo),
+ bind::Mode::Numa => bind::bind_numa(binding_cfg, &qmp_cpuinfo),
+ bind::Mode::NumaManual => bind::bind_numa_manual(binding_cfg, &qmp_cpuinfo),
+ }
+}
diff --git a/pve-qemu-server-cpu/src/qmp_cpu.rs b/pve-qemu-server-cpu/src/qmp_cpu.rs
new file mode 100644
index 0000000..00d88c5
--- /dev/null
+++ b/pve-qemu-server-cpu/src/qmp_cpu.rs
@@ -0,0 +1,99 @@
+//! Parsing of the QEMU `query-cpus-fast` QMP response.
+//!
+//! The response is a JSON array; each element describes one guest vCPU
+//! and carries both its guest-visible topology (socket/core/NUMA node)
+//! and its host kernel TID. Every binding mode consumes this shape, so
+//! the parsing lives here rather than inside any individual mode.
+//!
+//! The full QEMU schema is modeled, including fields not currently used
+//! by any binding mode. Serde ignores fields it doesn't know about, but
+//! keeping them explicit documents the wire format and makes future
+//! modes cheaper to implement.
+
+use serde::Deserialize;
+
+use crate::BindError;
+
+/// One entry of the `query-cpus-fast` array.
+#[derive(Debug, Clone, Deserialize)]
+pub struct QmpCpu {
+ /// Guest-visible vCPU index (0 .. sockets*cores).
+ #[serde(rename = "cpu-index")]
+ pub cpu_index: u32,
+
+ /// Host kernel TID of the QEMU vCPU thread.
+ #[serde(rename = "thread-id")]
+ pub thread_id: u32,
+
+ /// QOM path of the CPU object, e.g. `/machine/unattached/device[0]`.
+ #[expect(dead_code)]
+ #[serde(rename = "qom-path")]
+ pub qom_path: String,
+
+ /// QOM type of the CPU object, e.g. `host-x86_64-cpu`.
+ #[expect(dead_code)]
+ #[serde(rename = "qom-type")]
+ pub qom_type: String,
+
+ /// Target architecture, e.g. `x86_64`.
+ #[expect(dead_code)]
+ pub target: String,
+
+ /// Guest-visible topology properties of the vCPU.
+ pub props: QmpCpuProps,
+}
+
+/// Guest-visible topology properties of a vCPU.
+#[derive(Debug, Clone, Deserialize)]
+pub struct QmpCpuProps {
+ /// Guest socket the vCPU belongs to.
+ #[expect(dead_code)]
+ #[serde(rename = "socket-id")]
+ pub socket_id: u32,
+
+ /// Guest core within that socket.
+ #[expect(dead_code)]
+ #[serde(rename = "core-id")]
+ pub core_id: u32,
+
+ /// Guest NUMA node the vCPU belongs to.
+ ///
+ /// QEMU reports this even when `numa` is disabled in the VM config,
+ /// in which case every vCPU is on node 0.
+ #[serde(rename = "node-id")]
+ pub node_id: u32,
+
+ /// Guest thread within the core.
+ ///
+ /// Always 0 today: QEMU's CPU object model carries this field for
+ /// SMT support in the guest, but PVE does not expose SMT to guests,
+ /// so it is always zero in practice. Modeled for completeness.
+ /// TODO: PVE should expose SMT to the guests
+ #[expect(dead_code)]
+ #[serde(rename = "thread-id")]
+ pub thread_id: u32,
+}
+
+// returned vec is sorted by vCPU id (0,1,2,3...)
+// Guarantees vec[id] == vec[id].cpu_index
+pub fn parse_cpuinfo(cpuinfo_json: &str) -> Result<Vec<QmpCpu>, BindError> {
+ let mut cpus: Vec<QmpCpu> = serde_json::from_str(cpuinfo_json)
+ .map_err(|e| BindError::BadSyntax(format!("invalid query-cpus-fast JSON: {e}")))?;
+
+ // QEMU assigns cpu-index sequentially from 0. Sort by it so
+ // `vec[i]` indexes vCPU i, then verify the values really are
+ // 0..N-1 so that assumption holds.
+ cpus.sort_unstable_by_key(|c| c.cpu_index);
+
+ for (expected, cpu) in cpus.iter().enumerate() {
+ if cpu.cpu_index != expected as u32 {
+ return Err(BindError::BadSyntax(format!(
+ "query-cpus-fast returned non-contiguous cpu-index: \
+ expected {expected}, got {}",
+ cpu.cpu_index
+ )));
+ }
+ }
+
+ Ok(cpus)
+}
--
2.47.3
^ permalink raw reply related [flat|nested] 4+ messages in thread* [PATCH qemu-server v3 3/3] cpu: add CPU binding
2026-10-22 9:22 [PATCH proxmox-perl-rs/qemu-server/qemu-server-rs v3 0/3] vCPU binding for qemu guests Elias Huhsovitz
2026-10-22 9:22 ` [PATCH qemu-server-rs v3 1/3] cpu: add pve-qemu-server-cpu crate for vCPU binding Elias Huhsovitz
2026-10-22 9:22 ` [PATCH proxmox-perl-rs v3 2/3] pve: cpu: add PVE::RS::CPU::Bind perl bindings Elias Huhsovitz
@ 2026-10-22 9:22 ` Elias Huhsovitz
2 siblings, 0 replies; 4+ messages in thread
From: Elias Huhsovitz @ 2026-10-22 9:22 UTC (permalink / raw)
To: pve-devel; +Cc: Elias Huhsovitz
Add two new VM config options:
* cpu-binding-mode none | manual | one-to-one | numa | numa-manual
* cpu-binding-cfg mode-dependent syntax
When cpu-binding-mode is unset or "none", behavior is unchanged.
Otherwise, vCPU threads are pinned to host CPUs after QEMU has spawned
them, driven by the query-cpus-fast QMP response.
Five modes are supported:
* manual - explicit vCPU:hostCPU mapping, e.g. "0:0;1:1;2:2;3:3"
* one-to-one - each vCPU gets its own physical core from a user pool,
SMT stripped, e.g. "0-3"
* numa - per-vNUMA-node set-to-set binding, sequential, e.g. "3;4"
* numa-manual - explicit vNUMA->host NUMA mapping, e.g. "0:2;1:3"
Two entry points are provided in PVE::QemuServer::CPUBinding:
* validate_binding() - called at VM start, before QEMU spawns. Rejects
invalid configs against the current topology.
* bind_vcpus() - called after QEMU is up. Encodes the QMP cpu
information as JSON and hands it to Rust together with the mode and cfg
string.
Signed-off-by: Elias Huhsovitz <e.huhsovitz@proxmox.com>
---
src/PVE/QemuServer.pm | 36 +++++++++
src/PVE/QemuServer/CPUBinding.pm | 133 +++++++++++++++++++++++++++++++
src/PVE/QemuServer/Makefile | 1 +
3 files changed, 170 insertions(+)
create mode 100644 src/PVE/QemuServer/CPUBinding.pm
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 1efeeb66..3ba266be 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -60,6 +60,7 @@ use PVE::QemuServer::Helpers
qw(config_aware_timeout get_iscsi_initiator_name get_host_arch min_version kvm_user_version windows_version);
use PVE::QemuServer::Cloudinit;
use PVE::QemuServer::CGroup;
+use PVE::QemuServer::CPUBinding;
use PVE::QemuServer::CPUConfig qw(
print_cpu_device
get_cpu_options
@@ -739,6 +740,18 @@ EODESCR
optional => 1,
default => 1,
},
+ 'cpu-binding-mode' => {
+ type => 'string',
+ format => $PVE::QemuServer::Bind::bind_mode_fmt,
+ description => "Set CPU binding mode for the guest",
+ optional => 1,
+ },
+ 'cpu-binding-cfg' => {
+ type => 'string',
+ format => $PVE::QemuServer::Bind::bind_cfg_fmt,
+ description => "Configuration for CPU binding for the guest",
+ optional => 1,
+ },
};
my $cicustom_fmt = {
@@ -3218,6 +3231,15 @@ sub config_to_command {
push @$cmd, '/usr/bin/taskset', '--cpu-list', '--all-tasks', $conf->{affinity};
}
+ if ($conf->{'cpu-binding-mode'}) {
+ PVE::QemuServer::CPUBinding::validate_binding(
+ $conf->{'cpu-binding-mode'},
+ $conf->{'sockets'},
+ $conf->{'cores'},
+ $conf->{'cpu-binding-cfg'},
+ );
+ }
+
push @$cmd, $kvm_binary;
push @$cmd, '-id', $vmid;
@@ -5715,6 +5737,20 @@ sub vm_start_nolock {
my $qemu_pkg_string = PVE::QemuServer::QMPHelpers::get_qemu_package_string($vmid);
syslog("info", "VM $vmid started with PID $pid using $qemu_pkg_string.");
+ if ($conf->{'cpu-binding-mode'}) {
+ eval {
+ PVE::QemuServer::CPUBinding::bind_vcpus(
+ $vmid,
+ $conf->{'cpu-binding-mode'},
+ $conf->{'cpu-binding-cfg'},
+ );
+ 1;
+ } or do {
+ my $error = $@ || 'Unknown failure';
+ log_warn("unable to bind vCPU threads - $error");
+ };
+ }
+
PVE::QemuServer::RunState::create_cleanup_flag($vmid);
if (defined(my $migrate = $res->{migrate})) {
diff --git a/src/PVE/QemuServer/CPUBinding.pm b/src/PVE/QemuServer/CPUBinding.pm
new file mode 100644
index 00000000..3ce3653b
--- /dev/null
+++ b/src/PVE/QemuServer/CPUBinding.pm
@@ -0,0 +1,133 @@
+package PVE::QemuServer::CPUBinding;
+
+use v5.36;
+
+use JSON;
+
+use PVE::QemuServer::Monitor;
+use PVE::RS::CPU::Bind;
+
+our $bind_mode_fmt = {
+ mode => {
+ type => 'string',
+ enum => [qw(none manual one-to-one numa numa-manual)],
+ description => "Set the CPU binding mode for the guest.",
+ verbose_description => <<'EODESCR',
+Selects how guest vCPUs are pinned to host CPUs. The mode determines
+how cpu-binding-cfg is interpreted.
+
+* none (default): no binding.
+
+* manual: explicit vCPU-to-hostCPU mapping. The config string is a
+ semicolon-separated list of `<vcpu-set>:<hostcpu-set>` entries.
+ Example: `0:0;1:1;2:2;3:3` pins vCPU 0 to CPU 0, vCPU 1 to CPU 1,
+ etc. Multi-value sets use `a-b` ranges or `{a,b,c}` unions.
+
+* one-to-one: pin each vCPU to its own physical core, chosen from a
+ user-supplied pool. The config string is an upper bound: the list of
+ host cores the vCPUs may be assigned to. SMT siblings are stripped
+ from that pool before assignment, so each vCPU gets its own distinct
+ physical core, not a hyperthread.
+
+ If the pool has N cores and the VM has M vCPUs, each vCPU is pinned
+ to exactly one of the N cores (N must be >= M, otherwise validation
+ fails). For example, with `0-3` on a 4-vCPU VM, vCPU 0 gets core 0,
+ vCPU 1 gets core 1, and so on. Each vCPU is fixed to a single core,
+ it does not float within the pool. With `0;1;4;5` on the same VM,
+ vCPU 2 is pinned to core 4 and vCPU 3 to core 5, skipping cores 2-3.
+
+ An empty string means "use all physical cores on the host" as the
+ upper bound. On a host with 16 cores, that pins an N-vCPU VM to
+ cores 0, 1, ..., N-1 — one core per vCPU, SMT stripped.
+
+ Unlike `manual`, you do not specify which core each vCPU gets; that
+ assignment is up to hwloc. Use `manual` if you need a specific
+ vCPU-to-core mapping.
+
+* numa: bind the set of vCPUs of each guest vNUMA node to the set of
+ CPUs of a host NUMA node. The config string is a `;`-separated list
+ of host NUMA node indices, consumed in order: guest vNUMA 0 gets the
+ first host node, guest vNUMA 1 the second, and so on.
+
+ Each guest vNUMA node's vCPUs form a group that may run on any CPU
+ of the assigned host NUMA node — including its cores and SMT threads.
+ The host scheduler is free to migrate vCPUs within the node, but not
+ outside it. This is the intended trade-off: memory allocated by a
+ vCPU stays local to the host node it runs on (preserving NUMA
+ memory locality), while the scheduler still has room to balance
+ load across the node's cores and hyperthreads.
+
+ This is a set-to-set binding, not a per-vCPU assignment. All vCPUs
+ of a guest vNUMA node share the same host CPU pool. If you need
+ each vCPU pinned to its own distinct core, use one-to-one or manual
+ instead.
+
+ Example: `3;4` lets all vCPUs on guest vNUMA 0 run on any CPU of
+ host NUMA node 3, and all vCPUs on guest vNUMA 1 run on any CPU of
+ host NUMA node 4.
+
+ An empty string uses host NUMA nodes in ascending index order:
+ guest vNUMA 0 → host node 0, guest vNUMA 1 → host node 1, etc.
+
+* numa-manual: explicit variant of numa. The config string is a
+ `;`-separated list of `<guest-vnuma>:<host-numa>` pairs. Each guest
+ vNUMA node's set of vCPUs is bound to the set of CPUs of the
+ specified host NUMA node.
+
+ Semantics are identical to numa — vCPUs of a guest vNUMA node may
+ be scheduled on any CPU of the assigned host node, but cannot leave
+ it. Only the node mapping is explicit here.
+
+ Example: `0:2;1:3` binds all vCPUs on guest vNUMA 0 to host NUMA
+ node 2's CPUs, and all vCPUs on guest vNUMA 1 to host NUMA node 3's
+ CPUs.
+EODESCR
+ default => 'none',
+ default_key => 1,
+ },
+};
+
+our $bind_cfg_fmt = {
+ cfg => {
+ type => 'string',
+ description => "CPU binding configuration (syntax depends on cpu-binding-mode).",
+ verbose_description => <<'EODESCR',
+The exact syntax depends on cpu-binding-mode:
+
+* manual, one-to-one:
+ `;`-separated list of `<vcpu-set>:<hostcpu-set>` entries (manual)
+ or a plain host CPU list (one-to-one). CPU sets accept `a-b` ranges
+ and `{a,b,c}` unions.
+
+* numa:
+ `;`-separated list of host NUMA node indices, e.g. `3;4`.
+
+* numa-manual:
+ `;`-separated list of `<guest-vnuma>:<host-numa>` pairs, e.g. `0:2;1:3`.
+
+Host CPU and NUMA node indices are OS indices as reported by `lstopo -p`.
+EODESCR
+ default => '',
+ default_key => 1,
+ },
+};
+
+sub validate_binding($mode, $sockets, $cores, $binding_cfg) {
+ return if $mode eq 'none';
+
+ $binding_cfg //= '';
+
+ PVE::RS::CPU::Bind::validate_binding($mode, $sockets, $cores, $binding_cfg);
+}
+
+sub bind_vcpus($vmid, $mode, $binding_cfg) {
+ return if $mode eq 'none';
+
+ $binding_cfg //= '';
+
+ my $cpuinfo = PVE::QemuServer::Monitor::mon_cmd($vmid, 'query-cpus-fast');
+ my $cpuinfo_json = JSON::encode_json($cpuinfo);
+ PVE::RS::CPU::Bind::bind_vcpus($mode, $binding_cfg, $cpuinfo_json);
+}
+
+1;
diff --git a/src/PVE/QemuServer/Makefile b/src/PVE/QemuServer/Makefile
index 61ed1a4e..ee123cf5 100644
--- a/src/PVE/QemuServer/Makefile
+++ b/src/PVE/QemuServer/Makefile
@@ -8,6 +8,7 @@ SOURCES=Agent.pm \
Cfg2Cmd.pm \
CGroup.pm \
Cloudinit.pm \
+ CPUBinding.pm \
CPUConfig.pm \
CPUFlags.pm \
DBusVMState.pm \
--
2.47.3
^ permalink raw reply related [flat|nested] 4+ messages in thread