From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 439E11FF0DF for ; Thu, 19 Nov 2026 10:17:09 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 4F2E82169D; Thu, 19 Nov 2026 10:17:08 +0100 (CET) Content-Type: text/plain; charset=UTF-8 Date: Thu, 19 Nov 2026 10:16:45 +0100 Message-Id: Subject: Re: [PATCH qemu-server-rs v4 1/4] cpu: add pve-qemu-server-cpu crate for vCPU binding From: "Elias Huhsovitz" To: "Dominik Csapak" , Mime-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Mailer: aerc 0.20.0 References: <20261111152027.260117-1-e.huhsovitz@proxmox.com> <20261111152027.260117-2-e.huhsovitz@proxmox.com> <88f12126-6022-47cb-b1b6-ab1895736cff@proxmox.com> In-Reply-To: <88f12126-6022-47cb-b1b6-ab1895736cff@proxmox.com> X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1795079806063 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.409 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) KAM_SHORT 0.001 Use of a URL Shortener for very short URL RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: SQITWPLWXCK3QG67K7CRVL2QLTB3UH7E X-Message-ID-Hash: SQITWPLWXCK3QG67K7CRVL2QLTB3UH7E X-MailFrom: e.huhsovitz@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: On Tue Nov 17, 2026 at 5:28 PM CET, Dominik Csapak wrote: > high level: i forgot it the last time, but how does this interact with > cpu hotplugging? it's totally legit to forbid cpu hotplugging > with this feature enabled for now, but i guess we'd need to > pin again after each hotplug step (and possibly take > into account that not all vcpus are present) > My current idea is indeed to simply re-try pinning after hotplug. Another idea was to restrict the hotplugging when binding is active, e.g. allow a maximum of `maxcpus`. Assert the pinning constrains using `maxcpus`, and disallow plugging > `maxcpus` so that we can guarantee a successful pin for a new vcpu. (The vm would have to start with=20 vcpus > maxcpus) But this seems a bit too rigid. > otherwise comments inline > > On 11/11/26 4:22 PM, Elias Huhsovitz wrote: >> Add a new library crate that implements vCPU to host CPU binding of qemu >> guests. >>=20 >> 2 main functions are exposed: gen_vcpu_bindings: (1) Reads the CPU >> Toplogy from sysfs, validates the configured cpu binding config and >> Generates a vCPU:hostCPU binding map. This function should be called >> before the guest starts. (2) bind_vcpus: Applies the binding map >> generated by gen_vcpu_bindings to the specified vm. The cpu binding is >> applied using sched_setaffinity(2). >>=20 >> 2 modes are implemented: (1) one-to-one: pin each vCPU to its own host >> cpu. (2) numa: bind the set of vCPUs of each vNUMA node to the set of >> CPUs of a host NUMA node. The cpu bindings are determined by the numa[N] >> configuration of the guest. >>=20 >> Both settings respect the configured affinity. >>=20 >> Thread binding uses sched_setaffinity(2), which accepts a kernel >> TID directly. The TID used is reported by qemu via query-cpus-fast. >>=20 >> Originally-by: Dominik Csapak >> Signed-off-by: Elias Huhsovitz >> --- >> Cargo.toml | 2 +- >> build.sh | 2 +- >> pve-qemu-server-cpu/Cargo.toml | 16 ++ >> pve-qemu-server-cpu/debian/changelog | 5 + >> pve-qemu-server-cpu/debian/control | 41 ++++ >> pve-qemu-server-cpu/debian/copyright | 18 ++ >> pve-qemu-server-cpu/debian/debcargo.toml | 9 + >> pve-qemu-server-cpu/src/bind.rs | 136 +++++++++++++ >> pve-qemu-server-cpu/src/config.rs | 146 ++++++++++++++ >> pve-qemu-server-cpu/src/cpuset.rs | 125 ++++++++++++ >> pve-qemu-server-cpu/src/error.rs | 32 ++++ >> pve-qemu-server-cpu/src/lib.rs | 92 +++++++++ >> pve-qemu-server-cpu/src/qmp.rs | 91 +++++++++ >> pve-qemu-server-cpu/src/syscall.rs | 24 +++ >> pve-qemu-server-cpu/src/topology/mod.rs | 222 ++++++++++++++++++++++ >> pve-qemu-server-cpu/src/topology/sysfs.rs | 6 + >> 16 files changed, 965 insertions(+), 2 deletions(-) >> create mode 100644 pve-qemu-server-cpu/Cargo.toml >> create mode 100644 pve-qemu-server-cpu/debian/changelog >> create mode 100644 pve-qemu-server-cpu/debian/control >> create mode 100644 pve-qemu-server-cpu/debian/copyright >> create mode 100644 pve-qemu-server-cpu/debian/debcargo.toml >> create mode 100644 pve-qemu-server-cpu/src/bind.rs >> create mode 100644 pve-qemu-server-cpu/src/config.rs >> create mode 100644 pve-qemu-server-cpu/src/cpuset.rs >> create mode 100644 pve-qemu-server-cpu/src/error.rs >> create mode 100644 pve-qemu-server-cpu/src/lib.rs >> create mode 100644 pve-qemu-server-cpu/src/qmp.rs >> create mode 100644 pve-qemu-server-cpu/src/syscall.rs >> create mode 100644 pve-qemu-server-cpu/src/topology/mod.rs >> create mode 100644 pve-qemu-server-cpu/src/topology/sysfs.rs >>=20 >> diff --git a/Cargo.toml b/Cargo.toml >> index 5eefc79..80f87ff 100644 >> --- a/Cargo.toml >> +++ b/Cargo.toml >> @@ -1,5 +1,6 @@ >> [workspace] >> members =3D [ >> + "pve-qemu-server-cpu", >> "pve-qemu-server-pci" >> ] >> exclude =3D [ >> @@ -25,4 +26,3 @@ strum =3D { version =3D "0.26", features =3D ["derive"= ] } >> =20 >> # proxmox dependencies >> pve-api-types =3D "8.1" >> - >> diff --git a/build.sh b/build.sh >> index 3c2293b..c3ee7b6 100755 >> --- a/build.sh >> +++ b/build.sh >> @@ -28,7 +28,7 @@ debcargo package \ >> --no-overlay-write-back \ >> --directory "$PWD/"${BUILDDIR}"/${CRATE}" \ >> "${CRATE}" \ >> - "$(dpkg-parsechangelog -l "${CRATE}/debian/changelog" -SVersion | s= ed -e 's/-.*//')" >> + "$(dpkg-parsechangelog -l "${CRATE}/debian/changelog" -SVersion | s= ed -e 's/-.*//' -e 's/~/-/')" >> =20 >> cd "${BUILDDIR}/${CRATE}" >> rm -f debian/source/format.debcargo.hint >> diff --git a/pve-qemu-server-cpu/Cargo.toml b/pve-qemu-server-cpu/Cargo.= toml >> new file mode 100644 >> index 0000000..b236796 >> --- /dev/null >> +++ b/pve-qemu-server-cpu/Cargo.toml >> @@ -0,0 +1,16 @@ >> +[package] >> +name =3D "pve-qemu-server-cpu" >> +version =3D "0.1.0" >> +authors.workspace =3D true >> +edition.workspace =3D true >> +license.workspace =3D true >> +repository.workspace =3D true >> +homepage.workspace =3D true >> +exclude.workspace =3D true >> +rust-version.workspace =3D true >> + >> +[dependencies] >> +serde =3D { version =3D "1", features =3D ["derive"] } >> +serde_json =3D "1" >> +thiserror =3D "2" >> +libc =3D "0.2" >> diff --git a/pve-qemu-server-cpu/debian/changelog b/pve-qemu-server-cpu/= debian/changelog >> new file mode 100644 >> index 0000000..3731cfd >> --- /dev/null >> +++ b/pve-qemu-server-cpu/debian/changelog >> @@ -0,0 +1,5 @@ >> +rust-pve-qemu-server-cpu (0.1.0) trixie; urgency=3Dmedium >> + >> + * initial release >> + >> + -- Proxmox Support Team Tue, 01 Sep 2026 14:02:= 25 +0200 >> diff --git a/pve-qemu-server-cpu/debian/control b/pve-qemu-server-cpu/de= bian/control >> new file mode 100644 >> index 0000000..1298d61 >> --- /dev/null >> +++ b/pve-qemu-server-cpu/debian/control >> @@ -0,0 +1,41 @@ >> +Source: rust-pve-qemu-server-cpu >> +Section: rust >> +Priority: optional >> +Build-Depends: debhelper-compat (=3D 13), >> + dh-sequence-cargo, >> + libhwloc-dev (>=3D 2.12) >> +Build-Depends-Arch: cargo:native , >> + rustc:native (>=3D 1.94) , >> + libstd-rust-dev , >> + librust-libc-0.2+default-dev , >> + librust-serde-1+default-dev , >> + librust-serde-1+derive-dev , >> + librust-serde-json-1+default-dev , >> + librust-thiserror-2+default-dev >> +Maintainer: Proxmox Support Team >> +Standards-Version: 4.7.2 >> +Vcs-Git: git://git.proxmox.com/git/pve-qemu-server-rs.git >> +Vcs-Browser: https://git.proxmox.com/?p=3Dpve-qemu-server-rs.git >> +Homepage: https://proxmox.com >> +X-Cargo-Crate: pve-qemu-server-cpu >> + >> +Package: librust-pve-qemu-server-cpu-dev >> +Architecture: any >> +Multi-Arch: same >> +Depends: >> + ${misc:Depends}, >> + librust-libc-0.2+default-dev, >> + librust-serde-1+default-dev, >> + librust-serde-1+derive-dev, >> + librust-serde-json-1+default-dev, >> + librust-thiserror-2+default-dev >> +Provides: >> + librust-pve-qemu-server-cpu+default-dev (=3D ${binary:Version}), >> + librust-pve-qemu-server-cpu-0-dev (=3D ${binary:Version}), >> + librust-pve-qemu-server-cpu-0+default-dev (=3D ${binary:Version}), >> + librust-pve-qemu-server-cpu-0.1-dev (=3D ${binary:Version}), >> + librust-pve-qemu-server-cpu-0.1+default-dev (=3D ${binary:Version}), >> + librust-pve-qemu-server-cpu-0.1.0-dev (=3D ${binary:Version}), >> + librust-pve-qemu-server-cpu-0.1.0+default-dev (=3D ${binary:Version}) >> +Description: Rust crate "pve-qemu-server-cpu" - Rust source code >> + Source code for Debianized Rust crate "pve-qemu-server-cpu" >> diff --git a/pve-qemu-server-cpu/debian/copyright b/pve-qemu-server-cpu/= debian/copyright >> new file mode 100644 >> index 0000000..01138fa >> --- /dev/null >> +++ b/pve-qemu-server-cpu/debian/copyright >> @@ -0,0 +1,18 @@ >> +Format: https://www.debian.org/doc/packaging-manuals/copyright-format/1= .0/ >> + >> +Files: >> + * >> +Copyright: 2026 Proxmox Server Solutions GmbH >> +License: AGPL-3.0-or-later >> + This program is free software: you can redistribute it and/or modify i= t under >> + the terms of the GNU Affero General Public License as published by the= Free >> + Software Foundation, either version 3 of the License, or (at your opti= on) any >> + later version. >> + . >> + This program is distributed in the hope that it will be useful, but WI= THOUT >> + ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or = FITNESS >> + FOR A PARTICULAR PURPOSE. See the GNU Affero General Public License fo= r more >> + details. >> + . >> + You should have received a copy of the GNU Affero General Public Licen= se along >> + with this program. If not, see . >> diff --git a/pve-qemu-server-cpu/debian/debcargo.toml b/pve-qemu-server-= cpu/debian/debcargo.toml >> new file mode 100644 >> index 0000000..23feea4 >> --- /dev/null >> +++ b/pve-qemu-server-cpu/debian/debcargo.toml >> @@ -0,0 +1,9 @@ >> +overlay =3D "." >> +crate_src_path =3D ".." >> +maintainer =3D "Proxmox Support Team " >> +allow_prerelease_deps =3D true >> + >> +[source] >> +vcs_git =3D "git://git.proxmox.com/git/pve-qemu-server-rs.git" >> +vcs_browser =3D "https://git.proxmox.com/?p=3Dpve-qemu-server-rs.git" >> +build_depends =3D ["libhwloc-dev (>=3D 2.12)"] >> diff --git a/pve-qemu-server-cpu/src/bind.rs b/pve-qemu-server-cpu/src/b= ind.rs >> new file mode 100644 >> index 0000000..d7f52f8 >> --- /dev/null >> +++ b/pve-qemu-server-cpu/src/bind.rs >> @@ -0,0 +1,136 @@ >> +use std::collections::BTreeMap; >> + >> +use crate::cpuset::CpuSet; >> +use crate::config::NumaConfig; >> +use crate::qmp::QmpCpu; >> +use crate::syscall::sched_setaffinity; >> +use crate::topology::Topology; >> +use crate::error::CpuError; >> + >> +#[derive(Debug, Clone, Copy, PartialEq, Eq)] >> +pub enum Mode { >> + None, >> + OneToOne, >> + Numa, >> +} >> + >> +impl std::str::FromStr for Mode { >> + type Err =3D CpuError; >> + >> + fn from_str(s: &str) -> Result { >> + match s { >> + "none" =3D> Ok(Mode::None), >> + "one-to-one" =3D> Ok(Mode::OneToOne), >> + "numa" =3D> Ok(Mode::Numa), >> + other =3D> Err(CpuError::BadSyntax(format!( >> + "unknown cpu-bind-mode '{other}'" >> + ))), >> + } >> + } >> +} >> + >> +pub fn gen_one_to_one_bindings( >> + sockets: u32, >> + cores: u32, >> + affinity: &Option, >> + only_real_cores: bool, >> +) -> Result, CpuError> { >> + let topo =3D Topology::new()?; >> + let mut allowed =3D topo.online_cpus().clone(); >> + >> + if let Some(aff) =3D affinity { >> + allowed.intersect(aff); >> + } >> + >> + let nr_vcpus =3D sockets * cores; >> + >> + if only_real_cores { >> + allowed =3D topo.singlify_per_core(&allowed); >> + } > > nit: this is hardcoded, so no reason to have this code here for now > > we can always add it, should we need it > Ok >> + >> + if allowed.size() < nr_vcpus as usize { >> + return Err(CpuError::BadSyntax(format!( >> + "one-to-one mode requires {nr_vcpus} host cpus but only {} = are available", >> + allowed.size() >> + ))); >> + } >> + >> + let available: Vec =3D allowed.to_vec(); >> + >> + let mut bindings: BTreeMap =3D BTreeMap::new(); >> + >> + // qmp::parse_cpuinfo should yield vCPU IDs that are 0..N-1. >> + // If not: we proably messed up somewhere else >> + for (vcpu_id, cpu) in available.iter().take(nr_vcpus as usize).enum= erate() { >> + let mut single_cpu =3D CpuSet::new(); >> + single_cpu.set(*cpu); >> + bindings.insert(vcpu_id as u32, single_cpu); >> + } >> + >> + Ok(bindings) >> +} >> + >> +pub fn gen_numa_bindings( >> + numa_cfg: &Vec, >> + affinity: &Option, >> +) -> Result, CpuError> { >> + let topo =3D Topology::new()?; >> + let mut bindings =3D BTreeMap::new(); >> + >> + for (vnode_id, cfg) in numa_cfg.iter().enumerate() { >> + if cfg.hostnodes.len() !=3D 1 { >> + return Err(CpuError::BadSyntax(format!( >> + "vNUMA node {vnode_id} must be bound to exactly one hos= t NUMA node" >> + ))); >> + } >> + >> + let host_node_id =3D *cfg.hostnodes.first().unwrap(); >> + let host_cpus =3D topo.numa_cpus(host_node_id).ok_or_else(|| { >> + CpuError::BadSyntax(format!( >> + "host NUMA node {host_node_id} does not exist on this h= ost" >> + )) >> + })?; >> + >> + let mut allowed =3D host_cpus.clone(); >> + >> + if let Some(aff) =3D affinity { >> + allowed.intersect(aff); >> + } >> + >> + if allowed.is_empty() { >> + return Err(CpuError::BadSyntax(format!( >> + "no allowed CPUs available for host NUMA node {host_nod= e_id} \ >> + after applying affinity mask." >> + ))); >> + } >> + >> + for &vcpu_id in &cfg.vcpus { >> + bindings.insert(vcpu_id, allowed.clone()); >> + } >> + } >> + >> + Ok(bindings) >> +} >> + >> +/// Applies the generated CPU bindings to the running vCPU threads. >> +pub fn apply_bindings( >> + bindings: &BTreeMap, >> + qmp_cpuinfo: &[QmpCpu], >> +) -> Result<(), CpuError> { >> + for (&vcpu_id, cpuset) in bindings { >> + let vcpu_index =3D vcpu_id as usize; >> + if vcpu_index >=3D qmp_cpuinfo.len() { >> + return Err(CpuError::VcpuOutOfRange { >> + vcpu: vcpu_id, >> + max: qmp_cpuinfo.len() as u32, >> + }); > > here we probably get an error when not all vcpus > are there (hotplugging) > >> + } >> + >> + let tid =3D qmp_cpuinfo[vcpu_index].thread_id; >> + sched_setaffinity(tid as libc::pid_t, cpuset).map_err(|e| CpuEr= ror::Bind { >> + vcpu: vcpu_id, >> + message: e.to_string(), >> + })?; >> + } >> + Ok(()) >> +} >> diff --git a/pve-qemu-server-cpu/src/config.rs b/pve-qemu-server-cpu/src= /config.rs >> new file mode 100644 >> index 0000000..a89c62c >> --- /dev/null >> +++ b/pve-qemu-server-cpu/src/config.rs >> @@ -0,0 +1,146 @@ >> +//! Parsing of the Proxmox VE CPU and memory configuration. >> + >> +use serde::de::{self, Deserializer, SeqAccess, Visitor}; >> +use serde::Deserialize; >> +use std::collections::{BTreeMap, BTreeSet, HashMap}; >> +use std::fmt; >> + >> +use crate::cpuset::CpuSet; >> +use crate::error::CpuError; >> + >> +#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)] >> +#[serde(rename_all =3D "lowercase")] >> +pub enum NumaPolicy { >> + Bind, >> + Interleave, >> + Preferred, >> +} >> + >> +/// Configuration for a single numa[N] node. >> +#[derive(Debug, Clone, PartialEq, Eq, Deserialize)] >> +pub struct NumaConfig { >> + /// Guest vCPU IDs assigned to this node. >> + #[serde(default, deserialize_with =3D "deserialize_range_set")] >> + pub vcpus: BTreeSet, >> + >> + /// Host NUMA node IDs. >> + #[serde(default, deserialize_with =3D "deserialize_range_set")] >> + pub hostnodes: BTreeSet, >> + >> + /// Memory provided by this node, in MiB. >> + pub memory: u32, >> + >> + /// NUMA allocation policy. >> + pub policy: Option, >> +} >> + >> +/// Parses the NUMA configuration from a JSON string. >> +/// >> +/// Returns a `Vec` where the index corresponds to the NUMA node ID. >> +/// Assumes the Perl layer has already validated that node IDs are >> +/// contiguous from `0` to `N-1`. >> +/// Parses the NUMA configuration from a JSON string. >> +/// >> +/// Returns a `Vec` where the index corresponds to the NUMA node ID. >> +/// Assumes the Perl layer has already validated that node IDs are >> +/// contiguous from `0` to `N-1`. >> +pub fn numa_config_from_json(cfg_json: &str) -> Result,= CpuError> { >> + let raw_map: HashMap =3D serde_json::from_str(c= fg_json) >> + .map_err(|e| CpuError::BadSyntax(format!("invalid NUMA JSON: {e= }")))?; >> + >> + let len =3D raw_map.len(); >> + >> + let mut configs: Vec> =3D vec![None; len]; >> + >> + for (key, config) in raw_map { >> + let id_str =3D key.strip_prefix("numa").unwrap_or(&key); >> + let id =3D id_str.parse::().map_err(|e| { >> + CpuError::BadSyntax(format!("invalid NUMA node ID '{key}': = {e}")) >> + })?; >> + >> + configs[id] =3D Some(config); >> + } >> + >> + // Unwrap the options. If Perl did its job, every slot will be fill= ed. >> + // We return an error instead of panicking just in case the invaria= nt is broken. >> + configs >> + .into_iter() >> + .enumerate() >> + .map(|(i, opt)| { >> + opt.ok_or_else(|| CpuError::BadSyntax(format!("missing NUMA= node {i}"))) >> + }) >> + .collect() >> +} >> + >> +/// Expands a JSON array of ranges into a `BTreeSet`. >> +/// >> +/// Perl outputs ranges as `[[0, 3], [8, null]]`. This visitor expands = them >> +/// into a flat set of integers: `{0, 1, 2, 3, 8}`. >> +fn deserialize_range_set<'de, D>(deserializer: D) -> Result, D::Error> >> +where >> + D: Deserializer<'de>, >> +{ >> + struct RangeSetVisitor; >> + >> + impl<'de> Visitor<'de> for RangeSetVisitor { >> + type Value =3D BTreeSet; >> + >> + fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Res= ult { >> + formatter.write_str("a JSON array of ranges (e.g., [[0, 3],= [8, null]])") >> + } >> + >> + fn visit_seq(self, mut seq: A) -> Result >> + where >> + A: SeqAccess<'de>, >> + { >> + let mut set =3D BTreeSet::new(); >> + >> + while let Some((start, end_opt)) =3D seq.next_element::<(u3= 2, Option)>()? { >> + let end =3D end_opt.unwrap_or(start); >> + if end < start { >> + return Err(de::Error::custom(format!( >> + "invalid range: end ({end}) < start ({start})" >> + ))); >> + } >> + set.extend(start..=3Dend); >> + } >> + >> + Ok(set) >> + } >> + >> + fn visit_unit(self) -> Result { >> + Ok(BTreeSet::new()) >> + } >> + >> + fn visit_none(self) -> Result { >> + Ok(BTreeSet::new()) >> + } >> + } >> + >> + deserializer.deserialize_any(RangeSetVisitor) >> +} >> + >> +pub fn bindings_from_json(bind_map_json: &str) -> Result, CpuError> { >> + let raw_map: BTreeMap> =3D serde_json::from_str(bind_= map_json) >> + .map_err(|e| CpuError::BadSyntax(format!("invalid binding map J= SON: {e}")))?; >> + >> + let mut bindings =3D BTreeMap::new(); >> + for (vcpu, cpus) in raw_map { >> + let mut set =3D CpuSet::new(); >> + for cpu in cpus { >> + set.set(cpu); >> + } >> + bindings.insert(vcpu, set); >> + } >> + Ok(bindings) >> +} >> + >> +pub fn bindings_to_json(bindings: &BTreeMap) -> Result { >> + let map: BTreeMap> =3D bindings >> + .iter() >> + .map(|(&vcpu, cpuset)| (vcpu, cpuset.iter().collect())) >> + .collect(); >> + >> + serde_json::to_string(&map) >> + .map_err(|e| CpuError::BadSyntax(format!("failed to serialize b= indings: {e}"))) >> +} >> diff --git a/pve-qemu-server-cpu/src/cpuset.rs b/pve-qemu-server-cpu/src= /cpuset.rs >> new file mode 100644 >> index 0000000..d2ffcf8 >> --- /dev/null >> +++ b/pve-qemu-server-cpu/src/cpuset.rs >> @@ -0,0 +1,125 @@ >> +//! Arbitrary-precision CPU bitmask for efficient set operations. >> + >> +#[derive(Clone, Debug, Default, PartialEq, Eq)] >> +pub struct CpuSet { >> + bits: Vec, >> +} > > this is a very nice optimization here,and a good way to be efficient, > but IMHO a bit overkill for this? > > we only calculate this once per start > (and maybe once per hotplug) so having a e.g. Vec > or even a HashSet could be enought? > > this would remove most of the implementation code here > and keep the usage mostly the same? > > OTOH, you did the work already so :shrug: > This is analog to the approach by the hwloc library and how taskset computes the cpuset internally (AFAIK). So implementing this was straightforward. >> + >> +struct WordBits { >> + base: u32, >> + remaining: u64, >> +} >> + >> +impl Iterator for WordBits { >> + type Item =3D u32; >> + >> + fn next(&mut self) -> Option { >> + if self.remaining =3D=3D 0 { >> + return None; >> + } >> + let bit =3D self.remaining.trailing_zeros(); >> + // Clear the lowest set bit so the next call finds the followin= g one. >> + self.remaining &=3D self.remaining - 1; >> + Some(self.base + bit) >> + } >> +} >> + >> +impl CpuSet { >> + pub fn new() -> Self { >> + Self { bits: Vec::new() } >> + } >> + >> + pub fn set(&mut self, cpu: u32) { >> + let idx =3D (cpu / 64) as usize; >> + let bit =3D cpu % 64; >> + if idx >=3D self.bits.len() { >> + self.bits.resize(idx + 1, 0); >> + } >> + self.bits[idx] |=3D 1u64 << bit; >> + } >> + >> + pub fn intersect(&mut self, other: &CpuSet) { >> + for (i, word) in self.bits.iter_mut().enumerate() { >> + if i < other.bits.len() { >> + *word &=3D other.bits[i]; >> + } else { >> + *word =3D 0; >> + } >> + } >> + } >> + >> + pub fn union(&mut self, other: &CpuSet) { >> + if other.bits.len() > self.bits.len() { >> + self.bits.resize(other.bits.len(), 0); >> + } >> + for (i, word) in other.bits.iter().enumerate() { >> + self.bits[i] |=3D word; >> + } >> + } >> + >> + pub fn is_empty(&self) -> bool { >> + self.bits.iter().all(|&w| w =3D=3D 0) >> + } >> + >> + pub fn size(&self) -> usize { >> + self.bits.iter().map(|w| w.count_ones() as usize).sum() >> + } >> + >> + pub fn first_set(&self) -> Option { >> + self.iter().next() >> + } >> + >> + pub fn iter(&self) -> impl Iterator + '_ { >> + self.bits >> + .iter() >> + .enumerate() >> + .flat_map(|(idx, &word)| WordBits { >> + base: (idx as u32) * 64, >> + remaining: word, >> + }) >> + } >> + >> + /// Returns the set members as an ascending vector. >> + pub fn to_vec(&self) -> Vec { >> + self.iter().collect() >> + } >> + >> + /// Converts the bitmask to a `libc::cpu_set_t` for `sched_setaffin= ity`. >> + pub fn to_cpu_set_t(&self) -> libc::cpu_set_t { >> + unsafe { >> + let mut set: libc::cpu_set_t =3D std::mem::zeroed(); >> + for cpu in self.iter() { >> + libc::CPU_SET(cpu as usize, &mut set); >> + } >> + set >> + } >> + } >> + >> + /// Parses sysfs CPU lists, for example `online`, `cpulist`, `core_= cpus_list`. >> + /// >> + /// Format: comma-separated CPU IDs and inclusive ranges, such as `= 0-15,18-31`. >> + pub fn from_list(list: &str) -> Self { >> + let mut set =3D CpuSet::new(); >> + let list =3D list.trim(); >> + if list.is_empty() { >> + return set; >> + } >> + >> + for part in list.split(',') { >> + let part =3D part.trim(); >> + if part.is_empty() { >> + continue; >> + } >> + if let Some((start_raw, end_raw)) =3D part.split_once('-') = { >> + let start: u32 =3D start_raw.trim().parse().unwrap_or(0= ); >> + let end: u32 =3D end_raw.trim().parse().unwrap_or(start= ); >> + for i in start..=3Dend { >> + set.set(i); >> + } >> + } else if let Ok(cpu) =3D part.parse::() { >> + set.set(cpu); >> + } >> + } >> + set >> + } >> +} >> diff --git a/pve-qemu-server-cpu/src/error.rs b/pve-qemu-server-cpu/src/= error.rs >> new file mode 100644 >> index 0000000..2efbc12 >> --- /dev/null >> +++ b/pve-qemu-server-cpu/src/error.rs >> @@ -0,0 +1,32 @@ >> +use thiserror::Error; >> + >> +#[derive(Debug, Error)] >> +#[error("cpu binding: {0}")] >> +pub enum CpuError { >> + #[error("invalid syntax in: '{0}'")] >> + BadSyntax(String), >> + >> + // Validation errors (pre-start) >> + #[error("host CPU {0} does not exist in this topology")] >> + UnknownHostCpu(u32), >> + >> + #[error("vCPU {vcpu} is out of range (VM has vCPUs 0..{max})")] >> + VcpuOutOfRange { vcpu: u32, max: u32 }, >> + >> + // Runtime errors (post-start) >> + #[error("no running thread found for vCPU {0}")] >> + ThreadNotFound(u32), >> + >> + #[error("invalid kernel TID {tid} for vCPU {vcpu}")] >> + BadThreadId { vcpu: u32, tid: u32 }, >> + >> + #[error("failed to init host topology: '{0}'")] >> + TopologyInit(String), >> + >> + #[error("binding vCPU {vcpu} failed: {message}")] >> + Bind { >> + vcpu: u32, >> + /// Human-readable error from sched_setaffinity(2) >> + message: String, >> + }, >> +} >> diff --git a/pve-qemu-server-cpu/src/lib.rs b/pve-qemu-server-cpu/src/li= b.rs >> new file mode 100644 >> index 0000000..cf19aa8 >> --- /dev/null >> +++ b/pve-qemu-server-cpu/src/lib.rs >> @@ -0,0 +1,92 @@ >> +mod bind; >> +mod cpuset; >> +mod error; >> +mod config; >> +mod qmp; >> +mod syscall; >> +mod topology; >> + >> +/// Generate vcpu:cpu(s) binding map for a guest >> +/// >> +/// # Arguments >> +/// >> +/// * `mode` - Pinning mode. >> +/// * `sockets` - number of sockets configured in >> +/// * `cores` - number of cores configured in >> +/// * `affinity` - Upper bound of host cpus that can be used for bindin= g. >> +/// * `cfg_json` - Additional config options. (e.g. numa config) >> +/// * `cpuinfo_json` - JSON output from QMP `query-cpus-fast`. >> +/// >> +/// # Modes >> +/// >> +/// | Mode | Meaning | >> +/// |---|---| >> +/// | `none` | Do not pin vCPUs. | >> +/// | `one-to-one` | Assign each vCPU to its own cpu. | >> +/// | `numa` | Map guest NUMA nodes to host NUMA nodes | >> +/// >> +/// # Examples >> +/// >> +/// ```ignore >> +/// // One-to-one mode: >> +/// // Distribute vCPUs over host CPUs 0-7. >> +/// gen_vcpu_bindings("one-to-one", 1, 8, "0-7", "", &cpuinfo_json)?; >> +/// >> +/// // One-to-one mode over all allowed host CPUs. >> +/// gen_vcpu_bindings("one-to-one", 1, 4 "", "", &cpuinfo_json)?; >> +/// >> +/// // NUMA mode: >> +/// // NUMA mode over all allowed host NUMA nodes. >> +/// gen_vcpu_bindings("numa", 2, 4, "", &cfg_json, &cpuinfo_json)?; >> +/// >> +/// // NUMA mode respecting the set affinity >> +/// gen_vcpu_bindings("numa", 4, 2, "0-7", &cfg_json, &cpuinfo_json)?; >> +/// ``` >> +/// >> +/// # Errors >> +/// >> +/// Returns [`CpuError`] if the mode or binding syntax is invalid, if t= he >> +/// requested host CPUs or NUMA nodes do not exist, or if binding fails= . >> +pub fn gen_vcpu_bindings( >> + mode: &str, >> + sockets: u32, >> + cores: u32, >> + affinity: &str, >> + cfg_json: &str, >> +) -> Result { > > i'd much prefer 'real' types here at the boundary > e.g. we can just export the `QmpCpu` etc. structs > (they already have to impl deserialize) and > should use that in the perl bindings, or is that not possible for some=20 > reason? If there is one, it would definitely belong in > the commit message or cover letter No specific reason for the json here tbh. Mostly just a habit. I will=20 implement it in a similar fashion to your PCI series! (Assuming the code will remain implemented in rust) >> + let cpu_affinity =3D if affinity.is_empty() { >> + None >> + } else { >> + Some(cpuset::CpuSet::from_list(affinity)) >> + }; >> + >> + let bindings: std::collections::BTreeMap =3D m= atch mode.parse::()? { >> + bind::Mode::None =3D> return Ok("{}".to_string()), > > together with the qemu-server patch 4/4 this now skips the > affinity when cpu-binding-mode is 'none', since the > qemu-server check skips the taskset and here we return nothing. > > we should either return the affinity map for all cores here > or let the taskset run throuh in qemu-server > > (actually we could let the taskset limit the threads anyway, this > code here can only further restrict it AFAICS) > Good catch! I would prefer to just have something like if (cpu-binding-mode =3D=3D 'none') { // init qemu using taskset } But I am thinking of simply removing the 'none' state alltogether, as it seems redundant. >> + bind::Mode::OneToOne =3D> { >> + // only_real_cores is hardcoded to false for now, until we = device if we actually want this setting >> + bind::gen_one_to_one_bindings(sockets, cores, &cpu_affinity= , false)? >> + } >> + bind::Mode::Numa =3D> { >> + let numa_cfg =3D config::numa_config_from_json(cfg_json)?; >> + bind::gen_numa_bindings(&numa_cfg, &cpu_affinity)? >> + } >> + }; >> + >> + // This shouldn't happen, but we should still avoid passing empty c= onfig to perl >> + if bindings.is_empty() { >> + return Err(error::CpuError::BadSyntax( >> + "cpu-binding-mode '{mode}' is enabled but no vCPU bindings = were generated".to_string(), >> + )); >> + } >> + >> + config::bindings_to_json(&bindings) >> +} >> + >> +/// Apply vcpu:cpu(s) the binding map >> +/// The function parses the QMP `query-cpus-fast` response and binds ea= ch >> +/// vCPU thread according to the selected mode. >> +pub fn bind_vcpus(bind_map_json: &str, qmp_cpuinfo_json: &str) -> Resul= t<(), error::CpuError> { >> + let bindings =3D config::bindings_from_json(bind_map_json)?; >> + let qmp_cpuinfo =3D qmp::parse_cpuinfo(qmp_cpuinfo_json)?; >> + bind::apply_bindings(&bindings, &qmp_cpuinfo) >> +} >> diff --git a/pve-qemu-server-cpu/src/qmp.rs b/pve-qemu-server-cpu/src/qm= p.rs >> new file mode 100644 >> index 0000000..78fb8bd >> --- /dev/null >> +++ b/pve-qemu-server-cpu/src/qmp.rs >> @@ -0,0 +1,91 @@ >> +//! Parsing of the QEMU `query-cpus-fast` QMP response. >> +//! >> +//! The response is a JSON array; each element describes one guest vCPU >> +//! and carries both its guest-visible topology (socket/core/NUMA node) >> +//! and its host kernel TID. Every binding mode consumes this shape, so >> +//! the parsing lives here rather than inside any individual mode. >> +//! >> +//! The full QEMU schema is modeled, including fields not currently use= d >> +//! by any binding mode. Serde ignores fields it doesn't know about, bu= t >> +//! keeping them explicit documents the wire format and makes future >> +//! modes cheaper to implement. >> + >> +use serde::Deserialize; >> + >> +use crate::error::CpuError; >> + >> +/// One entry of the `query-cpus-fast` array. >> +#[derive(Debug, Clone, Deserialize)] >> +pub struct QmpCpu { >> + /// Guest-visible vCPU index (0 .. sockets*cores). >> + #[serde(rename =3D "cpu-index")] >> + pub cpu_index: u32, >> + >> + /// Host kernel TID of the QEMU vCPU thread. >> + #[serde(rename =3D "thread-id")] >> + pub thread_id: u32, >> + >> + /// QOM path of the CPU object, e.g. `/machine/unattached/device[0]= `. >> + #[expect(dead_code)] >> + #[serde(rename =3D "qom-path")] >> + pub qom_path: String, >> + >> + /// QOM type of the CPU object, e.g. `host-x86_64-cpu`. >> + #[expect(dead_code)] >> + #[serde(rename =3D "qom-type")] >> + pub qom_type: String, >> + >> + /// Target architecture, e.g. `x86_64`. >> + #[expect(dead_code)] >> + pub target: String, >> + >> + /// Guest-visible topology properties of the vCPU. >> + #[expect(dead_code)] >> + pub props: QmpCpuProps, >> +} >> + >> +/// Guest-visible topology properties of a vCPU. >> +#[derive(Debug, Clone, Deserialize)] >> +pub struct QmpCpuProps { >> + /// Guest socket the vCPU belongs to. >> + #[expect(dead_code)] >> + #[serde(rename =3D "socket-id")] >> + pub socket_id: u32, >> + >> + /// Guest core within that socket. >> + #[expect(dead_code)] >> + #[serde(rename =3D "core-id")] >> + pub core_id: u32, >> + >> + /// Guest NUMA node the vCPU belongs to. >> + /// >> + /// QEMU reports this even when `numa` is disabled in the VM config= , >> + /// in which case every vCPU is on node 0. >> + #[expect(dead_code)] >> + #[serde(rename =3D "node-id")] >> + pub node_id: u32, >> + >> + /// Guest thread within the core. >> + /// >> + /// Always 0 today: QEMU's CPU object model carries this field for >> + /// SMT support in the guest, but PVE does not expose SMT to guests= , >> + /// so it is always zero in practice. Modeled for completeness. >> + /// TODO: PVE should expose SMT to the guests >> + #[expect(dead_code)] >> + #[serde(rename =3D "thread-id")] >> + pub thread_id: u32, >> +} > > not sure about it, but are these fields always present? if not > the deserialization will fail. marking them with #[serde(default)] or > making them options would be a solution for that > Noted. As far as I can tell these fields are always present. >> + >> +// returned vec is sorted by vCPU id (0,1,2,3...) >> +// Guarantees vec[id] =3D=3D vec[id].cpu_index >> +pub fn parse_cpuinfo(cpuinfo_json: &str) -> Result, CpuErro= r> { >> + let mut cpus: Vec =3D serde_json::from_str(cpuinfo_json) >> + .map_err(|e| CpuError::BadSyntax(format!("invalid query-cpus-fa= st JSON: {e}")))?; >> + >> + // QEMU assigns cpu-index sequentially from 0. Sort by it so >> + // `vec[i]` indexes vCPU i, >> + // value should be 0..N-1 so that assumption holds. >> + cpus.sort_unstable_by_key(|c| c.cpu_index); >> + >> + Ok(cpus) >> +} >> diff --git a/pve-qemu-server-cpu/src/syscall.rs b/pve-qemu-server-cpu/sr= c/syscall.rs >> new file mode 100644 >> index 0000000..59b74a9 >> --- /dev/null >> +++ b/pve-qemu-server-cpu/src/syscall.rs >> @@ -0,0 +1,24 @@ >> +//! Safe wrappers around Linux syscalls. >> + >> +#[cfg(target_os =3D "linux")] >> +use std::io; >> + >> +use crate::cpuset::CpuSet; >> + >> +pub fn sched_setaffinity(tid: libc::pid_t, cpus: &CpuSet) -> io::Result= <()> { >> + let set =3D cpus.to_cpu_set_t(); >> + >> + unsafe { >> + let rc =3D libc::sched_setaffinity( >> + tid, >> + std::mem::size_of::(), >> + &set, >> + ); >> + >> + if rc !=3D 0 { >> + return Err(io::Error::last_os_error()); >> + } >> + } >> + >> + Ok(()) >> +} >> diff --git a/pve-qemu-server-cpu/src/topology/mod.rs b/pve-qemu-server-c= pu/src/topology/mod.rs >> new file mode 100644 >> index 0000000..3d7ffaa >> --- /dev/null >> +++ b/pve-qemu-server-cpu/src/topology/mod.rs >> @@ -0,0 +1,222 @@ >> +//! Hierarchical hardware topology discovered from sysfs. >> +//! >> +//! The topology is a tree rooted at the machine. Every node owns its >> +//! children and there are no parent pointers, so traversal is top-down= . >> +//! >> +//! On modern systems (like AMD EPYC), the hierarchy typically looks li= ke: >> +//! >> +//! ```text >> +//! Machine -> Package -> Die -> NumaNode -> Core -> Pu >> +//! ``` >> + >> +mod sysfs; >> + >> +use std::collections::HashMap; >> +use std::fs; >> +use std::path::Path; >> + >> +use crate::cpuset::CpuSet; >> +use crate::error::CpuError; >> + >> +const CPU_DIR: &str =3D "/sys/devices/system/cpu"; >> +const NODE_DIR: &str =3D "/sys/devices/system/node"; >> + >> +#[derive(Debug, Clone, Copy, PartialEq, Eq)] >> +pub enum ObjectKind { >> + Machine, >> + Package, >> + Die, >> + NumaNode, >> + // Cache, // Future extension >> + Core, >> + Pu, >> +} > > probably nice to have this (in the future), but we probably should only= =20 > model stuff we can represent for now? we can't e.g. model multiple dies > of a cpu in our config, so this is kinda wasted? > Hmm ok, this design is heavily inspired by hwloc, since IMO there might be some neat future use cases (automatic placement/rebalancing, or different pinning options entirely). But I will omitt it for now, to keep it simple. >> + >> +/// A node in the hardware topology tree. >> +/// >> +/// Each node owns its children. The `cpuset` covers every CPU at or be= low >> +/// this node. `os_index` is the operating-system index (hwloc's `P#`). >> +#[derive(Debug, Clone)] >> +pub struct TopoObject { >> + pub kind: ObjectKind, >> + pub os_index: u32, >> + pub cpuset: CpuSet, >> + pub children: Vec, >> +} >> + >> +impl TopoObject { >> + fn new(kind: ObjectKind, os_index: u32) -> Self { >> + Self { >> + kind, >> + os_index, >> + cpuset: CpuSet::new(), >> + children: Vec::new(), >> + } >> + } >> + >> + /// Finds an existing child by kind and index, or creates and appen= ds a new one. >> + fn get_or_add_child(&mut self, kind: ObjectKind, os_index: u32) -> = &mut TopoObject { >> + let pos =3D self >> + .children >> + .iter() >> + .position(|c| c.kind =3D=3D kind && c.os_index =3D=3D os_in= dex) >> + .unwrap_or_else(|| { >> + self.children.push(TopoObject::new(kind, os_index)); >> + self.children.len() - 1 >> + }); >> + >> + &mut self.children[pos] >> + } >> + >> + /// Finds the first node matching `kind` and `os_index` in this sub= tree. >> + pub fn find(&self, kind: ObjectKind, os_index: u32) -> Option<&Topo= Object> { >> + if self.kind =3D=3D kind && self.os_index =3D=3D os_index { >> + return Some(self); >> + } >> + for child in &self.children { >> + if let Some(found) =3D child.find(kind, os_index) { >> + return Some(found); >> + } >> + } >> + None >> + } >> + >> + /// Calls `f` for every node matching `kind` in this subtree. >> + pub fn for_each(&self, kind: ObjectKind, f: &mut dyn FnMut(&TopoObj= ect)) { >> + if self.kind =3D=3D kind { >> + f(self); >> + } >> + for child in &self.children { >> + child.for_each(kind, &mut *f); >> + } >> + } >> + >> + /// Computes this node's cpuset bottom-up from its children. >> + fn compute_cpuset(&mut self) { >> + self.cpuset =3D CpuSet::new(); >> + if self.kind =3D=3D ObjectKind::Pu { >> + self.cpuset.set(self.os_index); >> + } else { >> + for child in &mut self.children { >> + child.compute_cpuset(); >> + self.cpuset.union(&child.cpuset); >> + } >> + } >> + } >> +} >> + >> +#[derive(Debug, Clone)] >> +pub struct Topology { >> + online_cpus: CpuSet, >> + root: TopoObject, >> +} >> + >> +impl Topology { >> + pub fn new() -> Result { >> + let online_cpus =3D sysfs::read_file(&Path::new(CPU_DIR).join("= online")) >> + .map(|s| CpuSet::from_list(&s)) >> + .ok_or_else(|| { >> + CpuError::TopologyInit("cannot read the online CPU mask= ".to_string()) >> + })?; >> + >> + let cpu_to_numa =3D Self::read_cpu_to_numa(); >> + >> + let mut root =3D TopoObject::new(ObjectKind::Machine, 0); >> + Self::build_tree(&mut root, &online_cpus, &cpu_to_numa)?; >> + >> + root.compute_cpuset(); >> + >> + Ok(Topology { online_cpus, root }) >> + } >> + >> + fn build_tree( >> + root: &mut TopoObject, >> + online_cpus: &CpuSet, >> + cpu_to_numa: &HashMap, >> + ) -> Result<(), CpuError> { >> + let cpu_dir =3D Path::new(CPU_DIR); >> + >> + for cpu_id in online_cpus.iter() { >> + let topo_dir =3D cpu_dir.join(format!("cpu{cpu_id}")).join(= "topology"); >> + >> + let pkg_id =3D sysfs::read_file(&topo_dir.join("physical_pa= ckage_id")) >> + .and_then(|s| s.parse().ok()) >> + .unwrap_or(0); >> + >> + let die_id =3D sysfs::read_file(&topo_dir.join("die_id")) >> + .and_then(|s| s.parse().ok()) >> + .unwrap_or(0); >> + >> + let core_id =3D sysfs::read_file(&topo_dir.join("core_id")) >> + .and_then(|s| s.parse().ok()) >> + .unwrap_or(cpu_id); >> + >> + let pkg =3D root.get_or_add_child(ObjectKind::Package, pkg_= id); >> + let die =3D pkg.get_or_add_child(ObjectKind::Die, die_id); >> + >> + // NumaNode is optional. If present, it sits between Die an= d Core. >> + let parent_of_core =3D if let Some(numa_id) =3D cpu_to_numa= .get(&cpu_id).copied() { >> + die.get_or_add_child(ObjectKind::NumaNode, numa_id) >> + } else { >> + &mut *die >> + }; >> + >> + let core =3D parent_of_core.get_or_add_child(ObjectKind::Co= re, core_id); >> + core.children.push(TopoObject::new(ObjectKind::Pu, cpu_id))= ; >> + } >> + >> + Ok(()) >> + } >> + >> + pub fn online_cpus(&self) -> &CpuSet { >> + &self.online_cpus >> + } >> + >> + pub fn numa_cpus(&self, os_index: u32) -> Option<&CpuSet> { >> + self.root >> + .find(ObjectKind::NumaNode, os_index) >> + .map(|node| &node.cpuset) >> + } >> + >> + /// Reduces `cpus` to at most one thread per physical core. >> + /// (Avoids SMT siblings) >> + pub fn singlify_per_core(&self, cpus: &CpuSet) -> CpuSet { >> + let mut result =3D CpuSet::new(); >> + self.root.for_each(ObjectKind::Core, &mut |core| { >> + let mut core_cpus =3D core.cpuset.clone(); >> + core_cpus.intersect(cpus); >> + if let Some(first) =3D core_cpus.first_set() { >> + result.set(first); >> + } >> + }); >> + result >> + } >> + >> + fn read_cpu_to_numa() -> HashMap { >> + let mut map =3D HashMap::new(); >> + let node_dir =3D Path::new(NODE_DIR); >> + >> + let entries =3D match fs::read_dir(node_dir) { >> + Ok(entries) =3D> entries, >> + Err(_) =3D> return map, >> + }; >> + >> + for entry in entries.flatten() { >> + let name =3D entry.file_name(); >> + let name_str =3D name.to_string_lossy(); >> + let node_id =3D match name_str.strip_prefix("node").and_the= n(|s| s.parse::().ok()) { >> + Some(id) =3D> id, >> + None =3D> continue, >> + }; >> + >> + let cpuset =3D sysfs::read_file(&entry.path().join("cpulist= ")) >> + .map(|s| CpuSet::from_list(&s)) >> + .unwrap_or_else(CpuSet::new); >> + >> + for cpu in cpuset.iter() { >> + map.insert(cpu, node_id); >> + } >> + } >> + map >> + } >> +} >> diff --git a/pve-qemu-server-cpu/src/topology/sysfs.rs b/pve-qemu-server= -cpu/src/topology/sysfs.rs >> new file mode 100644 >> index 0000000..a1ff67a >> --- /dev/null >> +++ b/pve-qemu-server-cpu/src/topology/sysfs.rs >> @@ -0,0 +1,6 @@ >> +use std::fs; >> +use std::path::Path; >> + >> +pub fn read_file(path: &Path) -> Option { >> + fs::read_to_string(path).ok().map(|s| s.trim().to_string()) >> +} > > > there are already helpers in proxmox-sys::fs::file for things like this= =20 > (e.g. `file_read_string`) > > it's just a small function but reusing the ones we have is better IMO Thanks, I wasn't aware of this!