From: Elias Huhsovitz <e.huhsovitz@proxmox.com>
To: pve-devel@lists.proxmox.com
Cc: Elias Huhsovitz <e.huhsovitz@proxmox.com>
Subject: [RFC qemu-server v2 1/4] pinning: add topology discovery and config parsing
Date: Mon, 21 Sep 2026 11:54:00 +0200 [thread overview]
Message-ID: <20260921095404.61552-2-e.huhsovitz@proxmox.com> (raw)
In-Reply-To: <20260921095404.61552-1-e.huhsovitz@proxmox.com>
Introduce Topology.pm and Config.pm to parse host CPU layouts from sysfs
and handle guest pinning configuration.
The config module parses 3 new modes that buid the mapping between guest
vCPUs and virtual NUMA nodes:
balanced: Automatically selects the host NUMA node(s) with the most
uncommitted memory, using as many as needed to fit the VM's CPU and
memory requirements.
1. numa: Pins the sets of vCPUs of each virtual NUMA node to a
corresponding host NUMA node.
2. one-to-one: Pins each vCPU to a specific host core. This prevents
vCPUs from being rescheduled on other cores entirely.
3. The topology module reads physical packages, cores, and SMT siblings,
and detects heterogeneous CPUs (e.g., Intel P/E cores, ARM big.LITTLE).
Originally-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Elias Huhsovitz <e.huhsovitz@proxmox.com>
---
src/PVE/QemuServer/Pinning/Config.pm | 244 +++++++++++++
src/PVE/QemuServer/Pinning/Topology.pm | 462 +++++++++++++++++++++++++
2 files changed, 706 insertions(+)
create mode 100644 src/PVE/QemuServer/Pinning/Config.pm
create mode 100644 src/PVE/QemuServer/Pinning/Topology.pm
diff --git a/src/PVE/QemuServer/Pinning/Config.pm b/src/PVE/QemuServer/Pinning/Config.pm
new file mode 100644
index 00000000..e71fdd84
--- /dev/null
+++ b/src/PVE/QemuServer/Pinning/Config.pm
@@ -0,0 +1,244 @@
+package PVE::QemuServer::Pinning::Config;
+
+use v5.36;
+
+use PVE::JSONSchema qw(parse_property_string);
+use PVE::QemuServer::Helpers;
+use PVE::QemuServer::Memory;
+
+use constant DEFAULT_VCPU_PINNING => 'none';
+
+our $pinning_fmt = {
+ mode => {
+ type => 'string',
+ enum => ['balanced', 'one-to-one', 'numa', DEFAULT_VCPU_PINNING],
+ description => 'Set the vCPU pinning mode.',
+ verbose_description => <<'EODESCR',
+There are multiple ways to pin vCPUs to host cores:
+
+* none (default): Applies no pinning.
+
+* balanced: Automatically selects the host NUMA node(s) with the most
+ uncommitted memory, using as many as needed to fit the VM's CPU and memory
+ requirements. Binds the VM's memory to the selected host node(s) with
+ policy=bind. The QEMU process is confined to the selected host nodes CPUs,
+ but individual vCPU threads are NOT pinned. The host scheduler may move
+ them freely within the selected nodes. Requires numa: 1 to be enabled.
+ Use 'hostnodes' to restrict which host nodes are considered.
+
+* numa: Pins the sets of vCPUs of each virtual NUMA node to a corresponding
+ host NUMA node. This makes memory access consistent for each vCPU because
+ the scheduler will not move it to a different NUMA node. It considers the
+ 'numaX' setting when binding vCPUs to host nodes.
+ Requires numa: 1 to be enabled.
+
+* one-to-one: Pins each vCPU to a specific host core. This prevents vCPUs
+ from being rescheduled on other cores entirely. It provides the highest
+ performance but is the least flexible for the host scheduler. It considers
+ the virtual and physical NUMA layout and avoids SMT collisions by spreading
+ vCPUs across distinct physical cores whenever possible. When used with
+ numa: 1, memory is also bound to host NUMA nodes. When used without numa: 1,
+ only CPU pinning is performed, memory is not bound to specific host nodes.
+
+All settings are affected by 'affinity', to only consider those cores.
+
+These options do not consider pinning settings from other virtual machines or
+containers. To achieve the best and most consistent performance, use the
+combination of 'numaX' and 'affinity' options to ensure host cores are not
+crowded with vCPU assignments.
+EODESCR
+ default => DEFAULT_VCPU_PINNING,
+ default_key => 1,
+ },
+ hostnodes => {
+ type => 'string',
+ pattern => qr/\d+(?:-\d+)?(?:;\d+(?:-\d+)?)*/,
+ description => 'Restrict host NUMA nodes the VM may use for pinning.',
+ verbose_description => <<'EODESCR',
+Semicolon-separated list of host NUMA node IDs (or ranges) that the VM is
+allowed to use for vCPU pinning and memory allocation. When set, the
+pinning allocator only considers these host nodes as candidates for
+auto-assignment.
+
+If not set, all host NUMA nodes are considered.
+
+Example: pinning=balanced,hostnodes=0;1 restricts the VM to host NUMA
+nodes 0 and 1 only.
+EODESCR
+ format_description => 'id[-id];...',
+ optional => 1,
+ },
+};
+
+PVE::JSONSchema::register_format('pve-qm-cpu-pinning', $pinning_fmt);
+
+our $pinning_desc = {
+ optional => 1,
+ type => 'string',
+ format => 'pve-qm-cpu-pinning',
+ description => 'CPU pinning settings.',
+};
+
+PVE::JSONSchema::register_standard_option('pve-qm-cpu-pinning', $pinning_desc);
+
+# Parses a semicolon-separated list of host NUMA node IDs/ranges,
+# into a sorted arrayref of node ID
+# in: 3-5;0;1 out: [0,1,3,4,5]
+my sub parse_host_node_list($id_ranges) {
+ return undef if !defined($id_ranges) || $id_ranges eq '';
+
+ my $ranges = PVE::QemuServer::Helpers::parse_number_sets($id_ranges);
+ my @ids;
+ for my $range ($ranges->@*) {
+ my ($start, $end) = $range->@*;
+ $end //= $start;
+ push @ids, ($start .. $end);
+ }
+ @ids = sort { $a <=> $b } @ids;
+ return @ids ? \@ids : undef;
+}
+
+# in: "balanced,hostnodes=0;1";
+# out: {mode => "balanced", hostnodes => [0,1]}
+sub parse_pinning($setting_str) {
+ return { mode => DEFAULT_VCPU_PINNING } if !defined($setting_str);
+
+ my $settings_map = parse_property_string($pinning_fmt, $setting_str);
+ $settings_map->{mode} = DEFAULT_VCPU_PINNING if !defined($settings_map->{mode});
+ $settings_map->{hostnodes} = parse_host_node_list($settings_map->{hostnodes}) if (defined($settings_map->{hostnodes}));
+
+ return $settings_map;
+}
+
+# convert NUMA node range list to vCPU set
+# in: {cpus => [ [0, 1], [3]]}
+# out: {0, 1, 3}
+my sub expand_numa_vcpus($numa_map) {
+ my %vcpus;
+ for my $range ($numa_map->{cpus}->@*) {
+ my ($start, $end) = $range->@*;
+ $vcpus{$_} = 1 for $start .. $end // $start;
+ }
+ return \%vcpus;
+}
+
+# in: {hostnodes => [[1, 1]]};
+# out: 1
+my sub extract_numa_hostnode($numa_map) {
+ my $hostnodes = $numa_map->{hostnodes};
+ return undef if !defined($hostnodes);
+
+ die "Pinning only available for 1-to-1 NUMA node mapping\n"
+ if $hostnodes->@* > 1 || defined($hostnodes->[0]->[1]);
+
+ return $hostnodes->[0]->[0];
+}
+
+# in:
+# {
+# numa => 1,
+# numa0 => "cpus=0-1,hostnodes=0",
+# numa1 => "cpus=2-3,hostnodes=1",
+# }
+# out:
+# {
+# 0 => {vcpus => {0, 1} hostnode => 0},
+# 1 => {vcpus => {2, 3}, hostnode => 1,}
+# }
+my sub parse_numa_vcpu_map($conf) {
+ my $map = {};
+
+ for my $i (0 .. $PVE::QemuServer::Memory::MAX_NUMA - 1) {
+ my $entry = $conf->{"numa$i"} or next;
+ my $numa = PVE::QemuServer::Memory::parse_numa($entry) or next;
+
+ $map->{$i} = {
+ vcpus => expand_numa_vcpus($numa),
+ hostnode => extract_numa_hostnode($numa)
+ };
+ }
+
+ return $map;
+}
+
+# Parse cpu settings to map keyed by numa node.
+# in: ($sockets, $cores, $threads_per_core) = (2, 2, 1);
+# out: { 0 => {vcpus => {0, 1}}, 1 => {vcpus => {2, 3}} }
+my sub build_socket_vcpu_map($sockets, $cores, $threads_per_core) {
+ my $map = {};
+ my $vcpu_id = 0;
+
+ for my $socket (0 .. $sockets - 1) {
+ my %vcpus;
+ for (1 .. ($cores * $threads_per_core)) {
+ $vcpus{ $vcpu_id++ } = 1;
+ }
+ $map->{$socket} = { vcpus => \%vcpus };
+ }
+
+ return $map;
+}
+
+# in: ($sockets, $cores, $threads_per_core) = (2, 2, 1);
+# out: { 0 => {vcpus => {0, 1, 2, 3}} }
+my sub build_flat_vcpu_map($sockets, $cores, $threads_per_core) {
+ my $total = $sockets * $cores * $threads_per_core;
+ return {
+ 0 => { vcpus => { map { $_ => 1 } 0 .. $total - 1 } },
+ };
+}
+
+# Parse qm.conf cpu settings to map.
+# The map is keyed by numa-node
+# in: {
+# sockets => 1,
+# cores => 4,
+# vcpus => 4,
+# numa => 1,
+# numa0 => "cpus=0-1,hostnodes=0",
+# numa1 => "cpus=2-3,hostnodes=1",
+# }
+# out: {
+# 0 => {vcpus => {0, 1}, hostnode => 0},
+# 1 => {vcpus => {2, 3}, hostnode => 1},
+# }
+sub build_vnuma_vcpu_map($conf) {
+ my $sockets = $conf->{sockets} // 1;
+ my $cores = $conf->{cores} // 1;
+ # Fix 4.4: honour $conf->{vcpus} when it exceeds sockets*cores (SMT/hotplug).
+ my $vcpus = $conf->{vcpus} // ($sockets * $cores);
+ my $threads_per_core = $vcpus > ($sockets * $cores)
+ ? int($vcpus / ($sockets * $cores))
+ : 1;
+
+ my $map;
+ if ($conf->{numa}) {
+ $map = parse_numa_vcpu_map($conf);
+ $map = build_socket_vcpu_map($sockets, $cores, $threads_per_core)
+ if !keys $map->%*;
+ } else {
+ $map = build_flat_vcpu_map($sockets, $cores, $threads_per_core);
+ }
+
+ my $assigned = 0;
+ $assigned += scalar keys $_->{vcpus}->%* for values $map->%*;
+
+ my $expected = $sockets * $cores * $threads_per_core;
+ die "Invalid NUMA configuration for pinning, some vCPUs missing in numa"
+ . " binding (assigned=$assigned, expected=$expected)\n"
+ if $assigned != $expected;
+
+ return $map;
+}
+
+# in: $conf = { sockets => 2, cores => 4, vcpus => 6 };
+# out: (2, 4, 8, 6)
+sub get_config_topology($conf) {
+ my $sockets = $conf->{sockets} || $conf->{smp} || 1;
+ my $cores = $conf->{cores} || 1;
+ my $maxcpus = $sockets * $cores;
+ my $vcpus = $conf->{vcpus} || $maxcpus;
+ return ($sockets, $cores, $maxcpus, $vcpus);
+}
+
+1;
diff --git a/src/PVE/QemuServer/Pinning/Topology.pm b/src/PVE/QemuServer/Pinning/Topology.pm
new file mode 100644
index 00000000..daddea0b
--- /dev/null
+++ b/src/PVE/QemuServer/Pinning/Topology.pm
@@ -0,0 +1,462 @@
+package PVE::QemuServer::Pinning::Topology;
+
+use v5.36;
+
+use PVE::CpuSet;
+use PVE::Tools qw(file_read_firstline dir_glob_foreach);
+
+# Build { cpu_id => numa_node_id } from /sys/devices/system/node.
+# key: cpu, val: numa-node
+# e.g. reads: 8 core CPU with 2 NUMA nodes
+# out: { 0 => 0, 1 => 0, 2 => 0, 3 => 0, 4 => 1, 5 => 1, 6 => 1, 7 => 1 }
+my sub cpu_to_numa_map() {
+ my $cpu_to_numa = {};
+ my $numa_path = "/sys/devices/system/node";
+
+ return $cpu_to_numa if !-d $numa_path;
+
+ dir_glob_foreach($numa_path, 'node(\d+)', sub ($node_dir, $node_id) {
+ dir_glob_foreach("$numa_path/$node_dir", 'cpu(\d+)', sub ($cpu_dir, $cpu_id) {
+ $cpu_to_numa->{$cpu_id} = $node_id;
+ });
+ });
+
+ return $cpu_to_numa;
+}
+
+# Build { cpu_id => topology_dir_path } for all CPUs under /sys.
+# in: (no args, reads sysfs)
+# out: { 0 => "/sys/devices/system/cpu/cpu0/topology",
+# 1 => "/sys/devices/system/cpu/cpu1/topology", ... }
+my sub cpu_topology_dirs() {
+ my $base_path = "/sys/devices/system/cpu";
+ opendir(my $dirfd, $base_path) || die "cannot open $base_path\n";
+
+ my $dirs = {};
+ for my $entry (readdir($dirfd)) {
+ next if $entry !~ m/^cpu(\d+)$/;
+ my $topo_path = "$base_path/$entry/topology";
+ next if !-d $topo_path;
+ $dirs->{$1} = $topo_path;
+ }
+ closedir($dirfd);
+
+ return $dirs;
+}
+
+# Read physical_package_id and core_id, normalizing sentinel values to 0.
+# in: "/sys/devices/system/cpu/cpu1/topology"
+# out: (0, 1)
+# core_id seems to be set by the vendor and non-linear.
+# e.g. for the AMD EPYC 7351P 16-Core Processor cpu 4
+# and its thread sibling cpu 16 reside on core 8
+# e.g. /sys/devices/system/cpu/cpu4/topology/
+# out: (0, 8)
+my sub read_cpu_topology($topo_path) {
+ my $socket_id = file_read_firstline("$topo_path/physical_package_id") // 0;
+ my $core_id = file_read_firstline("$topo_path/core_id") // 0;
+
+ # missing topology info is represented as sential value (e.g. nested virtualization)
+ $socket_id = 0 if $socket_id < 0 || $socket_id == 65535;
+ $core_id = 0 if $core_id < 0 || $core_id == 65535;
+
+ return ($socket_id, $core_id);
+}
+
+my sub make_cpu_info($socket_id, $core_id, $numa_node) {
+ return {
+ socket => $socket_id,
+ core => "${socket_id}_${core_id}",
+ numa_node => $numa_node,
+ };
+}
+
+# Add one CPU into topology (mutates cpus/cores/numa_nodes).
+# in: {cpus=>{},cores=>{},numa_nodes=>{}}, 3, {socket=>0,core=>"0_3",numa_node=>1}
+# out: (mutated topology)
+# cpus => {3 => {socket=>0, core=>"0_3", numa_node=>1}}
+# cores => {"0_3" => {numa_node=>1, cpus=>[3]}}
+# numa_nodes => {1 => {cores=>{"0_3"}, cpus=>{3}}}
+my sub add_cpu_to_topology($topology, $cpu_id, $cpu_info) {
+ $topology->{cpus}->{$cpu_id} = $cpu_info;
+
+ my $core_key = $cpu_info->{core};
+ my $numa_node = $cpu_info->{numa_node};
+
+ $topology->{cores}->{$core_key} //= {
+ numa_node => $numa_node,
+ cpus => [],
+ };
+
+ push $topology->{cores}->{$core_key}->{cpus}->@*, $cpu_id;
+
+ $topology->{numa_nodes}->{$numa_node} //= {
+ cores => {},
+ cpus => {},
+ };
+ $topology->{numa_nodes}->{$numa_node}->{cores}->{$core_key} = 1;
+ $topology->{numa_nodes}->{$numa_node}->{cpus}->{$cpu_id} = 1;
+}
+
+=head2 get_host_topology
+
+Reads the host CPU topology from sysfs and returns it as a hashref with
+three keys:
+
+=over
+
+=item * C<cpus> - C<< { $cpu_id => { socket, core, numa_node } } >>, one
+entry per CPU. C<core> is a C<"${socket}_${core_id}"> string.
+
+=item * C<cores> - C<< { $core_key => { numa_node, cpus } } >>, where
+C<cpus> is an arrayref of the CPU ids sharing that physical core.
+
+=item * C<numa_nodes> - C<< { $node_id => { cores, cpus } } >>, where
+C<cores> and C<cpus> are sets (hashrefs of id => 1).
+
+=back
+
+Missing or unavailable topology information is normalized to socket and
+core C<0>, and CPUs without an entry under C</sys/devices/system/node> are
+placed on NUMA node C<0>. B<Dies> if C</sys/devices/system/cpu> cannot be
+opened.
+
+B<Example> - x86 CPU:
+1 socket, 8 cores, 2 CPUs per core (SMT siblings), 2 NUMA nodes
+
+ {
+ # Keyed by CPU id. socket = physical package, core = "socket_coreid"
+ # (the key used to index C<cores> below), numa_node = host NUMA node.
+ #
+ # CPUs 8..15 are the SMT siblings of 0..7: same socket and
+ # core, therefore same NUMA node (e.g. 0 and 8 both on node 0).
+ cpus => {
+ 0 => { socket => 0, core => "0_0", numa_node => 0 },
+ 1 => { socket => 0, core => "0_1", numa_node => 0 },
+ 2 => { socket => 0, core => "0_2", numa_node => 0 },
+ 3 => { socket => 0, core => "0_3", numa_node => 0 },
+ 4 => { socket => 0, core => "0_4", numa_node => 1 },
+ 5 => { socket => 0, core => "0_5", numa_node => 1 },
+ 6 => { socket => 0, core => "0_6", numa_node => 1 },
+ 7 => { socket => 0, core => "0_7", numa_node => 1 },
+ 8 => { socket => 0, core => "0_0", numa_node => 0 },
+ 9 => { socket => 0, core => "0_1", numa_node => 0 },
+ 10 => { socket => 0, core => "0_2", numa_node => 0 },
+ 11 => { socket => 0, core => "0_3", numa_node => 0 },
+ 12 => { socket => 0, core => "0_4", numa_node => 1 },
+ 13 => { socket => 0, core => "0_5", numa_node => 1 },
+ 14 => { socket => 0, core => "0_6", numa_node => 1 },
+ 15 => { socket => 0, core => "0_7", numa_node => 1 },
+ },
+
+ # Keyed by core key ("socket_coreid").
+ cores => {
+ # CPUs 0 and 8 are SMT siblings
+ "0_0" => { numa_node => 0, cpus => [0, 8] },
+ "0_1" => { numa_node => 0, cpus => [1, 9] },
+ "0_2" => { numa_node => 0, cpus => [2, 10] },
+ # CPUs 3 and 11 are SMT siblings (see their entries above).
+ "0_3" => { numa_node => 0, cpus => [3, 11] },
+ "0_4" => { numa_node => 1, cpus => [4, 12] },
+ "0_5" => { numa_node => 1, cpus => [5, 13] },
+ "0_6" => { numa_node => 1, cpus => [6, 14] },
+ "0_7" => { numa_node => 1, cpus => [7, 15] },
+ },
+
+ # Keyed by NUMA node id. Each value holds two sets (id => 1): the
+ # core keys on that node, and the CPU ids on that node. The 1 values
+ # are not counts; only the keys are meaningful.
+ numa_nodes => {
+ 0 => {
+ cores => { "0_0" => 1, "0_1" => 1, "0_2" => 1, "0_3" => 1 },
+ cpus => { 0 => 1, 1 => 1, 2 => 1, 3 => 1,
+ 8 => 1, 9 => 1, 10 => 1, 11 => 1 },
+ },
+ 1 => {
+ cores => { "0_4" => 1, "0_5" => 1, "0_6" => 1, "0_7" => 1 },
+ cpus => { 4 => 1, 5 => 1, 6 => 1, 7 => 1,
+ 12 => 1, 13 => 1, 14 => 1, 15 => 1 },
+ },
+ },
+ }
+=cut
+sub get_host_topology() {
+ my $cpu_to_numa = cpu_to_numa_map();
+
+ my $topology = {
+ cpus => {},
+ cores => {},
+ numa_nodes => {},
+ };
+
+ my $cpu_dirs = cpu_topology_dirs();
+
+ for my $cpu_id (sort { $a <=> $b } keys $cpu_dirs->%*) {
+ my ($socket_id, $core_id) = read_cpu_topology($cpu_dirs->{$cpu_id});
+
+ my $numa_node = $cpu_to_numa->{$cpu_id} // 0;
+
+ my $cpu_info = make_cpu_info($socket_id, $core_id, $numa_node);
+ add_cpu_to_topology($topology, $cpu_id, $cpu_info);
+ }
+
+ return $topology;
+}
+
+my sub read_sysfs_trim($path) {
+ return undef if !-e $path;
+ my $val = file_read_firstline($path);
+ return undef if !defined($val);
+ $val =~ s/^\s+|\s+$//g;
+ return $val;
+}
+
+# in: /sys/devices/system/cpu/online = 0-7
+# out: [0, 1, 2, 3, 4, 5, 6, 7]
+my sub online_cpu_ids() {
+ my $content = read_sysfs_trim("/sys/devices/system/cpu/online");
+ return [] if !$content;
+
+ my @ids;
+ for my $part (split(/,/, $content)) {
+ if ($part =~ /^(\d+)-(\d+)$/) {
+ push @ids, ($1 .. $2);
+ } elsif ($part =~ /^(\d+)$/) {
+ push @ids, $1;
+ }
+ }
+ return \@ids;
+}
+
+# Build a stable "CPU type" key for a single online CPU.
+# On ARM: MIDR register + cache geometry.
+# On x86 (no MIDR): cache geometry alone (level/type/line/ways/sets).
+# On intel CPUs different P/E cores or big.LITTLE cores produce different keys because
+# they have different cache geometries even when /proc/cpuinfo model names
+# are identical or absent.
+# Note: Only tested on AMD x86 CPUs, so take this with a grain of salt.
+my sub cpu_type_key($cpu_id) {
+ my $cpu_path = "/sys/devices/system/cpu/cpu$cpu_id";
+
+ my $key = '';
+
+ # MIDR (ARM only). Absent on x86.
+ my $midr = read_sysfs_trim("$cpu_path/regs/identification/midr_el1");
+ $key .= defined($midr) ? $midr : 'no-midr';
+
+ # Cache geometry: sorted cache indices, each with level/type/line/ways/sets.
+ # This distinguishes P-cores from E-cores on x86 (different L2/L3 sizes)
+ # and provides a secondary signal on ARM.
+ my $cache_base = "$cpu_path/cache";
+ if (-d $cache_base) {
+ opendir(my $dh, $cache_base) or return $key;
+ my @indices = sort { $a <=> $b }
+ map { /^index(\d+)$/ ? $1 : () }
+ readdir($dh);
+ closedir($dh);
+
+ for my $idx (@indices) {
+ my $idx_path = "$cache_base/index$idx";
+ for my $field (qw(level type coherency_line_size ways_of_associativity number_of_sets)) {
+ my $val = read_sysfs_trim("$idx_path/$field");
+ $key .= '/' . (defined($val) ? $val : '');
+ }
+ }
+ }
+
+ return $key;
+}
+
+# Group online CPUs by cpu_type_key. capacity defaults to 1024/CPU.
+# out: ({ "abc/..." => {cpus => {0, 1, 2, 3}, capacity => 4096} },
+# [ "abc/..." ])
+sub group_cpus_by_type() {
+ my $online_ids = online_cpu_ids();
+ return ({}, []) if !$online_ids || !@$online_ids;
+
+ my %groups;
+ my @keys;
+
+ for my $cpu_id (@$online_ids) {
+ my $key = cpu_type_key($cpu_id);
+ if (!$groups{$key}) {
+ $groups{$key} = { cpus => [], capacity => 0 };
+ push @keys, $key;
+ }
+ push $groups{$key}{cpus}->@*, $cpu_id;
+
+ my $cap = 1024;
+ my $cap_val = read_sysfs_trim("/sys/devices/system/cpu/cpu$cpu_id/cpu_capacity");
+ if (defined($cap_val) && $cap_val =~ /^(\d+)$/) {
+ $cap = int($1);
+ }
+ $groups{$key}{capacity} += $cap;
+ }
+
+ # Deterministic key order: by first CPU id in each group.
+ @keys = sort { $groups{$a}{cpus}[0] <=> $groups{$b}{cpus}[0] } @keys;
+
+ return (\%groups, \@keys);
+}
+
+sub are_cpus_heterogenous($cpu_ids_arrayref) {
+ my ($groups, $keys) = group_cpus_by_type();
+ return 0 if !@$keys || @$keys <= 1;
+
+ my %requested = map { $_ => 1 } @$cpu_ids_arrayref;
+ my $seen_key;
+
+ for my $key (@$keys) {
+ for my $cpu_id (@{$groups->{$key}{cpus}}) {
+ next if !$requested{$cpu_id};
+ if (!defined $seen_key) {
+ $seen_key = $key;
+ } elsif ($key ne $seen_key) {
+ return 1;
+ }
+ last;
+ }
+ }
+ return 0;
+}
+
+sub filter_by_affinity($topology, $affinity_string) {
+ return $topology if !defined($affinity_string) || $affinity_string eq '';
+
+ my ($_count, $affinity_members) = PVE::CpuSet::parse_cpuset($affinity_string);
+
+ my $filtered = {
+ cpus => {},
+ cores => {},
+ numa_nodes => {},
+ };
+
+ for my $cpu_id (sort { $a <=> $b } keys $topology->{cpus}->%*) {
+ next if !$affinity_members->{$cpu_id};
+ my $cpu_info = $topology->{cpus}->{$cpu_id};
+ add_cpu_to_topology($filtered, $cpu_id, $cpu_info);
+ }
+
+ return $filtered;
+}
+
+# Group selected host CPUs by socket -> core -> threads.
+# in: $topology, [0, 1, 4, 5]
+# out: ({ 0 => { "0_0" => [0, 4], "0_1" => [1, 5] } }, undef)
+# in: $topology, [0, 99]
+# out: (undef, "CPU 99 not found in host topology")
+my sub group_cpus_by_socket_core($topology, $selected_cpus) {
+ my %sockets;
+ for my $cpu_id (@$selected_cpus) {
+ my $info = $topology->{cpus}->{$cpu_id};
+ return (undef, "CPU $cpu_id not found in host topology") if !$info;
+ push @{$sockets{$info->{socket}}{$info->{core}}}, $cpu_id;
+ }
+ return (\%sockets, undef);
+}
+
+# Check every socket has same core count and every core same thread count.
+# in: { 0 => { "0_0" => [0, 8], "0_4" => [4, 12] } }
+# out: (2, 2)
+# in: { 0 => { "0_0" => [0, 8] }, 1 => { "1_0" => [1, 9], "1_1" => [2, 10] } }
+# out: (undef, undef)
+my sub validate_consistency($sockets) {
+ my ($expected_cores, $expected_threads) = (-1, -1);
+
+ for my $socket (keys %$sockets) {
+ my $core_count = scalar keys %{$sockets->{$socket}};
+ return (undef, undef)
+ if $expected_cores != -1 && $core_count != $expected_cores;
+ $expected_cores = $core_count;
+
+ for my $core (keys %{$sockets->{$socket}}) {
+ my $thread_count = scalar @{$sockets->{$socket}->{$core}};
+ return (undef, undef)
+ if $expected_threads != -1 && $thread_count != $expected_threads;
+ $expected_threads = $thread_count;
+ }
+ }
+
+ return ($expected_cores, $expected_threads);
+}
+
+my sub flat_fallback_result($total, $warning = undef) {
+ $warning //= "Instance uses a CPU pinning profile which doesn't match"
+ . " hardware layout; falling back to flat topology"
+ . " (sockets=1, cores=$total, threads=1)";
+ return {
+ valid => 0,
+ sockets => 1,
+ cores => $total,
+ threads => 1,
+ warning => $warning,
+ };
+}
+
+=encoding utf8
+
+=head2 validate_pinned_topology
+
+Validates that the host CPUs in C<$selected_cpus> (an arrayref of host CPU
+IDs) form a balanced topology matching the guest configuration.
+
+=over 4
+
+=item 1. Groups selected host CPUs by physical socket and physical core.
+
+=item 2. Checks that every socket contains the same number of cores.
+
+=item 3. Checks that every core contains the same number of threads.
+
+=item 4. Checks that C<sockets * cores_per_socket * threads_per_core> equals
+the total number of selected CPUs.
+
+=back
+
+Returns a hashref:
+
+ {
+ valid => 1 | 0,
+ sockets => $nr_sockets,
+ cores => $nr_cores_per_socket,
+ threads => $nr_threads_per_core,
+ warning => $msg | undef,
+ }
+
+If the topology is invalid, the result contains a flat fallback
+(C<sockets=1, cores=N, threads=1>) and a warning string.
+
+=cut
+sub validate_pinned_topology($topology, $selected_cpus, $guest_sockets, $guest_cores) {
+ my $total = scalar @$selected_cpus;
+
+ # Empty selection: trust the guest-configured topology.
+ return {
+ valid => 1,
+ sockets => $guest_sockets,
+ cores => $guest_cores,
+ threads => 1,
+ warning => undef,
+ } if $total == 0;
+
+ my ($sockets, $err) = group_cpus_by_socket_core($topology, $selected_cpus);
+ return flat_fallback_result($total, $err) if defined($err);
+
+ my ($expected_cores, $expected_threads) = validate_consistency($sockets);
+ return flat_fallback_result($total)
+ if !defined $expected_cores || !defined $expected_threads;
+
+ my $nr_sockets = scalar keys %$sockets;
+ return flat_fallback_result($total)
+ if $nr_sockets * $expected_cores * $expected_threads != $total;
+
+ return {
+ valid => 1,
+ sockets => $nr_sockets,
+ cores => $expected_cores,
+ threads => $expected_threads,
+ warning => undef,
+ };
+}
+
+1;
--
2.47.3
next prev parent reply other threads:[~2026-09-21 9:54 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-21 9:53 [RFC qemu-server v2 0/4] fix #7282: allow (NUMA aware) vCPU pinning Elias Huhsovitz
2026-09-21 9:54 ` Elias Huhsovitz [this message]
2026-09-21 9:54 ` [RFC qemu-server v2 2/4] pinning: add NUMA allocator and reservation tracking Elias Huhsovitz
2026-09-21 9:54 ` [RFC qemu-server v2 3/4] memory: integrate pinning-aware NUMA memory binding Elias Huhsovitz
2026-09-21 9:54 ` [RFC qemu-server v2 4/4] pinning: integrate cpu pinning into vm lifecycle Elias Huhsovitz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260921095404.61552-2-e.huhsovitz@proxmox.com \
--to=e.huhsovitz@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox