From: Elias Huhsovitz <e.huhsovitz@proxmox.com>
To: pve-devel@lists.proxmox.com
Cc: Elias Huhsovitz <e.huhsovitz@proxmox.com>
Subject: [RFC qemu-server v2 1/4] pinning: add topology discovery and config parsing
Date: Mon, 21 Sep 2026 11:54:00 +0200 [thread overview]
Message-ID: <20260921095404.61552-2-e.huhsovitz@proxmox.com> (raw)
In-Reply-To: <20260921095404.61552-1-e.huhsovitz@proxmox.com>
Introduce Topology.pm and Config.pm to parse host CPU layouts from sysfs
and handle guest pinning configuration.
The config module parses 3 new modes that buid the mapping between guest
vCPUs and virtual NUMA nodes:
balanced: Automatically selects the host NUMA node(s) with the most
uncommitted memory, using as many as needed to fit the VM's CPU and
memory requirements.
1. numa: Pins the sets of vCPUs of each virtual NUMA node to a
corresponding host NUMA node.
2. one-to-one: Pins each vCPU to a specific host core. This prevents
vCPUs from being rescheduled on other cores entirely.
3. The topology module reads physical packages, cores, and SMT siblings,
and detects heterogeneous CPUs (e.g., Intel P/E cores, ARM big.LITTLE).
Originally-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Elias Huhsovitz <e.huhsovitz@proxmox.com>
---
src/PVE/QemuServer/Pinning/Config.pm | 244 +++++++++++++
src/PVE/QemuServer/Pinning/Topology.pm | 462 +++++++++++++++++++++++++
2 files changed, 706 insertions(+)
create mode 100644 src/PVE/QemuServer/Pinning/Config.pm
create mode 100644 src/PVE/QemuServer/Pinning/Topology.pm
diff --git a/src/PVE/QemuServer/Pinning/Config.pm b/src/PVE/QemuServer/Pinning/Config.pm
new file mode 100644
index 00000000..e71fdd84
--- /dev/null
+++ b/src/PVE/QemuServer/Pinning/Config.pm
@@ -0,0 +1,244 @@
+package PVE::QemuServer::Pinning::Config;
+
+use v5.36;
+
+use PVE::JSONSchema qw(parse_property_string);
+use PVE::QemuServer::Helpers;
+use PVE::QemuServer::Memory;
+
+use constant DEFAULT_VCPU_PINNING => 'none';
+
+our $pinning_fmt = {
+ mode => {
+ type => 'string',
+ enum => ['balanced', 'one-to-one', 'numa', DEFAULT_VCPU_PINNING],
+ description => 'Set the vCPU pinning mode.',
+ verbose_description => <<'EODESCR',
+There are multiple ways to pin vCPUs to host cores:
+
+* none (default): Applies no pinning.
+
+* balanced: Automatically selects the host NUMA node(s) with the most
+ uncommitted memory, using as many as needed to fit the VM's CPU and memory
+ requirements. Binds the VM's memory to the selected host node(s) with
+ policy=bind. The QEMU process is confined to the selected host nodes CPUs,
+ but individual vCPU threads are NOT pinned. The host scheduler may move
+ them freely within the selected nodes. Requires numa: 1 to be enabled.
+ Use 'hostnodes' to restrict which host nodes are considered.
+
+* numa: Pins the sets of vCPUs of each virtual NUMA node to a corresponding
+ host NUMA node. This makes memory access consistent for each vCPU because
+ the scheduler will not move it to a different NUMA node. It considers the
+ 'numaX' setting when binding vCPUs to host nodes.
+ Requires numa: 1 to be enabled.
+
+* one-to-one: Pins each vCPU to a specific host core. This prevents vCPUs
+ from being rescheduled on other cores entirely. It provides the highest
+ performance but is the least flexible for the host scheduler. It considers
+ the virtual and physical NUMA layout and avoids SMT collisions by spreading
+ vCPUs across distinct physical cores whenever possible. When used with
+ numa: 1, memory is also bound to host NUMA nodes. When used without numa: 1,
+ only CPU pinning is performed, memory is not bound to specific host nodes.
+
+All settings are affected by 'affinity', to only consider those cores.
+
+These options do not consider pinning settings from other virtual machines or
+containers. To achieve the best and most consistent performance, use the
+combination of 'numaX' and 'affinity' options to ensure host cores are not
+crowded with vCPU assignments.
+EODESCR
+ default => DEFAULT_VCPU_PINNING,
+ default_key => 1,
+ },
+ hostnodes => {
+ type => 'string',
+ pattern => qr/\d+(?:-\d+)?(?:;\d+(?:-\d+)?)*/,
+ description => 'Restrict host NUMA nodes the VM may use for pinning.',
+ verbose_description => <<'EODESCR',
+Semicolon-separated list of host NUMA node IDs (or ranges) that the VM is
+allowed to use for vCPU pinning and memory allocation. When set, the
+pinning allocator only considers these host nodes as candidates for
+auto-assignment.
+
+If not set, all host NUMA nodes are considered.
+
+Example: pinning=balanced,hostnodes=0;1 restricts the VM to host NUMA
+nodes 0 and 1 only.
+EODESCR
+ format_description => 'id[-id];...',
+ optional => 1,
+ },
+};
+
+PVE::JSONSchema::register_format('pve-qm-cpu-pinning', $pinning_fmt);
+
+our $pinning_desc = {
+ optional => 1,
+ type => 'string',
+ format => 'pve-qm-cpu-pinning',
+ description => 'CPU pinning settings.',
+};
+
+PVE::JSONSchema::register_standard_option('pve-qm-cpu-pinning', $pinning_desc);
+
+# Parses a semicolon-separated list of host NUMA node IDs/ranges,
+# into a sorted arrayref of node ID
+# in: 3-5;0;1 out: [0,1,3,4,5]
+my sub parse_host_node_list($id_ranges) {
+ return undef if !defined($id_ranges) || $id_ranges eq '';
+
+ my $ranges = PVE::QemuServer::Helpers::parse_number_sets($id_ranges);
+ my @ids;
+ for my $range ($ranges->@*) {
+ my ($start, $end) = $range->@*;
+ $end //= $start;
+ push @ids, ($start .. $end);
+ }
+ @ids = sort { $a <=> $b } @ids;
+ return @ids ? \@ids : undef;
+}
+
+# in: "balanced,hostnodes=0;1";
+# out: {mode => "balanced", hostnodes => [0,1]}
+sub parse_pinning($setting_str) {
+ return { mode => DEFAULT_VCPU_PINNING } if !defined($setting_str);
+
+ my $settings_map = parse_property_string($pinning_fmt, $setting_str);
+ $settings_map->{mode} = DEFAULT_VCPU_PINNING if !defined($settings_map->{mode});
+ $settings_map->{hostnodes} = parse_host_node_list($settings_map->{hostnodes}) if (defined($settings_map->{hostnodes}));
+
+ return $settings_map;
+}
+
+# convert NUMA node range list to vCPU set
+# in: {cpus => [ [0, 1], [3]]}
+# out: {0, 1, 3}
+my sub expand_numa_vcpus($numa_map) {
+ my %vcpus;
+ for my $range ($numa_map->{cpus}->@*) {
+ my ($start, $end) = $range->@*;
+ $vcpus{$_} = 1 for $start .. $end // $start;
+ }
+ return \%vcpus;
+}
+
+# in: {hostnodes => [[1, 1]]};
+# out: 1
+my sub extract_numa_hostnode($numa_map) {
+ my $hostnodes = $numa_map->{hostnodes};
+ return undef if !defined($hostnodes);
+
+ die "Pinning only available for 1-to-1 NUMA node mapping\n"
+ if $hostnodes->@* > 1 || defined($hostnodes->[0]->[1]);
+
+ return $hostnodes->[0]->[0];
+}
+
+# in:
+# {
+# numa => 1,
+# numa0 => "cpus=0-1,hostnodes=0",
+# numa1 => "cpus=2-3,hostnodes=1",
+# }
+# out:
+# {
+# 0 => {vcpus => {0, 1} hostnode => 0},
+# 1 => {vcpus => {2, 3}, hostnode => 1,}
+# }
+my sub parse_numa_vcpu_map($conf) {
+ my $map = {};
+
+ for my $i (0 .. $PVE::QemuServer::Memory::MAX_NUMA - 1) {
+ my $entry = $conf->{"numa$i"} or next;
+ my $numa = PVE::QemuServer::Memory::parse_numa($entry) or next;
+
+ $map->{$i} = {
+ vcpus => expand_numa_vcpus($numa),
+ hostnode => extract_numa_hostnode($numa)
+ };
+ }
+
+ return $map;
+}
+
+# Parse cpu settings to map keyed by numa node.
+# in: ($sockets, $cores, $threads_per_core) = (2, 2, 1);
+# out: { 0 => {vcpus => {0, 1}}, 1 => {vcpus => {2, 3}} }
+my sub build_socket_vcpu_map($sockets, $cores, $threads_per_core) {
+ my $map = {};
+ my $vcpu_id = 0;
+
+ for my $socket (0 .. $sockets - 1) {
+ my %vcpus;
+ for (1 .. ($cores * $threads_per_core)) {
+ $vcpus{ $vcpu_id++ } = 1;
+ }
+ $map->{$socket} = { vcpus => \%vcpus };
+ }
+
+ return $map;
+}
+
+# in: ($sockets, $cores, $threads_per_core) = (2, 2, 1);
+# out: { 0 => {vcpus => {0, 1, 2, 3}} }
+my sub build_flat_vcpu_map($sockets, $cores, $threads_per_core) {
+ my $total = $sockets * $cores * $threads_per_core;
+ return {
+ 0 => { vcpus => { map { $_ => 1 } 0 .. $total - 1 } },
+ };
+}
+
+# Parse qm.conf cpu settings to map.
+# The map is keyed by numa-node
+# in: {
+# sockets => 1,
+# cores => 4,
+# vcpus => 4,
+# numa => 1,
+# numa0 => "cpus=0-1,hostnodes=0",
+# numa1 => "cpus=2-3,hostnodes=1",
+# }
+# out: {
+# 0 => {vcpus => {0, 1}, hostnode => 0},
+# 1 => {vcpus => {2, 3}, hostnode => 1},
+# }
+sub build_vnuma_vcpu_map($conf) {
+ my $sockets = $conf->{sockets} // 1;
+ my $cores = $conf->{cores} // 1;
+ # Fix 4.4: honour $conf->{vcpus} when it exceeds sockets*cores (SMT/hotplug).
+ my $vcpus = $conf->{vcpus} // ($sockets * $cores);
+ my $threads_per_core = $vcpus > ($sockets * $cores)
+ ? int($vcpus / ($sockets * $cores))
+ : 1;
+
+ my $map;
+ if ($conf->{numa}) {
+ $map = parse_numa_vcpu_map($conf);
+ $map = build_socket_vcpu_map($sockets, $cores, $threads_per_core)
+ if !keys $map->%*;
+ } else {
+ $map = build_flat_vcpu_map($sockets, $cores, $threads_per_core);
+ }
+
+ my $assigned = 0;
+ $assigned += scalar keys $_->{vcpus}->%* for values $map->%*;
+
+ my $expected = $sockets * $cores * $threads_per_core;
+ die "Invalid NUMA configuration for pinning, some vCPUs missing in numa"
+ . " binding (assigned=$assigned, expected=$expected)\n"
+ if $assigned != $expected;
+
+ return $map;
+}
+
+# in: $conf = { sockets => 2, cores => 4, vcpus => 6 };
+# out: (2, 4, 8, 6)
+sub get_config_topology($conf) {
+ my $sockets = $conf->{sockets} || $conf->{smp} || 1;
+ my $cores = $conf->{cores} || 1;
+ my $maxcpus = $sockets * $cores;
+ my $vcpus = $conf->{vcpus} || $maxcpus;
+ return ($sockets, $cores, $maxcpus, $vcpus);
+}
+
+1;
diff --git a/src/PVE/QemuServer/Pinning/Topology.pm b/src/PVE/QemuServer/Pinning/Topology.pm
new file mode 100644
index 00000000..daddea0b
--- /dev/null
+++ b/src/PVE/QemuServer/Pinning/Topology.pm
@@ -0,0 +1,462 @@
+package PVE::QemuServer::Pinning::Topology;
+
+use v5.36;
+
+use PVE::CpuSet;
+use PVE::Tools qw(file_read_firstline dir_glob_foreach);
+
+# Build { cpu_id => numa_node_id } from /sys/devices/system/node.
+# key: cpu, val: numa-node
+# e.g. reads: 8 core CPU with 2 NUMA nodes
+# out: { 0 => 0, 1 => 0, 2 => 0, 3 => 0, 4 => 1, 5 => 1, 6 => 1, 7 => 1 }
+my sub cpu_to_numa_map() {
+ my $cpu_to_numa = {};
+ my $numa_path = "/sys/devices/system/node";
+
+ return $cpu_to_numa if !-d $numa_path;
+
+ dir_glob_foreach($numa_path, 'node(\d+)', sub ($node_dir, $node_id) {
+ dir_glob_foreach("$numa_path/$node_dir", 'cpu(\d+)', sub ($cpu_dir, $cpu_id) {
+ $cpu_to_numa->{$cpu_id} = $node_id;
+ });
+ });
+
+ return $cpu_to_numa;
+}
+
+# Build { cpu_id => topology_dir_path } for all CPUs under /sys.
+# in: (no args, reads sysfs)
+# out: { 0 => "/sys/devices/system/cpu/cpu0/topology",
+# 1 => "/sys/devices/system/cpu/cpu1/topology", ... }
+my sub cpu_topology_dirs() {
+ my $base_path = "/sys/devices/system/cpu";
+ opendir(my $dirfd, $base_path) || die "cannot open $base_path\n";
+
+ my $dirs = {};
+ for my $entry (readdir($dirfd)) {
+ next if $entry !~ m/^cpu(\d+)$/;
+ my $topo_path = "$base_path/$entry/topology";
+ next if !-d $topo_path;
+ $dirs->{$1} = $topo_path;
+ }
+ closedir($dirfd);
+
+ return $dirs;
+}
+
+# Read physical_package_id and core_id, normalizing sentinel values to 0.
+# in: "/sys/devices/system/cpu/cpu1/topology"
+# out: (0, 1)
+# core_id seems to be set by the vendor and non-linear.
+# e.g. for the AMD EPYC 7351P 16-Core Processor cpu 4
+# and its thread sibling cpu 16 reside on core 8
+# e.g. /sys/devices/system/cpu/cpu4/topology/
+# out: (0, 8)
+my sub read_cpu_topology($topo_path) {
+ my $socket_id = file_read_firstline("$topo_path/physical_package_id") // 0;
+ my $core_id = file_read_firstline("$topo_path/core_id") // 0;
+
+ # missing topology info is represented as sential value (e.g. nested virtualization)
+ $socket_id = 0 if $socket_id < 0 || $socket_id == 65535;
+ $core_id = 0 if $core_id < 0 || $core_id == 65535;
+
+ return ($socket_id, $core_id);
+}
+
+my sub make_cpu_info($socket_id, $core_id, $numa_node) {
+ return {
+ socket => $socket_id,
+ core => "${socket_id}_${core_id}",
+ numa_node => $numa_node,
+ };
+}
+
+# Add one CPU into topology (mutates cpus/cores/numa_nodes).
+# in: {cpus=>{},cores=>{},numa_nodes=>{}}, 3, {socket=>0,core=>"0_3",numa_node=>1}
+# out: (mutated topology)
+# cpus => {3 => {socket=>0, core=>"0_3", numa_node=>1}}
+# cores => {"0_3" => {numa_node=>1, cpus=>[3]}}
+# numa_nodes => {1 => {cores=>{"0_3"}, cpus=>{3}}}
+my sub add_cpu_to_topology($topology, $cpu_id, $cpu_info) {
+ $topology->{cpus}->{$cpu_id} = $cpu_info;
+
+ my $core_key = $cpu_info->{core};
+ my $numa_node = $cpu_info->{numa_node};
+
+ $topology->{cores}->{$core_key} //= {
+ numa_node => $numa_node,
+ cpus => [],
+ };
+
+ push $topology->{cores}->{$core_key}->{cpus}->@*, $cpu_id;
+
+ $topology->{numa_nodes}->{$numa_node} //= {
+ cores => {},
+ cpus => {},
+ };
+ $topology->{numa_nodes}->{$numa_node}->{cores}->{$core_key} = 1;
+ $topology->{numa_nodes}->{$numa_node}->{cpus}->{$cpu_id} = 1;
+}
+
+=head2 get_host_topology
+
+Reads the host CPU topology from sysfs and returns it as a hashref with
+three keys:
+
+=over
+
+=item * C<cpus> - C<< { $cpu_id => { socket, core, numa_node } } >>, one
+entry per CPU. C<core> is a C<"${socket}_${core_id}"> string.
+
+=item * C<cores> - C<< { $core_key => { numa_node, cpus } } >>, where
+C<cpus> is an arrayref of the CPU ids sharing that physical core.
+
+=item * C<numa_nodes> - C<< { $node_id => { cores, cpus } } >>, where
+C<cores> and C<cpus> are sets (hashrefs of id => 1).
+
+=back
+
+Missing or unavailable topology information is normalized to socket and
+core C<0>, and CPUs without an entry under C</sys/devices/system/node> are
+placed on NUMA node C<0>. B<Dies> if C</sys/devices/system/cpu> cannot be
+opened.
+
+B<Example> - x86 CPU:
+1 socket, 8 cores, 2 CPUs per core (SMT siblings), 2 NUMA nodes
+
+ {
+ # Keyed by CPU id. socket = physical package, core = "socket_coreid"
+ # (the key used to index C<cores> below), numa_node = host NUMA node.
+ #
+ # CPUs 8..15 are the SMT siblings of 0..7: same socket and
+ # core, therefore same NUMA node (e.g. 0 and 8 both on node 0).
+ cpus => {
+ 0 => { socket => 0, core => "0_0", numa_node => 0 },
+ 1 => { socket => 0, core => "0_1", numa_node => 0 },
+ 2 => { socket => 0, core => "0_2", numa_node => 0 },
+ 3 => { socket => 0, core => "0_3", numa_node => 0 },
+ 4 => { socket => 0, core => "0_4", numa_node => 1 },
+ 5 => { socket => 0, core => "0_5", numa_node => 1 },
+ 6 => { socket => 0, core => "0_6", numa_node => 1 },
+ 7 => { socket => 0, core => "0_7", numa_node => 1 },
+ 8 => { socket => 0, core => "0_0", numa_node => 0 },
+ 9 => { socket => 0, core => "0_1", numa_node => 0 },
+ 10 => { socket => 0, core => "0_2", numa_node => 0 },
+ 11 => { socket => 0, core => "0_3", numa_node => 0 },
+ 12 => { socket => 0, core => "0_4", numa_node => 1 },
+ 13 => { socket => 0, core => "0_5", numa_node => 1 },
+ 14 => { socket => 0, core => "0_6", numa_node => 1 },
+ 15 => { socket => 0, core => "0_7", numa_node => 1 },
+ },
+
+ # Keyed by core key ("socket_coreid").
+ cores => {
+ # CPUs 0 and 8 are SMT siblings
+ "0_0" => { numa_node => 0, cpus => [0, 8] },
+ "0_1" => { numa_node => 0, cpus => [1, 9] },
+ "0_2" => { numa_node => 0, cpus => [2, 10] },
+ # CPUs 3 and 11 are SMT siblings (see their entries above).
+ "0_3" => { numa_node => 0, cpus => [3, 11] },
+ "0_4" => { numa_node => 1, cpus => [4, 12] },
+ "0_5" => { numa_node => 1, cpus => [5, 13] },
+ "0_6" => { numa_node => 1, cpus => [6, 14] },
+ "0_7" => { numa_node => 1, cpus => [7, 15] },
+ },
+
+ # Keyed by NUMA node id. Each value holds two sets (id => 1): the
+ # core keys on that node, and the CPU ids on that node. The 1 values
+ # are not counts; only the keys are meaningful.
+ numa_nodes => {
+ 0 => {
+ cores => { "0_0" => 1, "0_1" => 1, "0_2" => 1, "0_3" => 1 },
+ cpus => { 0 => 1, 1 => 1, 2 => 1, 3 => 1,
+ 8 => 1, 9 => 1, 10 => 1, 11 => 1 },
+ },
+ 1 => {
+ cores => { "0_4" => 1, "0_5" => 1, "0_6" => 1, "0_7" => 1 },
+ cpus => { 4 => 1, 5 => 1, 6 => 1, 7 => 1,
+ 12 => 1, 13 => 1, 14 => 1, 15 => 1 },
+ },
+ },
+ }
+=cut
+sub get_host_topology() {
+ my $cpu_to_numa = cpu_to_numa_map();
+
+ my $topology = {
+ cpus => {},
+ cores => {},
+ numa_nodes => {},
+ };
+
+ my $cpu_dirs = cpu_topology_dirs();
+
+ for my $cpu_id (sort { $a <=> $b } keys $cpu_dirs->%*) {
+ my ($socket_id, $core_id) = read_cpu_topology($cpu_dirs->{$cpu_id});
+
+ my $numa_node = $cpu_to_numa->{$cpu_id} // 0;
+
+ my $cpu_info = make_cpu_info($socket_id, $core_id, $numa_node);
+ add_cpu_to_topology($topology, $cpu_id, $cpu_info);
+ }
+
+ return $topology;
+}
+
+my sub read_sysfs_trim($path) {
+ return undef if !-e $path;
+ my $val = file_read_firstline($path);
+ return undef if !defined($val);
+ $val =~ s/^\s+|\s+$//g;
+ return $val;
+}
+
+# in: /sys/devices/system/cpu/online = 0-7
+# out: [0, 1, 2, 3, 4, 5, 6, 7]
+my sub online_cpu_ids() {
+ my $content = read_sysfs_trim("/sys/devices/system/cpu/online");
+ return [] if !$content;
+
+ my @ids;
+ for my $part (split(/,/, $content)) {
+ if ($part =~ /^(\d+)-(\d+)$/) {
+ push @ids, ($1 .. $2);
+ } elsif ($part =~ /^(\d+)$/) {
+ push @ids, $1;
+ }
+ }
+ return \@ids;
+}
+
+# Build a stable "CPU type" key for a single online CPU.
+# On ARM: MIDR register + cache geometry.
+# On x86 (no MIDR): cache geometry alone (level/type/line/ways/sets).
+# On intel CPUs different P/E cores or big.LITTLE cores produce different keys because
+# they have different cache geometries even when /proc/cpuinfo model names
+# are identical or absent.
+# Note: Only tested on AMD x86 CPUs, so take this with a grain of salt.
+my sub cpu_type_key($cpu_id) {
+ my $cpu_path = "/sys/devices/system/cpu/cpu$cpu_id";
+
+ my $key = '';
+
+ # MIDR (ARM only). Absent on x86.
+ my $midr = read_sysfs_trim("$cpu_path/regs/identification/midr_el1");
+ $key .= defined($midr) ? $midr : 'no-midr';
+
+ # Cache geometry: sorted cache indices, each with level/type/line/ways/sets.
+ # This distinguishes P-cores from E-cores on x86 (different L2/L3 sizes)
+ # and provides a secondary signal on ARM.
+ my $cache_base = "$cpu_path/cache";
+ if (-d $cache_base) {
+ opendir(my $dh, $cache_base) or return $key;
+ my @indices = sort { $a <=> $b }
+ map { /^index(\d+)$/ ? $1 : () }
+ readdir($dh);
+ closedir($dh);
+
+ for my $idx (@indices) {
+ my $idx_path = "$cache_base/index$idx";
+ for my $field (qw(level type coherency_line_size ways_of_associativity number_of_sets)) {
+ my $val = read_sysfs_trim("$idx_path/$field");
+ $key .= '/' . (defined($val) ? $val : '');
+ }
+ }
+ }
+
+ return $key;
+}
+
+# Group online CPUs by cpu_type_key. capacity defaults to 1024/CPU.
+# out: ({ "abc/..." => {cpus => {0, 1, 2, 3}, capacity => 4096} },
+# [ "abc/..." ])
+sub group_cpus_by_type() {
+ my $online_ids = online_cpu_ids();
+ return ({}, []) if !$online_ids || !@$online_ids;
+
+ my %groups;
+ my @keys;
+
+ for my $cpu_id (@$online_ids) {
+ my $key = cpu_type_key($cpu_id);
+ if (!$groups{$key}) {
+ $groups{$key} = { cpus => [], capacity => 0 };
+ push @keys, $key;
+ }
+ push $groups{$key}{cpus}->@*, $cpu_id;
+
+ my $cap = 1024;
+ my $cap_val = read_sysfs_trim("/sys/devices/system/cpu/cpu$cpu_id/cpu_capacity");
+ if (defined($cap_val) && $cap_val =~ /^(\d+)$/) {
+ $cap = int($1);
+ }
+ $groups{$key}{capacity} += $cap;
+ }
+
+ # Deterministic key order: by first CPU id in each group.
+ @keys = sort { $groups{$a}{cpus}[0] <=> $groups{$b}{cpus}[0] } @keys;
+
+ return (\%groups, \@keys);
+}
+
+sub are_cpus_heterogenous($cpu_ids_arrayref) {
+ my ($groups, $keys) = group_cpus_by_type();
+ return 0 if !@$keys || @$keys <= 1;
+
+ my %requested = map { $_ => 1 } @$cpu_ids_arrayref;
+ my $seen_key;
+
+ for my $key (@$keys) {
+ for my $cpu_id (@{$groups->{$key}{cpus}}) {
+ next if !$requested{$cpu_id};
+ if (!defined $seen_key) {
+ $seen_key = $key;
+ } elsif ($key ne $seen_key) {
+ return 1;
+ }
+ last;
+ }
+ }
+ return 0;
+}
+
+sub filter_by_affinity($topology, $affinity_string) {
+ return $topology if !defined($affinity_string) || $affinity_string eq '';
+
+ my ($_count, $affinity_members) = PVE::CpuSet::parse_cpuset($affinity_string);
+
+ my $filtered = {
+ cpus => {},
+ cores => {},
+ numa_nodes => {},
+ };
+
+ for my $cpu_id (sort { $a <=> $b } keys $topology->{cpus}->%*) {
+ next if !$affinity_members->{$cpu_id};
+ my $cpu_info = $topology->{cpus}->{$cpu_id};
+ add_cpu_to_topology($filtered, $cpu_id, $cpu_info);
+ }
+
+ return $filtered;
+}
+
+# Group selected host CPUs by socket -> core -> threads.
+# in: $topology, [0, 1, 4, 5]
+# out: ({ 0 => { "0_0" => [0, 4], "0_1" => [1, 5] } }, undef)
+# in: $topology, [0, 99]
+# out: (undef, "CPU 99 not found in host topology")
+my sub group_cpus_by_socket_core($topology, $selected_cpus) {
+ my %sockets;
+ for my $cpu_id (@$selected_cpus) {
+ my $info = $topology->{cpus}->{$cpu_id};
+ return (undef, "CPU $cpu_id not found in host topology") if !$info;
+ push @{$sockets{$info->{socket}}{$info->{core}}}, $cpu_id;
+ }
+ return (\%sockets, undef);
+}
+
+# Check every socket has same core count and every core same thread count.
+# in: { 0 => { "0_0" => [0, 8], "0_4" => [4, 12] } }
+# out: (2, 2)
+# in: { 0 => { "0_0" => [0, 8] }, 1 => { "1_0" => [1, 9], "1_1" => [2, 10] } }
+# out: (undef, undef)
+my sub validate_consistency($sockets) {
+ my ($expected_cores, $expected_threads) = (-1, -1);
+
+ for my $socket (keys %$sockets) {
+ my $core_count = scalar keys %{$sockets->{$socket}};
+ return (undef, undef)
+ if $expected_cores != -1 && $core_count != $expected_cores;
+ $expected_cores = $core_count;
+
+ for my $core (keys %{$sockets->{$socket}}) {
+ my $thread_count = scalar @{$sockets->{$socket}->{$core}};
+ return (undef, undef)
+ if $expected_threads != -1 && $thread_count != $expected_threads;
+ $expected_threads = $thread_count;
+ }
+ }
+
+ return ($expected_cores, $expected_threads);
+}
+
+my sub flat_fallback_result($total, $warning = undef) {
+ $warning //= "Instance uses a CPU pinning profile which doesn't match"
+ . " hardware layout; falling back to flat topology"
+ . " (sockets=1, cores=$total, threads=1)";
+ return {
+ valid => 0,
+ sockets => 1,
+ cores => $total,
+ threads => 1,
+ warning => $warning,
+ };
+}
+
+=encoding utf8
+
+=head2 validate_pinned_topology
+
+Validates that the host CPUs in C<$selected_cpus> (an arrayref of host CPU
+IDs) form a balanced topology matching the guest configuration.
+
+=over 4
+
+=item 1. Groups selected host CPUs by physical socket and physical core.
+
+=item 2. Checks that every socket contains the same number of cores.
+
+=item 3. Checks that every core contains the same number of threads.
+
+=item 4. Checks that C<sockets * cores_per_socket * threads_per_core> equals
+the total number of selected CPUs.
+
+=back
+
+Returns a hashref:
+
+ {
+ valid => 1 | 0,
+ sockets => $nr_sockets,
+ cores => $nr_cores_per_socket,
+ threads => $nr_threads_per_core,
+ warning => $msg | undef,
+ }
+
+If the topology is invalid, the result contains a flat fallback
+(C<sockets=1, cores=N, threads=1>) and a warning string.
+
+=cut
+sub validate_pinned_topology($topology, $selected_cpus, $guest_sockets, $guest_cores) {
+ my $total = scalar @$selected_cpus;
+
+ # Empty selection: trust the guest-configured topology.
+ return {
+ valid => 1,
+ sockets => $guest_sockets,
+ cores => $guest_cores,
+ threads => 1,
+ warning => undef,
+ } if $total == 0;
+
+ my ($sockets, $err) = group_cpus_by_socket_core($topology, $selected_cpus);
+ return flat_fallback_result($total, $err) if defined($err);
+
+ my ($expected_cores, $expected_threads) = validate_consistency($sockets);
+ return flat_fallback_result($total)
+ if !defined $expected_cores || !defined $expected_threads;
+
+ my $nr_sockets = scalar keys %$sockets;
+ return flat_fallback_result($total)
+ if $nr_sockets * $expected_cores * $expected_threads != $total;
+
+ return {
+ valid => 1,
+ sockets => $nr_sockets,
+ cores => $expected_cores,
+ threads => $expected_threads,
+ warning => undef,
+ };
+}
+
+1;
--
2.47.3
next prev parent reply other threads:[~2026-09-21 9:54 UTC|newest]
Thread overview: 5+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-09-21 9:53 [RFC qemu-server v2 0/4] fix #7282: allow (NUMA aware) vCPU pinning Elias Huhsovitz
2026-09-21 9:54 ` Elias Huhsovitz [this message]
2026-09-21 9:54 ` [RFC qemu-server v2 2/4] pinning: add NUMA allocator and reservation tracking Elias Huhsovitz
2026-09-21 9:54 ` [RFC qemu-server v2 3/4] memory: integrate pinning-aware NUMA memory binding Elias Huhsovitz
2026-09-21 9:54 ` [RFC qemu-server v2 4/4] pinning: integrate cpu pinning into vm lifecycle Elias Huhsovitz
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20260921095404.61552-2-e.huhsovitz@proxmox.com \
--to=e.huhsovitz@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.