From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id A1A971FF09C for ; Mon, 21 Sep 2026 11:54:50 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 430E1215F8; Mon, 21 Sep 2026 11:54:33 +0200 (CEST) From: Elias Huhsovitz To: pve-devel@lists.proxmox.com Subject: [RFC qemu-server v2 1/4] pinning: add topology discovery and config parsing Date: Mon, 21 Sep 2026 11:54:00 +0200 Message-ID: <20260921095404.61552-2-e.huhsovitz@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20260921095404.61552-1-e.huhsovitz@proxmox.com> References: <20260921095404.61552-1-e.huhsovitz@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1789984465350 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.504 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SCC_5_SHORT_WORD_LINES 0.25 5 lines with many short words SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: FEGQ7HQSIQW5RSXSJ52IGE3ZYSOWQ6ZQ X-Message-ID-Hash: FEGQ7HQSIQW5RSXSJ52IGE3ZYSOWQ6ZQ X-MailFrom: e.huhsovitz@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header CC: Elias Huhsovitz X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Introduce Topology.pm and Config.pm to parse host CPU layouts from sysfs and handle guest pinning configuration. The config module parses 3 new modes that buid the mapping between guest vCPUs and virtual NUMA nodes: balanced: Automatically selects the host NUMA node(s) with the most uncommitted memory, using as many as needed to fit the VM's CPU and memory requirements. 1. numa: Pins the sets of vCPUs of each virtual NUMA node to a corresponding host NUMA node. 2. one-to-one: Pins each vCPU to a specific host core. This prevents vCPUs from being rescheduled on other cores entirely. 3. The topology module reads physical packages, cores, and SMT siblings, and detects heterogeneous CPUs (e.g., Intel P/E cores, ARM big.LITTLE). Originally-by: Dominik Csapak Signed-off-by: Elias Huhsovitz --- src/PVE/QemuServer/Pinning/Config.pm | 244 +++++++++++++ src/PVE/QemuServer/Pinning/Topology.pm | 462 +++++++++++++++++++++++++ 2 files changed, 706 insertions(+) create mode 100644 src/PVE/QemuServer/Pinning/Config.pm create mode 100644 src/PVE/QemuServer/Pinning/Topology.pm diff --git a/src/PVE/QemuServer/Pinning/Config.pm b/src/PVE/QemuServer/Pinning/Config.pm new file mode 100644 index 00000000..e71fdd84 --- /dev/null +++ b/src/PVE/QemuServer/Pinning/Config.pm @@ -0,0 +1,244 @@ +package PVE::QemuServer::Pinning::Config; + +use v5.36; + +use PVE::JSONSchema qw(parse_property_string); +use PVE::QemuServer::Helpers; +use PVE::QemuServer::Memory; + +use constant DEFAULT_VCPU_PINNING => 'none'; + +our $pinning_fmt = { + mode => { + type => 'string', + enum => ['balanced', 'one-to-one', 'numa', DEFAULT_VCPU_PINNING], + description => 'Set the vCPU pinning mode.', + verbose_description => <<'EODESCR', +There are multiple ways to pin vCPUs to host cores: + +* none (default): Applies no pinning. + +* balanced: Automatically selects the host NUMA node(s) with the most + uncommitted memory, using as many as needed to fit the VM's CPU and memory + requirements. Binds the VM's memory to the selected host node(s) with + policy=bind. The QEMU process is confined to the selected host nodes CPUs, + but individual vCPU threads are NOT pinned. The host scheduler may move + them freely within the selected nodes. Requires numa: 1 to be enabled. + Use 'hostnodes' to restrict which host nodes are considered. + +* numa: Pins the sets of vCPUs of each virtual NUMA node to a corresponding + host NUMA node. This makes memory access consistent for each vCPU because + the scheduler will not move it to a different NUMA node. It considers the + 'numaX' setting when binding vCPUs to host nodes. + Requires numa: 1 to be enabled. + +* one-to-one: Pins each vCPU to a specific host core. This prevents vCPUs + from being rescheduled on other cores entirely. It provides the highest + performance but is the least flexible for the host scheduler. It considers + the virtual and physical NUMA layout and avoids SMT collisions by spreading + vCPUs across distinct physical cores whenever possible. When used with + numa: 1, memory is also bound to host NUMA nodes. When used without numa: 1, + only CPU pinning is performed, memory is not bound to specific host nodes. + +All settings are affected by 'affinity', to only consider those cores. + +These options do not consider pinning settings from other virtual machines or +containers. To achieve the best and most consistent performance, use the +combination of 'numaX' and 'affinity' options to ensure host cores are not +crowded with vCPU assignments. +EODESCR + default => DEFAULT_VCPU_PINNING, + default_key => 1, + }, + hostnodes => { + type => 'string', + pattern => qr/\d+(?:-\d+)?(?:;\d+(?:-\d+)?)*/, + description => 'Restrict host NUMA nodes the VM may use for pinning.', + verbose_description => <<'EODESCR', +Semicolon-separated list of host NUMA node IDs (or ranges) that the VM is +allowed to use for vCPU pinning and memory allocation. When set, the +pinning allocator only considers these host nodes as candidates for +auto-assignment. + +If not set, all host NUMA nodes are considered. + +Example: pinning=balanced,hostnodes=0;1 restricts the VM to host NUMA +nodes 0 and 1 only. +EODESCR + format_description => 'id[-id];...', + optional => 1, + }, +}; + +PVE::JSONSchema::register_format('pve-qm-cpu-pinning', $pinning_fmt); + +our $pinning_desc = { + optional => 1, + type => 'string', + format => 'pve-qm-cpu-pinning', + description => 'CPU pinning settings.', +}; + +PVE::JSONSchema::register_standard_option('pve-qm-cpu-pinning', $pinning_desc); + +# Parses a semicolon-separated list of host NUMA node IDs/ranges, +# into a sorted arrayref of node ID +# in: 3-5;0;1 out: [0,1,3,4,5] +my sub parse_host_node_list($id_ranges) { + return undef if !defined($id_ranges) || $id_ranges eq ''; + + my $ranges = PVE::QemuServer::Helpers::parse_number_sets($id_ranges); + my @ids; + for my $range ($ranges->@*) { + my ($start, $end) = $range->@*; + $end //= $start; + push @ids, ($start .. $end); + } + @ids = sort { $a <=> $b } @ids; + return @ids ? \@ids : undef; +} + +# in: "balanced,hostnodes=0;1"; +# out: {mode => "balanced", hostnodes => [0,1]} +sub parse_pinning($setting_str) { + return { mode => DEFAULT_VCPU_PINNING } if !defined($setting_str); + + my $settings_map = parse_property_string($pinning_fmt, $setting_str); + $settings_map->{mode} = DEFAULT_VCPU_PINNING if !defined($settings_map->{mode}); + $settings_map->{hostnodes} = parse_host_node_list($settings_map->{hostnodes}) if (defined($settings_map->{hostnodes})); + + return $settings_map; +} + +# convert NUMA node range list to vCPU set +# in: {cpus => [ [0, 1], [3]]} +# out: {0, 1, 3} +my sub expand_numa_vcpus($numa_map) { + my %vcpus; + for my $range ($numa_map->{cpus}->@*) { + my ($start, $end) = $range->@*; + $vcpus{$_} = 1 for $start .. $end // $start; + } + return \%vcpus; +} + +# in: {hostnodes => [[1, 1]]}; +# out: 1 +my sub extract_numa_hostnode($numa_map) { + my $hostnodes = $numa_map->{hostnodes}; + return undef if !defined($hostnodes); + + die "Pinning only available for 1-to-1 NUMA node mapping\n" + if $hostnodes->@* > 1 || defined($hostnodes->[0]->[1]); + + return $hostnodes->[0]->[0]; +} + +# in: +# { +# numa => 1, +# numa0 => "cpus=0-1,hostnodes=0", +# numa1 => "cpus=2-3,hostnodes=1", +# } +# out: +# { +# 0 => {vcpus => {0, 1} hostnode => 0}, +# 1 => {vcpus => {2, 3}, hostnode => 1,} +# } +my sub parse_numa_vcpu_map($conf) { + my $map = {}; + + for my $i (0 .. $PVE::QemuServer::Memory::MAX_NUMA - 1) { + my $entry = $conf->{"numa$i"} or next; + my $numa = PVE::QemuServer::Memory::parse_numa($entry) or next; + + $map->{$i} = { + vcpus => expand_numa_vcpus($numa), + hostnode => extract_numa_hostnode($numa) + }; + } + + return $map; +} + +# Parse cpu settings to map keyed by numa node. +# in: ($sockets, $cores, $threads_per_core) = (2, 2, 1); +# out: { 0 => {vcpus => {0, 1}}, 1 => {vcpus => {2, 3}} } +my sub build_socket_vcpu_map($sockets, $cores, $threads_per_core) { + my $map = {}; + my $vcpu_id = 0; + + for my $socket (0 .. $sockets - 1) { + my %vcpus; + for (1 .. ($cores * $threads_per_core)) { + $vcpus{ $vcpu_id++ } = 1; + } + $map->{$socket} = { vcpus => \%vcpus }; + } + + return $map; +} + +# in: ($sockets, $cores, $threads_per_core) = (2, 2, 1); +# out: { 0 => {vcpus => {0, 1, 2, 3}} } +my sub build_flat_vcpu_map($sockets, $cores, $threads_per_core) { + my $total = $sockets * $cores * $threads_per_core; + return { + 0 => { vcpus => { map { $_ => 1 } 0 .. $total - 1 } }, + }; +} + +# Parse qm.conf cpu settings to map. +# The map is keyed by numa-node +# in: { +# sockets => 1, +# cores => 4, +# vcpus => 4, +# numa => 1, +# numa0 => "cpus=0-1,hostnodes=0", +# numa1 => "cpus=2-3,hostnodes=1", +# } +# out: { +# 0 => {vcpus => {0, 1}, hostnode => 0}, +# 1 => {vcpus => {2, 3}, hostnode => 1}, +# } +sub build_vnuma_vcpu_map($conf) { + my $sockets = $conf->{sockets} // 1; + my $cores = $conf->{cores} // 1; + # Fix 4.4: honour $conf->{vcpus} when it exceeds sockets*cores (SMT/hotplug). + my $vcpus = $conf->{vcpus} // ($sockets * $cores); + my $threads_per_core = $vcpus > ($sockets * $cores) + ? int($vcpus / ($sockets * $cores)) + : 1; + + my $map; + if ($conf->{numa}) { + $map = parse_numa_vcpu_map($conf); + $map = build_socket_vcpu_map($sockets, $cores, $threads_per_core) + if !keys $map->%*; + } else { + $map = build_flat_vcpu_map($sockets, $cores, $threads_per_core); + } + + my $assigned = 0; + $assigned += scalar keys $_->{vcpus}->%* for values $map->%*; + + my $expected = $sockets * $cores * $threads_per_core; + die "Invalid NUMA configuration for pinning, some vCPUs missing in numa" + . " binding (assigned=$assigned, expected=$expected)\n" + if $assigned != $expected; + + return $map; +} + +# in: $conf = { sockets => 2, cores => 4, vcpus => 6 }; +# out: (2, 4, 8, 6) +sub get_config_topology($conf) { + my $sockets = $conf->{sockets} || $conf->{smp} || 1; + my $cores = $conf->{cores} || 1; + my $maxcpus = $sockets * $cores; + my $vcpus = $conf->{vcpus} || $maxcpus; + return ($sockets, $cores, $maxcpus, $vcpus); +} + +1; diff --git a/src/PVE/QemuServer/Pinning/Topology.pm b/src/PVE/QemuServer/Pinning/Topology.pm new file mode 100644 index 00000000..daddea0b --- /dev/null +++ b/src/PVE/QemuServer/Pinning/Topology.pm @@ -0,0 +1,462 @@ +package PVE::QemuServer::Pinning::Topology; + +use v5.36; + +use PVE::CpuSet; +use PVE::Tools qw(file_read_firstline dir_glob_foreach); + +# Build { cpu_id => numa_node_id } from /sys/devices/system/node. +# key: cpu, val: numa-node +# e.g. reads: 8 core CPU with 2 NUMA nodes +# out: { 0 => 0, 1 => 0, 2 => 0, 3 => 0, 4 => 1, 5 => 1, 6 => 1, 7 => 1 } +my sub cpu_to_numa_map() { + my $cpu_to_numa = {}; + my $numa_path = "/sys/devices/system/node"; + + return $cpu_to_numa if !-d $numa_path; + + dir_glob_foreach($numa_path, 'node(\d+)', sub ($node_dir, $node_id) { + dir_glob_foreach("$numa_path/$node_dir", 'cpu(\d+)', sub ($cpu_dir, $cpu_id) { + $cpu_to_numa->{$cpu_id} = $node_id; + }); + }); + + return $cpu_to_numa; +} + +# Build { cpu_id => topology_dir_path } for all CPUs under /sys. +# in: (no args, reads sysfs) +# out: { 0 => "/sys/devices/system/cpu/cpu0/topology", +# 1 => "/sys/devices/system/cpu/cpu1/topology", ... } +my sub cpu_topology_dirs() { + my $base_path = "/sys/devices/system/cpu"; + opendir(my $dirfd, $base_path) || die "cannot open $base_path\n"; + + my $dirs = {}; + for my $entry (readdir($dirfd)) { + next if $entry !~ m/^cpu(\d+)$/; + my $topo_path = "$base_path/$entry/topology"; + next if !-d $topo_path; + $dirs->{$1} = $topo_path; + } + closedir($dirfd); + + return $dirs; +} + +# Read physical_package_id and core_id, normalizing sentinel values to 0. +# in: "/sys/devices/system/cpu/cpu1/topology" +# out: (0, 1) +# core_id seems to be set by the vendor and non-linear. +# e.g. for the AMD EPYC 7351P 16-Core Processor cpu 4 +# and its thread sibling cpu 16 reside on core 8 +# e.g. /sys/devices/system/cpu/cpu4/topology/ +# out: (0, 8) +my sub read_cpu_topology($topo_path) { + my $socket_id = file_read_firstline("$topo_path/physical_package_id") // 0; + my $core_id = file_read_firstline("$topo_path/core_id") // 0; + + # missing topology info is represented as sential value (e.g. nested virtualization) + $socket_id = 0 if $socket_id < 0 || $socket_id == 65535; + $core_id = 0 if $core_id < 0 || $core_id == 65535; + + return ($socket_id, $core_id); +} + +my sub make_cpu_info($socket_id, $core_id, $numa_node) { + return { + socket => $socket_id, + core => "${socket_id}_${core_id}", + numa_node => $numa_node, + }; +} + +# Add one CPU into topology (mutates cpus/cores/numa_nodes). +# in: {cpus=>{},cores=>{},numa_nodes=>{}}, 3, {socket=>0,core=>"0_3",numa_node=>1} +# out: (mutated topology) +# cpus => {3 => {socket=>0, core=>"0_3", numa_node=>1}} +# cores => {"0_3" => {numa_node=>1, cpus=>[3]}} +# numa_nodes => {1 => {cores=>{"0_3"}, cpus=>{3}}} +my sub add_cpu_to_topology($topology, $cpu_id, $cpu_info) { + $topology->{cpus}->{$cpu_id} = $cpu_info; + + my $core_key = $cpu_info->{core}; + my $numa_node = $cpu_info->{numa_node}; + + $topology->{cores}->{$core_key} //= { + numa_node => $numa_node, + cpus => [], + }; + + push $topology->{cores}->{$core_key}->{cpus}->@*, $cpu_id; + + $topology->{numa_nodes}->{$numa_node} //= { + cores => {}, + cpus => {}, + }; + $topology->{numa_nodes}->{$numa_node}->{cores}->{$core_key} = 1; + $topology->{numa_nodes}->{$numa_node}->{cpus}->{$cpu_id} = 1; +} + +=head2 get_host_topology + +Reads the host CPU topology from sysfs and returns it as a hashref with +three keys: + +=over + +=item * C - C<< { $cpu_id => { socket, core, numa_node } } >>, one +entry per CPU. C is a C<"${socket}_${core_id}"> string. + +=item * C - C<< { $core_key => { numa_node, cpus } } >>, where +C is an arrayref of the CPU ids sharing that physical core. + +=item * C - C<< { $node_id => { cores, cpus } } >>, where +C and C are sets (hashrefs of id => 1). + +=back + +Missing or unavailable topology information is normalized to socket and +core C<0>, and CPUs without an entry under C are +placed on NUMA node C<0>. B if C cannot be +opened. + +B - x86 CPU: +1 socket, 8 cores, 2 CPUs per core (SMT siblings), 2 NUMA nodes + + { + # Keyed by CPU id. socket = physical package, core = "socket_coreid" + # (the key used to index C below), numa_node = host NUMA node. + # + # CPUs 8..15 are the SMT siblings of 0..7: same socket and + # core, therefore same NUMA node (e.g. 0 and 8 both on node 0). + cpus => { + 0 => { socket => 0, core => "0_0", numa_node => 0 }, + 1 => { socket => 0, core => "0_1", numa_node => 0 }, + 2 => { socket => 0, core => "0_2", numa_node => 0 }, + 3 => { socket => 0, core => "0_3", numa_node => 0 }, + 4 => { socket => 0, core => "0_4", numa_node => 1 }, + 5 => { socket => 0, core => "0_5", numa_node => 1 }, + 6 => { socket => 0, core => "0_6", numa_node => 1 }, + 7 => { socket => 0, core => "0_7", numa_node => 1 }, + 8 => { socket => 0, core => "0_0", numa_node => 0 }, + 9 => { socket => 0, core => "0_1", numa_node => 0 }, + 10 => { socket => 0, core => "0_2", numa_node => 0 }, + 11 => { socket => 0, core => "0_3", numa_node => 0 }, + 12 => { socket => 0, core => "0_4", numa_node => 1 }, + 13 => { socket => 0, core => "0_5", numa_node => 1 }, + 14 => { socket => 0, core => "0_6", numa_node => 1 }, + 15 => { socket => 0, core => "0_7", numa_node => 1 }, + }, + + # Keyed by core key ("socket_coreid"). + cores => { + # CPUs 0 and 8 are SMT siblings + "0_0" => { numa_node => 0, cpus => [0, 8] }, + "0_1" => { numa_node => 0, cpus => [1, 9] }, + "0_2" => { numa_node => 0, cpus => [2, 10] }, + # CPUs 3 and 11 are SMT siblings (see their entries above). + "0_3" => { numa_node => 0, cpus => [3, 11] }, + "0_4" => { numa_node => 1, cpus => [4, 12] }, + "0_5" => { numa_node => 1, cpus => [5, 13] }, + "0_6" => { numa_node => 1, cpus => [6, 14] }, + "0_7" => { numa_node => 1, cpus => [7, 15] }, + }, + + # Keyed by NUMA node id. Each value holds two sets (id => 1): the + # core keys on that node, and the CPU ids on that node. The 1 values + # are not counts; only the keys are meaningful. + numa_nodes => { + 0 => { + cores => { "0_0" => 1, "0_1" => 1, "0_2" => 1, "0_3" => 1 }, + cpus => { 0 => 1, 1 => 1, 2 => 1, 3 => 1, + 8 => 1, 9 => 1, 10 => 1, 11 => 1 }, + }, + 1 => { + cores => { "0_4" => 1, "0_5" => 1, "0_6" => 1, "0_7" => 1 }, + cpus => { 4 => 1, 5 => 1, 6 => 1, 7 => 1, + 12 => 1, 13 => 1, 14 => 1, 15 => 1 }, + }, + }, + } +=cut +sub get_host_topology() { + my $cpu_to_numa = cpu_to_numa_map(); + + my $topology = { + cpus => {}, + cores => {}, + numa_nodes => {}, + }; + + my $cpu_dirs = cpu_topology_dirs(); + + for my $cpu_id (sort { $a <=> $b } keys $cpu_dirs->%*) { + my ($socket_id, $core_id) = read_cpu_topology($cpu_dirs->{$cpu_id}); + + my $numa_node = $cpu_to_numa->{$cpu_id} // 0; + + my $cpu_info = make_cpu_info($socket_id, $core_id, $numa_node); + add_cpu_to_topology($topology, $cpu_id, $cpu_info); + } + + return $topology; +} + +my sub read_sysfs_trim($path) { + return undef if !-e $path; + my $val = file_read_firstline($path); + return undef if !defined($val); + $val =~ s/^\s+|\s+$//g; + return $val; +} + +# in: /sys/devices/system/cpu/online = 0-7 +# out: [0, 1, 2, 3, 4, 5, 6, 7] +my sub online_cpu_ids() { + my $content = read_sysfs_trim("/sys/devices/system/cpu/online"); + return [] if !$content; + + my @ids; + for my $part (split(/,/, $content)) { + if ($part =~ /^(\d+)-(\d+)$/) { + push @ids, ($1 .. $2); + } elsif ($part =~ /^(\d+)$/) { + push @ids, $1; + } + } + return \@ids; +} + +# Build a stable "CPU type" key for a single online CPU. +# On ARM: MIDR register + cache geometry. +# On x86 (no MIDR): cache geometry alone (level/type/line/ways/sets). +# On intel CPUs different P/E cores or big.LITTLE cores produce different keys because +# they have different cache geometries even when /proc/cpuinfo model names +# are identical or absent. +# Note: Only tested on AMD x86 CPUs, so take this with a grain of salt. +my sub cpu_type_key($cpu_id) { + my $cpu_path = "/sys/devices/system/cpu/cpu$cpu_id"; + + my $key = ''; + + # MIDR (ARM only). Absent on x86. + my $midr = read_sysfs_trim("$cpu_path/regs/identification/midr_el1"); + $key .= defined($midr) ? $midr : 'no-midr'; + + # Cache geometry: sorted cache indices, each with level/type/line/ways/sets. + # This distinguishes P-cores from E-cores on x86 (different L2/L3 sizes) + # and provides a secondary signal on ARM. + my $cache_base = "$cpu_path/cache"; + if (-d $cache_base) { + opendir(my $dh, $cache_base) or return $key; + my @indices = sort { $a <=> $b } + map { /^index(\d+)$/ ? $1 : () } + readdir($dh); + closedir($dh); + + for my $idx (@indices) { + my $idx_path = "$cache_base/index$idx"; + for my $field (qw(level type coherency_line_size ways_of_associativity number_of_sets)) { + my $val = read_sysfs_trim("$idx_path/$field"); + $key .= '/' . (defined($val) ? $val : ''); + } + } + } + + return $key; +} + +# Group online CPUs by cpu_type_key. capacity defaults to 1024/CPU. +# out: ({ "abc/..." => {cpus => {0, 1, 2, 3}, capacity => 4096} }, +# [ "abc/..." ]) +sub group_cpus_by_type() { + my $online_ids = online_cpu_ids(); + return ({}, []) if !$online_ids || !@$online_ids; + + my %groups; + my @keys; + + for my $cpu_id (@$online_ids) { + my $key = cpu_type_key($cpu_id); + if (!$groups{$key}) { + $groups{$key} = { cpus => [], capacity => 0 }; + push @keys, $key; + } + push $groups{$key}{cpus}->@*, $cpu_id; + + my $cap = 1024; + my $cap_val = read_sysfs_trim("/sys/devices/system/cpu/cpu$cpu_id/cpu_capacity"); + if (defined($cap_val) && $cap_val =~ /^(\d+)$/) { + $cap = int($1); + } + $groups{$key}{capacity} += $cap; + } + + # Deterministic key order: by first CPU id in each group. + @keys = sort { $groups{$a}{cpus}[0] <=> $groups{$b}{cpus}[0] } @keys; + + return (\%groups, \@keys); +} + +sub are_cpus_heterogenous($cpu_ids_arrayref) { + my ($groups, $keys) = group_cpus_by_type(); + return 0 if !@$keys || @$keys <= 1; + + my %requested = map { $_ => 1 } @$cpu_ids_arrayref; + my $seen_key; + + for my $key (@$keys) { + for my $cpu_id (@{$groups->{$key}{cpus}}) { + next if !$requested{$cpu_id}; + if (!defined $seen_key) { + $seen_key = $key; + } elsif ($key ne $seen_key) { + return 1; + } + last; + } + } + return 0; +} + +sub filter_by_affinity($topology, $affinity_string) { + return $topology if !defined($affinity_string) || $affinity_string eq ''; + + my ($_count, $affinity_members) = PVE::CpuSet::parse_cpuset($affinity_string); + + my $filtered = { + cpus => {}, + cores => {}, + numa_nodes => {}, + }; + + for my $cpu_id (sort { $a <=> $b } keys $topology->{cpus}->%*) { + next if !$affinity_members->{$cpu_id}; + my $cpu_info = $topology->{cpus}->{$cpu_id}; + add_cpu_to_topology($filtered, $cpu_id, $cpu_info); + } + + return $filtered; +} + +# Group selected host CPUs by socket -> core -> threads. +# in: $topology, [0, 1, 4, 5] +# out: ({ 0 => { "0_0" => [0, 4], "0_1" => [1, 5] } }, undef) +# in: $topology, [0, 99] +# out: (undef, "CPU 99 not found in host topology") +my sub group_cpus_by_socket_core($topology, $selected_cpus) { + my %sockets; + for my $cpu_id (@$selected_cpus) { + my $info = $topology->{cpus}->{$cpu_id}; + return (undef, "CPU $cpu_id not found in host topology") if !$info; + push @{$sockets{$info->{socket}}{$info->{core}}}, $cpu_id; + } + return (\%sockets, undef); +} + +# Check every socket has same core count and every core same thread count. +# in: { 0 => { "0_0" => [0, 8], "0_4" => [4, 12] } } +# out: (2, 2) +# in: { 0 => { "0_0" => [0, 8] }, 1 => { "1_0" => [1, 9], "1_1" => [2, 10] } } +# out: (undef, undef) +my sub validate_consistency($sockets) { + my ($expected_cores, $expected_threads) = (-1, -1); + + for my $socket (keys %$sockets) { + my $core_count = scalar keys %{$sockets->{$socket}}; + return (undef, undef) + if $expected_cores != -1 && $core_count != $expected_cores; + $expected_cores = $core_count; + + for my $core (keys %{$sockets->{$socket}}) { + my $thread_count = scalar @{$sockets->{$socket}->{$core}}; + return (undef, undef) + if $expected_threads != -1 && $thread_count != $expected_threads; + $expected_threads = $thread_count; + } + } + + return ($expected_cores, $expected_threads); +} + +my sub flat_fallback_result($total, $warning = undef) { + $warning //= "Instance uses a CPU pinning profile which doesn't match" + . " hardware layout; falling back to flat topology" + . " (sockets=1, cores=$total, threads=1)"; + return { + valid => 0, + sockets => 1, + cores => $total, + threads => 1, + warning => $warning, + }; +} + +=encoding utf8 + +=head2 validate_pinned_topology + +Validates that the host CPUs in C<$selected_cpus> (an arrayref of host CPU +IDs) form a balanced topology matching the guest configuration. + +=over 4 + +=item 1. Groups selected host CPUs by physical socket and physical core. + +=item 2. Checks that every socket contains the same number of cores. + +=item 3. Checks that every core contains the same number of threads. + +=item 4. Checks that C equals +the total number of selected CPUs. + +=back + +Returns a hashref: + + { + valid => 1 | 0, + sockets => $nr_sockets, + cores => $nr_cores_per_socket, + threads => $nr_threads_per_core, + warning => $msg | undef, + } + +If the topology is invalid, the result contains a flat fallback +(C) and a warning string. + +=cut +sub validate_pinned_topology($topology, $selected_cpus, $guest_sockets, $guest_cores) { + my $total = scalar @$selected_cpus; + + # Empty selection: trust the guest-configured topology. + return { + valid => 1, + sockets => $guest_sockets, + cores => $guest_cores, + threads => 1, + warning => undef, + } if $total == 0; + + my ($sockets, $err) = group_cpus_by_socket_core($topology, $selected_cpus); + return flat_fallback_result($total, $err) if defined($err); + + my ($expected_cores, $expected_threads) = validate_consistency($sockets); + return flat_fallback_result($total) + if !defined $expected_cores || !defined $expected_threads; + + my $nr_sockets = scalar keys %$sockets; + return flat_fallback_result($total) + if $nr_sockets * $expected_cores * $expected_threads != $total; + + return { + valid => 1, + sockets => $nr_sockets, + cores => $expected_cores, + threads => $expected_threads, + warning => undef, + }; +} + +1; -- 2.47.3