public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Elias Huhsovitz <e.huhsovitz@proxmox.com>
To: pve-devel@lists.proxmox.com
Cc: Elias Huhsovitz <e.huhsovitz@proxmox.com>
Subject: [RFC qemu-server v2 1/4] pinning: add topology discovery and config parsing
Date: Mon, 21 Sep 2026 11:54:00 +0200	[thread overview]
Message-ID: <20260921095404.61552-2-e.huhsovitz@proxmox.com> (raw)
In-Reply-To: <20260921095404.61552-1-e.huhsovitz@proxmox.com>

Introduce Topology.pm and Config.pm to parse host CPU layouts from sysfs
and handle guest pinning configuration.

The config module parses 3 new modes that buid the mapping between guest
vCPUs and virtual NUMA nodes:
balanced: Automatically selects the host NUMA node(s) with the most
uncommitted memory, using as many as needed to fit the VM's CPU and
memory requirements.

1. numa: Pins the sets of vCPUs of each virtual NUMA node to a
corresponding host NUMA node.

2. one-to-one: Pins each vCPU to a specific host core. This prevents
vCPUs from being rescheduled on other cores entirely.

3. The topology module reads physical packages, cores, and SMT siblings,
and detects heterogeneous CPUs (e.g., Intel P/E cores, ARM big.LITTLE).

Originally-by: Dominik Csapak <d.csapak@proxmox.com>
Signed-off-by: Elias Huhsovitz <e.huhsovitz@proxmox.com>
---
 src/PVE/QemuServer/Pinning/Config.pm   | 244 +++++++++++++
 src/PVE/QemuServer/Pinning/Topology.pm | 462 +++++++++++++++++++++++++
 2 files changed, 706 insertions(+)
 create mode 100644 src/PVE/QemuServer/Pinning/Config.pm
 create mode 100644 src/PVE/QemuServer/Pinning/Topology.pm

diff --git a/src/PVE/QemuServer/Pinning/Config.pm b/src/PVE/QemuServer/Pinning/Config.pm
new file mode 100644
index 00000000..e71fdd84
--- /dev/null
+++ b/src/PVE/QemuServer/Pinning/Config.pm
@@ -0,0 +1,244 @@
+package PVE::QemuServer::Pinning::Config;
+
+use v5.36;
+
+use PVE::JSONSchema qw(parse_property_string);
+use PVE::QemuServer::Helpers;
+use PVE::QemuServer::Memory;
+
+use constant DEFAULT_VCPU_PINNING => 'none';
+
+our $pinning_fmt = {
+    mode => {
+        type => 'string',
+        enum => ['balanced', 'one-to-one', 'numa', DEFAULT_VCPU_PINNING],
+        description => 'Set the vCPU pinning mode.',
+        verbose_description => <<'EODESCR',
+There are multiple ways to pin vCPUs to host cores:
+
+* none (default): Applies no pinning.
+
+* balanced: Automatically selects the host NUMA node(s) with the most
+  uncommitted memory, using as many as needed to fit the VM's CPU and memory
+  requirements. Binds the VM's memory to the selected host node(s) with
+  policy=bind. The QEMU process is confined to the selected host nodes CPUs,
+  but individual vCPU threads are NOT pinned. The host scheduler may move
+  them freely within the selected nodes. Requires numa: 1 to be enabled.
+  Use 'hostnodes' to restrict which host nodes are considered.
+
+* numa: Pins the sets of vCPUs of each virtual NUMA node to a corresponding
+  host NUMA node. This makes memory access consistent for each vCPU because
+  the scheduler will not move it to a different NUMA node. It considers the
+  'numaX' setting when binding vCPUs to host nodes.
+  Requires numa: 1 to be enabled.
+
+* one-to-one: Pins each vCPU to a specific host core. This prevents vCPUs
+  from being rescheduled on other cores entirely. It provides the highest
+  performance but is the least flexible for the host scheduler. It considers
+  the virtual and physical NUMA layout and avoids SMT collisions by spreading
+  vCPUs across distinct physical cores whenever possible. When used with
+  numa: 1, memory is also bound to host NUMA nodes. When used without numa: 1,
+  only CPU pinning is performed, memory is not bound to specific host nodes.
+
+All settings are affected by 'affinity', to only consider those cores.
+
+These options do not consider pinning settings from other virtual machines or
+containers. To achieve the best and most consistent performance, use the
+combination of 'numaX' and 'affinity' options to ensure host cores are not
+crowded with vCPU assignments.
+EODESCR
+        default => DEFAULT_VCPU_PINNING,
+        default_key => 1,
+    },
+    hostnodes => {
+        type => 'string',
+        pattern => qr/\d+(?:-\d+)?(?:;\d+(?:-\d+)?)*/,
+        description => 'Restrict host NUMA nodes the VM may use for pinning.',
+        verbose_description => <<'EODESCR',
+Semicolon-separated list of host NUMA node IDs (or ranges) that the VM is
+allowed to use for vCPU pinning and memory allocation. When set, the
+pinning allocator only considers these host nodes as candidates for
+auto-assignment.
+
+If not set, all host NUMA nodes are considered.
+
+Example: pinning=balanced,hostnodes=0;1 restricts the VM to host NUMA
+nodes 0 and 1 only.
+EODESCR
+        format_description => 'id[-id];...',
+        optional => 1,
+    },
+};
+
+PVE::JSONSchema::register_format('pve-qm-cpu-pinning', $pinning_fmt);
+
+our $pinning_desc = {
+    optional => 1,
+    type => 'string',
+    format => 'pve-qm-cpu-pinning',
+    description => 'CPU pinning settings.',
+};
+
+PVE::JSONSchema::register_standard_option('pve-qm-cpu-pinning', $pinning_desc);
+
+# Parses a semicolon-separated list of host NUMA node IDs/ranges,
+# into a sorted arrayref of node ID
+# in: 3-5;0;1 out: [0,1,3,4,5]
+my sub parse_host_node_list($id_ranges) {
+    return undef if !defined($id_ranges) || $id_ranges eq '';
+
+    my $ranges = PVE::QemuServer::Helpers::parse_number_sets($id_ranges);
+    my @ids;
+    for my $range ($ranges->@*) {
+        my ($start, $end) = $range->@*;
+        $end //= $start;
+        push @ids, ($start .. $end);
+    }
+    @ids = sort { $a <=> $b } @ids;
+    return @ids ? \@ids : undef;
+}
+
+# in: "balanced,hostnodes=0;1";
+# out: {mode => "balanced", hostnodes => [0,1]}
+sub parse_pinning($setting_str) {
+    return { mode => DEFAULT_VCPU_PINNING } if !defined($setting_str);
+
+    my $settings_map = parse_property_string($pinning_fmt, $setting_str);
+    $settings_map->{mode} = DEFAULT_VCPU_PINNING if !defined($settings_map->{mode});
+    $settings_map->{hostnodes} = parse_host_node_list($settings_map->{hostnodes}) if (defined($settings_map->{hostnodes}));
+
+    return $settings_map;
+}
+
+# convert NUMA node range list to vCPU set
+# in: {cpus => [ [0, 1], [3]]}
+# out: {0, 1, 3}
+my sub expand_numa_vcpus($numa_map) {
+    my %vcpus;
+    for my $range ($numa_map->{cpus}->@*) {
+        my ($start, $end) = $range->@*;
+        $vcpus{$_} = 1 for $start .. $end // $start;
+    }
+    return \%vcpus;
+}
+
+# in: {hostnodes => [[1, 1]]};
+# out: 1
+my sub extract_numa_hostnode($numa_map) {
+    my $hostnodes = $numa_map->{hostnodes};
+    return undef if !defined($hostnodes);
+
+    die "Pinning only available for 1-to-1 NUMA node mapping\n"
+        if $hostnodes->@* > 1 || defined($hostnodes->[0]->[1]);
+
+    return $hostnodes->[0]->[0];
+}
+
+# in:
+# {
+#     numa  => 1,
+#     numa0 => "cpus=0-1,hostnodes=0",
+#     numa1 => "cpus=2-3,hostnodes=1",
+# }
+# out:
+# {
+#     0 => {vcpus => {0, 1} hostnode => 0},
+#     1 => {vcpus => {2, 3}, hostnode => 1,}
+# }
+my sub parse_numa_vcpu_map($conf) {
+    my $map = {};
+
+    for my $i (0 .. $PVE::QemuServer::Memory::MAX_NUMA - 1) {
+        my $entry = $conf->{"numa$i"} or next;
+        my $numa = PVE::QemuServer::Memory::parse_numa($entry) or next;
+
+        $map->{$i} = {
+            vcpus => expand_numa_vcpus($numa),
+            hostnode => extract_numa_hostnode($numa)
+        };
+    }
+
+    return $map;
+}
+
+# Parse cpu settings to map keyed by numa node.
+# in: ($sockets, $cores, $threads_per_core) = (2, 2, 1);
+# out: { 0 => {vcpus => {0, 1}}, 1 => {vcpus => {2, 3}} }
+my sub build_socket_vcpu_map($sockets, $cores, $threads_per_core) {
+    my $map = {};
+    my $vcpu_id = 0;
+
+    for my $socket (0 .. $sockets - 1) {
+        my %vcpus;
+        for (1 .. ($cores * $threads_per_core)) {
+            $vcpus{ $vcpu_id++ } = 1;
+        }
+        $map->{$socket} = { vcpus => \%vcpus };
+    }
+
+    return $map;
+}
+
+# in: ($sockets, $cores, $threads_per_core) = (2, 2, 1);
+# out: { 0 => {vcpus => {0, 1, 2, 3}} }
+my sub build_flat_vcpu_map($sockets, $cores, $threads_per_core) {
+    my $total = $sockets * $cores * $threads_per_core;
+    return {
+        0 => { vcpus => { map { $_ => 1 } 0 .. $total - 1 } },
+    };
+}
+
+# Parse qm.conf cpu settings to map.
+# The map is keyed by numa-node
+#  in: {
+#     sockets => 1,
+#     cores   => 4,
+#     vcpus   => 4,
+#     numa    => 1,
+#     numa0   => "cpus=0-1,hostnodes=0",
+#     numa1   => "cpus=2-3,hostnodes=1",
+# }
+# out: {
+#     0 => {vcpus => {0, 1}, hostnode => 0},
+#     1 => {vcpus => {2, 3}, hostnode => 1},
+# }
+sub build_vnuma_vcpu_map($conf) {
+    my $sockets = $conf->{sockets} // 1;
+    my $cores   = $conf->{cores}   // 1;
+    # Fix 4.4: honour $conf->{vcpus} when it exceeds sockets*cores (SMT/hotplug).
+    my $vcpus = $conf->{vcpus} // ($sockets * $cores);
+    my $threads_per_core = $vcpus > ($sockets * $cores)
+        ? int($vcpus / ($sockets * $cores))
+        : 1;
+
+    my $map;
+    if ($conf->{numa}) {
+        $map = parse_numa_vcpu_map($conf);
+        $map = build_socket_vcpu_map($sockets, $cores, $threads_per_core)
+            if !keys $map->%*;
+    } else {
+        $map = build_flat_vcpu_map($sockets, $cores, $threads_per_core);
+    }
+
+    my $assigned = 0;
+    $assigned += scalar keys $_->{vcpus}->%* for values $map->%*;
+
+    my $expected = $sockets * $cores * $threads_per_core;
+    die "Invalid NUMA configuration for pinning, some vCPUs missing in numa"
+        . " binding (assigned=$assigned, expected=$expected)\n"
+        if $assigned != $expected;
+
+    return $map;
+}
+
+# in: $conf = { sockets => 2, cores => 4, vcpus => 6 };
+# out: (2, 4, 8, 6)
+sub get_config_topology($conf) {
+    my $sockets = $conf->{sockets} || $conf->{smp} || 1;
+    my $cores = $conf->{cores} || 1;
+    my $maxcpus = $sockets * $cores;
+    my $vcpus = $conf->{vcpus} || $maxcpus;
+    return ($sockets, $cores, $maxcpus, $vcpus);
+}
+
+1;
diff --git a/src/PVE/QemuServer/Pinning/Topology.pm b/src/PVE/QemuServer/Pinning/Topology.pm
new file mode 100644
index 00000000..daddea0b
--- /dev/null
+++ b/src/PVE/QemuServer/Pinning/Topology.pm
@@ -0,0 +1,462 @@
+package PVE::QemuServer::Pinning::Topology;
+
+use v5.36;
+
+use PVE::CpuSet;
+use PVE::Tools qw(file_read_firstline dir_glob_foreach);
+
+# Build { cpu_id => numa_node_id } from /sys/devices/system/node.
+# key: cpu, val: numa-node
+# e.g. reads: 8 core CPU with 2 NUMA nodes
+# out: { 0 => 0, 1 => 0, 2 => 0, 3 => 0, 4 => 1, 5 => 1, 6 => 1, 7 => 1 }
+my sub cpu_to_numa_map() {
+    my $cpu_to_numa = {};
+    my $numa_path = "/sys/devices/system/node";
+
+    return $cpu_to_numa if !-d $numa_path;
+
+    dir_glob_foreach($numa_path, 'node(\d+)', sub ($node_dir, $node_id) {
+        dir_glob_foreach("$numa_path/$node_dir", 'cpu(\d+)', sub ($cpu_dir, $cpu_id) {
+            $cpu_to_numa->{$cpu_id} = $node_id;
+        });
+    });
+
+    return $cpu_to_numa;
+}
+
+# Build { cpu_id => topology_dir_path } for all CPUs under /sys.
+# in: (no args, reads sysfs)
+# out: { 0 => "/sys/devices/system/cpu/cpu0/topology",
+#        1 => "/sys/devices/system/cpu/cpu1/topology", ... }
+my sub cpu_topology_dirs() {
+    my $base_path = "/sys/devices/system/cpu";
+    opendir(my $dirfd, $base_path) || die "cannot open $base_path\n";
+
+    my $dirs = {};
+    for my $entry (readdir($dirfd)) {
+        next if $entry !~ m/^cpu(\d+)$/;
+        my $topo_path = "$base_path/$entry/topology";
+        next if !-d $topo_path;
+        $dirs->{$1} = $topo_path;
+    }
+    closedir($dirfd);
+
+    return $dirs;
+}
+
+# Read physical_package_id and core_id, normalizing sentinel values to 0.
+# in: "/sys/devices/system/cpu/cpu1/topology"
+# out: (0, 1)
+# core_id seems to be set by the vendor and non-linear.
+# e.g. for the AMD EPYC 7351P 16-Core Processor cpu 4
+# and its thread sibling cpu 16 reside on core 8
+# e.g. /sys/devices/system/cpu/cpu4/topology/
+# out: (0, 8)
+my sub read_cpu_topology($topo_path) {
+    my $socket_id = file_read_firstline("$topo_path/physical_package_id") // 0;
+    my $core_id   = file_read_firstline("$topo_path/core_id")             // 0;
+
+    # missing topology info is represented as sential value (e.g. nested virtualization)
+    $socket_id = 0 if $socket_id < 0 || $socket_id == 65535;
+    $core_id   = 0 if $core_id   < 0 || $core_id   == 65535;
+
+    return ($socket_id, $core_id);
+}
+
+my sub make_cpu_info($socket_id, $core_id, $numa_node) {
+    return {
+        socket    => $socket_id,
+        core      => "${socket_id}_${core_id}",
+        numa_node => $numa_node,
+    };
+}
+
+# Add one CPU into topology (mutates cpus/cores/numa_nodes).
+# in: {cpus=>{},cores=>{},numa_nodes=>{}}, 3, {socket=>0,core=>"0_3",numa_node=>1}
+# out: (mutated topology)
+#      cpus       => {3 => {socket=>0, core=>"0_3", numa_node=>1}}
+#      cores      => {"0_3" => {numa_node=>1, cpus=>[3]}}
+#      numa_nodes => {1 => {cores=>{"0_3"}, cpus=>{3}}}
+my sub add_cpu_to_topology($topology, $cpu_id, $cpu_info) {
+    $topology->{cpus}->{$cpu_id} = $cpu_info;
+
+    my $core_key  = $cpu_info->{core};
+    my $numa_node = $cpu_info->{numa_node};
+
+    $topology->{cores}->{$core_key} //= {
+        numa_node => $numa_node,
+        cpus => [],
+    };
+
+    push $topology->{cores}->{$core_key}->{cpus}->@*, $cpu_id;
+
+    $topology->{numa_nodes}->{$numa_node} //= {
+        cores => {},
+        cpus => {},
+    };
+    $topology->{numa_nodes}->{$numa_node}->{cores}->{$core_key} = 1;
+    $topology->{numa_nodes}->{$numa_node}->{cpus}->{$cpu_id} = 1;
+}
+
+=head2 get_host_topology
+
+Reads the host CPU topology from sysfs and returns it as a hashref with
+three keys:
+
+=over
+
+=item * C<cpus> - C<< { $cpu_id => { socket, core, numa_node } } >>, one
+entry per CPU. C<core> is a C<"${socket}_${core_id}"> string.
+
+=item * C<cores> - C<< { $core_key => { numa_node, cpus } } >>, where
+C<cpus> is an arrayref of the CPU ids sharing that physical core.
+
+=item * C<numa_nodes> - C<< { $node_id => { cores, cpus } } >>, where
+C<cores> and C<cpus> are sets (hashrefs of id => 1).
+
+=back
+
+Missing or unavailable topology information is normalized to socket and
+core C<0>, and CPUs without an entry under C</sys/devices/system/node> are
+placed on NUMA node C<0>. B<Dies> if C</sys/devices/system/cpu> cannot be
+opened.
+
+B<Example> - x86 CPU:
+1 socket, 8 cores, 2 CPUs per core (SMT siblings), 2 NUMA nodes
+
+    {
+        # Keyed by CPU id. socket = physical package, core = "socket_coreid"
+        # (the key used to index C<cores> below), numa_node = host NUMA node.
+        #
+        # CPUs 8..15 are the SMT siblings of 0..7: same socket and
+        # core, therefore same NUMA node (e.g. 0 and 8 both on node 0).
+        cpus => {
+            0 => { socket => 0, core => "0_0", numa_node => 0 },
+            1 => { socket => 0, core => "0_1", numa_node => 0 },
+            2 => { socket => 0, core => "0_2", numa_node => 0 },
+            3 => { socket => 0, core => "0_3", numa_node => 0 },
+            4 => { socket => 0, core => "0_4", numa_node => 1 },
+            5 => { socket => 0, core => "0_5", numa_node => 1 },
+            6 => { socket => 0, core => "0_6", numa_node => 1 },
+            7 => { socket => 0, core => "0_7", numa_node => 1 },
+            8  => { socket => 0, core => "0_0", numa_node => 0 },
+            9  => { socket => 0, core => "0_1", numa_node => 0 },
+            10 => { socket => 0, core => "0_2", numa_node => 0 },
+            11 => { socket => 0, core => "0_3", numa_node => 0 },
+            12 => { socket => 0, core => "0_4", numa_node => 1 },
+            13 => { socket => 0, core => "0_5", numa_node => 1 },
+            14 => { socket => 0, core => "0_6", numa_node => 1 },
+            15 => { socket => 0, core => "0_7", numa_node => 1 },
+        },
+
+        # Keyed by core key ("socket_coreid").
+        cores => {
+            # CPUs 0 and 8 are SMT siblings
+            "0_0" => { numa_node => 0, cpus => [0, 8] },
+            "0_1" => { numa_node => 0, cpus => [1, 9] },
+            "0_2" => { numa_node => 0, cpus => [2, 10] },
+            # CPUs 3 and 11 are SMT siblings (see their entries above).
+            "0_3" => { numa_node => 0, cpus => [3, 11] },
+            "0_4" => { numa_node => 1, cpus => [4, 12] },
+            "0_5" => { numa_node => 1, cpus => [5, 13] },
+            "0_6" => { numa_node => 1, cpus => [6, 14] },
+            "0_7" => { numa_node => 1, cpus => [7, 15] },
+        },
+
+        # Keyed by NUMA node id. Each value holds two sets (id => 1): the
+        # core keys on that node, and the CPU ids on that node. The 1 values
+        # are not counts; only the keys are meaningful.
+        numa_nodes => {
+            0 => {
+                cores => { "0_0" => 1, "0_1" => 1, "0_2" => 1, "0_3" => 1 },
+                cpus  => { 0 => 1, 1 => 1, 2 => 1, 3 => 1,
+                           8 => 1, 9 => 1, 10 => 1, 11 => 1 },
+            },
+            1 => {
+                cores => { "0_4" => 1, "0_5" => 1, "0_6" => 1, "0_7" => 1 },
+                cpus  => { 4 => 1, 5 => 1, 6 => 1, 7 => 1,
+                           12 => 1, 13 => 1, 14 => 1, 15 => 1 },
+            },
+        },
+    }
+=cut
+sub get_host_topology() {
+    my $cpu_to_numa = cpu_to_numa_map();
+
+    my $topology = {
+        cpus => {},
+        cores => {},
+        numa_nodes => {},
+    };
+
+    my $cpu_dirs = cpu_topology_dirs();
+
+    for my $cpu_id (sort { $a <=> $b } keys $cpu_dirs->%*) {
+        my ($socket_id, $core_id) = read_cpu_topology($cpu_dirs->{$cpu_id});
+
+        my $numa_node = $cpu_to_numa->{$cpu_id} // 0;
+
+        my $cpu_info = make_cpu_info($socket_id, $core_id, $numa_node);
+        add_cpu_to_topology($topology, $cpu_id, $cpu_info);
+    }
+
+    return $topology;
+}
+
+my sub read_sysfs_trim($path) {
+    return undef if !-e $path;
+    my $val = file_read_firstline($path);
+    return undef if !defined($val);
+    $val =~ s/^\s+|\s+$//g;
+    return $val;
+}
+
+# in: /sys/devices/system/cpu/online = 0-7
+# out: [0, 1, 2, 3, 4, 5, 6, 7]
+my sub online_cpu_ids() {
+    my $content = read_sysfs_trim("/sys/devices/system/cpu/online");
+    return [] if !$content;
+
+    my @ids;
+    for my $part (split(/,/, $content)) {
+        if ($part =~ /^(\d+)-(\d+)$/) {
+            push @ids, ($1 .. $2);
+        } elsif ($part =~ /^(\d+)$/) {
+            push @ids, $1;
+        }
+    }
+    return \@ids;
+}
+
+# Build a stable "CPU type" key for a single online CPU.
+# On ARM: MIDR register + cache geometry.
+# On x86 (no MIDR): cache geometry alone (level/type/line/ways/sets).
+# On intel CPUs different P/E cores or big.LITTLE cores produce different keys because
+# they have different cache geometries even when /proc/cpuinfo model names
+# are identical or absent.
+# Note: Only tested on AMD x86 CPUs, so take this with a grain of salt.
+my sub cpu_type_key($cpu_id) {
+    my $cpu_path = "/sys/devices/system/cpu/cpu$cpu_id";
+
+    my $key = '';
+
+    # MIDR (ARM only). Absent on x86.
+    my $midr = read_sysfs_trim("$cpu_path/regs/identification/midr_el1");
+    $key .= defined($midr) ? $midr : 'no-midr';
+
+    # Cache geometry: sorted cache indices, each with level/type/line/ways/sets.
+    # This distinguishes P-cores from E-cores on x86 (different L2/L3 sizes)
+    # and provides a secondary signal on ARM.
+    my $cache_base = "$cpu_path/cache";
+    if (-d $cache_base) {
+        opendir(my $dh, $cache_base) or return $key;
+        my @indices = sort { $a <=> $b }
+            map { /^index(\d+)$/ ? $1 : () }
+            readdir($dh);
+        closedir($dh);
+
+        for my $idx (@indices) {
+            my $idx_path = "$cache_base/index$idx";
+            for my $field (qw(level type coherency_line_size ways_of_associativity number_of_sets)) {
+                my $val = read_sysfs_trim("$idx_path/$field");
+                $key .= '/' . (defined($val) ? $val : '');
+            }
+        }
+    }
+
+    return $key;
+}
+
+# Group online CPUs by cpu_type_key. capacity defaults to 1024/CPU.
+# out: ({ "abc/..." => {cpus => {0, 1, 2, 3}, capacity => 4096} },
+#       [ "abc/..." ])
+sub group_cpus_by_type() {
+    my $online_ids = online_cpu_ids();
+    return ({}, []) if !$online_ids || !@$online_ids;
+
+    my %groups;
+    my @keys;
+
+    for my $cpu_id (@$online_ids) {
+        my $key = cpu_type_key($cpu_id);
+        if (!$groups{$key}) {
+            $groups{$key} = { cpus => [], capacity => 0 };
+            push @keys, $key;
+        }
+        push $groups{$key}{cpus}->@*, $cpu_id;
+
+        my $cap = 1024;
+        my $cap_val = read_sysfs_trim("/sys/devices/system/cpu/cpu$cpu_id/cpu_capacity");
+        if (defined($cap_val) && $cap_val =~ /^(\d+)$/) {
+            $cap = int($1);
+        }
+        $groups{$key}{capacity} += $cap;
+    }
+
+    # Deterministic key order: by first CPU id in each group.
+    @keys = sort { $groups{$a}{cpus}[0] <=> $groups{$b}{cpus}[0] } @keys;
+
+    return (\%groups, \@keys);
+}
+
+sub are_cpus_heterogenous($cpu_ids_arrayref) {
+    my ($groups, $keys) = group_cpus_by_type();
+    return 0 if !@$keys || @$keys <= 1;
+
+    my %requested = map { $_ => 1 } @$cpu_ids_arrayref;
+    my $seen_key;
+
+    for my $key (@$keys) {
+        for my $cpu_id (@{$groups->{$key}{cpus}}) {
+            next if !$requested{$cpu_id};
+            if (!defined $seen_key) {
+                $seen_key = $key;
+            } elsif ($key ne $seen_key) {
+                return 1;
+            }
+            last;
+        }
+    }
+    return 0;
+}
+
+sub filter_by_affinity($topology, $affinity_string) {
+    return $topology if !defined($affinity_string) || $affinity_string eq '';
+
+    my ($_count, $affinity_members) = PVE::CpuSet::parse_cpuset($affinity_string);
+
+    my $filtered = {
+        cpus => {},
+        cores => {},
+        numa_nodes => {},
+    };
+
+    for my $cpu_id (sort { $a <=> $b } keys $topology->{cpus}->%*) {
+        next if !$affinity_members->{$cpu_id};
+        my $cpu_info = $topology->{cpus}->{$cpu_id};
+        add_cpu_to_topology($filtered, $cpu_id,  $cpu_info);
+    }
+
+    return $filtered;
+}
+
+# Group selected host CPUs by socket -> core -> threads.
+# in:  $topology, [0, 1, 4, 5]
+# out: ({ 0 => { "0_0" => [0, 4], "0_1" => [1, 5] } }, undef)
+# in:  $topology, [0, 99]
+# out: (undef, "CPU 99 not found in host topology")
+my sub group_cpus_by_socket_core($topology, $selected_cpus) {
+    my %sockets;
+    for my $cpu_id (@$selected_cpus) {
+        my $info = $topology->{cpus}->{$cpu_id};
+        return (undef, "CPU $cpu_id not found in host topology") if !$info;
+        push @{$sockets{$info->{socket}}{$info->{core}}}, $cpu_id;
+    }
+    return (\%sockets, undef);
+}
+
+# Check every socket has same core count and every core same thread count.
+# in: { 0 => { "0_0" => [0, 8], "0_4" => [4, 12] } }
+# out: (2, 2)
+# in: { 0 => { "0_0" => [0, 8] }, 1 => { "1_0" => [1, 9], "1_1" => [2, 10] } }
+# out: (undef, undef)
+my sub validate_consistency($sockets) {
+    my ($expected_cores, $expected_threads) = (-1, -1);
+
+    for my $socket (keys %$sockets) {
+        my $core_count = scalar keys %{$sockets->{$socket}};
+        return (undef, undef)
+            if $expected_cores != -1 && $core_count != $expected_cores;
+        $expected_cores = $core_count;
+
+        for my $core (keys %{$sockets->{$socket}}) {
+            my $thread_count = scalar @{$sockets->{$socket}->{$core}};
+            return (undef, undef)
+                if $expected_threads != -1 && $thread_count != $expected_threads;
+            $expected_threads = $thread_count;
+        }
+    }
+
+    return ($expected_cores, $expected_threads);
+}
+
+my sub flat_fallback_result($total, $warning = undef) {
+    $warning //= "Instance uses a CPU pinning profile which doesn't match"
+        . " hardware layout; falling back to flat topology"
+        . " (sockets=1, cores=$total, threads=1)";
+    return {
+        valid   => 0,
+        sockets => 1,
+        cores   => $total,
+        threads => 1,
+        warning => $warning,
+    };
+}
+
+=encoding utf8
+
+=head2 validate_pinned_topology
+
+Validates that the host CPUs in C<$selected_cpus> (an arrayref of host CPU
+IDs) form a balanced topology matching the guest configuration.
+
+=over 4
+
+=item 1. Groups selected host CPUs by physical socket and physical core.
+
+=item 2. Checks that every socket contains the same number of cores.
+
+=item 3. Checks that every core contains the same number of threads.
+
+=item 4. Checks that C<sockets * cores_per_socket * threads_per_core> equals
+the total number of selected CPUs.
+
+=back
+
+Returns a hashref:
+
+  {
+      valid   => 1 | 0,
+      sockets => $nr_sockets,
+      cores   => $nr_cores_per_socket,
+      threads => $nr_threads_per_core,
+      warning => $msg | undef,
+  }
+
+If the topology is invalid, the result contains a flat fallback
+(C<sockets=1, cores=N, threads=1>) and a warning string.
+
+=cut
+sub validate_pinned_topology($topology, $selected_cpus, $guest_sockets, $guest_cores) {
+    my $total = scalar @$selected_cpus;
+
+    # Empty selection: trust the guest-configured topology.
+    return {
+        valid   => 1,
+        sockets => $guest_sockets,
+        cores   => $guest_cores,
+        threads => 1,
+        warning => undef,
+    } if $total == 0;
+
+     my ($sockets, $err) = group_cpus_by_socket_core($topology, $selected_cpus);
+     return flat_fallback_result($total, $err) if defined($err);
+
+    my ($expected_cores, $expected_threads) = validate_consistency($sockets);
+    return flat_fallback_result($total)
+        if !defined $expected_cores || !defined $expected_threads;
+
+    my $nr_sockets = scalar keys %$sockets;
+    return flat_fallback_result($total)
+        if $nr_sockets * $expected_cores * $expected_threads != $total;
+
+    return {
+        valid   => 1,
+        sockets => $nr_sockets,
+        cores   => $expected_cores,
+        threads => $expected_threads,
+        warning => undef,
+    };
+}
+
+1;
-- 
2.47.3





  reply	other threads:[~2026-09-21  9:54 UTC|newest]

Thread overview: 5+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-09-21  9:53 [RFC qemu-server v2 0/4] fix #7282: allow (NUMA aware) vCPU pinning Elias Huhsovitz
2026-09-21  9:54 ` Elias Huhsovitz [this message]
2026-09-21  9:54 ` [RFC qemu-server v2 2/4] pinning: add NUMA allocator and reservation tracking Elias Huhsovitz
2026-09-21  9:54 ` [RFC qemu-server v2 3/4] memory: integrate pinning-aware NUMA memory binding Elias Huhsovitz
2026-09-21  9:54 ` [RFC qemu-server v2 4/4] pinning: integrate cpu pinning into vm lifecycle Elias Huhsovitz

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20260921095404.61552-2-e.huhsovitz@proxmox.com \
    --to=e.huhsovitz@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal