[pve-devel] [PATCH SERIES v2 manager/docs/widget-toolkit/qemu-server 0/4] fix #3593

[pve-devel] [PATCH SERIES v2 manager/docs/widget-toolkit/qemu-server 0/4] fix #3593

[Daniel Bowder]

Summary:
Allows for virtual machines to be pinned to CPU cores via an affinity value
added to the qemu configuration. I previously created a v1 of this patch,
but it was my first submission and had lots of formatting issues. This patch
is v2. The affinity input can be found on the CPU Edit page for virtual
machines.

v1 can be found here:
https://lists.proxmox.com/pipermail/pve-devel/2022-June/053219.html

V1 -> V2:
* Adhere better to style guides
* Reuse PVE::CpuSet instead of reinventing the wheel
* Change new qemu conf parameter from 'cpuset' to 'affinity'
* Exec kvm using taskset to pin processes


qemu-server:

Daniel Bowder (1):
  fix #3593: add affinity to qemu conf, launch taskset before kvm

 PVE/QemuServer.pm | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)


docs:

Daniel Bowder (1):
  fix #3593: add affinity to qm.conf docs

 qm.adoc             | 15 +++++++++++++++
 qm.conf.5-opts.adoc |  4 ++++
 2 files changed, 19 insertions(+)


widget-toolkit:

Daniel Bowder (1):
  fix #3593: add CpuSet type to js frontend

 src/Toolkit.js | 20 ++++++++++++++++++++
 src/Utils.js   |  2 ++
 2 files changed, 22 insertions(+)


manager:

Daniel Bowder (1):
  fix #3593: add js input for affinity

 www/manager6/qemu/ProcessorEdit.js | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

-- 
2.36.1

[pve-devel] [PATCH v2 qemu-server 1/4] fix #3593: add affinity to qemu

[Daniel Bowder]

Reuse the PVE::CpuSet to validate cpuset formatting.
Add new qemu property called 'affinity' to store the cpuset.
Push taskset command in front of kvm if 'affinity' is set.

Signed-off-by: Daniel Bowder <daniel@bowdernet.com>
---
 PVE/QemuServer.pm | 27 +++++++++++++++++++++++++++
 1 file changed, 27 insertions(+)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 7d9cf22..94b44ae 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -28,6 +28,7 @@ use UUID;
 
 use PVE::Cluster qw(cfs_register_file cfs_read_file cfs_write_file);
 use PVE::CGroup;
+use PVE::CpuSet;
 use PVE::DataCenterConfig;
 use PVE::Exception qw(raise raise_param_exc);
 use PVE::Format qw(render_duration render_bytes);
@@ -713,6 +714,11 @@ EODESCR
 	description => "Some (read-only) meta-information about this guest.",
 	optional => 1,
     },
+    affinity => {
+	type => 'string', format => 'pve-cpuset',
+	description => "List of host cores used to execute guest processes.",
+	optional => 1,
+    },
 };
 
 my $cicustom_fmt = {
@@ -1032,6 +1038,20 @@ foreach my $key (keys %$confdesc_cloudinit) {
     $confdesc->{$key} = $confdesc_cloudinit->{$key};
 }
 
+PVE::JSONSchema::register_format('pve-cpuset', \&pve_verify_cpuset);
+sub pve_verify_cpuset {
+    my ($set_text, $noerr) = @_;
+
+    my ($count, $members) = eval { PVE::CpuSet::parse_cpuset($set_text) };
+
+    if ($@) {
+	return if $noerr;
+	die "unable to parse cpuset option\n";
+    }
+
+    return PVE::CpuSet->new($members)->short_string();
+}
+
 PVE::JSONSchema::register_format('pve-volume-id-or-qm-path', \&verify_volume_id_or_qm_path);
 sub verify_volume_id_or_qm_path {
     my ($volid, $noerr) = @_;
@@ -3535,6 +3555,13 @@ sub config_to_command {
     my $use_old_bios_files = undef;
     ($use_old_bios_files, $machine_type) = qemu_use_old_bios_files($machine_type);
 
+    if ($conf->{affinity}) {
+	push @$cmd, "taskset";
+	push @$cmd, "--cpu-list";
+	push @$cmd, "--all-tasks";
+	push @$cmd, $conf->{affinity};
+    }
+
     push @$cmd, $kvm_binary;
 
     push @$cmd, '-id', $vmid;
-- 
2.36.1

[pve-devel] [PATCH v2 pve-docs 2/4] fix #3593: add affinity to docs

[Daniel Bowder]

Added affinity to the resource limiting info in qm.adoc.
Added affinity to the qemu conf spec in qm.conf.5-opts.adoc.

Signed-off-by: Daniel Bowder <daniel@bowdernet.com>
---
 qm.adoc             | 15 +++++++++++++++
 qm.conf.5-opts.adoc |  4 ++++
 2 files changed, 19 insertions(+)

diff --git a/qm.adoc b/qm.adoc
index 4d0c7c4..3722726 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -328,6 +328,21 @@ For more information see `man systemd.resource-control`, here `CPUQuota`
 corresponds to `cpulimit` and `CPUShares` corresponds to our `cpuunits`
 setting, visit its Notes section for references and implementation details.
 
+The third CPU resource limiting setting, *affinity*, controls what host cores
+the virtual machine will be permitted to execute on. E.g., if an affinity value
+of `0-3,8-11` is provided, the virtual machine will be restricted to using the
+host cores `0,1,2,3,8,9,10,` and `11`. Valid *affinity* values are written in
+cpuset `List Format`. List Format is a comma-separated list of CPU numbers and
+ranges of numbers, in ASCII decimal.
+
+NOTE: CPU *affinity* uses the `taskset` command to restrict virtual machines to
+a given set of cores. This restriction will not take effect for some types of
+processes that may be created for IO. *CPU affinity is not a security feature.*
+
+For more information regarding *affinity* see `man cpuset`. Here the
+`List Format` corresponds to valid *affinity* values. Visit its `Formats`
+section for more examples.
+
 CPU Type
 ^^^^^^^^
 
diff --git a/qm.conf.5-opts.adoc b/qm.conf.5-opts.adoc
index a56dc5d..e46b4c2 100644
--- a/qm.conf.5-opts.adoc
+++ b/qm.conf.5-opts.adoc
@@ -155,6 +155,10 @@ Limit of CPU usage.
 +
 NOTE: If the computer has 2 CPUs, it has total of '2' CPU time. Value '0' indicates no CPU limit.
 
+`affinity`: `<string>`::
+
+Set of CPU cores to pin the virtual machine processes to. This is a comma sepparated list of numbers or ranges in list format as defined by the Linux man page for cpuset. ( e.g `0,4-6,9` )
+
 `cpuunits`: `<integer> (1 - 262144)` ('default =' `cgroup v1: 1024, cgroup v2: 100`)::
 
 CPU weight for a VM. Argument is used in the kernel fair scheduler. The larger the number is, the more CPU time this VM gets. Number is relative to weights of all the other running VMs.
-- 
2.36.1

[pve-devel] [PATCH v2 proxmox-widget-toolkit 3/4] fix #3593: add CpuSet type to js

[Daniel Bowder]

Regex parses a cpuset via 2 matches. Find number(s) or range(s) folowed
by a comma, then, find a single number or a single range not followed
by a comma. E.g., 0-1,4-5,6,7,10,11,14-15
CpuSet function first checks regex, then ensures left num <= right num

Signed-off-by: Daniel Bowder <daniel@bowdernet.com>
---
 src/Toolkit.js | 20 ++++++++++++++++++++
 src/Utils.js   |  2 ++
 2 files changed, 22 insertions(+)

diff --git a/src/Toolkit.js b/src/Toolkit.js
index a1d291e..ebdc545 100644
--- a/src/Toolkit.js
+++ b/src/Toolkit.js
@@ -121,6 +121,26 @@ Ext.apply(Ext.form.field.VTypes, {
     },
     HttpProxyText: gettext('Example') + ": http://username:password&#64;host:port/",
 
+    CpuSet: function(v) {
+	if (!Proxmox.Utils.CpuSet_match.test(v)) {
+	    return false;
+	}
+	let groups = v.split(",");
+	for (let i = 0; i < groups.length; i++) {
+	    if (!groups[i].includes("-")) {
+		continue;
+	    }
+	    let values = groups[i].split("-");
+	    let left = parseInt(values[0], 10);
+	    let right = parseInt(values[1], 10);
+	    if (left > right) {
+		return false;
+	    }
+	}
+	return true;
+    },
+    CpuSetText: gettext('This is not a valid CpuSet'),
+
     DnsName: function(v) {
 	return Proxmox.Utils.DnsName_match.test(v);
     },
diff --git a/src/Utils.js b/src/Utils.js
index 6a03057..87afbea 100644
--- a/src/Utils.js
+++ b/src/Utils.js
@@ -1315,6 +1315,8 @@ utilities: {
 	me.DnsName_match = new RegExp("^" + DnsName_REGEXP + "$");
 	me.DnsName_or_Wildcard_match = new RegExp("^(?:\\*\\.)?" + DnsName_REGEXP + "$");
 
+	me.CpuSet_match = /^(?:(?:[0-9]+,)|(?:[0-9]+-[0-9]+,))*(?:(?:[0-9]+)$|(?:[0-9]+-[0-9]+)$)/;
+
 	me.HostPort_match = new RegExp("^(" + IPV4_REGEXP + "|" + DnsName_REGEXP + ")(?::(\\d+))?$");
 	me.HostPortBrackets_match = new RegExp("^\\[(" + IPV6_REGEXP + "|" + IPV4_REGEXP + "|" + DnsName_REGEXP + ")\\](?::(\\d+))?$");
 	me.IP6_dotnotation_match = new RegExp("^(" + IPV6_REGEXP + ")(?:\\.(\\d+))?$");
-- 
2.36.1

[pve-devel] [PATCH v2 pve-manager 4/4] fix #3593: add js input for affinity

[Daniel Bowder]

Adds a textfield for the affinity value. Delete that affinity value
when the textbox is empty.

Signed-off-by: Daniel Bowder <daniel@bowdernet.com>
---
 www/manager6/qemu/ProcessorEdit.js | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/www/manager6/qemu/ProcessorEdit.js b/www/manager6/qemu/ProcessorEdit.js
index 1bed2877..42c919f2 100644
--- a/www/manager6/qemu/ProcessorEdit.js
+++ b/www/manager6/qemu/ProcessorEdit.js
@@ -27,6 +27,17 @@ Ext.define('PVE.qemu.ProcessorInputPanel', {
 	    values.delete = values.delete.join(',');
 	}
 
+	if (values.affinity === undefined ||
+	    values.affinity === null ||
+	    values.affinity === '') {
+		if (values.delete.length > 0) {
+		    values.delete = values.delete + ",affinity";
+		} else {
+		    values.delete = "affinity";
+		}
+		delete values.affinity;
+	}
+
 	PVE.Utils.delete_if_default(values, 'cpulimit', '0', 0);
 	PVE.Utils.delete_if_default(values, 'cpuunits', '1024', 0);
 
@@ -180,6 +191,18 @@ Ext.define('PVE.qemu.ProcessorInputPanel', {
 	    allowBlank: true,
 	    emptyText: gettext('unlimited'),
 	},
+	{
+	    xtype: 'textfield',
+	    name: 'affinity',
+	    vtype: 'CpuSet',
+	    value: '',
+	    fieldLabel: gettext('CPU affinity'),
+	    allowBlank: true,
+	    emptyText: gettext("all cores"),
+	    bind: {
+		value: '{affinity}',
+	    },
+	},
     ],
 
     advancedColumn2: [
-- 
2.36.1

[pve-devel] applied-series: [PATCH SERIES v2 manager/docs/widget-toolkit/qemu-server 0/4] fix #3593

[Wolfgang Bumiller]

sorry for the delay, applied, thanks

On Thu, Jun 30, 2022 at 05:09:44PM -0700, Daniel Bowder wrote:
> Summary:
> Allows for virtual machines to be pinned to CPU cores via an affinity value
> added to the qemu configuration. I previously created a v1 of this patch,
> but it was my first submission and had lots of formatting issues. This patch
> is v2. The affinity input can be found on the CPU Edit page for virtual
> machines.
> 
> v1 can be found here:
> https://lists.proxmox.com/pipermail/pve-devel/2022-June/053219.html
> 
> V1 -> V2:
> * Adhere better to style guides
> * Reuse PVE::CpuSet instead of reinventing the wheel
> * Change new qemu conf parameter from 'cpuset' to 'affinity'
> * Exec kvm using taskset to pin processes
> 
> 
> qemu-server:
> 
> Daniel Bowder (1):
>   fix #3593: add affinity to qemu conf, launch taskset before kvm
> 
>  PVE/QemuServer.pm | 27 +++++++++++++++++++++++++++
>  1 file changed, 27 insertions(+)
> 
> 
> docs:
> 
> Daniel Bowder (1):
>   fix #3593: add affinity to qm.conf docs
> 
>  qm.adoc             | 15 +++++++++++++++
>  qm.conf.5-opts.adoc |  4 ++++
>  2 files changed, 19 insertions(+)
> 
> 
> widget-toolkit:
> 
> Daniel Bowder (1):
>   fix #3593: add CpuSet type to js frontend
> 
>  src/Toolkit.js | 20 ++++++++++++++++++++
>  src/Utils.js   |  2 ++
>  2 files changed, 22 insertions(+)
> 
> 
> manager:
> 
> Daniel Bowder (1):
>   fix #3593: add js input for affinity
> 
>  www/manager6/qemu/ProcessorEdit.js | 23 +++++++++++++++++++++++
>  1 file changed, 23 insertions(+)
> 
> -- 
> 2.36.1