[pve-devel] [PATCH qemu-server 0/4] qemu 7.1: virtio-net: add new default values

[pve-devel] [PATCH qemu-server 0/4] qemu 7.1: virtio-net: add new default values

[Alexandre Derumier]

patch 1 was already submitted last month.

this patch series also add a new patch to increase rx|tx-queue-size to 1024
and tests.

I'm running theses values in production and don't have seen any performance
regression.


Alexandre Derumier (4):
  fix #4296: virtio-net: enable packed queues for qemu 7.1
  virtio-net: increase defaults rx|tx-queue-size to 1024
  test: add qemu 7.1 default netdev rx|tx_queue_size=1024
  test: add qemu 7.1 multiqueue netdev test

 PVE/QemuServer.pm                             | 15 ++++++++---
 .../cputype-icelake-client-deprecation.conf   |  1 -
 ...putype-icelake-client-deprecation.conf.cmd |  2 --
 test/cfg2cmd/netdev-7.1-multiqueues.conf      |  8 ++++++
 test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd  | 26 +++++++++++++++++++
 test/cfg2cmd/netdev-7.1.conf                  |  8 ++++++
 test/cfg2cmd/netdev-7.1.conf.cmd              | 26 +++++++++++++++++++
 7 files changed, 80 insertions(+), 6 deletions(-)
 create mode 100644 test/cfg2cmd/netdev-7.1-multiqueues.conf
 create mode 100644 test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd
 create mode 100644 test/cfg2cmd/netdev-7.1.conf
 create mode 100644 test/cfg2cmd/netdev-7.1.conf.cmd

-- 
2.30.2

[pve-devel] [PATCH qemu-server 1/4] fix #4296: virtio-net: enable packed queues for qemu 7.1

[Alexandre Derumier]

virtio 1.1 have improve virtio multiqueue performance,
with a new implementation called "packed queues".

https://www.redhat.com/en/blog/packed-virtqueue-how-reduce-overhead-virtio
https://archive.fosdem.org/2018/schedule/event/virtio/attachments/slides/2167/export/events/attachments/virtio/slides/2167/fosdem_virtio1_1.pdf

This patch enable it by default for qemu 7.1

This don't break old guests with old virtio 1.0 drivers,
as virtio device/devices are forward/backward compatible.

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 PVE/QemuServer.pm | 10 +++++++---
 1 file changed, 7 insertions(+), 3 deletions(-)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 513a248..2fed06d 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -1730,7 +1730,7 @@ sub print_pbs_blockdev {
 }
 
 sub print_netdevice_full {
-    my ($vmid, $conf, $net, $netid, $bridges, $use_old_bios_files, $arch, $machine_type) = @_;
+    my ($vmid, $conf, $net, $netid, $bridges, $use_old_bios_files, $arch, $machine_type, $machine_version) = @_;
 
     my $device = $net->{model};
     if ($net->{model} eq 'virtio') {
@@ -1744,6 +1744,9 @@ sub print_netdevice_full {
 	# and out of each queue plus one config interrupt and control vector queue
 	my $vectors = $net->{queues} * 2 + 2;
 	$tmpstr .= ",vectors=$vectors,mq=on";
+	if (min_version($machine_version, 7, 1)) {
+	    $tmpstr .= ",packed=on";
+	}
     }
     $tmpstr .= ",bootindex=$net->{bootindex}" if $net->{bootindex} ;
 
@@ -4021,7 +4024,7 @@ sub config_to_command {
 	push @$devices, '-netdev', $netdevfull;
 
 	my $netdevicefull = print_netdevice_full(
-	    $vmid, $conf, $d, $netname, $bridges, $use_old_bios_files, $arch, $machine_type);
+	    $vmid, $conf, $d, $netname, $bridges, $use_old_bios_files, $arch, $machine_type, $machine_version);
 
 	push @$devices, '-device', $netdevicefull;
     }
@@ -4252,11 +4255,12 @@ sub vm_deviceplug {
 	return if !qemu_netdevadd($vmid, $conf, $arch, $device, $deviceid);
 
 	my $machine_type = PVE::QemuServer::Machine::qemu_machine_pxe($vmid, $conf);
+	my $machine_version = PVE::QemuServer::Machine::extract_version($machine_type);
 	my $use_old_bios_files = undef;
 	($use_old_bios_files, $machine_type) = qemu_use_old_bios_files($machine_type);
 
 	my $netdevicefull = print_netdevice_full(
-	    $vmid, $conf, $device, $deviceid, undef, $use_old_bios_files, $arch, $machine_type);
+	    $vmid, $conf, $device, $deviceid, undef, $use_old_bios_files, $arch, $machine_type, $machine_version);
 	qemu_deviceadd($vmid, $netdevicefull);
 	eval {
 	    qemu_deviceaddverify($vmid, $deviceid);
-- 
2.30.2

[pve-devel] [PATCH qemu-server 2/4] virtio-net: increase defaults rx|tx-queue-size to 1024

[Alexandre Derumier]

This is reducing packet drop on high pps, and also needed for dpdk.

Redhat already have use it by default in rhev and his openstack platform too
since 2019.

I'm using it in production since 6 months, I don't have seen performance regression.

fix: (which ask for custom option, but setting it by default seem fine for me)

https://bugzilla.proxmox.com/show_bug.cgi?id=1546
https://bugzilla.proxmox.com/show_bug.cgi?id=2349
Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 PVE/QemuServer.pm | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 2fed06d..e636c98 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -1748,6 +1748,11 @@ sub print_netdevice_full {
 	    $tmpstr .= ",packed=on";
 	}
     }
+
+    if (min_version($machine_version, 7, 1) && $net->{model} eq 'virtio'){
+	$tmpstr .= ",rx_queue_size=1024,tx_queue_size=1024";
+    }
+
     $tmpstr .= ",bootindex=$net->{bootindex}" if $net->{bootindex} ;
 
     if (my $mtu = $net->{mtu}) {
-- 
2.30.2

[pve-devel] [PATCH qemu-server 3/4] test: add qemu 7.1 default netdev rx|tx_queue_size=1024

[Alexandre Derumier]

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 .../cputype-icelake-client-deprecation.conf   |  1 -
 ...putype-icelake-client-deprecation.conf.cmd |  2 --
 test/cfg2cmd/netdev-7.1.conf                  |  8 ++++++
 test/cfg2cmd/netdev-7.1.conf.cmd              | 26 +++++++++++++++++++
 4 files changed, 34 insertions(+), 3 deletions(-)
 create mode 100644 test/cfg2cmd/netdev-7.1.conf
 create mode 100644 test/cfg2cmd/netdev-7.1.conf.cmd

diff --git a/test/cfg2cmd/cputype-icelake-client-deprecation.conf b/test/cfg2cmd/cputype-icelake-client-deprecation.conf
index 668a323..523dd27 100644
--- a/test/cfg2cmd/cputype-icelake-client-deprecation.conf
+++ b/test/cfg2cmd/cputype-icelake-client-deprecation.conf
@@ -6,7 +6,6 @@ cpu: Icelake-Client
 ide2: none,media=cdrom
 memory: 768
 name: simple
-net0: virtio=A2:C0:43:77:08:A0,bridge=vmbr0
 ostype: l26
 scsi0: local:8006/base-8006-disk-0.qcow2,discard=on,size=104858K
 scsihw: virtio-scsi-pci
diff --git a/test/cfg2cmd/cputype-icelake-client-deprecation.conf.cmd b/test/cfg2cmd/cputype-icelake-client-deprecation.conf.cmd
index 9960e5e..bf08443 100644
--- a/test/cfg2cmd/cputype-icelake-client-deprecation.conf.cmd
+++ b/test/cfg2cmd/cputype-icelake-client-deprecation.conf.cmd
@@ -28,6 +28,4 @@
   -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' \
   -drive 'file=/var/lib/vz/images/8006/base-8006-disk-0.qcow2,if=none,id=drive-scsi0,discard=on,format=qcow2,cache=none,aio=io_uring,detect-zeroes=unmap' \
   -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,bootindex=100' \
-  -netdev 'type=tap,id=net0,ifname=tap8006i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
-  -device 'virtio-net-pci,mac=A2:C0:43:77:08:A0,netdev=net0,bus=pci.0,addr=0x12,id=net0,bootindex=300' \
   -machine 'type=pc+pve0'
diff --git a/test/cfg2cmd/netdev-7.1.conf b/test/cfg2cmd/netdev-7.1.conf
new file mode 100644
index 0000000..82be056
--- /dev/null
+++ b/test/cfg2cmd/netdev-7.1.conf
@@ -0,0 +1,8 @@
+# TEST: Simple test for netdev related stuff
+# QEMU_VERSION: 7.1
+bootdisk: scsi0
+cores: 3
+memory: 768
+name: netdev
+net0: virtio=A2:C0:43:77:08:A0,bridge=vmbr0,mtu=900
+ostype: l26
diff --git a/test/cfg2cmd/netdev-7.1.conf.cmd b/test/cfg2cmd/netdev-7.1.conf.cmd
new file mode 100644
index 0000000..6f9c177
--- /dev/null
+++ b/test/cfg2cmd/netdev-7.1.conf.cmd
@@ -0,0 +1,26 @@
+/usr/bin/kvm \
+  -id 8006 \
+  -name 'netdev,debug-threads=on' \
+  -no-shutdown \
+  -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \
+  -mon 'chardev=qmp,mode=control' \
+  -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
+  -mon 'chardev=qmp-event,mode=control' \
+  -pidfile /var/run/qemu-server/8006.pid \
+  -daemonize \
+  -smp '3,sockets=1,cores=3,maxcpus=3' \
+  -nodefaults \
+  -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
+  -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \
+  -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \
+  -m 768 \
+  -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
+  -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
+  -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
+  -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
+  -device 'VGA,id=vga,bus=pci.0,addr=0x2' \
+  -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \
+  -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \
+  -netdev 'type=tap,id=net0,ifname=tap8006i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on' \
+  -device 'virtio-net-pci,mac=A2:C0:43:77:08:A0,netdev=net0,bus=pci.0,addr=0x12,id=net0,rx_queue_size=1024,tx_queue_size=1024,bootindex=300,host_mtu=900' \
+  -machine 'type=pc+pve0'
-- 
2.30.2

[pve-devel] [PATCH qemu-server 4/4] test: add qemu 7.1 multiqueue netdev test

[Alexandre Derumier]

Signed-off-by: Alexandre Derumier <aderumier@odiso.com>
---
 test/cfg2cmd/netdev-7.1-multiqueues.conf     |  8 ++++++
 test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd | 26 ++++++++++++++++++++
 2 files changed, 34 insertions(+)
 create mode 100644 test/cfg2cmd/netdev-7.1-multiqueues.conf
 create mode 100644 test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd

diff --git a/test/cfg2cmd/netdev-7.1-multiqueues.conf b/test/cfg2cmd/netdev-7.1-multiqueues.conf
new file mode 100644
index 0000000..da5f111
--- /dev/null
+++ b/test/cfg2cmd/netdev-7.1-multiqueues.conf
@@ -0,0 +1,8 @@
+# TEST: Simple test for netdev related stuff
+# QEMU_VERSION: 7.1
+bootdisk: scsi0
+cores: 3
+memory: 768
+name: netdev
+net0: virtio=A2:C0:43:77:08:A0,bridge=vmbr0,mtu=900,queues=2
+ostype: l26
diff --git a/test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd b/test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd
new file mode 100644
index 0000000..2842fcd
--- /dev/null
+++ b/test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd
@@ -0,0 +1,26 @@
+/usr/bin/kvm \
+  -id 8006 \
+  -name 'netdev,debug-threads=on' \
+  -no-shutdown \
+  -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \
+  -mon 'chardev=qmp,mode=control' \
+  -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \
+  -mon 'chardev=qmp-event,mode=control' \
+  -pidfile /var/run/qemu-server/8006.pid \
+  -daemonize \
+  -smp '3,sockets=1,cores=3,maxcpus=3' \
+  -nodefaults \
+  -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
+  -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \
+  -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \
+  -m 768 \
+  -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \
+  -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \
+  -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \
+  -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \
+  -device 'VGA,id=vga,bus=pci.0,addr=0x2' \
+  -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \
+  -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \
+  -netdev 'type=tap,id=net0,ifname=tap8006i0,script=/var/lib/qemu-server/pve-bridge,downscript=/var/lib/qemu-server/pve-bridgedown,vhost=on,queues=2' \
+  -device 'virtio-net-pci,mac=A2:C0:43:77:08:A0,netdev=net0,bus=pci.0,addr=0x12,id=net0,vectors=6,mq=on,packed=on,rx_queue_size=1024,tx_queue_size=1024,bootindex=300,host_mtu=900'
+  -machine 'type=pc+pve0'
-- 
2.30.2

[pve-devel] applied-series: [PATCH qemu-server 0/4] qemu 7.1: virtio-net: add new default values

[Thomas Lamprecht]

Am 13/11/2022 um 15:37 schrieb Alexandre Derumier:
> patch 1 was already submitted last month.
> 
> this patch series also add a new patch to increase rx|tx-queue-size to 1024
> and tests.
> 
> I'm running theses values in production and don't have seen any performance
> regression.
> 
> 
> Alexandre Derumier (4):
>   fix #4296: virtio-net: enable packed queues for qemu 7.1
>   virtio-net: increase defaults rx|tx-queue-size to 1024
>   test: add qemu 7.1 default netdev rx|tx_queue_size=1024
>   test: add qemu 7.1 multiqueue netdev test
> 
>  PVE/QemuServer.pm                             | 15 ++++++++---
>  .../cputype-icelake-client-deprecation.conf   |  1 -
>  ...putype-icelake-client-deprecation.conf.cmd |  2 --
>  test/cfg2cmd/netdev-7.1-multiqueues.conf      |  8 ++++++
>  test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd  | 26 +++++++++++++++++++
>  test/cfg2cmd/netdev-7.1.conf                  |  8 ++++++
>  test/cfg2cmd/netdev-7.1.conf.cmd              | 26 +++++++++++++++++++
>  7 files changed, 80 insertions(+), 6 deletions(-)
>  create mode 100644 test/cfg2cmd/netdev-7.1-multiqueues.conf
>  create mode 100644 test/cfg2cmd/netdev-7.1-multiqueues.conf.cmd
>  create mode 100644 test/cfg2cmd/netdev-7.1.conf
>  create mode 100644 test/cfg2cmd/netdev-7.1.conf.cmd
> 


applied, thanks!

There where some other tests that needed updating for the increased tx/rx Q size
too as they had a net device and weren't fixed to a specific QEMU version.

Oh, and the last patch missed a trailing \ in the penultimate line, I squashed
that in.