[PATCH qemu-server 0/2] use configured serial as SCSI device ID

[PATCH qemu-server 0/2] use configured serial as SCSI device ID

[Michael Köppl]

Since the switch to '-blockdev' with machine version 10.0, SCSI disks
always get the drive name (e.g. 'drive-scsi1') as device ID, even if a
serial is configured. The /dev/disk/by-id paths in the guest therefore
no longer contain the serial, and in a guest running Proxmox VE, the
disk list API endpoint does not report it either.

1/2 sets the device ID explicitly to the configured serial. Unlike
QEMU's default, this also works for serials longer than 20 bytes. The
change is tied to the new machine version 11.1+pve1, which only VMs
with a SCSI disk with a serial require.

2/2 also adds cfg2cmd tests for both sides of the version check.

Tested this on my local test node with explicitly set machine versions
11.1 and 11.1+pve1 as well as latest. Checked that serials longer than
20 bytes are accepted and the VM starts and that with versions older
than 11.1+pve1, the drive name was used, as before.

qemu-server:

Michael Köppl (2):
  drive device: use configured serial as SCSI device ID
  test: cfg2cmd: add tests for SCSI disks with a serial

 src/PVE/QemuServer.pm                      |  7 +++--
 src/PVE/QemuServer/DriveDevice.pm          | 22 ++++++++++++--
 src/PVE/QemuServer/Machine.pm              |  6 ++++
 src/test/cfg2cmd/scsi-serial-11.0.conf     | 10 +++++++
 src/test/cfg2cmd/scsi-serial-11.0.conf.cmd | 33 +++++++++++++++++++++
 src/test/cfg2cmd/scsi-serial.conf          | 10 +++++++
 src/test/cfg2cmd/scsi-serial.conf.cmd      | 34 ++++++++++++++++++++++
 7 files changed, 116 insertions(+), 6 deletions(-)
 create mode 100644 src/test/cfg2cmd/scsi-serial-11.0.conf
 create mode 100644 src/test/cfg2cmd/scsi-serial-11.0.conf.cmd
 create mode 100644 src/test/cfg2cmd/scsi-serial.conf
 create mode 100644 src/test/cfg2cmd/scsi-serial.conf.cmd


Summary over all repositories:
  7 files changed, 116 insertions(+), 6 deletions(-)

--
Generated by murpp 0.12.1

[PATCH qemu-server 1/2] drive device: use configured serial as SCSI device ID

[Michael Köppl]

Since the switch to '-blockdev', the device ID of SCSI disks is
explicitly set to the drive name, even if serial is configured. So with
machine version 10.0 or newer, a SCSI disk with a serial reports the
drive name (e.g. 'drive-scsi1') and the /dev/disk/by-id paths that udev
derives from it no longer contain the serial. In a guest running Proxmox
VE, the disk list API endpoint (/nodes/<node>/disks/list) therefore does
not report the configured serial either.

Set the device ID explicitly to the serial if one is configured, like
libvirt does since its switch to '-blockdev' [0]. For serials of up to
20 bytes, this is the same device ID that QEMU used by default with
'-drive'.

Relying on QEMU's default instead would not work for all serials. QEMU
accepts serials of up to 36 bytes [1], but only uses serials of up to
20 bytes as the default device ID and refuses to start the device for
longer ones instead of truncating them [2][3].

As this changes what the guest sees, only do so starting with machine
version 11.1+pve1. Check it with the version guard, so that only VMs
with a SCSI disk with a serial require the new revision.

[0]: https://gitlab.com/libvirt/libvirt/-/commit/a1dce96236f6d35167924fa7e6a70f58f394b23c
[1]: https://lists.gnu.org/archive/html/qemu-stable/2016-09/msg00117.html
[2]: https://lists.gnu.org/archive/html/qemu-block/2024-06/msg00027.html
[3]: https://github.com/systemd/systemd/issues/39262

Fixes: 46d1225e ("drive device: fix regression with missing '/dev/disk/by-id' paths")

Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
---
Added a machine version 11.1+pve1 specifically for this. If it makes
more sense to wait for 11.2, considering this is not an urgent issue,
I'll of course send a v2. Should be a bit simpler to implement anyway.

 src/PVE/QemuServer.pm             |  7 ++++---
 src/PVE/QemuServer/DriveDevice.pm | 22 +++++++++++++++++++---
 src/PVE/QemuServer/Machine.pm     |  6 ++++++
 3 files changed, 29 insertions(+), 6 deletions(-)

diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 1efeeb66..36471bdf 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -3686,9 +3686,10 @@ sub config_to_command {
                 push @$devices, '-drive', $drive_cmd;
             }

-            push @$devices, '-device', print_drivedevice_full(
-                $storecfg, $conf, $vmid, $drive, $arch, $machine_type,
-            );
+            push @$devices, '-device',
+                print_drivedevice_full(
+                    $storecfg, $conf, $vmid, $drive, $arch, $machine_type, $version_guard,
+                );
         },
     );

diff --git a/src/PVE/QemuServer/DriveDevice.pm b/src/PVE/QemuServer/DriveDevice.pm
index 621fcc2b..980a217a 100644
--- a/src/PVE/QemuServer/DriveDevice.pm
+++ b/src/PVE/QemuServer/DriveDevice.pm
@@ -69,13 +69,23 @@ sub get_max_scsihw_index {
 }

 sub print_drivedevice_full {
-    my ($storecfg, $conf, $vmid, $drive, $arch, $machine_type) = @_;
+    my (
+        $storecfg,
+        $conf,
+        $vmid,
+        $drive,
+        $arch,
+        $machine_type,
+        $version_guard, # checks the machine version and records the required +pveX revision
+    ) = @_;

     my $device = '';
     my $maxdev = 0;

     my $machine_version =
         PVE::QemuServer::Machine::extract_version($machine_type, kvm_user_version());
+    # when hotplugging, the machine is already running, so there is no required version to record
+    $version_guard //= sub { return min_version($machine_version, @_); };
     my $has_write_cache = 1; # whether the device has a 'write-cache' option

     my $use_blockdev = min_version($machine_version, 10, 0);
@@ -112,9 +122,15 @@ sub print_drivedevice_full {
         $device .= ",id=$drive_id";

         # For the switch to -blockdev, the SCSI device ID needs to be explicitly specified. Note
-        # that only ide-cd and ide-hd have a 'device_id' option.
+        # that only ide-cd and ide-hd have a 'device_id' option. Since machine version 11.1+pve1,
+        # a configured serial is used as the device ID, like QEMU's default with -drive.
         if ($use_blockdev && ($device_type eq 'cd' || $device_type eq 'hd')) {
-            $device .= ",device_id=drive-${drive_id}";
+            if ($drive->{serial} && $version_guard->(11, 1, 1)) {
+                my $serial = _escape_qemu_arg_value(URI::Escape::uri_unescape($drive->{serial}));
+                $device .= ",device_id=$serial";
+            } else {
+                $device .= ",device_id=drive-${drive_id}";
+            }
         }

         if ($drive->{ssd} && ($device_type eq 'block' || $device_type eq 'hd')) {
diff --git a/src/PVE/QemuServer/Machine.pm b/src/PVE/QemuServer/Machine.pm
index 35161d1f..68cc5daf 100644
--- a/src/PVE/QemuServer/Machine.pm
+++ b/src/PVE/QemuServer/Machine.pm
@@ -50,6 +50,12 @@ our $PVE_MACHINE_VERSION = {
             '+pve2' => 'Enable hv_emcs for Windows 11/2022/2025 to mitigate issues with VBS.',
         },
     },
+    '11.1' => {
+        highest => 1,
+        revisions => {
+            '+pve1' => 'Use the configured serial as device ID for SCSI disks.',
+        },
+    },
 };

 my $machine_fmt = {
--
2.47.3

[PATCH qemu-server 2/2] test: cfg2cmd: add tests for SCSI disks with a serial

[Michael Köppl]

Cover both sides of the machine version check for the device ID of a
SCSI disk with a serial. Starting from machine version 11.1+pve1, the
serial is used as the device ID, while a machine pinned to an older
version (11.0 in the case of the test) keeps the drive name. The
second disk without a serial ensures that the drive name is still
used as device ID in both cases.

Signed-off-by: Michael Köppl <m.koeppl@proxmox.com>
---
 src/test/cfg2cmd/scsi-serial-11.0.conf     | 10 +++++++
 src/test/cfg2cmd/scsi-serial-11.0.conf.cmd | 33 +++++++++++++++++++++
 src/test/cfg2cmd/scsi-serial.conf          | 10 +++++++
 src/test/cfg2cmd/scsi-serial.conf.cmd      | 34 ++++++++++++++++++++++
 4 files changed, 87 insertions(+)
 create mode 100644 src/test/cfg2cmd/scsi-serial-11.0.conf
 create mode 100644 src/test/cfg2cmd/scsi-serial-11.0.conf.cmd
 create mode 100644 src/test/cfg2cmd/scsi-serial.conf
 create mode 100644 src/test/cfg2cmd/scsi-serial.conf.cmd

diff --git a/src/test/cfg2cmd/scsi-serial-11.0.conf b/src/test/cfg2cmd/scsi-serial-11.0.conf
new file mode 100644
index 00000000..ea10040d
--- /dev/null
+++ b/src/test/cfg2cmd/scsi-serial-11.0.conf
@@ -0,0 +1,10 @@
+# TEST: Test that a SCSI disk with a serial keeps the drive name as device ID with machine version 11.0
+# QEMU_VERSION: 11.1.0
+bootdisk: scsi0
+machine: pc-q35-11.0
+name: simple
+scsi0: local:8006/vm-8006-disk-0.qcow2,serial=disk0,size=104858K
+scsi1: local:8006/vm-8006-disk-1.qcow2,size=104858K
+scsihw: virtio-scsi-pci
+smbios1: uuid=7b10d7af-b932-4c66-b2c3-3996152ec465
+vmgenid: c773c261-d800-4348-9f5d-167fadd53cf8
diff --git a/src/test/cfg2cmd/scsi-serial-11.0.conf.cmd b/src/test/cfg2cmd/scsi-serial-11.0.conf.cmd
new file mode 100644
index 00000000..c7cfb0f1
--- /dev/null
+++ b/src/test/cfg2cmd/scsi-serial-11.0.conf.cmd
@@ -0,0 +1,33 @@
+/usr/bin/kvm \
+  -id 8006 \
+  -name simple \
+  -no-shutdown \
+  -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \
+  -object 'monitor-qmp,chardev=qmp,id=monitor-qmp' \
+  -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' \
+  -object 'monitor-qmp,chardev=qmp-event,id=monitor-qmp-event' \
+  -pidfile /var/run/qemu-server/8006.pid \
+  -daemonize \
+  -smbios 'type=1,uuid=7b10d7af-b932-4c66-b2c3-3996152ec465' \
+  -smp '1,sockets=1,cores=1,maxcpus=1' \
+  -nodefaults \
+  -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
+  -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \
+  -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \
+  -m 512 \
+  -object '{"id":"throttle-drive-scsi0","limits":{},"qom-type":"throttle-group"}' \
+  -object '{"id":"throttle-drive-scsi1","limits":{},"qom-type":"throttle-group"}' \
+  -global 'ICH9-LPC.disable_s3=1' \
+  -global 'ICH9-LPC.disable_s4=1' \
+  -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \
+  -device 'vmgenid,guid=c773c261-d800-4348-9f5d-167fadd53cf8' \
+  -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \
+  -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \
+  -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \
+  -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \
+  -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' \
+  -blockdev '{"detect-zeroes":"on","discard":"ignore","driver":"throttle","file":{"cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"qcow2","file":{"aio":"io_uring","cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"file","filename":"/var/lib/vz/images/8006/vm-8006-disk-0.qcow2","node-name":"ecd04be4259153b8293415fefa2a84c","read-only":false},"node-name":"fcd04be4259153b8293415fefa2a84c","read-only":false},"node-name":"drive-scsi0","read-only":false,"throttle-group":"throttle-drive-scsi0"}' \
+  -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,device_id=drive-scsi0,bootindex=100,serial=disk0,write-cache=on' \
+  -blockdev '{"detect-zeroes":"on","discard":"ignore","driver":"throttle","file":{"cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"qcow2","file":{"aio":"io_uring","cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"file","filename":"/var/lib/vz/images/8006/vm-8006-disk-1.qcow2","node-name":"e70a10e5fb0473d3fc55686f720db61","read-only":false},"node-name":"f70a10e5fb0473d3fc55686f720db61","read-only":false},"node-name":"drive-scsi1","read-only":false,"throttle-group":"throttle-drive-scsi1"}' \
+  -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1,device_id=drive-scsi1,write-cache=on' \
+  -machine 'type=pc-q35-11.0+pve0'
diff --git a/src/test/cfg2cmd/scsi-serial.conf b/src/test/cfg2cmd/scsi-serial.conf
new file mode 100644
index 00000000..12e57b01
--- /dev/null
+++ b/src/test/cfg2cmd/scsi-serial.conf
@@ -0,0 +1,10 @@
+# TEST: Test that a SCSI disk's serial is used as device ID with machine version 11.1+pve1
+# QEMU_VERSION: 11.1.0
+bootdisk: scsi0
+machine: q35
+name: simple
+scsi0: local:8006/vm-8006-disk-0.qcow2,serial=disk0,size=104858K
+scsi1: local:8006/vm-8006-disk-1.qcow2,size=104858K
+scsihw: virtio-scsi-pci
+smbios1: uuid=7b10d7af-b932-4c66-b2c3-3996152ec465
+vmgenid: c773c261-d800-4348-9f5d-167fadd53cf8
diff --git a/src/test/cfg2cmd/scsi-serial.conf.cmd b/src/test/cfg2cmd/scsi-serial.conf.cmd
new file mode 100644
index 00000000..c8f5bc05
--- /dev/null
+++ b/src/test/cfg2cmd/scsi-serial.conf.cmd
@@ -0,0 +1,34 @@
+/usr/bin/kvm \
+  -id 8006 \
+  -name simple \
+  -no-shutdown \
+  -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \
+  -object 'monitor-qmp,chardev=qmp,id=monitor-qmp' \
+  -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' \
+  -object 'monitor-qmp,chardev=qmp-event,id=monitor-qmp-event' \
+  -pidfile /var/run/qemu-server/8006.pid \
+  -daemonize \
+  -smbios 'type=1,uuid=7b10d7af-b932-4c66-b2c3-3996152ec465' \
+  -smp '1,sockets=1,cores=1,maxcpus=1' \
+  -nodefaults \
+  -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
+  -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \
+  -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \
+  -m 512 \
+  -object '{"id":"throttle-drive-scsi0","limits":{},"qom-type":"throttle-group"}' \
+  -object '{"id":"throttle-drive-scsi1","limits":{},"qom-type":"throttle-group"}' \
+  -global 'ICH9-LPC.disable_s3=1' \
+  -global 'ICH9-LPC.disable_s4=1' \
+  -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \
+  -device 'pci-bridge,id=pci.4,chassis_nr=4,bus=pci.1,addr=0x1c' \
+  -device 'vmgenid,guid=c773c261-d800-4348-9f5d-167fadd53cf8' \
+  -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \
+  -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \
+  -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \
+  -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \
+  -device 'virtio-scsi-pci,id=scsihw0,bus=pci.0,addr=0x5' \
+  -blockdev '{"detect-zeroes":"on","discard":"ignore","driver":"throttle","file":{"cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"qcow2","file":{"aio":"io_uring","cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"file","filename":"/var/lib/vz/images/8006/vm-8006-disk-0.qcow2","node-name":"ecd04be4259153b8293415fefa2a84c","read-only":false},"node-name":"fcd04be4259153b8293415fefa2a84c","read-only":false},"node-name":"drive-scsi0","read-only":false,"throttle-group":"throttle-drive-scsi0"}' \
+  -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=0,drive=drive-scsi0,id=scsi0,device_id=disk0,bootindex=100,serial=disk0,write-cache=on' \
+  -blockdev '{"detect-zeroes":"on","discard":"ignore","driver":"throttle","file":{"cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"qcow2","file":{"aio":"io_uring","cache":{"direct":true,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"file","filename":"/var/lib/vz/images/8006/vm-8006-disk-1.qcow2","node-name":"e70a10e5fb0473d3fc55686f720db61","read-only":false},"node-name":"f70a10e5fb0473d3fc55686f720db61","read-only":false},"node-name":"drive-scsi1","read-only":false,"throttle-group":"throttle-drive-scsi1"}' \
+  -device 'scsi-hd,bus=scsihw0.0,channel=0,scsi-id=0,lun=1,drive=drive-scsi1,id=scsi1,device_id=drive-scsi1,write-cache=on' \
+  -machine 'type=q35+pve1'
-- 
2.47.3