From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 021571FF0DF for ; Thu, 15 Oct 2026 10:12:06 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 9DB172160E; Thu, 15 Oct 2026 10:11:36 +0200 (CEST) From: Dominik Csapak To: pve-devel@lists.proxmox.com Subject: [PATCH qemu-server v2 6/9] pci: use PVE::RS::PCI bindings for addresses and bus devices Date: Thu, 15 Oct 2026 10:10:28 +0200 Message-ID: <20261015081041.743422-7-d.csapak@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261015081041.743422-1-d.csapak@proxmox.com> References: <20261015081041.743422-1-d.csapak@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.634 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) KAM_MAILER 2 Automated Mailer Tag Left in Email RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: OIJ7G3BNIV7FRYDIW3MATYENYPFY4ERS X-Message-ID-Hash: OIJ7G3BNIV7FRYDIW3MATYENYPFY4ERS X-MailFrom: d.csapak@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Instead of the hardcoded address maps, use the pve-qemu-server-pci crate via `PVE::RS::PCI::HardwareConfig` for the addresses of all devices, the pci bridges and the root ports of passed through pcie devices. The object contains the relevant parts of the guest config and the machine version (for hotplug the one of the running instance), so we construct it once for the commandline or a hotplug operation and pass it to the functions that need an address, instead of the arch. The functions that also check the machine version still get it separately. The only functional change is that the root ports of the hostpcie devices are now generated together with the bridges, so they move up on the commandline. This also fixes hostpcie devices whose root port had no function 0 in its slot (e.g. with only hostpci5 configured), which the guest did not see. Adapt the hostpci tests accordingly (see [0] for the initial bug). Since this is a configuration that is obviously wrong for the user (so unlikely to occur in the real world), and the fact that pci live migration is still marked experimental, it should be ok to not guard that change behind a machine version. The checks of run_pci_addr_checks.pl are now covered by the tests of the rust crate, so remove it. 0: https://lore.proxmox.com/all/11ce85d1-eb5d-48d1-a4ba-046424613f29@pultar.org/ Fixes: c4e16381 (Add support for up to 16 PCI(e) devices) Signed-off-by: Dominik Csapak --- src/PVE/QemuServer.pm | 192 ++++++---- src/PVE/QemuServer/DriveDevice.pm | 5 +- src/PVE/QemuServer/PCI.pm | 339 ++---------------- src/PVE/QemuServer/RNG.pm | 6 +- src/PVE/QemuServer/USB.pm | 26 +- src/test/Makefile | 5 +- src/test/TestsCommon/CommandLineMocks.pm | 3 + .../q35-linux-hostpci-driver-keep.conf.cmd | 4 +- src/test/cfg2cmd/q35-linux-hostpci-wrong.conf | 16 + .../cfg2cmd/q35-linux-hostpci-wrong.conf.cmd | 41 +++ src/test/cfg2cmd/q35-linux-hostpci.conf.cmd | 4 +- src/test/cfg2cmd/q35-win10-hostpci.conf.cmd | 2 +- src/test/run_pci_addr_checks.pl | 141 -------- 13 files changed, 223 insertions(+), 561 deletions(-) create mode 100644 src/test/cfg2cmd/q35-linux-hostpci-wrong.conf create mode 100644 src/test/cfg2cmd/q35-linux-hostpci-wrong.conf.cmd delete mode 100755 src/test/run_pci_addr_checks.pl diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm index 4a67d3a0..a393db78 100644 --- a/src/PVE/QemuServer.pm +++ b/src/PVE/QemuServer.pm @@ -86,7 +86,7 @@ use PVE::QemuServer::MetaInfo; use PVE::QemuServer::Monitor qw(mon_cmd qmp_cmd vm_qmp_peer); use PVE::QemuServer::Network; use PVE::QemuServer::OVMF; -use PVE::QemuServer::PCI qw(print_pci_addr print_pcie_addr parse_hostpci get_pci_bridges); +use PVE::QemuServer::PCI qw(parse_hostpci); use PVE::QemuServer::PCI::Mdev; use PVE::QemuServer::QemuImage; use PVE::QemuServer::QMPHelpers @@ -1180,17 +1180,17 @@ sub assert_clipboard_config { } sub print_tabletdevice_full { - my ($conf, $arch) = @_; + my ($hw_config) = @_; - my $usbbus = PVE::QemuServer::USB::get_default_usb_bus($conf, $arch); + my $usbbus = PVE::QemuServer::USB::get_default_usb_bus($hw_config); return "usb-tablet,id=tablet,bus=$usbbus.0,port=1"; } sub print_keyboarddevice_full { - my ($conf, $arch) = @_; + my ($hw_config) = @_; - return if $arch ne 'aarch64'; + return if $hw_config->arch() ne 'aarch64'; return "usb-kbd,id=keyboard,bus=ehci.0,port=2"; } @@ -1318,7 +1318,7 @@ sub print_netdevice_full { $net, $netid, $use_old_bios_files, - $arch, + $hw_config, $version_guard, # checks the machine version and records the required +pveX revision $host_mtu_migration, # force this value for host_mtu, 0 means force absence of param ) = @_; @@ -1328,7 +1328,7 @@ sub print_netdevice_full { $device = 'virtio-net-pci'; } - my $pciaddr = print_pci_addr("$netid", $arch); + my $pciaddr = $hw_config->addr("$netid"); my $tmpstr = "$device,mac=$net->{macaddr},netdev=$netid$pciaddr,id=$netid"; if ($net->{queues} && $net->{queues} > 1 && $net->{model} eq 'virtio') { # Consider we have N queues, the number of vectors needed is 2 * N + 2, i.e., one per in @@ -1488,9 +1488,9 @@ my sub map_vga_model { } sub print_vga_device { - my ($conf, $vga, $arch, $machine_version, $id, $qxlnum) = @_; + my ($conf, $vga, $hw_config, $machine_version, $id, $qxlnum) = @_; - my $type = map_vga_model($vga->{type}, $arch); + my $type = map_vga_model($vga->{type}, $hw_config->arch()); my $vgamem_mb = $vga->{memory}; my $max_outputs = ''; @@ -1530,15 +1530,8 @@ sub print_vga_device { $edidoff = ",edid=off" if (!defined($conf->{bios}) || $conf->{bios} ne 'ovmf'); } - my $q35 = PVE::QemuServer::Machine::machine_type_is_q35($conf); my $vgaid = "vga" . ($id // ''); - my $pciaddr; - if ($q35 && $vgaid eq 'vga') { - # the first display uses pcie.0 bus on q35 machines - $pciaddr = print_pcie_addr($vgaid); - } else { - $pciaddr = print_pci_addr($vgaid, $arch); - } + my $pciaddr = $hw_config->addr($vgaid); if ($vga->{type} eq 'virtio-gl') { # VirGL renders on the host, so the guest arch is irrelevant for the library lookup @@ -2810,10 +2803,11 @@ sub conf_has_audio { } sub audio_devs { - my ($audio, $audiopciaddr, $machine_version) = @_; + my ($audio, $hw_config, $machine_version) = @_; my $devs = []; + my $audiopciaddr = $hw_config->addr("audio0"); my $id = $audio->{dev_id}; my $audiodev = ""; if (min_version($machine_version, 4, 2)) { @@ -3267,6 +3261,11 @@ sub config_to_command { push $machineFlags->@*, $ovmf_machine_flags->@*; } + my $max_scsihw = PVE::QemuServer::DriveDevice::get_max_scsihw_index($conf); + my $hw_config = PVE::QemuServer::PCI::hardware_config( + $conf, $arch, $max_scsihw, $machine_version, $forcemachine, + ); + if ($q35) { # tell QEMU to load q35 config early # we use different pcie-port hardware for qemu >= 4.0 for passthrough if (min_version($machine_version, 4, 0)) { @@ -3280,9 +3279,8 @@ sub config_to_command { push @$cmd, $fixups->@*; } - my $max_scsihw = PVE::QemuServer::DriveDevice::get_max_scsihw_index($conf); - if (my $bridges = get_pci_bridges($conf, $arch, $q35, $max_scsihw, $version_guard)) { - push @$devices, $bridges->@*; + if (my $bus_devices = $hw_config->pci_bus_devices()) { + push @$devices, $bus_devices->@*; } if ($conf->{vmgenid}) { @@ -3290,7 +3288,8 @@ sub config_to_command { } # add usb controllers - my @usbcontrollers = PVE::QemuServer::USB::get_usb_controllers($conf, $arch, $machine_version); + my @usbcontrollers = + PVE::QemuServer::USB::get_usb_controllers($conf, $hw_config, $machine_version); push @$devices, @usbcontrollers if @usbcontrollers; my ($vga, $qxlnum) = get_vga_properties($conf, $arch, $machine_version, $winversion); @@ -3304,8 +3303,8 @@ sub config_to_command { } if ($tablet) { - push @$devices, '-device', print_tabletdevice_full($conf, $arch) if $tablet; - my $kbd = print_keyboarddevice_full($conf, $arch); + push @$devices, '-device', print_tabletdevice_full($hw_config) if $tablet; + my $kbd = print_keyboarddevice_full($hw_config); push @$devices, '-device', $kbd if defined($kbd); } @@ -3314,7 +3313,7 @@ sub config_to_command { # host pci device passthrough my ($kvm_off, $gpu_passthrough, $legacy_igd, $pci_devices) = PVE::QemuServer::PCI::print_hostpci_devices( - $vmid, $conf, $devices, $vga, $winversion, $arch, $bootorder, $dry_run, + $vmid, $conf, $devices, $vga, $winversion, $hw_config, $bootorder, $dry_run, ); my @usbdevices = PVE::QemuServer::USB::get_usb_devices($conf, $bootorder, $machine_version); @@ -3352,8 +3351,7 @@ sub config_to_command { } if (min_version($machine_version, 4, 0) && (my $audio = conf_has_audio($conf))) { - my $audiopciaddr = print_pci_addr("audio0", $arch); - my $audio_devs = audio_devs($audio, $audiopciaddr, $machine_version); + my $audio_devs = audio_devs($audio, $hw_config, $machine_version); push @$devices, @$audio_devs; } @@ -3397,7 +3395,7 @@ sub config_to_command { if ($vga->{type} && $vga->{type} !~ m/^serial\d+$/ && $vga->{type} ne 'none') { push @$devices, '-device', - print_vga_device($conf, $vga, $arch, $machine_version, undef, $qxlnum); + print_vga_device($conf, $vga, $hw_config, $machine_version, undef, $qxlnum); push @$cmd, '-display', 'egl-headless,gl=core' if $vga->{type} eq 'virtio-gl'; # VIRGL @@ -3457,7 +3455,7 @@ sub config_to_command { push @$devices, '-chardev', "socket,path=$qgasocket,server=on,wait=off,id=qga0"; if (!$guest_agent->{type} || $guest_agent->{type} eq 'virtio') { - my $pciaddr = print_pci_addr("qga0", $arch); + my $pciaddr = $hw_config->addr("qga0"); push @$devices, '-device', "virtio-serial,id=qga0$pciaddr"; push @$devices, '-device', 'virtserialport,chardev=qga0,name=org.qemu.guest_agent.0'; } elsif ($guest_agent->{type} eq 'isa') { @@ -3468,7 +3466,7 @@ sub config_to_command { my $rng = $conf->{rng0} ? parse_rng($conf->{rng0}) : undef; if ($rng && $version_guard->(4, 1, 2)) { my $rng_object = print_rng_object_commandline('rng0', $rng); - my $rng_device = print_rng_device_commandline('rng0', $rng, $arch); + my $rng_device = print_rng_device_commandline('rng0', $rng, $hw_config); push @$devices, '-object', $rng_object; push @$devices, '-device', $rng_device; } @@ -3482,9 +3480,8 @@ sub config_to_command { if ($qxlnum > 1) { if ($winversion) { for (my $i = 1; $i < $qxlnum; $i++) { - push @$devices, '-device', print_vga_device( - $conf, $vga, $arch, $machine_version, $i, $qxlnum, - ); + push @$devices, '-device', + print_vga_device($conf, $vga, $hw_config, $machine_version, $i, $qxlnum); } } else { # assume other OS works like Linux @@ -3498,7 +3495,7 @@ sub config_to_command { } } - my $pciaddr = print_pci_addr("spice", $arch); + my $pciaddr = $hw_config->addr("spice"); push @$devices, '-device', "virtio-serial,id=spice$pciaddr"; if ($vga->{'clipboard'} && $vga->{'clipboard'} eq 'vnc') { @@ -3536,7 +3533,7 @@ sub config_to_command { # enable balloon by default, unless explicitly disabled if (!defined($conf->{balloon}) || $conf->{balloon}) { - my $pciaddr = print_pci_addr("balloon0", $arch); + my $pciaddr = $hw_config->addr("balloon0"); my $ballooncmd = "virtio-balloon-pci,id=balloon0$pciaddr"; $ballooncmd .= ",free-page-reporting=on" if min_version($machine_version, 6, 2); push @$devices, '-device', $ballooncmd; @@ -3544,7 +3541,7 @@ sub config_to_command { if ($conf->{watchdog}) { my $wdopts = parse_watchdog($conf->{watchdog}); - my $pciaddr = print_pci_addr("watchdog", $arch); + my $pciaddr = $hw_config->addr("watchdog"); my $watchdog = $wdopts->{model} || 'i6300esb'; push @$devices, '-device', "$watchdog$pciaddr"; push @$devices, '-watchdog-action', $wdopts->{action} if $wdopts->{action}; @@ -3586,7 +3583,7 @@ sub config_to_command { "scsi$drive->{index}: machine version 4.1~pve2 or higher is required to use more than 14 SCSI disks\n" if $drive->{index} > 13 && !&$version_guard(4, 1, 2); - my $pciaddr = print_pci_addr("$controller_prefix$controller", $arch); + my $pciaddr = $hw_config->addr("$controller_prefix$controller"); my $scsihw_type = $scsihw =~ m/^virtio-scsi-single/ ? "virtio-scsi-pci" : $scsihw; @@ -3621,7 +3618,7 @@ sub config_to_command { if ($drive->{interface} eq 'sata') { my $controller = int($drive->{index} / $PVE::QemuServer::Drive::MAX_SATA_DISKS); - my $pciaddr = print_pci_addr("ahci$controller", $arch); + my $pciaddr = $hw_config->addr("ahci$controller"); push @$devices, '-device', "ahci,id=ahci$controller,multifunction=on$pciaddr" if !$ahcicontroller->{$controller}; $ahcicontroller->{$controller} = 1; @@ -3666,7 +3663,7 @@ sub config_to_command { } push @$devices, '-device', print_drivedevice_full( - $storecfg, $conf, $vmid, $drive, $arch, $machine_type, + $storecfg, $conf, $vmid, $drive, $hw_config, $machine_type, ); }, ); @@ -3696,7 +3693,7 @@ sub config_to_command { $d, $netname, $use_old_bios_files, - $arch, + $hw_config, $version_guard, $nets_host_mtu->{$netname}, ); @@ -3707,12 +3704,7 @@ sub config_to_command { if ($conf->{ivshmem}) { my $ivshmem = parse_property_string($ivshmem_fmt, $conf->{ivshmem}); - my $bus; - if ($q35) { - $bus = print_pcie_addr("ivshmem"); - } else { - $bus = print_pci_addr("ivshmem", $arch); - } + my $bus = $hw_config->addr("ivshmem"); my $ivshmem_name = $ivshmem->{name} // $vmid; my $path = '/dev/shm/pve-shm-' . $ivshmem_name; @@ -3859,18 +3851,18 @@ sub vm_devices_list { } sub vm_deviceplug { - my ($storecfg, $conf, $vmid, $deviceid, $device, $arch, $machine_type) = @_; + my ($storecfg, $conf, $vmid, $deviceid, $device, $hw_config, $machine_type) = @_; my $devices_list = vm_devices_list($vmid); return 1 if defined($devices_list->{$deviceid}); # we can't hotplug bridges, so check if the necessary one exists - assert_pci_bridge_present($vmid, $deviceid); + assert_pci_bridge_present($vmid, $deviceid, $hw_config); if ($deviceid eq 'tablet') { - qemu_deviceadd($vmid, print_tabletdevice_full($conf, $arch)); + qemu_deviceadd($vmid, print_tabletdevice_full($hw_config)); } elsif ($deviceid eq 'keyboard') { - qemu_deviceadd($vmid, print_keyboarddevice_full($conf, $arch)); + qemu_deviceadd($vmid, print_keyboarddevice_full($hw_config)); } elsif ($deviceid =~ m/^usbredirdev(\d+)$/) { my $id = $1; qemu_spice_usbredir_chardev_add($vmid, "usbredirchardev$id"); @@ -3885,7 +3877,7 @@ sub vm_deviceplug { qemu_driveadd($storecfg, $vmid, $device); my $devicefull = - print_drivedevice_full($storecfg, $conf, $vmid, $device, $arch, $machine_type); + print_drivedevice_full($storecfg, $conf, $vmid, $device, $hw_config, $machine_type); qemu_deviceadd($vmid, $devicefull); eval { qemu_deviceaddverify($vmid, $deviceid); }; @@ -3898,7 +3890,7 @@ sub vm_deviceplug { } } elsif ($deviceid =~ m/^(virtioscsi|scsihw)(\d+)$/) { my $scsihw = defined($conf->{scsihw}) ? $conf->{scsihw} : "lsi"; - my $pciaddr = print_pci_addr($deviceid, $arch); + my $pciaddr = $hw_config->addr($deviceid); my $scsihw_type = $scsihw eq 'virtio-scsi-single' ? "virtio-scsi-pci" : $scsihw; my $devicefull = "$scsihw_type,id=$deviceid$pciaddr"; @@ -3922,11 +3914,11 @@ sub vm_deviceplug { die $err; } } elsif ($deviceid =~ m/^(scsi)(\d+)$/) { - qemu_findorcreatescsihw($storecfg, $conf, $vmid, $device, $arch, $machine_type); + qemu_findorcreatescsihw($storecfg, $conf, $vmid, $device, $hw_config, $machine_type); qemu_driveadd($storecfg, $vmid, $device); my $devicefull = - print_drivedevice_full($storecfg, $conf, $vmid, $device, $arch, $machine_type); + print_drivedevice_full($storecfg, $conf, $vmid, $device, $hw_config, $machine_type); eval { qemu_deviceadd($vmid, $devicefull); }; if (my $err = $@) { eval { qemu_drivedel($vmid, $deviceid); }; @@ -3934,7 +3926,7 @@ sub vm_deviceplug { die $err; } } elsif ($deviceid =~ m/^(net)(\d+)$/) { - return if !qemu_netdevadd($vmid, $conf, $arch, $device, $deviceid); + return if !qemu_netdevadd($vmid, $conf, $hw_config->arch(), $device, $deviceid); my $machine_type = PVE::QemuServer::Machine::qemu_machine_pxe($vmid, $conf); my $machine_version = PVE::QemuServer::Machine::extract_version($machine_type); @@ -3947,7 +3939,13 @@ sub vm_deviceplug { return min_version($machine_version, $major, $minor, $pve); }; my $netdevicefull = print_netdevice_full( - $vmid, $conf, $device, $deviceid, $use_old_bios_files, $arch, $version_guard, + $vmid, + $conf, + $device, + $deviceid, + $use_old_bios_files, + $hw_config, + $version_guard, ); qemu_deviceadd($vmid, $netdevicefull); eval { @@ -4129,7 +4127,7 @@ sub qemu_devicedelverify { } sub qemu_findorcreatescsihw { - my ($storecfg, $conf, $vmid, $device, $arch, $machine_type) = @_; + my ($storecfg, $conf, $vmid, $device, $hw_config, $machine_type) = @_; my ($maxdev, $controller, $controller_prefix) = scsihw_infos($conf->{scsihw}, $device->{index}); @@ -4137,7 +4135,7 @@ sub qemu_findorcreatescsihw { my $devices_list = vm_devices_list($vmid); if (!defined($devices_list->{$scsihwid})) { - vm_deviceplug($storecfg, $conf, $vmid, $scsihwid, $device, $arch, $machine_type); + vm_deviceplug($storecfg, $conf, $vmid, $scsihwid, $device, $hw_config, $machine_type); } return 1; @@ -4176,12 +4174,11 @@ sub qemu_deletescsihw { } sub assert_pci_bridge_present { - my ($vmid, $device) = @_; + my ($vmid, $device, $hw_config) = @_; - my $bridgeid = PVE::QemuServer::PCI::get_pci_bridge_for_device($device); - return 1 if !defined($bridgeid) || $bridgeid < 1; + my $bridge = $hw_config->pci_bridge_for_device($device); + return 1 if !defined($bridge) || $bridge eq 'pci.0' || $bridge eq 'pcie.0'; - my $bridge = "pci.$bridgeid"; my $devices_list = vm_devices_list($vmid); if (!defined($devices_list->{$bridge})) { @@ -4226,7 +4223,7 @@ sub qemu_netdevdel { } sub qemu_usb_hotplug { - my ($storecfg, $conf, $vmid, $deviceid, $device, $arch, $machine_type) = @_; + my ($storecfg, $conf, $vmid, $deviceid, $device, $hw_config, $machine_type) = @_; return if !$device; @@ -4238,13 +4235,13 @@ sub qemu_usb_hotplug { my $devicelist = vm_devices_list($vmid); if (!$devicelist->{xhci}) { - my $pciaddr = print_pci_addr("xhci", $arch); + my $pciaddr = $hw_config->addr("xhci"); qemu_deviceadd($vmid, PVE::QemuServer::USB::print_qemu_xhci_controller($pciaddr)); $added_xhci = 1; } # add the new one - eval { vm_deviceplug($storecfg, $conf, $vmid, $deviceid, $device, $arch, $machine_type); }; + eval { vm_deviceplug($storecfg, $conf, $vmid, $deviceid, $device, $hw_config, $machine_type); }; if (my $err = $@) { if ($added_xhci) { eval { vm_deviceunplug($vmid, $conf, 'xhci'); }; @@ -4577,6 +4574,21 @@ sub vmconfig_hotplug_pending { return $_machine_type; }; + my $_hw_config; + my $hw_config = sub { + return $_hw_config if defined($_hw_config); + + my $max_scsihw = PVE::QemuServer::DriveDevice::get_max_scsihw_index($conf); + $_hw_config = PVE::QemuServer::PCI::hardware_config( + $conf, + $arch, + $max_scsihw, + PVE::QemuServer::Machine::extract_version($machine_type->()), + ); + + return $_hw_config; + }; + # commit values which do not have any impact on running VM first # Note: those option cannot raise errors, we we do not care about # $selection and always apply them. @@ -4696,10 +4708,22 @@ sub vmconfig_hotplug_pending { die "skip\n" if !$hotplug_features->{usb}; if ($defaults->{tablet}) { vm_deviceplug( - $storecfg, $conf, $vmid, 'tablet', undef, $arch, $machine_type->(), + $storecfg, + $conf, + $vmid, + 'tablet', + undef, + $hw_config->(), + $machine_type->(), ); vm_deviceplug( - $storecfg, $conf, $vmid, 'keyboard', undef, $arch, $machine_type->(), + $storecfg, + $conf, + $vmid, + 'keyboard', + undef, + $hw_config->(), + $machine_type->(), ) if $arch eq 'aarch64'; } else { vm_deviceunplug($vmid, $conf, 'tablet'); @@ -4770,10 +4794,22 @@ sub vmconfig_hotplug_pending { die "skip\n" if !$hotplug_features->{usb}; if ($value == 1) { vm_deviceplug( - $storecfg, $conf, $vmid, 'tablet', undef, $arch, $machine_type->(), + $storecfg, + $conf, + $vmid, + 'tablet', + undef, + $hw_config->(), + $machine_type->(), ); vm_deviceplug( - $storecfg, $conf, $vmid, 'keyboard', undef, $arch, $machine_type->(), + $storecfg, + $conf, + $vmid, + 'keyboard', + undef, + $hw_config->(), + $machine_type->(), ) if $arch eq 'aarch64'; } elsif ($value == 0) { vm_deviceunplug($vmid, $conf, 'tablet'); @@ -4787,7 +4823,9 @@ sub vmconfig_hotplug_pending { if ($d->{host} =~ m/^spice$/i) { $id = "usbredirdev$index"; } - qemu_usb_hotplug($storecfg, $conf, $vmid, $id, $d, $arch, $machine_type->()); + qemu_usb_hotplug( + $storecfg, $conf, $vmid, $id, $d, $hw_config->(), $machine_type->(), + ); } elsif ($opt eq 'vcpus') { die "skip\n" if !$hotplug_features->{cpu}; qemu_cpu_hotplug($vmid, $conf, $value); @@ -4813,7 +4851,7 @@ sub vmconfig_hotplug_pending { $vmid, $opt, $value, - $arch, + $hw_config->(), $machine_type->(), ); } elsif (is_valid_drivename($opt)) { @@ -4832,7 +4870,7 @@ sub vmconfig_hotplug_pending { $vmid, $opt, $value, - $arch, + $hw_config->(), $machine_type->(), ); } elsif ($opt =~ m/^memory$/) { #dimms @@ -4873,7 +4911,7 @@ sub vmconfig_hotplug_pending { $vmid, $opt, $value, - $arch, + $hw_config->(), $machine_type->(), ); }; @@ -5068,7 +5106,7 @@ sub vmconfig_apply_pending { } sub vmconfig_update_net { - my ($storecfg, $conf, $hotplug, $vmid, $opt, $value, $arch, $machine_type) = @_; + my ($storecfg, $conf, $hotplug, $vmid, $opt, $value, $hw_config, $machine_type) = @_; my $newnet = PVE::QemuServer::Network::parse_net($value); @@ -5160,7 +5198,7 @@ sub vmconfig_update_net { PVE::Network::SDN::Vnets::add_dhcp_mapping( $newnet->{bridge}, $newnet->{macaddr}, $vmid, $conf->{name}, ); - vm_deviceplug($storecfg, $conf, $vmid, $opt, $newnet, $arch, $machine_type); + vm_deviceplug($storecfg, $conf, $vmid, $opt, $newnet, $hw_config, $machine_type); } else { die "skip\n"; } @@ -5190,7 +5228,7 @@ sub vmconfig_update_agent { } sub vmconfig_update_disk { - my ($storecfg, $conf, $hotplug, $vmid, $opt, $value, $arch, $machine_type) = @_; + my ($storecfg, $conf, $hotplug, $vmid, $opt, $value, $hw_config, $machine_type) = @_; my $drive = parse_drive($opt, $value); @@ -5296,7 +5334,7 @@ sub vmconfig_update_disk { die "skip\n" if !$hotplug || $opt =~ m/(ide|sata)(\d+)/; # hotplug new disks PVE::Storage::activate_volumes($storecfg, [$drive->{file}]) if $drive->{file} !~ m|^/dev/.+|; - vm_deviceplug($storecfg, $conf, $vmid, $opt, $drive, $arch, $machine_type); + vm_deviceplug($storecfg, $conf, $vmid, $opt, $drive, $hw_config, $machine_type); } sub vmconfig_update_cloudinit_drive { diff --git a/src/PVE/QemuServer/DriveDevice.pm b/src/PVE/QemuServer/DriveDevice.pm index 621fcc2b..28588433 100644 --- a/src/PVE/QemuServer/DriveDevice.pm +++ b/src/PVE/QemuServer/DriveDevice.pm @@ -8,7 +8,6 @@ use URI::Escape; use PVE::QemuServer::Drive qw (drive_is_cdrom); use PVE::QemuServer::Helpers qw(kvm_user_version min_version); use PVE::QemuServer::Machine; -use PVE::QemuServer::PCI qw(print_pci_addr); use base qw(Exporter); @@ -69,7 +68,7 @@ sub get_max_scsihw_index { } sub print_drivedevice_full { - my ($storecfg, $conf, $vmid, $drive, $arch, $machine_type) = @_; + my ($storecfg, $conf, $vmid, $drive, $hw_config, $machine_type) = @_; my $device = ''; my $maxdev = 0; @@ -82,7 +81,7 @@ sub print_drivedevice_full { my $drive_id = PVE::QemuServer::Drive::get_drive_id($drive); if ($drive->{interface} eq 'virtio') { - my $pciaddr = print_pci_addr("$drive_id", $arch); + my $pciaddr = $hw_config->addr("$drive_id"); $device = 'virtio-blk-pci'; # for the switch to -blockdev, there is no blockdev for 'none' if (!$use_blockdev || $drive->{file} ne 'none') { diff --git a/src/PVE/QemuServer/PCI.pm b/src/PVE/QemuServer/PCI.pm index d7dc6121..6b4eb179 100644 --- a/src/PVE/QemuServer/PCI.pm +++ b/src/PVE/QemuServer/PCI.pm @@ -7,6 +7,7 @@ use IO::File; use PVE::JSONSchema; use PVE::Mapping::PCI; +use PVE::RS::PCI::HardwareConfig; use PVE::SysFSTools; use PVE::Tools; @@ -17,10 +18,7 @@ use PVE::QemuServer::PCI::Mdev; use base 'Exporter'; our @EXPORT_OK = qw( - print_pci_addr - print_pcie_addr parse_hostpci - get_pci_bridges ); our $MAX_HOSTPCI_DEVICES = 16; @@ -152,266 +150,6 @@ EODESCR }; PVE::JSONSchema::register_standard_option("pve-qm-hostpci", $hostpcidesc); -my $pci_addr_map; - -sub get_pci_addr_map { - $pci_addr_map = { - piix3 => { bus => 0, addr => 1, conflict_ok => qw(ehci) }, - ehci => { bus => 0, addr => 1, conflict_ok => qw(piix3) }, # instead of piix3 on arm - vga => { bus => 0, addr => 2, conflict_ok => qw(legacy-igd) }, - 'legacy-igd' => { bus => 0, addr => 2, conflict_ok => qw(vga) }, # legacy-igd requires vga=none - balloon0 => { bus => 0, addr => 3 }, - watchdog => { bus => 0, addr => 4 }, - scsihw0 => { bus => 0, addr => 5, conflict_ok => qw(pci.3) }, - 'pci.3' => { bus => 0, addr => 5, conflict_ok => qw(scsihw0) }, # also used for virtio-scsi-single bridge - scsihw1 => { bus => 0, addr => 6 }, - ahci0 => { bus => 0, addr => 7 }, - qga0 => { bus => 0, addr => 8 }, - spice => { bus => 0, addr => 9 }, - virtio0 => { bus => 0, addr => 10 }, - virtio1 => { bus => 0, addr => 11 }, - virtio2 => { bus => 0, addr => 12 }, - virtio3 => { bus => 0, addr => 13 }, - virtio4 => { bus => 0, addr => 14 }, - virtio5 => { bus => 0, addr => 15 }, - hostpci0 => { bus => 0, addr => 16 }, - hostpci1 => { bus => 0, addr => 17 }, - net0 => { bus => 0, addr => 18 }, - net1 => { bus => 0, addr => 19 }, - net2 => { bus => 0, addr => 20 }, - net3 => { bus => 0, addr => 21 }, - net4 => { bus => 0, addr => 22 }, - net5 => { bus => 0, addr => 23 }, - vga1 => { bus => 0, addr => 24 }, - vga2 => { bus => 0, addr => 25 }, - vga3 => { bus => 0, addr => 26 }, - hostpci2 => { bus => 0, addr => 27 }, - hostpci3 => { bus => 0, addr => 28 }, - #addr29 : usb-host (pve-usb.cfg) - 'pci.1' => { bus => 0, addr => 30 }, - 'pci.2' => { bus => 0, addr => 31 }, - 'net6' => { bus => 1, addr => 1 }, - 'net7' => { bus => 1, addr => 2 }, - 'net8' => { bus => 1, addr => 3 }, - 'net9' => { bus => 1, addr => 4 }, - 'net10' => { bus => 1, addr => 5 }, - 'net11' => { bus => 1, addr => 6 }, - 'net12' => { bus => 1, addr => 7 }, - 'net13' => { bus => 1, addr => 8 }, - 'net14' => { bus => 1, addr => 9 }, - 'net15' => { bus => 1, addr => 10 }, - 'net16' => { bus => 1, addr => 11 }, - 'net17' => { bus => 1, addr => 12 }, - 'net18' => { bus => 1, addr => 13 }, - 'net19' => { bus => 1, addr => 14 }, - 'net20' => { bus => 1, addr => 15 }, - 'net21' => { bus => 1, addr => 16 }, - 'net22' => { bus => 1, addr => 17 }, - 'net23' => { bus => 1, addr => 18 }, - 'net24' => { bus => 1, addr => 19 }, - 'net25' => { bus => 1, addr => 20 }, - 'net26' => { bus => 1, addr => 21 }, - 'net27' => { bus => 1, addr => 22 }, - 'net28' => { bus => 1, addr => 23 }, - 'net29' => { bus => 1, addr => 24 }, - 'net30' => { bus => 1, addr => 25 }, - 'net31' => { bus => 1, addr => 26 }, - 'xhci' => { bus => 1, addr => 27 }, - 'pci.4' => { bus => 1, addr => 28 }, - 'rng0' => { bus => 1, addr => 29 }, - 'pci.2-igd' => { bus => 1, addr => 30 }, # replaces pci.2 in case a legacy IGD device is passed through - 'virtio6' => { bus => 2, addr => 1 }, - 'virtio7' => { bus => 2, addr => 2 }, - 'virtio8' => { bus => 2, addr => 3 }, - 'virtio9' => { bus => 2, addr => 4 }, - 'virtio10' => { bus => 2, addr => 5 }, - 'virtio11' => { bus => 2, addr => 6 }, - 'virtio12' => { bus => 2, addr => 7 }, - 'virtio13' => { bus => 2, addr => 8 }, - 'virtio14' => { bus => 2, addr => 9 }, - 'virtio15' => { bus => 2, addr => 10 }, - 'ivshmem' => { bus => 2, addr => 11 }, - 'audio0' => { bus => 2, addr => 12 }, - hostpci4 => { bus => 2, addr => 13 }, - hostpci5 => { bus => 2, addr => 14 }, - hostpci6 => { bus => 2, addr => 15 }, - hostpci7 => { bus => 2, addr => 16 }, - hostpci8 => { bus => 2, addr => 17 }, - hostpci9 => { bus => 2, addr => 18 }, - hostpci10 => { bus => 2, addr => 19 }, - hostpci11 => { bus => 2, addr => 20 }, - hostpci12 => { bus => 2, addr => 21 }, - hostpci13 => { bus => 2, addr => 22 }, - hostpci14 => { bus => 2, addr => 23 }, - hostpci15 => { bus => 2, addr => 24 }, - 'virtioscsi0' => { bus => 3, addr => 1 }, - 'virtioscsi1' => { bus => 3, addr => 2 }, - 'virtioscsi2' => { bus => 3, addr => 3 }, - 'virtioscsi3' => { bus => 3, addr => 4 }, - 'virtioscsi4' => { bus => 3, addr => 5 }, - 'virtioscsi5' => { bus => 3, addr => 6 }, - 'virtioscsi6' => { bus => 3, addr => 7 }, - 'virtioscsi7' => { bus => 3, addr => 8 }, - 'virtioscsi8' => { bus => 3, addr => 9 }, - 'virtioscsi9' => { bus => 3, addr => 10 }, - 'virtioscsi10' => { bus => 3, addr => 11 }, - 'virtioscsi11' => { bus => 3, addr => 12 }, - 'virtioscsi12' => { bus => 3, addr => 13 }, - 'virtioscsi13' => { bus => 3, addr => 14 }, - 'virtioscsi14' => { bus => 3, addr => 15 }, - 'virtioscsi15' => { bus => 3, addr => 16 }, - 'virtioscsi16' => { bus => 3, addr => 17 }, - 'virtioscsi17' => { bus => 3, addr => 18 }, - 'virtioscsi18' => { bus => 3, addr => 19 }, - 'virtioscsi19' => { bus => 3, addr => 20 }, - 'virtioscsi20' => { bus => 3, addr => 21 }, - 'virtioscsi21' => { bus => 3, addr => 22 }, - 'virtioscsi22' => { bus => 3, addr => 23 }, - 'virtioscsi23' => { bus => 3, addr => 24 }, - 'virtioscsi24' => { bus => 3, addr => 25 }, - 'virtioscsi25' => { bus => 3, addr => 26 }, - 'virtioscsi26' => { bus => 3, addr => 27 }, - 'virtioscsi27' => { bus => 3, addr => 28 }, - 'virtioscsi28' => { bus => 3, addr => 29 }, - 'virtioscsi29' => { bus => 3, addr => 30 }, - 'virtioscsi30' => { bus => 3, addr => 31 }, - 'scsihw2' => { bus => 4, addr => 1 }, - 'scsihw3' => { bus => 4, addr => 2 }, - 'scsihw4' => { bus => 4, addr => 3 }, - } - if !defined($pci_addr_map); - return $pci_addr_map; -} - -my $get_addr_mapping_from_id = sub { - my ($map, $id) = @_; - - my $d = $map->{$id}; - return if !defined($d) || !defined($d->{bus}) || !defined($d->{addr}); - - return { bus => $d->{bus}, addr => sprintf("0x%x", $d->{addr}) }; -}; - -sub get_pci_bridge_for_device { - my ($id) = @_; - - my $map = get_pci_addr_map(); - if (my $d = $get_addr_mapping_from_id->($map, $id)) { - return $d->{bus}; - } - - return; -} - -sub print_pci_addr { - my ($id, $arch) = @_; - - die "aarch64 cannot use IDE devices\n" if $arch eq 'aarch64' && $id =~ /^ide/; - - my $res = ''; - - my $map = get_pci_addr_map(); - if (my $d = $get_addr_mapping_from_id->($map, $id)) { - # Using same bus slots on all HW, so we need to check special cases here. For aarch64, the - # virt machine has an initial pcie.0. The other pci bridges that get added are called pci.N. - my $busname = $arch eq 'aarch64' && $d->{bus} eq 0 ? 'pcie' : 'pci'; - - $res = ",bus=$busname.$d->{bus},addr=$d->{addr}"; - } - - return $res; -} - -my $pcie_addr_map; - -sub get_pcie_addr_map { - $pcie_addr_map = { - vga => { bus => 'pcie.0', addr => 1 }, - hostpci0 => { bus => "ich9-pcie-port-1", addr => 0 }, - hostpci1 => { bus => "ich9-pcie-port-2", addr => 0 }, - hostpci2 => { bus => "ich9-pcie-port-3", addr => 0 }, - hostpci3 => { bus => "ich9-pcie-port-4", addr => 0 }, - hostpci4 => { bus => "ich9-pcie-port-5", addr => 0 }, - hostpci5 => { bus => "ich9-pcie-port-6", addr => 0 }, - hostpci6 => { bus => "ich9-pcie-port-7", addr => 0 }, - hostpci7 => { bus => "ich9-pcie-port-8", addr => 0 }, - hostpci8 => { bus => "ich9-pcie-port-9", addr => 0 }, - hostpci9 => { bus => "ich9-pcie-port-10", addr => 0 }, - hostpci10 => { bus => "ich9-pcie-port-11", addr => 0 }, - hostpci11 => { bus => "ich9-pcie-port-12", addr => 0 }, - hostpci12 => { bus => "ich9-pcie-port-13", addr => 0 }, - hostpci13 => { bus => "ich9-pcie-port-14", addr => 0 }, - hostpci14 => { bus => "ich9-pcie-port-15", addr => 0 }, - hostpci15 => { bus => "ich9-pcie-port-16", addr => 0 }, - # win7 is picky about pcie assignments - hostpci0bus0 => { bus => "pcie.0", addr => 16 }, - hostpci1bus0 => { bus => "pcie.0", addr => 17 }, - hostpci2bus0 => { bus => "pcie.0", addr => 18 }, - hostpci3bus0 => { bus => "pcie.0", addr => 19 }, - ivshmem => { bus => 'pcie.0', addr => 20 }, - hostpci4bus0 => { bus => "pcie.0", addr => 9 }, - hostpci5bus0 => { bus => "pcie.0", addr => 10 }, - hostpci6bus0 => { bus => "pcie.0", addr => 11 }, - hostpci7bus0 => { bus => "pcie.0", addr => 12 }, - hostpci8bus0 => { bus => "pcie.0", addr => 13 }, - hostpci9bus0 => { bus => "pcie.0", addr => 14 }, - hostpci10bus0 => { bus => "pcie.0", addr => 15 }, - hostpci11bus0 => { bus => "pcie.0", addr => 21 }, - hostpci12bus0 => { bus => "pcie.0", addr => 22 }, - hostpci13bus0 => { bus => "pcie.0", addr => 23 }, - hostpci14bus0 => { bus => "pcie.0", addr => 24 }, - hostpci15bus0 => { bus => "pcie.0", addr => 25 }, - } - if !defined($pcie_addr_map); - - return $pcie_addr_map; -} - -sub print_pcie_addr { - my ($id) = @_; - - my $res = ''; - - my $map = get_pcie_addr_map($id); - if (my $d = $get_addr_mapping_from_id->($map, $id)) { - $res = ",bus=$d->{bus},addr=$d->{addr}"; - } - - return $res; -} - -# Generates the device strings for additional pcie root ports. The first 4 pcie -# root ports are defined in the pve-q35*.cfg files. -my sub print_pcie_root_port { - my ($i) = @_; - my $res = ''; - - my $root_port_addresses = { - 4 => "10.0", - 5 => "10.1", - 6 => "10.2", - 7 => "10.3", - 8 => "10.4", - 9 => "10.5", - 10 => "10.6", - 11 => "10.7", - 12 => "11.0", - 13 => "11.1", - 14 => "11.2", - 15 => "11.3", - }; - - if (defined($root_port_addresses->{$i})) { - my $id = $i + 1; - $res = "pcie-root-port,id=ich9-pcie-port-${id}"; - $res .= ",addr=$root_port_addresses->{$i}"; - $res .= ",x-speed=16,x-width=32,multifunction=on,bus=pcie.0"; - $res .= ",port=${id},chassis=${id}"; - } - - return $res; -} - # returns the parsed pci config but parses the 'host' part into # a list if lists into the 'id' property like this: # @@ -627,7 +365,7 @@ sub choose_hostpci_devices { } sub print_hostpci_devices { - my ($vmid, $conf, $devices, $vga, $winversion, $arch, $bootorder, $dry_run) = @_; + my ($vmid, $conf, $devices, $vga, $winversion, $hw_config, $bootorder, $dry_run) = @_; my $kvm_off = 0; my $gpu_passthrough = 0; @@ -644,21 +382,11 @@ sub print_hostpci_devices { $legacy_igd = 1 if $d->{'legacy-igd'}; if (my $pcie = $d->{pcie}) { - # win7 wants to have the pcie devices directly on the pcie bus - # instead of in the root port - if ($winversion == 7) { - $pciaddr = print_pcie_addr("${id}bus0"); - } else { - # add more root ports if needed, 4 are present by default - # by pve-q35 cfgs, rest added here on demand. - if ($i > 3) { - push @$devices, '-device', print_pcie_root_port($i); - } - $pciaddr = print_pcie_addr($id); - } + # the root ports come from pci_bus_devices + $pciaddr = $hw_config->addr("hostpcie$i"); } else { my $pci_name = $d->{'legacy-igd'} ? 'legacy-igd' : $id; - $pciaddr = print_pci_addr($pci_name, $arch); + $pciaddr = $hw_config->addr($pci_name); } my $num_devices = scalar($d->{ids}->@*); @@ -881,50 +609,35 @@ sub reserve_pci_usage { die $@ if $@; } -# Returns a list of bridge devices which are necessary for the remaining -# devices. -sub get_pci_bridges { - my ($conf, $arch, $q35, $max_scsihw, $version_guard) = @_; +sub hardware_config { + my ($conf, $arch, $max_scsihw, $machine_version, $forcemachine) = @_; - my $bridges = { - # 0 => 1, always present - 1 => 1, - 2 => 1, - }; - - $bridges->{3} = 1 if ($conf->{scsihw} // '') =~ m/^virtio-scsi-single/; - - # some scsi controllers can only have 7 scsi disks per controller, - # so scsi14 and upwards need scsihw2,3,4 which live on bridge 4 - $bridges->{4} = 1 if $max_scsihw > 1 || $version_guard->(11, 1); - - # use cheap legacy igd check instead of a full parse_hostpci my $legacy_igd = 0; + my $hostpcie = []; for (my $i = 0; $i < $MAX_HOSTPCI_DEVICES; $i++) { next if !defined($conf->{"hostpci$i"}); my $res = PVE::JSONSchema::parse_property_string($hostpci_fmt, $conf->{"hostpci$i"}); next if !defined($res); - if ($res->{'legacy-igd'}) { - $legacy_igd = 1; - last; - } + $legacy_igd = 1 if $res->{'legacy-igd'}; + push @$hostpcie, $i if $res->{pcie}; } - my $devices = []; - for my $k (sort { $a <=> $b } keys %$bridges) { - next if $q35 && $k < 4; # q35.cfg already includes bridges up to 3 - - my $k_name = $k; - if ($k == 2 && $legacy_igd) { - $k_name = "$k-igd"; - } - my $pciaddr = print_pci_addr("pci.$k_name", $arch); - my $devstr = "pci-bridge,id=pci.$k,chassis_nr=$k$pciaddr"; - - push @$devices, '-device', $devstr; - } - - return $devices; + my $machine_type = PVE::QemuServer::Machine::get_vm_machine($conf, $forcemachine); + my ($major, $minor, $patch, $pve) = PVE::QemuServer::Helpers::version_parts($machine_version); + + return PVE::RS::PCI::HardwareConfig->new( + $arch // '', + $machine_type, + $conf->{scsihw}, + $max_scsihw + 0, + $legacy_igd, + $conf->{ostype}, + $major, + $minor, + $pve, + 'legacy', + $hostpcie, + ); } 1; diff --git a/src/PVE/QemuServer/RNG.pm b/src/PVE/QemuServer/RNG.pm index f854dd50..eabc4540 100644 --- a/src/PVE/QemuServer/RNG.pm +++ b/src/PVE/QemuServer/RNG.pm @@ -6,8 +6,6 @@ use warnings; use PVE::JSONSchema; use PVE::Tools qw(file_read_firstline); -use PVE::QemuServer::PCI qw(print_pci_addr); - use base 'Exporter'; our @EXPORT_OK = qw( @@ -87,7 +85,7 @@ sub check_rng_source { } sub print_rng_device_commandline { - my ($id, $rng, $arch) = @_; + my ($id, $rng, $hw_config) = @_; die "no rng device specified\n" if !$rng; @@ -98,7 +96,7 @@ sub print_rng_device_commandline { $limiter_str = ",max-bytes=$max_bytes,period=$period"; } - my $rng_addr = print_pci_addr($id, $arch); + my $rng_addr = $hw_config->addr($id); return "virtio-rng-pci,rng=$id$limiter_str$rng_addr"; } diff --git a/src/PVE/QemuServer/USB.pm b/src/PVE/QemuServer/USB.pm index 0ab43614..f5e2e31f 100644 --- a/src/PVE/QemuServer/USB.pm +++ b/src/PVE/QemuServer/USB.pm @@ -2,7 +2,6 @@ package PVE::QemuServer::USB; use strict; use warnings; -use PVE::QemuServer::PCI qw(print_pci_addr); use PVE::QemuServer::Machine; use PVE::QemuServer::Helpers qw(min_version windows_version); use PVE::JSONSchema; @@ -120,16 +119,14 @@ my sub assert_usb_index_is_useable { } sub get_default_usb_bus { - my ($conf, $arch) = @_; - - my $q35 = PVE::QemuServer::Machine::machine_type_is_q35($conf); + my ($hw_config) = @_; # we use uhci for old VMs because tablet driver was buggy in older qemu - return $q35 || $arch eq 'aarch64' ? 'ehci' : 'uhci'; + return $hw_config->machine_type() eq 'i440fx' ? 'uhci' : 'ehci'; } sub get_usb_controllers { - my ($conf, $arch, $machine_version) = @_; + my ($conf, $hw_config, $machine_version) = @_; my $devices = []; my $pciaddr = ""; @@ -140,14 +137,15 @@ sub get_usb_controllers { min_version($machine_version, 7, 1) && defined($ostype) && ($ostype eq 'l26' || windows_version($ostype) > 7); - my $is_q35 = PVE::QemuServer::Machine::machine_type_is_q35($conf); - if ($arch eq 'aarch64') { - $pciaddr = print_pci_addr('ehci', $arch); - push @$devices, '-device', "usb-ehci,id=ehci$pciaddr"; - } elsif (!$is_q35) { - $pciaddr = print_pci_addr("piix3", $arch); + my $machine_type = $hw_config->machine_type(); + + if ($machine_type eq 'i440fx') { + $pciaddr = $hw_config->addr("piix3"); push @$devices, '-device', "piix3-usb-uhci,id=uhci$pciaddr.0x2"; + } elsif ($machine_type eq 'virt') { + $pciaddr = $hw_config->addr("ehci"); + push @$devices, '-device', "usb-ehci,id=ehci$pciaddr"; } my ($use_usb2, $use_usb3) = 0; @@ -161,12 +159,12 @@ sub get_usb_controllers { $use_usb2 = 1 if !$d->{usb3}; } - if (!$use_qemu_xhci && !$is_q35 && $use_usb2 && $arch ne 'aarch64') { + if (!$use_qemu_xhci && $use_usb2 && $machine_type eq 'i440fx') { # include usb device config if still on x86 before-xhci machines and if USB 3 is not used push @$devices, '-readconfig', '/usr/share/qemu-server/pve-usb.cfg'; } - $pciaddr = print_pci_addr("xhci", $arch); + $pciaddr = $hw_config->addr("xhci"); if ($use_qemu_xhci && $any_usb) { push @$devices, '-device', print_qemu_xhci_controller($pciaddr); } elsif ($use_usb3) { diff --git a/src/test/Makefile b/src/test/Makefile index 702c9c03..07fadb99 100644 --- a/src/test/Makefile +++ b/src/test/Makefile @@ -1,6 +1,6 @@ all: test -test: test_snapshot test_cfg_to_cmd test_cfg_to_cmd_aarch64 test_hotplug test_hotplug_aarch64 test_pci_addr_conflicts test_pci_reservation test_qemu_img_convert test_migration test_restore_config test_parse_config +test: test_snapshot test_cfg_to_cmd test_cfg_to_cmd_aarch64 test_hotplug test_hotplug_aarch64 test_pci_reservation test_qemu_img_convert test_migration test_restore_config test_parse_config test_snapshot: run_snapshot_tests.pl ./run_snapshot_tests.pl @@ -21,9 +21,6 @@ test_hotplug_aarch64: run_hotplug_tests.pl TestsCommon/*.pm hotplug/aarch64/*.co test_qemu_img_convert: run_qemu_img_convert_tests.pl perl -I../ ./run_qemu_img_convert_tests.pl -test_pci_addr_conflicts: run_pci_addr_checks.pl - ./run_pci_addr_checks.pl - test_pci_reservation: run_pci_reservation_tests.pl ./run_pci_reservation_tests.pl diff --git a/src/test/TestsCommon/CommandLineMocks.pm b/src/test/TestsCommon/CommandLineMocks.pm index 50540e8f..a4b91136 100644 --- a/src/test/TestsCommon/CommandLineMocks.pm +++ b/src/test/TestsCommon/CommandLineMocks.pm @@ -347,6 +347,9 @@ $qemu_server_helpers->mock( get_host_phys_address_bits => sub { return 46; }, + kvm_user_version => sub { + return get_test_qemu_version(); + }, ); our $qemu_server_memory; diff --git a/src/test/cfg2cmd/q35-linux-hostpci-driver-keep.conf.cmd b/src/test/cfg2cmd/q35-linux-hostpci-driver-keep.conf.cmd index 9adc2e9e..d64264ec 100644 --- a/src/test/cfg2cmd/q35-linux-hostpci-driver-keep.conf.cmd +++ b/src/test/cfg2cmd/q35-linux-hostpci-driver-keep.conf.cmd @@ -27,15 +27,15 @@ -global 'ICH9-LPC.disable_s4=1' \ -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \ -device 'pci-bridge,id=pci.4,chassis_nr=4,bus=pci.1,addr=0x1c' \ + -device 'pcie-root-port,id=ich9-pcie-port-5,addr=10.0,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=5,chassis=5' \ + -device 'pcie-root-port,id=ich9-pcie-port-8,addr=10.3,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=8,chassis=8' \ -device 'vmgenid,guid=54d1c06c-8f5b-440f-b5b2-6eab1380e13d' \ -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \ -device 'vfio-pci,host=0000:00:ff.1,id=hostpci0,bus=pci.0,addr=0x10' \ -device 'vfio-pci,host=0000:d0:13.0,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0' \ -device 'vfio-pci,host=0000:00:f4.0,id=hostpci2,bus=pci.0,addr=0x1b' \ -device 'vfio-pci,host=0000:d0:15.1,id=hostpci3,bus=ich9-pcie-port-4,addr=0x0' \ - -device 'pcie-root-port,id=ich9-pcie-port-5,addr=10.0,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=5,chassis=5' \ -device 'vfio-pci,host=0000:d0:17.0,id=hostpci4,bus=ich9-pcie-port-5,addr=0x0,rombar=0' \ - -device 'pcie-root-port,id=ich9-pcie-port-8,addr=10.3,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=8,chassis=8' \ -device 'vfio-pci,host=0000:d0:15.2,id=hostpci7,bus=ich9-pcie-port-8,addr=0x0' \ -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \ -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \ diff --git a/src/test/cfg2cmd/q35-linux-hostpci-wrong.conf b/src/test/cfg2cmd/q35-linux-hostpci-wrong.conf new file mode 100644 index 00000000..85458507 --- /dev/null +++ b/src/test/cfg2cmd/q35-linux-hostpci-wrong.conf @@ -0,0 +1,16 @@ +# TEST: Config with q35, NUMA, hostpci passthrough, EFI & Linux, only configure some hostpci devices that don't live on 0 function root port +bios: ovmf +bootdisk: scsi0 +cores: 1 +efidisk0: local:100/vm-100-disk-1.qcow2,size=128K +hostpci7: d0:15.2,pcie=1 +hostpci15: 01:00.4,pcie=1 +machine: pc-q35-11.0 +memory: 512 +net0: virtio=2E:01:68:F9:9C:87,bridge=vmbr0 +numa: 1 +ostype: l26 +scsihw: virtio-scsi-pci +smbios1: uuid=3dd750ce-d910-44d0-9493-525c0be4e687 +sockets: 2 +vmgenid: 54d1c06c-8f5b-440f-b5b2-6eab1380e13d diff --git a/src/test/cfg2cmd/q35-linux-hostpci-wrong.conf.cmd b/src/test/cfg2cmd/q35-linux-hostpci-wrong.conf.cmd new file mode 100644 index 00000000..7a2ef382 --- /dev/null +++ b/src/test/cfg2cmd/q35-linux-hostpci-wrong.conf.cmd @@ -0,0 +1,41 @@ +/usr/bin/kvm \ + -id 8006 \ + -name vm8006 \ + -no-shutdown \ + -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \ + -object 'monitor-qmp,chardev=qmp,id=monitor-qmp' \ + -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' \ + -object 'monitor-qmp,chardev=qmp-event,id=monitor-qmp-event' \ + -pidfile /var/run/qemu-server/8006.pid \ + -daemonize \ + -smbios 'type=1,uuid=3dd750ce-d910-44d0-9493-525c0be4e687' \ + -object '{"id":"throttle-drive-efidisk0","limits":{},"qom-type":"throttle-group"}' \ + -blockdev '{"driver":"raw","file":{"driver":"file","filename":"/usr/share/pve-edk2-firmware//OVMF_CODE.fd"},"node-name":"pflash0","read-only":true}' \ + -blockdev '{"detect-zeroes":"on","discard":"ignore","driver":"throttle","file":{"cache":{"direct":false,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"qcow2","file":{"aio":"io_uring","cache":{"direct":false,"no-flush":false},"detect-zeroes":"on","discard":"ignore","driver":"file","filename":"/var/lib/vz/images/100/vm-100-disk-1.qcow2","node-name":"e70e3017c5a79fdee5a04aa92ac1e9c","read-only":false},"node-name":"f70e3017c5a79fdee5a04aa92ac1e9c","read-only":false},"node-name":"drive-efidisk0","read-only":false,"throttle-group":"throttle-drive-efidisk0"}' \ + -smp '2,sockets=2,cores=1,maxcpus=2' \ + -nodefaults \ + -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ + -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \ + -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \ + -m 512 \ + -object 'memory-backend-ram,id=ram-node0,size=256M' \ + -numa 'node,nodeid=0,cpus=0,memdev=ram-node0' \ + -object 'memory-backend-ram,id=ram-node1,size=256M' \ + -numa 'node,nodeid=1,cpus=1,memdev=ram-node1' \ + -global 'ICH9-LPC.disable_s3=1' \ + -global 'ICH9-LPC.disable_s4=1' \ + -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \ + -device 'pcie-root-port,id=ich9-pcie-port-5,addr=10.0,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=5,chassis=5' \ + -device 'pcie-root-port,id=ich9-pcie-port-8,addr=10.3,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=8,chassis=8' \ + -device 'pcie-root-port,id=ich9-pcie-port-13,addr=11.0,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=13,chassis=13' \ + -device 'pcie-root-port,id=ich9-pcie-port-16,addr=11.3,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=16,chassis=16' \ + -device 'vmgenid,guid=54d1c06c-8f5b-440f-b5b2-6eab1380e13d' \ + -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \ + -device 'vfio-pci,host=0000:d0:15.2,id=hostpci7,bus=ich9-pcie-port-8,addr=0x0' \ + -device 'vfio-pci,host=0000:01:00.4,id=hostpci15,bus=ich9-pcie-port-16,addr=0x0' \ + -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \ + -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \ + -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \ + -netdev 'type=tap,id=net0,ifname=tap8006i0,script=/usr/libexec/qemu-server/pve-bridge,downscript=/usr/libexec/qemu-server/pve-bridgedown,vhost=on' \ + -device 'virtio-net-pci,mac=2E:01:68:F9:9C:87,netdev=net0,bus=pci.0,addr=0x12,id=net0,rx_queue_size=1024,tx_queue_size=256,bootindex=300,host_mtu=1500' \ + -machine 'pflash0=pflash0,pflash1=drive-efidisk0,hpet=off,type=pc-q35-11.0+pve0' diff --git a/src/test/cfg2cmd/q35-linux-hostpci.conf.cmd b/src/test/cfg2cmd/q35-linux-hostpci.conf.cmd index 9adc2e9e..d64264ec 100644 --- a/src/test/cfg2cmd/q35-linux-hostpci.conf.cmd +++ b/src/test/cfg2cmd/q35-linux-hostpci.conf.cmd @@ -27,15 +27,15 @@ -global 'ICH9-LPC.disable_s4=1' \ -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \ -device 'pci-bridge,id=pci.4,chassis_nr=4,bus=pci.1,addr=0x1c' \ + -device 'pcie-root-port,id=ich9-pcie-port-5,addr=10.0,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=5,chassis=5' \ + -device 'pcie-root-port,id=ich9-pcie-port-8,addr=10.3,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=8,chassis=8' \ -device 'vmgenid,guid=54d1c06c-8f5b-440f-b5b2-6eab1380e13d' \ -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \ -device 'vfio-pci,host=0000:00:ff.1,id=hostpci0,bus=pci.0,addr=0x10' \ -device 'vfio-pci,host=0000:d0:13.0,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0' \ -device 'vfio-pci,host=0000:00:f4.0,id=hostpci2,bus=pci.0,addr=0x1b' \ -device 'vfio-pci,host=0000:d0:15.1,id=hostpci3,bus=ich9-pcie-port-4,addr=0x0' \ - -device 'pcie-root-port,id=ich9-pcie-port-5,addr=10.0,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=5,chassis=5' \ -device 'vfio-pci,host=0000:d0:17.0,id=hostpci4,bus=ich9-pcie-port-5,addr=0x0,rombar=0' \ - -device 'pcie-root-port,id=ich9-pcie-port-8,addr=10.3,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=8,chassis=8' \ -device 'vfio-pci,host=0000:d0:15.2,id=hostpci7,bus=ich9-pcie-port-8,addr=0x0' \ -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \ -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3,free-page-reporting=on' \ diff --git a/src/test/cfg2cmd/q35-win10-hostpci.conf.cmd b/src/test/cfg2cmd/q35-win10-hostpci.conf.cmd index bc7d1120..2506c7fd 100644 --- a/src/test/cfg2cmd/q35-win10-hostpci.conf.cmd +++ b/src/test/cfg2cmd/q35-win10-hostpci.conf.cmd @@ -23,11 +23,11 @@ -object 'memory-backend-ram,id=ram-node1,size=256M' \ -numa 'node,nodeid=1,cpus=1,memdev=ram-node1' \ -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \ + -device 'pcie-root-port,id=ich9-pcie-port-5,addr=10.0,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=5,chassis=5' \ -device 'vmgenid,guid=54d1c06c-8f5b-440f-b5b2-6eab1380e13d' \ -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \ -device 'vfio-pci,host=0000:f0:42.0,id=hostpci0,bus=pci.0,addr=0x10' \ -device 'vfio-pci,host=0000:f0:43.0,id=hostpci1,bus=ich9-pcie-port-2,addr=0x0' \ - -device 'pcie-root-port,id=ich9-pcie-port-5,addr=10.0,x-speed=16,x-width=32,multifunction=on,bus=pcie.0,port=5,chassis=5' \ -device 'vfio-pci,host=0000:00:43.1,id=hostpci4,bus=ich9-pcie-port-5,addr=0x0' \ -device 'VGA,id=vga,bus=pcie.0,addr=0x1' \ -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \ diff --git a/src/test/run_pci_addr_checks.pl b/src/test/run_pci_addr_checks.pl deleted file mode 100755 index 866c43eb..00000000 --- a/src/test/run_pci_addr_checks.pl +++ /dev/null @@ -1,141 +0,0 @@ -#!/usr/bin/perl - -use strict; -use warnings; -use experimental 'smartmatch'; - -use lib qw(..); - -use Test::More; - -use PVE::Tools qw(file_get_contents); -use PVE::QemuServer::PCI; - -my $qemu_cfg_base_path = "../usr"; - -# not our format but that what QEMU gets passed with '-readconfig' -sub slurp_qemu_config { - my ($fn) = @_; - - my $raw = file_get_contents($fn); - - my $lineno = 0; - my $cfg = {}; - my $group; - my $skip_to_next_group; - while ($raw =~ /^\h*(.*?)\h*$/gm) { - my $line = $1; - $lineno++; - next if !$line || $line =~ /^#/; - - # tried to follow qemu's qemu_config_parse function - if ($line =~ /\[(\S{1,63}) "([^"\]]{1,63})"\]/) { - $group = $2; - $skip_to_next_group = 0; - if ($1 ne 'device') { - $group = undef; - $skip_to_next_group = 1; - } - } elsif ($line =~ /\[([^\]]{1,63})\]/) { - $group = undef; - $skip_to_next_group = 1; - } elsif ($group) { - if ($line =~ /(\S{1,63}) = "([^\"]{1,1023})"/) { - my ($k, $v) = ($1, $2); - $cfg->{$group}->{$k} = $v; - } else { - print "ignoring $fn:$lineno: $line\n"; - } - } else { - warn "ignore $fn:$lineno, currently no group\n" if !$skip_to_next_group; - } - } - - return $cfg; -} - -sub extract_qemu_config_addrs { - my ($qemu_cfg) = @_; - - my $addr_map = {}; - for my $k (keys %$qemu_cfg) { - my $v = $qemu_cfg->{$k}; - next if !$v || !defined($v->{bus}) || !defined($v->{addr}); - - my $bus = $v->{bus}; - $bus =~ s/pci\.//; - - $addr_map->{$k} = { bus => $bus, addr => $v->{addr} }; - } - - return $addr_map; -} - -print "testing PCI(e) address conflicts\n"; - -# exec tests - -#FIXME: make cross PCI <-> PCIe check sense at all?? -my $addr_map = {}; -my ($fail, $ignored) = (0, 0); - -sub check_conflict { - my ($id, $what, $ignore_if_same_key) = @_; - - my ($bus, $addr) = $what->@{qw(bus addr)}; - my $full_addr = "$bus:$addr"; - - if (defined(my $conflict = $addr_map->{$full_addr})) { - if (my @ignores = $what->{conflict_ok}) { - if ($conflict ~~ @ignores) { - note("OK: ignore conflict for '$full_addr' between '$id' and '$conflict'"); - $ignored++; - return; - } - } - # this allows to read multiple pve-*.cfg qemu configs, and check them - # normally their OK if they conflict is on the same key. Else TODO?? - return if $ignore_if_same_key && $id eq $conflict; - - note("ERR: conflict for '$full_addr' between '$id' and '$conflict'"); - $fail++; - } else { - $addr_map->{$full_addr} = $id; - } -} - -my $pci_map = PVE::QemuServer::PCI::get_pci_addr_map(); -while (my ($id, $what) = each %$pci_map) { - check_conflict($id, $what); -} - -my $pcie_map = PVE::QemuServer::PCI::get_pcie_addr_map(); -while (my ($id, $what) = each %$pcie_map) { - check_conflict($id, $what); -} - -my $pve_qm_cfg = slurp_qemu_config("$qemu_cfg_base_path/pve-q35.cfg"); -my $pve_qm_cfg_map = extract_qemu_config_addrs($pve_qm_cfg); -while (my ($id, $what) = each %$pve_qm_cfg_map) { - check_conflict($id, $what); -} - -# FIXME: restart with clean conflict $addr_map with only get_pci*_addr_map ones? -my $pve_qm4_cfg = slurp_qemu_config("$qemu_cfg_base_path/pve-q35-4.0.cfg"); -my $pve_qm4_cfg_map = extract_qemu_config_addrs($pve_qm4_cfg); -while (my ($id, $what) = each %$pve_qm4_cfg_map) { - check_conflict($id, $what, 1); -} -my $pve_qm_usb_cfg = slurp_qemu_config("$qemu_cfg_base_path/pve-usb.cfg"); -my $pve_qm_usb_cfg_map = extract_qemu_config_addrs($pve_qm_usb_cfg); -while (my ($id, $what) = each %$pve_qm_usb_cfg_map) { - check_conflict($id, $what, 1); -} - -if ($fail) { - fail("PCI(e) address conflict check, ignored: $ignored, conflicts: $fail"); -} else { - pass("PCI(e) address conflict check, ignored: $ignored"); -} - -done_testing(); -- 2.47.3