From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 18EEE61BF0 for ; Mon, 28 Sep 2020 14:19:04 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id F0EEC2DF29 for ; Mon, 28 Sep 2020 14:18:33 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id CF8612DEF2 for ; Mon, 28 Sep 2020 14:18:30 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 9E592445AC for ; Mon, 28 Sep 2020 14:18:30 +0200 (CEST) To: Proxmox VE development discussion , Stefan Reiter References: <20200924141142.15842-1-s.reiter@proxmox.com> From: Thomas Lamprecht Message-ID: Date: Mon, 28 Sep 2020 14:18:28 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:82.0) Gecko/20100101 Thunderbird/82.0 MIME-Version: 1.0 In-Reply-To: <20200924141142.15842-1-s.reiter@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 AWL -0.166 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.011 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [PATCH qemu-server 1/2] fix #3010: add 'bootorder' parameter for better control of boot devices X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Sep 2020 12:19:04 -0000 On 24.09.20 16:11, Stefan Reiter wrote: > (also fixes #3011) >=20 > Deprecates the old 'boot' and 'bootdisk' options (they still work the > same, but will get removed if a user sets a 'bootorder' parameter and > ignored if both are set). I'd rather re-use boot instead of adding a new property. Move the current boot format out into it's own format definition, and change it to a formatsting key=3Dvalue list with the old format as defaul= t_key for backward compatibillity. The new stuff can be added as new format the= re, e.g., "order=3D". >=20 > This allows a user to specify more than one disk in the boot order, > helping with newer versions of SeaBIOS/OVMF where disks without a > bootindex won't be initialized at all (breaks soft-raid and some LVM > setups). >=20 > This also allows specifying a bootindex for USB and hostpci devices, > which was not possible before. Floppy boot support is not supported in > the new model, but I doubt that will be a problem (AFAICT we can't even= > attach floppy disks to a VM?). >=20 > Default behaviour is intended to stay the same, i.e. while new VMs will= > receive the new 'bootorder' property, it will be set so the VM starts > the same as before (see get_default_bootorder). We probably want to add a boot and/or bootorder config2cmd test before this patch, helps to guarantee that. >=20 > The API is updated to handle the deprecation correctly, i.e. when > updating the 'bootorder' attribute, the old properties are removed > (would now be useless). When removing a device that is in the bootorder= > list, it will be removed from the aforementioned. Note that non-existin= g > devices in the list will not cause an error - they will simply be > ignored - but it's still nice to not have them in there. But you do not always rewrite it to the new format, i.e., if just another= , unrelated, config property changed, or? some other comments inline. > Includes a cfg2cmd test. >=20 > Signed-off-by: Stefan Reiter > --- >=20 > The diff of the network part in config_to_command looks a bit weird, th= at's > because the indentation was off by one in the entire block. >=20 > PVE/API2/Qemu.pm | 32 +++++- > PVE/CLI/qm.pm | 4 +- > PVE/QemuServer.pm | 178 ++++++++++++++++++++++++--------= > PVE/QemuServer/Drive.pm | 29 ++++-- > PVE/QemuServer/PCI.pm | 3 +- > PVE/QemuServer/USB.pm | 14 ++- > test/cfg2cmd/bootorder.conf | 16 +++ > test/cfg2cmd/bootorder.conf.cmd | 38 +++++++ > 8 files changed, 254 insertions(+), 60 deletions(-) > create mode 100644 test/cfg2cmd/bootorder.conf > create mode 100644 test/cfg2cmd/bootorder.conf.cmd >=20 > diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm > index 8da616a..f9be475 100644 > --- a/PVE/API2/Qemu.pm > +++ b/PVE/API2/Qemu.pm > @@ -316,6 +316,7 @@ my $vmpoweroptions =3D { > my $diskoptions =3D { > 'boot' =3D> 1, > 'bootdisk' =3D> 1, > + 'bootorder' =3D> 1, > 'vmstatestorage' =3D> 1, > }; > =20 > @@ -656,9 +657,9 @@ __PACKAGE__->register_method({ > eval { > $vollist =3D &$create_disks($rpcenv, $authuser, $conf, $arch, $s= torecfg, $vmid, $pool, $param, $storage); > =20 > - if (!$conf->{bootdisk}) { > - my $firstdisk =3D PVE::QemuServer::Drive::resolve_first_disk($conf)= ; > - $conf->{bootdisk} =3D $firstdisk if $firstdisk; > + if (!$conf->{bootorder}) { > + my $order =3D PVE::QemuServer::get_default_bootorder($conf); > + $conf->{bootorder} =3D $order; > } > =20 > # auto generate uuid if user did not specify smbios1 option > @@ -1191,6 +1192,9 @@ my $update_vm_api =3D sub { > =20 > my $modified =3D {}; # record what $option we modify > =20 > + my @bootorder =3D PVE::Tools::split_list($conf->{bootorder}) if $= conf->{bootorder}; > + my $bootorder_deleted =3D grep {$_ eq 'bootorder'} @delete; > + > foreach my $opt (@delete) { > $modified->{$opt} =3D 1; > $conf =3D PVE::QemuConfig->load_config($vmid); # update/reload > @@ -1205,6 +1209,13 @@ my $update_vm_api =3D sub { > my $is_pending_val =3D defined($conf->{pending}->{$opt}); > delete $conf->{pending}->{$opt}; > =20 > + # remove from bootorder if necessary > + if (!$bootorder_deleted && @bootorder && grep {$_ eq $opt} @bootorde= r) { > + @bootorder =3D grep {$_ ne $opt} @bootorder; > + $conf->{pending}->{bootorder} =3D join(',', @bootorder); > + $modified->{bootorder} =3D 1; > + } > + > if ($opt =3D~ m/^unused/) { > my $drive =3D PVE::QemuServer::parse_drive($opt, $val); > PVE::QemuConfig->check_protection($conf, "can't remove unused di= sk '$drive->{file}'"); > @@ -1283,6 +1294,21 @@ my $update_vm_api =3D sub { > $conf->{pending}->{$opt} =3D $param->{$opt}; > } else { > $conf->{pending}->{$opt} =3D $param->{$opt}; > + > + if ($opt eq 'bootorder') { > + for my $dev (PVE::Tools::split_list($param->{$opt})) { > + my $exists =3D $conf->{$dev} || $conf->{pending}->{$dev}; > + my $deleted =3D grep {$_ eq $dev} @delete; > + die "invalid bootorder: device '$dev' does not exist'\n" > + if !$exists || $deleted; > + } > + > + # remove legacy boot order settings if new one set > + PVE::QemuConfig->add_to_pending_delete($conf, "boot") > + if $conf->{boot}; > + PVE::QemuConfig->add_to_pending_delete($conf, "bootdisk") > + if $conf->{bootdisk}; > + } > } > PVE::QemuConfig->remove_from_pending_delete($conf, $opt); > PVE::QemuConfig->write_config($vmid, $conf); > diff --git a/PVE/CLI/qm.pm b/PVE/CLI/qm.pm > index 282fa86..ff16636 100755 > --- a/PVE/CLI/qm.pm > +++ b/PVE/CLI/qm.pm > @@ -656,8 +656,8 @@ __PACKAGE__->register_method ({ > =20 > # reload after disks entries have been created > $conf =3D PVE::QemuConfig->load_config($vmid); > - my $firstdisk =3D PVE::QemuServer::Drive::resolve_first_disk($con= f); > - $conf->{bootdisk} =3D $firstdisk if $firstdisk; > + my $order =3D PVE::QemuServer::get_default_bootorder($conf); > + $conf->{bootorder} =3D $order; > PVE::QemuConfig->write_config($vmid, $conf); > }; > =20 > diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm > index 2747c66..3689ef1 100644 > --- a/PVE/QemuServer.pm > +++ b/PVE/QemuServer.pm > @@ -398,15 +398,33 @@ EODESC > boot =3D> { > optional =3D> 1, > type =3D> 'string', > - description =3D> "Boot on floppy (a), hard disk (c), CD-ROM (d), or n= etwork (n).", > + description =3D> "Boot on floppy (a), hard disk (c), CD-ROM (d), or n= etwork (n)." > + . " Deprecated: Use 'bootorder' instead.", > pattern =3D> '[acdn]{1,4}', > default =3D> 'cdn', > }, > bootdisk =3D> { > optional =3D> 1, > type =3D> 'string', format =3D> 'pve-qm-bootdisk', > - description =3D> "Enable booting from specified disk.", > - pattern =3D> '(ide|sata|scsi|virtio)\d+', > + description =3D> "Enable booting from specified disk. Deprecated: Use= 'bootorder' instead.", > + }, > + bootorder =3D> { > + optional =3D> 1, > + type =3D> 'string', > + format =3D> 'pve-qm-bootdev-list', > + description =3D> < +The guest will attempt to boot from devices in the order they appear h= ere. > + > +Disks, optical drives and passed-through storage USB devices will be d= irectly > +booted from, NICs will load PXE, and PCIe devices will either behave l= ike disks > +(e.g. NVMe) or load an option ROM (e.g. RAID controller, hardware NIC)= =2E > + > +Note that only devices in this list will be marked as bootable and thu= s loaded > +by the guest firmware (BIOS/UEFI). If you require multiple disks for b= ooting > +(e.g. software-raid), you need to specify all of them here. > + > +Overrides the deprecated 'boot' and 'bootdisk' properties when given. > +EODESC > }, > smp =3D> { > optional =3D> 1, > @@ -689,6 +707,27 @@ EODESCR > }, > }; > =20 > +PVE::JSONSchema::register_format('pve-qm-bootdev', \&verify_bootdev); > +sub verify_bootdev { > + my ($dev, $noerr) =3D @_; > + > + return $dev if PVE::QemuServer::Drive::is_valid_drivename($dev) &&= $dev !~ m/^efidisk/; > + > + my $check =3D sub { > + my ($base) =3D @_; > + return 0 if $dev !~ m/^$base\d+$/; > + return 0 if !$confdesc->{$dev}; > + return 1; > + }; > + > + return $dev if $check->("net"); > + return $dev if $check->("usb"); > + return $dev if $check->("hostpci"); > + > + return undef if $noerr; > + die "invalid boot device '$dev'\n"; > +} > + > my $cicustom_fmt =3D { > meta =3D> { > type =3D> 'string', > @@ -1552,8 +1591,6 @@ sub print_drive_commandline_full { > sub print_netdevice_full { > my ($vmid, $conf, $net, $netid, $bridges, $use_old_bios_files, $ar= ch, $machine_type) =3D @_; > =20 > - my $bootorder =3D $conf->{boot} || $confdesc->{boot}->{default}; > - > my $device =3D $net->{model}; > if ($net->{model} eq 'virtio') { > $device =3D 'virtio-net-pci'; > @@ -3151,17 +3188,29 @@ sub config_to_command { > push @$devices, '-device', $kbd if defined($kbd); > } > =20 > + my $bootorder =3D {}; > + if ($conf->{bootorder}) { > + # start at 100 to allow user to insert devices before us with -args > + my $i =3D 100; > + for my $dev (PVE::Tools::split_list($conf->{bootorder})) { > + $bootorder->{$dev} =3D $i++; > + } > + } else { > + $bootorder =3D bootorder_from_legacy($conf); > + } > + > # host pci device passthrough > my ($kvm_off, $gpu_passthrough, $legacy_igd) =3D PVE::QemuServer::= PCI::print_hostpci_devices( > - $vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, $machine_= type); > + $vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, $machine_= type, $bootorder); > =20 > # usb devices > my $usb_dev_features =3D {}; > $usb_dev_features->{spice_usb3} =3D 1 if min_version($machine_vers= ion, 4, 0); > =20 > my @usbdevices =3D PVE::QemuServer::USB::get_usb_devices( > - $conf, $usbdesc->{format}, $MAX_USB_DEVICES, $usb_dev_features= ); > + $conf, $usbdesc->{format}, $MAX_USB_DEVICES, $usb_dev_features= , $bootorder); > push @$devices, @usbdevices if @usbdevices; > + > # serial devices > for (my $i =3D 0; $i < $MAX_SERIAL_PORTS; $i++) { > if (my $path =3D $conf->{"serial$i"}) { > @@ -3229,15 +3278,6 @@ sub config_to_command { > } > push @$cmd, '-nodefaults'; > =20 > - my $bootorder =3D $conf->{boot} || $confdesc->{boot}->{default}; > - > - my $bootindex_hash =3D {}; > - my $i =3D 1; > - foreach my $o (split(//, $bootorder)) { > - $bootindex_hash->{$o} =3D $i*100; > - $i++; > - } > - > push @$cmd, '-boot', "menu=3Don,strict=3Don,reboot-timeout=3D1000,= splash=3D/usr/share/qemu-server/bootsplash.jpg"; > =20 > push @$cmd, '-no-acpi' if defined($conf->{acpi}) && $conf->{acpi} = =3D=3D 0; > @@ -3407,17 +3447,7 @@ sub config_to_command { > =20 > $use_virtio =3D 1 if $ds =3D~ m/^virtio/; > =20 > - if (drive_is_cdrom ($drive)) { > - if ($bootindex_hash->{d}) { > - $drive->{bootindex} =3D $bootindex_hash->{d}; > - $bootindex_hash->{d} +=3D 1; > - } > - } else { > - if ($bootindex_hash->{c}) { > - $drive->{bootindex} =3D $bootindex_hash->{c} if $conf->{bootdisk} &&= ($conf->{bootdisk} eq $ds); > - $bootindex_hash->{c} +=3D 1; > - } > - } > + $drive->{bootindex} =3D $bootorder->{$ds} if $bootorder->{$ds}; > =20 > if ($drive->{interface} eq 'virtio'){ > push @$cmd, '-object', "iothread,id=3Diothread-$ds" if $dri= ve->{iothread}; > @@ -3468,24 +3498,23 @@ sub config_to_command { > }); > =20 > for (my $i =3D 0; $i < $MAX_NETS; $i++) { > - next if !$conf->{"net$i"}; > - my $d =3D parse_net($conf->{"net$i"}); > - next if !$d; > + my $netname =3D "net$i"; > =20 > - $use_virtio =3D 1 if $d->{model} eq 'virtio'; > + next if !$conf->{$netname}; > + my $d =3D parse_net($conf->{$netname}); > + next if !$d; > =20 those changes here seem mostly unrelated to this patch, maybe pull them o= ut in a extra patch. > - if ($bootindex_hash->{n}) { > - $d->{bootindex} =3D $bootindex_hash->{n}; > - $bootindex_hash->{n} +=3D 1; > - } > + $use_virtio =3D 1 if $d->{model} eq 'virtio'; > =20 > - my $netdevfull =3D print_netdev_full($vmid, $conf, $arch, $d, "net$i= "); > - push @$devices, '-netdev', $netdevfull; > + $d->{bootindex} =3D $bootorder->{$netname} if $bootorder->{$netname};= > =20 > - my $netdevicefull =3D print_netdevice_full( > - $vmid, $conf, $d, "net$i", $bridges, $use_old_bios_files, $arch, = $machine_type); > + my $netdevfull =3D print_netdev_full($vmid, $conf, $arch, $d, $netnam= e); > + push @$devices, '-netdev', $netdevfull; > =20 > - push @$devices, '-device', $netdevicefull; > + my $netdevicefull =3D print_netdevice_full( > + $vmid, $conf, $d, $netname, $bridges, $use_old_bios_files, $arch, $ma= chine_type); > + > + push @$devices, '-device', $netdevicefull; > } > =20 > if ($conf->{ivshmem}) { > @@ -3765,7 +3794,8 @@ sub vm_deviceunplug { > my $devices_list =3D vm_devices_list($vmid); > return 1 if !defined($devices_list->{$deviceid}); > =20 > - die "can't unplug bootdisk" if $conf->{bootdisk} && $conf->{bootdi= sk} eq $deviceid; > + my $bootdisks =3D PVE::QemuServer::Drive::get_bootdisks($conf); > + die "can't unplug bootdisk '$deviceid'\n" if grep {$_ eq $deviceid= } @$bootdisks; > =20 > if ($deviceid eq 'tablet' || $deviceid eq 'keyboard') { > =20 > @@ -7152,6 +7182,72 @@ sub clear_reboot_request { > return $res; > } > =20 > +sub bootorder_from_legacy { > + my ($conf) =3D @_; we could add that in a patch before this already, switching call sites ov= er, that would reduce the size (reviewabillity) of this. I.e., new order could be: # general cleanups, no new stuff # test for current boot/bootorder stuff # factor out current stuff # add new functionality > + > + my $boot =3D $conf->{boot} || $confdesc->{boot}->{default}; > + my $bootindex_hash =3D {}; > + my $i =3D 1; > + foreach my $o (split(//, $boot)) { > + $bootindex_hash->{$o} =3D $i*100; > + $i++; > + } > + > + my $bootorder =3D {}; > + > + PVE::QemuConfig->foreach_volume($conf, sub { > + my ($ds, $drive) =3D @_; > + > + if (drive_is_cdrom ($drive, 1)) { > + if ($bootindex_hash->{d}) { > + $bootorder->{$ds} =3D $bootindex_hash->{d}; > + $bootindex_hash->{d} +=3D 1; > + } > + } elsif ($bootindex_hash->{c} && $conf->{bootdisk} && $conf->{bootdis= k} eq $ds) { > + $bootorder->{$ds} =3D $bootindex_hash->{c}; > + } > + }); > + > + if ($bootindex_hash->{n}) { > + for (my $i =3D 0; $i < $MAX_NETS; $i++) { > + my $netname =3D "net$i"; > + next if !$conf->{$netname}; > + $bootorder->{$netname} =3D $bootindex_hash->{n}; > + $bootindex_hash->{n} +=3D 1; > + } > + } > + > + return $bootorder; > +} > + > +# Matches legacy default boot order, but with explicit device names. T= his is > +# somewhat important, since the fallback for when neither 'bootorder' = nor the > +# old 'boot'/'bootdisk' is specified relies on 'bootorder_from_legacy'= above, > +# and it would be confusing if this diverges. > +sub get_default_bootorder { > + my ($conf) =3D @_; > + > + my @ret =3D (); > + > + # harddisk > + my $first =3D PVE::QemuServer::Drive::resolve_first_disk($conf, 0)= ; > + push @ret, $first if $first; > + > + # cdrom > + $first =3D PVE::QemuServer::Drive::resolve_first_disk($conf, 1); > + push @ret, $first if $first; > + > + # network > + for (my $i =3D 0; $i < $MAX_NETS; $i++) { > + my $netname =3D "net$i"; > + next if !$conf->{$netname}; > + push @ret, $netname; > + last; > + } > + > + return join(',', @ret); > +} > + > # bash completion helper > =20 > sub complete_backup_archives { > diff --git a/PVE/QemuServer/Drive.pm b/PVE/QemuServer/Drive.pm > index 91c33f8..af25882 100644 > --- a/PVE/QemuServer/Drive.pm > +++ b/PVE/QemuServer/Drive.pm > @@ -501,11 +501,25 @@ sub print_drive { > return PVE::JSONSchema::print_property_string($drive, $alldrive_fm= t, $skip); > } > =20 > +sub get_bootdisks { > + my ($conf) =3D @_; > + > + if (!$conf->{bootorder}) { > + return [$conf->{bootdisk}] if $conf->{bootdisk}; > + return []; > + } > + > + my @list =3D PVE::Tools::split_list($conf->{bootorder}); > + @list =3D grep {is_valid_drivename($_)} @list; > + return \@list; > +} > + > sub bootdisk_size { > my ($storecfg, $conf) =3D @_; > =20 > - my $bootdisk =3D $conf->{bootdisk}; > - return undef if !$bootdisk; > + my $bootdisks =3D get_bootdisks($conf); > + return undef if !@$bootdisks; > + my $bootdisk =3D $bootdisks->[0]; > return undef if !is_valid_drivename($bootdisk); > =20 > return undef if !$conf->{$bootdisk}; > @@ -584,16 +598,15 @@ sub is_volume_in_use { > } > =20 > sub resolve_first_disk { > - my $conf =3D shift; > + my ($conf, $cdrom) =3D @_; > my @disks =3D valid_drive_names(); > - my $firstdisk; > - foreach my $ds (reverse @disks) { > + foreach my $ds (@disks) { > next if !$conf->{$ds}; > my $disk =3D parse_drive($ds, $conf->{$ds}); > - next if drive_is_cdrom($disk); > - $firstdisk =3D $ds; > + next if !(drive_is_cdrom($disk) xor $cdrom); > + return $ds; > } > - return $firstdisk; > + return undef; > } > =20 > 1; > diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/PCI.pm > index cb36845..2df2708 100644 > --- a/PVE/QemuServer/PCI.pm > +++ b/PVE/QemuServer/PCI.pm > @@ -357,7 +357,7 @@ sub parse_hostpci { > } > =20 > sub print_hostpci_devices { > - my ($vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, $m= achine_type) =3D @_; > + my ($vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, $m= achine_type, $bootorder) =3D @_; > =20 > my $kvm_off =3D 0; > my $gpu_passthrough =3D 0; > @@ -446,6 +446,7 @@ sub print_hostpci_devices { > $devicestr .=3D "$xvga"; > $devicestr .=3D ",multifunction=3Don" if $multifunction; > $devicestr .=3D ",romfile=3D/usr/share/kvm/$d->{romfile}" if $d->{ro= mfile}; > + $devicestr .=3D ",bootindex=3D$bootorder->{$id}" if $bootorder->{$id= }; > } > =20 > push @$devices, '-device', $devicestr; > diff --git a/PVE/QemuServer/USB.pm b/PVE/QemuServer/USB.pm > index d328148..4a843cd 100644 > --- a/PVE/QemuServer/USB.pm > +++ b/PVE/QemuServer/USB.pm > @@ -74,13 +74,14 @@ sub get_usb_controllers { > } > =20 > sub get_usb_devices { > - my ($conf, $format, $max_usb_devices, $features) =3D @_; > + my ($conf, $format, $max_usb_devices, $features, $bootorder) =3D @= _; > =20 > my $devices =3D []; > =20 > for (my $i =3D 0; $i < $max_usb_devices; $i++) { > - next if !$conf->{"usb$i"}; > - my $d =3D eval { PVE::JSONSchema::parse_property_string($format,$conf= ->{"usb$i"}) }; > + my $devname =3D "usb$i"; > + next if !$conf->{$devname}; > + my $d =3D eval { PVE::JSONSchema::parse_property_string($format,$conf= ->{$devname}) }; > next if !$d; > =20 > if (defined($d->{host})) { > @@ -93,8 +94,10 @@ sub get_usb_devices { > =20 > push @$devices, '-chardev', "spicevmc,id=3Dusbredirchardev$i,name=3D= usbredir"; > push @$devices, '-device', "usb-redir,chardev=3Dusbredirchardev$i,id= =3Dusbredirdev$i,bus=3D$bus.0"; > + > + warn "warning: spice usb port set as bootdevice, ignoring\n" if $boo= torder->{$devname}; > } else { > - push @$devices, '-device', print_usbdevice_full($conf, "usb$i", $hos= tdevice); > + push @$devices, '-device', print_usbdevice_full($conf, $devname, $ho= stdevice, $bootorder); > } > } > } > @@ -103,7 +106,7 @@ sub get_usb_devices { > } > =20 > sub print_usbdevice_full { > - my ($conf, $deviceid, $device) =3D @_; > + my ($conf, $deviceid, $device, $bootorder) =3D @_; > =20 > return if !$device; > my $usbdevice =3D "usb-host"; > @@ -120,6 +123,7 @@ sub print_usbdevice_full { > } > =20 > $usbdevice .=3D ",id=3D$deviceid"; > + $usbdevice .=3D ",bootindex=3D$bootorder->{$deviceid}" if $bootord= er->{$deviceid}; > return $usbdevice; > } > =20 > diff --git a/test/cfg2cmd/bootorder.conf b/test/cfg2cmd/bootorder.conf > new file mode 100644 > index 0000000..7d296da > --- /dev/null > +++ b/test/cfg2cmd/bootorder.conf > @@ -0,0 +1,16 @@ > +# TEST: Test for a specific bootorder given by 'bootorder' parameter > +# QEMU_VERSION: 5.1 > +cores: 3 > +bootorder: virtio1,net0,scsi4,ide2 > +ide2: none,media=3Dcdrom > +memory: 768 > +name: simple > +net0: virtio=3DA2:C0:43:77:08:A0,bridge=3Dvmbr0 > +numa: 0 > +ostype: l26 > +scsi4: local:8006/vm-8006-disk-0.qcow2,discard=3Don,size=3D104858K > +smbios1: uuid=3D7b10d7af-b932-4c66-b2c3-3996152ec465 > +sockets: 1 > +virtio0: local:8006/vm-8006-disk-0.qcow2,discard=3Don,iothread=3D1,siz= e=3D104858K > +virtio1: local:8006/vm-8006-disk-0.qcow2,discard=3Don,iothread=3D1,siz= e=3D104858K > +vmgenid: c773c261-d800-4348-9f5d-167fadd53cf8 > diff --git a/test/cfg2cmd/bootorder.conf.cmd b/test/cfg2cmd/bootorder.c= onf.cmd > new file mode 100644 > index 0000000..86cae07 > --- /dev/null > +++ b/test/cfg2cmd/bootorder.conf.cmd > @@ -0,0 +1,38 @@ > +/usr/bin/kvm \ > + -id 8006 \ > + -name simple \ > + -chardev 'socket,id=3Dqmp,path=3D/var/run/qemu-server/8006.qmp,serve= r,nowait' \ > + -mon 'chardev=3Dqmp,mode=3Dcontrol' \ > + -chardev 'socket,id=3Dqmp-event,path=3D/var/run/qmeventd.sock,reconn= ect=3D5' \ > + -mon 'chardev=3Dqmp-event,mode=3Dcontrol' \ > + -pidfile /var/run/qemu-server/8006.pid \ > + -daemonize \ > + -smbios 'type=3D1,uuid=3D7b10d7af-b932-4c66-b2c3-3996152ec465' \ > + -smp '3,sockets=3D1,cores=3D3,maxcpus=3D3' \ > + -nodefaults \ > + -boot 'menu=3Don,strict=3Don,reboot-timeout=3D1000,splash=3D/usr/sha= re/qemu-server/bootsplash.jpg' \ > + -vnc unix:/var/run/qemu-server/8006.vnc,password \ > + -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \ > + -m 768 \ > + -object 'iothread,id=3Diothread-virtio0' \ > + -object 'iothread,id=3Diothread-virtio1' \ > + -device 'pci-bridge,id=3Dpci.1,chassis_nr=3D1,bus=3Dpci.0,addr=3D0x1= e' \ > + -device 'pci-bridge,id=3Dpci.2,chassis_nr=3D2,bus=3Dpci.0,addr=3D0x1= f' \ > + -device 'vmgenid,guid=3Dc773c261-d800-4348-9f5d-167fadd53cf8' \ > + -device 'piix3-usb-uhci,id=3Duhci,bus=3Dpci.0,addr=3D0x1.0x2' \ > + -device 'usb-tablet,id=3Dtablet,bus=3Duhci.0,port=3D1' \ > + -device 'VGA,id=3Dvga,bus=3Dpci.0,addr=3D0x2' \ > + -device 'virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.0,addr=3D0x3' \ > + -iscsi 'initiator-name=3Diqn.1993-08.org.debian:01:aabbccddeeff' \ > + -drive 'if=3Dnone,id=3Ddrive-ide2,media=3Dcdrom,aio=3Dthreads' \ > + -device 'ide-cd,bus=3Dide.1,unit=3D0,drive=3Ddrive-ide2,id=3Dide2,bo= otindex=3D103' \ > + -device 'lsi,id=3Dscsihw0,bus=3Dpci.0,addr=3D0x5' \ > + -drive 'file=3D/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=3Dnon= e,id=3Ddrive-scsi4,discard=3Don,format=3Dqcow2,cache=3Dnone,aio=3Dnative,= detect-zeroes=3Dunmap' \ > + -device 'scsi-hd,bus=3Dscsihw0.0,scsi-id=3D4,drive=3Ddrive-scsi4,id=3D= scsi4,bootindex=3D102' \ > + -drive 'file=3D/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=3Dnon= e,id=3Ddrive-virtio0,discard=3Don,format=3Dqcow2,cache=3Dnone,aio=3Dnativ= e,detect-zeroes=3Dunmap' \ > + -device 'virtio-blk-pci,drive=3Ddrive-virtio0,id=3Dvirtio0,bus=3Dpci= =2E0,addr=3D0xa,iothread=3Diothread-virtio0' \ > + -drive 'file=3D/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=3Dnon= e,id=3Ddrive-virtio1,discard=3Don,format=3Dqcow2,cache=3Dnone,aio=3Dnativ= e,detect-zeroes=3Dunmap' \ > + -device 'virtio-blk-pci,drive=3Ddrive-virtio1,id=3Dvirtio1,bus=3Dpci= =2E0,addr=3D0xb,iothread=3Diothread-virtio1,bootindex=3D100' \ > + -netdev 'type=3Dtap,id=3Dnet0,ifname=3Dtap8006i0,script=3D/var/lib/q= emu-server/pve-bridge,downscript=3D/var/lib/qemu-server/pve-bridgedown,vh= ost=3Don' \ > + -device 'virtio-net-pci,mac=3DA2:C0:43:77:08:A0,netdev=3Dnet0,bus=3D= pci.0,addr=3D0x12,id=3Dnet0,bootindex=3D101' \ > + -machine 'type=3Dpc+pve0' >=20