From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <t.lamprecht@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 18EEE61BF0
for <pve-devel@lists.proxmox.com>; Mon, 28 Sep 2020 14:19:04 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id F0EEC2DF29
for <pve-devel@lists.proxmox.com>; Mon, 28 Sep 2020 14:18:33 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[212.186.127.180])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id CF8612DEF2
for <pve-devel@lists.proxmox.com>; Mon, 28 Sep 2020 14:18:30 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 9E592445AC
for <pve-devel@lists.proxmox.com>; Mon, 28 Sep 2020 14:18:30 +0200 (CEST)
To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>,
Stefan Reiter <s.reiter@proxmox.com>
References: <20200924141142.15842-1-s.reiter@proxmox.com>
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
Message-ID: <b976aeee-38fe-4538-30cb-1f30743bd728@proxmox.com>
Date: Mon, 28 Sep 2020 14:18:28 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:82.0) Gecko/20100101
Thunderbird/82.0
MIME-Version: 1.0
In-Reply-To: <20200924141142.15842-1-s.reiter@proxmox.com>
Content-Type: text/plain; charset=UTF-8
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.166 Adjusted score from AWL reputation of From: address
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
NICE_REPLY_A -0.011 Looks like a legit reply (A)
RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/,
medium trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Subject: Re: [pve-devel] [PATCH qemu-server 1/2] fix #3010: add 'bootorder'
parameter for better control of boot devices
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Mon, 28 Sep 2020 12:19:04 -0000
On 24.09.20 16:11, Stefan Reiter wrote:
> (also fixes #3011)
>=20
> Deprecates the old 'boot' and 'bootdisk' options (they still work the
> same, but will get removed if a user sets a 'bootorder' parameter and
> ignored if both are set).
I'd rather re-use boot instead of adding a new property.
Move the current boot format out into it's own format definition, and
change it to a formatsting key=3Dvalue list with the old format as defaul=
t_key
for backward compatibillity. The new stuff can be added as new format the=
re,
e.g., "order=3D<boot-order-list>".
>=20
> This allows a user to specify more than one disk in the boot order,
> helping with newer versions of SeaBIOS/OVMF where disks without a
> bootindex won't be initialized at all (breaks soft-raid and some LVM
> setups).
>=20
> This also allows specifying a bootindex for USB and hostpci devices,
> which was not possible before. Floppy boot support is not supported in
> the new model, but I doubt that will be a problem (AFAICT we can't even=
> attach floppy disks to a VM?).
>=20
> Default behaviour is intended to stay the same, i.e. while new VMs will=
> receive the new 'bootorder' property, it will be set so the VM starts
> the same as before (see get_default_bootorder).
We probably want to add a boot and/or bootorder config2cmd test before
this patch, helps to guarantee that.
>=20
> The API is updated to handle the deprecation correctly, i.e. when
> updating the 'bootorder' attribute, the old properties are removed
> (would now be useless). When removing a device that is in the bootorder=
> list, it will be removed from the aforementioned. Note that non-existin=
g
> devices in the list will not cause an error - they will simply be
> ignored - but it's still nice to not have them in there.
But you do not always rewrite it to the new format, i.e., if just another=
,
unrelated, config property changed, or?
some other comments inline.
> Includes a cfg2cmd test.
>=20
> Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
> ---
>=20
> The diff of the network part in config_to_command looks a bit weird, th=
at's
> because the indentation was off by one in the entire block.
>=20
> PVE/API2/Qemu.pm | 32 +++++-
> PVE/CLI/qm.pm | 4 +-
> PVE/QemuServer.pm | 178 ++++++++++++++++++++++++--------=
> PVE/QemuServer/Drive.pm | 29 ++++--
> PVE/QemuServer/PCI.pm | 3 +-
> PVE/QemuServer/USB.pm | 14 ++-
> test/cfg2cmd/bootorder.conf | 16 +++
> test/cfg2cmd/bootorder.conf.cmd | 38 +++++++
> 8 files changed, 254 insertions(+), 60 deletions(-)
> create mode 100644 test/cfg2cmd/bootorder.conf
> create mode 100644 test/cfg2cmd/bootorder.conf.cmd
>=20
> diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
> index 8da616a..f9be475 100644
> --- a/PVE/API2/Qemu.pm
> +++ b/PVE/API2/Qemu.pm
> @@ -316,6 +316,7 @@ my $vmpoweroptions =3D {
> my $diskoptions =3D {
> 'boot' =3D> 1,
> 'bootdisk' =3D> 1,
> + 'bootorder' =3D> 1,
> 'vmstatestorage' =3D> 1,
> };
> =20
> @@ -656,9 +657,9 @@ __PACKAGE__->register_method({
> eval {
> $vollist =3D &$create_disks($rpcenv, $authuser, $conf, $arch, $s=
torecfg, $vmid, $pool, $param, $storage);
> =20
> - if (!$conf->{bootdisk}) {
> - my $firstdisk =3D PVE::QemuServer::Drive::resolve_first_disk($conf)=
;
> - $conf->{bootdisk} =3D $firstdisk if $firstdisk;
> + if (!$conf->{bootorder}) {
> + my $order =3D PVE::QemuServer::get_default_bootorder($conf);
> + $conf->{bootorder} =3D $order;
> }
> =20
> # auto generate uuid if user did not specify smbios1 option
> @@ -1191,6 +1192,9 @@ my $update_vm_api =3D sub {
> =20
> my $modified =3D {}; # record what $option we modify
> =20
> + my @bootorder =3D PVE::Tools::split_list($conf->{bootorder}) if $=
conf->{bootorder};
> + my $bootorder_deleted =3D grep {$_ eq 'bootorder'} @delete;
> +
> foreach my $opt (@delete) {
> $modified->{$opt} =3D 1;
> $conf =3D PVE::QemuConfig->load_config($vmid); # update/reload
> @@ -1205,6 +1209,13 @@ my $update_vm_api =3D sub {
> my $is_pending_val =3D defined($conf->{pending}->{$opt});
> delete $conf->{pending}->{$opt};
> =20
> + # remove from bootorder if necessary
> + if (!$bootorder_deleted && @bootorder && grep {$_ eq $opt} @bootorde=
r) {
> + @bootorder =3D grep {$_ ne $opt} @bootorder;
> + $conf->{pending}->{bootorder} =3D join(',', @bootorder);
> + $modified->{bootorder} =3D 1;
> + }
> +
> if ($opt =3D~ m/^unused/) {
> my $drive =3D PVE::QemuServer::parse_drive($opt, $val);
> PVE::QemuConfig->check_protection($conf, "can't remove unused di=
sk '$drive->{file}'");
> @@ -1283,6 +1294,21 @@ my $update_vm_api =3D sub {
> $conf->{pending}->{$opt} =3D $param->{$opt};
> } else {
> $conf->{pending}->{$opt} =3D $param->{$opt};
> +
> + if ($opt eq 'bootorder') {
> + for my $dev (PVE::Tools::split_list($param->{$opt})) {
> + my $exists =3D $conf->{$dev} || $conf->{pending}->{$dev};
> + my $deleted =3D grep {$_ eq $dev} @delete;
> + die "invalid bootorder: device '$dev' does not exist'\n"
> + if !$exists || $deleted;
> + }
> +
> + # remove legacy boot order settings if new one set
> + PVE::QemuConfig->add_to_pending_delete($conf, "boot")
> + if $conf->{boot};
> + PVE::QemuConfig->add_to_pending_delete($conf, "bootdisk")
> + if $conf->{bootdisk};
> + }
> }
> PVE::QemuConfig->remove_from_pending_delete($conf, $opt);
> PVE::QemuConfig->write_config($vmid, $conf);
> diff --git a/PVE/CLI/qm.pm b/PVE/CLI/qm.pm
> index 282fa86..ff16636 100755
> --- a/PVE/CLI/qm.pm
> +++ b/PVE/CLI/qm.pm
> @@ -656,8 +656,8 @@ __PACKAGE__->register_method ({
> =20
> # reload after disks entries have been created
> $conf =3D PVE::QemuConfig->load_config($vmid);
> - my $firstdisk =3D PVE::QemuServer::Drive::resolve_first_disk($con=
f);
> - $conf->{bootdisk} =3D $firstdisk if $firstdisk;
> + my $order =3D PVE::QemuServer::get_default_bootorder($conf);
> + $conf->{bootorder} =3D $order;
> PVE::QemuConfig->write_config($vmid, $conf);
> };
> =20
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index 2747c66..3689ef1 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -398,15 +398,33 @@ EODESC
> boot =3D> {
> optional =3D> 1,
> type =3D> 'string',
> - description =3D> "Boot on floppy (a), hard disk (c), CD-ROM (d), or n=
etwork (n).",
> + description =3D> "Boot on floppy (a), hard disk (c), CD-ROM (d), or n=
etwork (n)."
> + . " Deprecated: Use 'bootorder' instead.",
> pattern =3D> '[acdn]{1,4}',
> default =3D> 'cdn',
> },
> bootdisk =3D> {
> optional =3D> 1,
> type =3D> 'string', format =3D> 'pve-qm-bootdisk',
> - description =3D> "Enable booting from specified disk.",
> - pattern =3D> '(ide|sata|scsi|virtio)\d+',
> + description =3D> "Enable booting from specified disk. Deprecated: Use=
'bootorder' instead.",
> + },
> + bootorder =3D> {
> + optional =3D> 1,
> + type =3D> 'string',
> + format =3D> 'pve-qm-bootdev-list',
> + description =3D> <<EODESC,
> +The guest will attempt to boot from devices in the order they appear h=
ere.
> +
> +Disks, optical drives and passed-through storage USB devices will be d=
irectly
> +booted from, NICs will load PXE, and PCIe devices will either behave l=
ike disks
> +(e.g. NVMe) or load an option ROM (e.g. RAID controller, hardware NIC)=
=2E
> +
> +Note that only devices in this list will be marked as bootable and thu=
s loaded
> +by the guest firmware (BIOS/UEFI). If you require multiple disks for b=
ooting
> +(e.g. software-raid), you need to specify all of them here.
> +
> +Overrides the deprecated 'boot' and 'bootdisk' properties when given.
> +EODESC
> },
> smp =3D> {
> optional =3D> 1,
> @@ -689,6 +707,27 @@ EODESCR
> },
> };
> =20
> +PVE::JSONSchema::register_format('pve-qm-bootdev', \&verify_bootdev);
> +sub verify_bootdev {
> + my ($dev, $noerr) =3D @_;
> +
> + return $dev if PVE::QemuServer::Drive::is_valid_drivename($dev) &&=
$dev !~ m/^efidisk/;
> +
> + my $check =3D sub {
> + my ($base) =3D @_;
> + return 0 if $dev !~ m/^$base\d+$/;
> + return 0 if !$confdesc->{$dev};
> + return 1;
> + };
> +
> + return $dev if $check->("net");
> + return $dev if $check->("usb");
> + return $dev if $check->("hostpci");
> +
> + return undef if $noerr;
> + die "invalid boot device '$dev'\n";
> +}
> +
> my $cicustom_fmt =3D {
> meta =3D> {
> type =3D> 'string',
> @@ -1552,8 +1591,6 @@ sub print_drive_commandline_full {
> sub print_netdevice_full {
> my ($vmid, $conf, $net, $netid, $bridges, $use_old_bios_files, $ar=
ch, $machine_type) =3D @_;
> =20
> - my $bootorder =3D $conf->{boot} || $confdesc->{boot}->{default};
> -
> my $device =3D $net->{model};
> if ($net->{model} eq 'virtio') {
> $device =3D 'virtio-net-pci';
> @@ -3151,17 +3188,29 @@ sub config_to_command {
> push @$devices, '-device', $kbd if defined($kbd);
> }
> =20
> + my $bootorder =3D {};
> + if ($conf->{bootorder}) {
> + # start at 100 to allow user to insert devices before us with -args
> + my $i =3D 100;
> + for my $dev (PVE::Tools::split_list($conf->{bootorder})) {
> + $bootorder->{$dev} =3D $i++;
> + }
> + } else {
> + $bootorder =3D bootorder_from_legacy($conf);
> + }
> +
> # host pci device passthrough
> my ($kvm_off, $gpu_passthrough, $legacy_igd) =3D PVE::QemuServer::=
PCI::print_hostpci_devices(
> - $vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, $machine_=
type);
> + $vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, $machine_=
type, $bootorder);
> =20
> # usb devices
> my $usb_dev_features =3D {};
> $usb_dev_features->{spice_usb3} =3D 1 if min_version($machine_vers=
ion, 4, 0);
> =20
> my @usbdevices =3D PVE::QemuServer::USB::get_usb_devices(
> - $conf, $usbdesc->{format}, $MAX_USB_DEVICES, $usb_dev_features=
);
> + $conf, $usbdesc->{format}, $MAX_USB_DEVICES, $usb_dev_features=
, $bootorder);
> push @$devices, @usbdevices if @usbdevices;
> +
> # serial devices
> for (my $i =3D 0; $i < $MAX_SERIAL_PORTS; $i++) {
> if (my $path =3D $conf->{"serial$i"}) {
> @@ -3229,15 +3278,6 @@ sub config_to_command {
> }
> push @$cmd, '-nodefaults';
> =20
> - my $bootorder =3D $conf->{boot} || $confdesc->{boot}->{default};
> -
> - my $bootindex_hash =3D {};
> - my $i =3D 1;
> - foreach my $o (split(//, $bootorder)) {
> - $bootindex_hash->{$o} =3D $i*100;
> - $i++;
> - }
> -
> push @$cmd, '-boot', "menu=3Don,strict=3Don,reboot-timeout=3D1000,=
splash=3D/usr/share/qemu-server/bootsplash.jpg";
> =20
> push @$cmd, '-no-acpi' if defined($conf->{acpi}) && $conf->{acpi} =
=3D=3D 0;
> @@ -3407,17 +3447,7 @@ sub config_to_command {
> =20
> $use_virtio =3D 1 if $ds =3D~ m/^virtio/;
> =20
> - if (drive_is_cdrom ($drive)) {
> - if ($bootindex_hash->{d}) {
> - $drive->{bootindex} =3D $bootindex_hash->{d};
> - $bootindex_hash->{d} +=3D 1;
> - }
> - } else {
> - if ($bootindex_hash->{c}) {
> - $drive->{bootindex} =3D $bootindex_hash->{c} if $conf->{bootdisk} &&=
($conf->{bootdisk} eq $ds);
> - $bootindex_hash->{c} +=3D 1;
> - }
> - }
> + $drive->{bootindex} =3D $bootorder->{$ds} if $bootorder->{$ds};
> =20
> if ($drive->{interface} eq 'virtio'){
> push @$cmd, '-object', "iothread,id=3Diothread-$ds" if $dri=
ve->{iothread};
> @@ -3468,24 +3498,23 @@ sub config_to_command {
> });
> =20
> for (my $i =3D 0; $i < $MAX_NETS; $i++) {
> - next if !$conf->{"net$i"};
> - my $d =3D parse_net($conf->{"net$i"});
> - next if !$d;
> + my $netname =3D "net$i";
> =20
> - $use_virtio =3D 1 if $d->{model} eq 'virtio';
> + next if !$conf->{$netname};
> + my $d =3D parse_net($conf->{$netname});
> + next if !$d;
> =20
those changes here seem mostly unrelated to this patch, maybe pull them o=
ut in a extra
patch.
> - if ($bootindex_hash->{n}) {
> - $d->{bootindex} =3D $bootindex_hash->{n};
> - $bootindex_hash->{n} +=3D 1;
> - }
> + $use_virtio =3D 1 if $d->{model} eq 'virtio';
> =20
> - my $netdevfull =3D print_netdev_full($vmid, $conf, $arch, $d, "net$i=
");
> - push @$devices, '-netdev', $netdevfull;
> + $d->{bootindex} =3D $bootorder->{$netname} if $bootorder->{$netname};=
> =20
> - my $netdevicefull =3D print_netdevice_full(
> - $vmid, $conf, $d, "net$i", $bridges, $use_old_bios_files, $arch, =
$machine_type);
> + my $netdevfull =3D print_netdev_full($vmid, $conf, $arch, $d, $netnam=
e);
> + push @$devices, '-netdev', $netdevfull;
> =20
> - push @$devices, '-device', $netdevicefull;
> + my $netdevicefull =3D print_netdevice_full(
> + $vmid, $conf, $d, $netname, $bridges, $use_old_bios_files, $arch, $ma=
chine_type);
> +
> + push @$devices, '-device', $netdevicefull;
> }
> =20
> if ($conf->{ivshmem}) {
> @@ -3765,7 +3794,8 @@ sub vm_deviceunplug {
> my $devices_list =3D vm_devices_list($vmid);
> return 1 if !defined($devices_list->{$deviceid});
> =20
> - die "can't unplug bootdisk" if $conf->{bootdisk} && $conf->{bootdi=
sk} eq $deviceid;
> + my $bootdisks =3D PVE::QemuServer::Drive::get_bootdisks($conf);
> + die "can't unplug bootdisk '$deviceid'\n" if grep {$_ eq $deviceid=
} @$bootdisks;
> =20
> if ($deviceid eq 'tablet' || $deviceid eq 'keyboard') {
> =20
> @@ -7152,6 +7182,72 @@ sub clear_reboot_request {
> return $res;
> }
> =20
> +sub bootorder_from_legacy {
> + my ($conf) =3D @_;
we could add that in a patch before this already, switching call sites ov=
er,
that would reduce the size (reviewabillity) of this.
I.e., new order could be:
# general cleanups, no new stuff
# test for current boot/bootorder stuff
# factor out current stuff
# add new functionality
> +
> + my $boot =3D $conf->{boot} || $confdesc->{boot}->{default};
> + my $bootindex_hash =3D {};
> + my $i =3D 1;
> + foreach my $o (split(//, $boot)) {
> + $bootindex_hash->{$o} =3D $i*100;
> + $i++;
> + }
> +
> + my $bootorder =3D {};
> +
> + PVE::QemuConfig->foreach_volume($conf, sub {
> + my ($ds, $drive) =3D @_;
> +
> + if (drive_is_cdrom ($drive, 1)) {
> + if ($bootindex_hash->{d}) {
> + $bootorder->{$ds} =3D $bootindex_hash->{d};
> + $bootindex_hash->{d} +=3D 1;
> + }
> + } elsif ($bootindex_hash->{c} && $conf->{bootdisk} && $conf->{bootdis=
k} eq $ds) {
> + $bootorder->{$ds} =3D $bootindex_hash->{c};
> + }
> + });
> +
> + if ($bootindex_hash->{n}) {
> + for (my $i =3D 0; $i < $MAX_NETS; $i++) {
> + my $netname =3D "net$i";
> + next if !$conf->{$netname};
> + $bootorder->{$netname} =3D $bootindex_hash->{n};
> + $bootindex_hash->{n} +=3D 1;
> + }
> + }
> +
> + return $bootorder;
> +}
> +
> +# Matches legacy default boot order, but with explicit device names. T=
his is
> +# somewhat important, since the fallback for when neither 'bootorder' =
nor the
> +# old 'boot'/'bootdisk' is specified relies on 'bootorder_from_legacy'=
above,
> +# and it would be confusing if this diverges.
> +sub get_default_bootorder {
> + my ($conf) =3D @_;
> +
> + my @ret =3D ();
> +
> + # harddisk
> + my $first =3D PVE::QemuServer::Drive::resolve_first_disk($conf, 0)=
;
> + push @ret, $first if $first;
> +
> + # cdrom
> + $first =3D PVE::QemuServer::Drive::resolve_first_disk($conf, 1);
> + push @ret, $first if $first;
> +
> + # network
> + for (my $i =3D 0; $i < $MAX_NETS; $i++) {
> + my $netname =3D "net$i";
> + next if !$conf->{$netname};
> + push @ret, $netname;
> + last;
> + }
> +
> + return join(',', @ret);
> +}
> +
> # bash completion helper
> =20
> sub complete_backup_archives {
> diff --git a/PVE/QemuServer/Drive.pm b/PVE/QemuServer/Drive.pm
> index 91c33f8..af25882 100644
> --- a/PVE/QemuServer/Drive.pm
> +++ b/PVE/QemuServer/Drive.pm
> @@ -501,11 +501,25 @@ sub print_drive {
> return PVE::JSONSchema::print_property_string($drive, $alldrive_fm=
t, $skip);
> }
> =20
> +sub get_bootdisks {
> + my ($conf) =3D @_;
> +
> + if (!$conf->{bootorder}) {
> + return [$conf->{bootdisk}] if $conf->{bootdisk};
> + return [];
> + }
> +
> + my @list =3D PVE::Tools::split_list($conf->{bootorder});
> + @list =3D grep {is_valid_drivename($_)} @list;
> + return \@list;
> +}
> +
> sub bootdisk_size {
> my ($storecfg, $conf) =3D @_;
> =20
> - my $bootdisk =3D $conf->{bootdisk};
> - return undef if !$bootdisk;
> + my $bootdisks =3D get_bootdisks($conf);
> + return undef if !@$bootdisks;
> + my $bootdisk =3D $bootdisks->[0];
> return undef if !is_valid_drivename($bootdisk);
> =20
> return undef if !$conf->{$bootdisk};
> @@ -584,16 +598,15 @@ sub is_volume_in_use {
> }
> =20
> sub resolve_first_disk {
> - my $conf =3D shift;
> + my ($conf, $cdrom) =3D @_;
> my @disks =3D valid_drive_names();
> - my $firstdisk;
> - foreach my $ds (reverse @disks) {
> + foreach my $ds (@disks) {
> next if !$conf->{$ds};
> my $disk =3D parse_drive($ds, $conf->{$ds});
> - next if drive_is_cdrom($disk);
> - $firstdisk =3D $ds;
> + next if !(drive_is_cdrom($disk) xor $cdrom);
> + return $ds;
> }
> - return $firstdisk;
> + return undef;
> }
> =20
> 1;
> diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/PCI.pm
> index cb36845..2df2708 100644
> --- a/PVE/QemuServer/PCI.pm
> +++ b/PVE/QemuServer/PCI.pm
> @@ -357,7 +357,7 @@ sub parse_hostpci {
> }
> =20
> sub print_hostpci_devices {
> - my ($vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, $m=
achine_type) =3D @_;
> + my ($vmid, $conf, $devices, $winversion, $q35, $bridges, $arch, $m=
achine_type, $bootorder) =3D @_;
> =20
> my $kvm_off =3D 0;
> my $gpu_passthrough =3D 0;
> @@ -446,6 +446,7 @@ sub print_hostpci_devices {
> $devicestr .=3D "$xvga";
> $devicestr .=3D ",multifunction=3Don" if $multifunction;
> $devicestr .=3D ",romfile=3D/usr/share/kvm/$d->{romfile}" if $d->{ro=
mfile};
> + $devicestr .=3D ",bootindex=3D$bootorder->{$id}" if $bootorder->{$id=
};
> }
> =20
> push @$devices, '-device', $devicestr;
> diff --git a/PVE/QemuServer/USB.pm b/PVE/QemuServer/USB.pm
> index d328148..4a843cd 100644
> --- a/PVE/QemuServer/USB.pm
> +++ b/PVE/QemuServer/USB.pm
> @@ -74,13 +74,14 @@ sub get_usb_controllers {
> }
> =20
> sub get_usb_devices {
> - my ($conf, $format, $max_usb_devices, $features) =3D @_;
> + my ($conf, $format, $max_usb_devices, $features, $bootorder) =3D @=
_;
> =20
> my $devices =3D [];
> =20
> for (my $i =3D 0; $i < $max_usb_devices; $i++) {
> - next if !$conf->{"usb$i"};
> - my $d =3D eval { PVE::JSONSchema::parse_property_string($format,$conf=
->{"usb$i"}) };
> + my $devname =3D "usb$i";
> + next if !$conf->{$devname};
> + my $d =3D eval { PVE::JSONSchema::parse_property_string($format,$conf=
->{$devname}) };
> next if !$d;
> =20
> if (defined($d->{host})) {
> @@ -93,8 +94,10 @@ sub get_usb_devices {
> =20
> push @$devices, '-chardev', "spicevmc,id=3Dusbredirchardev$i,name=3D=
usbredir";
> push @$devices, '-device', "usb-redir,chardev=3Dusbredirchardev$i,id=
=3Dusbredirdev$i,bus=3D$bus.0";
> +
> + warn "warning: spice usb port set as bootdevice, ignoring\n" if $boo=
torder->{$devname};
> } else {
> - push @$devices, '-device', print_usbdevice_full($conf, "usb$i", $hos=
tdevice);
> + push @$devices, '-device', print_usbdevice_full($conf, $devname, $ho=
stdevice, $bootorder);
> }
> }
> }
> @@ -103,7 +106,7 @@ sub get_usb_devices {
> }
> =20
> sub print_usbdevice_full {
> - my ($conf, $deviceid, $device) =3D @_;
> + my ($conf, $deviceid, $device, $bootorder) =3D @_;
> =20
> return if !$device;
> my $usbdevice =3D "usb-host";
> @@ -120,6 +123,7 @@ sub print_usbdevice_full {
> }
> =20
> $usbdevice .=3D ",id=3D$deviceid";
> + $usbdevice .=3D ",bootindex=3D$bootorder->{$deviceid}" if $bootord=
er->{$deviceid};
> return $usbdevice;
> }
> =20
> diff --git a/test/cfg2cmd/bootorder.conf b/test/cfg2cmd/bootorder.conf
> new file mode 100644
> index 0000000..7d296da
> --- /dev/null
> +++ b/test/cfg2cmd/bootorder.conf
> @@ -0,0 +1,16 @@
> +# TEST: Test for a specific bootorder given by 'bootorder' parameter
> +# QEMU_VERSION: 5.1
> +cores: 3
> +bootorder: virtio1,net0,scsi4,ide2
> +ide2: none,media=3Dcdrom
> +memory: 768
> +name: simple
> +net0: virtio=3DA2:C0:43:77:08:A0,bridge=3Dvmbr0
> +numa: 0
> +ostype: l26
> +scsi4: local:8006/vm-8006-disk-0.qcow2,discard=3Don,size=3D104858K
> +smbios1: uuid=3D7b10d7af-b932-4c66-b2c3-3996152ec465
> +sockets: 1
> +virtio0: local:8006/vm-8006-disk-0.qcow2,discard=3Don,iothread=3D1,siz=
e=3D104858K
> +virtio1: local:8006/vm-8006-disk-0.qcow2,discard=3Don,iothread=3D1,siz=
e=3D104858K
> +vmgenid: c773c261-d800-4348-9f5d-167fadd53cf8
> diff --git a/test/cfg2cmd/bootorder.conf.cmd b/test/cfg2cmd/bootorder.c=
onf.cmd
> new file mode 100644
> index 0000000..86cae07
> --- /dev/null
> +++ b/test/cfg2cmd/bootorder.conf.cmd
> @@ -0,0 +1,38 @@
> +/usr/bin/kvm \
> + -id 8006 \
> + -name simple \
> + -chardev 'socket,id=3Dqmp,path=3D/var/run/qemu-server/8006.qmp,serve=
r,nowait' \
> + -mon 'chardev=3Dqmp,mode=3Dcontrol' \
> + -chardev 'socket,id=3Dqmp-event,path=3D/var/run/qmeventd.sock,reconn=
ect=3D5' \
> + -mon 'chardev=3Dqmp-event,mode=3Dcontrol' \
> + -pidfile /var/run/qemu-server/8006.pid \
> + -daemonize \
> + -smbios 'type=3D1,uuid=3D7b10d7af-b932-4c66-b2c3-3996152ec465' \
> + -smp '3,sockets=3D1,cores=3D3,maxcpus=3D3' \
> + -nodefaults \
> + -boot 'menu=3Don,strict=3Don,reboot-timeout=3D1000,splash=3D/usr/sha=
re/qemu-server/bootsplash.jpg' \
> + -vnc unix:/var/run/qemu-server/8006.vnc,password \
> + -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \
> + -m 768 \
> + -object 'iothread,id=3Diothread-virtio0' \
> + -object 'iothread,id=3Diothread-virtio1' \
> + -device 'pci-bridge,id=3Dpci.1,chassis_nr=3D1,bus=3Dpci.0,addr=3D0x1=
e' \
> + -device 'pci-bridge,id=3Dpci.2,chassis_nr=3D2,bus=3Dpci.0,addr=3D0x1=
f' \
> + -device 'vmgenid,guid=3Dc773c261-d800-4348-9f5d-167fadd53cf8' \
> + -device 'piix3-usb-uhci,id=3Duhci,bus=3Dpci.0,addr=3D0x1.0x2' \
> + -device 'usb-tablet,id=3Dtablet,bus=3Duhci.0,port=3D1' \
> + -device 'VGA,id=3Dvga,bus=3Dpci.0,addr=3D0x2' \
> + -device 'virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.0,addr=3D0x3' \
> + -iscsi 'initiator-name=3Diqn.1993-08.org.debian:01:aabbccddeeff' \
> + -drive 'if=3Dnone,id=3Ddrive-ide2,media=3Dcdrom,aio=3Dthreads' \
> + -device 'ide-cd,bus=3Dide.1,unit=3D0,drive=3Ddrive-ide2,id=3Dide2,bo=
otindex=3D103' \
> + -device 'lsi,id=3Dscsihw0,bus=3Dpci.0,addr=3D0x5' \
> + -drive 'file=3D/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=3Dnon=
e,id=3Ddrive-scsi4,discard=3Don,format=3Dqcow2,cache=3Dnone,aio=3Dnative,=
detect-zeroes=3Dunmap' \
> + -device 'scsi-hd,bus=3Dscsihw0.0,scsi-id=3D4,drive=3Ddrive-scsi4,id=3D=
scsi4,bootindex=3D102' \
> + -drive 'file=3D/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=3Dnon=
e,id=3Ddrive-virtio0,discard=3Don,format=3Dqcow2,cache=3Dnone,aio=3Dnativ=
e,detect-zeroes=3Dunmap' \
> + -device 'virtio-blk-pci,drive=3Ddrive-virtio0,id=3Dvirtio0,bus=3Dpci=
=2E0,addr=3D0xa,iothread=3Diothread-virtio0' \
> + -drive 'file=3D/var/lib/vz/images/8006/vm-8006-disk-0.qcow2,if=3Dnon=
e,id=3Ddrive-virtio1,discard=3Don,format=3Dqcow2,cache=3Dnone,aio=3Dnativ=
e,detect-zeroes=3Dunmap' \
> + -device 'virtio-blk-pci,drive=3Ddrive-virtio1,id=3Dvirtio1,bus=3Dpci=
=2E0,addr=3D0xb,iothread=3Diothread-virtio1,bootindex=3D100' \
> + -netdev 'type=3Dtap,id=3Dnet0,ifname=3Dtap8006i0,script=3D/var/lib/q=
emu-server/pve-bridge,downscript=3D/var/lib/qemu-server/pve-bridgedown,vh=
ost=3Don' \
> + -device 'virtio-net-pci,mac=3DA2:C0:43:77:08:A0,netdev=3Dnet0,bus=3D=
pci.0,addr=3D0x12,id=3Dnet0,bootindex=3D101' \
> + -machine 'type=3Dpc+pve0'
>=20