[pve-devel] [PATCH docs/guest-common/qemu-server 0/3] add new pci passthrough specific hookscript phase

[pve-devel] [PATCH docs/guest-common/qemu-server 0/3] add new pci passthrough specific hookscript phase

[Dominik Csapak]

this series adds a new phase to the guest hookscript that is called for each
passed throug pci device after it's prepared, but before the qemu process is
started.

See the qemu-server commit for why that is interesting.

pve-guest-common:

Dominik Csapak (1):
  helpers: exec hookscript: add optional parameters

 src/PVE/GuestHelpers.pm | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)


qemu-server:

Dominik Csapak (1):
  pci: call hookscript for each prepared pci device

 src/PVE/QemuServer.pm     | 18 +++++++++++++++---
 src/PVE/QemuServer/PCI.pm |  1 +
 2 files changed, 16 insertions(+), 3 deletions(-)


pve-docs:

Dominik Csapak (1):
  examples: add new hookscript phase to example hookscript

 examples/guest-example-hookscript.pl | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)


Summary over all repositories:
  4 files changed, 43 insertions(+), 5 deletions(-)

-- 
Generated by git-murpp 0.8.1


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH guest-common 1/1] helpers: exec hookscript: add optional parameters

[Dominik Csapak]

sometimes we may want to call the hookscript with additional parameters
in some phases, e.g. we want to call it for each pci device that was
prepared before starting with the correct uuid or pci id.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/PVE/GuestHelpers.pm | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/src/PVE/GuestHelpers.pm b/src/PVE/GuestHelpers.pm
index f8d112b..1da7619 100644
--- a/src/PVE/GuestHelpers.pm
+++ b/src/PVE/GuestHelpers.pm
@@ -115,15 +115,19 @@ sub check_hookscript {
 }
 
 sub exec_hookscript {
-    my ($conf, $vmid, $phase, $stop_on_error) = @_;
+    my ($conf, $vmid, $phase, $stop_on_error, $params) = @_;
 
     return if !$conf->{hookscript};
 
+    $params //= [];
+
     eval {
         my $hookscript = check_hookscript($conf->{hookscript});
         die $@ if $@;
 
-        PVE::Tools::run_command([$hookscript, $vmid, $phase]);
+        my $cmd = [$hookscript, $vmid, $phase];
+        push $cmd->@*, $params->@*;
+        PVE::Tools::run_command($cmd);
     };
     if (my $err = $@) {
         my $errmsg = "hookscript error for $vmid on $phase: $err\n";
-- 
2.47.3



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server 1/1] pci: call hookscript for each prepared pci device

[Dominik Csapak]

There are situations where a user might want to do extra things
for a passed through PCI device after it has been prepared/created (e.g.
in case of vGPU/mdev) but before the actual QEMU process is started.

Two examples are (both are used with NVIDIA vGPUs):
* setting 'vgpu_params' such as removing the frame-rate-limiter
* setting the gpu_instance_id for MIG devices

So instead of creating (nvidia-specific) interfaces for these, give a
user the ability to do it themselves via the hookscript as a first step.

Call it for each prepared device, so that we can give the hookscript the
'hostpciX' id, and the used uuid (in case of mdevs) or the pci id (in
case of regular or modern vGPU passthrough).

Include the generated mdev uuid in the return value of
`prepare_pci_device`, to avoid having to generate that multiple times.
With that we can get rid of one extra generation here too.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/PVE/QemuServer.pm     | 18 +++++++++++++++---
 src/PVE/QemuServer/PCI.pm |  1 +
 2 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 9d2cf44d..a676c9f4 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -5632,10 +5632,23 @@ sub vm_start_nolock {
                 if ($d->{mdev} || $d->{nvidia}) {
                     warn $@ if $@;
                     $chosen_mdev = $info;
-                    last if $chosen_mdev; # if successful, we're done
+                    if (defined($chosen_mdev)) {
+                        my $params = [$id, $chosen_mdev->{uuid} // $chosen_mdev->{name}];
+                        PVE::GuestHelpers::exec_hookscript(
+                            $conf, $vmid, 'post-pci-prepare', 1, $params,
+                        );
+                        last;
+                    }
                 } else {
                     die $@ if $@;
+                    if (defined($info)) {
+                        my $params = [$id, $info->{name}];
+                        PVE::GuestHelpers::exec_hookscript(
+                            $conf, $vmid, 'post-pci-prepare', 1, $params,
+                        );
+                    }
                 }
+
             }
 
             next if !$d->{mdev} && !$d->{nvidia};
@@ -5647,8 +5660,7 @@ sub vm_start_nolock {
                     my $smbios_conf = parse_smbios1($conf->{smbios1});
                     $uuid = $smbios_conf->{uuid} if defined($smbios_conf->{uuid});
                 }
-                $uuid = PVE::QemuServer::PCI::generate_mdev_uuid($vmid, $index)
-                    if !defined($uuid);
+                $uuid = $chosen_mdev->{uuid} if !defined($uuid);
             }
         }
         push @$cmd, '-uuid', $uuid if defined($uuid);
diff --git a/src/PVE/QemuServer/PCI.pm b/src/PVE/QemuServer/PCI.pm
index c9cf8de0..9603f5ea 100644
--- a/src/PVE/QemuServer/PCI.pm
+++ b/src/PVE/QemuServer/PCI.pm
@@ -750,6 +750,7 @@ sub prepare_pci_device {
     } elsif (my $mdev = $device->{mdev}) {
         my $uuid = generate_mdev_uuid($vmid, $index);
         PVE::SysFSTools::pci_create_mdev_device($pciid, $uuid, $mdev);
+        $info->{uuid} = $uuid;
     } else {
         die "can't unbind/bind PCI group to VFIO '$pciid'\n"
             if !PVE::SysFSTools::pci_dev_group_bind_to_vfio($pciid);
-- 
2.47.3



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs 1/1] examples: add new hookscript phase to example hookscript

[Dominik Csapak]

qemu-server has a new phase 'post-pci-prepare' that is called for vms
with pci passthrough for each device prepared.

add that to the example hookscript and explain when it's called and it's
parameters with a comment.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 examples/guest-example-hookscript.pl | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/examples/guest-example-hookscript.pl b/examples/guest-example-hookscript.pl
index 1cce2e3..d677ab3 100755
--- a/examples/guest-example-hookscript.pl
+++ b/examples/guest-example-hookscript.pl
@@ -38,6 +38,27 @@ if ($phase eq 'pre-start') {
 
     print "$vmid started successfully.\n";
 
+} elsif ($phase eq 'post-pci-prepare') {
+
+    # Only called for virtual machines, not containers.
+    #
+    # This phase will be called for each pci device that is passed through,
+    # after it was prepared by the PVE stack. In other words when either
+    # * the mdev/vGPU was created
+    # * the driver was changed to vfio-pci and the device was reset
+    #
+    # This phase has 2 additional parameters
+    #
+    # First is the 'hostpciX' index:
+
+    my $hostpci_x = shift;
+
+    # Second is either the mdev uuid or the pci id
+
+    my $mdev_or_pciid = shift;
+
+    print "Prepared PCI device for $hostpci_x with uuid/pciid: $mdev_or_pciid \n.";
+
 } elsif ($phase eq 'pre-stop') {
 
     # Third phase 'pre-stop' will be executed before stopping the guest
-- 
2.47.3



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel