[pve-devel] [PATCH manager/nvidia-vgpu-helper v4 0/5] reduce setup steps for nvidia vgpu drivers

[pve-devel] [PATCH manager/nvidia-vgpu-helper v4 0/5] reduce setup steps for nvidia vgpu drivers

[Hannes Duerr]

Changes in v4:
in commits

Changes in v3:
* install headers for every installed kernel version by default
* additionally add patch to only install headers for running kernel
  version and newer ones, this requires the new dependency
  "libdpkg-perl"
* remove unnecessary intrusive "Dpkg:Options::=--force-confnew"
* rename systemd template unit to "pve-nvidia-sriov@.service"
* check if path "/usr/lib/nvidia/sriov-manage" exists in systemd
  template unit


Changes in v2:
* patches contain all changes to build new repository
* make pve-manager depend on this package instead of the other way around
* install the script to /usr/bin/
* rename the script to pve-nvidia-vgpu-helper because it is only
  relevant for PVE(the repository should therefore also be renamed
  when created)

The aim of the repository is to reduce the necessary installation
steps for the Nvidia VGPU drivers [0]. The package installs a script
which can be used to check and install necessary dependencies and a
systemd template service which can be used to configure the SR-IOV per
pci-id

Part of the changes would later be the adjustment of the wiki page

[0] https://pve.proxmox.com/wiki/NVIDIA_vGPU_on_Proxmox_VE

pve-nvidia-vgpu-helper:

Hannes Duerr (4):
  create a debian package to make the installation of Nvidia vGPU
    drivers more convenient
  debian/control: add dependency for helper script
  add pve-nvidia-vgpu-helper and Makefile to make dependency
    installation more convenient
  debian: add and install pve-nvidia-sriov systemd template unit file


pve-manager:

Hannes Duerr (1):
  debian/control: add pve-nvidia-vgpu-helper as dependency

 debian/control | 1 +
 1 file changed, 1 insertion(+)


Summary over all repositories:
  1 files changed, 1 insertions(+), 0 deletions(-)

-- 
Generated by git-murpp 0.8.0


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-nvidia-vgpu-helper v4 1/4] create a debian package to make the installation of Nvidia vGPU drivers more convenient

[Hannes Duerr]

Signed-off-by: Hannes Duerr <h.duerr@proxmox.com>
---
 debian/changelog     |  5 +++++
 debian/control       | 15 +++++++++++++++
 debian/copyright     | 14 ++++++++++++++
 debian/rules         |  8 ++++++++
 debian/source/format |  1 +
 5 files changed, 43 insertions(+)
 create mode 100644 debian/changelog
 create mode 100644 debian/control
 create mode 100644 debian/copyright
 create mode 100755 debian/rules
 create mode 100644 debian/source/format

diff --git a/debian/changelog b/debian/changelog
new file mode 100644
index 0000000..de5e10a
--- /dev/null
+++ b/debian/changelog
@@ -0,0 +1,5 @@
+pve-nvidia-vgpu-helper (8.3.3) UNRELEASED; urgency=medium
+
+  * Initial release.
+
+ -- Proxmox Support Team <support@proxmox.com>  Mon, 20 Jan 2025 17:02:52 +0100
diff --git a/debian/control b/debian/control
new file mode 100644
index 0000000..334bf25
--- /dev/null
+++ b/debian/control
@@ -0,0 +1,15 @@
+Source: pve-nvidia-vgpu-helper
+Section: admin
+Priority: optional
+Maintainer: Proxmox Support Team <support@proxmox.com>
+Build-Depends: debhelper-compat (= 13), lintian,
+Standards-Version: 4.6.2
+Homepage: https://www.proxmox.com
+
+Package: pve-nvidia-vgpu-helper
+Architecture: all
+Depends: ${misc:Depends},
+Description: Proxmox Nvidia vGPU helper script and systemd service
+ This package provides a script, that helps with installing all required
+ packages for the Nvidia vGPU driver, and also a systemd template service which
+ configures the option SRI-OV per pci-id
diff --git a/debian/copyright b/debian/copyright
new file mode 100644
index 0000000..046356b
--- /dev/null
+++ b/debian/copyright
@@ -0,0 +1,14 @@
+Copyright (C) 2016 - 2024 Proxmox Server Solutions GmbH <support@proxmox.com>
+
+   This program is free software: you can redistribute it and/or modify
+   it under the terms of the GNU Affero General Public License as
+   published by the Free Software Foundation, either version 3 of the
+   License, or (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU Affero General Public License for more details.
+
+   You should have received a copy of the GNU Affero General Public License
+   along with this program.  If not, see <https://www.gnu.org/licenses/>.
diff --git a/debian/rules b/debian/rules
new file mode 100755
index 0000000..218df65
--- /dev/null
+++ b/debian/rules
@@ -0,0 +1,8 @@
+#!/usr/bin/make -f
+# -*- makefile -*-
+
+# Uncomment this to turn on verbose mode.
+#export DH_VERBOSE=1
+
+%:
+	dh $@
diff --git a/debian/source/format b/debian/source/format
new file mode 100644
index 0000000..89ae9db
--- /dev/null
+++ b/debian/source/format
@@ -0,0 +1 @@
+3.0 (native)
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-nvidia-vgpu-helper v4 2/4] debian/control: add dependency for helper script

[Hannes Duerr]

Signed-off-by: Hannes Duerr <h.duerr@proxmox.com>
---
 debian/control | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/debian/control b/debian/control
index 334bf25..352e63a 100644
--- a/debian/control
+++ b/debian/control
@@ -8,7 +8,9 @@ Homepage: https://www.proxmox.com
 
 Package: pve-nvidia-vgpu-helper
 Architecture: all
-Depends: ${misc:Depends},
+Depends: libapt-pkg-perl,
+         libdpkg-perl,
+         ${misc:Depends},
 Description: Proxmox Nvidia vGPU helper script and systemd service
  This package provides a script, that helps with installing all required
  packages for the Nvidia vGPU driver, and also a systemd template service which
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-nvidia-vgpu-helper v4 3/4] add pve-nvidia-vgpu-helper and Makefile to make dependency installation more convenient

[Hannes Duerr]

We add the pve-nvidia-vgpu-helper script to simplify the installation of
the required Nvidia Vgpu driver dependencies.
The script performs the following tasks
- install consistent dependencies
- check the currently running kernel and install the necessary kernel
  headers for the running kernel and any newer kernels installed
- blacklist the competing nouveau driver and add the opt-out flag
  --no-blacklist
We also add a Makefile to help build the Debian package and install the
script

Signed-off-by: Hannes Duerr <h.duerr@proxmox.com>
---

Notes:
    Changes in V4:
    * add `--help` option displaying the usage
    * as suggested by @Dominik we squash the patch, to only install headers
      for running kernel version and newer ones, into this one
    * install `proxmox-headers-$major.$minor-pve` package so that the
      headers for future updates are also installed directly
    * blacklist the nouveau driver by default and add opt-out flag
      `--no-blacklist`

 Makefile               | 54 +++++++++++++++++++++++
 pve-nvidia-vgpu-helper | 97 ++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 151 insertions(+)
 create mode 100644 Makefile
 create mode 100755 pve-nvidia-vgpu-helper

diff --git a/Makefile b/Makefile
new file mode 100644
index 0000000..c6e461d
--- /dev/null
+++ b/Makefile
@@ -0,0 +1,54 @@
+include /usr/share/dpkg/default.mk
+
+PACKAGE=pve-nvidia-vgpu-helper
+
+BINDIR=/usr/bin/
+DESTDIR=
+
+GITVERSION:=$(shell git rev-parse HEAD)
+
+BUILDDIR ?= $(PACKAGE)-$(DEB_VERSION)
+DSC=$(PACKAGE)_$(DEB_VERSION).dsc
+
+DEB=$(PACKAGE)_$(DEB_VERSION_UPSTREAM_REVISION)_all.deb
+
+all:
+deb: $(DEB)
+
+$(BUILDDIR): debian
+	rm -rf $@ $@.tmp
+	rsync -a * $@.tmp/
+	echo "git clone git://git.proxmox.com/git/pve-nvidia-vgpu-helper.git\\ngit checkout $(GITVERSION)" > $@.tmp/debian/SOURCE
+	mv $@.tmp $@
+
+$(DEB): $(BUILDDIR)
+	cd $(BUILDDIR); dpkg-buildpackage -b -uc -us
+	lintian $(DEB)
+
+dsc: $(DSC)
+	$(MAKE) clean
+	$(MAKE) $(DSC)
+	lintian $(DSC)
+
+$(DSC): $(BUILDDIR)
+	cd $(BUILDDIR); dpkg-buildpackage -S -uc -us
+
+sbuild: $(DSC)
+	sbuild $(DSC)
+
+.PHONY: install
+install: pve-nvidia-vgpu-helper
+	install -d $(DESTDIR)$(BINDIR)
+	install -m 0755 pve-nvidia-vgpu-helper $(DESTDIR)$(BINDIR)
+
+.PHONY: upload
+upload: UPLOAD_DIST ?= $(DEB_DISTRIBUTION)
+upload: $(DEB)
+	tar cf - $(DEB)|ssh repoman@repo.proxmox.com -- upload --product pve --dist $(UPLOAD_DIST)
+
+.PHONY: distclean
+distclean: clean
+
+.PHONY: clean
+clean:
+	rm -rf *~ $(PACKAGE)-[0-9]*/ $(PACKAGE)*.tar.* *.deb *.dsc *.changes *.build *.buildinfo
diff --git a/pve-nvidia-vgpu-helper b/pve-nvidia-vgpu-helper
new file mode 100755
index 0000000..8921c6f
--- /dev/null
+++ b/pve-nvidia-vgpu-helper
@@ -0,0 +1,97 @@
+#!/usr/bin/perl
+
+use strict;
+use warnings;
+
+use PVE::Tools qw(run_command);
+use PVE::SysFSTools;
+
+use AptPkg::Cache;
+use Dpkg::Version;
+use Getopt::Long;
+
+my @apt_install = qw(apt-get --no-install-recommends install --);
+my @dependencies = qw(dkms libc6-dev proxmox-default-headers);
+my @missing_packages;
+
+die "Please execute the script with root privileges\n" if $>;
+
+my $apt_cache = AptPkg::Cache->new();
+die "unable to initialize AptPkg::Cache\n" if !$apt_cache; 
+
+GetOptions('no-blacklist' => \my $no_blacklist, 'help' => \my $help);
+
+if (defined($help)) {
+    print("USAGE:\tpve-nvidia-vgpu-helper [OPTIONS]\n");
+    print("\t --help\n");
+    print("\t --no-blacklist\n");
+    exit;
+}
+
+if (!defined($no_blacklist) && !-e "/etc/modprobe.d/block-nouveau.conf") {
+    run_command(["mkdir", "-p", "/etc/modprobe.d/"]);
+    PVE::SysFSTools::file_write( "/etc/modprobe.d/block-nouveau.conf",
+        "blacklist nouveau" )
+      || die "Could not create block-nouveau.conf";
+
+    run_command(["update-initramfs", "-u", "-k", "all"]);
+}
+
+sub package_is_installed {
+    my ($package) = @_;
+    my $p = $apt_cache->{$package};
+    if (!defined($p->{CurrentState}) || $p->{CurrentState} ne "Installed") {
+	push(@missing_packages, $package);
+    }
+}
+
+sub install_newer_headers {
+    my (%installed_versions) = @_;
+    for my $version (keys(%installed_versions)) {
+	# install header for the running kernel and newer kernel versions
+	package_is_installed("proxmox-headers-$version");
+    }
+}
+
+foreach my $dependency (@dependencies) {
+    package_is_installed($dependency);
+}
+
+
+my $running_kernel;
+run_command( ['/usr/bin/uname', '-r' ],
+    outfunc => sub { $running_kernel = shift } );
+
+if ($running_kernel =~ m/^(\d+\.\d+\.\d+-\d+)-pve$/) {
+    print "You are running the proxmox kernel `proxmox-kernel-$running_kernel`\n";
+    $running_kernel = $1;
+} else {
+    die "You are not using a proxmox-kernel, please make sure that the appropriate header package is installed.\n";
+}
+
+my %installed_versions;
+run_command(['/usr/bin/dpkg-query', '-W', 'proxmox-kernel-*-pve'],
+    outfunc => sub {
+	my $installed_kernel = shift;
+	$installed_kernel =~ m/^\s*proxmox-kernel-(\d+.\d+)(.\d+-\d+)-pve\s*$/;
+	if (Dpkg::Version::version_compare("$running_kernel", "$1$2") != 1){
+	    $installed_versions{$1} = 1;
+	}
+    });
+
+install_newer_headers(%installed_versions);
+
+if (!@missing_packages){
+    print "All required packages are installed, you can continue with the Nvidia vGPU driver installation.\n";
+    exit;
+} else {
+    print "The following packages are missing:\n" . join("\n", @missing_packages) ."\n";
+    print "Would you like to install them now (y/n)?\n";
+}
+
+my $answer = <STDIN>;
+if (defined($answer) && $answer =~ m/^\s*y(?:es)?\s*$/i) {
+    if (system(@apt_install, @missing_packages) != 0) {
+	die "apt failed during the installation: ($?)\n";
+    }
+}
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-nvidia-vgpu-helper v4 4/4] debian: add and install pve-nvidia-sriov systemd template unit file

[Hannes Duerr]

SR-IOV must be enabled each time the system is restarted.
This systemd service should take over this task and enable SR-IOV per
pci-id/gpu after a system restart.

Signed-off-by: Hannes Duerr <h.duerr@proxmox.com>
---

Notes:
    Changes in v4:
    * Change nvidia-vgpud.service nvidia-vgpu-mgr.service to `Before=`
      targets and remove the 5 seconds sleep in `ExecStartPre=` because it
      is not needed anymore

 debian/pve-nvidia-sriov@.service | 12 ++++++++++++
 debian/rules                     |  3 +++
 2 files changed, 15 insertions(+)
 create mode 100644 debian/pve-nvidia-sriov@.service

diff --git a/debian/pve-nvidia-sriov@.service b/debian/pve-nvidia-sriov@.service
new file mode 100644
index 0000000..f2e4c83
--- /dev/null
+++ b/debian/pve-nvidia-sriov@.service
@@ -0,0 +1,12 @@
+[Unit]
+Description=Enable NVIDIA SR-IOV for PCI ID %i
+ConditionPathExists=/usr/lib/nvidia/sriov-manage
+After=network.target 
+Before=pve-guests.service nvidia-vgpud.service nvidia-vgpu-mgr.service
+
+[Service]
+Type=oneshot
+ExecStart=/usr/lib/nvidia/sriov-manage -e %i
+
+[Install]
+WantedBy=multi-user.target
diff --git a/debian/rules b/debian/rules
index 218df65..d5fe1f6 100755
--- a/debian/rules
+++ b/debian/rules
@@ -6,3 +6,6 @@
 
 %:
 	dh $@
+
+override_dh_installsystemd:
+	dh_installsystemd --no-start --no-enable --name pve-nvidia-sriov@ pve-nvidia-sriov@.service
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH pve-manager v4 1/1] debian/control: add pve-nvidia-vgpu-helper as dependency

[Hannes Duerr]

the package ships a script that helps to set up Nvidia vgpu drivers.

Signed-off-by: Hannes Duerr <h.duerr@proxmox.com>
---
 debian/control | 1 +
 1 file changed, 1 insertion(+)

diff --git a/debian/control b/debian/control
index 6c94df09..ab02fd76 100644
--- a/debian/control
+++ b/debian/control
@@ -89,6 +89,7 @@ Depends: apt (>= 1.5~),
          pve-firewall,
          pve-ha-manager,
          pve-i18n (>= 3.2.0~),
+         pve-nvidia-vgpu-helper,
          pve-xtermjs (>= 4.7.0-1),
          qemu-server (>= 8.2.7),
          rsync,
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager/nvidia-vgpu-helper v4 0/5] reduce setup steps for nvidia vgpu drivers

[Dominik Csapak]

One super tiny nit (that's IMHO almost bike-shed territory) but otherwise LGTM:

Maybe we want to print a short log line before blacklisting the module?
(though we can always add that later)

So consider this

Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager/nvidia-vgpu-helper v4 0/5] reduce setup steps for nvidia vgpu drivers

[Hannes Dürr]

Good point, that will certainly help one or the other.
If nobody has anything against it, I would add it to V5.

On 2/12/25 14:38, Dominik Csapak wrote:
> One super tiny nit (that's IMHO almost bike-shed territory) but 
> otherwise LGTM:
>
> Maybe we want to print a short log line before blacklisting the module?
> (though we can always add that later)
>
> So consider this
>
> Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
> Tested-by: Dominik Csapak <d.csapak@proxmox.com>
>


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel