From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 5A31B7361C for ; Thu, 7 Oct 2021 14:01:15 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 468CA1994B for ; Thu, 7 Oct 2021 14:00:45 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 90E8D1993C for ; Thu, 7 Oct 2021 14:00:44 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 5FAB24586F for ; Thu, 7 Oct 2021 14:00:44 +0200 (CEST) Message-ID: <16b65490-9445-07d5-9374-c9649d6735ec@proxmox.com> Date: Thu, 7 Oct 2021 13:59:31 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:93.0) Gecko/20100101 Thunderbird/93.0 Content-Language: en-US To: Proxmox VE development discussion , Dominik Csapak References: <20211007093728.2071756-1-d.csapak@proxmox.com> <20211007093728.2071756-2-d.csapak@proxmox.com> From: Thomas Lamprecht In-Reply-To: <20211007093728.2071756-2-d.csapak@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 1.196 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -1.964 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [pci.pm] Subject: Re: [pve-devel] [PATCH qemu-server v2 1/2] pci: add helpers to (un)reserve pciids for a vm X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Oct 2021 12:01:15 -0000 On 07.10.21 11:37, Dominik Csapak wrote: > saves a list of pciid <-> vmid mappings in /var/run > that we can check when we start a vm > > if we're not given a pid but a timeout, we save the time when the > reservation will run out (current time + timeout + 5s) since each > vm start (until we can save the pid) varies from config to config > a few comments, some of them could have happend for v1 so sorry for any churn.. > Signed-off-by: Dominik Csapak > --- > PVE/QemuServer/PCI.pm | 99 +++++++++++++++++++++++++++++++++++++++++++ > 1 file changed, 99 insertions(+) > > diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/PCI.pm > index 5608207..364ceb7 100644 > --- a/PVE/QemuServer/PCI.pm > +++ b/PVE/QemuServer/PCI.pm > @@ -5,6 +5,7 @@ use strict; > > use PVE::JSONSchema; > use PVE::SysFSTools; > +use PVE::Tools; > > use base 'Exporter'; > > @@ -461,4 +462,102 @@ sub print_hostpci_devices { > return ($kvm_off, $gpu_passthrough, $legacy_igd); > } > > +my $PCIID_RESERVATION_FILE = "/var/run/pve-reserved-pciids"; > +my $PCIID_RESERVATION_LCK = "/var/lock/pve-reserved-pciids.lck"; > + > +my $parse_pci_reservation = sub { > + my $pciids = {}; > + > + if (my $fh = IO::File->new ($PCIID_RESERVATION_FILE, "r")) { > + while (my $line = <$fh>) { > + if ($line =~ m/^($PCIRE)\s(\d+)\stime\:(\d+)$/) { > + $pciids->{$1} = { > + vmid => $2, > + timestamp => $3, > + }; > + } elsif ($line =~ m/^($PCIRE)\s(\d+)\spid\:(\d+)$/s) { > + $pciids->{$1} = { > + vmid => $2, > + pid => $3, > + }; > + } could probably be (untested): if ($line =~ m/^($PCIRE)\s(\d+)\s(time|pid)\:(\d+)$/) { $pciids->{$1} = { vmid => $2, "$3" => $4, }; } Alternative: use a format like "$vmid $timestamp [$pid]" we'd keep the time of initial reservation info that way, not sure how much use that has though ^^ > + } > + } > + > + return $pciids; > +}; > + > +my $write_pci_reservation = sub { > + my ($pciids) = @_; > + > + my $data = ""; > + foreach my $p (keys %$pciids) { > + my $entry = $pciids->{$p}; > + if (defined($entry->{pid})) { > + $data .= "$p $entry->{vmid} pid:$entry->{pid}\n"; > + } else { > + $data .= "$p $entry->{vmid} time:$entry->{timestamp}\n"; > + } > + } > + > + PVE::Tools::file_set_contents($PCIID_RESERVATION_FILE, $data); > +}; > + > +sub remove_pci_reservation { > + my ($id) = @_; it could be an optimization to allow passing a array ref of IDs, less open+writes+rename, while thanks to /run being a tmpfs no IO is happening but syscalls are expensive, so when they're as easy to reduce as here I'd opt for that. Could do the same in a similar spirit for registering then. > + > + my $code = sub { > + my $pciids = $parse_pci_reservation->(); > + > + delete $pciids->{$id}; > + > + $write_pci_reservation->($pciids); > + }; > + > + PVE::Tools::lock_file($PCIID_RESERVATION_LCK, 10, $code); > + die $@ if $@; > + > + return; > +} > + > +sub reserve_pci_usage { > + my ($id, $vmid, $timeout, $pid) = @_; > + > + PVE::Tools::lock_file($PCIID_RESERVATION_LCK, 10, sub { > + > + my $ctime = time(); > + my $errmsg = "PCI device '$id' already in use\n"; > + > + my $pciids = $parse_pci_reservation->(); > + > + if (my $pciid = $pciids->{$id}) { > + if ($pciid->{vmid} != $vmid) { > + # check time based reservation > + die $errmsg if defined($pciid->{timestamp}) && $pciid->{timestamp} > $ctime; > + > + # check running vm > + my $pid = PVE::QemuServer::Helpers::vm_running_locally($pciid->{vmid}); > + die $errmsg if defined($pciid->{pid}) && $pid == $pciid->{pid}; Maybe we should warn in the != case, as that would mean a registration was not cleaned up nicely and recording that in the task log should not hurt. > + } > + } > + > + $pciids->{$id} = { > + vmid => $vmid, > + }; > + if (defined($timeout)) { > + # we save the timestamp when the reservation gets invalid (+5s), > + # since the start timeout depends on the config > + $pciids->{$id}->{timestamp} = $ctime + $timeout + 5; > + } > + $pciids->{$id}->{pid} = $pid if defined($pid); > + > + $write_pci_reservation->($pciids); > + > + return; > + }); > + die $@ if $@; > + > + return; > +} > + > 1; >