[pve-devel] [PATCH guest-common/qemu-server/manager/docs v5 0/3] implement experimental vgpu live migration

[pve-devel] [PATCH guest-common/qemu-server/manager/docs v5 0/3] implement experimental vgpu live migration

[Dominik Csapak]

and some useful cleanups

This is implemented for mapped resources. This requires driver and
hardware support, but aside from nvidia vgpus there don't seem to be
many drivers (if any) that do support that.

qemu already supports that for vfio-pci devices, so nothing to be
done there besides actively enabling it.

Since we currently can't properly test it here and very much depends on
hardware/driver support, mark it as experimental everywhere (docs/api/gui).
(though i tested the live-migration part manually here by using
"exec:cat > /tmp/test" for the migration target, and "exec: cat
/tmp/test" as the 'incoming' parameter for a new vm start, which worked ;) )

i opted for marking them migratable at the mapping level, but we could
theoretically also put it in the hostpciX config instead.
(though imho it fits better in the cluster-wide resource mapping config)

also the naming/texts could probably be improved, but i think
'live-migration-capable' is very descriptive and i didn't want to
use an overly short name for it (which can be confusing, see the
'shared' flag for storages)

should mostly be the same as v4 functionality/code-wise but still a bit
changed due to the recent nvidia changes from our side, so probably
warrants a bit of a closer look in any case

changes from v4:
* rebased on master (some work due to the recent nvidia changes)
* incorporated thomas/alexanders feedback from v4

changes from v3:
* rebased on master
* split first guest-common patch into 3
* instead of merging keys, just write all expected keys in to expected_props
* made $cfg optional so it does not break callers that don't call it
* added patch to fix the cfg2cmd tests for mdev check
* added patch to show vfio state transferred for migration
* incorporated fionas feedback (mostly minor stuff)

for more details see the individual patches

changes from v2:
* rebased on master
* rework the rework of the properties check (pve-guest-common 1/4)
* properly check mdev in the gui (pve-manager 1/5)

manager patches depend on pve-guest-common/qemu-server patches
qemu-server depends on pve-guest-common patches

guest-common 3/3 breaks older qemu-server version before applying
qemu-server patches 1&2

pve-guest-common:

Dominik Csapak (3):
  mapping: pci: check the mdev configuration on the device too
  mapping: pci: add 'live-migration-capable' flag to mappings
  mapping: remove find_on_current_node

 src/PVE/Mapping/PCI.pm | 27 +++++++++++++++------------
 src/PVE/Mapping/USB.pm | 10 ----------
 2 files changed, 15 insertions(+), 22 deletions(-)

qemu-server:

Dominik Csapak (11):
  usb: mapping: move implementation of find_on_current_node here
  pci: mapping: move implementation of find_on_current_node here
  pci: mapping: check mdev config against hardware
  vm stop-cleanup: allow callers to decide error behavior
  migrate: call vm_stop_cleanup after stopping in phase3_cleanup
  pci: set 'enable-migration' to on for live-migration marked mapped
    devices
  check_local_resources: add more info per mapped device and return as
    hash
  api: enable live migration for marked mapped pci devices
  api: include not mapped resources for running vms in migrate
    preconditions
  tests: cfg2cmd: fix mdev tests
  migration: show vfio state transferred too

 PVE/API2/Qemu.pm                 | 55 ++++++++++++++++++++------------
 PVE/CLI/qm.pm                    |  2 +-
 PVE/QemuMigrate.pm               | 44 +++++++++++++++++--------
 PVE/QemuServer.pm                | 30 ++++++++++-------
 PVE/QemuServer/PCI.pm            | 24 ++++++++++++--
 PVE/QemuServer/USB.pm            | 17 ++++++++--
 test/MigrationTest/Shared.pm     |  3 ++
 test/run_config2command_tests.pl |  2 +-
 8 files changed, 123 insertions(+), 54 deletions(-)

pve-manager

Dominik Csapak (5):
  mapping: pci: include mdev in config checks
  bulk migrate: improve precondition checks
  bulk migrate: include checks for live-migratable local resources
  ui: adapt migration window to precondition api change
  fix #5175: ui: allow configuring and live migration of mapped pci
    resources

 PVE/API2/Cluster/Mapping/PCI.pm   |  2 +-
 PVE/API2/Nodes.pm                 | 27 ++++++++++++++--
 www/manager6/dc/PCIMapView.js     |  6 ++++
 www/manager6/window/Migrate.js    | 51 ++++++++++++++++++++-----------
 www/manager6/window/PCIMapEdit.js | 12 ++++++++
 5 files changed, 76 insertions(+), 22 deletions(-)

pve-docs:

Dominik Csapak (2):
  qm: resource mapping: add description for `mdev` option
  qm: resource mapping: document `live-migration-capable` setting

 qm.adoc | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH guest-common v5 1/3] mapping: pci: check the mdev configuration on the device too

[Dominik Csapak]

but that lives int he 'global' part of the mapping config, not in a
specific mapping. To check that, add it to the $configured_props from
there.

this requires all call sites to be adapted otherwise the check will
always fail for devices that are capable of mediated devices

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v4:
* rename $cfg to $cluster_mapping_cfg

 src/PVE/Mapping/PCI.pm | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)

diff --git a/src/PVE/Mapping/PCI.pm b/src/PVE/Mapping/PCI.pm
index aa56496..cdd73d9 100644
--- a/src/PVE/Mapping/PCI.pm
+++ b/src/PVE/Mapping/PCI.pm
@@ -131,7 +131,7 @@ sub options {
 
 # checks if the given config is valid for the current node
 sub assert_valid {
-    my ($name, $mapping) = @_;
+    my ($name, $mapping, $cluster_mapping_cfg) = @_;
 
     my @paths = split(';', $mapping->{path} // '');
 
@@ -161,6 +161,12 @@ sub assert_valid {
 
 	my $configured_props = { $mapping->%{qw(id iommugroup subsystem-id)} };
 
+	# check mdev from globabl mapping config, if that is given
+	if (defined($cluster_mapping_cfg)) {
+	    $expected_props->{mdev} = $info->{mdev} ? 1 : 0;
+	    $configured_props->{mdev} = $cluster_mapping_cfg->{mdev} ? 1 : 0;
+	}
+
 	for my $prop (sort keys $expected_props->%*) {
 	    next if $prop eq 'iommugroup' && $idx > 0; # check iommu only on the first device
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH guest-common v5 2/3] mapping: pci: add 'live-migration-capable' flag to mappings

[Dominik Csapak]

so that we can decide in qemu-server to allow live-migration.
The driver and QEMU must be capable of that, and it's the
admin's responsibility to know and configure that

Mark the option as experimental in the description.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/PVE/Mapping/PCI.pm | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/src/PVE/Mapping/PCI.pm b/src/PVE/Mapping/PCI.pm
index cdd73d9..3e93429 100644
--- a/src/PVE/Mapping/PCI.pm
+++ b/src/PVE/Mapping/PCI.pm
@@ -105,6 +105,13 @@ my $defaultData = {
 	    optional => 1,
 	    default => 0,
 	},
+	'live-migration-capable' => {
+	    description => "Marks the device(s) as being able to be live-migrated (Experimental)."
+		." This needs hardware and driver support to work.",
+	    type => 'boolean',
+	    optional => 1,
+	    default => 0,
+	},
 	map => {
 	    type => 'array',
 	    description => 'A list of maps for the cluster nodes.',
@@ -125,6 +132,7 @@ sub options {
     return {
 	description => { optional => 1 },
 	mdev => { optional => 1 },
+	'live-migration-capable' => { optional => 1 },
 	map => {},
     };
 }
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH guest-common v5 3/3] mapping: remove find_on_current_node

[Dominik Csapak]

they only have one user each (where we can inline the implementation).
It's easy enough to recreate should we need to.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
breaks older qemu-server that rely on this function

 src/PVE/Mapping/PCI.pm | 11 -----------
 src/PVE/Mapping/USB.pm | 10 ----------
 2 files changed, 21 deletions(-)

diff --git a/src/PVE/Mapping/PCI.pm b/src/PVE/Mapping/PCI.pm
index 3e93429..4c6aa8a 100644
--- a/src/PVE/Mapping/PCI.pm
+++ b/src/PVE/Mapping/PCI.pm
@@ -9,7 +9,6 @@ use PVE::Cluster qw(
     cfs_register_file
     cfs_write_file
 );
-use PVE::INotify ();
 use PVE::JSONSchema qw(get_standard_option parse_property_string);
 use PVE::SysFSTools ();
 
@@ -218,16 +217,6 @@ sub write_pci_config {
     cfs_write_file($FILENAME, $cfg);
 }
 
-sub find_on_current_node {
-    my ($id) = @_;
-
-    my $cfg = PVE::Mapping::PCI::config();
-    my $node = PVE::INotify::nodename();
-
-    # ignore errors
-    return get_node_mapping($cfg, $id, $node);
-}
-
 sub get_node_mapping {
     my ($cfg, $id, $nodename) = @_;
 
diff --git a/src/PVE/Mapping/USB.pm b/src/PVE/Mapping/USB.pm
index 34bc3cb..19468d8 100644
--- a/src/PVE/Mapping/USB.pm
+++ b/src/PVE/Mapping/USB.pm
@@ -9,7 +9,6 @@ use PVE::Cluster qw(
     cfs_register_file
     cfs_write_file
 );
-use PVE::INotify ();
 use PVE::JSONSchema qw(get_standard_option parse_property_string);
 use PVE::SysFSTools ();
 
@@ -155,15 +154,6 @@ sub write_usb_config {
     cfs_write_file($FILENAME, $cfg);
 }
 
-sub find_on_current_node {
-    my ($id) = @_;
-
-    my $cfg = config();
-    my $node = PVE::INotify::nodename();
-
-    return get_node_mapping($cfg, $id, $node);
-}
-
 sub get_node_mapping {
     my ($cfg, $id, $nodename) = @_;
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH guest-common v5 3/3] mapping: remove find_on_current_node

[Dominik Csapak]

On 1/20/25 15:51, Dominik Csapak wrote:
> they only have one user each (where we can inline the implementation).
> It's easy enough to recreate should we need to.
> 

turns out i forgot that we added a second user of the pci function in pve-manager

we still need to adapt the qemu-server side code still, so this would have one user after
again...

i could still do the changes similar to this version (remove the find_on_current_node here,
add a new sub in qemu-server) but add a new patch for pve-manager that makes
use of the new qemu-server sub

alternatively we could omit this patch and simply change the one place in qemu-server
where find_on_current_node is not enough

seems variant 2 is less breakage & work, any input on this @thomas?
(I'm asking you because you started to review the patches in v5)

but I'll wait with a v6 until i get more feedback on this series
(at least a user on the bugzilla reported that it works correct except the VFIO state in the 
migration log)


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH guest-common v5 3/3] mapping: remove find_on_current_node

[Thomas Lamprecht]

Am 22.01.25 um 09:16 schrieb Dominik Csapak:
> On 1/20/25 15:51, Dominik Csapak wrote:
>> they only have one user each (where we can inline the implementation).
>> It's easy enough to recreate should we need to.
>>
> 
> turns out i forgot that we added a second user of the pci function in pve-manager
> 
> we still need to adapt the qemu-server side code still, so this would have one user after
> again...
> 
> i could still do the changes similar to this version (remove the find_on_current_node here,
> add a new sub in qemu-server) but add a new patch for pve-manager that makes
> use of the new qemu-server sub
> 
> alternatively we could omit this patch and simply change the one place in qemu-server
> where find_on_current_node is not enough
> 
> seems variant 2 is less breakage & work, any input on this @thomas?
> (I'm asking you because you started to review the patches in v5)

Yeah, just skipping applying this patch here seems the least amount of
work for now.

Especially as this seems just to be for refactoring reasons, or is there
any other reason this is done? I checked the patches for further modifications
to these methods that require qemu-server specific code or the like in
subsequent patches and could not find any; but I might have overlooked
something and possibly also just forgot the discussion on this.

That said, moving can be fine for me, just would try to limit work required
to do it if there's really little benefit of doing so and if there's any
other reason it naturally would be good to have that stated in the commit
message, even if it's to help to make planned future changes easier.

> 
> but I'll wait with a v6 until i get more feedback on this series
> (at least a user on the bugzilla reported that it works correct except the VFIO state in the 
> migration log)

I skimmed over the patches and most seem OK to me from a higher level
perspective, just found one small potential patch ordering thing that
might be improved (will reply to the respective patch), so I'd like to
see some testing/review from Christoph here – ideally that results in
a slightly polished v6 with review/test trailers included that can be
directly rolled out.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 01/11] usb: mapping: move implementation of find_on_current_node here

[Dominik Csapak]

this was the only user, and it's easy enough

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v4:
* refactor code into sub to keep the 'parse_usb_device' function smaller

 PVE/QemuServer/USB.pm | 17 ++++++++++++++---
 1 file changed, 14 insertions(+), 3 deletions(-)

diff --git a/PVE/QemuServer/USB.pm b/PVE/QemuServer/USB.pm
index 017ef9c0..c701485b 100644
--- a/PVE/QemuServer/USB.pm
+++ b/PVE/QemuServer/USB.pm
@@ -5,6 +5,7 @@ use warnings;
 use PVE::QemuServer::PCI qw(print_pci_addr);
 use PVE::QemuServer::Machine;
 use PVE::QemuServer::Helpers qw(min_version windows_version);
+use PVE::INotify;
 use PVE::JSONSchema;
 use PVE::Mapping::USB;
 use base 'Exporter';
@@ -74,6 +75,18 @@ our $usbdesc = {
 };
 PVE::JSONSchema::register_standard_option("pve-qm-usb", $usbdesc);
 
+my sub get_current_node_mapping {
+    my ($mapping) = @_;
+
+    my $config = PVE::Mapping::USB::config();
+    my $node = PVE::INotify::nodename();
+    my $devices = PVE::Mapping::USB::get_node_mapping($config, $mapping, $node);
+    die "USB device mapping not found for '$mapping'\n" if !$devices || !scalar($devices->@*);
+    die "More than one USB mapping per host not supported\n" if scalar($devices->@*) > 1;
+
+    return $devices;
+}
+
 sub parse_usb_device {
     my ($value, $mapping) = @_;
 
@@ -91,9 +104,7 @@ sub parse_usb_device {
 	    $res->{spice} = 1;
 	}
     } elsif (defined($mapping)) {
-	my $devices = PVE::Mapping::USB::find_on_current_node($mapping);
-	die "USB device mapping not found for '$mapping'\n" if !$devices || !scalar($devices->@*);
-	die "More than one USB mapping per host not supported\n" if scalar($devices->@*) > 1;
+	my $devices = get_current_node_mapping($mapping);
 	eval {
 	    PVE::Mapping::USB::assert_valid($mapping, $devices->[0]);
 	};
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 02/11] pci: mapping: move implementation of find_on_current_node here

[Dominik Csapak]

this was the only user, and it's easy enough

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v4:
* factor out some code into it's own sub to keep parse_hostpci smaller
  (we need $config again later, so keep it outside the sub)

 PVE/QemuServer/PCI.pm | 15 +++++++++++++--
 1 file changed, 13 insertions(+), 2 deletions(-)

diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/PCI.pm
index d758ae9d..d6655b76 100644
--- a/PVE/QemuServer/PCI.pm
+++ b/PVE/QemuServer/PCI.pm
@@ -3,6 +3,7 @@ package PVE::QemuServer::PCI;
 use warnings;
 use strict;
 
+use PVE::INotify;
 use PVE::JSONSchema;
 use PVE::Mapping::PCI;
 use PVE::SysFSTools;
@@ -388,6 +389,16 @@ sub print_pcie_root_port {
     return $res;
 }
 
+my sub get_current_node_mapping {
+    my ($mapping_config, $mapping_name) = @_;
+
+    my $node = PVE::INotify::nodename();
+    my $devices = PVE::Mapping::PCI::get_node_mapping($mapping_config, $mapping_name, $node);
+    die "PCI device mapping not found for '$mapping_name'\n" if !$devices || !scalar($devices->@*);
+
+    return $devices;
+}
+
 # returns the parsed pci config but parses the 'host' part into
 # a list if lists into the 'id' property like this:
 #
@@ -429,8 +440,8 @@ sub parse_hostpci {
 
     if ($mapping) {
 	# we have no ordinary pci id, must be a mapping
-	my $devices = PVE::Mapping::PCI::find_on_current_node($mapping);
-	die "PCI device mapping not found for '$mapping'\n" if !$devices || !scalar($devices->@*);
+	my $config = PVE::Mapping::PCI::config();
+	my $devices = get_current_node_mapping($config, $mapping);
 
 	for my $device ($devices->@*) {
 	    eval { PVE::Mapping::PCI::assert_valid($mapping, $device) };
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH qemu-server v5 02/11] pci: mapping: move implementation of find_on_current_node here

[Christoph Heiss]

On Mon Jan 20, 2025 at 3:51 PM CET, Dominik Csapak wrote:
[..]
> +my sub get_current_node_mapping {
> +    my ($mapping_config, $mapping_name) = @_;
> +
> +    my $node = PVE::INotify::nodename();
> +    my $devices = PVE::Mapping::PCI::get_node_mapping($mapping_config, $mapping_name, $node);
> +    die "PCI device mapping not found for '$mapping_name'\n" if !$devices || !scalar($devices->@*);
> +
> +    return $devices;
> +}
> +
>  # returns the parsed pci config but parses the 'host' part into
>  # a list if lists into the 'id' property like this:
>  #
> @@ -429,8 +440,8 @@ sub parse_hostpci {
>  
>      if ($mapping) {
>  	# we have no ordinary pci id, must be a mapping
> -	my $devices = PVE::Mapping::PCI::find_on_current_node($mapping);
> -	die "PCI device mapping not found for '$mapping'\n" if !$devices || !scalar($devices->@*);
> +	my $config = PVE::Mapping::PCI::config();

Nit: Maybe move this line into get_current_node_mapping() too, much like
is done in the previous patch for PVE::QemuServer::USB? Would be nice
from a consistency point-of-view.

> +	my $devices = get_current_node_mapping($config, $mapping);


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH qemu-server v5 02/11] pci: mapping: move implementation of find_on_current_node here

[Dominik Csapak]

On 2/11/25 13:45, Christoph Heiss wrote:
> On Mon Jan 20, 2025 at 3:51 PM CET, Dominik Csapak wrote:
> [..]
>> +my sub get_current_node_mapping {
>> +    my ($mapping_config, $mapping_name) = @_;
>> +
>> +    my $node = PVE::INotify::nodename();
>> +    my $devices = PVE::Mapping::PCI::get_node_mapping($mapping_config, $mapping_name, $node);
>> +    die "PCI device mapping not found for '$mapping_name'\n" if !$devices || !scalar($devices->@*);
>> +
>> +    return $devices;
>> +}
>> +
>>   # returns the parsed pci config but parses the 'host' part into
>>   # a list if lists into the 'id' property like this:
>>   #
>> @@ -429,8 +440,8 @@ sub parse_hostpci {
>>   
>>       if ($mapping) {
>>   	# we have no ordinary pci id, must be a mapping
>> -	my $devices = PVE::Mapping::PCI::find_on_current_node($mapping);
>> -	die "PCI device mapping not found for '$mapping'\n" if !$devices || !scalar($devices->@*);
>> +	my $config = PVE::Mapping::PCI::config();
> 
> Nit: Maybe move this line into get_current_node_mapping() too, much like
> is done in the previous patch for PVE::QemuServer::USB? Would be nice
> from a consistency point-of-view.
> 

just note, this would not have worked, because I needed the $config outside of that function

but aside: i'll leave the 'find_on_current_node' code in guest-common, since we don't really
gain anything by moving that here and it makes more trouble with patch ordering and
depends/breaks cycle...

>> +	my $devices = get_current_node_mapping($config, $mapping);



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 03/11] pci: mapping: check mdev config against hardware

[Dominik Csapak]

by giving the mapping config to assert_valid, not only the specific mapping

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/QemuServer/PCI.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/PCI.pm
index d6655b76..ad6d5ff6 100644
--- a/PVE/QemuServer/PCI.pm
+++ b/PVE/QemuServer/PCI.pm
@@ -444,7 +444,7 @@ sub parse_hostpci {
 	my $devices = get_current_node_mapping($config, $mapping);
 
 	for my $device ($devices->@*) {
-	    eval { PVE::Mapping::PCI::assert_valid($mapping, $device) };
+	    eval { PVE::Mapping::PCI::assert_valid($mapping, $device, $config->{ids}->{$mapping}) };
 	    die "PCI device mapping invalid (hardware probably changed): $@\n" if $@;
 	    push $alternatives->@*, [split(/;/, $device->{path})];
 	}
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 04/11] vm stop-cleanup: allow callers to decide error behavior

[Dominik Csapak]

and keep it the same for all current callers as before by setting the
additional 'noerr' parameter to '1'.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/CLI/qm.pm     |  2 +-
 PVE/QemuServer.pm | 13 ++++++++-----
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/PVE/CLI/qm.pm b/PVE/CLI/qm.pm
index 4214a7ca..3e3a4c91 100755
--- a/PVE/CLI/qm.pm
+++ b/PVE/CLI/qm.pm
@@ -963,7 +963,7 @@ __PACKAGE__->register_method({
 
 	    if (!$clean || $guest) {
 		# vm was shutdown from inside the guest or crashed, doing api cleanup
-		PVE::QemuServer::vm_stop_cleanup($storecfg, $vmid, $conf, 0, 0);
+		PVE::QemuServer::vm_stop_cleanup($storecfg, $vmid, $conf, 0, 0, 1);
 	    }
 	    PVE::GuestHelpers::exec_hookscript($conf, $vmid, 'post-stop');
 
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 808c0e1c..fc27d0e6 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -6109,7 +6109,7 @@ sub cleanup_pci_devices {
 }
 
 sub vm_stop_cleanup {
-    my ($storecfg, $vmid, $conf, $keepActive, $apply_pending_changes) = @_;
+    my ($storecfg, $vmid, $conf, $keepActive, $apply_pending_changes, $noerr) = @_;
 
     eval {
 
@@ -6135,7 +6135,10 @@ sub vm_stop_cleanup {
 
 	vmconfig_apply_pending($vmid, $conf, $storecfg) if $apply_pending_changes;
     };
-    warn $@ if $@; # avoid errors - just warn
+    if (my $err = $@) {
+	die $err if !$noerr;
+	warn $err;
+    }
 }
 
 # call only in locked context
@@ -6186,7 +6189,7 @@ sub _do_vm_stop {
 		die "VM quit/powerdown failed - got timeout\n";
 	    }
 	} else {
-	    vm_stop_cleanup($storecfg, $vmid, $conf, $keepActive, 1) if $conf;
+	    vm_stop_cleanup($storecfg, $vmid, $conf, $keepActive, 1, 1) if $conf;
 	    return;
 	}
     } else {
@@ -6217,7 +6220,7 @@ sub _do_vm_stop {
 	sleep 1;
     }
 
-    vm_stop_cleanup($storecfg, $vmid, $conf, $keepActive, 1) if $conf;
+    vm_stop_cleanup($storecfg, $vmid, $conf, $keepActive, 1, 1) if $conf;
 }
 
 # Note: use $nocheck to skip tests if VM configuration file exists.
@@ -6232,7 +6235,7 @@ sub vm_stop {
 	my $pid = check_running($vmid, $nocheck, $migratedfrom);
 	kill 15, $pid if $pid;
 	my $conf = PVE::QemuConfig->load_config($vmid, $migratedfrom);
-	vm_stop_cleanup($storecfg, $vmid, $conf, $keepActive, 0);
+	vm_stop_cleanup($storecfg, $vmid, $conf, $keepActive, 0, 1);
 	return;
     }
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 05/11] migrate: call vm_stop_cleanup after stopping in phase3_cleanup

[Dominik Csapak]

we currently only call deactivate_volumes, but we actually want to call
the whole vm_stop_cleanup, since that is not invoked by the vm_stop
above (we cannot parse the config anymore) and might do other cleanups
we also want to do (like mdev cleanup).

For this to work properly we have to clone the original config at the
beginning, since we might modify the volids.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/QemuMigrate.pm           | 12 ++++++------
 test/MigrationTest/Shared.pm |  3 +++
 2 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index ed5ede30..c2e36334 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -5,6 +5,7 @@ use warnings;
 
 use IO::File;
 use IPC::Open2;
+use Storable qw(dclone);
 use Time::HiRes qw( usleep );
 
 use PVE::AccessControl;
@@ -1457,7 +1458,8 @@ sub phase3_cleanup {
 
     my $tunnel = $self->{tunnel};
 
-    my $sourcevollist = PVE::QemuServer::get_vm_volumes($conf);
+    # we'll need an unmodified copy of the config later for the cleanup
+    my $oldconf = dclone($conf);
 
     if ($self->{volume_map} && !$self->{opts}->{remote}) {
 	my $target_drives = $self->{target_drive};
@@ -1588,12 +1590,10 @@ sub phase3_cleanup {
 	$self->{errors} = 1;
     }
 
-    # always deactivate volumes - avoid lvm LVs to be active on several nodes
-    eval {
-	PVE::Storage::deactivate_volumes($self->{storecfg}, $sourcevollist);
-    };
+    # stop with nocheck does not do a cleanup, so do it here with the original config
+    eval { PVE::QemuServer::vm_stop_cleanup($self->{storecfg}, $vmid, $oldconf) };
     if (my $err = $@) {
-	$self->log('err', $err);
+	$self->log('err', "Cleanup after stopping VM failed - $err");
 	$self->{errors} = 1;
     }
 
diff --git a/test/MigrationTest/Shared.pm b/test/MigrationTest/Shared.pm
index aa7203d1..e69bf84f 100644
--- a/test/MigrationTest/Shared.pm
+++ b/test/MigrationTest/Shared.pm
@@ -130,6 +130,9 @@ $qemu_server_module->mock(
     clear_reboot_request => sub {
 	return 1;
     },
+    vm_stop_cleanup => sub {
+	return;
+    },
     get_efivars_size => sub {
 	 return 128 * 1024;
     },
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 06/11] pci: set 'enable-migration' to on for live-migration marked mapped devices

[Dominik Csapak]

the default is 'auto', but for those which are marked as capable for
live migration, we want to explicitly enable that, so we get an early
error on start if the driver does not support that.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/QemuServer/PCI.pm | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/PVE/QemuServer/PCI.pm b/PVE/QemuServer/PCI.pm
index ad6d5ff6..20c48f1c 100644
--- a/PVE/QemuServer/PCI.pm
+++ b/PVE/QemuServer/PCI.pm
@@ -443,8 +443,11 @@ sub parse_hostpci {
 	my $config = PVE::Mapping::PCI::config();
 	my $devices = get_current_node_mapping($config, $mapping);
 
+	my $cfg = $config->{ids}->{$mapping};
+	$res->{'live-migration-capable'} = 1 if $cfg->{'live-migration-capable'};
+
 	for my $device ($devices->@*) {
-	    eval { PVE::Mapping::PCI::assert_valid($mapping, $device, $config->{ids}->{$mapping}) };
+	    eval { PVE::Mapping::PCI::assert_valid($mapping, $device, $cfg) };
 	    die "PCI device mapping invalid (hardware probably changed): $@\n" if $@;
 	    push $alternatives->@*, [split(/;/, $device->{path})];
 	}
@@ -701,6 +704,10 @@ sub print_hostpci_devices {
 		$devicestr .= ",host=$pcidevice->{id}";
 	    }
 
+	    if ($d->{'live-migration-capable'}) {
+		$devicestr .= ",enable-migration=on"
+	    }
+
 	    my $mf_addr = $multifunction ? ".$j" : '';
 	    $devicestr .= ",id=${id}${mf_addr}${pciaddr}${mf_addr}";
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 07/11] check_local_resources: add more info per mapped device and return as hash

[Dominik Csapak]

such as the mapping name and if it's marked for live-migration (pci only)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/API2/Qemu.pm   |  2 +-
 PVE/QemuMigrate.pm |  7 ++++---
 PVE/QemuServer.pm  | 17 ++++++++++-------
 3 files changed, 15 insertions(+), 11 deletions(-)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 45fe6ea6..1fb643f4 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4729,7 +4729,7 @@ __PACKAGE__->register_method({
 	$res->{local_disks} = [ values %$local_disks ];;
 
 	$res->{local_resources} = $local_resources;
-	$res->{'mapped-resources'} = $mapped_resources;
+	$res->{'mapped-resources'} = [ sort keys $mapped_resources->%* ];
 
 	return $res;
 
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index c2e36334..2153ac42 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -229,7 +229,7 @@ sub prepare {
     my ($loc_res, $mapped_res, $missing_mappings_by_node) = PVE::QemuServer::check_local_resources($conf, $running, 1);
     my $blocking_resources = [];
     for my $res ($loc_res->@*) {
-	if (!grep($res, $mapped_res->@*)) {
+	if (!defined($mapped_res->{$res})) {
 	    push $blocking_resources->@*, $res;
 	}
     }
@@ -241,10 +241,11 @@ sub prepare {
 	}
     }
 
-    if (scalar($mapped_res->@*)) {
+    if (scalar(keys $mapped_res->%*)) {
 	my $missing_mappings = $missing_mappings_by_node->{$self->{node}};
 	if ($running) {
-	    die "can't migrate running VM which uses mapped devices: " . join(", ", $mapped_res->@*) . "\n";
+	    my $mapped_text = join(", ", sort keys $mapped_res->%*);
+	    die "can't migrate running VM which uses mapped devices: $mapped_text\n";
 	} elsif (scalar($missing_mappings->@*)) {
 	    die "can't migrate to '$self->{node}': missing mapped devices " . join(", ", $missing_mappings->@*) . "\n";
 	} else {
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index fc27d0e6..f3497583 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -2471,7 +2471,7 @@ sub check_local_resources {
     my ($conf, $state, $noerr) = @_;
 
     my @loc_res = ();
-    my $mapped_res = [];
+    my $mapped_res = {};
 
     my @non_migratable_resources = check_non_migratable_resources($conf, $state, $noerr);
     push(@loc_res, @non_migratable_resources);
@@ -2506,16 +2506,19 @@ sub check_local_resources {
 	if ($k =~ m/^usb/) {
 	    my $entry = parse_property_string('pve-qm-usb', $conf->{$k});
 	    next if $entry->{host} && $entry->{host} =~ m/^spice$/i;
-	    if ($entry->{mapping}) {
-		$add_missing_mapping->('usb', $k, $entry->{mapping});
-		push @$mapped_res, $k;
+	    if (my $name = $entry->{mapping}) {
+		$add_missing_mapping->('usb', $k, $name);
+		$mapped_res->{$k} = { name => $name };
 	    }
 	}
 	if ($k =~ m/^hostpci/) {
 	    my $entry = parse_property_string('pve-qm-hostpci', $conf->{$k});
-	    if ($entry->{mapping}) {
-		$add_missing_mapping->('pci', $k, $entry->{mapping});
-		push @$mapped_res, $k;
+	    if (my $name = $entry->{mapping}) {
+		$add_missing_mapping->('pci', $k, $name);
+		my $mapped_device = { name => $name };
+		$mapped_device->{'live-migration'} = 1
+		    if $pci_map->{ids}->{$name}->{'live-migration-capable'};
+		$mapped_res->{$k} = $mapped_device;
 	    }
 	}
 	# sockets are safe: they will recreated be on the target side post-migrate
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 08/11] api: enable live migration for marked mapped pci devices

[Dominik Csapak]

They have to be marked as 'live-migration-capable' in the mapping
config, and the driver and qemu must support it.

For the gui checks, we now return the whole object of the mapped
resources, which includes info like the name and if it's marked as
live-migration capable. (while deprecating the old 'mapped-resource'
return value, since that returns strictly less information)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/API2/Qemu.pm   |  8 +++++++-
 PVE/QemuMigrate.pm | 17 ++++++++++++-----
 2 files changed, 19 insertions(+), 6 deletions(-)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 1fb643f4..047d8da0 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4658,13 +4658,18 @@ __PACKAGE__->register_method({
 		},
 		description => "List local resources e.g. pci, usb"
 	    },
+	    # FIXME: remove with 9.0
 	    'mapped-resources' => {
 		type => 'array',
 		items => {
 		    type => 'string',
 		    description => "A mapped resource",
 		},
-		description => "List of mapped resources e.g. pci, usb"
+		description => "List of mapped resources e.g. pci, usb. Deprecated, use 'mapped-resource-info' instead."
+	    },
+	    'mapped-resource-info' => {
+		type => 'object',
+		description => "Object of mapped resources with additional information such if they're live migratable.",
 	    },
 	},
     },
@@ -4730,6 +4735,7 @@ __PACKAGE__->register_method({
 
 	$res->{local_resources} = $local_resources;
 	$res->{'mapped-resources'} = [ sort keys $mapped_resources->%* ];
+	$res->{'mapped-resource-info'} = $mapped_resources;
 
 	return $res;
 
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 2153ac42..6909fc82 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -243,11 +243,18 @@ sub prepare {
 
     if (scalar(keys $mapped_res->%*)) {
 	my $missing_mappings = $missing_mappings_by_node->{$self->{node}};
-	if ($running) {
-	    my $mapped_text = join(", ", sort keys $mapped_res->%*);
-	    die "can't migrate running VM which uses mapped devices: $mapped_text\n";
-	} elsif (scalar($missing_mappings->@*)) {
-	    die "can't migrate to '$self->{node}': missing mapped devices " . join(", ", $missing_mappings->@*) . "\n";
+	my $missing_live_mappings = [];
+	for my $key (sort keys $mapped_res->%*) {
+	    my $res = $mapped_res->{$key};
+	    my $name = "$key:$res->{name}";
+	    push $missing_live_mappings->@*, $name if !$res->{'live-migration'};
+	}
+	if (scalar($missing_mappings->@*)) {
+	    my $missing = join(", ", $missing_mappings->@*);
+	    die "can't migrate to '$self->{node}': missing mapped devices $missing\n";
+	} elsif ($running && scalar($missing_live_mappings->@*)) {
+	    my $missing = join(", ", $missing_live_mappings->@*);
+	    die "can't live migrate running VM which uses following mapped devices: $missing\n";
 	} else {
 	    $self->log('info', "migrating VM which uses mapped local devices");
 	}
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 09/11] api: include not mapped resources for running vms in migrate preconditions

[Dominik Csapak]

so that we can show a proper warning in the migrate dialog and check it
in the bulk migrate precondition check

the unavailable_storages and should be the same as before, but
we now always return (not_)allowed_nodes too.

to make the code a bit easier to read, reorganize how we construct
the (not_)allowed_nodes properties.

also add a note that we want to redesign the return values here, to make
* the api call simpler
* return better structured values

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/API2/Qemu.pm | 45 ++++++++++++++++++++++++++-------------------
 1 file changed, 26 insertions(+), 19 deletions(-)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 047d8da0..915d788d 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4594,6 +4594,8 @@ __PACKAGE__->register_method({
 	},
     },
     returns => {
+	# TODO 9.x: rework the api call to return more sensible structures
+	# e.g. a simple list of nodes with their blockers and/or notices to show
 	type => "object",
 	properties => {
 	    running => {
@@ -4607,7 +4609,7 @@ __PACKAGE__->register_method({
 		    description => "An allowed node",
 		},
 		optional => 1,
-		description => "List nodes allowed for offline migration, only passed if VM is offline"
+		description => "List nodes allowed for migration.",
 	    },
 	    not_allowed_nodes => {
 		type => 'object',
@@ -4623,7 +4625,7 @@ __PACKAGE__->register_method({
 			},
 		    },
 		},
-		description => "List not allowed nodes with additional information, only passed if VM is offline"
+		description => "List not allowed nodes with additional information.",
 	    },
 	    local_disks => {
 		type => 'array',
@@ -4701,7 +4703,6 @@ __PACKAGE__->register_method({
 
 	my ($local_resources, $mapped_resources, $missing_mappings_by_node) =
 	    PVE::QemuServer::check_local_resources($vmconf, $res->{running}, 1);
-	delete $missing_mappings_by_node->{$localnode};
 
 	my $vga = PVE::QemuServer::parse_vga($vmconf->{vga});
 	if ($res->{running} && $vga->{'clipboard'} && $vga->{'clipboard'} eq 'vnc') {
@@ -4710,28 +4711,34 @@ __PACKAGE__->register_method({
 
 	# if vm is not running, return target nodes where local storage/mapped devices are available
 	# for offline migration
-	if (!$res->{running}) {
-	    $res->{allowed_nodes} = [];
-	    my $checked_nodes = PVE::QemuServer::check_local_storage_availability($vmconf, $storecfg);
-	    delete $checked_nodes->{$localnode};
-
-	    foreach my $node (keys %$checked_nodes) {
-		my $missing_mappings = $missing_mappings_by_node->{$node};
-		if (scalar($missing_mappings->@*)) {
-		    $checked_nodes->{$node}->{'unavailable-resources'} = $missing_mappings;
-		    next;
-		}
+	$res->{allowed_nodes} = [];
+	$res->{not_allowed_nodes} = {};
 
-		if (!defined($checked_nodes->{$node}->{unavailable_storages})) {
-		    push @{$res->{allowed_nodes}}, $node;
-		}
+	my $storage_nodehash = PVE::QemuServer::check_local_storage_availability($vmconf, $storecfg);
+
+	my $nodelist = PVE::Cluster::get_nodelist();
+	for my $node ($nodelist->@*) {
+	    next if $node eq $localnode;
+
+	    # extract missing storage info
+	    if (my $storage_info = $storage_nodehash->{$node}) {
+		$res->{not_allowed_nodes}->{$node} = $storage_info;
+	    }
+
+	    # extract missing mappings info
+	    my $missing_mappings = $missing_mappings_by_node->{$node};
+	    if (scalar($missing_mappings->@*)) {
+		$res->{not_allowed_nodes}->{$node}->{'unavailable-resources'} = $missing_mappings;
+	    }
 
+	    # if nothing came up, add it to the allowed nodes
+	    if (!$res->{not_allowed_nodes}->{$node}) {
+		push $res->{allowed_nodes}->@*, $node;
 	    }
-	    $res->{not_allowed_nodes} = $checked_nodes;
 	}
 
 	my $local_disks = &$check_vm_disks_local($storecfg, $vmconf, $vmid);
-	$res->{local_disks} = [ values %$local_disks ];;
+	$res->{local_disks} = [ values %$local_disks ];
 
 	$res->{local_resources} = $local_resources;
 	$res->{'mapped-resources'} = [ sort keys $mapped_resources->%* ];
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 10/11] tests: cfg2cmd: fix mdev tests

[Dominik Csapak]

this will fail with the new checks for mdev when we don't have the
correct config.

namely a device that has mediated devices, should have 'mdev' set in the
mapping config

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 test/run_config2command_tests.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/test/run_config2command_tests.pl b/test/run_config2command_tests.pl
index 2feebd4a..b2cda648 100755
--- a/test/run_config2command_tests.pl
+++ b/test/run_config2command_tests.pl
@@ -101,6 +101,7 @@ my $pci_map_config = {
     ids => {
 	someGpu => {
 	    type => 'pci',
+	    mdev => 1,
 	    map => [
 		'node=localhost,path=0000:01:00.4,id=10de:2231,iommugroup=1',
 		'node=localhost,path=0000:01:00.5,id=10de:2231,iommugroup=1',
@@ -323,7 +324,6 @@ $pve_common_sysfstools->mock(
 	} elsif ($path =~ m/^0000:07:10/) {
 	    return {
 		iommugroup => 2,
-		mdev => 0,
 		vendor => "0x8086",
 		device => "0x1520",
 	    };
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH qemu-server v5 10/11] tests: cfg2cmd: fix mdev tests

[Thomas Lamprecht]

Am 20.01.25 um 15:51 schrieb Dominik Csapak:
> this will fail with the new checks for mdev when we don't have the
> correct config.

IIUC this patch should then be ordered earlier before any patch that
needs a newer guest-common that contains the adapted check.I 


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH qemu-server v5 11/11] migration: show vfio state transferred too

[Dominik Csapak]

there is no total here, so we can't show any, just what was transferred
up until now (if any).

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/API2/Qemu.pm   |  2 +-
 PVE/QemuMigrate.pm | 12 +++++++++++-
 2 files changed, 12 insertions(+), 2 deletions(-)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 915d788d..d0b5cb4c 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4732,7 +4732,7 @@ __PACKAGE__->register_method({
 	    }
 
 	    # if nothing came up, add it to the allowed nodes
-	    if (!$res->{not_allowed_nodes}->{$node}) {
+	    if (scalar($res->{not_allowed_nodes}->{$node}->%*) == 0) {
 		push $res->{allowed_nodes}->@*, $node;
 	    }
 	}
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 6909fc82..d6f9132b 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -1241,6 +1241,7 @@ sub phase2 {
     $self->log('info', "migrate uri => $migrate_uri failed: $merr") if $merr;
 
     my $last_mem_transferred = 0;
+    my $last_vfio_transferred = 0;
     my $usleep = 1000000;
     my $i = 0;
     my $err_count = 0;
@@ -1300,8 +1301,11 @@ sub phase2 {
 	    last;
 	}
 
-	if ($memstat->{transferred} ne $last_mem_transferred) {
+	if ($memstat->{transferred} ne $last_mem_transferred ||
+	    $stat->{vfio}->{transferred} ne $last_vfio_transferred
+	) {
 	    my $trans = $memstat->{transferred} || 0;
+	    my $vfio_transferred = $stat->{vfio}->{transferred} || 0;
 	    my $rem = $memstat->{remaining} || 0;
 	    my $total = $memstat->{total} || 0;
 	    my $speed = ($memstat->{'pages-per-second'} // 0) * ($memstat->{'page-size'} // 0);
@@ -1319,6 +1323,11 @@ sub phase2 {
 
 	    my $progress = "transferred $transferred_h of $total_h VM-state, ${speed_h}/s";
 
+	    if ($vfio_transferred > 0) {
+		my $vfio_h = render_bytes($vfio_transferred, 1);
+		$progress .= " (+ $vfio_h VFIO-state)";
+	    }
+
 	    if ($dirty_rate > $speed) {
 		my $dirty_rate_h = render_bytes($dirty_rate, 1);
 		$progress .= ", VM dirties lots of memory: $dirty_rate_h/s";
@@ -1360,6 +1369,7 @@ sub phase2 {
 	}
 
 	$last_mem_transferred = $memstat->{transferred};
+	$last_vfio_transferred = $stat->{vfio}->{transferred};
     }
 
     if ($self->{storage_migration}) {
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v5 1/5] mapping: pci: include mdev in config checks

[Dominik Csapak]

by also providing the global config in assert_valid, and by also
adding the mdev config in the 'toCheck' object in the gui

For the gui, we extract the mdev property from the global entry, and add
it to the individual mapping entries, that way we can reuse the checking
logic of the other properties.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/API2/Cluster/Mapping/PCI.pm | 2 +-
 www/manager6/dc/PCIMapView.js   | 6 ++++++
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/PVE/API2/Cluster/Mapping/PCI.pm b/PVE/API2/Cluster/Mapping/PCI.pm
index 64462d25..f85623bb 100644
--- a/PVE/API2/Cluster/Mapping/PCI.pm
+++ b/PVE/API2/Cluster/Mapping/PCI.pm
@@ -115,7 +115,7 @@ __PACKAGE__->register_method ({
 			};
 		    }
 		    for my $mapping ($mappings->@*) {
-			eval { PVE::Mapping::PCI::assert_valid($id, $mapping) };
+			eval { PVE::Mapping::PCI::assert_valid($id, $mapping, $entry) };
 			if (my $err = $@) {
 			    push $entry->{checks}->@*, {
 				severity => 'error',
diff --git a/www/manager6/dc/PCIMapView.js b/www/manager6/dc/PCIMapView.js
index 80fe3c0f..f9d43aa2 100644
--- a/www/manager6/dc/PCIMapView.js
+++ b/www/manager6/dc/PCIMapView.js
@@ -20,10 +20,15 @@ Ext.define('PVE.dc.PCIMapView', {
 	data.forEach((entry) => {
 	    ids[entry.id] = entry;
 	});
+	let mdev;
 	me.getRootNode()?.cascade(function(rec) {
+	    if (rec.data.type === 'entry') {
+		mdev = rec.data.mdev ?? 0;
+	    }
 	    if (rec.data.node !== node || rec.data.type !== 'map') {
 		return;
 	    }
+	    rec.data.mdev = mdev;
 
 	    let id = rec.data.path;
 	    if (!id.match(/\.\d$/)) {
@@ -44,6 +49,7 @@ Ext.define('PVE.dc.PCIMapView', {
 	    let toCheck = {
 		id: deviceId,
 		'subsystem-id': subId,
+		mdev: device.mdev ?? 0,
 		iommugroup: device.iommugroup !== -1 ? device.iommugroup : undefined,
 	    };
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v5 2/5] bulk migrate: improve precondition checks

[Dominik Csapak]

this now takes into account the 'not_allowed_nodes' hash we get from the
api call. With that, we can now limit the 'local_resources' check for
online vms only, as for offline guests, the 'unavailable-resources' hash
already includes mapped devices that don't exist on the target node.

This now also includes unavailable storages on target nodes.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/API2/Nodes.pm | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index 9cdf19db..6fe7a5fd 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -2331,11 +2331,23 @@ my $create_migrate_worker = sub {
 	    $invalidConditions .= join(', ', map { $_->{volid} } @{$preconditions->{local_disks}});
 	}
 
-	if (@{$preconditions->{local_resources}}) {
+	if ($online && scalar($preconditions->{local_resources}->@*)) {
 	    $invalidConditions .= "\n  Has local resources: ";
 	    $invalidConditions .= join(', ', @{$preconditions->{local_resources}});
 	}
 
+	if (my $not_allowed_nodes = $preconditions->{not_allowed_nodes}) {
+	    if (my $unavail_storages = $not_allowed_nodes->{$target}->{unavailable_storages}) {
+		$invalidConditions .= "\n  Has unavailable storages: ";
+		$invalidConditions .= join(', ', $unavail_storages->@*);
+	    }
+
+	    if (my $unavail_resources = $not_allowed_nodes->{$target}->{'unavailable-resources'}) {
+		$invalidConditions .= "\n  Has unavailable resources ";
+		$invalidConditions .= join(', ', $unavail_resources->@*);
+	    }
+	}
+
 	if ($invalidConditions && $invalidConditions ne '') {
 	    print STDERR "skip VM $vmid - precondition check failed:";
 	    die "$invalidConditions\n";
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH manager v5 2/5] bulk migrate: improve precondition checks

[Christoph Heiss]

On Mon Jan 20, 2025 at 3:51 PM CET, Dominik Csapak wrote:
[..]
> diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
> index 9cdf19db..6fe7a5fd 100644
> --- a/PVE/API2/Nodes.pm
> +++ b/PVE/API2/Nodes.pm
> @@ -2331,11 +2331,23 @@ my $create_migrate_worker = sub {
>  	    $invalidConditions .= join(', ', map { $_->{volid} } @{$preconditions->{local_disks}});
>  	}
>  
> -	if (@{$preconditions->{local_resources}}) {
> +	if ($online && scalar($preconditions->{local_resources}->@*)) {
>  	    $invalidConditions .= "\n  Has local resources: ";
>  	    $invalidConditions .= join(', ', @{$preconditions->{local_resources}});
>  	}
>  
> +	if (my $not_allowed_nodes = $preconditions->{not_allowed_nodes}) {
> +	    if (my $unavail_storages = $not_allowed_nodes->{$target}->{unavailable_storages}) {
> +		$invalidConditions .= "\n  Has unavailable storages: ";
> +		$invalidConditions .= join(', ', $unavail_storages->@*);
> +	    }
> +
> +	    if (my $unavail_resources = $not_allowed_nodes->{$target}->{'unavailable-resources'}) {
> +		$invalidConditions .= "\n  Has unavailable resources ";

Nit: Above both messages have colons before the item names, so add it
here too?

> +		$invalidConditions .= join(', ', $unavail_resources->@*);
> +	    }
> +	}
> +


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v5 3/5] bulk migrate: include checks for live-migratable local resources

[Dominik Csapak]

those should be able to migrate even for online vms. If the mapping does
not exist on the target node, that will be caught further down anyway.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 PVE/API2/Nodes.pm | 13 +++++++++++--
 1 file changed, 11 insertions(+), 2 deletions(-)

diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index 6fe7a5fd..3747f082 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -2331,9 +2331,18 @@ my $create_migrate_worker = sub {
 	    $invalidConditions .= join(', ', map { $_->{volid} } @{$preconditions->{local_disks}});
 	}
 
+	# for a live migration all local_resources must be marked as live-migratable
 	if ($online && scalar($preconditions->{local_resources}->@*)) {
-	    $invalidConditions .= "\n  Has local resources: ";
-	    $invalidConditions .= join(', ', @{$preconditions->{local_resources}});
+	    my $resource_not_live = [];
+	    for my $resource ($preconditions->{local_resources}->@*) {
+		next if $preconditions->{'mapped-resource-info'}->{$resource}->{'live-migration'};
+		push $resource_not_live->@*, $resource;
+	    }
+
+	    if (scalar($resource_not_live->@*)) {
+		$invalidConditions .= "\n  Has local resources not marked as live migratable: ";
+		$invalidConditions .= join(', ', $resource_not_live->@*);
+	    }
 	}
 
 	if (my $not_allowed_nodes = $preconditions->{not_allowed_nodes}) {
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v5 4/5] ui: adapt migration window to precondition api change

[Dominik Csapak]

we now return the 'allowed_nodes'/'not_allowed_nodes' also if the vm is
running, when it has mapped resources. So do that checks independently
so that the user has instant feedback where those resources exist.

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 www/manager6/window/Migrate.js | 26 +++++++++++++-------------
 1 file changed, 13 insertions(+), 13 deletions(-)

diff --git a/www/manager6/window/Migrate.js b/www/manager6/window/Migrate.js
index 78d03921..604b63e7 100644
--- a/www/manager6/window/Migrate.js
+++ b/www/manager6/window/Migrate.js
@@ -104,7 +104,7 @@ Ext.define('PVE.window.Migrate', {
 	onTargetChange: function(nodeSelector) {
 	    // Always display the storages of the currently selected migration target
 	    this.lookup('pveDiskStorageSelector').setNodename(nodeSelector.value);
-	    this.checkMigratePreconditions();
+	    this.checkMigratePreconditions(true);
 	},
 
 	startMigration: function() {
@@ -214,12 +214,12 @@ Ext.define('PVE.window.Migrate', {
 		migration.possible = true;
 	    }
 	    migration.preconditions = [];
+	    let target = me.lookup('pveNodeSelector').value;
+	    let disallowed = migrateStats.not_allowed_nodes?.[target] ?? {};
 
-	    if (migrateStats.allowed_nodes) {
+	    if (migrateStats.allowed_nodes && !vm.get('running')) {
 		migration.allowedNodes = migrateStats.allowed_nodes;
-		let target = me.lookup('pveNodeSelector').value;
 		if (target.length && !migrateStats.allowed_nodes.includes(target)) {
-		    let disallowed = migrateStats.not_allowed_nodes[target] ?? {};
 		    if (disallowed.unavailable_storages !== undefined) {
 			let missingStorages = disallowed.unavailable_storages.join(', ');
 
@@ -230,17 +230,17 @@ Ext.define('PVE.window.Migrate', {
 			    severity: 'error',
 			});
 		    }
+		}
+	    }
 
-		    if (disallowed['unavailable-resources'] !== undefined) {
-			let unavailableResources = disallowed['unavailable-resources'].join(', ');
+	    if (disallowed['unavailable-resources'] !== undefined) {
+		let unavailableResources = disallowed['unavailable-resources'].join(', ');
 
-			migration.possible = false;
-			migration.preconditions.push({
-			    text: 'Mapped Resources (' + unavailableResources + ') not available on selected target. ',
-			    severity: 'error',
-			});
-		    }
-		}
+		migration.possible = false;
+		migration.preconditions.push({
+		    text: 'Mapped Resources (' + unavailableResources + ') not available on selected target. ',
+		    severity: 'error',
+		});
 	    }
 
 	    let blockingResources = [];
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH manager v5 5/5] fix #5175: ui: allow configuring and live migration of mapped pci resources

[Dominik Csapak]

if the hardware/driver is capable, the admin can now mark a pci device
as 'live-migration-capable', which then tries enabling live migration
for such devices.

mark it as experimental when configuring and in the migrate window

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 www/manager6/window/Migrate.js    | 25 ++++++++++++++++++++-----
 www/manager6/window/PCIMapEdit.js | 12 ++++++++++++
 2 files changed, 32 insertions(+), 5 deletions(-)

diff --git a/www/manager6/window/Migrate.js b/www/manager6/window/Migrate.js
index 604b63e7..46fc0cf3 100644
--- a/www/manager6/window/Migrate.js
+++ b/www/manager6/window/Migrate.js
@@ -244,10 +244,10 @@ Ext.define('PVE.window.Migrate', {
 	    }
 
 	    let blockingResources = [];
-	    let mappedResources = migrateStats['mapped-resources'] ?? [];
+	    let mappedResources = migrateStats['mapped-resource-info'] ?? {};
 
 	    for (const res of migrateStats.local_resources) {
-		if (mappedResources.indexOf(res) === -1) {
+		if (!mappedResources[res]) {
 		    blockingResources.push(res);
 		}
 	    }
@@ -271,14 +271,29 @@ Ext.define('PVE.window.Migrate', {
 		}
 	    }
 
-	    if (mappedResources && mappedResources.length) {
-		if (vm.get('running')) {
+	    if (mappedResources && vm.get('running')) {
+		let allowed = [];
+		let notAllowed = [];
+		for (const [key, resource] of Object.entries(mappedResources)) {
+		    if (resource['live-migration']) {
+			allowed.push(key);
+		    } else {
+			notAllowed.push(key);
+		    }
+		}
+		if (notAllowed.length > 0) {
 		    migration.possible = false;
 		    migration.preconditions.push({
 			text: Ext.String.format('Can\'t migrate running VM with mapped resources: {0}',
-			mappedResources.join(', ')),
+			notAllowed.join(', ')),
 			severity: 'error',
 		    });
+		} else if (allowed.length > 0) {
+		    migration.preconditions.push({
+			text: Ext.String.format('Live-migrating running VM with mapped resources (Experimental): {0}',
+			allowed.join(', ')),
+			severity: 'warning',
+		    });
 		}
 	    }
 
diff --git a/www/manager6/window/PCIMapEdit.js b/www/manager6/window/PCIMapEdit.js
index faf58255..9f2ea651 100644
--- a/www/manager6/window/PCIMapEdit.js
+++ b/www/manager6/window/PCIMapEdit.js
@@ -244,6 +244,18 @@ Ext.define('PVE.window.PCIMapEditWindow', {
 			disabled: '{hideComment}',
 		    },
 		},
+		{
+		    xtype: 'proxmoxcheckbox',
+		    fieldLabel: gettext('Live Migration Capable'),
+		    labelWidth: 200,
+		    boxLabel: `<i class="fa fa-exclamation-triangle warning"></i> ${gettext('Experimental')}`,
+		    reference: 'live-migration-capable',
+		    name: 'live-migration-capable',
+		    cbind: {
+			deleteEmpty: '{!isCreate}',
+			disabled: '{hideComment}',
+		    },
+		},
 	    ],
 
 	    columnB: [
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs v5 1/2] qm: resource mapping: add description for `mdev` option

[Dominik Csapak]

in a new section about additional options

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v4:
* use different asciidoc style to mark section

 qm.adoc | 12 ++++++++++++
 1 file changed, 12 insertions(+)

diff --git a/qm.adoc b/qm.adoc
index 4bb8f2c..6f337fe 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -1944,6 +1944,18 @@ To create mappings `Mapping.Modify` on `/mapping/<type>/<name>` is necessary
 To use these mappings, `Mapping.Use` on `/mapping/<type>/<name>` is necessary
 (in addition to the normal guest privileges to edit the configuration).
 
+.Additional Options
+
+There are additional options when defining a cluster wide resource mapping.
+Currently there are the following options:
+
+* `mdev` (PCI): This marks the PCI device as being capable of providing
+  `mediated devices`. When this is enabled, you can select a type when
+  configuring it on the guest. If multiple PCI devices are selected for the
+  mapping, the mediated device will be created on the first one where there are
+  any available instances of the selected type.
+
+
 Managing Virtual Machines with `qm`
 ------------------------------------
 
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

[pve-devel] [PATCH docs v5 2/2] qm: resource mapping: document `live-migration-capable` setting

[Dominik Csapak]

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v4:
* use same wording as the previous option (`the PCI device`)

 qm.adoc | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/qm.adoc b/qm.adoc
index 6f337fe..0d18d7e 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -1955,6 +1955,12 @@ Currently there are the following options:
   mapping, the mediated device will be created on the first one where there are
   any available instances of the selected type.
 
+* `live-migration-capable` (PCI): This marks the PCI device as being capable of
+  being live migrated between nodes. This requires driver and hardware support.
+  Only NVIDIA GPUs with recent kernel are known to support this. Note that live
+  migrating passed through devices is an experimental feature and may not work
+  or cause issues.
+
 
 Managing Virtual Machines with `qm`
 ------------------------------------
-- 
2.39.5



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH guest-common/qemu-server/manager/docs v5 0/3] implement experimental vgpu live migration

[Christoph Heiss]

I'm not _that_ familiar (yet) really with the resource mapping and
live-migration code. So I'd not consider this a full in-depth review.

But looking over the patches, nothing odd stood out to me, at least.

Small nit: personally I'd just squash the two pve-docs patches, since
they touch the exact same section anyway. But no hard feelings on that,
really.

Other than that + the two nits as noted directly in replies, looks good
to them.

Please consider this series thus:

Reviewed-by: Christoph Heiss <c.heiss@proxmox.com>

I'll do some testing when v6 is posted, after talking shortly with
Dominik about it - as there are already some fixes prepared.


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH guest-common/qemu-server/manager/docs v5 0/3] implement experimental vgpu live migration

[Eneko Lacunza via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 10852 bytes --]

From: Eneko Lacunza <elacunza@binovo.es>
To: pve-devel@lists.proxmox.com
Subject: Re: [pve-devel] [PATCH guest-common/qemu-server/manager/docs v5 0/3] implement experimental vgpu live migration
Date: Wed, 5 Mar 2025 11:34:55 +0100
Message-ID: <3492611b-e733-40f5-a542-9c03eb4bec9b@binovo.es>

Hi Dominik,

It is very likely we'll have access to a suitable cluster to test this 
before summer, provided these patches are in published packages.

I can test and report back if that's helpful.

Regards

El 20/1/25 a las 15:51, Dominik Csapak escribió:
> and some useful cleanups
>
> This is implemented for mapped resources. This requires driver and
> hardware support, but aside from nvidia vgpus there don't seem to be
> many drivers (if any) that do support that.
>
> qemu already supports that for vfio-pci devices, so nothing to be
> done there besides actively enabling it.
>
> Since we currently can't properly test it here and very much depends on
> hardware/driver support, mark it as experimental everywhere (docs/api/gui).
> (though i tested the live-migration part manually here by using
> "exec:cat > /tmp/test" for the migration target, and "exec: cat
> /tmp/test" as the 'incoming' parameter for a new vm start, which worked ;) )
>
> i opted for marking them migratable at the mapping level, but we could
> theoretically also put it in the hostpciX config instead.
> (though imho it fits better in the cluster-wide resource mapping config)
>
> also the naming/texts could probably be improved, but i think
> 'live-migration-capable' is very descriptive and i didn't want to
> use an overly short name for it (which can be confusing, see the
> 'shared' flag for storages)
>
> should mostly be the same as v4 functionality/code-wise but still a bit
> changed due to the recent nvidia changes from our side, so probably
> warrants a bit of a closer look in any case
>
> changes from v4:
> * rebased on master (some work due to the recent nvidia changes)
> * incorporated thomas/alexanders feedback from v4
>
> changes from v3:
> * rebased on master
> * split first guest-common patch into 3
> * instead of merging keys, just write all expected keys in to expected_props
> * made $cfg optional so it does not break callers that don't call it
> * added patch to fix the cfg2cmd tests for mdev check
> * added patch to show vfio state transferred for migration
> * incorporated fionas feedback (mostly minor stuff)
>
> for more details see the individual patches
>
> changes from v2:
> * rebased on master
> * rework the rework of the properties check (pve-guest-common 1/4)
> * properly check mdev in the gui (pve-manager 1/5)
>
> manager patches depend on pve-guest-common/qemu-server patches
> qemu-server depends on pve-guest-common patches
>
> guest-common 3/3 breaks older qemu-server version before applying
> qemu-server patches 1&2
>
> pve-guest-common:
>
> Dominik Csapak (3):
>    mapping: pci: check the mdev configuration on the device too
>    mapping: pci: add 'live-migration-capable' flag to mappings
>    mapping: remove find_on_current_node
>
>   src/PVE/Mapping/PCI.pm | 27 +++++++++++++++------------
>   src/PVE/Mapping/USB.pm | 10 ----------
>   2 files changed, 15 insertions(+), 22 deletions(-)
>
> qemu-server:
>
> Dominik Csapak (11):
>    usb: mapping: move implementation of find_on_current_node here
>    pci: mapping: move implementation of find_on_current_node here
>    pci: mapping: check mdev config against hardware
>    vm stop-cleanup: allow callers to decide error behavior
>    migrate: call vm_stop_cleanup after stopping in phase3_cleanup
>    pci: set 'enable-migration' to on for live-migration marked mapped
>      devices
>    check_local_resources: add more info per mapped device and return as
>      hash
>    api: enable live migration for marked mapped pci devices
>    api: include not mapped resources for running vms in migrate
>      preconditions
>    tests: cfg2cmd: fix mdev tests
>    migration: show vfio state transferred too
>
>   PVE/API2/Qemu.pm                 | 55 ++++++++++++++++++++------------
>   PVE/CLI/qm.pm                    |  2 +-
>   PVE/QemuMigrate.pm               | 44 +++++++++++++++++--------
>   PVE/QemuServer.pm                | 30 ++++++++++-------
>   PVE/QemuServer/PCI.pm            | 24 ++++++++++++--
>   PVE/QemuServer/USB.pm            | 17 ++++++++--
>   test/MigrationTest/Shared.pm     |  3 ++
>   test/run_config2command_tests.pl |  2 +-
>   8 files changed, 123 insertions(+), 54 deletions(-)
>
> pve-manager
>
> Dominik Csapak (5):
>    mapping: pci: include mdev in config checks
>    bulk migrate: improve precondition checks
>    bulk migrate: include checks for live-migratable local resources
>    ui: adapt migration window to precondition api change
>    fix #5175: ui: allow configuring and live migration of mapped pci
>      resources
>
>   PVE/API2/Cluster/Mapping/PCI.pm   |  2 +-
>   PVE/API2/Nodes.pm                 | 27 ++++++++++++++--
>   www/manager6/dc/PCIMapView.js     |  6 ++++
>   www/manager6/window/Migrate.js    | 51 ++++++++++++++++++++-----------
>   www/manager6/window/PCIMapEdit.js | 12 ++++++++
>   5 files changed, 76 insertions(+), 22 deletions(-)
>
> pve-docs:
>
> Dominik Csapak (2):
>    qm: resource mapping: add description for `mdev` option
>    qm: resource mapping: document `live-migration-capable` setting
>
>   qm.adoc | 18 ++++++++++++++++++
>   1 file changed, 18 insertions(+)
>

Eneko Lacunza
Zuzendari teknikoa | Director técnico
Binovo IT Human Project

Tel. +34 943 569 206 | https://www.binovo.es
Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun

https://www.youtube.com/user/CANALBINOVO
https://www.linkedin.com/company/37269706/



[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH guest-common/qemu-server/manager/docs v5 0/3] implement experimental vgpu live migration

[Dominik Csapak]

On 3/5/25 11:34, Eneko Lacunza via pve-devel wrote:
> Hi Dominik,
> 
> It is very likely we'll have access to a suitable cluster to test this before summer, provided these 
> patches are in published packages.
> 
> I can test and report back if that's helpful.

Note that this series was superseded by a next version v6 (disregard the wrong subject...)
https://lore.proxmox.com/pve-devel/20250213131716.3062383-1-d.csapak@proxmox.com/

I'll see that I can get a colleague to test/review the new version so we can include this, maybe soon.

If you would be able to test (after or before it's applied) it would be great :)
(see also the discussion on the bug tracker: https://bugzilla.proxmox.com/show_bug.cgi?id=5175 )

> 
> Regards



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

Re: [pve-devel] [PATCH guest-common/qemu-server/manager/docs v5 0/3] implement experimental vgpu live migration

[Eneko Lacunza via pve-devel]

[-- Attachment #1: Type: message/rfc822, Size: 6875 bytes --]

From: Eneko Lacunza <elacunza@binovo.es>
To: Dominik Csapak <d.csapak@proxmox.com>, Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Subject: Re: [pve-devel] [PATCH guest-common/qemu-server/manager/docs v5 0/3] implement experimental vgpu live migration
Date: Thu, 6 Mar 2025 15:42:54 +0100
Message-ID: <c1d24cde-d4f2-4bcd-b955-8c3e4aa6f4f9@binovo.es>

Hi Dominik,

El 6/3/25 a las 12:06, Dominik Csapak escribió:
> On 3/5/25 11:34, Eneko Lacunza via pve-devel wrote:
>> Hi Dominik,
>>
>> It is very likely we'll have access to a suitable cluster to test 
>> this before summer, provided these patches are in published packages.
>>
>> I can test and report back if that's helpful.
>
> Note that this series was superseded by a next version v6 (disregard 
> the wrong subject...)
> https://lore.proxmox.com/pve-devel/20250213131716.3062383-1-d.csapak@proxmox.com/ 
>
>
> I'll see that I can get a colleague to test/review the new version so 
> we can include this, maybe soon.
>
> If you would be able to test (after or before it's applied) it would 
> be great :)
> (see also the discussion on the bug tracker: 
> https://bugzilla.proxmox.com/show_bug.cgi?id=5175 )
>

Thank you, I'll reply in the bug tracker with our findings when I get 
access.

Regards

Eneko Lacunza
Zuzendari teknikoa | Director técnico
Binovo IT Human Project

Tel. +34 943 569 206 | https://www.binovo.es
Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun

https://www.youtube.com/user/CANALBINOVO
https://www.linkedin.com/company/37269706/



[-- Attachment #2: Type: text/plain, Size: 160 bytes --]

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel