Re: [pve-devel] [PATCH common/qemu-server/manager] improve vGPU (mdev) usage for NVIDIA

[Dominik Csapak <d.csapak@proxmox.com>]

On 8/17/22 01:15, DERUMIER, Alexandre wrote:
> Le 9/08/22 à 10:39, Dominik Csapak a écrit :
>> On 8/9/22 09:59, DERUMIER, Alexandre wrote:
>>> Le 26/07/22 à 08:55, Dominik Csapak a écrit :
>>>> so maybe someone can look at that and give some feedback?
>>>> my idea there would be to allow multiple device mappings per node
>>>> (instead of one only) and the qemu code would select one automatically
>>> Hi Dominik,
>>>
>>> do you want to create some kind of pool of pci devices in your ""add
>>> cluster-wide hardware device mapping" patches series ?
>>>
>>> Maybe in hardwaremap, allow to define multiple pci address on same node ?
>>>
>>> Then, for mdev, look if a mdev already exist in 1 of the device.
>>> If not, try to create the mdev if 1 device, if it's failing (max
>>> number of mdev reached), try to create mdev on the other device,...
>>>
>>> if not mdev, choose a pci device in the pool not yet detached from host.
>>>
>>
>> yes i plan to do this in my next iteration of the mapping series
>> (basically what you describe)
> Hi, sorry to be late.
> 
> 
>> my (rough) idea:
>>
>> have a list of pci paths in mapping (e.g. 01:00.0;01:00.4;...)
>> (should be enough, i don't think grouping unrelated devices (different
>> vendor/product) makes much sense?)
> yes, that's enough for me. we don't want to mix unrelated devices.
> 
> BTW, I'm finally able to do live migration with nvidia mdev vgpu. (need
> to compile the nvidia vfio driver with an option to enable it + add
> "-device vfio-pci,x-enable-migration=on,..."

nice (what flag do you need on the driver install? i did not find it)
i'll see if i can test that on a single card (only have one here)

> 
> So, maybe adding a "livemigrate" flag on the hardwaremap could be great :)

it's probably better suited for the hostpci setting in the qemu config,
since that's the place we need it

> 
> Could be usefull for stateless usb device, like usb dongle,where we
> could unplug usb/livemigrate/replug usb.
> 
> 
> 
also probably better suited for the usbX setting

but those can be done after (some version of) this series
is applied