From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id EEA8B1FF15C
for <inbox@lore.proxmox.com>; Wed, 5 Mar 2025 11:35:43 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id CF0F113522;
Wed, 5 Mar 2025 11:35:37 +0100 (CET)
Date: Wed, 5 Mar 2025 11:34:55 +0100
To: pve-devel@lists.proxmox.com
References: <20250120145203.282363-1-d.csapak@proxmox.com>
In-Reply-To: <20250120145203.282363-1-d.csapak@proxmox.com>
MIME-Version: 1.0
Message-ID: <mailman.778.1741170936.293.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Eneko Lacunza via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Eneko Lacunza <elacunza@binovo.es>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: Re: [pve-devel] [PATCH guest-common/qemu-server/manager/docs v5
0/3] implement experimental vgpu live migration
Content-Type: multipart/mixed; boundary="===============3542951793713858005=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
--===============3542951793713858005==
Content-Type: message/rfc822
Content-Disposition: inline
Return-Path: <elacunza@binovo.es>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 27508D3860
for <pve-devel@lists.proxmox.com>; Wed, 5 Mar 2025 11:35:36 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 010C9134E8
for <pve-devel@lists.proxmox.com>; Wed, 5 Mar 2025 11:35:06 +0100 (CET)
Received: from mail-wm1-x329.google.com (mail-wm1-x329.google.com [IPv6:2a00:1450:4864:20::329])
(using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Wed, 5 Mar 2025 11:35:04 +0100 (CET)
Received: by mail-wm1-x329.google.com with SMTP id 5b1f17b1804b1-43bbc8b7c65so35404275e9.0
for <pve-devel@lists.proxmox.com>; Wed, 05 Mar 2025 02:35:04 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=binovo.es; s=google; t=1741170898; x=1741775698; darn=lists.proxmox.com;
h=content-transfer-encoding:in-reply-to:content-language:from
:references:to:subject:user-agent:mime-version:date:message-id:from
:to:cc:subject:date:message-id:reply-to;
bh=F0bNqddREROFDvR3WTuCAEdlT0v39Qo0uzsyKaU0m4w=;
b=E8rdkwNx+VgOOjW03680RbvabE7B6RhMqVpg9mjtcK2HgpPTm7pWyo/t/8xI2IUGjD
pQK0Y4dAXDho+CNKV0c2mM/roc3JBKdsJtnhSOaW4UCS0HDgEw9hvwv3KIuFbUwoNB2z
m/JBzIh2E60Cw1jJRzbIs25pdTa2WQ+w20892YsRttZk+1ip2ykXXJYXaUdWZfC7nV4Q
xk9DBbfceXEq79m6q4vcMWFRn6YDfxv4W+HcY8AosHd0b70WGpCI0XFyPA0YQ2veEGuF
36zh40/YaR5lcPpCRya5qi+QYnhRzVJ6mgJW1W+udibDViNnCrzCY8ED5mocGXrO5em4
6VAQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1741170898; x=1741775698;
h=content-transfer-encoding:in-reply-to:content-language:from
:references:to:subject:user-agent:mime-version:date:message-id
:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=F0bNqddREROFDvR3WTuCAEdlT0v39Qo0uzsyKaU0m4w=;
b=NT3HPX38vJXzttfps2BoKK901Ah/rm6V1g1MQID8agcbFFTlOnZe35KoyEjI+jSBtO
YrAza98K5k8EZX0QUtQd/8bmBCLN15CTAvAj+5jObOUlWy1gCwjhM5RWIUIzg+6wJ+dJ
FtQykvR76pezoM1PmSoLqnsZgL7cKq3qwXxzF8vOpxydsFh5a/7pJ392HMrUD53DBDUy
V658SosSJZ1fMl9Peam3Qrn7JRtZqw650eoIIVvl+mzD/slmOq3OLAYwu4QR6RWpK/jK
AGzddHX4pfSAjnN9p+SATx4ACZk1v93tWYrZpl4YRS6JOELPpB8kEiby37AcFQCnlYqV
dsaA==
X-Gm-Message-State: AOJu0Yx7GAVnMN6osOkSpIUXjzb6tsFml5iIOm7qUDnV6ANvBitsLCMu
b6PZcyXLoMReXQKU2x8u71AwQnD6NvLT18u5NIUrJakD3I0HptoEAq0huj+zeorPxlWdV3ZcufQ
=
X-Gm-Gg: ASbGncuM5ddhAsfLEhvhzojOd+ibGS4xHA6Y3eu0rxoLbv/W/5KHA3Y4NMIL4H2dicf
YLlKzMekiWgEFKnWl6qPJ1+/8einalRZpfbYwDr+ebiXJtUwpWkinZBcohoEGYqRXhn9BEesrsG
mqWKqaw9SOQXgcTuAReJcMNyMP8ZgLADBMHMmtxF9bTiEV0YplCpJUxr7Lap3ydZLfoLIFzBgLj
yQqAvpmTI1dmSOPiXQ9UjXtniyMdi75VO5Ckru/2zT+CF7eX72IyRl4bLpWlY12oIusqDLE7S/r
SWJZH907D5QO4IECxarPUAxCCdvSmotiNxvU/HDnAyb8xZL9G03ssUGgElFb1ig2WcYppBkkU51
1VuYP3k3TCp7gK5xIfA==
X-Google-Smtp-Source: AGHT+IEfk824TjNuUWe7BntSncJsVhxkPLNH6YUpP1rOt79BF++JsNoD2cFzruCA7tC5HXruxUtypA==
X-Received: by 2002:a05:600c:5114:b0:43b:8198:f713 with SMTP id 5b1f17b1804b1-43bd292db63mr19452665e9.4.1741170897523;
Wed, 05 Mar 2025 02:34:57 -0800 (PST)
Received: from [172.29.37.147] (249.red-213-96-132.staticip.rima-tde.net. [213.96.132.249])
by smtp.gmail.com with ESMTPSA id 5b1f17b1804b1-43bd42c5ccesm13413295e9.19.2025.03.05.02.34.56
for <pve-devel@lists.proxmox.com>
(version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
Wed, 05 Mar 2025 02:34:56 -0800 (PST)
Message-ID: <3492611b-e733-40f5-a542-9c03eb4bec9b@binovo.es>
Date: Wed, 5 Mar 2025 11:34:55 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [pve-devel] [PATCH guest-common/qemu-server/manager/docs v5 0/3]
implement experimental vgpu live migration
To: pve-devel@lists.proxmox.com
References: <20250120145203.282363-1-d.csapak@proxmox.com>
From: Eneko Lacunza <elacunza@binovo.es>
Content-Language: es-CO
In-Reply-To: <20250120145203.282363-1-d.csapak@proxmox.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.133 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid
DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature
DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain
DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain
DMARC_PASS -0.1 DMARC pass policy
RCVD_IN_DNSWL_NONE -0.0001 Sender listed at https://www.dnswl.org/, no trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
Hi Dominik,
It is very likely we'll have access to a suitable cluster to test this
before summer, provided these patches are in published packages.
I can test and report back if that's helpful.
Regards
El 20/1/25 a las 15:51, Dominik Csapak escribió:
> and some useful cleanups
>
> This is implemented for mapped resources. This requires driver and
> hardware support, but aside from nvidia vgpus there don't seem to be
> many drivers (if any) that do support that.
>
> qemu already supports that for vfio-pci devices, so nothing to be
> done there besides actively enabling it.
>
> Since we currently can't properly test it here and very much depends on
> hardware/driver support, mark it as experimental everywhere (docs/api/gui).
> (though i tested the live-migration part manually here by using
> "exec:cat > /tmp/test" for the migration target, and "exec: cat
> /tmp/test" as the 'incoming' parameter for a new vm start, which worked ;) )
>
> i opted for marking them migratable at the mapping level, but we could
> theoretically also put it in the hostpciX config instead.
> (though imho it fits better in the cluster-wide resource mapping config)
>
> also the naming/texts could probably be improved, but i think
> 'live-migration-capable' is very descriptive and i didn't want to
> use an overly short name for it (which can be confusing, see the
> 'shared' flag for storages)
>
> should mostly be the same as v4 functionality/code-wise but still a bit
> changed due to the recent nvidia changes from our side, so probably
> warrants a bit of a closer look in any case
>
> changes from v4:
> * rebased on master (some work due to the recent nvidia changes)
> * incorporated thomas/alexanders feedback from v4
>
> changes from v3:
> * rebased on master
> * split first guest-common patch into 3
> * instead of merging keys, just write all expected keys in to expected_props
> * made $cfg optional so it does not break callers that don't call it
> * added patch to fix the cfg2cmd tests for mdev check
> * added patch to show vfio state transferred for migration
> * incorporated fionas feedback (mostly minor stuff)
>
> for more details see the individual patches
>
> changes from v2:
> * rebased on master
> * rework the rework of the properties check (pve-guest-common 1/4)
> * properly check mdev in the gui (pve-manager 1/5)
>
> manager patches depend on pve-guest-common/qemu-server patches
> qemu-server depends on pve-guest-common patches
>
> guest-common 3/3 breaks older qemu-server version before applying
> qemu-server patches 1&2
>
> pve-guest-common:
>
> Dominik Csapak (3):
> mapping: pci: check the mdev configuration on the device too
> mapping: pci: add 'live-migration-capable' flag to mappings
> mapping: remove find_on_current_node
>
> src/PVE/Mapping/PCI.pm | 27 +++++++++++++++------------
> src/PVE/Mapping/USB.pm | 10 ----------
> 2 files changed, 15 insertions(+), 22 deletions(-)
>
> qemu-server:
>
> Dominik Csapak (11):
> usb: mapping: move implementation of find_on_current_node here
> pci: mapping: move implementation of find_on_current_node here
> pci: mapping: check mdev config against hardware
> vm stop-cleanup: allow callers to decide error behavior
> migrate: call vm_stop_cleanup after stopping in phase3_cleanup
> pci: set 'enable-migration' to on for live-migration marked mapped
> devices
> check_local_resources: add more info per mapped device and return as
> hash
> api: enable live migration for marked mapped pci devices
> api: include not mapped resources for running vms in migrate
> preconditions
> tests: cfg2cmd: fix mdev tests
> migration: show vfio state transferred too
>
> PVE/API2/Qemu.pm | 55 ++++++++++++++++++++------------
> PVE/CLI/qm.pm | 2 +-
> PVE/QemuMigrate.pm | 44 +++++++++++++++++--------
> PVE/QemuServer.pm | 30 ++++++++++-------
> PVE/QemuServer/PCI.pm | 24 ++++++++++++--
> PVE/QemuServer/USB.pm | 17 ++++++++--
> test/MigrationTest/Shared.pm | 3 ++
> test/run_config2command_tests.pl | 2 +-
> 8 files changed, 123 insertions(+), 54 deletions(-)
>
> pve-manager
>
> Dominik Csapak (5):
> mapping: pci: include mdev in config checks
> bulk migrate: improve precondition checks
> bulk migrate: include checks for live-migratable local resources
> ui: adapt migration window to precondition api change
> fix #5175: ui: allow configuring and live migration of mapped pci
> resources
>
> PVE/API2/Cluster/Mapping/PCI.pm | 2 +-
> PVE/API2/Nodes.pm | 27 ++++++++++++++--
> www/manager6/dc/PCIMapView.js | 6 ++++
> www/manager6/window/Migrate.js | 51 ++++++++++++++++++++-----------
> www/manager6/window/PCIMapEdit.js | 12 ++++++++
> 5 files changed, 76 insertions(+), 22 deletions(-)
>
> pve-docs:
>
> Dominik Csapak (2):
> qm: resource mapping: add description for `mdev` option
> qm: resource mapping: document `live-migration-capable` setting
>
> qm.adoc | 18 ++++++++++++++++++
> 1 file changed, 18 insertions(+)
>
Eneko Lacunza
Zuzendari teknikoa | Director técnico
Binovo IT Human Project
Tel. +34 943 569 206 | https://www.binovo.es
Astigarragako Bidea, 2 - 2º izda. Oficina 10-11, 20180 Oiartzun
https://www.youtube.com/user/CANALBINOVO
https://www.linkedin.com/company/37269706/
--===============3542951793713858005==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--===============3542951793713858005==--