Re: [RFC qemu-server 0/4] remote migrate: extract preconditions and add check endpoint

[Fiona Ebner <f.ebner@proxmox.com>]

Am 21.07.26 um 1:58 PM schrieb Erik Fastermann:
> This series reworks how the QEMU remote-migration precondition checks
> are structured and exposes them through a new endpoint, so blockers and
> warnings can be surfaced before a migration is started rather than
> mid-flight.
> 
> It is deliberately sent as an RFC: the change touches a critical path,
> and the larger direction (see the open questions below) matters more
> than the current diff. Thanks to @Fabian Gruenbichler for the discussion
> so far.
> 
> Motivation
> ==========
> 
> Today the checks run at the very start of a remote migration and abort
> on the first error via die. Two problems follow:
> 
> - Many prerequisites are not checked up front at all; they only surface
>   once the migration is already running, e.g. local/mapped devices or a
>   VNC clipboard that is not live-migratable. A user fixes one blocker,
>   retries, and hits the next. A forum user collected a checklist of such
>   prerequisites [0] (thanks @Arthur Bied-Charreton for pointing this
>   out).
> 
> - The qm CLI wrapper only ran a subset of the checks the API path ran,
>   so direct API callers (e.g. the web UI) and the CLI disagreed on what
>   was validated.
> 
> What the series does
> ====================
> 
> 1. Drop the ineffective fingerprint auto-detection.
> 
> 2. Extract the checks into a validate_remote_migrate_preconditions
>    helper.
> 
> 3. Register the remote-migrate command against PVE::API2::Qemu
>    directly and drop the CLI wrapper, so CLI and API run the exact same
>    checks.
> 
> 4. Add a remote_migrate_vm_precondition endpoint that runs the checks
>    without starting a migration and returns the full findings list. It
>    reuses the same helper as the migrate endpoint, so the precheck
>    cannot drift from what is actually enforced.
> 
> The precondition results are returned as a flat list of findings. Does
> this shape look reasonable, and could a similar structure be reused for
> the intra-cluster migration precondition endpoint?

For the intra-cluster endpoint we also collect information for each
node, not just for a single target.

> 
> Open questions (RFC)
> ====================
> 
> Scope of the checks
> - The checks should stay semi-static, i.e. mostly config-derived, and
>   run quickly, i.e. avoid slow probes such as querying storage
>   contents. But we still want to query the remote, since a precheck
>   that never talks to the target is not very useful. Where/how can we
>   reasonably reduce the number of API round trips to the remote?

I think it can be done as part of the tunnel, see below. But this can be
done after (or on top of your current series).

> 
> Where should the checks run?
> - The source node currently runs the checks with a few local and fast
>   remote API calls. Since we want migrations between major versions to
>   work in a mostly forward- and backward-compatible way (older source
>   to newer target being the typical case), it might be cleaner to run
>   the checks on the remote by sending the relevant local config over.
>   When the remote is newer, it can also run more or patched checks.
>   Of course this would be a larger change, as the general structure
>   currently goes from source to target.

Yes, I would prefer the source passing the relevant info and have all
checks that can be done by the target to be done by the target.

> - Collecting results from both sides in PDM and comparing there seems
>   wrong: the checks still have to run locally for the real migration,
>   and both paths would then need to agree. We should rather keep a
>   single authority.

Yes, PDM is not a prerequisite for remote migration, so it should not be
dependent on PDM. The source should do its own checks and call into the
endpoint for the target side.

> - Running the checks over the Tunnel command instead would add a new
>   command that older endpoints do not understand. Both here and for the
>   API, a capability/feature endpoint could help: source and/or target
>   query what each supports for a migration, and older versions fall
>   back to the current checks. Is that worth introducing now?

The capability negotiation can be done via a new tunnel command, see:
https://lore.proxmox.com/pve-devel/b072f9d7-b0f8-41fc-8c37-7ea90798aa1b@proxmox.com/

It should be possible to issue the capabilities command to the target
and detect if the target side understands it (look at the error message
to see if unknown or failed for other reasons) and otherwise it's too
old and has no capabilities at all.

> 
> Two-phase refactor
> - The cleaner fix for "checks only run mid-migration" is to extract all
>   preconditions and split execution into two phases. Phase 1 is shared
>   by the precondition endpoint and the migration: it runs all the
>   reasonable checks and builds a plan hash of the queried info. Phase 2
>   performs the actual migration, ideally with its remaining checks up
>   front and ordered fast-to-slow. This is a large refactor of a
>   critical path, as the cluster migration code is mostly shared with
>   the remote migration. In addition, the cluster migration precondition
>   endpoint can also be refactored, which is an open TODO. Is that the
>   direction we want?

In the long term checking as much as possible up-front would be best,
yes. But some checks are a bit more involved to do up-front (e.g.
storage migration format negotiation also needs a brush-up). For the
current series, it's better to focus on things that are sensible/easier
to do. The rest can follow, once we have the mechanism in place. I don't
think we should drop existing checks from the migration path, or even
can, because they are still relevant for a source/target without the
precondition capability.

> 
> Containers
> - A similar series could later cover remote container migration. What
>   pitfalls should I watch out for there?

I don't think there's a fundamental difference with the design. In fact,
it should be simpler, because there is no live migration, so there are
fewer scenarios.

> 
> References
> ==========
> [0] https://forum.proxmox.com/threads/pdm-cross-cluster-migration-prerequisites-checklist-i-wish-i-had-before-my-first-attempt.184485
> 
> Thanks in advance for the feedback!
> 
> 
> Erik Fastermann (4):
>   remote migrate: drop ineffective fingerprint auto-detection
>   remote migrate: collect preconditions as structured findings
>   qm: remote-migrate: call API endpoint directly
>   remote migrate: add precondition check endpoint
> 
>  src/PVE/API2/Qemu.pm | 472 +++++++++++++++++++++++++++++--------------
>  src/PVE/CLI/qm.pm    | 128 +-----------
>  2 files changed, 323 insertions(+), 277 deletions(-)
>