[PATCH-SERIES common/cluster/ha-manager/qemu-server/container v3 00/21] migration: strict config check for intra-cluster migration

[Fiona Ebner <f.ebner@proxmox.com>]

Changes in v3:
* Add patches to teach HA to pass along certain migration parameters
  and fix #7053 by passing along 'with-conntrack-state'.
* Avoid re-using root-only 'force' flag, add a new 'skip-config-check'
  flag that can also be used by regular users.
* Use pvecfg node version to detect if a target supports config
  checking via mtunnel.

Changes in v2:
* add patches for container
* log when skipping strict configuration check

First part
==========

Additional migration options can now be passed from the guest API
endpoints to the HA stack. From the HA resource migrate API endpoint,
the options are passed all the way to the LRM and resource plugins,
which will then use it again when invoking the resource's migrate API
endpoint itself.

The migration API endpoints for containers and VMs use different
parameters. For example, the endpoint for containers does not have
the 'with-conntrack-state' option. To keep the schema for the more
abstract endpoint for HA resources clean, separate by type using a
oneOf schema and let the resource plugins declare additional
parameters via a migrate_json_properties() function.

Known issue: because of cyclic dependencies, the PVE::API::{LXC,QEMU}
modules can't be included by the respective resource plugin, when the
build happens in a PVE_GENERATING_DOCS environment. Because of this,
the extra migration options for the HA endpoint won't show up in the
docs.

To pass these along to CRM, support for JSON CRM commands is
introduced, with only the migration command being supported for now.
If not all nodes are recent enough to support the new-style JSON CRM
command, produce a warning and fall-back to the previous behavior
ignoring the additional migration options.

Initially, this is used to fix #7053 by passing along the
'with-conntrack-state' migration option.

Second part
===========

For remote migration, we already check that the config can be parsed
on the target. Do the same for intra-cluster migration, to avoid
issues like [0] for future new settings, with settings being
unexpectedly ignored on the target, and in the case of qemu-server
even relatively silently dropped (there are warnings in the target's
system logs). In case of containers, an example is migrating a
container with a mountpoint with 'keepattrs' to a node with a too old
pve-container version, resulting in the mountpoint not being mounted
on the target.

Unfortunately, before patch "qm: mtunnel: reply when a command is
unknown", when a command is unknown, mtunnel did not reply at all.
Therefore, check the target node's version to see if it is new enough
to support the new mtunnel command.

Still missing
=============

UI integration for the new flag
HA manager tests with the new JSON commands

Useful to pull in early
=======================

pvecfg node version helpers moved to pve-cluster rather than just
pve-guest-common, because David needs them in pve-cluster too [1].

[0]: https://bugzilla.proxmox.com/show_bug.cgi?id=7341
[1]: https://lore.proxmox.com/pve-devel/20260724142528.109453-1-d.riley@proxmox.com/

Patch 15/21 qm: mtunnel: reply when a command is unknown would also
be nice to have.

Dependencies
============

Dependency bump qemu-server,pve-container -> common,pve-cluster needed!

qemu-server also build-depends on new common and pve-cluster.

pve-manager needs to depend on new ha-manager so the node version
checking makes sense.

Dependency bump pve-container -> guest-common needed!

common:

Fiona Ebner (2):
  tools: move version_cmp() helper from qemu-server
  rest handler: handle: respect schema's 'type-property' when resolving
    type

 src/PVE/JSONSchema.pm    |  10 +++
 src/PVE/RESTHandler.pm   |  21 ++++--
 src/PVE/Tools.pm         |  26 +++++++
 test/get-options-test.pl | 150 +++++++++++++++++++++++++++++++++++++++
 4 files changed, 200 insertions(+), 7 deletions(-)


cluster:

Fiona Ebner (1):
  cluster: move pvecfg node version helpers from qemu-server

 src/PVE/Cluster.pm | 30 ++++++++++++++++++++++++++++++
 1 file changed, 30 insertions(+)


ha-manager:

Fiona Ebner (7):
  next state {stopped,started}: factor out helper to handle motion
    command
  lrm: resource migration: support extra migration options
  change service state: support hash as a parameter value
  next state: handle motion command: support migration options
  queue resource motion: support migration options
  crm command: support JSON-style migrate command
  api: resources: migration: support additional migration options

 src/PVE/API2/HA/Resources.pm                  |  99 ++++++++++--
 src/PVE/HA/LRM.pm                             |  15 +-
 src/PVE/HA/Manager.pm                         | 152 ++++++++++++------
 src/PVE/HA/Resources.pm                       |   2 +-
 src/PVE/HA/Resources/PVECT.pm                 |  15 +-
 src/PVE/HA/Resources/PVEVM.pm                 |  15 +-
 src/PVE/HA/Sim/Resources.pm                   |   2 +-
 src/PVE/HA/Sim/Resources/VirtCT.pm            |   2 +-
 src/PVE/HA/Sim/Resources/VirtFail.pm          |   2 +-
 .../test-relocate-to-inactive-node/log.expect |   1 +
 src/test/test-service-stopped3/log.expect     |   1 +
 11 files changed, 229 insertions(+), 77 deletions(-)


qemu-server:

Fiona Ebner (7):
  helpers: move version_cmp() helper to pve-common
  helpers: move pvecfg node version helpers to pve-cluster
  api: migrate: allow forwarding certain migration properties to HA
  fix #7053: api: migrate: pass 'with-conntrack-state' flag to HA
    migration
  qm: mtunnel: reply when a command is unknown
  qm: mtunnel: add 'conf' command to do strict configuration parsing
  migration: intra-cluster: check config can be parsed on target
    node

 src/PVE/API2/Qemu.pm                      | 158 +++++++++++++---------
 src/PVE/CLI/qm.pm                         |  15 ++
 src/PVE/QemuMigrate.pm                    |  31 ++++-
 src/PVE/QemuServer/Helpers.pm             |  51 +------
 src/PVE/QemuServer/Machine.pm             |  13 +-
 src/PVE/QemuServer/QMPHelpers.pm          |   5 +-
 src/test/MigrationTest/QemuMigrateMock.pm |  11 +-
 7 files changed, 154 insertions(+), 130 deletions(-)


container:

Fiona Ebner (4):
  api: migrate: allow forwarding certain migration properties to HA
  pct: introduce mtunnel command
  d/control: bump versioned build dependency for libpve-common-perl to
    9.0.12
  migration: intra-cluster: check config can be parsed on target node

 debian/control         |   2 +-
 src/PVE/API2/LXC.pm    | 105 ++++++++++++++++++++++++++---------------
 src/PVE/CLI/pct.pm     |  57 ++++++++++++++++++++++
 src/PVE/LXC/Migrate.pm |  30 ++++++++++++
 4 files changed, 156 insertions(+), 38 deletions(-)


Summary over all repositories:
  27 files changed, 769 insertions(+), 252 deletions(-)

-- 
Generated by git-murpp 0.5.0