From: Dominik Csapak <d.csapak@proxmox.com>
To: Thomas Lamprecht <t.lamprecht@proxmox.com>, pve-devel@lists.proxmox.com
Subject: Re: [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID
Date: Wed, 18 Nov 2026 14:49:27 +0100 [thread overview]
Message-ID: <5c3aa7d2-aea2-462a-80a8-bfcf95c18247@proxmox.com> (raw)
In-Reply-To: <20261118043817.1659212-1-t.lamprecht@proxmox.com>
Read through it again (though mostly via diff from the v1)
and it looks good to me now:
* the rationale is given
* the flow is better visible (and with better wording)
* remote migrate is now blocked by default on conflicting uids
the biggest issue to discuss maybe still is if we want to allow
restore with a new uid in case the same uid exists already,
same argument as for the remote migration:
if the same vm already exists, restoring it again can have
bad implications like duplicate ips, etc.
but IMHO it's not a blocker, maybe even something the admin
would want to configure globally? (datacenter.cfg?)
one thing i noticed during testing was if i modified a guest
to have an invalid uid, on the next config write, the
ctime and creation-qemu vanished from the config and
only a new rtime and uid remained.
(though i had to manually edit the config, so not a problem IMO)
there are still some nits here and there, but they are mostly
taste questions and nothing that can't be improved further down the line
for example: when restoring a vm over another another, the failure i get is:
TASK ERROR: command 'set -o pipefail && zstd -q -d -c --
/var/lib/vz/dump/vzdump-qemu-9999-2026_11_18-14_37_58.vma.zst | vma
extract -v -r /var/tmp/vzdumptmp8272.fifo - /var/tmp/vzdumptmp8272'
failed: backup belongs to another guest: its uid '2xuof4nw6rnowo5mpgs2'
differs from uid 'itomggi4ofm33szeb3h2' of the existing VM 102 - use
'allow-identity-mismatch' to overwrite the VM anyway
which is not very readable in the task log window (still functional though)
also at least a few things in the ui i'm missing:
a way to show the cluster/node id (probably in the datacenter summary)
using the uid (maybe rtime) as variables in the templating for the notes
Tested it on a test cluster with some backup/restore cycles,
adopting of uid, manually editing configs, etc.
Since nothing of these blocks this series from my POV, consider it:
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
On 11/18/26 5:39 AM, Thomas Lamprecht wrote:
> A VMID can be reused for another guest, and a guest can get another
> VMID or move to another cluster through a restore or a remote
> migration. So backup software and other tools cannot tell whether they
> still look at the same guest, see [0].
>
> This series gives every VM and container a guest identity in the
> read-only 'meta' property: a random uid (100 bits as 20 base32
> characters) and the time the guest was registered in the cluster. Create
> and clone get a new uid. Config changes, migration, rollback, backup and
> restore keep it, also under another VMID or in another cluster, as long
> as no other guest there has it. Each cluster or standalone node also
> gets a cluster ID in the same format, generated and protected by pmxcfs.
> The docs patches describe the details.
>
> The uid is a plain random nonce: 100 bits make a duplicate practically
> impossible and fit into a 128-bit value with room to spare, and 20
> base32 characters stay short and need no quoting. The registration time
> is separate from the creation time, which clones, restores and remote
> migrations copy along. It changes when a guest gets registered again, so
> tools can tell such a guest from the one they knew.
>
> Three behavior changes come with it:
>
> * Restoring over an existing guest whose uid differs from the one of
> the backup is refused unless 'allow-identity-mismatch' is set. 'force'
> alone does not override this.
> * A migration to another cluster is refused if another guest there
> already has the uid, unless 'allow-new-uid' is set. The check runs
> before anything is transferred. PDM does not know the option yet.
> * Older versions drop the whole 'meta' property of a guest that has an
> identity. So migration and clone to such nodes is refused, and remote
> migration bumps the tunnel version. This also hits moves by the HA
> stack, which stops a container or a relocated VM before the migration
> gets refused. A target check in the LRM that avoids this will follow
> separately.
>
> The patches are in dependency order: guest-common needs the pve-cluster
> of this series, qemu-server and container need guest-common, and manager
> needs all of them. manager also has to depend on the qemu-server and
> container versions with this series, as the migration guard takes the
> pve-manager version of a node as proof that the node keeps the identity.
> MIN_PVE_MANAGER_VERSION in guest-common is a stand-in for that version.
>
> Tested with the affected tests of the repositories for every patch and
> their full suites at the end, pmxcfs with corosync in network
> namespaces, and end to end on a nested three-node cluster that was
> upgraded node by node, with remote migrations to a standalone node.
>
> changes v2 -> v3:
> * remote migration: a guest without uid gets one before the source
> sends it, so source and target no longer end up with unrelated uids;
> the identity command moved to pve-guest-common for both guest types
> (new patch 13)
> * restore: keep the guest if a restored volume or firewall config
> cannot be removed, fail if a container backup changes between its
> check and its extraction, and let the cleanup follow the reservation
> * reserving a VMID refuses an existing config that fails to load
> instead of replacing it
> * cluster ID: generate it right away when the configured nodes change
> (new patch 5)
> * feature API without version lookups for snapshot requests, and the
> cluster-wide bulk migration refuses its target up front
> * docs: failed restores, uid conflicts on migration, qm.conf(5) and
> pct.conf(5)
> * more independent fixes that the series builds on were applied on
> their own: failed and concurrent restores and creates, the schema of
> the migration preconditions, the lock of a VMA restore, a node list
> cut short in pmxcfs
>
> changes v1 -> v2, thanks to Dominik for the review:
> * rebase onto current master (tracking of used guest IDs, dfsm rework)
> * remote migration takes the uid over at its start and refuses a uid
> that is in use in the target cluster, unless 'allow-new-uid' is set
> * qemu-server and container share the restore checks and messages
> through pve-guest-common, with fewer special cases
> * IDs take each character from one random byte, without an encoder
> * the clone dialog does not offer nodes that would drop the identity,
> and a bulk migration checks its target once (patch 34)
> * independent fixes moved out of the series and got applied on their
> own: the IPC client error, the order of removing firewall and guest
> config, and the cleanup of a remote migration target after failures
> * comments, commit messages and docs give the reasons for the format,
> for rtime next to ctime and for the separate lock
> Details are in the notes below the "---" of each patch.
>
> v2: https://lore.proxmox.com/pve-devel/20261117223945.862914-1-t.lamprecht@proxmox.com/
> v1: https://lore.proxmox.com/pve-devel/20261116004703.2908294-1-t.lamprecht@proxmox.com/
>
> [0]: https://bugzilla.proxmox.com/show_bug.cgi?id=7846
>
> cluster:
>
> Thomas Lamprecht (8):
> pmxcfs: reject changes to the cluster ID file from user space
> pmxcfs: generate a cluster ID on standalone nodes
> pmxcfs: dfsm: add a callback that runs once all members are in sync
> pmxcfs: quorum: pass the corosync membership to an optional listener
> pmxcfs: confdb: call an optional listener when the nodes get published
> pmxcfs: dcdb: generate the cluster ID once the cluster is in sync
> cluster: add helper to read the cluster ID
> pmxcfs: allow removing the cluster ID file in forced local mode
>
> src/PVE/Cluster.pm | 16 +
> src/pmxcfs/Makefile | 12 +-
> src/pmxcfs/cfs-plug-memdb.c | 46 +++
> src/pmxcfs/cfs-utils.c | 8 +
> src/pmxcfs/cfs-utils.h | 5 +
> src/pmxcfs/check_dcdb.c | 563 ++++++++++++++++++++++++++++++++++++
> src/pmxcfs/check_memdb.c | 283 ++++++++++++++++++
> src/pmxcfs/cluster-id.c | 142 +++++++++
> src/pmxcfs/cluster-id.h | 50 ++++
> src/pmxcfs/confdb.c | 18 +-
> src/pmxcfs/confdb.h | 5 +-
> src/pmxcfs/dcdb.c | 153 ++++++++++
> src/pmxcfs/dcdb.h | 8 +
> src/pmxcfs/dfsm.c | 59 ++++
> src/pmxcfs/dfsm.h | 14 +
> src/pmxcfs/pmxcfs.c | 12 +-
> src/pmxcfs/quorum.c | 26 +-
> src/pmxcfs/quorum.h | 9 +-
> src/pmxcfs/status.c | 12 +
> src/pmxcfs/status.h | 3 +
> src/test/Makefile | 6 +-
> src/test/test_cluster_id.pl | 46 +++
> 22 files changed, 1478 insertions(+), 18 deletions(-)
> create mode 100644 src/pmxcfs/check_dcdb.c
> create mode 100644 src/pmxcfs/cluster-id.c
> create mode 100644 src/pmxcfs/cluster-id.h
> create mode 100644 src/test/test_cluster_id.pl
>
>
> guest-common:
>
> Thomas Lamprecht (5):
> guest identity: add helpers for the guest uid and registration time
> abstract config: assign a guest identity and keep it on config writes
> abstract config: let the VMID reservation return the previous state
> guest identity: allow taking over a uid that is unused in the cluster
> migrate: add shared helpers for the identity command of the migration tunnel
>
> src/Makefile | 1 +
> src/PVE/AbstractConfig.pm | 202 +++-
> src/PVE/AbstractMigrate.pm | 30 +
> src/PVE/GuestIdentity.pm | 510 ++++++++++
> src/tests/Makefile | 10 +-
> src/tests/config-identity-tests.pl | 1391 ++++++++++++++++++++++++++++
> src/tests/guest-identity-tests.pl | 530 +++++++++++
> 7 files changed, 2668 insertions(+), 6 deletions(-)
> create mode 100644 src/PVE/GuestIdentity.pm
> create mode 100755 src/tests/config-identity-tests.pl
> create mode 100755 src/tests/guest-identity-tests.pl
>
>
> qemu-server:
>
> Thomas Lamprecht (7):
> api: restore: release the VMID if a restore to an unused VMID fails
> api: migrate, clone: refuse targets that would drop the guest identity
> fix #7846: config: add a guest identity to the 'meta' property
> restore: take over the guest uid from the backup where possible
> api: remote migration: take over the guest uid from the source
> api: status: return the guest uid
> tests: remote migration: check a failed command and an early tunnel end
>
> src/PVE/API2/Qemu.pm | 118 +-
> src/PVE/CLI/qm.pm | 7 +
> src/PVE/CLI/qmrestore.pm | 11 +-
> src/PVE/QemuConfig.pm | 7 +
> src/PVE/QemuMigrate.pm | 18 +-
> src/PVE/QemuServer.pm | 155 ++-
> src/PVE/QemuServer/MetaInfo.pm | 2 +
> src/test/Makefile | 5 +-
> src/test/QemuServerTest/FakeCFS.pm | 189 +++
> .../q35-windows-pinning-identity-only.conf | 6 +
> ...q35-windows-pinning-identity-only.conf.cmd | 26 +
> .../meta-identity-invalid.conf | 6 +
> .../meta-identity-invalid.conf.strict.error | 2 +
> .../meta-identity-invalid.conf | 7 +
> .../parse-config-input/meta-identity.conf | 20 +
> src/test/restore-config-expected/150.conf | 9 +
> src/test/restore-config-input/150.conf | 10 +
> src/test/run_api_tests.pl | 206 ++++
> src/test/run_guest_identity_tests.pl | 803 +++++++++++++
> src/test/run_parse_config_tests.pl | 2 +-
> src/test/run_qemu_restore_config_tests.pl | 1053 ++++++++++++++++-
> .../create/qemu-server/401.conf | 31 +
> .../create/qemu-server/402.conf | 31 +
> .../rollback/qemu-server/401.conf | 36 +
> .../rollback/qemu-server/402.conf | 35 +
> .../create/qemu-server/401.conf | 13 +
> .../create/qemu-server/402.conf | 13 +
> .../rollback/qemu-server/401.conf | 35 +
> .../rollback/qemu-server/402.conf | 34 +
> src/test/snapshot-test.pm | 82 +-
> 30 files changed, 2899 insertions(+), 73 deletions(-)
> create mode 100644 src/test/QemuServerTest/FakeCFS.pm
> create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf
> create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf.cmd
> create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf
> create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf.strict.error
> create mode 100644 src/test/parse-config-input/meta-identity-invalid.conf
> create mode 100644 src/test/parse-config-input/meta-identity.conf
> create mode 100644 src/test/restore-config-expected/150.conf
> create mode 100644 src/test/restore-config-input/150.conf
> create mode 100755 src/test/run_guest_identity_tests.pl
> create mode 100644 src/test/snapshot-expected/create/qemu-server/401.conf
> create mode 100644 src/test/snapshot-expected/create/qemu-server/402.conf
> create mode 100644 src/test/snapshot-expected/rollback/qemu-server/401.conf
> create mode 100644 src/test/snapshot-expected/rollback/qemu-server/402.conf
> create mode 100644 src/test/snapshot-input/create/qemu-server/401.conf
> create mode 100644 src/test/snapshot-input/create/qemu-server/402.conf
> create mode 100644 src/test/snapshot-input/rollback/qemu-server/401.conf
> create mode 100644 src/test/snapshot-input/rollback/qemu-server/402.conf
>
>
> container:
>
> Thomas Lamprecht (8):
> config: add read-only meta property with the creation time
> fix #7846: config: add a guest identity to the 'meta' property
> api: migrate, clone: refuse targets that would drop the guest identity
> restore: take over the guest uid from the backup where possible
> restore: refuse overwriting a container with another guest's backup
> test: remote migration: check the cleanup of the target after failures
> api: remote migration: take over the guest uid from the source
> api: status: return the guest uid
>
> src/PVE/API2/LXC.pm | 190 ++-
> src/PVE/API2/LXC/Config.pm | 29 +-
> src/PVE/API2/LXC/Status.pm | 6 +
> src/PVE/CLI/pct.pm | 7 +
> src/PVE/LXC/Config.pm | 51 +-
> src/PVE/LXC/Migrate.pm | 8 +-
> src/test/Makefile | 8 +-
> src/test/config_tests.pl | 23 +
> src/test/lifecycle_tests.pl | 2446 +++++++++++++++++++++++++++++++++++
> 9 files changed, 2730 insertions(+), 38 deletions(-)
> create mode 100755 src/test/lifecycle_tests.pl
>
>
> manager:
>
> Thomas Lamprecht (6):
> api: cluster resources: add guest uid and registration time
> api: cluster status: add cluster ID
> ui: restore: allow overwriting a guest with a backup of another guest
> ui: migrate: refuse target nodes that would drop the guest identity
> ui: guest summary: show the guest identity
> api: bulk migrate: refuse targets that would drop the guest identity
>
> PVE/API2/Cluster.pm | 24 ++-
> PVE/API2/Cluster/BulkAction/Guest.pm | 5 +
> PVE/API2/Nodes.pm | 3 +
> test/ClusterResources_test.pl | 249 +++++++++++++++++++++++
> test/ClusterStatus_test.pl | 114 +++++++++++
> test/Makefile | 10 +-
> www/manager6/Makefile | 1 +
> www/manager6/panel/GuestIdentity.js | 142 +++++++++++++
> www/manager6/panel/GuestStatusView.js | 3 +
> www/manager6/panel/TemplateStatusView.js | 3 +
> www/manager6/window/Migrate.js | 29 +++
> www/manager6/window/Restore.js | 18 ++
> 12 files changed, 599 insertions(+), 2 deletions(-)
> create mode 100755 test/ClusterResources_test.pl
> create mode 100755 test/ClusterStatus_test.pl
> create mode 100644 www/manager6/panel/GuestIdentity.js
>
>
> docs:
>
> Thomas Lamprecht (4):
> vzdump: describe the guest identity of VMs and containers
> pvecm: document the cluster ID and how to get a new one
> qm: configuration: describe the read-only meta property
> pct: configuration: describe the read-only meta property
>
> pct.adoc | 35 ++++++
> pct.conf.adoc | 3 +
> pmxcfs.adoc | 1 +
> pvecm.adoc | 87 +++++++++++++
> qm.adoc | 38 ++++++
> qm.conf.adoc | 3 +
> vzdump.adoc | 335 ++++++++++++++++++++++++++++++++++++++++++++++++++
> 7 files changed, 502 insertions(+)
>
>
> Summary over all repositories:
> 87 files changed, 10876 insertions(+), 137 deletions(-)
>
next prev parent reply other threads:[~2026-11-18 13:49 UTC|newest]
Thread overview: 42+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 01/38] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 02/38] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 03/38] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 04/38] pmxcfs: quorum: pass the corosync membership to an optional listener Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 05/38] pmxcfs: confdb: call an optional listener when the nodes get published Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 06/38] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 07/38] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 08/38] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 09/38] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 10/38] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 11/38] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 12/38] guest identity: allow taking over a uid that is unused in the cluster Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 13/38] migrate: add shared helpers for the identity command of the migration tunnel Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 14/38] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 15/38] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 16/38] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 17/38] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 18/38] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 19/38] api: status: return the guest uid Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 20/38] tests: remote migration: check a failed command and an early tunnel end Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 21/38] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 22/38] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 23/38] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 24/38] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 25/38] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 26/38] test: remote migration: check the cleanup of the target after failures Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 27/38] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 28/38] api: status: return the guest uid Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 29/38] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 30/38] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 31/38] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 32/38] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 33/38] ui: guest summary: show " Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 34/38] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 35/38] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 36/38] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 37/38] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 38/38] pct: " Thomas Lamprecht
2026-11-18 13:49 ` Dominik Csapak [this message]
2026-11-18 14:13 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-19 10:38 ` superseded: " Thomas Lamprecht
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=5c3aa7d2-aea2-462a-80a8-bfcf95c18247@proxmox.com \
--to=d.csapak@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
--cc=t.lamprecht@proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox