From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 50FDC1FF0C0 for ; Wed, 18 Nov 2026 14:49:39 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 5E8C8216C9; Wed, 18 Nov 2026 14:49:36 +0100 (CET) Message-ID: <5c3aa7d2-aea2-462a-80a8-bfcf95c18247@proxmox.com> Date: Wed, 18 Nov 2026 14:49:27 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta Subject: Re: [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID To: Thomas Lamprecht , pve-devel@lists.proxmox.com References: <20261118043817.1659212-1-t.lamprecht@proxmox.com> Content-Language: en-US From: Dominik Csapak In-Reply-To: <20261118043817.1659212-1-t.lamprecht@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1795009767411 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.285 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: CNY7ASJBC3GXEGQIP7IWJCSXHYEHJAY3 X-Message-ID-Hash: CNY7ASJBC3GXEGQIP7IWJCSXHYEHJAY3 X-MailFrom: d.csapak@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: Read through it again (though mostly via diff from the v1) and it looks good to me now: * the rationale is given * the flow is better visible (and with better wording) * remote migrate is now blocked by default on conflicting uids the biggest issue to discuss maybe still is if we want to allow restore with a new uid in case the same uid exists already, same argument as for the remote migration: if the same vm already exists, restoring it again can have bad implications like duplicate ips, etc. but IMHO it's not a blocker, maybe even something the admin would want to configure globally? (datacenter.cfg?) one thing i noticed during testing was if i modified a guest to have an invalid uid, on the next config write, the ctime and creation-qemu vanished from the config and only a new rtime and uid remained. (though i had to manually edit the config, so not a problem IMO) there are still some nits here and there, but they are mostly taste questions and nothing that can't be improved further down the line for example: when restoring a vm over another another, the failure i get is: TASK ERROR: command 'set -o pipefail && zstd -q -d -c -- /var/lib/vz/dump/vzdump-qemu-9999-2026_11_18-14_37_58.vma.zst | vma extract -v -r /var/tmp/vzdumptmp8272.fifo - /var/tmp/vzdumptmp8272' failed: backup belongs to another guest: its uid '2xuof4nw6rnowo5mpgs2' differs from uid 'itomggi4ofm33szeb3h2' of the existing VM 102 - use 'allow-identity-mismatch' to overwrite the VM anyway which is not very readable in the task log window (still functional though) also at least a few things in the ui i'm missing: a way to show the cluster/node id (probably in the datacenter summary) using the uid (maybe rtime) as variables in the templating for the notes Tested it on a test cluster with some backup/restore cycles, adopting of uid, manually editing configs, etc. Since nothing of these blocks this series from my POV, consider it: Reviewed-by: Dominik Csapak Tested-by: Dominik Csapak On 11/18/26 5:39 AM, Thomas Lamprecht wrote: > A VMID can be reused for another guest, and a guest can get another > VMID or move to another cluster through a restore or a remote > migration. So backup software and other tools cannot tell whether they > still look at the same guest, see [0]. > > This series gives every VM and container a guest identity in the > read-only 'meta' property: a random uid (100 bits as 20 base32 > characters) and the time the guest was registered in the cluster. Create > and clone get a new uid. Config changes, migration, rollback, backup and > restore keep it, also under another VMID or in another cluster, as long > as no other guest there has it. Each cluster or standalone node also > gets a cluster ID in the same format, generated and protected by pmxcfs. > The docs patches describe the details. > > The uid is a plain random nonce: 100 bits make a duplicate practically > impossible and fit into a 128-bit value with room to spare, and 20 > base32 characters stay short and need no quoting. The registration time > is separate from the creation time, which clones, restores and remote > migrations copy along. It changes when a guest gets registered again, so > tools can tell such a guest from the one they knew. > > Three behavior changes come with it: > > * Restoring over an existing guest whose uid differs from the one of > the backup is refused unless 'allow-identity-mismatch' is set. 'force' > alone does not override this. > * A migration to another cluster is refused if another guest there > already has the uid, unless 'allow-new-uid' is set. The check runs > before anything is transferred. PDM does not know the option yet. > * Older versions drop the whole 'meta' property of a guest that has an > identity. So migration and clone to such nodes is refused, and remote > migration bumps the tunnel version. This also hits moves by the HA > stack, which stops a container or a relocated VM before the migration > gets refused. A target check in the LRM that avoids this will follow > separately. > > The patches are in dependency order: guest-common needs the pve-cluster > of this series, qemu-server and container need guest-common, and manager > needs all of them. manager also has to depend on the qemu-server and > container versions with this series, as the migration guard takes the > pve-manager version of a node as proof that the node keeps the identity. > MIN_PVE_MANAGER_VERSION in guest-common is a stand-in for that version. > > Tested with the affected tests of the repositories for every patch and > their full suites at the end, pmxcfs with corosync in network > namespaces, and end to end on a nested three-node cluster that was > upgraded node by node, with remote migrations to a standalone node. > > changes v2 -> v3: > * remote migration: a guest without uid gets one before the source > sends it, so source and target no longer end up with unrelated uids; > the identity command moved to pve-guest-common for both guest types > (new patch 13) > * restore: keep the guest if a restored volume or firewall config > cannot be removed, fail if a container backup changes between its > check and its extraction, and let the cleanup follow the reservation > * reserving a VMID refuses an existing config that fails to load > instead of replacing it > * cluster ID: generate it right away when the configured nodes change > (new patch 5) > * feature API without version lookups for snapshot requests, and the > cluster-wide bulk migration refuses its target up front > * docs: failed restores, uid conflicts on migration, qm.conf(5) and > pct.conf(5) > * more independent fixes that the series builds on were applied on > their own: failed and concurrent restores and creates, the schema of > the migration preconditions, the lock of a VMA restore, a node list > cut short in pmxcfs > > changes v1 -> v2, thanks to Dominik for the review: > * rebase onto current master (tracking of used guest IDs, dfsm rework) > * remote migration takes the uid over at its start and refuses a uid > that is in use in the target cluster, unless 'allow-new-uid' is set > * qemu-server and container share the restore checks and messages > through pve-guest-common, with fewer special cases > * IDs take each character from one random byte, without an encoder > * the clone dialog does not offer nodes that would drop the identity, > and a bulk migration checks its target once (patch 34) > * independent fixes moved out of the series and got applied on their > own: the IPC client error, the order of removing firewall and guest > config, and the cleanup of a remote migration target after failures > * comments, commit messages and docs give the reasons for the format, > for rtime next to ctime and for the separate lock > Details are in the notes below the "---" of each patch. > > v2: https://lore.proxmox.com/pve-devel/20261117223945.862914-1-t.lamprecht@proxmox.com/ > v1: https://lore.proxmox.com/pve-devel/20261116004703.2908294-1-t.lamprecht@proxmox.com/ > > [0]: https://bugzilla.proxmox.com/show_bug.cgi?id=7846 > > cluster: > > Thomas Lamprecht (8): > pmxcfs: reject changes to the cluster ID file from user space > pmxcfs: generate a cluster ID on standalone nodes > pmxcfs: dfsm: add a callback that runs once all members are in sync > pmxcfs: quorum: pass the corosync membership to an optional listener > pmxcfs: confdb: call an optional listener when the nodes get published > pmxcfs: dcdb: generate the cluster ID once the cluster is in sync > cluster: add helper to read the cluster ID > pmxcfs: allow removing the cluster ID file in forced local mode > > src/PVE/Cluster.pm | 16 + > src/pmxcfs/Makefile | 12 +- > src/pmxcfs/cfs-plug-memdb.c | 46 +++ > src/pmxcfs/cfs-utils.c | 8 + > src/pmxcfs/cfs-utils.h | 5 + > src/pmxcfs/check_dcdb.c | 563 ++++++++++++++++++++++++++++++++++++ > src/pmxcfs/check_memdb.c | 283 ++++++++++++++++++ > src/pmxcfs/cluster-id.c | 142 +++++++++ > src/pmxcfs/cluster-id.h | 50 ++++ > src/pmxcfs/confdb.c | 18 +- > src/pmxcfs/confdb.h | 5 +- > src/pmxcfs/dcdb.c | 153 ++++++++++ > src/pmxcfs/dcdb.h | 8 + > src/pmxcfs/dfsm.c | 59 ++++ > src/pmxcfs/dfsm.h | 14 + > src/pmxcfs/pmxcfs.c | 12 +- > src/pmxcfs/quorum.c | 26 +- > src/pmxcfs/quorum.h | 9 +- > src/pmxcfs/status.c | 12 + > src/pmxcfs/status.h | 3 + > src/test/Makefile | 6 +- > src/test/test_cluster_id.pl | 46 +++ > 22 files changed, 1478 insertions(+), 18 deletions(-) > create mode 100644 src/pmxcfs/check_dcdb.c > create mode 100644 src/pmxcfs/cluster-id.c > create mode 100644 src/pmxcfs/cluster-id.h > create mode 100644 src/test/test_cluster_id.pl > > > guest-common: > > Thomas Lamprecht (5): > guest identity: add helpers for the guest uid and registration time > abstract config: assign a guest identity and keep it on config writes > abstract config: let the VMID reservation return the previous state > guest identity: allow taking over a uid that is unused in the cluster > migrate: add shared helpers for the identity command of the migration tunnel > > src/Makefile | 1 + > src/PVE/AbstractConfig.pm | 202 +++- > src/PVE/AbstractMigrate.pm | 30 + > src/PVE/GuestIdentity.pm | 510 ++++++++++ > src/tests/Makefile | 10 +- > src/tests/config-identity-tests.pl | 1391 ++++++++++++++++++++++++++++ > src/tests/guest-identity-tests.pl | 530 +++++++++++ > 7 files changed, 2668 insertions(+), 6 deletions(-) > create mode 100644 src/PVE/GuestIdentity.pm > create mode 100755 src/tests/config-identity-tests.pl > create mode 100755 src/tests/guest-identity-tests.pl > > > qemu-server: > > Thomas Lamprecht (7): > api: restore: release the VMID if a restore to an unused VMID fails > api: migrate, clone: refuse targets that would drop the guest identity > fix #7846: config: add a guest identity to the 'meta' property > restore: take over the guest uid from the backup where possible > api: remote migration: take over the guest uid from the source > api: status: return the guest uid > tests: remote migration: check a failed command and an early tunnel end > > src/PVE/API2/Qemu.pm | 118 +- > src/PVE/CLI/qm.pm | 7 + > src/PVE/CLI/qmrestore.pm | 11 +- > src/PVE/QemuConfig.pm | 7 + > src/PVE/QemuMigrate.pm | 18 +- > src/PVE/QemuServer.pm | 155 ++- > src/PVE/QemuServer/MetaInfo.pm | 2 + > src/test/Makefile | 5 +- > src/test/QemuServerTest/FakeCFS.pm | 189 +++ > .../q35-windows-pinning-identity-only.conf | 6 + > ...q35-windows-pinning-identity-only.conf.cmd | 26 + > .../meta-identity-invalid.conf | 6 + > .../meta-identity-invalid.conf.strict.error | 2 + > .../meta-identity-invalid.conf | 7 + > .../parse-config-input/meta-identity.conf | 20 + > src/test/restore-config-expected/150.conf | 9 + > src/test/restore-config-input/150.conf | 10 + > src/test/run_api_tests.pl | 206 ++++ > src/test/run_guest_identity_tests.pl | 803 +++++++++++++ > src/test/run_parse_config_tests.pl | 2 +- > src/test/run_qemu_restore_config_tests.pl | 1053 ++++++++++++++++- > .../create/qemu-server/401.conf | 31 + > .../create/qemu-server/402.conf | 31 + > .../rollback/qemu-server/401.conf | 36 + > .../rollback/qemu-server/402.conf | 35 + > .../create/qemu-server/401.conf | 13 + > .../create/qemu-server/402.conf | 13 + > .../rollback/qemu-server/401.conf | 35 + > .../rollback/qemu-server/402.conf | 34 + > src/test/snapshot-test.pm | 82 +- > 30 files changed, 2899 insertions(+), 73 deletions(-) > create mode 100644 src/test/QemuServerTest/FakeCFS.pm > create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf > create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf.cmd > create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf > create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf.strict.error > create mode 100644 src/test/parse-config-input/meta-identity-invalid.conf > create mode 100644 src/test/parse-config-input/meta-identity.conf > create mode 100644 src/test/restore-config-expected/150.conf > create mode 100644 src/test/restore-config-input/150.conf > create mode 100755 src/test/run_guest_identity_tests.pl > create mode 100644 src/test/snapshot-expected/create/qemu-server/401.conf > create mode 100644 src/test/snapshot-expected/create/qemu-server/402.conf > create mode 100644 src/test/snapshot-expected/rollback/qemu-server/401.conf > create mode 100644 src/test/snapshot-expected/rollback/qemu-server/402.conf > create mode 100644 src/test/snapshot-input/create/qemu-server/401.conf > create mode 100644 src/test/snapshot-input/create/qemu-server/402.conf > create mode 100644 src/test/snapshot-input/rollback/qemu-server/401.conf > create mode 100644 src/test/snapshot-input/rollback/qemu-server/402.conf > > > container: > > Thomas Lamprecht (8): > config: add read-only meta property with the creation time > fix #7846: config: add a guest identity to the 'meta' property > api: migrate, clone: refuse targets that would drop the guest identity > restore: take over the guest uid from the backup where possible > restore: refuse overwriting a container with another guest's backup > test: remote migration: check the cleanup of the target after failures > api: remote migration: take over the guest uid from the source > api: status: return the guest uid > > src/PVE/API2/LXC.pm | 190 ++- > src/PVE/API2/LXC/Config.pm | 29 +- > src/PVE/API2/LXC/Status.pm | 6 + > src/PVE/CLI/pct.pm | 7 + > src/PVE/LXC/Config.pm | 51 +- > src/PVE/LXC/Migrate.pm | 8 +- > src/test/Makefile | 8 +- > src/test/config_tests.pl | 23 + > src/test/lifecycle_tests.pl | 2446 +++++++++++++++++++++++++++++++++++ > 9 files changed, 2730 insertions(+), 38 deletions(-) > create mode 100755 src/test/lifecycle_tests.pl > > > manager: > > Thomas Lamprecht (6): > api: cluster resources: add guest uid and registration time > api: cluster status: add cluster ID > ui: restore: allow overwriting a guest with a backup of another guest > ui: migrate: refuse target nodes that would drop the guest identity > ui: guest summary: show the guest identity > api: bulk migrate: refuse targets that would drop the guest identity > > PVE/API2/Cluster.pm | 24 ++- > PVE/API2/Cluster/BulkAction/Guest.pm | 5 + > PVE/API2/Nodes.pm | 3 + > test/ClusterResources_test.pl | 249 +++++++++++++++++++++++ > test/ClusterStatus_test.pl | 114 +++++++++++ > test/Makefile | 10 +- > www/manager6/Makefile | 1 + > www/manager6/panel/GuestIdentity.js | 142 +++++++++++++ > www/manager6/panel/GuestStatusView.js | 3 + > www/manager6/panel/TemplateStatusView.js | 3 + > www/manager6/window/Migrate.js | 29 +++ > www/manager6/window/Restore.js | 18 ++ > 12 files changed, 599 insertions(+), 2 deletions(-) > create mode 100755 test/ClusterResources_test.pl > create mode 100755 test/ClusterStatus_test.pl > create mode 100644 www/manager6/panel/GuestIdentity.js > > > docs: > > Thomas Lamprecht (4): > vzdump: describe the guest identity of VMs and containers > pvecm: document the cluster ID and how to get a new one > qm: configuration: describe the read-only meta property > pct: configuration: describe the read-only meta property > > pct.adoc | 35 ++++++ > pct.conf.adoc | 3 + > pmxcfs.adoc | 1 + > pvecm.adoc | 87 +++++++++++++ > qm.adoc | 38 ++++++ > qm.conf.adoc | 3 + > vzdump.adoc | 335 ++++++++++++++++++++++++++++++++++++++++++++++++++ > 7 files changed, 502 insertions(+) > > > Summary over all repositories: > 87 files changed, 10876 insertions(+), 137 deletions(-) >