From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [45.144.208.40]) by lore.proxmox.com (Postfix) with ESMTPS id 09C681FF0AA for ; Tue, 17 Nov 2026 11:47:31 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 91E7E215C2; Tue, 17 Nov 2026 11:47:27 +0100 (CET) Message-ID: <5aaeb776-c738-4e0a-bc6b-814b56f24b06@proxmox.com> Date: Tue, 17 Nov 2026 11:47:15 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Beta Subject: Re: [PATCH cluster/guest-common/qemu-server/container/manager/docs 00/34] fix #7846: guest identity (uid) and cluster ID To: Thomas Lamprecht , pve-devel@lists.proxmox.com References: <20261116004703.2908294-1-t.lamprecht@proxmox.com> Content-Language: en-US From: Dominik Csapak In-Reply-To: <20261116004703.2908294-1-t.lamprecht@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794912440049 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.293 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: JDDWMHK5FA3GQ2FKDS5L66WL5Y6GO5KA X-Message-ID-Hash: JDDWMHK5FA3GQ2FKDS5L66WL5Y6GO5KA X-MailFrom: d.csapak@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: I started reviewing this series and wanted to give some early feedback on the overall approach. More in depth code review see the individual patches (might take a bit going through every patch) Note that i might have overlooked something. If so, please do tell me. My high level feedback: Cluster IDs: Is there a particular reason why 100bit + base32? I couldn't find a rationale for the choice (did i overlook it?) and I think using something 'industry standard' like a UUIDv7 would be a better choice? I'ts just 28 bit more ( with a bit less randomness) but with a time stamp included and this should make each cluster uniquely identifiable too. If we want to stay on base32, maybe 80 or 120 bit is a better choice, since we then don't have to use half a byte (100bit => 12.5 byte) and still provide enough randomness. VM UIDs: Same question about the rationale behind the format, but with one change: the 'registering' time is already recorded in the meta object (as ctime), no? so providing an additional separate timestamp seems weird to me. Also using UUIDv7 here would include a timestamp too and also be more 'industry standard' while only using 1 byte extra (36 vs 20 + 15). Though we probably would save some space by not having two properties in the meta object. UID adoption/checks: The cluster wide check is necessary, i get it. Handling it with a cluster lock + reading the cofnig could make that lookup a bottleneck in big and dynamic clusters? Or did i misunderstood how we get the meta config out? Alternatively we could have that lookup in pmxcfs itself and replicate it like rrd, etc.? having a uuid saved as a 128bit number (16byte) would mean 160k for 10000 guests 16M for 1000000 guests which looks reasonable to me? (we already have to have much more data in pmxcfs with that many guests...) Let me know what you think On 11/16/26 1:48 AM, Thomas Lamprecht wrote: > A VMID can be reused for another guest, and a guest can get another > VMID or move to another cluster through a restore or a remote > migration. So backup software and other tools cannot tell whether they > still look at the same guest, see [0]. > > This series gives every VM and container a guest identity in the > read-only 'meta' property: a random uid (100 bits as 20 base32 > characters) and the time the guest was registered in the cluster. Create > and clone get a new uid. Config changes, migration, rollback, backup and > restore keep it, also under another VMID or in another cluster, as long > as no other guest there has it. Each cluster or standalone node also > gets a cluster ID in the same format, generated and protected by pmxcfs. > The docs patches describe the details. > > Two behavior changes come with it: > > * Restoring over an existing guest whose uid differs from the one of > the backup is refused unless 'allow-identity-mismatch' is set. 'force' > alone does not override this. > * Older versions drop the whole 'meta' property of a guest that has an > identity. So migration and clone to such nodes is refused, and remote > migration bumps the tunnel version. This also hits moves by the HA > stack, which stops a container or a relocated VM before the migration > gets refused. A target check in the LRM that avoids this will follow > separately. > > The patches are in dependency order: guest-common needs a pve-cluster > newer than 9.1.9, qemu-server and container need guest-common, and > manager needs all of them. manager also has to depend on the > qemu-server and container versions with this series, as the migration > guard takes the pve-manager version of a node as proof that the node > keeps the identity. MIN_PVE_MANAGER_VERSION in guest-common is a > stand-in for that pve-manager version. > > Tested with the test suites of the repositories and end to end on a > nested three-node cluster that was upgraded node by node. > > [0]: https://bugzilla.proxmox.com/show_bug.cgi?id=7846 > > cluster: > > Thomas Lamprecht (7): > pmxcfs: reject changes to the cluster ID file from user space > pmxcfs: generate a cluster ID on standalone nodes > pmxcfs: dfsm: add a callback that runs once all members are in sync > pmxcfs: quorum: pass the corosync membership to an optional listener > pmxcfs: dcdb: generate the cluster ID once the cluster is in sync > cluster: add helper to read the cluster ID > pmxcfs: allow removing the cluster ID file in forced local mode > > src/PVE/Cluster.pm | 16 ++ > src/pmxcfs/Makefile | 12 +- > src/pmxcfs/cfs-plug-memdb.c | 46 ++++ > src/pmxcfs/cfs-utils.c | 8 + > src/pmxcfs/cfs-utils.h | 5 + > src/pmxcfs/check_dcdb.c | 529 ++++++++++++++++++++++++++++++++++++ > src/pmxcfs/check_memdb.c | 311 +++++++++++++++++++++ > src/pmxcfs/cluster-id.c | 158 +++++++++++ > src/pmxcfs/cluster-id.h | 52 ++++ > src/pmxcfs/dcdb.c | 144 ++++++++++ > src/pmxcfs/dcdb.h | 5 + > src/pmxcfs/dfsm.c | 58 ++++ > src/pmxcfs/dfsm.h | 14 + > src/pmxcfs/pmxcfs.c | 10 +- > src/pmxcfs/quorum.c | 26 +- > src/pmxcfs/quorum.h | 8 +- > src/pmxcfs/status.c | 12 + > src/pmxcfs/status.h | 3 + > src/test/Makefile | 6 +- > src/test/test_cluster_id.pl | 47 ++++ > 20 files changed, 1458 insertions(+), 12 deletions(-) > create mode 100644 src/pmxcfs/check_dcdb.c > create mode 100644 src/pmxcfs/cluster-id.c > create mode 100644 src/pmxcfs/cluster-id.h > create mode 100644 src/test/test_cluster_id.pl > > > guest-common: > > Thomas Lamprecht (4): > guest identity: add helpers for the guest uid and registration time > abstract config: assign a guest identity and keep it on config writes > abstract config: let the VMID reservation return the previous state > guest identity: allow taking over a uid that is unused in the cluster > > debian/control | 1 + > src/Makefile | 1 + > src/PVE/AbstractConfig.pm | 235 +++++- > src/PVE/GuestIdentity.pm | 384 ++++++++++ > src/tests/Makefile | 10 +- > src/tests/config-identity-tests.pl | 1090 ++++++++++++++++++++++++++++ > src/tests/guest-identity-tests.pl | 421 +++++++++++ > 7 files changed, 2135 insertions(+), 7 deletions(-) > create mode 100644 src/PVE/GuestIdentity.pm > create mode 100755 src/tests/config-identity-tests.pl > create mode 100755 src/tests/guest-identity-tests.pl > > > qemu-server: > > Thomas Lamprecht (6): > api: restore: release the VMID if a restore to an unused VMID fails > api: migrate, clone: refuse targets that would drop the guest identity > fix #7846: config: add a guest identity to the 'meta' property > restore: take over the guest uid from the backup where possible > api: remote migration: take over the guest uid from the source > api: status: return the guest uid > > src/PVE/API2/Qemu.pm | 104 +- > src/PVE/CLI/qmrestore.pm | 7 + > src/PVE/QemuConfig.pm | 7 + > src/PVE/QemuMigrate.pm | 9 +- > src/PVE/QemuServer.pm | 150 ++- > src/PVE/QemuServer/MetaInfo.pm | 2 + > src/test/Makefile | 5 +- > src/test/QemuServerTest/FakeCFS.pm | 197 ++++ > .../q35-windows-pinning-identity-only.conf | 6 + > ...q35-windows-pinning-identity-only.conf.cmd | 26 + > .../meta-identity-invalid.conf | 6 + > .../meta-identity-invalid.conf.strict.error | 2 + > .../meta-identity-invalid.conf | 7 + > .../parse-config-input/meta-identity.conf | 20 + > src/test/restore-config-expected/150.conf | 9 + > src/test/restore-config-input/150.conf | 10 + > src/test/run_api_tests.pl | 157 +++ > src/test/run_guest_identity_tests.pl | 533 ++++++++++ > src/test/run_parse_config_tests.pl | 2 +- > src/test/run_qemu_restore_config_tests.pl | 916 +++++++++++++++++- > .../create/qemu-server/401.conf | 31 + > .../create/qemu-server/402.conf | 31 + > .../rollback/qemu-server/401.conf | 36 + > .../rollback/qemu-server/402.conf | 35 + > .../create/qemu-server/401.conf | 13 + > .../create/qemu-server/402.conf | 13 + > .../rollback/qemu-server/401.conf | 35 + > .../rollback/qemu-server/402.conf | 34 + > src/test/snapshot-test.pm | 85 +- > 29 files changed, 2439 insertions(+), 49 deletions(-) > create mode 100644 src/test/QemuServerTest/FakeCFS.pm > create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf > create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf.cmd > create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf > create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf.strict.error > create mode 100644 src/test/parse-config-input/meta-identity-invalid.conf > create mode 100644 src/test/parse-config-input/meta-identity.conf > create mode 100644 src/test/restore-config-expected/150.conf > create mode 100644 src/test/restore-config-input/150.conf > create mode 100755 src/test/run_guest_identity_tests.pl > create mode 100644 src/test/snapshot-expected/create/qemu-server/401.conf > create mode 100644 src/test/snapshot-expected/create/qemu-server/402.conf > create mode 100644 src/test/snapshot-expected/rollback/qemu-server/401.conf > create mode 100644 src/test/snapshot-expected/rollback/qemu-server/402.conf > create mode 100644 src/test/snapshot-input/create/qemu-server/401.conf > create mode 100644 src/test/snapshot-input/create/qemu-server/402.conf > create mode 100644 src/test/snapshot-input/rollback/qemu-server/401.conf > create mode 100644 src/test/snapshot-input/rollback/qemu-server/402.conf > > > container: > > Thomas Lamprecht (8): > config: add read-only meta property with the creation time > fix #7846: config: add a guest identity to the 'meta' property > api: migrate, clone: refuse targets that would drop the guest identity > restore: take over the guest uid from the backup where possible > restore: refuse overwriting a container with another guest's backup > migrate: remote: clean up the target on a tunnel version mismatch > api: remote migration: take over the guest uid from the source > api: status: return the guest uid > > src/PVE/API2/LXC.pm | 166 +++- > src/PVE/API2/LXC/Config.pm | 29 +- > src/PVE/API2/LXC/Status.pm | 10 + > src/PVE/LXC/Config.pm | 63 +- > src/PVE/LXC/Migrate.pm | 31 +- > src/test/Makefile | 8 +- > src/test/config_tests.pl | 23 + > src/test/lifecycle_tests.pl | 1878 +++++++++++++++++++++++++++++++++++ > 8 files changed, 2177 insertions(+), 31 deletions(-) > create mode 100755 src/test/lifecycle_tests.pl > > > manager: > > Thomas Lamprecht (5): > api: cluster resources: add guest uid and registration time > api: cluster status: add cluster ID > ui: restore: allow overwriting a guest with a backup of another guest > ui: migrate: refuse target nodes that would drop the guest identity > ui: guest summary: show the guest identity > > PVE/API2/Cluster.pm | 35 +++- > test/ClusterResources_test.pl | 249 +++++++++++++++++++++++ > test/ClusterStatus_test.pl | 114 +++++++++++ > test/Makefile | 10 +- > www/manager6/Makefile | 1 + > www/manager6/panel/GuestIdentity.js | 142 +++++++++++++ > www/manager6/panel/GuestStatusView.js | 3 + > www/manager6/panel/TemplateStatusView.js | 3 + > www/manager6/window/Migrate.js | 29 +++ > www/manager6/window/Restore.js | 18 ++ > 10 files changed, 602 insertions(+), 2 deletions(-) > create mode 100755 test/ClusterResources_test.pl > create mode 100755 test/ClusterStatus_test.pl > create mode 100644 www/manager6/panel/GuestIdentity.js > > > docs: > > Thomas Lamprecht (4): > vzdump: describe the guest identity of VMs and containers > pvecm: document the cluster ID and how to get a new one > qm: configuration: describe the read-only meta property > pct: configuration: describe the read-only meta property > > pct.adoc | 35 +++++++ > pmxcfs.adoc | 1 + > pvecm.adoc | 86 +++++++++++++++++ > qm.adoc | 38 ++++++++ > vzdump.adoc | 259 ++++++++++++++++++++++++++++++++++++++++++++++++++++ > 5 files changed, 419 insertions(+) > > > Summary over all repositories: > 79 files changed, 9230 insertions(+), 101 deletions(-) >