* [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID
@ 2026-11-18 4:37 Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 01/38] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
` (39 more replies)
0 siblings, 40 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A VMID can be reused for another guest, and a guest can get another
VMID or move to another cluster through a restore or a remote
migration. So backup software and other tools cannot tell whether they
still look at the same guest, see [0].
This series gives every VM and container a guest identity in the
read-only 'meta' property: a random uid (100 bits as 20 base32
characters) and the time the guest was registered in the cluster. Create
and clone get a new uid. Config changes, migration, rollback, backup and
restore keep it, also under another VMID or in another cluster, as long
as no other guest there has it. Each cluster or standalone node also
gets a cluster ID in the same format, generated and protected by pmxcfs.
The docs patches describe the details.
The uid is a plain random nonce: 100 bits make a duplicate practically
impossible and fit into a 128-bit value with room to spare, and 20
base32 characters stay short and need no quoting. The registration time
is separate from the creation time, which clones, restores and remote
migrations copy along. It changes when a guest gets registered again, so
tools can tell such a guest from the one they knew.
Three behavior changes come with it:
* Restoring over an existing guest whose uid differs from the one of
the backup is refused unless 'allow-identity-mismatch' is set. 'force'
alone does not override this.
* A migration to another cluster is refused if another guest there
already has the uid, unless 'allow-new-uid' is set. The check runs
before anything is transferred. PDM does not know the option yet.
* Older versions drop the whole 'meta' property of a guest that has an
identity. So migration and clone to such nodes is refused, and remote
migration bumps the tunnel version. This also hits moves by the HA
stack, which stops a container or a relocated VM before the migration
gets refused. A target check in the LRM that avoids this will follow
separately.
The patches are in dependency order: guest-common needs the pve-cluster
of this series, qemu-server and container need guest-common, and manager
needs all of them. manager also has to depend on the qemu-server and
container versions with this series, as the migration guard takes the
pve-manager version of a node as proof that the node keeps the identity.
MIN_PVE_MANAGER_VERSION in guest-common is a stand-in for that version.
Tested with the affected tests of the repositories for every patch and
their full suites at the end, pmxcfs with corosync in network
namespaces, and end to end on a nested three-node cluster that was
upgraded node by node, with remote migrations to a standalone node.
changes v2 -> v3:
* remote migration: a guest without uid gets one before the source
sends it, so source and target no longer end up with unrelated uids;
the identity command moved to pve-guest-common for both guest types
(new patch 13)
* restore: keep the guest if a restored volume or firewall config
cannot be removed, fail if a container backup changes between its
check and its extraction, and let the cleanup follow the reservation
* reserving a VMID refuses an existing config that fails to load
instead of replacing it
* cluster ID: generate it right away when the configured nodes change
(new patch 5)
* feature API without version lookups for snapshot requests, and the
cluster-wide bulk migration refuses its target up front
* docs: failed restores, uid conflicts on migration, qm.conf(5) and
pct.conf(5)
* more independent fixes that the series builds on were applied on
their own: failed and concurrent restores and creates, the schema of
the migration preconditions, the lock of a VMA restore, a node list
cut short in pmxcfs
changes v1 -> v2, thanks to Dominik for the review:
* rebase onto current master (tracking of used guest IDs, dfsm rework)
* remote migration takes the uid over at its start and refuses a uid
that is in use in the target cluster, unless 'allow-new-uid' is set
* qemu-server and container share the restore checks and messages
through pve-guest-common, with fewer special cases
* IDs take each character from one random byte, without an encoder
* the clone dialog does not offer nodes that would drop the identity,
and a bulk migration checks its target once (patch 34)
* independent fixes moved out of the series and got applied on their
own: the IPC client error, the order of removing firewall and guest
config, and the cleanup of a remote migration target after failures
* comments, commit messages and docs give the reasons for the format,
for rtime next to ctime and for the separate lock
Details are in the notes below the "---" of each patch.
v2: https://lore.proxmox.com/pve-devel/20261117223945.862914-1-t.lamprecht@proxmox.com/
v1: https://lore.proxmox.com/pve-devel/20261116004703.2908294-1-t.lamprecht@proxmox.com/
[0]: https://bugzilla.proxmox.com/show_bug.cgi?id=7846
cluster:
Thomas Lamprecht (8):
pmxcfs: reject changes to the cluster ID file from user space
pmxcfs: generate a cluster ID on standalone nodes
pmxcfs: dfsm: add a callback that runs once all members are in sync
pmxcfs: quorum: pass the corosync membership to an optional listener
pmxcfs: confdb: call an optional listener when the nodes get published
pmxcfs: dcdb: generate the cluster ID once the cluster is in sync
cluster: add helper to read the cluster ID
pmxcfs: allow removing the cluster ID file in forced local mode
src/PVE/Cluster.pm | 16 +
src/pmxcfs/Makefile | 12 +-
src/pmxcfs/cfs-plug-memdb.c | 46 +++
src/pmxcfs/cfs-utils.c | 8 +
src/pmxcfs/cfs-utils.h | 5 +
src/pmxcfs/check_dcdb.c | 563 ++++++++++++++++++++++++++++++++++++
src/pmxcfs/check_memdb.c | 283 ++++++++++++++++++
src/pmxcfs/cluster-id.c | 142 +++++++++
src/pmxcfs/cluster-id.h | 50 ++++
src/pmxcfs/confdb.c | 18 +-
src/pmxcfs/confdb.h | 5 +-
src/pmxcfs/dcdb.c | 153 ++++++++++
src/pmxcfs/dcdb.h | 8 +
src/pmxcfs/dfsm.c | 59 ++++
src/pmxcfs/dfsm.h | 14 +
src/pmxcfs/pmxcfs.c | 12 +-
src/pmxcfs/quorum.c | 26 +-
src/pmxcfs/quorum.h | 9 +-
src/pmxcfs/status.c | 12 +
src/pmxcfs/status.h | 3 +
src/test/Makefile | 6 +-
src/test/test_cluster_id.pl | 46 +++
22 files changed, 1478 insertions(+), 18 deletions(-)
create mode 100644 src/pmxcfs/check_dcdb.c
create mode 100644 src/pmxcfs/cluster-id.c
create mode 100644 src/pmxcfs/cluster-id.h
create mode 100644 src/test/test_cluster_id.pl
guest-common:
Thomas Lamprecht (5):
guest identity: add helpers for the guest uid and registration time
abstract config: assign a guest identity and keep it on config writes
abstract config: let the VMID reservation return the previous state
guest identity: allow taking over a uid that is unused in the cluster
migrate: add shared helpers for the identity command of the migration tunnel
src/Makefile | 1 +
src/PVE/AbstractConfig.pm | 202 +++-
src/PVE/AbstractMigrate.pm | 30 +
src/PVE/GuestIdentity.pm | 510 ++++++++++
src/tests/Makefile | 10 +-
src/tests/config-identity-tests.pl | 1391 ++++++++++++++++++++++++++++
src/tests/guest-identity-tests.pl | 530 +++++++++++
7 files changed, 2668 insertions(+), 6 deletions(-)
create mode 100644 src/PVE/GuestIdentity.pm
create mode 100755 src/tests/config-identity-tests.pl
create mode 100755 src/tests/guest-identity-tests.pl
qemu-server:
Thomas Lamprecht (7):
api: restore: release the VMID if a restore to an unused VMID fails
api: migrate, clone: refuse targets that would drop the guest identity
fix #7846: config: add a guest identity to the 'meta' property
restore: take over the guest uid from the backup where possible
api: remote migration: take over the guest uid from the source
api: status: return the guest uid
tests: remote migration: check a failed command and an early tunnel end
src/PVE/API2/Qemu.pm | 118 +-
src/PVE/CLI/qm.pm | 7 +
src/PVE/CLI/qmrestore.pm | 11 +-
src/PVE/QemuConfig.pm | 7 +
src/PVE/QemuMigrate.pm | 18 +-
src/PVE/QemuServer.pm | 155 ++-
src/PVE/QemuServer/MetaInfo.pm | 2 +
src/test/Makefile | 5 +-
src/test/QemuServerTest/FakeCFS.pm | 189 +++
.../q35-windows-pinning-identity-only.conf | 6 +
...q35-windows-pinning-identity-only.conf.cmd | 26 +
.../meta-identity-invalid.conf | 6 +
.../meta-identity-invalid.conf.strict.error | 2 +
.../meta-identity-invalid.conf | 7 +
.../parse-config-input/meta-identity.conf | 20 +
src/test/restore-config-expected/150.conf | 9 +
src/test/restore-config-input/150.conf | 10 +
src/test/run_api_tests.pl | 206 ++++
src/test/run_guest_identity_tests.pl | 803 +++++++++++++
src/test/run_parse_config_tests.pl | 2 +-
src/test/run_qemu_restore_config_tests.pl | 1053 ++++++++++++++++-
.../create/qemu-server/401.conf | 31 +
.../create/qemu-server/402.conf | 31 +
.../rollback/qemu-server/401.conf | 36 +
.../rollback/qemu-server/402.conf | 35 +
.../create/qemu-server/401.conf | 13 +
.../create/qemu-server/402.conf | 13 +
.../rollback/qemu-server/401.conf | 35 +
.../rollback/qemu-server/402.conf | 34 +
src/test/snapshot-test.pm | 82 +-
30 files changed, 2899 insertions(+), 73 deletions(-)
create mode 100644 src/test/QemuServerTest/FakeCFS.pm
create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf
create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf.cmd
create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf
create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf.strict.error
create mode 100644 src/test/parse-config-input/meta-identity-invalid.conf
create mode 100644 src/test/parse-config-input/meta-identity.conf
create mode 100644 src/test/restore-config-expected/150.conf
create mode 100644 src/test/restore-config-input/150.conf
create mode 100755 src/test/run_guest_identity_tests.pl
create mode 100644 src/test/snapshot-expected/create/qemu-server/401.conf
create mode 100644 src/test/snapshot-expected/create/qemu-server/402.conf
create mode 100644 src/test/snapshot-expected/rollback/qemu-server/401.conf
create mode 100644 src/test/snapshot-expected/rollback/qemu-server/402.conf
create mode 100644 src/test/snapshot-input/create/qemu-server/401.conf
create mode 100644 src/test/snapshot-input/create/qemu-server/402.conf
create mode 100644 src/test/snapshot-input/rollback/qemu-server/401.conf
create mode 100644 src/test/snapshot-input/rollback/qemu-server/402.conf
container:
Thomas Lamprecht (8):
config: add read-only meta property with the creation time
fix #7846: config: add a guest identity to the 'meta' property
api: migrate, clone: refuse targets that would drop the guest identity
restore: take over the guest uid from the backup where possible
restore: refuse overwriting a container with another guest's backup
test: remote migration: check the cleanup of the target after failures
api: remote migration: take over the guest uid from the source
api: status: return the guest uid
src/PVE/API2/LXC.pm | 190 ++-
src/PVE/API2/LXC/Config.pm | 29 +-
src/PVE/API2/LXC/Status.pm | 6 +
src/PVE/CLI/pct.pm | 7 +
src/PVE/LXC/Config.pm | 51 +-
src/PVE/LXC/Migrate.pm | 8 +-
src/test/Makefile | 8 +-
src/test/config_tests.pl | 23 +
src/test/lifecycle_tests.pl | 2446 +++++++++++++++++++++++++++++++++++
9 files changed, 2730 insertions(+), 38 deletions(-)
create mode 100755 src/test/lifecycle_tests.pl
manager:
Thomas Lamprecht (6):
api: cluster resources: add guest uid and registration time
api: cluster status: add cluster ID
ui: restore: allow overwriting a guest with a backup of another guest
ui: migrate: refuse target nodes that would drop the guest identity
ui: guest summary: show the guest identity
api: bulk migrate: refuse targets that would drop the guest identity
PVE/API2/Cluster.pm | 24 ++-
PVE/API2/Cluster/BulkAction/Guest.pm | 5 +
PVE/API2/Nodes.pm | 3 +
test/ClusterResources_test.pl | 249 +++++++++++++++++++++++
test/ClusterStatus_test.pl | 114 +++++++++++
test/Makefile | 10 +-
www/manager6/Makefile | 1 +
www/manager6/panel/GuestIdentity.js | 142 +++++++++++++
www/manager6/panel/GuestStatusView.js | 3 +
www/manager6/panel/TemplateStatusView.js | 3 +
www/manager6/window/Migrate.js | 29 +++
www/manager6/window/Restore.js | 18 ++
12 files changed, 599 insertions(+), 2 deletions(-)
create mode 100755 test/ClusterResources_test.pl
create mode 100755 test/ClusterStatus_test.pl
create mode 100644 www/manager6/panel/GuestIdentity.js
docs:
Thomas Lamprecht (4):
vzdump: describe the guest identity of VMs and containers
pvecm: document the cluster ID and how to get a new one
qm: configuration: describe the read-only meta property
pct: configuration: describe the read-only meta property
pct.adoc | 35 ++++++
pct.conf.adoc | 3 +
pmxcfs.adoc | 1 +
pvecm.adoc | 87 +++++++++++++
qm.adoc | 38 ++++++
qm.conf.adoc | 3 +
vzdump.adoc | 335 ++++++++++++++++++++++++++++++++++++++++++++++++++
7 files changed, 502 insertions(+)
Summary over all repositories:
87 files changed, 10876 insertions(+), 137 deletions(-)
--
2.47.3
^ permalink raw reply [flat|nested] 42+ messages in thread
* [PATCH cluster v3 01/38] pmxcfs: reject changes to the cluster ID file from user space
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 02/38] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
` (38 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
The cluster ID identifies a cluster or standalone node and must remain
stable during normal operation. Reserve /etc/pve/cluster-id for pmxcfs
by rejecting local changes to its content and stored metadata with
EACCES, and report the file as read-only.
Protect timestamps too: changing them updates the entry's version and
writer and replicates like a content change. Keep the existing chmod
and chown behavior, which does not change stored data.
Allow rmdir to remove a directory left at this path by an older node;
otherwise it would prevent ID generation. The kernel already rejects
rmdir on the regular file.
Cluster-delivered operations bypass these local filesystem guards, so
the ID still replicates normally, including between mixed versions.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
no changes since v1
src/pmxcfs/cfs-plug-memdb.c | 42 ++++++++++++++++++
src/pmxcfs/cfs-utils.c | 8 ++++
src/pmxcfs/cfs-utils.h | 4 ++
src/pmxcfs/check_memdb.c | 86 +++++++++++++++++++++++++++++++++++++
4 files changed, 140 insertions(+)
diff --git a/src/pmxcfs/cfs-plug-memdb.c b/src/pmxcfs/cfs-plug-memdb.c
index 0af3b18..0d0fa64 100644
--- a/src/pmxcfs/cfs-plug-memdb.c
+++ b/src/pmxcfs/cfs-plug-memdb.c
@@ -129,6 +129,10 @@ static int cfs_plug_memdb_getattr(cfs_plug_t *plug, const char *path, struct sta
if (te) {
tree_entry_stat(te, stbuf, cfs_is_quorate());
+ /* user space can change neither the content nor the stored metadata of the ID file */
+ if (path_is_cluster_id(path)) {
+ stbuf->st_mode &= ~(S_IWUSR | S_IWGRP | S_IWOTH);
+ }
memdb_tree_entry_free(te);
return 0;
}
@@ -186,6 +190,12 @@ static int cfs_plug_memdb_open(cfs_plug_t *plug, const char *path, struct fuse_f
g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
g_return_val_if_fail(fi != NULL, PARAM_CHECK_ERRNO);
+ /* only affects local user space, dcdb applies the operations of other nodes directly */
+ if (path_is_cluster_id(path) &&
+ ((fi->flags & O_ACCMODE) != O_RDONLY || (fi->flags & (O_TRUNC | O_APPEND)))) {
+ return -EACCES;
+ }
+
memdb_tree_entry_t *te;
cfs_plug_memdb_t *mdb = (cfs_plug_memdb_t *)plug;
@@ -255,6 +265,10 @@ static int cfs_plug_memdb_write(
g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
g_return_val_if_fail(buf != NULL, PARAM_CHECK_ERRNO);
+ if (path_is_cluster_id(path)) {
+ return -EACCES;
+ }
+
int res;
cfs_plug_memdb_t *mdb = (cfs_plug_memdb_t *)plug;
@@ -276,6 +290,10 @@ static int cfs_plug_memdb_truncate(cfs_plug_t *plug, const char *path, off_t siz
g_return_val_if_fail(plug->ops == &cfs_ops, PARAM_CHECK_ERRNO);
g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
+ if (path_is_cluster_id(path)) {
+ return -EACCES;
+ }
+
int res;
cfs_plug_memdb_t *mdb = (cfs_plug_memdb_t *)plug;
@@ -298,6 +316,10 @@ cfs_plug_memdb_create(cfs_plug_t *plug, const char *path, mode_t mode, struct fu
g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
g_return_val_if_fail(fi != NULL, PARAM_CHECK_ERRNO);
+ if (path_is_cluster_id(path)) {
+ return -EACCES;
+ }
+
int res;
cfs_plug_memdb_t *mdb = (cfs_plug_memdb_t *)plug;
@@ -318,6 +340,10 @@ static int cfs_plug_memdb_mkdir(cfs_plug_t *plug, const char *path, mode_t mode)
g_return_val_if_fail(plug->ops == &cfs_ops, PARAM_CHECK_ERRNO);
g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
+ if (path_is_cluster_id(path)) {
+ return -EACCES;
+ }
+
int res;
cfs_plug_memdb_t *mdb = (cfs_plug_memdb_t *)plug;
@@ -337,6 +363,10 @@ static int cfs_plug_memdb_rmdir(cfs_plug_t *plug, const char *path) {
g_return_val_if_fail(plug->ops == &cfs_ops, PARAM_CHECK_ERRNO);
g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
+ /* no cluster ID check: the kernel refuses rmdir on the regular file, and a directory that a
+ * node without the check created at that path must stay removable, as it keeps the ID from
+ * being written */
+
int res;
cfs_plug_memdb_t *mdb = (cfs_plug_memdb_t *)plug;
@@ -357,6 +387,10 @@ static int cfs_plug_memdb_rename(cfs_plug_t *plug, const char *from, const char
g_return_val_if_fail(from != NULL, PARAM_CHECK_ERRNO);
g_return_val_if_fail(to != NULL, PARAM_CHECK_ERRNO);
+ if (path_is_cluster_id(from) || path_is_cluster_id(to)) {
+ return -EACCES;
+ }
+
int res;
cfs_plug_memdb_t *mdb = (cfs_plug_memdb_t *)plug;
@@ -376,6 +410,10 @@ static int cfs_plug_memdb_unlink(cfs_plug_t *plug, const char *path) {
g_return_val_if_fail(plug->ops == &cfs_ops, PARAM_CHECK_ERRNO);
g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
+ if (path_is_cluster_id(path)) {
+ return -EACCES;
+ }
+
int res;
cfs_plug_memdb_t *mdb = (cfs_plug_memdb_t *)plug;
@@ -396,6 +434,10 @@ static int cfs_plug_memdb_utimens(cfs_plug_t *plug, const char *path, const stru
g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
g_return_val_if_fail(tv != NULL, PARAM_CHECK_ERRNO);
+ if (path_is_cluster_id(path)) {
+ return -EACCES;
+ }
+
int res;
cfs_plug_memdb_t *mdb = (cfs_plug_memdb_t *)plug;
diff --git a/src/pmxcfs/cfs-utils.c b/src/pmxcfs/cfs-utils.c
index 239d380..e303193 100644
--- a/src/pmxcfs/cfs-utils.c
+++ b/src/pmxcfs/cfs-utils.c
@@ -339,3 +339,11 @@ gboolean path_is_lockdir(const char *path) {
return (strncmp(path, "priv/lock/", 10) == 0) && path[10];
}
+
+gboolean path_is_cluster_id(const char *path) {
+ size_t len;
+ const char *component = path_next_component(&path, &len);
+
+ return component && path_component_is(component, len, CFS_CLUSTER_ID_FN) &&
+ !path_next_component(&path, &len);
+}
diff --git a/src/pmxcfs/cfs-utils.h b/src/pmxcfs/cfs-utils.h
index 85e60af..f66e944 100644
--- a/src/pmxcfs/cfs-utils.h
+++ b/src/pmxcfs/cfs-utils.h
@@ -34,6 +34,8 @@
#define VARLIBDIR "/var/lib/pve-cluster"
#define RUNDIR "/run/pve-cluster"
+#define CFS_CLUSTER_ID_FN "cluster-id"
+
#define CFS_MAX(a, b) (((a) > (b)) ? (a) : (b))
#define CFS_MIN(a, b) (((a) < (b)) ? (a) : (b))
@@ -98,4 +100,6 @@ gboolean path_is_lxc_conf(const char *path);
gboolean path_is_lockdir(const char *path);
+gboolean path_is_cluster_id(const char *path);
+
#endif /* _PVE_CFS_UTILS_H_ */
diff --git a/src/pmxcfs/check_memdb.c b/src/pmxcfs/check_memdb.c
index 74728df..caac740 100644
--- a/src/pmxcfs/check_memdb.c
+++ b/src/pmxcfs/check_memdb.c
@@ -26,6 +26,7 @@
#include <string.h>
#include <unistd.h>
+#include "cfs-plug-memdb.h"
#include "cfs-utils.h"
#include "memdb.h"
#include "status.h"
@@ -220,6 +221,87 @@ START_TEST(test_version_and_members_json) {
}
END_TEST
+START_TEST(test_path_is_cluster_id) {
+ ck_assert(path_is_cluster_id("cluster-id"));
+ ck_assert(path_is_cluster_id("/cluster-id"));
+ ck_assert(path_is_cluster_id("//cluster-id/"));
+
+ ck_assert(!path_is_cluster_id(""));
+ ck_assert(!path_is_cluster_id("/"));
+ ck_assert(!path_is_cluster_id("cluster-i"));
+ ck_assert(!path_is_cluster_id("cluster-id.new"));
+ ck_assert(!path_is_cluster_id("cluster-id/x"));
+ ck_assert(!path_is_cluster_id("nodes/n1/cluster-id"));
+}
+END_TEST
+
+START_TEST(test_cluster_id_fuse_read_only) {
+ time_t ctime = 1234;
+ const char *id = "abcdefghijklmnopqrst\n";
+ gpointer data;
+ struct fuse_file_info fi = {0};
+ struct fuse_file_info fi_write = {.flags = O_WRONLY};
+ struct fuse_file_info fi_trunc = {.flags = O_RDONLY | O_TRUNC};
+ struct fuse_file_info fi_append = {.flags = O_RDONLY | O_APPEND};
+ struct timespec tv[2] = {{.tv_sec = 42}, {.tv_sec = 42}};
+ struct stat stbuf;
+
+ cfs_plug_t *plug = (cfs_plug_t *)cfs_plug_memdb_new("memdb", memdb, NULL);
+ ck_assert(plug != NULL);
+
+ ck_assert(plug->ops->create(plug, "cluster-id", 0644, &fi) == -EACCES);
+ ck_assert(plug->ops->mkdir(plug, "cluster-id", 0755) == -EACCES);
+ ck_assert(memdb_read(memdb, "cluster-id", &data) == -ENOENT);
+
+ /* pmxcfs itself goes to the memdb directly */
+ ck_assert(memdb_create(memdb, "cluster-id", 0, ctime) == 0);
+ ck_assert(memdb_write(memdb, "cluster-id", 0, ctime, id, 21, 0, 1) == 21);
+
+ memdb_tree_entry_t *before = memdb_getattr(memdb, "cluster-id");
+ ck_assert(before != NULL);
+
+ ck_assert(plug->ops->open(plug, "cluster-id", &fi) == 0);
+ ck_assert(plug->ops->open(plug, "cluster-id", &fi_write) == -EACCES);
+ ck_assert(plug->ops->open(plug, "cluster-id", &fi_trunc) == -EACCES);
+ ck_assert(plug->ops->open(plug, "cluster-id", &fi_append) == -EACCES);
+ ck_assert(plug->ops->write(plug, "cluster-id", "x", 1, 0, &fi) == -EACCES);
+ ck_assert(plug->ops->truncate(plug, "cluster-id", 0) == -EACCES);
+ ck_assert(plug->ops->unlink(plug, "cluster-id") == -EACCES);
+ ck_assert(plug->ops->rename(plug, "cluster-id", "moved") == -EACCES);
+ ck_assert(plug->ops->mkdir(plug, "cluster-id", 0755) == -EACCES);
+ ck_assert(plug->ops->utimens(plug, "cluster-id", tv) == -EACCES);
+
+ ck_assert(plug->ops->create(plug, "other", 0644, &fi) == 0);
+ ck_assert(plug->ops->write(plug, "other", "x", 1, 0, &fi) == 1);
+ ck_assert(plug->ops->rename(plug, "other", "cluster-id") == -EACCES);
+
+ ck_assert(memdb_read(memdb, "cluster-id", &data) == 21);
+ ck_assert(memcmp(data, id, 21) == 0);
+ g_free(data);
+
+ /* not even the metadata changed */
+ memdb_tree_entry_t *after = memdb_getattr(memdb, "cluster-id");
+ ck_assert(after != NULL);
+ ck_assert(after->version == before->version);
+ ck_assert(after->mtime == before->mtime);
+ memdb_tree_entry_free(before);
+ memdb_tree_entry_free(after);
+
+ cfs_set_quorate(1, TRUE);
+ ck_assert(plug->ops->getattr(plug, "cluster-id", &stbuf) == 0);
+ ck_assert((stbuf.st_mode & 0777) == 0444);
+ ck_assert(plug->ops->getattr(plug, "other", &stbuf) == 0);
+ ck_assert((stbuf.st_mode & 0777) == 0666);
+
+ /* a directory at the path, as a node without the check can create, stays removable */
+ ck_assert(memdb_delete(memdb, "cluster-id", 0, ctime) == 0);
+ ck_assert(memdb_mkdir(memdb, "cluster-id", 0, ctime) == 0);
+ ck_assert(plug->ops->rmdir(plug, "cluster-id") == 0);
+
+ plug->destroy_plug(plug);
+}
+END_TEST
+
START_TEST(test_truncate_grow) {
const char *fn = "/f1";
time_t ctime = 1234;
@@ -504,6 +586,10 @@ static Suite *memdb_suite(void) {
add_test(s, test_guest_config_properties_last_wins, "guest_config_properties_last_wins");
+ add_test(s, test_path_is_cluster_id, "path_is_cluster_id");
+
+ add_test(s, test_cluster_id_fuse_read_only, "cluster_id_fuse_read_only");
+
add_test(s, test_indextest1, "indextest1");
add_test(s, test_loaddb1, "loaddb1");
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH cluster v3 02/38] pmxcfs: generate a cluster ID on standalone nodes
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 01/38] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 03/38] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
` (37 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Standalone nodes need a cluster ID to distinguish them from other
installations. Generate and persist it before mounting the filesystem
or starting the IPC server, when no other writer or reader can access
the database.
Use 20 random characters of the lowercase RFC 4648 base32 alphabet, each
from a random byte of /dev/urandom, followed by a newline. These 100
random bits make a duplicate between installations practically
impossible, fit into a 128-bit value with bits to spare, and stay short
and free of characters that need quoting. Keep a valid existing ID, with
or without its newline. Replace invalid content, including an empty file
left by interrupted initialization. Log a directory at the path and
leave it for an administrator to remove with rmdir.
Abort startup if storing the ID fails: the in-memory database could
otherwise expose an ID that is lost or replaced after a restart. A
failure to obtain random data only logs an error, since nothing was
written and the next startup can retry.
Do not generate an ID in forced local mode if corosync.conf exists.
That node belongs to a cluster and must not create a conflicting ID
before rejoining it.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* say in the comment that it is about a failed generation
changes v1 -> v2:
* take each character from a random byte of its own instead of encoding
13 random bytes, which drops the encoder and its test vectors
(Dominik)
* give the reasons for the format in the commit message
src/pmxcfs/Makefile | 2 +-
src/pmxcfs/check_memdb.c | 165 +++++++++++++++++++++++++++++++++++++++
src/pmxcfs/cluster-id.c | 142 +++++++++++++++++++++++++++++++++
src/pmxcfs/cluster-id.h | 50 ++++++++++++
src/pmxcfs/pmxcfs.c | 6 ++
5 files changed, 364 insertions(+), 1 deletion(-)
create mode 100644 src/pmxcfs/cluster-id.c
create mode 100644 src/pmxcfs/cluster-id.h
diff --git a/src/pmxcfs/Makefile b/src/pmxcfs/Makefile
index 547f258..cd93e72 100644
--- a/src/pmxcfs/Makefile
+++ b/src/pmxcfs/Makefile
@@ -21,7 +21,7 @@ all: pmxcfs create_pmxcfs_db logtest check_memdb
libpmxcfs.a: logger.o loop.o server.o status.o confdb.o quorum.o dcdb.o dfsm.o
libpmxcfs.a: cfs-plug.o cfs-plug-memdb.o cfs-plug-link.o cfs-plug-func.o
-libpmxcfs.a: cfs-utils.o memdb.o database.o
+libpmxcfs.a: cfs-utils.o memdb.o database.o cluster-id.o
libpmxcfs.a:
$(AR) $(ARFLAGS) $@ $^
diff --git a/src/pmxcfs/check_memdb.c b/src/pmxcfs/check_memdb.c
index caac740..db85cbd 100644
--- a/src/pmxcfs/check_memdb.c
+++ b/src/pmxcfs/check_memdb.c
@@ -19,8 +19,10 @@
*/
#include <check.h>
+#include <dirent.h>
#include <errno.h>
#include <glib.h>
+#include <sqlite3.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
@@ -28,6 +30,7 @@
#include "cfs-plug-memdb.h"
#include "cfs-utils.h"
+#include "cluster-id.h"
#include "memdb.h"
#include "status.h"
@@ -302,6 +305,158 @@ START_TEST(test_cluster_id_fuse_read_only) {
}
END_TEST
+START_TEST(test_cluster_id_is_valid) {
+ ck_assert(cluster_id_is_valid("abcdefghijklmnopqrst\n", 21));
+ ck_assert(cluster_id_is_valid("uvwxyz234567abcdefgh\n", 21));
+ ck_assert(cluster_id_is_valid("abcdefghijklmnopqrst", 20));
+
+ ck_assert(!cluster_id_is_valid(NULL, 21));
+ ck_assert(!cluster_id_is_valid("", 0));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmno", 15));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmnop", 16));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmnop\n", 17));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmnopqrs\n", 20));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmnopqrstu", 21));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmnopqrstu\n", 22));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmnopqrst\n\n", 22));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmnopqrst ", 21));
+ ck_assert(!cluster_id_is_valid("Abcdefghijklmnopqrst\n", 21));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmnopqrs1\n", 21));
+ ck_assert(!cluster_id_is_valid("abcdefghijklmnopqrs8\n", 21));
+ ck_assert(!cluster_id_is_valid("abcdefgh\0jklmnopqrst\n", 21));
+}
+END_TEST
+
+START_TEST(test_cluster_id_generate) {
+ char id1[CLUSTER_ID_FILE_SIZE], id2[CLUSTER_ID_FILE_SIZE];
+
+ ck_assert(cluster_id_generate(id1));
+ ck_assert(cluster_id_generate(id2));
+
+ ck_assert(cluster_id_is_valid(id1, sizeof(id1)));
+ ck_assert(cluster_id_is_valid(id2, sizeof(id2)));
+ ck_assert(memcmp(id1, id2, sizeof(id1)) != 0);
+}
+END_TEST
+
+START_TEST(test_cluster_id_state) {
+ time_t ctime = 1234;
+
+ ck_assert(cluster_id_state(memdb) == CLUSTER_ID_MISSING);
+
+ ck_assert(memdb_create(memdb, "cluster-id", 0, ctime) == 0);
+ ck_assert(cluster_id_state(memdb) == CLUSTER_ID_INVALID);
+
+ ck_assert(memdb_write(memdb, "cluster-id", 0, ctime, "abcdefghijklmnopqrst\n", 21, 0, 1) == 21);
+ ck_assert(cluster_id_state(memdb) == CLUSTER_ID_VALID);
+
+ ck_assert(memdb_write(memdb, "cluster-id", 0, ctime, "foo", 3, 0, 1) == 3);
+ ck_assert(cluster_id_state(memdb) == CLUSTER_ID_INVALID);
+
+ ck_assert(memdb_delete(memdb, "cluster-id", 0, ctime) == 0);
+ ck_assert(memdb_mkdir(memdb, "cluster-id", 0, ctime) == 0);
+ ck_assert(cluster_id_state(memdb) == CLUSTER_ID_NOT_A_FILE);
+}
+END_TEST
+
+START_TEST(test_cluster_id_standalone) {
+ time_t ctime = 1234;
+ gpointer data;
+ int len;
+
+ ck_assert(cluster_id_ensure_standalone(memdb));
+ len = memdb_read(memdb, "cluster-id", &data);
+ ck_assert(len == CLUSTER_ID_FILE_SIZE);
+ ck_assert(cluster_id_is_valid(data, len));
+
+ /* an existing ID is kept */
+ char id[CLUSTER_ID_FILE_SIZE];
+ memcpy(id, data, sizeof(id));
+ g_free(data);
+
+ ck_assert(cluster_id_ensure_standalone(memdb));
+ ck_assert(memdb_read(memdb, "cluster-id", &data) == CLUSTER_ID_FILE_SIZE);
+ ck_assert(memcmp(data, id, sizeof(id)) == 0);
+ g_free(data);
+
+ /* an empty file is what a crash between creating and writing leaves behind, it gets filled */
+ ck_assert(memdb_delete(memdb, "cluster-id", 0, ctime) == 0);
+ ck_assert(memdb_create(memdb, "cluster-id", 0, ctime) == 0);
+
+ ck_assert(cluster_id_ensure_standalone(memdb));
+ len = memdb_read(memdb, "cluster-id", &data);
+ ck_assert(len == CLUSTER_ID_FILE_SIZE);
+ ck_assert(cluster_id_is_valid(data, len));
+ g_free(data);
+
+ /* so is invalid content */
+ ck_assert(memdb_write(memdb, "cluster-id", 0, ctime, "foo\n", 4, 0, 1) == 4);
+
+ ck_assert(cluster_id_ensure_standalone(memdb));
+ len = memdb_read(memdb, "cluster-id", &data);
+ ck_assert(len == CLUSTER_ID_FILE_SIZE);
+ ck_assert(cluster_id_is_valid(data, len));
+ g_free(data);
+
+ /* a valid ID without newline is kept as is */
+ ck_assert(memdb_write(memdb, "cluster-id", 0, ctime, "abcdefghijklmnopqrst", 20, 0, 1) == 20);
+
+ ck_assert(cluster_id_ensure_standalone(memdb));
+ ck_assert(memdb_read(memdb, "cluster-id", &data) == 20);
+ ck_assert(memcmp(data, "abcdefghijklmnopqrst", 20) == 0);
+ g_free(data);
+
+ /* an ID with only 16 characters is malformed like anything else and gets replaced */
+ ck_assert(memdb_write(memdb, "cluster-id", 0, ctime, "abcdefghijklmnop\n", 17, 0, 1) == 17);
+
+ ck_assert(cluster_id_ensure_standalone(memdb));
+ len = memdb_read(memdb, "cluster-id", &data);
+ ck_assert(len == CLUSTER_ID_FILE_SIZE);
+ ck_assert(cluster_id_is_valid(data, len));
+ g_free(data);
+
+ /* a directory is left for the admin and does not stop the start */
+ ck_assert(memdb_delete(memdb, "cluster-id", 0, ctime) == 0);
+ ck_assert(memdb_mkdir(memdb, "cluster-id", 0, ctime) == 0);
+
+ ck_assert(cluster_id_ensure_standalone(memdb));
+ memdb_tree_entry_t *te = memdb_getattr(memdb, "cluster-id");
+ ck_assert(te != NULL);
+ ck_assert(te->type == DT_DIR);
+ memdb_tree_entry_free(te);
+}
+END_TEST
+
+static void sql_exec(const char *statement) {
+ sqlite3 *db = NULL;
+ ck_assert(sqlite3_open(TESTDB, &db) == SQLITE_OK);
+ ck_assert(sqlite3_exec(db, statement, NULL, NULL, NULL) == SQLITE_OK);
+ ck_assert(sqlite3_close(db) == SQLITE_OK);
+}
+
+START_TEST(test_cluster_id_standalone_write_failure) {
+ sql_exec(
+ "CREATE TRIGGER fail_cluster_id BEFORE INSERT ON tree "
+ "WHEN NEW.name = 'cluster-id' AND length(NEW.data) > 0 "
+ "BEGIN SELECT RAISE(FAIL, 'injected failure'); END;"
+ );
+
+ /* creating the empty file still works, storing the content fails */
+ ck_assert(!cluster_id_ensure_standalone(memdb));
+ ck_assert(memdb->errors);
+
+ memdb_close(memdb);
+ sql_exec("DROP TRIGGER fail_cluster_id;");
+ memdb = memdb_open(TESTDB);
+ ck_assert(memdb != NULL);
+
+ /* the next start finds the empty file and fills it */
+ ck_assert(cluster_id_state(memdb) == CLUSTER_ID_INVALID);
+ ck_assert(cluster_id_ensure_standalone(memdb));
+ ck_assert(cluster_id_state(memdb) == CLUSTER_ID_VALID);
+}
+END_TEST
+
START_TEST(test_truncate_grow) {
const char *fn = "/f1";
time_t ctime = 1234;
@@ -590,6 +745,16 @@ static Suite *memdb_suite(void) {
add_test(s, test_cluster_id_fuse_read_only, "cluster_id_fuse_read_only");
+ add_test(s, test_cluster_id_is_valid, "cluster_id_is_valid");
+
+ add_test(s, test_cluster_id_generate, "cluster_id_generate");
+
+ add_test(s, test_cluster_id_state, "cluster_id_state");
+
+ add_test(s, test_cluster_id_standalone, "cluster_id_standalone");
+
+ add_test(s, test_cluster_id_standalone_write_failure, "cluster_id_standalone_write_failure");
+
add_test(s, test_indextest1, "indextest1");
add_test(s, test_loaddb1, "loaddb1");
diff --git a/src/pmxcfs/cluster-id.c b/src/pmxcfs/cluster-id.c
new file mode 100644
index 0000000..381f638
--- /dev/null
+++ b/src/pmxcfs/cluster-id.c
@@ -0,0 +1,142 @@
+/*
+ Copyright (C) 2026 Proxmox Server Solutions GmbH
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+ Author: Thomas Lamprecht <t.lamprecht@proxmox.com>
+
+*/
+
+#ifdef HAVE_CONFIG_H
+#include <config.h>
+#endif /* HAVE_CONFIG_H */
+
+#include <dirent.h>
+#include <errno.h>
+#include <fcntl.h>
+#include <glib.h>
+#include <time.h>
+#include <unistd.h>
+
+#include "cfs-utils.h"
+#include "cluster-id.h"
+
+static const char base32_alphabet[] = "abcdefghijklmnopqrstuvwxyz234567";
+
+gboolean cluster_id_generate(char id[CLUSTER_ID_FILE_SIZE]) {
+ guint8 raw[CLUSTER_ID_LEN];
+
+ int fd = open("/dev/urandom", O_RDONLY | O_CLOEXEC);
+ if (fd == -1) {
+ cfs_critical("unable to open /dev/urandom - %s", g_strerror(errno));
+ return FALSE;
+ }
+
+ ssize_t res = safe_read(fd, (char *)raw, sizeof(raw));
+ int saved_errno = errno;
+ close(fd);
+
+ if (res != (ssize_t)sizeof(raw)) {
+ cfs_critical(
+ "unable to read from /dev/urandom - %s",
+ res < 0 ? g_strerror(saved_errno) : "short read"
+ );
+ return FALSE;
+ }
+
+ /* five bits of a random byte per character, so each one is uniformly distributed */
+ for (int i = 0; i < CLUSTER_ID_LEN; i++) {
+ id[i] = base32_alphabet[raw[i] & 0x1f];
+ }
+ id[CLUSTER_ID_LEN] = '\n';
+
+ return TRUE;
+}
+
+gboolean cluster_id_is_valid(gconstpointer data, size_t len) {
+ const char *id = data;
+
+ if (!id ||
+ !(len == CLUSTER_ID_LEN || (len == CLUSTER_ID_FILE_SIZE && id[CLUSTER_ID_LEN] == '\n'))) {
+ return FALSE;
+ }
+
+ for (int i = 0; i < CLUSTER_ID_LEN; i++) {
+ if (!((id[i] >= 'a' && id[i] <= 'z') || (id[i] >= '2' && id[i] <= '7'))) {
+ return FALSE;
+ }
+ }
+
+ return TRUE;
+}
+
+cluster_id_state_t cluster_id_state(memdb_t *memdb) {
+ g_return_val_if_fail(memdb != NULL, CLUSTER_ID_NOT_A_FILE);
+
+ memdb_tree_entry_t *te = memdb_getattr(memdb, CFS_CLUSTER_ID_FN);
+ if (!te) {
+ return CLUSTER_ID_MISSING;
+ }
+
+ gboolean is_file = te->type == DT_REG;
+ memdb_tree_entry_free(te);
+ if (!is_file) {
+ return CLUSTER_ID_NOT_A_FILE;
+ }
+
+ gpointer data = NULL;
+ int len = memdb_read(memdb, CFS_CLUSTER_ID_FN, &data);
+ gboolean valid = len >= 0 && cluster_id_is_valid(data, len);
+ g_free(data);
+
+ return valid ? CLUSTER_ID_VALID : CLUSTER_ID_INVALID;
+}
+
+gboolean cluster_id_ensure_standalone(memdb_t *memdb) {
+ g_return_val_if_fail(memdb != NULL, FALSE);
+
+ cluster_id_state_t state = cluster_id_state(memdb);
+ if (state == CLUSTER_ID_VALID) {
+ return TRUE;
+ }
+
+ if (state == CLUSTER_ID_NOT_A_FILE) {
+ cfs_message("not generating a cluster ID, '%s' is not a regular file", CFS_CLUSTER_ID_FN);
+ return TRUE;
+ }
+
+ /* on failure nothing got written, so the start can go on and the next one tries again */
+ char id[CLUSTER_ID_FILE_SIZE];
+ if (!cluster_id_generate(id)) {
+ return TRUE;
+ }
+
+ /* nothing is mounted yet, so no reader can see the file between creating and writing it, and
+ * a crash in between leaves an empty file that the next start fills */
+ guint32 mtime = time(NULL);
+ int res = state == CLUSTER_ID_MISSING ? memdb_create(memdb, CFS_CLUSTER_ID_FN, 0, mtime) : 0;
+ if (res == 0) {
+ res = memdb_write(memdb, CFS_CLUSTER_ID_FN, 0, mtime, id, CLUSTER_ID_FILE_SIZE, 0, TRUE);
+ }
+
+ /* the memdb already shows the new content if only storing it failed, so it must not be used */
+ if (res != CLUSTER_ID_FILE_SIZE) {
+ cfs_critical("unable to write '%s' - %s", CFS_CLUSTER_ID_FN, g_strerror(-res));
+ return FALSE;
+ }
+
+ cfs_message("generated cluster ID %.*s", CLUSTER_ID_LEN, id);
+
+ return TRUE;
+}
diff --git a/src/pmxcfs/cluster-id.h b/src/pmxcfs/cluster-id.h
new file mode 100644
index 0000000..2715f7d
--- /dev/null
+++ b/src/pmxcfs/cluster-id.h
@@ -0,0 +1,50 @@
+/*
+ Copyright (C) 2026 Proxmox Server Solutions GmbH
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+ Author: Thomas Lamprecht <t.lamprecht@proxmox.com>
+
+*/
+
+#ifndef _PVE_CLUSTER_ID_H_
+#define _PVE_CLUSTER_ID_H_
+
+#include <glib.h>
+
+#include "memdb.h"
+
+/* the ID is 20 random characters of the lowercase RFC 4648 base32 alphabet, so 100 random bits,
+ * pmxcfs writes it plus a newline */
+#define CLUSTER_ID_LEN 20
+#define CLUSTER_ID_FILE_SIZE (CLUSTER_ID_LEN + 1)
+
+gboolean cluster_id_generate(char id[CLUSTER_ID_FILE_SIZE]);
+
+gboolean cluster_id_is_valid(gconstpointer data, size_t len);
+
+typedef enum {
+ CLUSTER_ID_VALID,
+ CLUSTER_ID_MISSING,
+ CLUSTER_ID_INVALID,
+ /* a directory, which only rmdir can remove */
+ CLUSTER_ID_NOT_A_FILE,
+} cluster_id_state_t;
+
+cluster_id_state_t cluster_id_state(memdb_t *memdb);
+
+/* FALSE only if storing the ID failed, the memdb must not be used then */
+gboolean cluster_id_ensure_standalone(memdb_t *memdb);
+
+#endif /* _PVE_CLUSTER_ID_H_ */
diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c
index b8a212d..2bf9145 100644
--- a/src/pmxcfs/pmxcfs.c
+++ b/src/pmxcfs/pmxcfs.c
@@ -48,6 +48,7 @@
#include "cfs-plug-memdb.h"
#include "cfs-plug.h"
#include "cfs-utils.h"
+#include "cluster-id.h"
#include "confdb.h"
#include "dcdb.h"
#include "dfsm.h"
@@ -912,6 +913,11 @@ int main(int argc, char *argv[]) {
} else {
cfs_debug("using local mode (corosync.conf does not exist)");
cfs_set_quorate(1, TRUE);
+ /* only without a corosync.conf, forced local mode on a cluster node must not generate an ID
+ * of its own */
+ if (!cluster_id_ensure_standalone(memdb)) {
+ goto err;
+ }
}
if (conf_data) {
g_free(conf_data);
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH cluster v3 03/38] pmxcfs: dfsm: add a callback that runs once all members are in sync
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 01/38] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 02/38] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 04/38] pmxcfs: quorum: pass the corosync membership to an optional listener Thomas Lamprecht
` (36 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A service acting on shared state from the loop thread needs to know
when the local state matches that of all members. The existing synced
callback runs before queued messages are delivered. The sync leader
also enters that state before members needing updates have received
them, so their queued messages are still pending.
Add an optional callback after all members are marked in sync and the
queued messages have been delivered. Call it again on service timer
ticks while that condition holds, allowing failed or skipped actions
to be retried.
Other loop events can request the same check with dfsm_check_in_sync().
Use dfsm_nodeid_is_member() to check membership in the current group.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* comment: the condition is what this node has marked as synced, not an
acknowledgement of the other nodes
changes v1 -> v2:
* rebase onto the dfsm rework
* also reset the in-sync state when the connection gets finalized
* reword the comment of the callback
src/pmxcfs/dfsm.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++
src/pmxcfs/dfsm.h | 14 +++++++++++
2 files changed, 73 insertions(+)
diff --git a/src/pmxcfs/dfsm.c b/src/pmxcfs/dfsm.c
index c83cef7..b48ee13 100644
--- a/src/pmxcfs/dfsm.c
+++ b/src/pmxcfs/dfsm.c
@@ -124,6 +124,7 @@ struct dfsm {
uint32_t local_epoch_counter;
dfsm_sync_epoch_t sync_epoch;
uint32_t lowest_nodeid;
+ gboolean in_sync; /* see dfsm_in_sync_fn */
GSequence *msg_queue;
GList *sync_queue;
@@ -609,6 +610,27 @@ gboolean dfsm_restartable(dfsm_t *dfsm) {
void dfsm_set_errormode(dfsm_t *dfsm) { dfsm_set_mode(dfsm, DFSM_MODE_ERROR); }
+static void dfsm_notify_in_sync(dfsm_t *dfsm) {
+ g_return_if_fail(dfsm != NULL);
+ g_return_if_fail(dfsm->sync_info != NULL);
+
+ if (dfsm->in_sync || dfsm_get_mode(dfsm) != DFSM_MODE_SYNCED) {
+ return;
+ }
+
+ for (int i = 0; i < dfsm->sync_info->node_count; i++) {
+ if (!dfsm->sync_info->nodes[i].synced) {
+ return;
+ }
+ }
+
+ dfsm->in_sync = TRUE;
+
+ if (dfsm->dfsm_callbacks->dfsm_in_sync_fn) {
+ dfsm->dfsm_callbacks->dfsm_in_sync_fn(dfsm, dfsm->data);
+ }
+}
+
static void dfsm_release_sync_resources(
dfsm_t *dfsm, const struct cpg_address *member_list, size_t member_list_entries
) {
@@ -802,6 +824,8 @@ static void dfsm_cpg_deliver_callback(
goto leave;
}
+ dfsm_notify_in_sync(dfsm);
+
return;
} else if (base_header->type == DFSM_MESSAGE_VERIFY_REQUEST) {
@@ -990,6 +1014,8 @@ static void dfsm_cpg_deliver_callback(
goto leave;
}
+ dfsm_notify_in_sync(dfsm);
+
} else {
dfsm_set_mode(dfsm, DFSM_MODE_UPDATE);
@@ -1040,6 +1066,8 @@ static void dfsm_cpg_deliver_callback(
dfsm_release_sync_resources(dfsm, NULL, 0);
+ dfsm_notify_in_sync(dfsm);
+
return;
}
@@ -1212,6 +1240,7 @@ static void dfsm_cpg_confchg_callback(
dfsm->we_are_member = 0;
dfsm->state_sent = FALSE;
+ dfsm->in_sync = FALSE;
/* create new epoch */
dfsm->local_epoch_counter++;
@@ -1304,6 +1333,7 @@ static void dfsm_cpg_confchg_callback(
if (!dfsm_deliver_queue(dfsm)) {
goto leave;
}
+ dfsm_notify_in_sync(dfsm);
}
if (dfsm->dfsm_callbacks->dfsm_confchg_fn) {
@@ -1390,6 +1420,31 @@ gboolean dfsm_is_initialized(dfsm_t *dfsm) {
return (dfsm->cpg_handle != 0) ? TRUE : FALSE;
}
+void dfsm_check_in_sync(dfsm_t *dfsm) {
+ g_return_if_fail(dfsm != NULL);
+
+ if (dfsm->in_sync && dfsm_get_mode(dfsm) == DFSM_MODE_SYNCED &&
+ dfsm->dfsm_callbacks->dfsm_in_sync_fn) {
+ dfsm->dfsm_callbacks->dfsm_in_sync_fn(dfsm, dfsm->data);
+ }
+}
+
+gboolean dfsm_nodeid_is_member(dfsm_t *dfsm, uint32_t nodeid) {
+ g_return_val_if_fail(dfsm != NULL, FALSE);
+
+ if (!dfsm->sync_info) {
+ return FALSE;
+ }
+
+ for (int i = 0; i < dfsm->sync_info->node_count; i++) {
+ if (dfsm->sync_info->nodes[i].nodeid == nodeid) {
+ return TRUE;
+ }
+ }
+
+ return FALSE;
+}
+
gboolean dfsm_lowest_nodeid(dfsm_t *dfsm) {
g_return_val_if_fail(dfsm != NULL, FALSE);
@@ -1481,6 +1536,7 @@ cs_error_t dfsm_initialize(dfsm_t *dfsm, int *fd) {
dfsm->joined = FALSE;
g_mutex_unlock(&dfsm->cpg_mutex);
dfsm->we_are_member = 0;
+ dfsm->in_sync = FALSE;
dfsm_set_mode(dfsm, DFSM_MODE_START);
@@ -1608,6 +1664,7 @@ gboolean dfsm_finalize(dfsm_t *dfsm) {
dfsm->joined = FALSE;
g_mutex_unlock(&dfsm->cpg_mutex);
dfsm->we_are_member = 0;
+ dfsm->in_sync = FALSE;
}
return TRUE;
@@ -1759,6 +1816,8 @@ static void service_dfsm_timer(cfs_service_t *service, gpointer context) {
g_return_if_fail(dfsm != NULL);
dfsm_verify_request(dfsm);
+
+ dfsm_check_in_sync(dfsm);
}
static cfs_service_callbacks_t cfs_dfsm_callbacks = {
diff --git a/src/pmxcfs/dfsm.h b/src/pmxcfs/dfsm.h
index 5b193e8..6ec3eb1 100644
--- a/src/pmxcfs/dfsm.h
+++ b/src/pmxcfs/dfsm.h
@@ -79,6 +79,8 @@ typedef int (*dfsm_process_state_update_fn_t)(
typedef void (*dfsm_synced_fn_t)(dfsm_t *dfsm);
+typedef void (*dfsm_in_sync_fn_t)(dfsm_t *dfsm, gpointer data);
+
/* return values:
* res < 0 ... serious error, leave group
* res >= 0 ... OK
@@ -103,6 +105,12 @@ typedef struct {
dfsm_process_update_fn_t dfsm_process_update_fn;
dfsm_checksum_fn_t dfsm_checksum_fn;
dfsm_synced_fn_t dfsm_synced_fn;
+ /* Called whenever this node has marked all members as synced and delivered the messages queued
+ * meanwhile, so that the local state matches the one of all members: first right after a sync,
+ * then again on every service timer tick and dfsm_check_in_sync() call while that holds.
+ * dfsm_synced_fn runs earlier, when the local node enters the synced mode and before the queued
+ * messages are delivered. */
+ dfsm_in_sync_fn_t dfsm_in_sync_fn;
} dfsm_callbacks_t;
/* what became of a message sent with dfsm_send_message_sync */
@@ -168,6 +176,12 @@ cs_error_t dfsm_send_update_complete(dfsm_t *dfsm);
gboolean dfsm_lowest_nodeid(dfsm_t *dfsm);
+/* calls dfsm_in_sync_fn again if all members are still in sync, loop thread only */
+void dfsm_check_in_sync(dfsm_t *dfsm);
+
+/* whether a process of that node is a member of the current group, loop thread only */
+gboolean dfsm_nodeid_is_member(dfsm_t *dfsm, uint32_t nodeid);
+
gboolean dfsm_nodeid_is_local(dfsm_t *dfsm, uint32_t nodeid, uint32_t pid);
cs_error_t dfsm_verify_request(dfsm_t *dfsm);
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH cluster v3 04/38] pmxcfs: quorum: pass the corosync membership to an optional listener
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (2 preceding siblings ...)
2026-11-18 4:37 ` [PATCH cluster v3 03/38] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 05/38] pmxcfs: confdb: call an optional listener when the nodes get published Thomas Lamprecht
` (35 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Quorum alone does not tell a service whether every node contributing to
it has participated in the service's synchronization. Pass the node IDs
from each corosync quorum notification to an optional listener, and an
empty list when the connection to corosync is lost.
The listener runs on the loop thread, like the dfsm callbacks, so it
can compare corosync membership with dfsm group membership without
locking. Existing behavior is unchanged when no listener is registered.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* comment: the array is only valid during the call
no changes from v1 to v2
src/pmxcfs/pmxcfs.c | 2 +-
src/pmxcfs/quorum.c | 26 +++++++++++++++++++++-----
src/pmxcfs/quorum.h | 9 ++++++++-
3 files changed, 30 insertions(+), 7 deletions(-)
diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c
index 2bf9145..284bae1 100644
--- a/src/pmxcfs/pmxcfs.c
+++ b/src/pmxcfs/pmxcfs.c
@@ -1019,7 +1019,7 @@ int main(int argc, char *argv[]) {
if (dcdb) {
- service_quorum = service_quorum_new();
+ service_quorum = service_quorum_new(NULL, NULL);
cfs_loop_add_service(corosync_loop, service_quorum, QB_LOOP_HIGH);
diff --git a/src/pmxcfs/quorum.c b/src/pmxcfs/quorum.c
index 0ac3cfe..3b9e012 100644
--- a/src/pmxcfs/quorum.c
+++ b/src/pmxcfs/quorum.c
@@ -35,12 +35,25 @@
#include "cfs-utils.h"
#include "loop.h"
+#include "quorum.h"
#include "status.h"
typedef struct {
quorum_handle_t handle;
+ cfs_quorum_view_fn_t view_fn;
+ gpointer view_data;
} qs_private_t;
+static void quorum_set(
+ qs_private_t *private, uint32_t quorate, gboolean quiet, const uint32_t *nodeids, size_t count
+) {
+ cfs_set_quorate(quorate, quiet);
+
+ if (private->view_fn) {
+ private->view_fn(private->view_data, quorate, nodeids, count);
+ }
+}
+
static void quorum_notification_fn(
quorum_handle_t handle,
uint32_t quorate,
@@ -64,7 +77,7 @@ static void quorum_notification_fn(
return;
}
- cfs_set_quorate(quorate, FALSE);
+ quorum_set(private, quorate, FALSE, view_list, view_list_entries);
}
static quorum_callbacks_t quorum_callbacks = {
@@ -80,7 +93,7 @@ static gboolean service_quorum_finalize(cfs_service_t *service, gpointer context
cs_error_t result;
- cfs_set_quorate(0, TRUE);
+ quorum_set(private, 0, TRUE, NULL, 0);
result = quorum_finalize(handle);
private->handle = 0;
@@ -150,7 +163,7 @@ static int service_quorum_initialize(cfs_service_t *service, gpointer context) {
return quorum_fd;
err_finalize:
- cfs_set_quorate(0, FALSE);
+ quorum_set(private, 0, FALSE, NULL, 0);
quorum_finalize(handle);
err_reset_handle:
private
@@ -185,7 +198,7 @@ loop:
cfs_critical("quorum_dispatch failed: %s", cs_strerror(result));
- cfs_set_quorate(0, FALSE);
+ quorum_set(private, 0, FALSE, NULL, 0);
quorum_finalize(handle);
private->handle = 0;
return FALSE;
@@ -197,7 +210,7 @@ static cfs_service_callbacks_t cfs_quorum_callbacks = {
.cfs_service_dispatch_fn = service_quorum_dispatch,
};
-cfs_service_t *service_quorum_new(void) {
+cfs_service_t *service_quorum_new(cfs_quorum_view_fn_t view_fn, gpointer view_data) {
cfs_service_t *service;
qs_private_t *private = g_new0(qs_private_t, 1);
@@ -205,6 +218,9 @@ cfs_service_t *service_quorum_new(void) {
return NULL;
}
+ private->view_fn = view_fn;
+ private->view_data = view_data;
+
service = cfs_service_new(&cfs_quorum_callbacks, G_LOG_DOMAIN, private);
return service;
diff --git a/src/pmxcfs/quorum.h b/src/pmxcfs/quorum.h
index 3dec79c..06a3246 100644
--- a/src/pmxcfs/quorum.h
+++ b/src/pmxcfs/quorum.h
@@ -25,7 +25,14 @@
#include "loop.h"
-cfs_service_t *service_quorum_new(void);
+/* Called on the loop thread with the node IDs of the current corosync membership each time corosync
+ * reports a change, and with none when the connection to corosync is lost. The array is only valid
+ * during the call. */
+typedef void (*cfs_quorum_view_fn_t)(
+ gpointer data, uint32_t quorate, const uint32_t *nodeids, size_t count
+);
+
+cfs_service_t *service_quorum_new(cfs_quorum_view_fn_t view_fn, gpointer view_data);
void service_quorum_destroy(cfs_service_t *service);
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH cluster v3 05/38] pmxcfs: confdb: call an optional listener when the nodes get published
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (3 preceding siblings ...)
2026-11-18 4:37 ` [PATCH cluster v3 04/38] pmxcfs: quorum: pass the corosync membership to an optional listener Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 06/38] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
` (34 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A service that decides by the configured nodes has to decide again when
they change. Neither the quorum nor the group of a dfsm has to change
with them, for example when a node that is offline gets removed from
the configuration.
Call an optional listener each time the nodes of the corosync
configuration got published. It runs on the loop thread, like the
quorum listener. Existing behavior is unchanged when no listener is
registered.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* new in v3
src/pmxcfs/confdb.c | 18 ++++++++++++++----
src/pmxcfs/confdb.h | 5 ++++-
src/pmxcfs/pmxcfs.c | 2 +-
3 files changed, 19 insertions(+), 6 deletions(-)
diff --git a/src/pmxcfs/confdb.c b/src/pmxcfs/confdb.c
index 43427e0..7632f9f 100644
--- a/src/pmxcfs/confdb.c
+++ b/src/pmxcfs/confdb.c
@@ -38,6 +38,7 @@
#include <corosync/corotypes.h>
#include "cfs-utils.h"
+#include "confdb.h"
#include "loop.h"
#include "status.h"
@@ -46,6 +47,8 @@ typedef struct {
cmap_track_handle_t track_nodelist_handle;
cmap_track_handle_t track_version_handle;
gboolean changes;
+ cfs_confdb_changed_fn_t changed_fn;
+ gpointer changed_data;
} cs_private_t;
static cs_error_t cmap_read_clusternodes(cmap_handle_t handle, cfs_clinfo_t *clinfo) {
@@ -132,7 +135,8 @@ static cs_error_t cmap_read_clusternodes(cmap_handle_t handle, cfs_clinfo_t *cli
return result;
}
-static cs_error_t cmap_read_config(cmap_handle_t handle) {
+static cs_error_t cmap_read_config(cs_private_t *private) {
+ cmap_handle_t handle = private->handle;
cs_error_t result;
uint64_t config_version = 0;
@@ -156,6 +160,9 @@ static cs_error_t cmap_read_config(cmap_handle_t handle) {
result = cmap_read_clusternodes(handle, clinfo);
if (result == CS_OK) {
cfs_status_set_clinfo(clinfo);
+ if (private->changed_fn) {
+ private->changed_fn(private->changed_data);
+ }
} else {
cfs_clinfo_destroy(clinfo);
}
@@ -281,7 +288,7 @@ static int service_cmap_initialize(cfs_service_t *service, gpointer context) {
return -1;
}
- cmap_read_config(handle);
+ cmap_read_config(private);
return cmap_fd;
}
@@ -311,7 +318,7 @@ loop:
if (result == CS_OK || result == CS_ERR_TRY_AGAIN) {
if (private->changes) {
- result = cmap_read_config(handle);
+ result = cmap_read_config(private);
private->changes = FALSE;
@@ -334,7 +341,7 @@ static cfs_service_callbacks_t cfs_confdb_callbacks = {
.cfs_service_dispatch_fn = service_cmap_dispatch,
};
-cfs_service_t *service_confdb_new(void) {
+cfs_service_t *service_confdb_new(cfs_confdb_changed_fn_t changed_fn, gpointer changed_data) {
cfs_service_t *service;
cs_private_t *private = g_new0(cs_private_t, 1);
@@ -342,6 +349,9 @@ cfs_service_t *service_confdb_new(void) {
return NULL;
}
+ private->changed_fn = changed_fn;
+ private->changed_data = changed_data;
+
service = cfs_service_new(&cfs_confdb_callbacks, G_LOG_DOMAIN, private);
return service;
diff --git a/src/pmxcfs/confdb.h b/src/pmxcfs/confdb.h
index 46ec888..2286b34 100644
--- a/src/pmxcfs/confdb.h
+++ b/src/pmxcfs/confdb.h
@@ -25,7 +25,10 @@
#include "loop.h"
-cfs_service_t *service_confdb_new(void);
+/* Called on the loop thread each time the nodes of the corosync configuration got published. */
+typedef void (*cfs_confdb_changed_fn_t)(gpointer data);
+
+cfs_service_t *service_confdb_new(cfs_confdb_changed_fn_t changed_fn, gpointer changed_data);
void service_confdb_destroy(cfs_service_t *service);
diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c
index 284bae1..2ab48dc 100644
--- a/src/pmxcfs/pmxcfs.c
+++ b/src/pmxcfs/pmxcfs.c
@@ -1023,7 +1023,7 @@ int main(int argc, char *argv[]) {
cfs_loop_add_service(corosync_loop, service_quorum, QB_LOOP_HIGH);
- service_confdb = service_confdb_new();
+ service_confdb = service_confdb_new(NULL, NULL);
cfs_loop_add_service(corosync_loop, service_confdb, QB_LOOP_MED);
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH cluster v3 06/38] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (4 preceding siblings ...)
2026-11-18 4:37 ` [PATCH cluster v3 05/38] pmxcfs: confdb: call an optional listener when the nodes get published Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 07/38] cluster: add helper to read the cluster ID Thomas Lamprecht
` (33 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Generate a missing or invalid cluster ID in pmxcfs, including for
upgraded clusters. Only the member with the lowest node ID generates
it, preventing competing IDs within a group.
Quorum alone is insufficient. During a cold boot, corosync can be
quorate before all pmxcfs instances have joined their group. A node
with a stale database could generate an ID before learning the
existing one. At equal database versions, its newer mtime could
then make that ID win the next synchronization.
Require quorum, more than half of the configured nodes in the corosync
membership, and every member of that membership synchronized in the
pmxcfs group. Majorities of the same configured node set overlap;
together with normal database transfer on join, this prevents a stale
minority from generating before synchronizing with the other nodes.
Count nodes rather than votes so this also holds with a QDevice,
two_node, last_man_standing, or lowered expected votes. The tradeoff is
that no ID is generated while half or more of the nodes are down.
This does not prevent ordinary database synchronization from selecting
a database that lacks the ID for other reasons. An ID is generated
again in that case. Log why generation is blocked whenever the reason
changes, naming the nodes whose pmxcfs is missing.
Retry after synchronization, after quorum, membership or configuration
changes, and on the hourly dcdb timer. Generation uses the existing
ordered file messages without waiting on the loop thread. Interrupted
creation can leave an empty file, which a later attempt fills. Reuse the
generated ID until a valid one is observed, so retries before delivery
do not replace it. Leave a directory at the path untouched and log it.
Older nodes can replicate the file without a new message type. If the
lowest member runs an older version, generation waits until that node
is upgraded or leaves.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* check again when the configured nodes get published or change, with
the new confdb listener, instead of waiting for the next quorum or
group change or the hourly timer; test for nodes removed from the
configuration
* comments: quorum is still required next to the node majority, and the
test instances share the per-process state
changes v1 -> v2:
* rebase onto the dfsm rework: the test nodes have to count as joined to
send
src/pmxcfs/Makefile | 10 +-
src/pmxcfs/check_dcdb.c | 563 ++++++++++++++++++++++++++++++++++++++++
src/pmxcfs/dcdb.c | 153 +++++++++++
src/pmxcfs/dcdb.h | 8 +
src/pmxcfs/pmxcfs.c | 4 +-
src/pmxcfs/status.c | 11 +
src/pmxcfs/status.h | 3 +
7 files changed, 747 insertions(+), 5 deletions(-)
create mode 100644 src/pmxcfs/check_dcdb.c
diff --git a/src/pmxcfs/Makefile b/src/pmxcfs/Makefile
index cd93e72..64d6d6f 100644
--- a/src/pmxcfs/Makefile
+++ b/src/pmxcfs/Makefile
@@ -17,7 +17,7 @@ ARFLAGS = crs
.c.o:
$(CC) $(CFLAGS) -c -o $@ $< -MMD -MT $@ -MF $@.d
-all: pmxcfs create_pmxcfs_db logtest check_memdb
+all: pmxcfs create_pmxcfs_db logtest check_memdb check_dcdb
libpmxcfs.a: logger.o loop.o server.o status.o confdb.o quorum.o dcdb.o dfsm.o
libpmxcfs.a: cfs-plug.o cfs-plug-memdb.o cfs-plug-link.o cfs-plug-func.o
@@ -41,6 +41,9 @@ logtest2: logtest2.o libpmxcfs.a
check_memdb: check_memdb.o libpmxcfs.a
$(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) $(shell pkg-config --libs check)
+check_dcdb: check_dcdb.o libpmxcfs.a
+ $(CC) $(CFLAGS) -o $@ $^ $(LDFLAGS) $(shell pkg-config --libs check)
+
pmxcfs.8:
-include /usr/share/pve-doc-generator/pve-doc-generator.mk
@@ -57,11 +60,12 @@ format:
git ls-files ':/src/pmxcfs/*.[ch]' | xargs -n2 -P0 -- clang-format -i
.PHONY: check
-check: check_memdb
+check: check_memdb check_dcdb
./check_memdb
+ ./check_dcdb
.PHONY: clean
clean:
- rm -f *.o *.o.d *.a pmxcfs create_pmxcfs_db check_memdb logtest pmxcfs.8
+ rm -f *.o *.o.d *.a pmxcfs create_pmxcfs_db check_memdb check_dcdb logtest pmxcfs.8
-include *.o.d
diff --git a/src/pmxcfs/check_dcdb.c b/src/pmxcfs/check_dcdb.c
new file mode 100644
index 0000000..81909d3
--- /dev/null
+++ b/src/pmxcfs/check_dcdb.c
@@ -0,0 +1,563 @@
+/*
+ Copyright (C) 2026 Proxmox Server Solutions GmbH
+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as published by
+ the Free Software Foundation, either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+
+ Author: Thomas Lamprecht <t.lamprecht@proxmox.com>
+
+*/
+
+/* Runs several dcdb instances in one process, each with its own memdb, and replaces the CPG
+ * transport with an ordered in-process queue, so that membership changes, syncs and the cluster ID
+ * generation can be tested without corosync. The sources are included to reach the static CPG
+ * callbacks. The quorum view and the pending cluster ID of dcdb.c are per process, so all instances
+ * share them and the tests keep node 1 as the lowest node. */
+
+#include <check.h>
+#include <stdint.h>
+
+#include "dfsm.c"
+#undef G_LOG_DOMAIN
+#include "dcdb.c"
+
+#if CHECK_MAJOR_VERSION == 0 && CHECK_MINOR_VERSION <= 10
+#define _TEST_TYPE TFun
+#else
+#define _TEST_TYPE const TTest *
+#endif
+
+cfs_t cfs = {
+ .debug = 0,
+ .nodename = "testnode",
+};
+
+typedef struct {
+ uint32_t nodeid;
+ uint32_t pid;
+ size_t len;
+ void *data;
+} packet_t;
+
+static GQueue packets = G_QUEUE_INIT;
+
+cs_error_t cpg_context_get(cpg_handle_t handle, void **context) {
+ *context = (void *)(uintptr_t)handle;
+ return CS_OK;
+}
+
+cs_error_t cpg_mcast_joined(
+ cpg_handle_t handle, cpg_guarantee_t guarantee, const struct iovec *iov, unsigned int iov_len
+) {
+ dfsm_t *sender = (dfsm_t *)(uintptr_t)handle;
+ packet_t *p = g_new0(packet_t, 1);
+
+ p->nodeid = sender->nodeid;
+ p->pid = sender->pid;
+
+ for (unsigned int i = 0; i < iov_len; i++) {
+ p->len += iov[i].iov_len;
+ }
+
+ p->data = g_malloc(p->len);
+
+ size_t pos = 0;
+ for (unsigned int i = 0; i < iov_len; i++) {
+ if (iov[i].iov_len) {
+ memcpy((char *)p->data + pos, iov[i].iov_base, iov[i].iov_len);
+ }
+ pos += iov[i].iov_len;
+ }
+
+ g_queue_push_tail(&packets, p);
+
+ return CS_OK;
+}
+
+/* delivers all queued messages, including those sent meanwhile, to the given nodes in order */
+static void pump(dfsm_t **nodes, int count) {
+ int rounds = 0;
+
+ while (!g_queue_is_empty(&packets)) {
+ ck_assert(++rounds < 1000);
+
+ packet_t *p = g_queue_pop_head(&packets);
+ for (int i = 0; i < count; i++) {
+ dfsm_cpg_deliver_callback(
+ nodes[i]->cpg_handle, &nodes[i]->cpg_group_name, p->nodeid, p->pid, p->data, p->len
+ );
+ ck_assert(dfsm_get_mode(nodes[i]) < DFSM_ERROR_MODE_START);
+ }
+
+ g_free(p->data);
+ g_free(p);
+ }
+}
+
+/* the databases of all tests, removed at the end whatever the tests left behind */
+static char *tmpdir;
+
+static memdb_t *open_db(const char *name) {
+ char *filename = g_build_filename(tmpdir, name, NULL);
+ unlink(filename);
+ memdb_t *memdb = memdb_open(filename);
+ g_free(filename);
+ ck_assert(memdb != NULL);
+
+ /* a node in cluster mode always has a corosync.conf */
+ ck_assert(memdb_create(memdb, "corosync.conf", 0, 100) == 0);
+
+ return memdb;
+}
+
+static dfsm_t *start_node(memdb_t *memdb, uint32_t nodeid) {
+ dfsm_t *dfsm = dcdb_new(memdb);
+ ck_assert(dfsm != NULL);
+
+ dfsm->cpg_handle = (uintptr_t)dfsm;
+ dfsm->joined = TRUE; /* messages are only sent while joined */
+ dfsm->nodeid = nodeid;
+ dfsm->pid = 100 + nodeid;
+
+ return dfsm;
+}
+
+static void stop_nodes(dfsm_t **nodes, memdb_t **dbs, int count) {
+ for (int i = 0; i < count; i++) {
+ nodes[i]->cpg_handle = 0;
+ dfsm_destroy(nodes[i]);
+ memdb_close(dbs[i]);
+ }
+}
+
+/* the nodes in the corosync configuration, as the confdb service reads them */
+static void set_config_nodes(uint32_t count) {
+ cfs_clinfo_t *clinfo = cfs_clinfo_new("test", 1);
+
+ for (uint32_t i = 1; i <= count; i++) {
+ char *name = g_strdup_printf("node%u", i);
+ cfs_clinfo_add_node(clinfo, cfs_clnode_new(name, i, 1));
+ g_free(name);
+ }
+
+ cfs_status_set_clinfo(clinfo);
+}
+
+static void pump(dfsm_t **nodes, int count);
+
+static void publish_config_nodes(dfsm_t **nodes, int count, uint32_t config_count) {
+ set_config_nodes(config_count);
+
+ for (int i = 0; i < count; i++) {
+ dcdb_config_changed(nodes[i]);
+ }
+
+ pump(nodes, count);
+}
+
+/* a change of the dcdb group, as seen by its members, without delivering what it causes */
+static void change_group(dfsm_t **nodes, int count) {
+ struct cpg_address members[count];
+
+ for (int i = 0; i < count; i++) {
+ members[i].nodeid = nodes[i]->nodeid;
+ members[i].pid = nodes[i]->pid;
+ members[i].reason = 0;
+ }
+
+ for (int i = 0; i < count; i++) {
+ dfsm_cpg_confchg_callback(
+ nodes[i]->cpg_handle, &nodes[i]->cpg_group_name, members, count, NULL, 0, members, count
+ );
+ }
+}
+
+static void set_group(dfsm_t **nodes, int count) {
+ change_group(nodes, count);
+ pump(nodes, count);
+}
+
+/* a change of the corosync membership, as the quorum service reports it */
+static void
+set_quorum(dfsm_t **nodes, int count, uint32_t quorate, uint32_t *view, size_t view_len) {
+ cfs_set_quorate(quorate, TRUE);
+
+ for (int i = 0; i < count; i++) {
+ dcdb_quorum_view_changed(nodes[i], quorate, view, view_len);
+ }
+
+ pump(nodes, count);
+}
+
+static void read_id(memdb_t *memdb, char id[CLUSTER_ID_FILE_SIZE]) {
+ gpointer data = NULL;
+
+ ck_assert(memdb_read(memdb, "cluster-id", &data) == CLUSTER_ID_FILE_SIZE);
+ ck_assert(cluster_id_is_valid(data, CLUSTER_ID_FILE_SIZE));
+ memcpy(id, data, CLUSTER_ID_FILE_SIZE);
+
+ g_free(data);
+}
+
+static void assert_id(memdb_t *memdb, const char *expected) {
+ char id[CLUSTER_ID_FILE_SIZE];
+
+ read_id(memdb, id);
+ ck_assert(memcmp(id, expected, CLUSTER_ID_FILE_SIZE) == 0);
+}
+
+static void write_id(memdb_t *memdb, const char *id, guint32 writer, guint32 mtime) {
+ ck_assert(memdb_create(memdb, "cluster-id", writer, mtime) == 0);
+ ck_assert(memdb_write(memdb, "cluster-id", writer, mtime, id, strlen(id), 0, 1) == strlen(id));
+}
+
+static void other_writes(memdb_t *memdb, int count) {
+ ck_assert(memdb_create(memdb, "other", 2, 100) == 0);
+ for (int i = 0; i < count; i++) {
+ ck_assert(memdb_write(memdb, "other", 2, 100, "x", 1, 0, 1) == 1);
+ }
+}
+
+START_TEST(test_single_node_cluster) {
+ memdb_t *dbs[] = {open_db("1.db")};
+ dfsm_t *nodes[] = {start_node(dbs[0], 1)};
+ uint32_t view[] = {1};
+ char id[CLUSTER_ID_FILE_SIZE];
+
+ /* without the configured nodes known, nothing happens */
+ set_quorum(nodes, 1, 1, view, 1);
+ set_group(nodes, 1);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_MISSING);
+
+ publish_config_nodes(nodes, 1, 1);
+ read_id(dbs[0], id);
+
+ stop_nodes(nodes, dbs, 1);
+}
+END_TEST
+
+/* removing nodes from the configuration can lift the node majority blocker */
+START_TEST(test_config_nodes_removed) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db")};
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2)};
+ uint32_t view[] = {1, 2};
+ char id[CLUSTER_ID_FILE_SIZE];
+
+ publish_config_nodes(nodes, 2, 4);
+ set_quorum(nodes, 2, 1, view, 2);
+ set_group(nodes, 2);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_MISSING);
+
+ publish_config_nodes(nodes, 2, 2);
+ read_id(dbs[0], id);
+ assert_id(dbs[1], id);
+
+ stop_nodes(nodes, dbs, 2);
+}
+END_TEST
+
+START_TEST(test_only_lowest_generates) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db")};
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2)};
+ uint32_t view[] = {1, 2};
+ char id1[CLUSTER_ID_FILE_SIZE], id2[CLUSTER_ID_FILE_SIZE];
+
+ set_config_nodes(2);
+ set_quorum(nodes, 2, 1, view, 2);
+ set_group(nodes, 2);
+
+ read_id(dbs[0], id1);
+ read_id(dbs[1], id2);
+ ck_assert(memcmp(id1, id2, CLUSTER_ID_FILE_SIZE) == 0);
+
+ /* written by node 1, the lowest */
+ memdb_tree_entry_t *te = memdb_getattr(dbs[1], "cluster-id");
+ ck_assert(te != NULL);
+ ck_assert(te->writer == 1);
+ memdb_tree_entry_free(te);
+
+ stop_nodes(nodes, dbs, 2);
+}
+END_TEST
+
+/* A node with an outdated database starts first and is alone in the group, while corosync, with
+ * the other node running, makes it quorate. It must not publish an ID of its own. */
+START_TEST(test_stale_node_alone) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db")};
+ write_id(dbs[1], "abcdefghijklmnopqrst\n", 2, 100);
+ other_writes(dbs[1], 10);
+
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2)};
+ uint32_t view[] = {1, 2};
+
+ set_config_nodes(2);
+ set_quorum(nodes, 1, 1, view, 2);
+ set_group(nodes, 1);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_MISSING);
+
+ set_group(nodes, 2);
+ assert_id(dbs[0], "abcdefghijklmnopqrst\n");
+ assert_id(dbs[1], "abcdefghijklmnopqrst\n");
+
+ stop_nodes(nodes, dbs, 2);
+}
+END_TEST
+
+/* Like above, but the outdated node holds the empty file of an interrupted creation. Filling it
+ * would make its database the newest one, so its ID would replace the established one. */
+START_TEST(test_stale_node_alone_with_empty_file) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db")};
+ ck_assert(memdb_create(dbs[0], "cluster-id", 1, 100) == 0);
+ write_id(dbs[1], "abcdefghijklmnopqrst\n", 1, 100);
+
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2)};
+ uint32_t view[] = {1, 2};
+
+ set_config_nodes(2);
+ set_quorum(nodes, 1, 1, view, 2);
+ set_group(nodes, 1);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_INVALID);
+
+ set_group(nodes, 2);
+ assert_id(dbs[0], "abcdefghijklmnopqrst\n");
+ assert_id(dbs[1], "abcdefghijklmnopqrst\n");
+
+ stop_nodes(nodes, dbs, 2);
+}
+END_TEST
+
+/* Quorate through a QDevice, or two_node, last_man_standing or lowered expected votes, the
+ * corosync membership of a lone outdated node is just itself. */
+START_TEST(test_lone_node_without_node_majority) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db")};
+ write_id(dbs[1], "abcdefghijklmnopqrst\n", 2, 100);
+
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2)};
+ uint32_t view1[] = {1}, view12[] = {1, 2};
+
+ set_config_nodes(2);
+ set_quorum(nodes, 1, 1, view1, 1);
+ set_group(nodes, 1);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_MISSING);
+
+ set_quorum(nodes, 2, 1, view12, 2);
+ set_group(nodes, 2);
+ assert_id(dbs[0], "abcdefghijklmnopqrst\n");
+ assert_id(dbs[1], "abcdefghijklmnopqrst\n");
+
+ stop_nodes(nodes, dbs, 2);
+}
+END_TEST
+
+START_TEST(test_half_of_the_nodes) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db")};
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2)};
+ uint32_t view[] = {1, 2};
+
+ set_config_nodes(4);
+ set_quorum(nodes, 2, 1, view, 2);
+ set_group(nodes, 2);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_MISSING);
+ ck_assert(cluster_id_state(dbs[1]) == CLUSTER_ID_MISSING);
+
+ stop_nodes(nodes, dbs, 2);
+}
+END_TEST
+
+/* Neither gaining quorum nor a corosync membership change causes a sync of the group, so the
+ * generation must not wait for the next one. */
+START_TEST(test_quorum_change_triggers) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db")};
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2)};
+ uint32_t view12[] = {1, 2}, view123[] = {1, 2, 3};
+ char id[CLUSTER_ID_FILE_SIZE];
+
+ set_config_nodes(3);
+ set_quorum(nodes, 2, 0, view12, 2);
+ set_group(nodes, 2);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_MISSING);
+
+ /* node 3 only runs corosync */
+ set_quorum(nodes, 2, 1, view123, 3);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_MISSING);
+
+ /* node 3 left corosync again, the group did not change */
+ set_quorum(nodes, 2, 1, view12, 2);
+ read_id(dbs[0], id);
+ assert_id(dbs[1], id);
+
+ stop_nodes(nodes, dbs, 2);
+}
+END_TEST
+
+/* node 3 joins corosync and the group, in both orders the notifications can arrive */
+static void third_node_joins(gboolean quorum_first) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db"), open_db("3.db")};
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2), start_node(dbs[2], 3)};
+ uint32_t view12[] = {1, 2}, view123[] = {1, 2, 3};
+ char id[CLUSTER_ID_FILE_SIZE];
+
+ set_config_nodes(5);
+ set_quorum(nodes, 2, 0, view12, 2);
+ set_group(nodes, 2);
+
+ if (quorum_first) {
+ set_quorum(nodes, 3, 1, view123, 3);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_MISSING);
+ set_group(nodes, 3);
+ } else {
+ set_group(nodes, 3);
+ ck_assert(cluster_id_state(dbs[0]) == CLUSTER_ID_MISSING);
+ set_quorum(nodes, 3, 1, view123, 3);
+ }
+
+ read_id(dbs[0], id);
+ assert_id(dbs[1], id);
+ assert_id(dbs[2], id);
+
+ stop_nodes(nodes, dbs, 3);
+}
+
+START_TEST(test_third_node_joins_quorum_first) { third_node_joins(TRUE); }
+END_TEST
+
+START_TEST(test_third_node_joins_group_first) { third_node_joins(FALSE); }
+END_TEST
+
+/* attempts made before the messages of the first one got delivered must write the same ID */
+START_TEST(test_pending_id_reused) {
+ memdb_t *dbs[] = {open_db("1.db")};
+ dfsm_t *nodes[] = {start_node(dbs[0], 1)};
+ uint32_t view[] = {1};
+ char first[CLUSTER_ID_FILE_SIZE];
+
+ set_config_nodes(1);
+ set_quorum(nodes, 1, 1, view, 1);
+
+ change_group(nodes, 1);
+ ck_assert(g_queue_get_length(&packets) == 2);
+ ck_assert(have_pending_cluster_id);
+ memcpy(first, pending_cluster_id, CLUSTER_ID_FILE_SIZE);
+
+ dfsm_check_in_sync(nodes[0]);
+ dcdb_quorum_view_changed(nodes[0], 1, view, 1);
+ ck_assert(g_queue_get_length(&packets) == 6);
+
+ pump(nodes, 1);
+ assert_id(dbs[0], first);
+
+ /* once delivered, the next check finds it valid and forgets the pending one */
+ dfsm_check_in_sync(nodes[0]);
+ ck_assert(g_queue_is_empty(&packets));
+ ck_assert(!have_pending_cluster_id);
+
+ stop_nodes(nodes, dbs, 1);
+}
+END_TEST
+
+START_TEST(test_lowest_repairs_empty_file) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db")};
+ ck_assert(memdb_create(dbs[0], "cluster-id", 3, 100) == 0);
+ ck_assert(memdb_create(dbs[1], "cluster-id", 3, 100) == 0);
+
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2)};
+ uint32_t view[] = {1, 2};
+ char id[CLUSTER_ID_FILE_SIZE];
+
+ set_config_nodes(2);
+ set_quorum(nodes, 2, 1, view, 2);
+ set_group(nodes, 2);
+
+ read_id(dbs[0], id);
+ assert_id(dbs[1], id);
+
+ stop_nodes(nodes, dbs, 2);
+}
+END_TEST
+
+/* an ID with only 16 characters is malformed like anything else */
+START_TEST(test_lowest_replaces_short_id) {
+ memdb_t *dbs[] = {open_db("1.db"), open_db("2.db")};
+ write_id(dbs[0], "abcdefghijklmnop\n", 1, 100);
+ write_id(dbs[1], "abcdefghijklmnop\n", 1, 100);
+
+ dfsm_t *nodes[] = {start_node(dbs[0], 1), start_node(dbs[1], 2)};
+ uint32_t view[] = {1, 2};
+ char id[CLUSTER_ID_FILE_SIZE];
+
+ set_config_nodes(2);
+ set_quorum(nodes, 2, 1, view, 2);
+ set_group(nodes, 2);
+
+ read_id(dbs[0], id);
+ assert_id(dbs[1], id);
+
+ stop_nodes(nodes, dbs, 2);
+}
+END_TEST
+
+static void add_test(Suite *s, _TEST_TYPE tf, const char *name) {
+ TCase *tc = tcase_create(name);
+ tcase_add_test(tc, tf);
+ suite_add_tcase(s, tc);
+}
+
+static void remove_tmpdir(void) {
+ GDir *dir = g_dir_open(tmpdir, 0, NULL);
+ if (dir) {
+ const char *name;
+ while ((name = g_dir_read_name(dir))) {
+ char *filename = g_build_filename(tmpdir, name, NULL);
+ unlink(filename);
+ g_free(filename);
+ }
+ g_dir_close(dir);
+ }
+
+ rmdir(tmpdir);
+ g_free(tmpdir);
+}
+
+int main(void) {
+ cfs_status_init();
+
+ if (!(tmpdir = g_dir_make_tmp("check_dcdb-XXXXXX", NULL))) {
+ fprintf(stderr, "unable to create a temporary directory\n");
+ return EXIT_FAILURE;
+ }
+
+ Suite *s = suite_create("dcdb");
+
+ add_test(s, test_single_node_cluster, "single_node_cluster");
+ add_test(s, test_config_nodes_removed, "config_nodes_removed");
+ add_test(s, test_only_lowest_generates, "only_lowest_generates");
+ add_test(s, test_stale_node_alone, "stale_node_alone");
+ add_test(s, test_stale_node_alone_with_empty_file, "stale_node_alone_with_empty_file");
+ add_test(s, test_lone_node_without_node_majority, "lone_node_without_node_majority");
+ add_test(s, test_half_of_the_nodes, "half_of_the_nodes");
+ add_test(s, test_quorum_change_triggers, "quorum_change_triggers");
+ add_test(s, test_third_node_joins_quorum_first, "third_node_joins_quorum_first");
+ add_test(s, test_third_node_joins_group_first, "third_node_joins_group_first");
+ add_test(s, test_pending_id_reused, "pending_id_reused");
+ add_test(s, test_lowest_repairs_empty_file, "lowest_repairs_empty_file");
+ add_test(s, test_lowest_replaces_short_id, "lowest_replaces_short_id");
+
+ SRunner *sr = srunner_create(s);
+ srunner_run_all(sr, CK_NORMAL);
+ int number_failed = srunner_ntests_failed(sr);
+ srunner_free(sr);
+
+ remove_tmpdir();
+
+ return (number_failed == 0) ? EXIT_SUCCESS : EXIT_FAILURE;
+}
diff --git a/src/pmxcfs/dcdb.c b/src/pmxcfs/dcdb.c
index 0965331..7a3693b 100644
--- a/src/pmxcfs/dcdb.c
+++ b/src/pmxcfs/dcdb.c
@@ -38,6 +38,7 @@
#include <unistd.h>
#include "cfs-utils.h"
+#include "cluster-id.h"
#include "dcdb.h"
#include "loop.h"
#include "status.h"
@@ -985,6 +986,157 @@ leave:
goto ret;
}
+/* node IDs of the current corosync membership while quorate, loop thread only */
+static uint32_t *quorum_view;
+static size_t quorum_view_len;
+
+/* Kept until the file holds a valid ID, so that an attempt made before the messages of an earlier
+ * one got delivered writes the same ID again instead of replacing it. Loop thread only. */
+static char pending_cluster_id[CLUSTER_ID_FILE_SIZE];
+static gboolean have_pending_cluster_id;
+
+/* why the last check did not generate, to log only changes, loop thread only */
+static char *blocked_reason;
+
+static void set_blocked_reason(char *reason) {
+ if (reason && g_strcmp0(reason, blocked_reason) != 0) {
+ cfs_message("not generating a cluster ID yet, %s", reason);
+ }
+
+ g_free(blocked_reason);
+ blocked_reason = reason;
+}
+
+/* An ID is only generated while more than half of the configured nodes are in the corosync
+ * membership and all of those are in sync in this group. Majorities of the same configured node
+ * set overlap. Together with normal database transfer on join, this prevents a stale minority from
+ * generating an ID before it synchronizes with the other nodes. Counting nodes rather than relying
+ * on quorum alone keeps this with a QDevice, two_node, last_man_standing or lowered expected votes,
+ * at the price of not generating while half of the nodes or more are down. Returns why generating
+ * is blocked. */
+static char *generation_blocker(dfsm_t *dfsm) {
+ if (!quorum_view_len || !cfs_is_quorate()) {
+ return g_strdup("not quorate");
+ }
+
+ guint nodes = cfs_clinfo_node_count();
+ if (!nodes) {
+ return g_strdup("the configured nodes are not known");
+ }
+
+ if (quorum_view_len * 2 <= nodes) {
+ return g_strdup_printf(
+ "only %zu of %u configured nodes are in the corosync membership", quorum_view_len, nodes
+ );
+ }
+
+ GString *missing = NULL;
+ for (size_t i = 0; i < quorum_view_len; i++) {
+ if (!dfsm_nodeid_is_member(dfsm, quorum_view[i])) {
+ if (!missing) {
+ missing = g_string_new(NULL);
+ } else {
+ g_string_append(missing, ", ");
+ }
+ g_string_append_printf(missing, "%u", quorum_view[i]);
+ }
+ }
+
+ if (missing) {
+ char *reason = g_strdup_printf(
+ "node ID(s) %s of the corosync membership are not in the pmxcfs group", missing->str
+ );
+ g_string_free(missing, TRUE);
+ return reason;
+ }
+
+ return NULL;
+}
+
+/* Only the node with the lowest ID generates the cluster ID, like only that node expires locks, so
+ * that no two nodes write different IDs. The loop thread cannot wait for the result of its own
+ * messages, so CREATE and WRITE go out back to back, the messages of one sender stay in order. */
+static void dcdb_in_sync(dfsm_t *dfsm, gpointer data) {
+ g_return_if_fail(dfsm != NULL);
+ g_return_if_fail(data != NULL);
+
+ memdb_t *memdb = (memdb_t *)data;
+
+ if (!dfsm_lowest_nodeid(dfsm)) {
+ return;
+ }
+
+ cluster_id_state_t state = cluster_id_state(memdb);
+ if (state == CLUSTER_ID_VALID) {
+ have_pending_cluster_id = FALSE;
+ set_blocked_reason(NULL);
+ return;
+ }
+
+ if (state == CLUSTER_ID_NOT_A_FILE) {
+ set_blocked_reason(g_strdup_printf("'%s' is not a regular file", CFS_CLUSTER_ID_FN));
+ return;
+ }
+
+ char *reason = generation_blocker(dfsm);
+ set_blocked_reason(reason);
+ if (reason) {
+ return;
+ }
+
+ if (!have_pending_cluster_id) {
+ if (!cluster_id_generate(pending_cluster_id)) {
+ return;
+ }
+ have_pending_cluster_id = TRUE;
+ }
+
+ if (state == CLUSTER_ID_MISSING &&
+ dcdb_send_fuse_message_full(
+ dfsm, DCDB_MESSAGE_CFS_CREATE, CFS_CLUSTER_ID_FN, NULL, NULL, 0, 0, 0, NULL
+ ) != CS_OK) {
+ return;
+ }
+
+ if (dcdb_send_fuse_message_full(
+ dfsm, DCDB_MESSAGE_CFS_WRITE, CFS_CLUSTER_ID_FN, NULL, pending_cluster_id,
+ CLUSTER_ID_FILE_SIZE, 0, 1, NULL
+ ) != CS_OK) {
+ return;
+ }
+
+ cfs_message("generating cluster ID %.*s", CLUSTER_ID_LEN, pending_cluster_id);
+}
+
+void dcdb_quorum_view_changed(
+ gpointer data, uint32_t quorate, const uint32_t *nodeids, size_t count
+) {
+ dfsm_t *dfsm = (dfsm_t *)data;
+
+ g_return_if_fail(dfsm != NULL);
+
+ g_free(quorum_view);
+ quorum_view = NULL;
+ quorum_view_len = 0;
+
+ if (quorate && count) {
+ quorum_view = g_memdup2(nodeids, count * sizeof(*nodeids));
+ quorum_view_len = count;
+ }
+
+ /* neither gaining quorum nor a node that only joins corosync causes a sync of the group */
+ dfsm_check_in_sync(dfsm);
+}
+
+void dcdb_config_changed(gpointer data) {
+ dfsm_t *dfsm = (dfsm_t *)data;
+
+ g_return_if_fail(dfsm != NULL);
+
+ /* the configured nodes can become known or fewer without any change of the group */
+ dfsm_check_in_sync(dfsm);
+}
+
static dfsm_callbacks_t dcdb_dfsm_callbacks = {
.dfsm_deliver_fn = dcdb_deliver,
.dfsm_get_state_fn = dcdb_get_state,
@@ -993,6 +1145,7 @@ static dfsm_callbacks_t dcdb_dfsm_callbacks = {
.dfsm_commit_fn = dcdb_commit,
.dfsm_cleanup_fn = dcdb_cleanup,
.dfsm_checksum_fn = dcdb_checksum,
+ .dfsm_in_sync_fn = dcdb_in_sync,
};
dfsm_t *dcdb_new(memdb_t *memdb) {
diff --git a/src/pmxcfs/dcdb.h b/src/pmxcfs/dcdb.h
index d637cc7..de56fc2 100644
--- a/src/pmxcfs/dcdb.h
+++ b/src/pmxcfs/dcdb.h
@@ -61,4 +61,12 @@ int dcdb_send_fuse_message(
void dcdb_send_unlock(dfsm_t *dfsm, const char *path, const guchar csum[32], gboolean request);
+/* quorum view listener, data is the dcdb dfsm */
+void dcdb_quorum_view_changed(
+ gpointer data, uint32_t quorate, const uint32_t *nodeids, size_t count
+);
+
+/* confdb listener, data is the dcdb dfsm */
+void dcdb_config_changed(gpointer data);
+
#endif /* _PVE_DCDB_H_ */
diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c
index 2ab48dc..ce9d650 100644
--- a/src/pmxcfs/pmxcfs.c
+++ b/src/pmxcfs/pmxcfs.c
@@ -1019,11 +1019,11 @@ int main(int argc, char *argv[]) {
if (dcdb) {
- service_quorum = service_quorum_new(NULL, NULL);
+ service_quorum = service_quorum_new(dcdb_quorum_view_changed, dcdb);
cfs_loop_add_service(corosync_loop, service_quorum, QB_LOOP_HIGH);
- service_confdb = service_confdb_new(NULL, NULL);
+ service_confdb = service_confdb_new(dcdb_config_changed, dcdb);
cfs_loop_add_service(corosync_loop, service_confdb, QB_LOOP_MED);
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index 8538d8b..f16dc20 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -2000,6 +2000,17 @@ gboolean cfs_is_quorate(void) {
return res;
}
+guint cfs_clinfo_node_count(void) {
+ g_mutex_lock(&mutex);
+ guint count = 0;
+ if (cfs_status.clinfo && cfs_status.clinfo->nodes_byid) {
+ count = g_hash_table_size(cfs_status.clinfo->nodes_byid);
+ }
+ g_mutex_unlock(&mutex);
+
+ return count;
+}
+
void cfs_set_quorate(uint32_t quorate, gboolean quiet) {
g_mutex_lock(&mutex);
diff --git a/src/pmxcfs/status.h b/src/pmxcfs/status.h
index 709ef64..dfc134f 100644
--- a/src/pmxcfs/status.h
+++ b/src/pmxcfs/status.h
@@ -69,6 +69,9 @@ void cfs_set_quorate(uint32_t quorate, gboolean quiet);
gboolean cfs_is_quorate(void);
+/* number of nodes in the corosync configuration, 0 while not known yet */
+guint cfs_clinfo_node_count(void);
+
GHashTable *vmlist_hash_new(void);
gboolean vmlist_hash_insert_vm(
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH cluster v3 07/38] cluster: add helper to read the cluster ID
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (5 preceding siblings ...)
2026-11-18 4:37 ` [PATCH cluster v3 06/38] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 08/38] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
` (32 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Consumers such as the API should not need to know the cluster ID file
format. Add get_cluster_id() with the same validation rules as pmxcfs.
It returns undef for missing, empty, or invalid content, treating those
cases as "no ID yet".
Register the file for version tracking so cfs_read_file() refreshes its
cached value when the file changes. Do not provide a writer: pmxcfs
owns the file.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* rebase only (test Makefile)
changes v1 -> v2:
* use v5.36 in the new test script
src/PVE/Cluster.pm | 16 +++++++++++++
src/pmxcfs/status.c | 1 +
src/test/Makefile | 6 ++++-
src/test/test_cluster_id.pl | 46 +++++++++++++++++++++++++++++++++++++
4 files changed, 68 insertions(+), 1 deletion(-)
create mode 100644 src/test/test_cluster_id.pl
diff --git a/src/PVE/Cluster.pm b/src/PVE/Cluster.pm
index 6ace846..076e49f 100644
--- a/src/PVE/Cluster.pm
+++ b/src/PVE/Cluster.pm
@@ -54,6 +54,7 @@ my $observed = {
'replication.cfg' => 1,
'corosync.conf' => 1,
'corosync.conf.new' => 1,
+ 'cluster-id' => 1,
'firewall/cluster.fw' => 1,
'user.cfg' => 1,
'domains.cfg' => 1,
@@ -604,6 +605,21 @@ sub cfs_write_file {
PVE::Tools::file_set_contents($fsname, $raw, undef, $force_utf8);
}
+my $parse_cluster_id = sub {
+ my ($filename, $raw) = @_;
+
+ return undef if !defined($raw);
+ return $raw =~ m/^([a-z2-7]{20})\n?\z/ ? $1 : undef;
+};
+
+cfs_register_file('cluster-id', $parse_cluster_id);
+
+# Returns the ID of the cluster, or of the node if it is standalone, undef if there is none yet. IPC
+# errors other than a missing file are raised, like for other pmxcfs reads.
+sub get_cluster_id {
+ return cfs_read_file('cluster-id');
+}
+
my $cfs_lock = sub {
my ($lockid, $timeout, $code, @param) = @_;
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index f16dc20..bf69524 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -80,6 +80,7 @@ typedef struct {
static memdb_change_t memdb_change_array[] = {
{.path = "corosync.conf"},
{.path = "corosync.conf.new"},
+ {.path = "cluster-id"},
{.path = "storage.cfg"},
{.path = "user.cfg"},
{.path = "domains.cfg"},
diff --git a/src/test/Makefile b/src/test/Makefile
index 3a47b8c..9b65883 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -4,7 +4,7 @@ cpgtest: cpgtest.c
gcc -Wall cpgtest.c $(shell pkg-config --cflags --libs libcpg libqb) -o cpgtest
.PHONY: check install clean distclean
-check: corosync-parser-test test-mac-prefix test-token-policy
+check: corosync-parser-test test-mac-prefix test-token-policy test-cluster-id
.PHONY: corosync-parser-test
corosync-parser-test:
@@ -18,5 +18,9 @@ test-mac-prefix:
test-token-policy:
perl test_token_policy.pl
+.PHONY: test-cluster-id
+test-cluster-id:
+ perl test_cluster_id.pl
+
distclean: clean
clean:
diff --git a/src/test/test_cluster_id.pl b/src/test/test_cluster_id.pl
new file mode 100644
index 0000000..33102e0
--- /dev/null
+++ b/src/test/test_cluster_id.pl
@@ -0,0 +1,46 @@
+use v5.36;
+
+use Test::MockModule;
+use Test::More;
+
+use lib ('.', '..');
+
+use PVE::Cluster;
+
+my $raw;
+my $cluster_module = Test::MockModule->new('PVE::Cluster');
+$cluster_module->mock(
+ get_config => sub {
+ my ($path) = @_;
+ die "unexpected path '$path'\n" if $path ne 'cluster-id';
+ return $raw;
+ },
+);
+
+my $cases = [
+ ["abcdefghijklmnopqrst\n", 'abcdefghijklmnopqrst', 'valid ID'],
+ ["uvwxyz234567abcdefgh\n", 'uvwxyz234567abcdefgh', 'valid ID with digits'],
+ ['abcdefghijklmnopqrst', 'abcdefghijklmnopqrst', 'valid ID without newline'],
+ [undef, undef, 'missing file'],
+ ['', undef, 'empty file'],
+ ["\n", undef, 'only a newline'],
+ ["abcdefghijklmnopqrst\n\n", undef, 'trailing empty line'],
+ ["abcdefghijklmnopqrst \n", undef, 'trailing space'],
+ [" abcdefghijklmnopqrst\n", undef, 'leading space'],
+ ["abcdefghijklmnopqrs\n", undef, 'too short'],
+ ["abcdefghijklmnop\n", undef, '16 characters'],
+ ["abcdefghijklmnopqrstu\n", undef, 'too long'],
+ ['abcdefghijklmnopqrstu', undef, 'too long without newline'],
+ ["ABCDEFGHIJKLMNOPQRST\n", undef, 'uppercase'],
+ ["abcdefghijklmnopqr01\n", undef, 'digits outside the base32 alphabet'],
+ ["abcdefghijklmnopqr89\n", undef, 'more digits outside the base32 alphabet'],
+ ["abcdefgh\njklmnopqrst\n", undef, 'embedded newline'],
+];
+
+for my $case ($cases->@*) {
+ my ($content, $expected, $name) = $case->@*;
+ $raw = $content;
+ is(PVE::Cluster::get_cluster_id(), $expected, $name);
+}
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH cluster v3 08/38] pmxcfs: allow removing the cluster ID file in forced local mode
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (6 preceding siblings ...)
2026-11-18 4:37 ` [PATCH cluster v3 07/38] cluster: add helper to read the cluster ID Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 09/38] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
` (31 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A node that gets separated from its cluster without reinstalling, or a
cloned installation, keeps the cluster ID it had. With the file
protected against changes from user space, only editing the database by
hand would get it a new one.
Allow unlinking the file while pmxcfs runs in forced local mode (-l),
which the documented procedure for separating a node already uses to
remove the corosync.conf. The next start without a corosync.conf then
generates a new ID, like on any standalone node. Moving the file away
is allowed then too, as libfuse unlinks a file that is still open by
renaming it to a hidden name. Outside of forced local mode both stay
rejected, and nothing can be moved onto the path in any mode.
An unlink done in local mode on a node that later rejoins its cluster
replicates like any other change made in local mode: if the database
of that node wins the sync, the file is gone on all nodes, and the
cluster generates a new ID.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
no changes since v2
changes v1 -> v2:
* rebase only
src/pmxcfs/cfs-plug-memdb.c | 8 ++++++--
src/pmxcfs/cfs-utils.h | 1 +
src/pmxcfs/check_memdb.c | 32 ++++++++++++++++++++++++++++++++
src/pmxcfs/pmxcfs.c | 2 ++
4 files changed, 41 insertions(+), 2 deletions(-)
diff --git a/src/pmxcfs/cfs-plug-memdb.c b/src/pmxcfs/cfs-plug-memdb.c
index 0d0fa64..217585e 100644
--- a/src/pmxcfs/cfs-plug-memdb.c
+++ b/src/pmxcfs/cfs-plug-memdb.c
@@ -387,7 +387,9 @@ static int cfs_plug_memdb_rename(cfs_plug_t *plug, const char *from, const char
g_return_val_if_fail(from != NULL, PARAM_CHECK_ERRNO);
g_return_val_if_fail(to != NULL, PARAM_CHECK_ERRNO);
- if (path_is_cluster_id(from) || path_is_cluster_id(to)) {
+ /* moving it away is fine in forced local mode, like unlinking it, which libfuse does that way
+ * while the file is still open */
+ if ((path_is_cluster_id(from) && !cfs.force_local_mode) || path_is_cluster_id(to)) {
return -EACCES;
}
@@ -410,7 +412,9 @@ static int cfs_plug_memdb_unlink(cfs_plug_t *plug, const char *path) {
g_return_val_if_fail(plug->ops == &cfs_ops, PARAM_CHECK_ERRNO);
g_return_val_if_fail(path != NULL, PARAM_CHECK_ERRNO);
- if (path_is_cluster_id(path)) {
+ /* the way to give a node that got separated from its cluster an ID of its own, generated by its
+ * next start without a corosync.conf */
+ if (path_is_cluster_id(path) && !cfs.force_local_mode) {
return -EACCES;
}
diff --git a/src/pmxcfs/cfs-utils.h b/src/pmxcfs/cfs-utils.h
index f66e944..5016720 100644
--- a/src/pmxcfs/cfs-utils.h
+++ b/src/pmxcfs/cfs-utils.h
@@ -44,6 +44,7 @@ typedef struct {
char *ip;
gid_t gid;
int debug;
+ gboolean force_local_mode;
} cfs_t;
extern cfs_t cfs;
diff --git a/src/pmxcfs/check_memdb.c b/src/pmxcfs/check_memdb.c
index db85cbd..86b78a4 100644
--- a/src/pmxcfs/check_memdb.c
+++ b/src/pmxcfs/check_memdb.c
@@ -457,6 +457,36 @@ START_TEST(test_cluster_id_standalone_write_failure) {
}
END_TEST
+START_TEST(test_cluster_id_unlink_forced_local_mode) {
+ cfs_plug_t *plug = (cfs_plug_t *)cfs_plug_memdb_new("memdb", memdb, NULL);
+ ck_assert(plug != NULL);
+
+ ck_assert(memdb_create(memdb, "cluster-id", 0, 1234) == 0);
+ ck_assert(memdb_write(memdb, "cluster-id", 0, 1234, "abcdefghijklmnopqrst\n", 21, 0, 1) == 21);
+
+ ck_assert(memdb_create(memdb, "other", 0, 1234) == 0);
+
+ ck_assert(plug->ops->unlink(plug, "cluster-id") == -EACCES);
+ ck_assert(plug->ops->rename(plug, "cluster-id", ".fuse_hidden01") == -EACCES);
+
+ cfs.force_local_mode = TRUE;
+ ck_assert(plug->ops->truncate(plug, "cluster-id", 0) == -EACCES);
+ ck_assert(plug->ops->rename(plug, "other", "cluster-id") == -EACCES);
+
+ /* what libfuse does on unlink while the file is open */
+ ck_assert(plug->ops->rename(plug, "cluster-id", ".fuse_hidden01") == 0);
+ ck_assert(memdb_getattr(memdb, "cluster-id") == NULL);
+
+ ck_assert(memdb_create(memdb, "cluster-id", 0, 1234) == 0);
+ ck_assert(plug->ops->unlink(plug, "cluster-id") == 0);
+ ck_assert(memdb_getattr(memdb, "cluster-id") == NULL);
+
+ cfs.force_local_mode = FALSE;
+
+ plug->destroy_plug(plug);
+}
+END_TEST
+
START_TEST(test_truncate_grow) {
const char *fn = "/f1";
time_t ctime = 1234;
@@ -745,6 +775,8 @@ static Suite *memdb_suite(void) {
add_test(s, test_cluster_id_fuse_read_only, "cluster_id_fuse_read_only");
+ add_test(s, test_cluster_id_unlink_forced_local_mode, "cluster_id_unlink_forced_local_mode");
+
add_test(s, test_cluster_id_is_valid, "cluster_id_is_valid");
add_test(s, test_cluster_id_generate, "cluster_id_generate");
diff --git a/src/pmxcfs/pmxcfs.c b/src/pmxcfs/pmxcfs.c
index ce9d650..d5a1a23 100644
--- a/src/pmxcfs/pmxcfs.c
+++ b/src/pmxcfs/pmxcfs.c
@@ -812,6 +812,8 @@ int main(int argc, char *argv[]) {
update_qb_log_settings();
}
+ cfs.force_local_mode = force_local_mode;
+
struct utsname utsname;
if (uname(&utsname) != 0) {
cfs_critical("Unable to read local node name");
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH guest-common v3 09/38] guest identity: add helpers for the guest uid and registration time
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (7 preceding siblings ...)
2026-11-18 4:37 ` [PATCH cluster v3 08/38] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 10/38] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
` (30 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A VMID can be reused, and a guest can get a different VMID or move
to another cluster after a restore or remote migration. External tools
such as backup products need to recognize the same logical guest
independently of its VMID, as described in bug #7846.
Provide shared helpers for a random uid of 20 characters from the
lowercase base32 alphabet, which is 100 random bits, and a registration
time in milliseconds. Both are members of the guest config's 'meta'
property. Supply format definitions for both guest types and extract or
merge the identity without requiring knowledge of other members,
including newer ones. Also read it from a raw config, as the parser of a
guest type can drop a 'meta' property with members it does not know.
100 bits make a duplicate practically impossible, and the uid still fits
into a 128-bit value with 28 bits to spare, for example for a format
version and extra data, should a later format need them. Base32 keeps it
at 20 characters, about half the width of a UUID in its usual notation,
and only uses lowercase letters and digits. Each character comes from a
random byte of its own, so no encoder is needed.
The registration time is the time a guest got its uid in this cluster.
Unlike the creation time, which a clone, a restore and a migration from
another cluster copy along with the guest, it changes when a guest is
registered again, for example by a restore to a free VMID. Tools can use
it to tell such a guest from the one they knew, also if both have the
same uid.
Also provide a minimum node-version check for migration targets. Older
versions drop the identity on config writes. Report the required and
available versions so callers can explain a refusal, with a shared
schema for API results; missing version information counts as too old.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* shorter description of the uid, which the status and resource APIs
show too
* reword the comment about the size and the commit message about rtime:
an in-place restore keeps the rtime
changes v1 -> v2:
* use v5.36 and signatures
* take each character of the uid from a random byte of its own, without
an encoder (Dominik)
* add a helper to read the identity from a raw config, shared by the
config write and the VM restore, which picks the 'meta' line like
pmxcfs does
* add the schema of the migration precondition result, shared by both
guest types
* explain the format, the difference between rtime and ctime, and why
the generic property string helpers are not used (Dominik)
* shorter description of the uid
Note: MIN_PVE_MANAGER_VERSION (9.2.31) is a stand-in. It has to become
the first pve-manager version that depends on the qemu-server and
pve-container versions with this series, see the cover letter.
src/Makefile | 1 +
src/PVE/GuestIdentity.pm | 231 ++++++++++++++++++++
src/tests/Makefile | 7 +-
src/tests/guest-identity-tests.pl | 338 ++++++++++++++++++++++++++++++
4 files changed, 575 insertions(+), 2 deletions(-)
create mode 100644 src/PVE/GuestIdentity.pm
create mode 100755 src/tests/guest-identity-tests.pl
diff --git a/src/Makefile b/src/Makefile
index 161dbe5..af4c277 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -10,6 +10,7 @@ install: PVE
install -m 0644 PVE/AbstractMigrate.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/GuestHelpers.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/GuestID.pm ${PERL5DIR}/PVE/
+ install -m 0644 PVE/GuestIdentity.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/Replication.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/ReplicationConfig.pm ${PERL5DIR}/PVE/
install -m 0644 PVE/ReplicationState.pm ${PERL5DIR}/PVE/
diff --git a/src/PVE/GuestIdentity.pm b/src/PVE/GuestIdentity.pm
new file mode 100644
index 0000000..27dbf8b
--- /dev/null
+++ b/src/PVE/GuestIdentity.pm
@@ -0,0 +1,231 @@
+package PVE::GuestIdentity;
+
+use v5.36;
+
+use Time::HiRes;
+
+use PVE::Cluster;
+
+# The identity of a guest, VM or container alike, consists of two members of the 'meta' property
+# string in the top-level section of its config: 'uid', a random identifier of the guest's lineage
+# (100 bits as 20 characters of the lowercase RFC 4648 base32 alphabet), and 'rtime', the time the
+# config was registered in this cluster (milliseconds since the epoch). The functions here pass it
+# as a hash with these two keys.
+#
+# 100 random bits make a duplicate uid practically impossible: a billion uids contain one with a
+# probability of about 4 * 10^-13. They also leave 28 bits of a 128-bit value free, for example for
+# a format version and extra data, should a later format need them. Base32 keeps the uid at 20
+# characters, about half the width of a UUID in its usual notation, and only uses lowercase letters
+# and digits, which need no quoting in a property string.
+#
+# Unlike the creation time 'ctime' of the same property, which a clone, a restore and a migration
+# from another cluster copy along with the guest, the rtime belongs to the uid of the guest in this
+# cluster: it is the time the guest got its uid here, be it a new one or one it took over. So a
+# changed rtime tells that a guest was registered again, even if its uid is the same as before.
+
+# Must match the first pve-manager release that depends on the qemu-server and pve-container
+# versions handling the identity. Older nodes drop the whole 'meta' property on their next write
+# of a guest config.
+use constant MIN_PVE_MANAGER_VERSION => '9.2.31';
+
+my $UID_LENGTH = 20;
+my $UID_ALPHABET = 'abcdefghijklmnopqrstuvwxyz234567';
+my $UID_PATTERN = "[a-z2-7]{$UID_LENGTH}";
+
+# 15 digits of milliseconds last until the year 33658
+my $RTIME_MAX = 999_999_999_999_999;
+
+# Returns $len random bytes from the kernel's random number generator.
+sub __random_bytes($len) {
+ open(my $fh, '<', '/dev/urandom') or die "unable to open /dev/urandom - $!\n";
+ my $buf = '';
+ my $read = sysread($fh, $buf, $len);
+ die "unable to read from /dev/urandom - $!\n" if !defined($read);
+ die "short read from /dev/urandom\n" if $read != $len;
+ close($fh);
+
+ return $buf;
+}
+
+# Returns the current time in milliseconds since the epoch.
+sub __time_ms() {
+ my ($sec, $usec) = Time::HiRes::gettimeofday();
+
+ return $sec * 1000 + int($usec / 1000);
+}
+
+# Returns a random uid. Each character comes from the lowest 5 bits of a random byte of its own, so
+# it is uniformly distributed over the alphabet.
+my sub random_uid() {
+ my $bytes = __random_bytes($UID_LENGTH);
+
+ return join('', map { substr($UID_ALPHABET, ord($_) & 31, 1) } split(//, $bytes));
+}
+
+my sub is_valid_uid($uid) {
+ return defined($uid) && $uid =~ m/^$UID_PATTERN\z/;
+}
+
+my sub is_valid_rtime($rtime) {
+ return defined($rtime) && $rtime =~ m/^[0-9]{1,15}\z/;
+}
+
+# Returns the JSON schema properties of the identity members, for merging into the format of the
+# 'meta' property string of a guest type. Every call returns a new copy.
+sub meta_format_properties() {
+ return {
+ uid => {
+ type => 'string',
+ pattern => $UID_PATTERN,
+ description => "Random identifier of the guest lineage, independent of the VMID.",
+ optional => 1,
+ },
+ rtime => {
+ type => 'integer',
+ minimum => 0,
+ maximum => $RTIME_MAX,
+ description => "Time the guest was registered in this cluster, for example by"
+ . " creation, restore, or migration from another cluster, in milliseconds since"
+ . " the UNIX epoch.",
+ optional => 1,
+ },
+ };
+}
+
+# Returns a new identity with the current time as rtime. Its uid is $uid if given, otherwise a
+# random one.
+sub new_identity($uid = undef) {
+ die "invalid guest identity uid '$uid'\n" if defined($uid) && !is_valid_uid($uid);
+
+ return {
+ uid => $uid // random_uid(),
+ rtime => __time_ms(),
+ };
+}
+
+# Extracts the identity from the 'meta' property string $meta, which may contain any other
+# members, including ones unknown to this version.
+#
+# Returns undef if $meta has no valid uid, and an identity otherwise. Its rtime is undef if the
+# rtime member is missing or invalid. A member given more than once counts as invalid.
+#
+# This does not use PVE::JSONSchema::parse_property_string on purpose. That parser rejects the
+# whole string if one member is unknown or invalid. A guest without a readable identity gets a new
+# one with the next config write, so a problem with another member must not hide the uid.
+sub parse_identity($meta) {
+ return undef if !defined($meta);
+
+ my $values = {};
+ for my $part (split(/,/, $meta)) {
+ my ($key, $value) = $part =~ m/^([^=]+)=(.*)\z/ or next;
+ next if $key ne 'uid' && $key ne 'rtime';
+ $values->{$key} = exists($values->{$key}) ? undef : $value;
+ }
+
+ return undef if !is_valid_uid($values->{uid});
+
+ return {
+ uid => $values->{uid},
+ rtime => is_valid_rtime($values->{rtime}) ? $values->{rtime} : undef,
+ };
+}
+
+# Returns the 'meta' property string $meta, which may be undef, with the identity members set to
+# $identity. All other members are kept as they are, also ones unknown to this version, which
+# PVE::JSONSchema::print_property_string would reject.
+sub merge_identity($meta, $identity) {
+ die "invalid guest identity uid\n" if !is_valid_uid($identity->{uid});
+ die "invalid guest identity rtime\n" if !is_valid_rtime($identity->{rtime});
+
+ my (@plain, @members);
+ for my $part (split(/,/, $meta // '')) {
+ next if $part =~ m/^\s*\z/;
+
+ if (my ($key) = $part =~ m/^([^=]+)=/) {
+ push @members, [$key, $part] if $key ne 'uid' && $key ne 'rtime';
+ } else {
+ push @plain, $part;
+ }
+ }
+ push @members, ['rtime', "rtime=$identity->{rtime}"], ['uid', "uid=$identity->{uid}"];
+
+ # same order as PVE::JSONSchema::print_property_string uses, so that a guest type parsing and
+ # printing the property string again does not change it
+ return join(',', @plain, map { $_->[1] } sort { $a->[0] cmp $b->[0] } @members);
+}
+
+# Returns the identity in the top-level section of the raw guest config $raw, undef if it has none.
+# Only looks at the 'meta' property, so this also finds the identity in a config that the parser of
+# the guest type rejects or reads without 'meta', for example because of members from a newer
+# version.
+#
+# If a manual edit left more than one 'meta' line, the last one with anything after the colon
+# counts, also if that is only whitespace. The property lookup of pmxcfs does the same, and the
+# cluster-wide check of adopt uses it.
+sub parse_identity_from_raw_config($raw) {
+ my $meta;
+ for my $line (split(/\n/, $raw)) {
+ last if $line =~ m/^\[/; # snapshot or other section
+ $meta = $1 =~ s/^\s+|\s+$//gr if $line =~ m/^meta:(.+)$/;
+ }
+
+ return parse_identity($meta);
+}
+
+# Returns undef if $node runs a version of Proxmox VE that keeps the identity when writing guest
+# configs, see MIN_PVE_MANAGER_VERSION. Missing or invalid version information counts as too old.
+# Otherwise returns a hash with 'required-version', the minimum version, and 'version', the version
+# the node reports, if any.
+sub node_identity_unsupported($node) {
+ my $required = MIN_PVE_MANAGER_VERSION;
+ my ($major, $minor, $release) = split(/\./, $required);
+
+ my $supported =
+ eval { PVE::Cluster::node_pvecfg_version_at_least($node, $major, $minor, $release) };
+ return undef if $supported;
+
+ my $version = eval { PVE::Cluster::get_node_pvecfg_version($node) };
+
+ return { 'required-version' => $required, defined($version) ? (version => $version) : () };
+}
+
+# Returns the JSON schema of what node_identity_unsupported returns for a refused node, for the
+# result of API calls that report why a node is not allowed as migration target. Every call returns
+# a new copy.
+sub node_identity_unsupported_schema() {
+ return {
+ description =>
+ "Set if migrating to the node is refused because the node does not report a"
+ . " Proxmox VE version that keeps the guest identity.",
+ type => 'object',
+ optional => 1,
+ properties => {
+ version => {
+ description => "The Proxmox VE version the node reports.",
+ type => 'string',
+ optional => 1,
+ },
+ 'required-version' => {
+ description => "The minimum Proxmox VE version that keeps the guest identity.",
+ type => 'string',
+ },
+ },
+ };
+}
+
+# Dies unless $node runs a version of Proxmox VE that keeps the identity when writing guest
+# configs, see node_identity_unsupported.
+sub assert_node_supports_identity($node) {
+ my $unsupported = node_identity_unsupported($node) or return;
+ my ($version, $required) = $unsupported->@{qw(version required-version)};
+
+ if (!defined($version)) {
+ die "node '$node' does not report its Proxmox VE version, so it is unknown whether the node"
+ . " keeps the guest identity\n";
+ }
+
+ die "node '$node' runs Proxmox VE $version, keeping the guest identity requires $required or"
+ . " newer - upgrade the node first\n";
+}
+
+1;
diff --git a/src/tests/Makefile b/src/tests/Makefile
index 8eb38a8..e2c3d01 100644
--- a/src/tests/Makefile
+++ b/src/tests/Makefile
@@ -1,8 +1,8 @@
all:
-.PHONY: check abstract-config guest-id hookscript
-check: abstract-config guest-id hookscript
+.PHONY: check abstract-config guest-id guest-identity hookscript
+check: abstract-config guest-id guest-identity hookscript
abstract-config: abstract-config-tests.pl
perl -I.. ./abstract-config-tests.pl
@@ -10,5 +10,8 @@ abstract-config: abstract-config-tests.pl
guest-id: guest-id-tests.pl
perl -I.. ./guest-id-tests.pl
+guest-identity: guest-identity-tests.pl
+ perl -I.. ./guest-identity-tests.pl
+
hookscript: hookscript-tests.pl
perl -I.. ./hookscript-tests.pl
diff --git a/src/tests/guest-identity-tests.pl b/src/tests/guest-identity-tests.pl
new file mode 100755
index 0000000..241e18c
--- /dev/null
+++ b/src/tests/guest-identity-tests.pl
@@ -0,0 +1,338 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib qw(..);
+
+use Test::MockModule;
+use Test::More;
+use Time::HiRes;
+
+use PVE::JSONSchema;
+use PVE::Tools;
+
+use PVE::GuestIdentity;
+
+subtest 'generation with fixed random bytes and time' => sub {
+ my $random_bytes = [];
+ my $time_ms = 1_700_000_000_123;
+
+ my $identity_module = Test::MockModule->new('PVE::GuestIdentity');
+ $identity_module->redefine(
+ __random_bytes => sub {
+ my ($len) = @_;
+ my $bytes = shift(@$random_bytes) // die "test error: no random bytes left\n";
+ die "test error: requested $len random bytes\n" if length($bytes) != $len;
+ return $bytes;
+ },
+ __time_ms => sub { return $time_ms; },
+ );
+
+ # each byte gives one character, its lowest 5 bits select it from the lowercase RFC 4648 base32
+ # alphabet
+ my $vectors = [
+ ["\x00" x 20, 'a' x 20],
+ ["\xff" x 20, '7' x 20],
+ ['0123456789abcdefghij', 'qrstuvwxyzbcdefghijk'],
+ [pack('C*', 0 .. 19), 'abcdefghijklmnopqrst'],
+ [pack('C*', 12 .. 31), 'mnopqrstuvwxyz234567'],
+ [pack('C*', map { $_ | 0xe0 } 0 .. 19), 'abcdefghijklmnopqrst'],
+ ];
+
+ for my $vector (@$vectors) {
+ my ($bytes, $uid) = @$vector;
+ $random_bytes = [$bytes];
+ is_deeply(
+ PVE::GuestIdentity::new_identity(),
+ { uid => $uid, rtime => $time_ms },
+ "encodes " . unpack('H*', $bytes),
+ );
+ }
+
+ # every symbol is as likely at each position, as each of them comes from 8 of the 256 values of
+ # the byte at that position
+ my $alphabet = 'abcdefghijklmnopqrstuvwxyz234567';
+ my $mismatches = 0;
+ for my $position (0 .. 19) {
+ for my $value (0 .. 255) {
+ my $bytes = "\x00" x 20;
+ substr($bytes, $position, 1) = chr($value);
+ $random_bytes = [$bytes];
+ my $expected = 'a' x 20;
+ substr($expected, $position, 1) = substr($alphabet, $value % 32, 1);
+ $mismatches++ if PVE::GuestIdentity::new_identity()->{uid} ne $expected;
+ }
+ }
+ is($mismatches, 0, 'each character comes from the byte at its position');
+
+ $random_bytes = [];
+ is_deeply(
+ PVE::GuestIdentity::new_identity('gaytemzugu3doobzmfrg'),
+ { uid => 'gaytemzugu3doobzmfrg', rtime => $time_ms },
+ 'given uid is used',
+ );
+ eval { PVE::GuestIdentity::new_identity('GAYTEMZUGU3DOOBZMFRG') };
+ like($@, qr/invalid guest identity uid/, 'invalid given uid is rejected');
+};
+
+subtest 'generation' => sub {
+ my $first = PVE::GuestIdentity::new_identity();
+ my $second = PVE::GuestIdentity::new_identity();
+ like($first->{uid}, qr/^[a-z2-7]{20}\z/, 'uid has the expected format');
+ my @uids = map { PVE::GuestIdentity::new_identity()->{uid} } 1 .. 100;
+ is_deeply([grep { !m/^[a-z2-7]{20}\z/ } @uids], [], 'uids have 20 characters, no padding');
+ isnt($first->{uid}, $second->{uid}, 'uids differ');
+
+ my $before = int(Time::HiRes::time() * 1000);
+ my $rtime = PVE::GuestIdentity::new_identity()->{rtime};
+ my $after = int(Time::HiRes::time() * 1000) + 1;
+ like($rtime, qr/^\d+\z/, 'rtime is an integer');
+ ok($rtime >= $before && $rtime <= $after, 'rtime is the current time in milliseconds');
+};
+
+subtest 'format properties' => sub {
+ my $format = {
+ ctime => { type => 'integer', optional => 1 },
+ PVE::GuestIdentity::meta_format_properties()->%*,
+ };
+
+ is_deeply(
+ PVE::JSONSchema::parse_property_string(
+ $format,
+ 'ctime=1700000000,rtime=1700000000123,uid=gaytemzugu3doobzmfrg',
+ ),
+ { ctime => 1700000000, rtime => 1700000000123, uid => 'gaytemzugu3doobzmfrg' },
+ 'meta string with identity parses with a merged format',
+ );
+
+ for my $invalid (
+ 'uid=GAYTEMZUGU3DOOBZMFRG',
+ 'uid=gaytemzugu3doobzmfr',
+ 'uid=gaytemzugu3doobzmfrga',
+ 'uid=gaytemzugu3doobzmfr0',
+ 'uid=' . substr('gaytemzugu3doobzmfrg', 0, 16),
+ 'rtime=-1',
+ 'rtime=1.5',
+ 'rtime=1000000000000000',
+ ) {
+ eval { PVE::JSONSchema::parse_property_string($format, $invalid) };
+ ok($@, "format rejects '$invalid'");
+ }
+
+ my $copy = PVE::GuestIdentity::meta_format_properties();
+ $copy->{uid}->{pattern} = '.*';
+ isnt(
+ PVE::GuestIdentity::meta_format_properties()->{uid}->{pattern},
+ '.*',
+ 'callers get their own copy',
+ );
+};
+
+subtest 'parse identity' => sub {
+ my $uid = 'gaytemzugu3doobzmfrg';
+ my $tests = [
+ [undef, undef, 'no meta'],
+ ['', undef, 'empty meta'],
+ ['ctime=1700000000', undef, 'meta without identity'],
+ [
+ "creation-qemu=10.1.2,ctime=1700000000,rtime=123,uid=$uid",
+ { uid => $uid, rtime => 123 },
+ 'identity next to other members',
+ ],
+ [
+ "future=a b,uid=$uid,other,rtime=0",
+ { uid => $uid, rtime => 0 },
+ 'unknown members and odd parts are ignored',
+ ],
+ ["rtime=123,uid=$uid", { uid => $uid, rtime => 123 }, 'identity only'],
+ ["uid=$uid", { uid => $uid, rtime => undef }, 'missing rtime'],
+ ["uid=$uid,rtime=12a", { uid => $uid, rtime => undef }, 'invalid rtime'],
+ ["uid=$uid,rtime=-5", { uid => $uid, rtime => undef }, 'negative rtime'],
+ [
+ "uid=$uid,rtime=999999999999999",
+ { uid => $uid, rtime => 999999999999999 },
+ 'maximum rtime',
+ ],
+ ["uid=$uid,rtime=1000000000000000", { uid => $uid, rtime => undef }, 'too large rtime'],
+ [
+ "uid=$uid,rtime=\x{0661}\x{0662}",
+ { uid => $uid, rtime => undef },
+ 'non-ASCII digits',
+ ],
+ ["uid=$uid,rtime=1,rtime=2", { uid => $uid, rtime => undef }, 'duplicate rtime'],
+ ['rtime=123', undef, 'rtime without uid'],
+ ["uid=$uid,uid=$uid,rtime=1", undef, 'duplicate uid'],
+ ['uid=GAYTEMZUGU3DOOBZMFRG,rtime=1', undef, 'uppercase uid'],
+ ['uid=gaytemzugu3doobzmfr,rtime=1', undef, 'short uid'],
+ ['uid=gaytemzugu3doobzmfrga,rtime=1', undef, 'long uid'],
+ ['uid=' . substr($uid, 0, 16) . ',rtime=1', undef, '16-character uid'],
+ ['uid=gaytemzugu3doobzmfr1,rtime=1', undef, 'uid outside of the alphabet'],
+ ["uid= $uid,rtime=1", undef, 'uid with whitespace'],
+ ];
+
+ for my $test (@$tests) {
+ my ($meta, $expected, $name) = @$test;
+ is_deeply(PVE::GuestIdentity::parse_identity($meta), $expected, $name);
+ }
+};
+
+subtest 'parse identity from raw config' => sub {
+ my $uid = 'gaytemzugu3doobzmfrg';
+ my $other = 'aaaqeayeaudaocajbifq';
+ my $identity = { uid => $uid, rtime => 123 };
+ my $tests = [
+ ['', undef, 'empty config'],
+ ["memory: 512\n", undef, 'no meta'],
+ [
+ "memory: 512\nmeta: ctime=1,rtime=123,uid=$uid\ncores: 2\n",
+ $identity,
+ 'top-level meta',
+ ],
+ ["meta: rtime=123,uid=$uid \n", $identity, 'surrounding whitespace'],
+ ["meta: rtime=1,uid=$other\nmeta: rtime=123,uid=$uid\n", $identity, 'last meta wins'],
+ ["meta: future=a b,rtime=123,uid=$uid\n", $identity, 'unknown members'],
+ ["meta: rtime=123,uid=$uid\nmeta:\n", $identity, 'meta line without value is ignored'],
+ [
+ "meta: rtime=123,uid=$uid\nmeta: \n", undef, 'meta line with only spaces counts',
+ ],
+ ["#meta: rtime=123,uid=$uid\n", undef, 'description line'],
+ ["memory: 512\n\n[PENDING]\nmeta: uid=$uid\n", undef, 'pending section is ignored'],
+ [
+ "meta: rtime=123,uid=$uid\n\n[snap]\nmeta: rtime=1,uid=$other\n",
+ $identity,
+ 'snapshot section is ignored',
+ ],
+ ];
+
+ for my $test (@$tests) {
+ my ($raw, $expected, $name) = @$test;
+ is_deeply(PVE::GuestIdentity::parse_identity_from_raw_config($raw), $expected, $name);
+ }
+};
+
+subtest 'merge identity' => sub {
+ my $identity = { uid => 'gaytemzugu3doobzmfrg', rtime => 1700000000123 };
+ my $members = 'rtime=1700000000123,uid=gaytemzugu3doobzmfrg';
+ my $tests = [
+ [undef, $members, 'no meta'],
+ ['', $members, 'empty meta'],
+ [
+ 'creation-qemu=10.1.2,ctime=1700000000',
+ "creation-qemu=10.1.2,ctime=1700000000,$members",
+ 'other members are kept',
+ ],
+ [
+ 'ctime=1,rtime=5,uid=aaaaaaaaaaaaaaaaaaaa',
+ "ctime=1,$members",
+ 'existing identity is replaced',
+ ],
+ [
+ 'uid=bogus,zz-future=x,rtime=1,uid=aaaaaaaaaaaaaaaaaaaa,ctime=1',
+ "ctime=1,$members,zz-future=x",
+ 'invalid and duplicate identity members are dropped, unknown members kept',
+ ],
+ ['plain,ctime=1', "plain,ctime=1,$members", 'parts without key stay first'],
+ ];
+
+ for my $test (@$tests) {
+ my ($meta, $expected, $name) = @$test;
+ my $merged = PVE::GuestIdentity::merge_identity($meta, $identity);
+ is($merged, $expected, $name);
+ is_deeply(PVE::GuestIdentity::parse_identity($merged), $identity, "$name, parses back");
+ }
+
+ for my $invalid (
+ { uid => 'GAYTEMZUGU3DOOBZMFRG', rtime => 1 },
+ { uid => 'gaytemzugu3doobzmfrg', rtime => undef },
+ { uid => 'gaytemzugu3doobzmfrg', rtime => 'now' },
+ { uid => 'gaytemzugu3doobzmfrg', rtime => '1000000000000000' },
+ { uid => 'gaytemzugu3doobzmfrg,ctime=1', rtime => 1 },
+ { rtime => 1 },
+ ) {
+ eval { PVE::GuestIdentity::merge_identity('ctime=1', $invalid) };
+ ok($@, "invalid identity is rejected");
+ }
+};
+
+subtest 'node version check' => sub {
+ my $min = PVE::GuestIdentity::MIN_PVE_MANAGER_VERSION;
+ like($min, qr/^[0-9]+\.[0-9]+\.[0-9]+\z/, 'minimum version is a version string');
+ my ($major, $minor, $release) = split(/\./, $min);
+
+ my $versions = {
+ current => "$major.$minor.$release",
+ newer => "$major." . ($minor + 1) . ".0",
+ older => "$major.$minor." . ($release - 1),
+ invalid => 'invalid',
+ };
+
+ my $cluster_module = Test::MockModule->new('PVE::Cluster');
+ $cluster_module->mock(
+ get_node_pvecfg_version => sub {
+ my ($node) = @_;
+ die "malformed version info\n" if $node eq 'malformed';
+ return $versions->{$node};
+ },
+ node_pvecfg_version_at_least => sub {
+ my ($node, @min) = @_;
+ my $version = PVE::Cluster::get_node_pvecfg_version($node);
+ return $version && PVE::Tools::pvecfg_min_version($version, @min);
+ },
+ );
+
+ for my $node (qw(current newer)) {
+ eval { PVE::GuestIdentity::assert_node_supports_identity($node) };
+ is($@, '', "node with version '$versions->{$node}' is supported");
+ my $unsupported = PVE::GuestIdentity::node_identity_unsupported($node);
+ is($unsupported, undef, "no reason for '$node'");
+ }
+
+ for my $node (qw(older invalid missing malformed)) {
+ eval { PVE::GuestIdentity::assert_node_supports_identity($node) };
+ like($@, qr/node '$node'/, "node '$node' is refused");
+ }
+
+ my $schema = PVE::GuestIdentity::node_identity_unsupported_schema();
+ isnt(
+ $schema,
+ PVE::GuestIdentity::node_identity_unsupported_schema(),
+ 'schema is a new copy',
+ );
+ for my $node (qw(older invalid missing malformed)) {
+ my $unsupported = PVE::GuestIdentity::node_identity_unsupported($node);
+ eval { PVE::JSONSchema::validate($unsupported, $schema) };
+ is($@, '', "reason for '$node' matches the schema");
+ }
+
+ for my $node (qw(older invalid)) {
+ is_deeply(
+ PVE::GuestIdentity::node_identity_unsupported($node),
+ { version => $versions->{$node}, 'required-version' => $min },
+ "reason for '$node' has both versions",
+ );
+ eval { PVE::GuestIdentity::assert_node_supports_identity($node) };
+ is(
+ $@,
+ "node '$node' runs Proxmox VE $versions->{$node}, keeping the guest identity requires"
+ . " $min or newer - upgrade the node first\n",
+ "refusal of '$node' names both versions",
+ );
+ }
+
+ for my $node (qw(missing malformed)) {
+ is_deeply(
+ PVE::GuestIdentity::node_identity_unsupported($node),
+ { 'required-version' => $min },
+ "reason for '$node' has only the required version",
+ );
+ eval { PVE::GuestIdentity::assert_node_supports_identity($node) };
+ is(
+ $@,
+ "node '$node' does not report its Proxmox VE version, so it is unknown whether the"
+ . " node keeps the guest identity\n",
+ "refusal of '$node' says why",
+ );
+ }
+};
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH guest-common v3 10/38] abstract config: assign a guest identity and keep it on config writes
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (8 preceding siblings ...)
2026-11-18 4:37 ` [PATCH guest-common v3 09/38] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 11/38] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
` (29 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
With identities enabled for a guest type, every config write keeps the
identity that the guest already has, regardless of the identity in the
written config, and assigns a new one if the guest has none. This covers
both new guests and guests from older versions, and operations need no
identity handling of their own. A stale config, a config copied from a
clone source, or a snapshot rollback can neither change nor duplicate an
identity that way. Taking over the uid of a backup needs a separate
check that no other guest in the cluster uses it.
Read the existing identity directly from the cluster filesystem, as the
parsed config in the cache can be outdated. A read error aborts the
write. This relies on the IPC client of pve-cluster reporting a failed
connection as an error, and not as a missing or empty file.
Some writers, including the container post-stop hook, do not hold the
guest config lock. Read the identity and write the config under a short
local lock, so that concurrent writers cannot assign different
identities.
Guest types have to enable this explicitly: older qemu-server versions
validate 'meta' strictly on writes and would reject a config with the
identity members.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* commit message wording only
changes v1 -> v2:
* rebase onto the tracking of used guest IDs
* drop the override hooks and test seams that nothing used
* drop the cross-check of the config read against the mounted
filesystem, the IPC client now reports a failed connection
(pve-cluster commit e31fe10)
* say why there is a separate lock and a direct read (Dominik)
* use explicit conditionals instead of '$x && $x->{y}' (Dominik)
src/PVE/AbstractConfig.pm | 81 +++++-
src/tests/Makefile | 7 +-
src/tests/config-identity-tests.pl | 383 +++++++++++++++++++++++++++++
3 files changed, 468 insertions(+), 3 deletions(-)
create mode 100755 src/tests/config-identity-tests.pl
diff --git a/src/PVE/AbstractConfig.pm b/src/PVE/AbstractConfig.pm
index 78e384b..302994e 100644
--- a/src/PVE/AbstractConfig.pm
+++ b/src/PVE/AbstractConfig.pm
@@ -11,6 +11,7 @@ use PVE::Tools qw(lock_file lock_file_full);
use PVE::GuestHelpers qw(typesafe_ne);
use PVE::GuestID;
+use PVE::GuestIdentity;
use PVE::ReplicationConfig;
use PVE::Replication;
@@ -62,13 +63,91 @@ sub load_config {
return $conf;
}
+# Guest identity, see PVE::GuestIdentity
+
+# Whether writing a config keeps the stored guest identity, or assigns one if there is none,
+# see write_config. Only enable it for a guest type whose config format accepts the identity
+# members, as a strict parser would otherwise reject every written config.
+sub __identity_enabled {
+ my ($class) = @_;
+
+ return 0;
+}
+
+# Returns the guest identity contained in the parsed config $conf, for example of a backup, undef
+# if there is none.
+sub get_identity {
+ my ($class, $conf) = @_;
+
+ return PVE::GuestIdentity::parse_identity($conf->{meta});
+}
+
+# Runs $code with the local identity lock of $vmid held and returns its result. Every config write
+# takes it while it reads the existing identity and writes the config. The guest config lock
+# cannot do that job, as some writers do not hold it and cannot take it either: the post-stop hook
+# of a container writes the config while the task that stops the container holds that lock. Never
+# take another lock while holding the identity lock.
+sub __identity_lock {
+ my ($class, $vmid, $code, @param) = @_;
+
+ my $lockdir = $PVE::GuestHelpers::lockdir;
+ mkdir $lockdir;
+
+ my $res = lock_file("$lockdir/pve-guest-identity-$vmid", 10, $code, @param);
+ die $@ if $@;
+
+ return $res;
+}
+
+# Returns undef if no config file exists for $vmid on this node. Otherwise returns a hash whose
+# 'identity' member is the identity stored in the top-level section of the file, or undef if it
+# has none, as for an empty file. Dies on other read errors, like a failed connection to pmxcfs.
+#
+# Reads the file directly and not the parsed config from the file cache. A process only sees the
+# config writes of other processes after PVE::Cluster::cfs_update(), and some writers, like the
+# post-stop hook of a container, write a config that they loaded a while ago.
+sub __identity_read_stored {
+ my ($class, $vmid) = @_;
+
+ my $raw = PVE::Cluster::get_config($class->cfs_config_path($vmid));
+ return undef if !defined($raw);
+
+ return { identity => PVE::GuestIdentity::parse_identity_from_raw_config($raw) };
+}
+
# Generate and write config file for this VM/CT
+#
+# With guest identities enabled for the class, the identity stored in the existing config file
+# gets set in $conf, whatever $conf contains, or a new one if none is stored. So a config write
+# never changes a stored identity.
sub write_config {
my ($class, $vmid, $conf) = @_;
my $cfspath = $class->cfs_config_path($vmid);
- PVE::Cluster::cfs_write_file($cfspath, $conf);
+ if (!$class->__identity_enabled()) {
+ PVE::Cluster::cfs_write_file($cfspath, $conf);
+ return;
+ }
+
+ $class->__identity_lock(
+ $vmid,
+ sub {
+ my $stored = $class->__identity_read_stored($vmid);
+ my $identity = $stored ? $stored->{identity} : undef;
+
+ # a stored uid without valid rtime can only come from a manual edit
+ if (!$identity || !defined($identity->{rtime})) {
+ $identity =
+ PVE::GuestIdentity::new_identity($identity ? $identity->{uid} : undef);
+ }
+
+ $conf->{meta} = PVE::GuestIdentity::merge_identity($conf->{meta}, $identity);
+ PVE::Cluster::cfs_write_file($cfspath, $conf);
+ },
+ );
+
+ return;
}
# Pending changes related
diff --git a/src/tests/Makefile b/src/tests/Makefile
index e2c3d01..4d2036e 100644
--- a/src/tests/Makefile
+++ b/src/tests/Makefile
@@ -1,12 +1,15 @@
all:
-.PHONY: check abstract-config guest-id guest-identity hookscript
-check: abstract-config guest-id guest-identity hookscript
+.PHONY: check abstract-config config-identity guest-id guest-identity hookscript
+check: abstract-config config-identity guest-id guest-identity hookscript
abstract-config: abstract-config-tests.pl
perl -I.. ./abstract-config-tests.pl
+config-identity: config-identity-tests.pl
+ perl -I.. ./config-identity-tests.pl
+
guest-id: guest-id-tests.pl
perl -I.. ./guest-id-tests.pl
diff --git a/src/tests/config-identity-tests.pl b/src/tests/config-identity-tests.pl
new file mode 100755
index 0000000..fcec452
--- /dev/null
+++ b/src/tests/config-identity-tests.pl
@@ -0,0 +1,383 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib qw(..);
+
+use Fcntl qw(:flock);
+use File::Temp qw(tempdir);
+use Storable qw(dclone);
+use Test::MockModule;
+use Test::More;
+
+use PVE::AbstractConfig;
+use PVE::GuestHelpers;
+use PVE::GuestIdentity;
+
+my $tmpdir = tempdir('config-identity-XXXXXX', DIR => '.', CLEANUP => 1);
+$PVE::GuestHelpers::lockdir = "$tmpdir/lock";
+
+# guest config classes using a minimal config file format, see below
+
+package PVE::TestConfig {
+ use base qw(PVE::AbstractConfig);
+
+ sub guest_type { return 'VM'; }
+
+ sub config_file_lock($class, $vmid) {
+ return "$tmpdir/config-$vmid.lock";
+ }
+
+ sub cfs_config_path($class, $vmid, $node = undef) {
+ return "nodes/local/qemu-server/$vmid.conf";
+ }
+
+ # keeps any direct file access of the config class out of /etc/pve
+ sub config_file($class, $vmid, $node = undef) {
+ return "$tmpdir/etc-pve/" . $class->cfs_config_path($vmid, $node);
+ }
+
+ sub __identity_enabled { return 1; }
+
+ # just enough for snapshot rollbacks without volumes
+ sub valid_volume_keys { return (); }
+ sub __snapshot_check_running { return 0; }
+ sub __snapshot_rollback_vm_stop { return; }
+ sub __snapshot_rollback_get_unused { return []; }
+}
+
+package PVE::TestConfig::CT {
+ use base qw(PVE::TestConfig);
+
+ sub guest_type { return 'CT'; }
+
+ sub cfs_config_path($class, $vmid, $node = undef) {
+ return "nodes/local/lxc/$vmid.conf";
+ }
+}
+
+package PVE::TestConfig::NoIdentity {
+ use base qw(PVE::TestConfig);
+
+ sub __identity_enabled { return 0; }
+}
+
+package main;
+
+# fake cluster filesystem with raw config files, indexed by their cfs path, and a file cache that
+# behaves like the one of PVE::Cluster: it only reads a config again after a write of this process,
+# or after cfs_update noticed a newer version
+
+my $files = {};
+my $versions = {};
+my $version_counter = 0;
+my $seen_versions = {};
+my $cache = {};
+
+our $get_config_error;
+our $check_identity_lock = 1;
+
+my $reads = 0;
+my $writes = 0;
+
+sub parse_test_config($raw) {
+ my $conf = { snapshots => {} };
+ my $section = $conf;
+ for my $line (split(/\n/, $raw)) {
+ next if $line =~ m/^\s*$/;
+ if ($line =~ m/^\[([^\]]+)\]$/) {
+ $section = $conf->{snapshots}->{$1} = {};
+ } elsif ($line =~ m/^([a-z][a-z0-9_\-]*):\s*(.*?)\s*$/) {
+ $section->{$1} = $2;
+ } else {
+ die "test error: unable to parse config line '$line'\n";
+ }
+ }
+
+ return $conf;
+}
+
+sub write_test_config($conf) {
+ my $print_section = sub {
+ my ($section) = @_;
+ return join('',
+ map { "$_: $section->{$_}\n" } grep { !ref($section->{$_}) } sort keys %$section);
+ };
+
+ my $raw = $print_section->($conf);
+ for my $snapname (sort keys(($conf->{snapshots} // {})->%*)) {
+ $raw .= "\n[$snapname]\n" . $print_section->($conf->{snapshots}->{$snapname});
+ }
+
+ return $raw;
+}
+
+sub vmid_of_path($path) {
+ return $path =~ m!/(\d+)\.conf$! ? $1 : die "test error: unexpected path '$path'\n";
+}
+
+sub identity_lock_held($vmid) {
+ open(my $fh, '>>', "$PVE::GuestHelpers::lockdir/pve-guest-identity-$vmid") or return 0;
+ my $free = flock($fh, LOCK_EX | LOCK_NB);
+ close($fh);
+
+ return !$free;
+}
+
+my $cluster_module = Test::MockModule->new('PVE::Cluster');
+$cluster_module->redefine(
+ cfs_update => sub {
+ $seen_versions = { $versions->%* };
+ return;
+ },
+ cfs_read_file => sub {
+ my ($path) = @_;
+ my $version = $seen_versions->{$path};
+ my $entry = $cache->{$path};
+ if (!$entry || !$entry->{version} || !$version || $entry->{version} != $version) {
+ my $raw = $files->{$path};
+ my $data = defined($raw) ? parse_test_config($raw) : undef;
+ $entry = $cache->{$path} = { version => $version, data => $data };
+ }
+ return defined($entry->{data}) ? dclone($entry->{data}) : undef;
+ },
+ cfs_write_file => sub {
+ my ($path, $conf) = @_;
+ my $vmid = vmid_of_path($path);
+ die "test error: config of $vmid written without identity lock\n"
+ if $check_identity_lock && !identity_lock_held($vmid);
+ $writes++;
+ $files->{$path} = write_test_config($conf);
+ $versions->{$path} = ++$version_counter;
+ $cache->{$path}->{version} = undef if $cache->{$path};
+ },
+ get_config => sub {
+ my ($path) = @_;
+ my $vmid = vmid_of_path($path);
+ die "test error: config of $vmid read without identity lock\n"
+ if !identity_lock_held($vmid);
+ die $get_config_error if $get_config_error;
+ $reads++;
+ return $files->{$path};
+ },
+);
+
+# deterministic identities: uids count up, the time is set by the tests
+
+my $uid_counter = 0;
+my $now = 1_700_000_000_000;
+
+# recording used guest IDs needs the cluster filesystem and is not what gets tested here
+my $guest_id_module = Test::MockModule->new('PVE::GuestID');
+$guest_id_module->redefine(register_used_id => sub { });
+
+my $identity_module = Test::MockModule->new('PVE::GuestIdentity');
+$identity_module->redefine(
+ # one byte per uid character, the last three count up
+ __random_bytes => sub {
+ my ($len) = @_;
+ my $n = ++$uid_counter;
+ return ("\0" x ($len - 3)) . pack('C3', map { ($n >> $_) & 31 } (10, 5, 0));
+ },
+ __time_ms => sub { return $now; },
+);
+
+my sub new_uid {
+ return PVE::GuestIdentity::new_identity()->{uid};
+}
+
+my sub file_of($class, $vmid) {
+ return $files->{ $class->cfs_config_path($vmid) };
+}
+
+my sub conf_of($class, $vmid) {
+ my $raw = file_of($class, $vmid);
+ return defined($raw) ? parse_test_config($raw) : undef;
+}
+
+my sub identity_of($class, $vmid) {
+ my $conf = conf_of($class, $vmid) or return undef;
+ return PVE::GuestIdentity::parse_identity($conf->{meta});
+}
+
+# write of another process, the file cache of this process does not notice it
+my sub set_file($class, $vmid, $raw) {
+ my $path = $class->cfs_config_path($vmid);
+ $files->{$path} = $raw;
+ $versions->{$path} = ++$version_counter;
+}
+
+my $foreign_uid = new_uid();
+
+subtest 'disabled by default' => sub {
+ ok(!PVE::AbstractConfig->__identity_enabled(), 'abstract config does not enable it');
+
+ local $check_identity_lock = 0;
+ my $class = 'PVE::TestConfig::NoIdentity';
+ my $reads_before = $reads;
+
+ my $res = [$class->write_config(100, { name => 'a' })];
+ is(file_of($class, 100), "name: a\n", 'config without identity written as is');
+ is_deeply($res, [], 'nothing is returned');
+
+ my $meta = "ctime=1,rtime=5,uid=$foreign_uid";
+ $class->write_config(100, { name => 'a', meta => $meta });
+ is(file_of($class, 100), "meta: $meta\nname: a\n", 'carried identity written as is');
+
+ is($reads, $reads_before, 'stored identity is not read');
+};
+
+subtest 'none stored' => sub {
+ my $class = 'PVE::TestConfig';
+
+ my $conf = { name => 'new' };
+ my $res = [$class->write_config(101, $conf)];
+ my $identity = identity_of($class, 101);
+ ok($identity, 'new config gets an identity');
+ is($identity->{rtime}, $now, 'rtime is the current time');
+ is_deeply($class->get_identity($conf), $identity, 'identity is set in the written hash');
+ is_deeply($res, [], 'nothing is returned');
+
+ # copied from a clone source, a snapshot, or a backup
+ $class->write_config(102, { name => 'clone', meta => "ctime=7,rtime=5,uid=$foreign_uid" });
+ $identity = identity_of($class, 102);
+ isnt($identity->{uid}, $foreign_uid, 'carried identity is not stored');
+ is($identity->{rtime}, $now, 'carried rtime is not stored');
+ like(
+ file_of($class, 102),
+ qr/^meta: ctime=7,rtime=$now,uid=[a-z2-7]{20}$/m,
+ 'other members are kept',
+ );
+
+ set_file($class, 103, "meta: ctime=1\nname: old\n");
+ $class->write_config(103, conf_of($class, 103));
+ $identity = identity_of($class, 103);
+ ok($identity, 'existing config without identity gets one');
+ like(file_of($class, 103), qr/^meta: ctime=1,rtime=$now,uid=/m, 'other members are kept');
+
+ set_file($class, 104, "name: old\n\n[snap]\nmeta: rtime=1,uid=$foreign_uid\n");
+ $class->write_config(104, conf_of($class, 104));
+ isnt(identity_of($class, 104)->{uid}, $foreign_uid, 'identity of a snapshot is ignored');
+ like(
+ file_of($class, 104),
+ qr/^\[snap\]\nmeta: rtime=1,uid=$foreign_uid$/m,
+ 'snapshot keeps its copy',
+ );
+
+ set_file($class, 105, '');
+ $class->write_config(105, { name => 'empty' });
+ ok(identity_of($class, 105), 'existing empty config gets an identity');
+};
+
+subtest 'stored identity wins' => sub {
+ my $class = 'PVE::TestConfig';
+
+ $class->write_config(110, { name => 'a', meta => 'ctime=1' });
+ my $stored = identity_of($class, 110);
+ my $stale = new_uid();
+
+ $now += 1000;
+
+ my $tests = [
+ [{ name => 'b' }, 'missing identity'],
+ [{ name => 'b', meta => 'ctime=1' }, 'missing identity members'],
+ [{ name => 'b', meta => "ctime=1,rtime=5,uid=$stale" }, 'stale identity'],
+ [{ name => 'b', meta => "ctime=1,rtime=$now,uid=$foreign_uid" }, 'foreign identity'],
+ [{ name => 'b', meta => "ctime=1,rtime=5,uid=bogus" }, 'invalid identity'],
+ [{ name => 'b', meta => "ctime=1,uid=$stored->{uid}" }, 'identity without rtime'],
+ ];
+
+ for my $test (@$tests) {
+ my ($conf, $name) = @$test;
+ my $other_members = $conf->{meta} && $conf->{meta} =~ m/ctime=1/;
+
+ $class->write_config(110, $conf);
+ is_deeply(identity_of($class, 110), $stored, "$name: stored identity is kept");
+ is_deeply($class->get_identity($conf), $stored, "$name: hash has it too");
+ like(file_of($class, 110), qr/^meta: ctime=1,/m, "$name: other members are kept")
+ if $other_members;
+ }
+
+ set_file($class, 111, "meta: ctime=1,uid=$stale\nname: old\n");
+ $class->write_config(111, conf_of($class, 111));
+ is_deeply(
+ identity_of($class, 111),
+ { uid => $stale, rtime => $now },
+ 'stored uid without rtime is kept and gets the current time',
+ );
+
+ my ($first, $last) = (new_uid(), new_uid());
+ set_file($class, 112, "meta: rtime=1,uid=$first\nname: x\nmeta: rtime=2,uid=$last\n");
+ $class->write_config(112, { name => 'y' });
+ is_deeply(
+ identity_of($class, 112),
+ { uid => $last, rtime => 2 },
+ 'last top-level meta line counts, like for the config parsers',
+ );
+};
+
+subtest 'read and write under identity lock' => sub {
+ my $class = 'PVE::TestConfig';
+
+ # the cluster filesystem mocks die without the lock, the lock is gone afterwards
+ $class->write_config(120, { name => 'a' });
+ ok(identity_of($class, 120), 'written');
+ ok(!identity_lock_held(120), 'lock is released');
+
+ $class->lock_config(120, sub { $class->write_config(120, conf_of($class, 120)) });
+ ok(!identity_lock_held(120), 'lock is released when nested in config lock');
+};
+
+subtest 'read errors are fatal' => sub {
+ my $class = 'PVE::TestConfig';
+
+ $class->write_config(130, { name => 'a' });
+ my $raw = file_of($class, 130);
+
+ {
+ local $get_config_error = "Connection refused\n";
+ eval { $class->write_config(130, { name => 'b' }) };
+ is($@, "Connection refused\n", 'write dies');
+ is(file_of($class, 130), $raw, 'config is unchanged');
+
+ eval { $class->write_config(131, { name => 'b' }) };
+ is($@, "Connection refused\n", 'write of new config dies');
+ ok(!defined(file_of($class, 131)), 'config is not written');
+ }
+};
+
+subtest 'snapshot rollback' => sub {
+ my $class = 'PVE::TestConfig';
+
+ my $storage_module = Test::MockModule->new('PVE::Storage');
+ $storage_module->redefine(config => sub { return {}; });
+ my $replication_module = Test::MockModule->new('PVE::ReplicationConfig');
+ $replication_module->redefine(
+ new => sub { return bless({}, 'PVE::ReplicationConfig'); },
+ check_for_existing_jobs => sub { return 0; },
+ );
+
+ $class->write_config(140, { name => 'current', meta => 'ctime=2' });
+ my $stored = identity_of($class, 140);
+
+ my $conf = conf_of($class, 140);
+ $conf->{snapshots}->{old} = { name => 'old', meta => "ctime=1,rtime=5,uid=$foreign_uid" };
+ $conf->{snapshots}->{none} = { name => 'none' };
+ $class->write_config(140, $conf);
+
+ $now += 1000;
+
+ for my $snapname (qw(old none)) {
+ $class->snapshot_rollback(140, $snapname);
+ $conf = conf_of($class, 140);
+ is($conf->{name}, $snapname, "rolled back to '$snapname'");
+ is_deeply(identity_of($class, 140), $stored, "'$snapname': current identity is kept");
+ }
+ is(
+ $conf->{snapshots}->{old}->{meta},
+ "ctime=1,rtime=5,uid=$foreign_uid",
+ 'snapshot is unchanged',
+ );
+};
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH guest-common v3 11/38] abstract config: let the VMID reservation return the previous state
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (9 preceding siblings ...)
2026-11-18 4:37 ` [PATCH guest-common v3 10/38] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 12/38] guest identity: allow taking over a uid that is unused in the cluster Thomas Lamprecht
` (28 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A restore needs to know whether its target existed and had an identity
before the operation, to decide between keeping the identity of the
target and taking over the uid of the backup. With identities enabled,
the reservation already assigns an identity to a target without one,
so reading the config afterwards cannot tell anymore, and reading it
before the locked section races with other writers.
Return that state from within the locked section, together with the
identity that the guest has after the reservation. With identities
enabled, the reservation reads the existing identity under the same lock
as the write. That read also finds a config that exists but fails to
load. Refuse such a config instead of replacing it with an empty one,
which would drop its lock and protection even if the restore is then
refused.
Existing callers ignore the return value.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* refuse an existing config that fails to load instead of replacing it
with an empty one, which dropped its lock and protection
* tests for that and for the reservation of a VMID whose config appeared
after the request started (refused by an independent fix before the
series)
changes v1 -> v2:
* rebase: the used guest ID gets registered before the state is captured
* name the keys of the returned state after what they hold (previous,
assigned)
src/PVE/AbstractConfig.pm | 45 +++++++++++-
src/tests/config-identity-tests.pl | 110 ++++++++++++++++++++++++++++-
2 files changed, 152 insertions(+), 3 deletions(-)
diff --git a/src/PVE/AbstractConfig.pm b/src/PVE/AbstractConfig.pm
index 302994e..ccbfc8e 100644
--- a/src/PVE/AbstractConfig.pm
+++ b/src/PVE/AbstractConfig.pm
@@ -332,10 +332,20 @@ sub load_current_config {
return $conf;
}
+# Reserves $vmid for an operation by writing its config with the lock $lock, 'create' by default.
+# An existing config is only allowed with $allow_existing and must not be locked. With identities
+# enabled, one that fails to load is refused as well.
+#
+# Returns the state of the guest identity around the write, captured under the guest config lock,
+# as the write can assign an identity to a config without one. Taking over a uid later needs it.
+# existed - whether a config existed before
+# previous - the guest identity of that config, undef if it had none or did not exist
+# assigned - the guest identity of the config after the write, undef with identities
+# disabled
sub create_and_lock_config {
my ($class, $vmid, $allow_existing, $lock) = @_;
- $class->lock_config_full(
+ return $class->lock_config_full(
$vmid,
5,
sub {
@@ -344,13 +354,44 @@ sub create_and_lock_config {
PVE::GuestID::register_used_id($vmid);
my $conf = eval { $class->load_config($vmid) };
+ my $load_err = $@;
# the guest list of check_vmid_unused can predate a config created since
die $class->guest_type() . " $vmid already exists\n"
if defined($conf) && !$allow_existing;
+ my $existed = defined($conf);
+ my $previous = $existed ? $class->get_identity($conf) : undef;
+
$conf //= {};
$class->check_lock($conf);
$conf->{lock} = $lock // 'create';
- $class->write_config($vmid, $conf);
+
+ if ($class->__identity_enabled()) {
+ # use what the write itself sees as stored, see write_config
+ $class->__identity_lock(
+ $vmid,
+ sub {
+ my $stored = $class->__identity_read_stored($vmid);
+ # this also finds a config that failed to load, replacing it with an
+ # empty one would drop its lock and protection
+ if ($stored) {
+ die $class->guest_type() . " $vmid already exists\n"
+ if !$allow_existing;
+ die $load_err if !$existed;
+ }
+ $existed = defined($stored);
+ $previous = $stored ? $stored->{identity} : undef;
+ $class->write_config($vmid, $conf);
+ },
+ );
+ } else {
+ $class->write_config($vmid, $conf);
+ }
+
+ return {
+ existed => $existed ? 1 : 0,
+ previous => $previous,
+ assigned => $class->__identity_enabled() ? $class->get_identity($conf) : undef,
+ };
},
);
}
diff --git a/src/tests/config-identity-tests.pl b/src/tests/config-identity-tests.pl
index fcec452..28cb1a3 100755
--- a/src/tests/config-identity-tests.pl
+++ b/src/tests/config-identity-tests.pl
@@ -151,6 +151,12 @@ $cluster_module->redefine(
$versions->{$path} = ++$version_counter;
$cache->{$path}->{version} = undef if $cache->{$path};
},
+ check_vmid_unused => sub {
+ my ($vmid, $noerr) = @_;
+ return 1 if !grep { defined($files->{$_}) && vmid_of_path($_) == $vmid } keys %$files;
+ return undef if $noerr;
+ die "guest $vmid already exists\n";
+ },
get_config => sub {
my ($path) = @_;
my $vmid = vmid_of_path($path);
@@ -167,7 +173,7 @@ $cluster_module->redefine(
my $uid_counter = 0;
my $now = 1_700_000_000_000;
-# recording used guest IDs needs the cluster filesystem and is not what gets tested here
+# recording used guest IDs needs the cluster filesystem, only one subtest fakes the list of them
my $guest_id_module = Test::MockModule->new('PVE::GuestID');
$guest_id_module->redefine(register_used_id => sub { });
@@ -380,4 +386,106 @@ subtest 'snapshot rollback' => sub {
);
};
+subtest 'create and lock config' => sub {
+ my $class = 'PVE::TestConfig';
+
+ my $state = $class->create_and_lock_config(150);
+ my $reserved = identity_of($class, 150);
+ ok($reserved, 'free VMID: reservation gets an identity');
+ is_deeply(
+ $state,
+ { existed => 0, previous => undef, assigned => $reserved },
+ 'free VMID: no previous identity',
+ );
+ is(conf_of($class, 150)->{lock}, 'create', 'free VMID: reserved');
+
+ eval { $class->create_and_lock_config(150) };
+ like($@, qr/already exists/, 'existing config is refused by default');
+
+ $class->write_config(151, { name => 'target', meta => 'ctime=1' });
+ my $stored = identity_of($class, 151);
+ $now += 1000;
+ $state = $class->create_and_lock_config(151, 1, 'restore');
+ is_deeply(
+ $state,
+ { existed => 1, previous => $stored, assigned => $stored },
+ 'in place: previous identity',
+ );
+ my $conf = conf_of($class, 151);
+ is($conf->{lock}, 'restore', 'in place: locked');
+ is($conf->{name}, 'target', 'in place: config is kept');
+ is_deeply(identity_of($class, 151), $stored, 'in place: identity is kept');
+
+ eval { $class->create_and_lock_config(151, 1) };
+ like($@, qr/^VM is locked \(restore\)/, 'locked config is refused');
+
+ set_file($class, 152, "meta: ctime=1\nname: old\n");
+ $state = $class->create_and_lock_config(152, 1);
+ is_deeply(
+ $state,
+ { existed => 1, previous => undef, assigned => identity_of($class, 152) },
+ 'in place without identity: reservation gets one',
+ );
+ ok($state->{assigned}, 'in place without identity: reserved identity');
+
+ set_file($class, 157, '');
+ $state = $class->create_and_lock_config(157, 1);
+ is_deeply(
+ $state,
+ { existed => 1, previous => undef, assigned => identity_of($class, 157) },
+ 'in place with empty config: reservation gets an identity',
+ );
+ ok($state->{assigned}, 'in place with empty config: reserved identity');
+
+ my $unloadable = "lock: backup\nmeta: rtime=5,uid=$foreign_uid\nname: old\n!unparsable\n";
+ set_file($class, 156, $unloadable);
+ eval { $class->create_and_lock_config(156, 1) };
+ like($@, qr/^test error: unable to parse/, 'in place, config fails to load: refused');
+ is(file_of($class, 156), $unloadable, 'in place, config fails to load: file is unchanged');
+
+ # the guest list of the process can predate a config created since
+ my $check_vmid_unused = PVE::Cluster->can('check_vmid_unused');
+ $cluster_module->redefine(check_vmid_unused => sub { return 1; });
+ for my $raw ("name: other\n", $unloadable) {
+ set_file($class, 158, $raw);
+ eval { $class->create_and_lock_config(158) };
+ is($@, "VM 158 already exists\n", 'outdated guest list: existing config is refused');
+ is(file_of($class, 158), $raw, 'outdated guest list: file is unchanged');
+ }
+ $cluster_module->redefine(check_vmid_unused => $check_vmid_unused);
+
+ # clone: the target's first write must not store the source's identity
+ $state = $class->create_and_lock_config(153);
+ $conf = conf_of($class, 151);
+ delete $conf->{lock};
+ $class->write_config(153, $conf);
+ is_deeply(identity_of($class, 153), $state->{assigned},
+ 'clone keeps the reserved identity');
+ like(file_of($class, 153), qr/^meta: ctime=1,/m, 'clone keeps other members of the source');
+
+ local $check_identity_lock = 0;
+ $class = 'PVE::TestConfig::NoIdentity';
+
+ $state = $class->create_and_lock_config(154);
+ is_deeply(
+ $state,
+ { existed => 0, previous => undef, assigned => undef },
+ 'disabled, free VMID: no previous identity',
+ );
+ is(file_of($class, 154), "lock: create\n", 'disabled, free VMID: no identity assigned');
+
+ set_file($class, 155, "meta: ctime=1,rtime=5,uid=$foreign_uid\nname: old\n");
+ $state = $class->create_and_lock_config(155, 1);
+ is_deeply(
+ $state,
+ { existed => 1, previous => { uid => $foreign_uid, rtime => 5 }, assigned => undef },
+ 'disabled, in place: identity from the config',
+ );
+ is(
+ file_of($class, 155),
+ "lock: create\nmeta: ctime=1,rtime=5,uid=$foreign_uid\nname: old\n",
+ 'disabled, in place: config is kept',
+ );
+};
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH guest-common v3 12/38] guest identity: allow taking over a uid that is unused in the cluster
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (10 preceding siblings ...)
2026-11-18 4:37 ` [PATCH guest-common v3 11/38] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 13/38] migrate: add shared helpers for the identity command of the migration tunnel Thomas Lamprecht
` (27 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Restores and remote migrations should keep the uid of the backup or of
the source guest, but must not duplicate it within the cluster. Config
writes keep the identity a guest already has, so provide a separate step
that takes over a uid.
The step checks the top-level 'meta' property of all guests and writes
the uid under a cluster-wide lock shared by VMs and containers. A guest
that had an identity before the operation keeps it. Otherwise, the uid
replaces the identity that the guest got when the operation reserved its
VMID. Taking over the uid a guest already has leaves its registration
time unchanged.
A restore takes the uid over at its end. If it fails or gets aborted, it
can leave a guest behind with the identity of its reservation. So the
restore records the uid it wants to take over as a pending change of
'meta'. A retry over such a guest takes the pending uid over if it
matches, under the same cluster-wide check. A pending uid does not count
as in use, so a retry at another VMID stays possible.
Taking over removes the pending uid once it has a result. If the
cluster-wide lock cannot be taken, the pending uid stays for a later
operation. The config view with pending changes applied shows the
identity of the guest, while the list of pending changes still shows the
pending uid. Applying pending changes drops a pending uid with a
warning, as a retry of the restore then needs the override.
A remote migration takes the uid over right after it reserved the VMID
on the target, before anything is transferred. If another guest in the
target cluster has the uid, the migration is refused, unless the caller
allows a new uid for the migrated guest.
Both guest types share the checks and messages of these steps: the
refusal to restore a backup over a guest with another uid, the take-over
at the end of a restore and at the start of a remote migration, and the
log line for each result. If a newer target of a remote migration
reports a result that the source does not know, the source only logs a
warning.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* the lookup double of the tests reads a 'meta:' line with nothing after
the colon like pmxcfs does, with a test case
* comments: when the pending uid gets stored, rewrap
changes v1 -> v2:
* add shared helpers for the check of a backup against its target, for
the take-over at the end of a restore and for the take-over at the
start of a remote migration, so that both guest types share checks and
messages
* drop the result 'changed' with its check, which only a manual edit
during the operation could trigger (Dominik found the name confusing)
* drop a pending uid through one shared helper with a warning of its own
* add an overview of the steps and rework the comment of adopt (Dominik)
* add a test with the real registration of used guest IDs
src/PVE/AbstractConfig.pm | 78 +++-
src/PVE/GuestIdentity.pm | 240 ++++++++++
src/tests/config-identity-tests.pl | 706 ++++++++++++++++++++++++++++-
src/tests/guest-identity-tests.pl | 192 ++++++++
4 files changed, 1207 insertions(+), 9 deletions(-)
diff --git a/src/PVE/AbstractConfig.pm b/src/PVE/AbstractConfig.pm
index ccbfc8e..5f982d2 100644
--- a/src/PVE/AbstractConfig.pm
+++ b/src/PVE/AbstractConfig.pm
@@ -119,7 +119,7 @@ sub __identity_read_stored {
#
# With guest identities enabled for the class, the identity stored in the existing config file
# gets set in $conf, whatever $conf contains, or a new one if none is stored. So a config write
-# never changes a stored identity.
+# never changes the uid of a guest, only PVE::GuestIdentity::adopt does.
sub write_config {
my ($class, $vmid, $conf) = @_;
@@ -150,6 +150,49 @@ sub write_config {
return;
}
+# Replaces the stored identity of the config of $vmid with $identity and removes its pending uid
+# in the same write. Only for PVE::GuestIdentity::adopt, which checks that no other guest has the
+# uid.
+sub __identity_replace {
+ my ($class, $vmid, $identity) = @_;
+
+ die "guest identities are not enabled for this guest type\n" if !$class->__identity_enabled();
+
+ $class->__identity_lock(
+ $vmid,
+ sub {
+ # the file cache only notices writes of other processes after an update
+ PVE::Cluster::cfs_update();
+ my $conf = $class->load_config($vmid);
+
+ $conf->{meta} = PVE::GuestIdentity::merge_identity($conf->{meta}, $identity);
+ PVE::GuestIdentity::set_pending_uid($conf, undef);
+ PVE::Cluster::cfs_write_file($class->cfs_config_path($vmid), $conf);
+ },
+ );
+
+ return;
+}
+
+# Writes $uid as pending uid into the config of $vmid, or removes the pending uid if $uid is undef,
+# keeping everything else, see PVE::GuestIdentity::set_pending_uid. Does not write if the config
+# already has it that way. The caller must hold the config lock.
+sub record_pending_uid {
+ my ($class, $vmid, $uid) = @_;
+
+ # the file cache only notices writes of other processes after an update
+ PVE::Cluster::cfs_update();
+ my $conf = $class->load_config($vmid);
+ return if ($uid // '') eq (PVE::GuestIdentity::get_pending_uid($conf) // '');
+
+ PVE::GuestIdentity::set_pending_uid($conf, $uid);
+ # the parsed config lacks the stored identity if the parser could not read the 'meta'
+ # property, but write_config puts it back
+ $class->write_config($vmid, $conf);
+
+ return;
+}
+
# Pending changes related
sub parse_pending_delete {
@@ -314,14 +357,19 @@ sub load_current_config {
# take pending changes in
if (!$current) {
+ # a guest type with identities never applies a pending change of 'meta', which can hold
+ # the pending uid, see PVE::GuestIdentity::set_pending_uid
+ my $skip = $class->__identity_enabled() ? { meta => 1 } : {};
+
foreach my $opt (keys %{ $conf->{pending} }) {
- next if $opt eq 'delete';
+ next if $opt eq 'delete' || $skip->{$opt};
my $value = $conf->{pending}->{$opt};
next if ref($value); # just to be sure
$conf->{$opt} = $value;
}
my $pending_delete_hash = $class->parse_pending_delete($conf->{pending}->{delete});
foreach my $opt (keys %$pending_delete_hash) {
+ next if $skip->{$opt};
delete $conf->{$opt} if $conf->{$opt};
}
}
@@ -332,6 +380,29 @@ sub load_current_config {
return $conf;
}
+# 'meta' is read-only, so only a manual edit, an older version, or an unfinished restore, which
+# keeps the uid of its backup as pending uid, see PVE::GuestIdentity::set_pending_uid, can have
+# queued a change of it. Drops such a change from the parsed config $conf instead of applying it.
+sub drop_pending_meta {
+ my ($class, $conf) = @_;
+
+ my $pending_delete_hash = $class->parse_pending_delete($conf->{pending}->{delete});
+ return if !defined($conf->{pending}->{meta}) && !$pending_delete_hash->{meta};
+
+ my $uid = PVE::GuestIdentity::get_pending_uid($conf);
+ if (defined($uid)) {
+ warn "guest identity: dropping pending uid $uid of an unfinished restore - a retry now"
+ . " needs 'allow-identity-mismatch'\n";
+ } else {
+ warn "ignoring pending change of read-only option 'meta'\n";
+ }
+
+ delete $conf->{pending}->{meta};
+ $class->remove_from_pending_delete($conf, 'meta');
+
+ return;
+}
+
# Reserves $vmid for an operation by writing its config with the lock $lock, 'create' by default.
# An existing config is only allowed with $allow_existing and must not be locked. With identities
# enabled, one that fails to load is refused as well.
@@ -340,6 +411,7 @@ sub load_current_config {
# as the write can assign an identity to a config without one. Taking over a uid later needs it.
# existed - whether a config existed before
# previous - the guest identity of that config, undef if it had none or did not exist
+# previous-pending-uid - the pending uid of that config, see PVE::GuestIdentity::set_pending_uid
# assigned - the guest identity of the config after the write, undef with identities
# disabled
sub create_and_lock_config {
@@ -360,6 +432,7 @@ sub create_and_lock_config {
if defined($conf) && !$allow_existing;
my $existed = defined($conf);
my $previous = $existed ? $class->get_identity($conf) : undef;
+ my $pending = $existed ? PVE::GuestIdentity::get_pending_uid($conf) : undef;
$conf //= {};
$class->check_lock($conf);
@@ -390,6 +463,7 @@ sub create_and_lock_config {
return {
existed => $existed ? 1 : 0,
previous => $previous,
+ 'previous-pending-uid' => $pending,
assigned => $class->__identity_enabled() ? $class->get_identity($conf) : undef,
};
},
diff --git a/src/PVE/GuestIdentity.pm b/src/PVE/GuestIdentity.pm
index 27dbf8b..8474c88 100644
--- a/src/PVE/GuestIdentity.pm
+++ b/src/PVE/GuestIdentity.pm
@@ -5,6 +5,7 @@ use v5.36;
use Time::HiRes;
use PVE::Cluster;
+use PVE::RESTEnvironment;
# The identity of a guest, VM or container alike, consists of two members of the 'meta' property
# string in the top-level section of its config: 'uid', a random identifier of the guest's lineage
@@ -172,6 +173,245 @@ sub parse_identity_from_raw_config($raw) {
return parse_identity($meta);
}
+# A restore takes over a uid in three steps. An incoming remote migration does the same without
+# step 2: it calls adopt right after the reservation, before anything of the guest gets transferred,
+# so that a uid in use by another guest can still stop the migration. Its later config writes keep
+# the identity, see adopt_from_source.
+#
+# 1. Reserve: create_and_lock_config of the config class writes the config. The guest then has an
+# identity, the one it had before or a new one. The returned reservation records which
+# identity the guest had before, if any, and which one it has now.
+# 2. Remember: when the restore knows the uid of the backup, it stores that uid as pending uid, if
+# step 3 could take it over without 'unique', see adoptable_uid. The pending uid is a pending
+# change of 'meta' with only the uid member. It does not count as in use, and no config write
+# applies it. If the restore fails, the guest that it leaves behind still has the pending uid,
+# so a retry can take the uid over.
+# 3. Take over: at its end, the restore calls adopt. The guest gets the uid if it had no
+# identity before step 1, or had the same uid as pending uid, and if no other guest in the
+# cluster has that uid. Otherwise, the guest keeps the identity it has after step 1.
+
+# Returns the pending uid in the parsed config $conf, undef if there is none.
+sub get_pending_uid($conf) {
+ my $pending = $conf->{pending} ? parse_identity($conf->{pending}->{meta}) : undef;
+
+ return $pending ? $pending->{uid} : undef;
+}
+
+# Sets $uid as pending uid in the parsed config $conf, replacing any pending change of 'meta', or
+# removes such a change if $uid is undef.
+sub set_pending_uid($conf, $uid) {
+ if (defined($uid)) {
+ die "invalid guest identity uid '$uid'\n" if !is_valid_uid($uid);
+ $conf->{pending}->{meta} = "uid=$uid";
+ } elsif ($conf->{pending}) {
+ delete $conf->{pending}->{meta};
+ }
+
+ return;
+}
+
+# Returns $uid if adopt would try to make it the uid of the guest of the operation with the
+# reservation $reservation, which is the case if the guest had no identity before the operation, or
+# $uid as pending uid. Returns undef otherwise, or if $uid is undef. An operation keeps the uid it
+# returns as pending uid until it calls adopt.
+sub adoptable_uid($reservation, $uid) {
+ return undef if !defined($uid) || !$reservation;
+ return $uid
+ if !$reservation->{previous} || ($reservation->{'previous-pending-uid'} // '') eq $uid;
+
+ return undef;
+}
+
+# Returns whether $uid is the uid of the guest of the operation with the reservation $reservation,
+# stored or pending, before the operation. A guest without identity matches any uid.
+my sub uid_matches_target($reservation, $uid) {
+ my $identity = $reservation->{previous};
+
+ return
+ !$identity
+ || $identity->{uid} eq $uid
+ || ($reservation->{'previous-pending-uid'} // '') eq $uid;
+}
+
+# Checks $backup_uid, the uid of a backup or undef if it has none, against the guest $vmid of the
+# config class $class, which a restore with the reservation $reservation overwrites. If the guest
+# had another uid before the restore, see uid_matches_target, the backup most likely belongs to
+# another guest: this dies, unless $allow_mismatch is set, in which case the guest keeps its uid and
+# this only logs that. Must run before the restore changes the guest.
+sub assert_backup_matches_target($class, $vmid, $reservation, $backup_uid, $allow_mismatch) {
+ my $target = $reservation ? $reservation->{previous} : undef;
+ return if !$target || !defined($backup_uid) || uid_matches_target($reservation, $backup_uid);
+
+ my $guest_type = $class->guest_type();
+ if (!$allow_mismatch) {
+ die "backup belongs to another guest: its uid '$backup_uid' differs from uid"
+ . " '$target->{uid}' of the existing $guest_type $vmid - use"
+ . " 'allow-identity-mismatch' to overwrite the $guest_type anyway\n";
+ }
+
+ print "guest identity: ignoring backup uid $backup_uid, $guest_type $vmid keeps uid"
+ . " $target->{uid}\n";
+
+ return;
+}
+
+my sub take_over($class, $vmid, $uid, $reservation) {
+ my $outcome = sub($result, $current_uid, @extra) {
+ return { result => $result, uid => $current_uid, requested => $uid, @extra };
+ };
+
+ my $previous = $reservation->{previous};
+ return $outcome->('kept', $previous->{uid}) if $previous && !adoptable_uid($reservation, $uid);
+
+ my $assigned_uid = $reservation->{assigned}->{uid};
+ return $outcome->('skipped', $assigned_uid) if !defined($uid);
+
+ # shared by all guest types, as they share the VMID and uid namespaces
+ my $res = PVE::Cluster::cfs_lock_domain(
+ 'guest-identity',
+ undef,
+ sub {
+ my $metas = PVE::Cluster::get_guest_config_properties(['meta']);
+
+ my $current = parse_identity($metas->{$vmid} ? $metas->{$vmid}->{meta} : undef);
+ return $outcome->('kept', $uid) if $current && $current->{uid} eq $uid;
+
+ for my $other (sort { $a <=> $b } keys %$metas) {
+ my $identity = parse_identity($metas->{$other}->{meta});
+ next if !$identity || $identity->{uid} ne $uid;
+
+ my $current_uid = $current ? $current->{uid} : undef;
+ return $outcome->('in-use', $current_uid, 'in-use-by' => $other);
+ }
+
+ $class->__identity_replace($vmid, new_identity($uid));
+
+ return $outcome->('adopted', $uid);
+ },
+ );
+ die $@ if $@;
+
+ return $res;
+}
+
+# Makes $uid, for example the one of a restored backup or of the source guest of a remote
+# migration, the uid of the guest $vmid, with the current time as rtime, where the rules below
+# allow that.
+#
+# $class is the config class of the guest and must have guest identities enabled. $reservation is
+# what create_and_lock_config returned for the operation. The caller must still hold the config
+# lock of $vmid. A restore calls this after its final config write, as its writes keep the pending
+# uid until then. $uid can be undef if there is nothing to adopt, for example for a backup without
+# uid.
+#
+# A guest that had an identity before the operation keeps it, unless it had $uid as pending uid,
+# see adoptable_uid. Otherwise, $uid replaces the identity that the guest has since the
+# reservation, but only if no other guest in the cluster has $uid.
+#
+# Each result listed below is final, so adopt then removes the pending uid from the config. If the
+# cluster-wide identity lock cannot be taken, for example without quorum, there is no result: adopt
+# dies and the pending uid stays, so that a later operation can still take it over.
+#
+# Returns a hash with the keys 'result', 'uid' (the uid of $vmid afterwards), 'requested' ($uid)
+# and, for the result 'in-use', 'in-use-by' (the lowest VMID of the other guests with $uid). The
+# result is one of:
+# kept - the guest had an identity before the operation and keeps it, or it already has $uid
+# adopted - the guest now has $uid
+# in-use - another guest has $uid, so the guest keeps the identity of the reservation
+# skipped - $uid is undef, so the guest keeps the identity of the reservation
+sub adopt($class, $vmid, $uid, $reservation) {
+ die "guest identities are not enabled for this guest type\n" if !$class->__identity_enabled();
+ die "missing guest identity reservation\n"
+ if ref($reservation) ne 'HASH' || !$reservation->{assigned};
+ die "invalid guest identity uid '$uid'\n" if defined($uid) && !is_valid_uid($uid);
+
+ my $res = take_over($class, $vmid, $uid, $reservation);
+
+ # the config write that takes over a uid removes the pending uid, too
+ if ($res->{result} ne 'adopted') {
+ eval { $class->record_pending_uid($vmid, undef) };
+ die "unable to remove the pending uid - $@" if $@;
+ }
+
+ return $res;
+}
+
+# Returns the task log line for the result $res of adopt(), so that all guest types use the same
+# wording. $origin is where the requested uid comes from, either 'backup' or 'source' for the
+# source guest of a remote migration.
+sub adopt_result_message($res, $origin) {
+ my $origin_name = { backup => 'backup', source => 'source guest' }->{$origin}
+ // die "unknown origin of guest identity '$origin'\n";
+
+ my ($result, $requested) = $res->@{qw(result requested)};
+ my $uid = $res->{uid} // 'none';
+
+ if ($result eq 'kept') {
+ return "guest identity: kept uid $uid";
+ } elsif ($result eq 'adopted') {
+ return "guest identity: adopted uid $uid from the $origin_name";
+ } elsif ($result eq 'in-use') {
+ return "guest identity: $origin_name uid $requested is in use by guest"
+ . " $res->{'in-use-by'}, keeping uid $uid";
+ } elsif ($result eq 'skipped') {
+ return "guest identity: new uid $uid";
+ }
+
+ die "unknown result of guest identity adoption '$result'\n";
+}
+
+# Calls adopt for $uid, the uid of the backup or undef if the restore does not take one over, as
+# the last step of a restore, and logs the result. A failure only warns, as the guest itself got
+# restored and keeps the identity of its reservation.
+sub adopt_from_backup($class, $vmid, $uid, $reservation) {
+ my $res = eval { adopt($class, $vmid, $uid, $reservation) };
+ if (my $err = $@) {
+ chomp($err);
+ my $kept = $reservation->{assigned}->{uid} // 'none';
+ my $msg =
+ defined($uid)
+ ? "unable to adopt backup uid $uid, keeping uid $kept"
+ : "keeping uid $kept";
+ PVE::RESTEnvironment::log_warn("guest identity: $msg - $err");
+ return;
+ }
+
+ print adopt_result_message($res, 'backup') . "\n";
+
+ return;
+}
+
+# Calls adopt for $uid, the uid of the source guest of an incoming remote migration, on the target
+# right after the reservation, and logs the result. Refuses the migration with an error if another
+# guest in the cluster has $uid, unless $allow_new_uid is set, in which case the guest keeps the uid
+# of its reservation. Returns the result of adopt, for the source to log.
+sub adopt_from_source($class, $vmid, $uid, $reservation, $allow_new_uid) {
+ my $res = eval { adopt($class, $vmid, $uid, $reservation) };
+ die "unable to adopt the uid of the source guest - $@" if $@;
+
+ if ($res->{result} eq 'in-use' && !$allow_new_uid) {
+ die "guest $res->{'in-use-by'} in the target cluster already has the uid '$uid' of this"
+ . " guest - use 'allow-new-uid' to migrate it with a new uid\n";
+ }
+
+ print adopt_result_message($res, 'source') . "\n";
+
+ return $res;
+}
+
+# Returns the log level and line for the result $res of adopt that the target of a remote migration
+# reported for the uid of the source guest. A target with a newer version can report results
+# unknown here, which only get a warning, as the target accepted the uid by then.
+sub remote_adopt_result_log($res) {
+ my $message = eval { adopt_result_message($res, 'source') };
+ if (my $err = $@) {
+ chomp($err);
+ return ('warn', $err);
+ }
+
+ return ('info', $message);
+}
+
# Returns undef if $node runs a version of Proxmox VE that keeps the identity when writing guest
# configs, see MIN_PVE_MANAGER_VERSION. Missing or invalid version information counts as too old.
# Otherwise returns a hash with 'required-version', the minimum version, and 'version', the version
diff --git a/src/tests/config-identity-tests.pl b/src/tests/config-identity-tests.pl
index 28cb1a3..62e8692 100755
--- a/src/tests/config-identity-tests.pl
+++ b/src/tests/config-identity-tests.pl
@@ -76,16 +76,22 @@ my $cache = {};
our $get_config_error;
our $check_identity_lock = 1;
+our $lock_domain_error;
+our $domain_locked = 0;
+our $on_lookup;
my $reads = 0;
my $writes = 0;
+my $domain_writes = 0;
sub parse_test_config($raw) {
my $conf = { snapshots => {} };
my $section = $conf;
for my $line (split(/\n/, $raw)) {
next if $line =~ m/^\s*$/;
- if ($line =~ m/^\[([^\]]+)\]$/) {
+ if ($line =~ m/^\[PENDING\]$/) {
+ $section = $conf->{pending} = {};
+ } elsif ($line =~ m/^\[([^\]]+)\]$/) {
$section = $conf->{snapshots}->{$1} = {};
} elsif ($line =~ m/^([a-z][a-z0-9_\-]*):\s*(.*?)\s*$/) {
$section->{$1} = $2;
@@ -105,6 +111,8 @@ sub write_test_config($conf) {
};
my $raw = $print_section->($conf);
+ $raw .= "\n[PENDING]\n" . $print_section->($conf->{pending})
+ if scalar(keys(($conf->{pending} // {})->%*));
for my $snapname (sort keys(($conf->{snapshots} // {})->%*)) {
$raw .= "\n[$snapname]\n" . $print_section->($conf->{snapshots}->{$snapname});
}
@@ -147,6 +155,7 @@ $cluster_module->redefine(
die "test error: config of $vmid written without identity lock\n"
if $check_identity_lock && !identity_lock_held($vmid);
$writes++;
+ $domain_writes++ if $domain_locked;
$files->{$path} = write_test_config($conf);
$versions->{$path} = ++$version_counter;
$cache->{$path}->{version} = undef if $cache->{$path};
@@ -157,6 +166,39 @@ $cluster_module->redefine(
return undef if $noerr;
die "guest $vmid already exists\n";
},
+ get_guest_config_properties => sub {
+ my ($properties, $vmid) = @_;
+ die "test error: unexpected arguments\n" if defined($vmid) || "@$properties" ne 'meta';
+
+ my $res = {};
+ for my $path (grep { defined($files->{$_}) } keys %$files) {
+ for my $line (split(/\n/, $files->{$path})) {
+ last if $line =~ m/^\[/;
+ # like pmxcfs, a line with nothing after the colon does not count
+ $res->{ vmid_of_path($path) }->{meta} = $1 =~ s/^\s+|\s+$//gr
+ if $line =~ m/^meta:(.+)$/;
+ }
+ }
+
+ $on_lookup->() if $on_lookup;
+
+ return $res;
+ },
+ cfs_lock_domain => sub {
+ my ($domain, $timeout, $code, @param) = @_;
+ die "test error: unexpected lock domain '$domain'\n" if $domain ne 'guest-identity';
+ die "test error: nested domain lock\n" if $domain_locked;
+
+ if ($lock_domain_error) {
+ $@ = $lock_domain_error;
+ return undef;
+ }
+
+ local $domain_locked = 1;
+ PVE::Cluster::cfs_update();
+ # like the real one, errors of $code are passed on in $@
+ return eval { $code->(@param) };
+ },
get_config => sub {
my ($path) = @_;
my $vmid = vmid_of_path($path);
@@ -394,7 +436,12 @@ subtest 'create and lock config' => sub {
ok($reserved, 'free VMID: reservation gets an identity');
is_deeply(
$state,
- { existed => 0, previous => undef, assigned => $reserved },
+ {
+ existed => 0,
+ previous => undef,
+ 'previous-pending-uid' => undef,
+ assigned => $reserved,
+ },
'free VMID: no previous identity',
);
is(conf_of($class, 150)->{lock}, 'create', 'free VMID: reserved');
@@ -408,7 +455,12 @@ subtest 'create and lock config' => sub {
$state = $class->create_and_lock_config(151, 1, 'restore');
is_deeply(
$state,
- { existed => 1, previous => $stored, assigned => $stored },
+ {
+ existed => 1,
+ previous => $stored,
+ 'previous-pending-uid' => undef,
+ assigned => $stored,
+ },
'in place: previous identity',
);
my $conf = conf_of($class, 151);
@@ -423,7 +475,12 @@ subtest 'create and lock config' => sub {
$state = $class->create_and_lock_config(152, 1);
is_deeply(
$state,
- { existed => 1, previous => undef, assigned => identity_of($class, 152) },
+ {
+ existed => 1,
+ previous => undef,
+ 'previous-pending-uid' => undef,
+ assigned => identity_of($class, 152),
+ },
'in place without identity: reservation gets one',
);
ok($state->{assigned}, 'in place without identity: reserved identity');
@@ -432,7 +489,12 @@ subtest 'create and lock config' => sub {
$state = $class->create_and_lock_config(157, 1);
is_deeply(
$state,
- { existed => 1, previous => undef, assigned => identity_of($class, 157) },
+ {
+ existed => 1,
+ previous => undef,
+ 'previous-pending-uid' => undef,
+ assigned => identity_of($class, 157),
+ },
'in place with empty config: reservation gets an identity',
);
ok($state->{assigned}, 'in place with empty config: reserved identity');
@@ -469,7 +531,7 @@ subtest 'create and lock config' => sub {
$state = $class->create_and_lock_config(154);
is_deeply(
$state,
- { existed => 0, previous => undef, assigned => undef },
+ { existed => 0, previous => undef, 'previous-pending-uid' => undef, assigned => undef },
'disabled, free VMID: no previous identity',
);
is(file_of($class, 154), "lock: create\n", 'disabled, free VMID: no identity assigned');
@@ -478,7 +540,12 @@ subtest 'create and lock config' => sub {
$state = $class->create_and_lock_config(155, 1);
is_deeply(
$state,
- { existed => 1, previous => { uid => $foreign_uid, rtime => 5 }, assigned => undef },
+ {
+ existed => 1,
+ previous => { uid => $foreign_uid, rtime => 5 },
+ 'previous-pending-uid' => undef,
+ assigned => undef,
+ },
'disabled, in place: identity from the config',
);
is(
@@ -488,4 +555,629 @@ subtest 'create and lock config' => sub {
);
};
+subtest 'create and lock config with unique and enforced guest IDs' => sub {
+ my $class = 'PVE::TestConfig';
+
+ # PVE::GuestID imported the functions it uses to access its files, so fake them there, with the
+ # parser and writer it registered for the list of used guest IDs
+ my (undef, $id_list) = PVE::Cluster::cfs_file_version('virtual-guest/used-guest-ids');
+ my $used_ids = "500\n";
+ my $dc_conf = { 'next-id' => { unique => 1, enforce => 1 } };
+ $guest_id_module->unmock('register_used_id');
+ $guest_id_module->redefine(
+ cfs_read_file => sub($filename) {
+ return $dc_conf if $filename eq 'datacenter.cfg';
+ die "test error: unexpected read of '$filename'\n"
+ if $filename ne 'virtual-guest/used-guest-ids';
+ return $id_list->{parser}->("/etc/pve/$filename", $used_ids);
+ },
+ cfs_write_file => sub($filename, $ranges) {
+ $used_ids = $id_list->{writer}->("/etc/pve/$filename", $ranges);
+ },
+ cfs_lock_file => sub($filename, $timeout, $code) {
+ return eval { $code->() };
+ },
+ );
+
+ eval { $class->create_and_lock_config(500) };
+ is($@, "guest ID 500 was used before\n", 'used VMID is refused');
+ ok(!defined(file_of($class, 500)), 'refused VMID: no config is left behind');
+
+ my $created = $class->create_and_lock_config(510);
+ $class->write_config(510, { name => 'existing' });
+ my $state = $class->create_and_lock_config(510, 1, 'restore');
+ is_deeply(
+ $state,
+ {
+ existed => 1,
+ previous => $created->{assigned},
+ 'previous-pending-uid' => undef,
+ assigned => $created->{assigned},
+ },
+ 'existing guest: reservation is accepted and returns its identity',
+ );
+ is(conf_of($class, 510)->{lock}, 'restore', 'existing guest: locked');
+
+ $class->create_and_lock_config(520);
+ is($used_ids, "500\n510\n520\n", 'new reservations are recorded as used');
+
+ # like a failed restore that releases the VMID it reserved
+ my $path = $class->cfs_config_path(520);
+ delete $files->{$path};
+ $versions->{$path} = ++$version_counter;
+
+ for my $force (0, 1) {
+ eval { $class->create_and_lock_config(520, $force) };
+ is($@, "guest ID 520 was used before\n", "released VMID is refused (force: $force)");
+ }
+ ok(!defined(file_of($class, 520)), 'released VMID: no config is left behind');
+
+ $guest_id_module->unmock(qw(cfs_read_file cfs_write_file cfs_lock_file));
+ $guest_id_module->redefine(register_used_id => sub { });
+};
+
+subtest 'adopt' => sub {
+ my $class = 'PVE::TestConfig';
+ my $ct_class = 'PVE::TestConfig::CT';
+
+ my $backup_uid = new_uid();
+
+ # restore to a free VMID, the final write carries the identity of the backup
+ my $reservation = $class->create_and_lock_config(200);
+ my $restored = { name => 'restored', meta => "ctime=1,rtime=5,uid=$backup_uid" };
+ $class->write_config(200, { $restored->%*, lock => 'create' });
+ my $generated = identity_of($class, 200);
+ is_deeply($generated, $reservation->{assigned}, 'final write keeps the reserved identity');
+
+ $now += 1000;
+ my $res = PVE::GuestIdentity::adopt($class, 200, $backup_uid, $reservation);
+ is_deeply(
+ $res,
+ { result => 'adopted', uid => $backup_uid, requested => $backup_uid },
+ 'free uid is adopted',
+ );
+ is_deeply(
+ identity_of($class, 200),
+ { uid => $backup_uid, rtime => $now },
+ 'adopted uid is stored with the current time',
+ );
+ like(
+ file_of($class, 200),
+ qr/^meta: ctime=1,rtime=$now,uid=$backup_uid$/m,
+ 'other members are kept',
+ );
+ is(conf_of($class, 200)->{lock}, 'create', 'rest of the config is kept');
+ is($domain_writes, 1, 'stored under the cluster-wide lock');
+
+ my $adopted_rtime = $now;
+ $now += 1000;
+
+ my $stale = { $restored->%*, meta => "ctime=1,rtime=5,uid=$generated->{uid}" };
+ $class->write_config(200, $stale);
+ is_deeply(
+ identity_of($class, 200),
+ { uid => $backup_uid, rtime => $adopted_rtime },
+ 'later write of a stale hash keeps the adopted identity',
+ );
+
+ my $writes_before = $writes;
+ $res = PVE::GuestIdentity::adopt($class, 200, $backup_uid, $reservation);
+ is_deeply(
+ $res,
+ { result => 'kept', uid => $backup_uid, requested => $backup_uid },
+ 'repeated adoption keeps it',
+ );
+ is($writes, $writes_before, 'repeated adoption writes nothing');
+ is(identity_of($class, 200)->{rtime}, $adopted_rtime, 'repeated adoption keeps rtime');
+
+ # restore in place onto a guest with identity
+ $class->write_config(210, { name => 'target' });
+ my $target = identity_of($class, 210);
+ $reservation = $class->create_and_lock_config(210, 1, 'restore');
+ my $other_uid = new_uid();
+ {
+ local $lock_domain_error = "test error: unexpected lock\n";
+ $res = PVE::GuestIdentity::adopt($class, 210, $other_uid, $reservation);
+ }
+ is_deeply(
+ $res,
+ { result => 'kept', uid => $target->{uid}, requested => $other_uid },
+ 'in place: identity from before the operation is kept',
+ );
+ is_deeply(identity_of($class, 210), $target, 'in place: identity is unchanged');
+
+ set_file($class, 211, "meta: ctime=1\nname: old\n");
+ $reservation = $class->create_and_lock_config(211, 1);
+ $other_uid = new_uid();
+ $res = PVE::GuestIdentity::adopt($class, 211, $other_uid, $reservation);
+ is($res->{result}, 'adopted', 'in place without identity: uid is adopted');
+ is(identity_of($class, 211)->{uid}, $other_uid, 'in place without identity: stored');
+
+ $reservation = $class->create_and_lock_config(212);
+ {
+ local $lock_domain_error = "test error: unexpected lock\n";
+ $res = PVE::GuestIdentity::adopt($class, 212, undef, $reservation);
+ }
+ is_deeply(
+ $res,
+ { result => 'skipped', uid => $reservation->{assigned}->{uid}, requested => undef },
+ 'nothing to adopt',
+ );
+
+ $reservation = $class->create_and_lock_config(201);
+ $class->write_config(201, { name => 'second restore', lock => 'create' });
+ my $second = identity_of($class, 201);
+ $writes_before = $writes;
+ $res = PVE::GuestIdentity::adopt($class, 201, $backup_uid, $reservation);
+ is_deeply(
+ $res,
+ {
+ result => 'in-use',
+ uid => $second->{uid},
+ requested => $backup_uid,
+ 'in-use-by' => 200,
+ },
+ 'uid of another VM is not adopted',
+ );
+ is_deeply(identity_of($class, 201), $second, 'reserved identity stays');
+ is($writes, $writes_before, 'nothing is written');
+
+ # manual copies of a config
+ my $copied_uid = new_uid();
+ set_file($class, 220, "meta: rtime=1,uid=$copied_uid\n");
+ set_file($ct_class, 205, "meta: rtime=1,uid=$copied_uid\n");
+ $res = PVE::GuestIdentity::adopt($class, 201, $copied_uid, $reservation);
+ is($res->{'in-use-by'}, 205, 'lowest VMID of several holders is reported');
+
+ my $hidden_uid = new_uid();
+ set_file($class, 221, "meta: rtime=1,uid=$hidden_uid\nmeta:\n");
+ $res = PVE::GuestIdentity::adopt($class, 201, $hidden_uid, $reservation);
+ is($res->{'in-use-by'}, 221, 'meta line without value does not hide the uid before it');
+
+ my $ct_uid = new_uid();
+ my $ct_reservation = $ct_class->create_and_lock_config(300);
+ $res = PVE::GuestIdentity::adopt($ct_class, 300, $ct_uid, $ct_reservation);
+ is($res->{result}, 'adopted', 'container adopts a free uid');
+ like(file_of($ct_class, 300), qr/^meta: rtime=$now,uid=$ct_uid$/m, 'container config');
+
+ $res = PVE::GuestIdentity::adopt($class, 201, $ct_uid, $reservation);
+ is($res->{'in-use-by'}, 300, 'uid of a container is not adopted for a VM');
+
+ $ct_reservation = $ct_class->create_and_lock_config(301);
+ $res = PVE::GuestIdentity::adopt($ct_class, 301, $backup_uid, $ct_reservation);
+ is($res->{'in-use-by'}, 200, 'uid of a VM is not adopted for a container');
+
+ # another process writes the config after the cluster-wide lookup
+ $reservation = $class->create_and_lock_config(230);
+ $class->write_config(230, { name => 'restored', lock => 'create' });
+ PVE::Cluster::cfs_update();
+ $class->load_config(230);
+ {
+ local $on_lookup = sub {
+ set_file($class, 230, file_of($class, 230) =~ s/^name: .*$/name: hook/mr);
+ };
+ $res = PVE::GuestIdentity::adopt($class, 230, new_uid(), $reservation);
+ }
+ is($res->{result}, 'adopted', 'concurrent write: uid is adopted');
+ is(conf_of($class, 230)->{name}, 'hook', 'concurrent write: its change is kept');
+
+ $reservation = $class->create_and_lock_config(202);
+ my $before = identity_of($class, 202);
+ {
+ local $lock_domain_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+ eval { PVE::GuestIdentity::adopt($class, 202, new_uid(), $reservation) };
+ is($@, $lock_domain_error, 'lock error is passed on');
+ is_deeply(identity_of($class, 202), $before, 'lock error: identity is unchanged');
+ }
+
+ my $missing =
+ { existed => 0, previous => undef, assigned => PVE::GuestIdentity::new_identity() };
+ eval { PVE::GuestIdentity::adopt($class, 299, new_uid(), $missing) };
+ like($@, qr/does not exist/, 'missing config is an error');
+
+ eval { PVE::GuestIdentity::adopt($class, 202, 'GAYTEMZUGU3DOOBZMFRG', $reservation) };
+ like($@, qr/invalid guest identity uid/, 'invalid uid is refused');
+
+ for my $invalid (undef, {}, { existed => 0, previous => undef, assigned => undef }) {
+ eval { PVE::GuestIdentity::adopt($class, 202, new_uid(), $invalid) };
+ like($@, qr/missing guest identity reservation/, 'missing reservation is refused');
+ }
+
+ local $check_identity_lock = 0;
+ eval {
+ PVE::GuestIdentity::adopt('PVE::TestConfig::NoIdentity', 202, new_uid(), $reservation);
+ };
+ like($@, qr/not enabled/, 'guest type without identities is refused');
+};
+
+subtest 'adopt from backup' => sub {
+ my $class = 'PVE::TestConfig';
+
+ my $warnings = [];
+ my $restenv_module = Test::MockModule->new('PVE::RESTEnvironment');
+ $restenv_module->redefine(log_warn => sub($msg) { push $warnings->@*, $msg; });
+
+ my $adopt_from_backup = sub($vmid, $uid, $reservation) {
+ my $output = '';
+ open(my $fh, '>', \$output) or die "unable to capture output - $!\n";
+ my $previous_fh = select($fh);
+ eval { PVE::GuestIdentity::adopt_from_backup($class, $vmid, $uid, $reservation) };
+ my $err = $@;
+ select($previous_fh);
+ close($fh);
+ die $err if $err;
+ return $output;
+ };
+
+ my $backup_uid = new_uid();
+ my $reservation = $class->create_and_lock_config(240);
+ is(
+ $adopt_from_backup->(240, $backup_uid, $reservation),
+ "guest identity: adopted uid $backup_uid from the backup\n",
+ 'result is logged',
+ );
+ is(identity_of($class, 240)->{uid}, $backup_uid, 'uid is adopted');
+
+ $reservation = $class->create_and_lock_config(241);
+ my $assigned_uid = $reservation->{assigned}->{uid};
+ $backup_uid = new_uid();
+ $class->record_pending_uid(241, $backup_uid);
+ {
+ local $lock_domain_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+ is($adopt_from_backup->(241, $backup_uid, $reservation), '', 'failure: no result');
+ }
+ is_deeply(
+ $warnings,
+ [
+ "guest identity: unable to adopt backup uid $backup_uid, keeping uid $assigned_uid"
+ . " - cfs-lock 'domain-guest-identity' error: no quorum!",
+ ],
+ 'failure only warns',
+ );
+
+ $warnings = [];
+ {
+ local $get_config_error = "Connection refused\n";
+ is($adopt_from_backup->(241, undef, $reservation), '', 'nothing to adopt, failure');
+ }
+ is_deeply(
+ $warnings,
+ [
+ "guest identity: keeping uid $assigned_uid - unable to remove the pending uid -"
+ . " Connection refused",
+ ],
+ 'nothing to adopt, failure: warning without requested uid',
+ );
+};
+
+subtest 'adopt from source' => sub {
+ my $class = 'PVE::TestConfig::CT';
+
+ # returns the error, the output and the result
+ my $adopt_from_source = sub($vmid, $uid, $reservation, $allow_new_uid) {
+ my $output = '';
+ open(my $fh, '>', \$output) or die "unable to capture output - $!\n";
+ my $previous_fh = select($fh);
+ my $res = eval {
+ PVE::GuestIdentity::adopt_from_source(
+ $class, $vmid, $uid, $reservation, $allow_new_uid,
+ );
+ };
+ my $err = $@;
+ select($previous_fh);
+ close($fh);
+ return ($err, $output, $res);
+ };
+
+ my $source_uid = new_uid();
+ my $reservation = $class->create_and_lock_config(250);
+ my ($err, $output, $res) = $adopt_from_source->(250, $source_uid, $reservation, 0);
+ is($err, '', 'free uid: no error');
+ is_deeply(
+ $res,
+ { result => 'adopted', uid => $source_uid, requested => $source_uid },
+ 'free uid: adopted',
+ );
+ is($output, "guest identity: adopted uid $source_uid from the source guest\n", 'logged');
+ is(identity_of($class, 250)->{uid}, $source_uid, 'free uid: stored');
+
+ $reservation = $class->create_and_lock_config(251);
+ my $reserved = identity_of($class, 251);
+ my $writes_before = $writes;
+ ($err, $output) = $adopt_from_source->(251, $source_uid, $reservation, 0);
+ is(
+ $err,
+ "guest 250 in the target cluster already has the uid '$source_uid' of this guest - use"
+ . " 'allow-new-uid' to migrate it with a new uid\n",
+ 'uid in use: refused',
+ );
+ is($output, '', 'uid in use: no result logged');
+ is_deeply(identity_of($class, 251), $reserved, 'uid in use: reserved identity stays');
+ is($writes, $writes_before, 'uid in use: nothing is written');
+
+ ($err, $output, $res) = $adopt_from_source->(251, $source_uid, $reservation, 1);
+ is($err, '', 'uid in use with allow-new-uid: no error');
+ is($res->{result}, 'in-use', 'uid in use with allow-new-uid: result');
+ is(
+ $output,
+ "guest identity: source guest uid $source_uid is in use by guest 250, keeping uid"
+ . " $reserved->{uid}\n",
+ 'uid in use with allow-new-uid: logged',
+ );
+ is_deeply(identity_of($class, 251), $reserved, 'uid in use with allow-new-uid: kept');
+
+ $reservation = $class->create_and_lock_config(252);
+ {
+ local $lock_domain_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+ ($err, $output) = $adopt_from_source->(252, new_uid(), $reservation, 1);
+ }
+ is(
+ $err,
+ "unable to adopt the uid of the source guest - cfs-lock 'domain-guest-identity' error:"
+ . " no quorum!\n",
+ 'lock error is passed on',
+ );
+};
+
+# Returns whether a restore with the reservation $reservation over the guest $vmid accepts a backup
+# with the uid $uid without 'allow-identity-mismatch'.
+my sub backup_matches($class, $vmid, $reservation, $uid) {
+ return eval {
+ PVE::GuestIdentity::assert_backup_matches_target($class, $vmid, $reservation, $uid, 0);
+ 1;
+ };
+}
+
+# like the guest that a failed operation leaves behind, with the lock removed by its error handling
+my sub unlock_config($class, $vmid) {
+ my $conf = conf_of($class, $vmid);
+ delete $conf->{lock};
+ $class->write_config($vmid, $conf);
+}
+
+my sub pending_of($class, $vmid) {
+ my $conf = conf_of($class, $vmid);
+ return $conf->{pending} ? $conf->{pending}->{meta} : undef;
+}
+
+subtest 'pending uid' => sub {
+ my $class = 'PVE::TestConfig';
+
+ # a restore to a free VMID records the uid of the backup, then fails and leaves the guest
+ my $backup_uid = new_uid();
+ my $first = $class->create_and_lock_config(400);
+ my $leftover = $first->{assigned};
+ is(
+ PVE::GuestIdentity::adoptable_uid($first, $backup_uid),
+ $backup_uid,
+ 'free VMID: adoptable',
+ );
+ $class->record_pending_uid(400, $backup_uid);
+ is(pending_of($class, 400), "uid=$backup_uid", 'pending uid is recorded');
+ is_deeply(identity_of($class, 400), $leftover, 'stored identity is kept');
+ is(conf_of($class, 400)->{lock}, 'create', 'rest of the config is kept');
+ unlock_config($class, 400);
+ is(pending_of($class, 400), "uid=$backup_uid", 'write of the loaded config keeps it');
+
+ # a retry at another VMID takes over the uid, as a pending uid is not in use
+ my $elsewhere = $class->create_and_lock_config(401);
+ my $res = PVE::GuestIdentity::adopt($class, 401, $backup_uid, $elsewhere);
+ is($res->{result}, 'adopted',
+ 'retry elsewhere: pending uid of another guest is not in use');
+ is(pending_of($class, 400), "uid=$backup_uid", 'retry elsewhere: leftover keeps it');
+
+ # a retry over the leftover is a restore of the same backup, but the uid is in use by now
+ my $retry = $class->create_and_lock_config(400, 1);
+ is_deeply(
+ $retry,
+ {
+ existed => 1,
+ previous => $leftover,
+ 'previous-pending-uid' => $backup_uid,
+ assigned => $leftover,
+ },
+ 'reservation reports the pending uid',
+ );
+ ok(backup_matches($class, 400, $retry, $backup_uid), 'pending uid matches');
+ ok(backup_matches($class, 400, $retry, $leftover->{uid}), 'stored uid matches');
+ ok(!backup_matches($class, 400, $retry, new_uid()), 'other uid does not match');
+ is(
+ PVE::GuestIdentity::adoptable_uid($retry, $backup_uid),
+ $backup_uid,
+ 'pending: adoptable',
+ );
+ $res = PVE::GuestIdentity::adopt($class, 400, $backup_uid, $retry);
+ is_deeply(
+ $res,
+ {
+ result => 'in-use',
+ uid => $leftover->{uid},
+ requested => $backup_uid,
+ 'in-use-by' => 401,
+ },
+ 'retry in place after the retry elsewhere: in use',
+ );
+ is_deeply(identity_of($class, 400), $leftover, 'in use: identity is kept');
+ ok(!defined(pending_of($class, 400)), 'in use: pending uid is removed');
+
+ # a failed restore over a guest without identity, then retried in place
+ set_file($class, 410, "name: old\n");
+ $first = $class->create_and_lock_config(410, 1);
+ $backup_uid = new_uid();
+ is(
+ PVE::GuestIdentity::adoptable_uid($first, $backup_uid),
+ $backup_uid,
+ 'no identity: adoptable',
+ );
+ $class->record_pending_uid(410, $backup_uid);
+ unlock_config($class, 410);
+
+ $retry = $class->create_and_lock_config(410, 1);
+ is($retry->{'previous-pending-uid'}, $backup_uid, 'retry in place: pending uid reported');
+ $class->write_config(
+ 410,
+ { name => 'restored', lock => 'create', pending => { meta => "uid=$backup_uid" } },
+ );
+ $now += 1000;
+ my $writes_before = $writes;
+ $res = PVE::GuestIdentity::adopt($class, 410, $backup_uid, $retry);
+ is_deeply(
+ $res,
+ { result => 'adopted', uid => $backup_uid, requested => $backup_uid },
+ 'retry in place: pending uid is adopted',
+ );
+ is_deeply(identity_of($class, 410), { uid => $backup_uid, rtime => $now }, 'stored');
+ ok(!defined(pending_of($class, 410)), 'retry in place: pending uid is removed');
+ is($writes, $writes_before + 1, 'in the same write');
+ is(conf_of($class, 410)->{name}, 'restored', 'rest of the config is kept');
+
+ # an in-place restore of another backup keeps the identity, with a pending uid of the earlier
+ # backup
+ set_file($class, 430, "name: old\n");
+ $first = $class->create_and_lock_config(430, 1);
+ my $earlier_uid = new_uid();
+ $class->record_pending_uid(430, $earlier_uid);
+ unlock_config($class, 430);
+ $retry = $class->create_and_lock_config(430, 1);
+ my $target = identity_of($class, 430);
+ my $other_uid = new_uid();
+ ok(!backup_matches($class, 430, $retry, $other_uid), 'other backup does not match');
+ ok(!defined(PVE::GuestIdentity::adoptable_uid($retry, $other_uid)), 'other backup: none');
+ {
+ local $lock_domain_error = "test error: unexpected lock\n";
+ $res = PVE::GuestIdentity::adopt($class, 430, $other_uid, $retry);
+ }
+ is($res->{result}, 'kept', 'other backup: identity is kept');
+ is_deeply(identity_of($class, 430), $target, 'other backup: identity is unchanged');
+ ok(!defined(pending_of($class, 430)), 'other backup: pending uid is removed');
+
+ # nothing to adopt, for example with 'unique'
+ $first = $class->create_and_lock_config(440);
+ $class->record_pending_uid(440, new_uid());
+ $res = PVE::GuestIdentity::adopt($class, 440, undef, $first);
+ is($res->{result}, 'skipped', 'nothing to adopt: skipped');
+ ok(!defined(pending_of($class, 440)), 'nothing to adopt: pending uid is removed');
+
+ # adopting fails
+ $first = $class->create_and_lock_config(450);
+ $backup_uid = new_uid();
+ $class->record_pending_uid(450, $backup_uid);
+ {
+ local $lock_domain_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+ eval { PVE::GuestIdentity::adopt($class, 450, $backup_uid, $first) };
+ is($@, $lock_domain_error, 'lock error is passed on');
+ }
+ is_deeply(identity_of($class, 450), $first->{assigned},
+ 'lock error: identity is unchanged');
+ is(
+ pending_of($class, 450),
+ "uid=$backup_uid",
+ 'lock error: nothing decided, pending uid stays',
+ );
+
+ unlock_config($class, 450);
+ $retry = $class->create_and_lock_config(450, 1);
+ $res = PVE::GuestIdentity::adopt($class, 450, $backup_uid, $retry);
+ is($res->{result}, 'adopted', 'lock error: a later operation takes the pending uid over');
+ ok(!defined(pending_of($class, 450)), 'lock error: then the pending uid is removed');
+
+ # removing the pending uid keeps a stored identity that the parser could not read
+ my $config_module = Test::MockModule->new('PVE::TestConfig', no_auto => 1);
+ $config_module->mock(
+ load_config => sub {
+ my $conf = $config_module->original('load_config')->(@_);
+ delete $conf->{meta};
+ return $conf;
+ },
+ );
+ $class->write_config(470, { name => 'target', meta => 'ctime=1' });
+ $target = identity_of($class, 470);
+ $retry = $class->create_and_lock_config(470, 1);
+ $class->record_pending_uid(470, new_uid());
+ {
+ local $lock_domain_error = "test error: unexpected lock\n";
+ $res = PVE::GuestIdentity::adopt($class, 470, new_uid(), $retry);
+ }
+ $config_module->unmock('load_config');
+ is($res->{result}, 'kept', 'unreadable meta: identity is kept');
+ ok(!defined(pending_of($class, 470)), 'unreadable meta: pending uid is removed');
+ is_deeply(identity_of($class, 470), $target, 'unreadable meta: stored identity is kept');
+
+ # removing the pending uid fails
+ $class->write_config(460, { name => 'target' });
+ $retry = $class->create_and_lock_config(460, 1);
+ $class->record_pending_uid(460, new_uid());
+ {
+ local $get_config_error = "Connection refused\n";
+ eval { PVE::GuestIdentity::adopt($class, 460, new_uid(), $retry) };
+ is($@, "unable to remove the pending uid - Connection refused\n", 'error is passed on');
+ }
+ ok(defined(pending_of($class, 460)), 'pending uid stays');
+
+ eval { $class->record_pending_uid(460, 'GAYTEMZUGU3DOOBZMFRG') };
+ like($@, qr/invalid guest identity uid/, 'invalid pending uid is refused');
+};
+
+subtest 'config view with pending changes' => sub {
+ my $class = 'PVE::TestConfig';
+
+ my $pending_uid = new_uid();
+ $class->write_config(480, { name => 'old', meta => 'ctime=1' });
+ my $meta = conf_of($class, 480)->{meta};
+ $class->record_pending_uid(480, $pending_uid);
+ my $conf = conf_of($class, 480);
+ $conf->{pending}->{name} = 'new';
+ $class->write_config(480, $conf);
+
+ PVE::Cluster::cfs_update();
+ $conf = $class->load_current_config(480);
+ is($conf->{name}, 'new', 'other pending change is shown');
+ is($conf->{meta}, $meta, 'pending uid is not shown as meta');
+ ok(!exists($conf->{pending}), 'no pending section');
+ is($class->load_current_config(480, 1)->{name}, 'old', 'current values only');
+
+ set_file($class, 481, "meta: $meta\nname: old\n\n[PENDING]\ndelete: meta\n");
+ PVE::Cluster::cfs_update();
+ is($class->load_current_config(481)->{meta}, $meta, 'pending deletion of meta not shown');
+
+ # a guest type without identities may apply pending changes of 'meta'
+ local $check_identity_lock = 0;
+ $class = 'PVE::TestConfig::NoIdentity';
+ set_file($class, 482, "meta: ctime=1\nname: old\n\n[PENDING]\nmeta: ctime=2\n");
+ PVE::Cluster::cfs_update();
+ is($class->load_current_config(482)->{meta}, 'ctime=2', 'without identities: shown');
+};
+
+subtest 'drop pending meta' => sub {
+ my $class = 'PVE::TestConfig';
+ my $uid = new_uid();
+
+ my $uid_dropped = "guest identity: dropping pending uid $uid of an unfinished restore - a"
+ . " retry now needs 'allow-identity-mismatch'\n";
+ my $ignored = "ignoring pending change of read-only option 'meta'\n";
+ my $tests = [
+ ['pending uid', { meta => "uid=$uid", name => 'new' }, { name => 'new' }, $uid_dropped],
+ ['other change', { meta => 'ctime=2', name => 'new' }, { name => 'new' }, $ignored],
+ ['deletion', { delete => 'meta,name' }, { delete => 'name' }, $ignored],
+ ['uid and deletion', { meta => "uid=$uid", delete => 'meta' }, {}, $uid_dropped],
+ ['no change of meta', { name => 'new' }, { name => 'new' }, ''],
+ ];
+
+ for my $test (@$tests) {
+ my ($name, $pending, $expected, $expected_warning) = @$test;
+ my $conf = { meta => 'ctime=1', name => 'old', pending => { $pending->%* } };
+
+ my $warnings = '';
+ local $SIG{__WARN__} = sub { $warnings .= $_[0]; };
+ $class->drop_pending_meta($conf);
+
+ is_deeply(
+ $conf,
+ { meta => 'ctime=1', name => 'old', pending => $expected },
+ "$name: only a change of meta is dropped",
+ );
+ is($warnings, $expected_warning, "$name: warning");
+ }
+};
+
done_testing();
diff --git a/src/tests/guest-identity-tests.pl b/src/tests/guest-identity-tests.pl
index 241e18c..c130ed8 100755
--- a/src/tests/guest-identity-tests.pl
+++ b/src/tests/guest-identity-tests.pl
@@ -13,6 +13,27 @@ use PVE::Tools;
use PVE::GuestIdentity;
+package PVE::TestGuest::VM {
+ sub guest_type { return 'VM'; }
+}
+
+package PVE::TestGuest::CT {
+ sub guest_type { return 'CT'; }
+}
+
+package main;
+
+# Returns whether a restore over the guest of the reservation $reservation accepts a backup with
+# the uid $uid without 'allow-identity-mismatch'.
+sub backup_matches($reservation, $uid) {
+ return eval {
+ PVE::GuestIdentity::assert_backup_matches_target(
+ 'PVE::TestGuest::VM', 100, $reservation, $uid, 0,
+ );
+ 1;
+ };
+}
+
subtest 'generation with fixed random bytes and time' => sub {
my $random_bytes = [];
my $time_ms = 1_700_000_000_123;
@@ -335,4 +356,175 @@ subtest 'node version check' => sub {
}
};
+subtest 'adopt result message' => sub {
+ my $uid = 'gaytemzugu3doobzmfrg';
+ my $new = 'aaaaaaaaaaaaaaaaaaaa';
+
+ my $tests = [
+ [{ result => 'kept', uid => $uid, requested => $new }, 'backup', "kept uid $uid"],
+ [
+ { result => 'adopted', uid => $uid, requested => $uid },
+ 'backup',
+ "adopted uid $uid from the backup",
+ ],
+ [
+ { result => 'adopted', uid => $uid, requested => $uid },
+ 'source',
+ "adopted uid $uid from the source guest",
+ ],
+ [
+ { result => 'in-use', uid => $new, requested => $uid, 'in-use-by' => 105 },
+ 'backup',
+ "backup uid $uid is in use by guest 105, keeping uid $new",
+ ],
+ [
+ { result => 'in-use', uid => $new, requested => $uid, 'in-use-by' => 105 },
+ 'source',
+ "source guest uid $uid is in use by guest 105, keeping uid $new",
+ ],
+ [{ result => 'skipped', uid => $new, requested => undef }, 'backup', "new uid $new"],
+ ];
+
+ for my $test (@$tests) {
+ my ($res, $origin, $expected) = @$test;
+ is(
+ PVE::GuestIdentity::adopt_result_message($res, $origin),
+ "guest identity: $expected",
+ "$res->{result} ($origin)",
+ );
+ }
+
+ eval {
+ PVE::GuestIdentity::adopt_result_message({ result => 'kept', uid => $uid }, 'other');
+ };
+ like($@, qr/unknown origin/, 'unknown origin is refused');
+
+ eval {
+ PVE::GuestIdentity::adopt_result_message({ result => 'bogus', uid => $uid }, 'backup');
+ };
+ like($@, qr/unknown result/, 'unknown result is refused');
+
+ is_deeply(
+ [PVE::GuestIdentity::remote_adopt_result_log({ result => 'adopted', uid => $uid })],
+ ['info', "guest identity: adopted uid $uid from the source guest"],
+ 'remote result: known result is logged as info',
+ );
+ is_deeply(
+ [PVE::GuestIdentity::remote_adopt_result_log({ result => 'bogus', uid => $uid })],
+ ['warn', "unknown result of guest identity adoption 'bogus'"],
+ 'remote result: unknown result is a warning',
+ );
+};
+
+subtest 'pending uid' => sub {
+ my $uid = 'gaytemzugu3doobzmfrg';
+ my $other = 'aaaaaaaaaaaaaaaaaaaa';
+
+ is(PVE::GuestIdentity::get_pending_uid({}), undef, 'none without pending section');
+ is(PVE::GuestIdentity::get_pending_uid({ pending => { memory => 1 } }), undef, 'none');
+ is(
+ PVE::GuestIdentity::get_pending_uid({ pending => { meta => 'uid=' . uc($uid) } }),
+ undef,
+ 'invalid pending uid is none',
+ );
+ is(
+ PVE::GuestIdentity::get_pending_uid({ pending => { meta => "ctime=1,uid=$uid" } }),
+ $uid,
+ 'other members are ignored',
+ );
+
+ my $conf = { meta => "rtime=1,uid=$other", pending => { memory => 1 } };
+ PVE::GuestIdentity::set_pending_uid($conf, $uid);
+ is_deeply(
+ $conf,
+ { meta => "rtime=1,uid=$other", pending => { memory => 1, meta => "uid=$uid" } },
+ 'set: only the uid member, other pending changes are kept',
+ );
+ is(PVE::GuestIdentity::get_pending_uid($conf), $uid, 'set: read back');
+
+ PVE::GuestIdentity::set_pending_uid($conf, undef);
+ is_deeply(
+ $conf,
+ { meta => "rtime=1,uid=$other", pending => { memory => 1 } },
+ 'remove: other pending changes are kept',
+ );
+ $conf = {};
+ PVE::GuestIdentity::set_pending_uid($conf, undef);
+ is_deeply($conf, {}, 'remove without pending section changes nothing');
+
+ eval { PVE::GuestIdentity::set_pending_uid({}, 'GAYTEMZUGU3DOOBZMFRG') };
+ like($@, qr/invalid guest identity uid/, 'invalid uid is refused');
+
+ my $reservation = { previous => undef, 'previous-pending-uid' => undef };
+ is(PVE::GuestIdentity::adoptable_uid($reservation, $uid), $uid, 'no identity: adoptable');
+ is(PVE::GuestIdentity::adoptable_uid($reservation, undef), undef, 'nothing to adopt');
+ is(PVE::GuestIdentity::adoptable_uid(undef, $uid), undef, 'no reservation: none');
+ ok(backup_matches($reservation, $uid), 'no identity: matches');
+
+ $reservation =
+ { previous => { uid => $other, rtime => 1 }, 'previous-pending-uid' => undef };
+ is(PVE::GuestIdentity::adoptable_uid($reservation, $uid), undef, 'identity: not adoptable');
+ is(PVE::GuestIdentity::adoptable_uid($reservation, $other), undef,
+ 'own uid: not adoptable');
+ ok(!backup_matches($reservation, $uid), 'identity: no match');
+ ok(backup_matches($reservation, $other), 'own uid: matches');
+
+ $reservation->{'previous-pending-uid'} = $uid;
+ is(PVE::GuestIdentity::adoptable_uid($reservation, $uid), $uid, 'pending uid: adoptable');
+ ok(backup_matches($reservation, $uid), 'pending uid: matches');
+ is(PVE::GuestIdentity::adoptable_uid($reservation, $other), undef, 'own uid still not');
+};
+
+subtest 'backup check' => sub {
+ my $uid = 'gaytemzugu3doobzmfrg';
+ my $other = 'aaaaaaaaaaaaaaaaaaaa';
+ my $reservation =
+ { previous => { uid => $other, rtime => 1 }, 'previous-pending-uid' => undef };
+
+ my $check = sub($class, $reservation, $backup_uid, $allow_mismatch) {
+ my $output = '';
+ open(my $fh, '>', \$output) or die "unable to capture output - $!\n";
+ my $previous_fh = select($fh);
+ eval {
+ PVE::GuestIdentity::assert_backup_matches_target(
+ $class, 100, $reservation, $backup_uid, $allow_mismatch,
+ );
+ };
+ my $err = $@;
+ select($previous_fh);
+ close($fh);
+ return ($err, $output);
+ };
+
+ for my $class (qw(PVE::TestGuest::VM PVE::TestGuest::CT)) {
+ my $type = $class->guest_type();
+
+ is_deeply(
+ [$check->($class, $reservation, $uid, 0)],
+ [
+ "backup belongs to another guest: its uid '$uid' differs from uid '$other' of the"
+ . " existing $type 100 - use 'allow-identity-mismatch' to overwrite the $type"
+ . " anyway\n",
+ '',
+ ],
+ "$type: backup of another guest is refused",
+ );
+ is_deeply(
+ [$check->($class, $reservation, $uid, 1)],
+ ['', "guest identity: ignoring backup uid $uid, $type 100 keeps uid $other\n"],
+ "$type: override is logged",
+ );
+ }
+
+ my $class = 'PVE::TestGuest::VM';
+ is_deeply([$check->($class, $reservation, $other, 0)], ['', ''], 'same uid: no output');
+ is_deeply([$check->($class, $reservation, undef, 0)], ['', ''], 'backup without uid');
+ is_deeply(
+ [$check->($class, { previous => undef }, $uid, 0)],
+ ['', ''],
+ 'target without identity',
+ );
+ is_deeply([$check->($class, undef, $uid, 0)], ['', ''], 'no reservation');
+};
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH guest-common v3 13/38] migrate: add shared helpers for the identity command of the migration tunnel
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (11 preceding siblings ...)
2026-11-18 4:37 ` [PATCH guest-common v3 12/38] guest identity: allow taking over a uid that is unused in the cluster Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 14/38] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
` (26 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
The source and the target of a remote migration exchange the uid of the
guest with the 'identity' tunnel command. VMs and containers would each
define the command, the rule that the target accepts it only once and
only before the config arrives, and the sender with its timeout and log
line, which all have to stay in sync with the next tunnel change.
Provide the parameter schema and the handler of the target in
PVE::GuestIdentity, as PVE::StorageTunnel does for the storage commands,
and the sender in PVE::AbstractMigrate, next to the sender of the
'bwlimit' command. The wire format stays the same.
If the parsed config of the source has no identity, the sender writes it
first. A guest without uid gets one that way, so the source keeps the
uid that it sends, instead of getting another one with a later write.
The write also restores a stored uid that the parser of the guest type
dropped, for example for a 'meta' member it does not know. Callers send
the command only after the version check of the tunnel, so that a
refused target leaves the source config unchanged.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* new in v3: command schema, target handler and sender moved here from
qemu-server and container
* the sender gives a guest without uid one before it sends it, and sends
the stored uid if the parser dropped 'meta'
src/PVE/AbstractMigrate.pm | 30 +++++
src/PVE/GuestIdentity.pm | 39 ++++++
src/tests/config-identity-tests.pl | 208 +++++++++++++++++++++++++++++
3 files changed, 277 insertions(+)
diff --git a/src/PVE/AbstractMigrate.pm b/src/PVE/AbstractMigrate.pm
index 919f12d..4375cb0 100644
--- a/src/PVE/AbstractMigrate.pm
+++ b/src/PVE/AbstractMigrate.pm
@@ -8,6 +8,7 @@ use POSIX qw(strftime);
use PVE::Cluster;
use PVE::DataCenterConfig;
+use PVE::GuestIdentity;
use PVE::ReplicationState;
use PVE::SSHInfo;
use PVE::Tools;
@@ -384,6 +385,35 @@ sub get_bwlimit {
return $bwlimit;
}
+# Lets the target of a remote migration take over the uid of the guest, whose config class is
+# $config_class, through the tunnel $tunnel, before anything gets transferred there. The target
+# refuses a uid that another guest there has, unless the migration allows a new uid, see
+# PVE::GuestIdentity::handle_tunnel_identity. Call it only after the version check of the tunnel,
+# so that a target that gets refused leaves the config of the source unchanged.
+sub take_over_identity_on_target {
+ my ($self, $config_class, $tunnel) = @_;
+
+ my $conf = $self->{vmconf};
+ my $identity = $config_class->get_identity($conf);
+ if (!$identity) {
+ # A guest without uid gets one first, so the source keeps the uid that it sends. The write
+ # also restores a stored uid that the parser of the guest type dropped.
+ $config_class->write_config($self->{vmid}, $conf);
+ $identity = $config_class->get_identity($conf) or return;
+ }
+
+ my $params = {
+ uid => $identity->{uid},
+ 'allow-new-uid' => $self->{opts}->{'allow-new-uid'} ? 1 : 0,
+ };
+
+ # the target waits for the cluster-wide guest identity lock
+ my $res = PVE::Tunnel::write_tunnel($tunnel, 60, 'identity', $params);
+ $self->log(PVE::GuestIdentity::remote_adopt_result_log($res->{identity})) if $res->{identity};
+
+ return;
+}
+
sub map_storage {
my ($self, $scfg, $storeid) = @_;
diff --git a/src/PVE/GuestIdentity.pm b/src/PVE/GuestIdentity.pm
index 8474c88..9c2de3d 100644
--- a/src/PVE/GuestIdentity.pm
+++ b/src/PVE/GuestIdentity.pm
@@ -412,6 +412,45 @@ sub remote_adopt_result_log($res) {
return ('info', $message);
}
+# Returns the properties of the parameters of the 'identity' command of the migration tunnel, see
+# handle_tunnel_identity. Every call returns a new copy.
+sub tunnel_identity_cmd_schema() {
+ return {
+ uid => {
+ type => 'string',
+ description => 'uid of the source guest',
+ },
+ 'allow-new-uid' => {
+ type => 'boolean',
+ description => 'go on with a new uid if another guest has the uid',
+ optional => 1,
+ default => 0,
+ },
+ };
+}
+
+# Handles the 'identity' command with the parameters $params on the target of a remote migration of
+# a guest of the config class $class: takes over the uid of the source guest with adopt_from_source
+# for the guest that the tunnel reserved with $reservation, and returns the reply for the source.
+# $state is the state of the tunnel, with the VMID in 'vmid' and the lock of the guest in 'lock'.
+# The command is only accepted once, and only before the config arrives, while the guest still has
+# the 'create' lock of the reservation.
+sub handle_tunnel_identity($class, $state, $reservation, $params) {
+ die "the uid can only be taken over once, before the config arrives\n"
+ if ($state->{lock} // '') ne 'create' || $state->{'identity-handled'};
+ $state->{'identity-handled'} = 1;
+
+ my $res = adopt_from_source(
+ $class,
+ $state->{vmid},
+ $params->{uid},
+ $reservation,
+ $params->{'allow-new-uid'},
+ );
+
+ return { identity => $res };
+}
+
# Returns undef if $node runs a version of Proxmox VE that keeps the identity when writing guest
# configs, see MIN_PVE_MANAGER_VERSION. Missing or invalid version information counts as too old.
# Otherwise returns a hash with 'required-version', the minimum version, and 'version', the version
diff --git a/src/tests/config-identity-tests.pl b/src/tests/config-identity-tests.pl
index 62e8692..ae8b75d 100755
--- a/src/tests/config-identity-tests.pl
+++ b/src/tests/config-identity-tests.pl
@@ -11,8 +11,10 @@ use Test::MockModule;
use Test::More;
use PVE::AbstractConfig;
+use PVE::AbstractMigrate;
use PVE::GuestHelpers;
use PVE::GuestIdentity;
+use PVE::JSONSchema;
my $tmpdir = tempdir('config-identity-XXXXXX', DIR => '.', CLEANUP => 1);
$PVE::GuestHelpers::lockdir = "$tmpdir/lock";
@@ -1180,4 +1182,210 @@ subtest 'drop pending meta' => sub {
}
};
+# Runs $code with the output to STDOUT captured, returns the error and the output.
+my sub capture_output($code) {
+ my $output = '';
+ open(my $fh, '>', \$output) or die "unable to capture output - $!\n";
+ my $previous_fh = select($fh);
+ eval { $code->() };
+ my $err = $@;
+ select($previous_fh);
+ close($fh);
+
+ return ($err, $output);
+}
+
+subtest 'tunnel identity command on the target' => sub {
+ my $class = 'PVE::TestConfig';
+
+ # validates the parameters like the tunnel endpoints do, returns the error, output and reply
+ my $handle = sub($state, $reservation, $params) {
+ my $schema = {
+ type => 'object',
+ properties => PVE::GuestIdentity::tunnel_identity_cmd_schema(),
+ additionalProperties => 0,
+ };
+ my $reply;
+ my ($err, $output) = capture_output(sub {
+ PVE::JSONSchema::validate($params, $schema);
+ $reply =
+ PVE::GuestIdentity::handle_tunnel_identity($class, $state, $reservation, $params);
+ });
+ return ($err, $output, $reply);
+ };
+ my $once = "the uid can only be taken over once, before the config arrives\n";
+
+ my $uid = new_uid();
+ my $reservation = $class->create_and_lock_config(600);
+ my $state = { vmid => 600, lock => 'create' };
+ my ($err, $output, $reply) = $handle->($state, $reservation, { uid => $uid });
+ is($err, '', 'free uid: accepted');
+ is_deeply(
+ $reply,
+ { identity => { result => 'adopted', uid => $uid, requested => $uid } },
+ 'free uid: reply',
+ );
+ is($output, "guest identity: adopted uid $uid from the source guest\n", 'free uid: logged');
+ is(identity_of($class, 600)->{uid}, $uid, 'free uid: stored');
+
+ ($err) = $handle->($state, $reservation, { uid => new_uid() });
+ is($err, $once, 'second command is refused');
+ is(identity_of($class, 600)->{uid}, $uid, 'second command: uid is unchanged');
+
+ $reservation = $class->create_and_lock_config(601);
+ my $reserved = identity_of($class, 601);
+ ($err) = $handle->({ vmid => 601, lock => 'migrate' }, $reservation, { uid => new_uid() });
+ is($err, $once, 'command after the config is refused');
+ is_deeply(identity_of($class, 601), $reserved, 'after the config: identity is kept');
+
+ ($err, $output) =
+ $handle->({ vmid => 601, lock => 'create' }, $reservation, { uid => $uid });
+ is(
+ $err,
+ "guest 600 in the target cluster already has the uid '$uid' of this guest - use"
+ . " 'allow-new-uid' to migrate it with a new uid\n",
+ 'uid in use: refused',
+ );
+ is($output, '', 'uid in use: no result logged');
+ is_deeply(identity_of($class, 601), $reserved, 'uid in use: reserved identity stays');
+
+ ($err, $output, $reply) = $handle->(
+ { vmid => 601, lock => 'create' },
+ $reservation,
+ { uid => $uid, 'allow-new-uid' => 1 },
+ );
+ is($err, '', 'uid in use with allow-new-uid: accepted');
+ is_deeply(
+ $reply,
+ {
+ identity => {
+ result => 'in-use',
+ uid => $reserved->{uid},
+ requested => $uid,
+ 'in-use-by' => 600,
+ },
+ },
+ 'uid in use with allow-new-uid: reply',
+ );
+ is_deeply(identity_of($class, 601), $reserved, 'uid in use with allow-new-uid: kept');
+
+ $reservation = $class->create_and_lock_config(602);
+ ($err) = $handle->({ vmid => 602, lock => 'create' }, $reservation, {});
+ like($err, qr/^uid: property is missing/m, 'missing uid is refused');
+ ($err) =
+ $handle->({ vmid => 602, lock => 'create' }, $reservation, { uid => $uid, x => 1 });
+ like($err, qr/^x: property is not defined in schema/m, 'unknown parameter is refused');
+
+ delete PVE::GuestIdentity::tunnel_identity_cmd_schema()->{uid};
+ ok(PVE::GuestIdentity::tunnel_identity_cmd_schema()->{uid}, 'schema is a new copy');
+};
+
+subtest 'tunnel identity command on the source' => sub {
+ my $class = 'PVE::TestConfig';
+
+ my $tunnel = { version => 3 };
+ my ($commands, $reply);
+ local *PVE::Tunnel::write_tunnel = sub {
+ my ($used_tunnel, $timeout, $command, $params) = @_;
+ push $commands->@*, [$used_tunnel, $timeout, $command, $params];
+ die $reply if !ref($reply);
+ return $reply;
+ };
+
+ # like the migration of the guest $vmid with the options $opts after its preparation loaded
+ # the config
+ my $migration = sub($vmid, $opts = {}) {
+ PVE::Cluster::cfs_update();
+ my $conf = $class->load_config($vmid);
+ return bless { vmid => $vmid, vmconf => $conf, opts => $opts }, 'PVE::AbstractMigrate';
+ };
+ # returns the error, the output, and the sent parameters
+ my $take_over = sub($name, $migrate) {
+ $commands = [];
+ my ($err, $output) =
+ capture_output(sub { $migrate->take_over_identity_on_target($class, $tunnel) });
+ my @sent = map { [$_->@[0 .. 2]] } $commands->@*;
+ is_deeply(\@sent, [[$tunnel, 60, 'identity']], "$name: one command, up to a minute");
+ return ($err, $output, $commands->[0]->[3]);
+ };
+
+ $class->write_config(610, { name => 'source' });
+ my $source = identity_of($class, 610);
+ $reply =
+ { identity => { result => 'adopted', uid => $source->{uid}, requested => $source->{uid} } };
+ my $writes_before = $writes;
+ my ($err, $output, $params) = $take_over->('source with uid', $migration->(610));
+ is($err, '', 'source with uid: no error');
+ is_deeply($params, { uid => $source->{uid}, 'allow-new-uid' => 0 },
+ 'source with uid: sent');
+ is($writes, $writes_before, 'source with uid: nothing is written');
+ like(
+ $output,
+ qr/^\S+ \S+ guest identity: adopted uid $source->{uid} from the source guest\n\z/,
+ 'source with uid: result is logged',
+ );
+
+ $reply = {
+ identity => {
+ result => 'in-use',
+ uid => 'targettargettargetta',
+ requested => $source->{uid},
+ 'in-use-by' => 7,
+ },
+ };
+ ($err, $output, $params) =
+ $take_over->('allow-new-uid', $migration->(610, { 'allow-new-uid' => 1 }));
+ is($params->{'allow-new-uid'}, 1, 'allow-new-uid is sent');
+ like(
+ $output,
+ qr/ guest identity: source guest uid $source->{uid} is in use by guest 7, keeping uid/,
+ 'uid in use on the target: result is logged',
+ );
+
+ set_file($class, 611, "name: source\n");
+ my $migrate = $migration->(611);
+ $reply = { success => 1 };
+ $writes_before = $writes;
+ ($err, $output, $params) = $take_over->('source without uid', $migrate);
+ my $assigned = identity_of($class, 611);
+ ok($assigned, 'source without uid: gets one');
+ is($writes, $writes_before + 1, 'source without uid: written once');
+ is($params->{uid}, $assigned->{uid}, 'source without uid: sends the uid it got');
+ is_deeply(
+ $class->get_identity($migrate->{vmconf}),
+ $assigned,
+ 'source without uid: loaded config has it',
+ );
+ is($output, '', 'source without uid: reply without result is not logged');
+
+ my $stored_uid = new_uid();
+ set_file($class, 612, "meta: ctime=1,rtime=5,uid=$stored_uid\nname: source\n");
+ $migrate = $migration->(612);
+ # like a parser that drops a 'meta' property with a member it does not know
+ delete $migrate->{vmconf}->{meta};
+ ($err, $output, $params) = $take_over->('stored uid only', $migrate);
+ is(
+ $params->{uid}, $stored_uid, 'stored uid only: sends the stored uid',
+ );
+ is_deeply(
+ identity_of($class, 612),
+ { uid => $stored_uid, rtime => 5 },
+ 'stored uid only: stored identity is kept',
+ );
+
+ $reply =
+ { identity => { result => 'future', uid => $stored_uid, requested => $stored_uid } };
+ ($err, $output) = $take_over->('unknown result', $migration->(612));
+ is($err, '', 'unknown result: no error');
+ like(
+ $output,
+ qr/ unknown result of guest identity adoption 'future'$/m,
+ 'unknown result: warning',
+ );
+
+ $reply = "error - tunnel command 'identity' failed - refused\n";
+ ($err, $output) = $take_over->('failed command', $migration->(612));
+ is($err, $reply, 'failed command: error is passed on');
+};
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH qemu-server v3 14/38] api: restore: release the VMID if a restore to an unused VMID fails
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (12 preceding siblings ...)
2026-11-18 4:37 ` [PATCH guest-common v3 13/38] migrate: add shared helpers for the identity command of the migration tunnel Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 15/38] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
` (25 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A failed restore to a previously unused VMID left its temporary or
empty config behind, even when the restore cleaned up the volumes it
had allocated. The VMID stayed taken and a retry needed 'force'.
Containers already remove their config in this case.
Remove the VM config and the firewall config that the restore wrote if
the VMID was unused before the operation and the restore has not
reached the point from which it keeps the restored volumes. Warn if the
removed firewall config had replaced one that existed for the VMID
before, for example one made by hand, as that one is gone. Keep the VM
if a volume that the restore allocated or the firewall config cannot be
removed, so that what is left behind keeps its VMID. Keep it as well
after later failures, such as a failure in the live phase of a live
restore. If the next-id setting enforces unique guest IDs, the released
VMID counts as used, so a retry needs another VMID, as for containers.
Remove the config before releasing the guest config lock, so that no
other operation can take the VMID in between and lose its config to
this cleanup. Remove the firewall config before the VM config for a
similar reason: the guest config lock is local to the node, so another
node can take the VMID once the VM config is gone. The lock is held, so
the VM cannot run unprotected in between.
Only warn that the VM was not cleaned up if the restore changed the
target and left it behind. A failure to read the backup should not
produce that warning.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* keep the VM if a restored volume or the restored firewall config
cannot be removed
* only remove a firewall config that the restore wrote, and warn if that
one had replaced a firewall config that existed for the free VMID
before
* tests: retry without force, and cases for the independent fixes before
the series ('force' and a VM created in between, lock of the temporary
VMA config)
changes v1 -> v2:
* remove the firewall config before the VM config, with a test for the
order
* note the effect with enforced unique guest IDs in the commit message
* test helper: use v5.36, mock the registration of used guest IDs
src/PVE/API2/Qemu.pm | 45 +-
src/PVE/QemuServer.pm | 56 ++-
src/test/QemuServerTest/FakeCFS.pm | 186 ++++++++
src/test/run_qemu_restore_config_tests.pl | 509 +++++++++++++++++++++-
4 files changed, 768 insertions(+), 28 deletions(-)
create mode 100644 src/test/QemuServerTest/FakeCFS.pm
diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index cad8eb06..1a3f2ce9 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -1423,10 +1423,19 @@ __PACKAGE__->register_method({
my $emsg = $is_restore ? "unable to restore VM $vmid -" : "unable to create VM $vmid -";
# 'force' only covers the VM that the checks above saw, not one created in the meantime
- eval { PVE::QemuConfig->create_and_lock_config($vmid, $force && $vm_exists) };
+ my $reservation =
+ eval { PVE::QemuConfig->create_and_lock_config($vmid, $force && $vm_exists) };
die "$emsg $@" if $@;
my $restored_data = 0;
+ # the restore paths record whether they changed the VM or wrote its firewall config, and
+ # whether restored volumes stay
+ my $restore_progress = {};
+ # a free VMID can have a firewall config, for example one made by hand
+ my $firewall_existed =
+ $is_restore
+ && !$reservation->{existed}
+ && PVE::QemuServer::firewall_config_exists($vmid);
my $restorefn = sub {
my $conf = PVE::QemuConfig->load_config($vmid);
@@ -1442,6 +1451,7 @@ __PACKAGE__->register_method({
bwlimit => $bwlimit,
live => $live_restore,
override_conf => $param,
+ progress => $restore_progress,
};
if (my $volid = $archive->{volid}) {
# best effort, real check is after restoring!
@@ -1499,7 +1509,34 @@ __PACKAGE__->register_method({
if $unique;
};
- PVE::QemuConfig->lock_config_full($vmid, 1, $realcmd);
+ my $restore_locked = sub {
+ eval { $realcmd->() };
+ my $err = $@ or return;
+
+ if (!$reservation->{existed} && !$restore_progress->{'volumes-kept'}) {
+ # Remove the reservation before releasing the guest lock, so another
+ # operation cannot take ownership first. The firewall config that the restore
+ # wrote goes first, before the VMID is freed. The lock is held, so the VM cannot
+ # run unprotected.
+ eval {
+ if ($restore_progress->{'firewall-restored'}) {
+ my $removed = PVE::Firewall::remove_vmfw_conf($vmid);
+ die "unable to remove the firewall config - $!\n"
+ if !$removed && !$!{ENOENT};
+ if ($removed && $firewall_existed) {
+ log_warn("removed the firewall config of VM $vmid, the failed"
+ . " restore had replaced the one that existed before");
+ }
+ }
+ PVE::QemuConfig->destroy_config($vmid);
+ $restore_progress->{'reservation-removed'} = 1;
+ };
+ warn $@ if $@;
+ }
+ die $err;
+ };
+
+ PVE::QemuConfig->lock_config_full($vmid, 1, $restore_locked);
if ($start_after_create && !$live_restore) {
print "Execute autostart\n";
@@ -1632,13 +1669,15 @@ __PACKAGE__->register_method({
$code = sub {
eval { $restorefn->() };
if (my $err = $@) {
+ die $err if $restore_progress->{'reservation-removed'};
+
eval { PVE::QemuConfig->remove_lock($vmid, 'create') };
warn $@ if $@;
if ($restored_data) {
warn
"error after data was restored, VM disks should be OK but config may "
. "require adaptions. VM $vmid state is NOT cleaned up.\n";
- } else {
+ } elsif ($restore_progress->{'target-changed'}) {
warn "error before or during data restore, some or all disks were not "
. "completely restored. VM $vmid state is NOT cleaned up.\n";
}
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 54c49b58..855ede59 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -6407,11 +6407,13 @@ sub tar_archive_read_firstfile {
return $firstfile;
}
+# Returns the number of volumes that could not be removed.
sub tar_restore_cleanup {
my ($storecfg, $statfile) = @_;
print STDERR "starting cleanup\n";
+ my $left = 0;
if (my $fd = IO::File->new($statfile, "r")) {
while (defined(my $line = <$fd>)) {
if ($line =~ m/vzdump:([^\s:]*):(\S+)$/) {
@@ -6424,13 +6426,18 @@ sub tar_restore_cleanup {
}
print STDERR "temporary volume '$volid' successfully removed\n";
};
- print STDERR "unable to cleanup '$volid' - $@" if $@;
+ if (my $err = $@) {
+ print STDERR "unable to cleanup '$volid' - $err";
+ $left++;
+ }
} else {
print STDERR "unable to parse line in statfile - $line";
}
}
$fd->close();
}
+
+ return $left;
}
sub restore_file_archive {
@@ -6697,17 +6704,24 @@ my $restore_deactivate_volumes = sub {
print STDERR $@ if $@;
};
+# Returns the number of volumes that could not be removed.
my $restore_destroy_volumes = sub {
my ($storecfg, $virtdev_hash) = @_;
+ my $left = 0;
for my $dev (values $virtdev_hash->%*) {
my $volid = $dev->{volid} or next;
eval {
PVE::Storage::vdisk_free($storecfg, $volid);
print STDERR "temporary volume '$volid' successfully removed\n";
};
- print STDERR "unable to cleanup '$volid' - $@" if $@;
+ if (my $err = $@) {
+ print STDERR "unable to cleanup '$volid' - $err";
+ $left++;
+ }
}
+
+ return $left;
};
sub restore_merge_config {
@@ -6877,6 +6891,13 @@ sub restore_firewall_config {
PVE::Tools::file_copy($path, "${pve_firewall_dir}/$vmid.fw");
}
+# Returns whether the VM $vmid has a firewall config.
+sub firewall_config_exists {
+ my ($vmid) = @_;
+
+ return -e "/etc/pve/firewall/$vmid.fw";
+}
+
sub restore_proxmox_backup_archive {
my ($archive, $vmid, $user, $options) = @_;
@@ -6960,12 +6981,14 @@ sub restore_proxmox_backup_archive {
$param = [$pbs_backup_name, "qemu-server.conf", $cfgfn];
PVE::Storage::PBSPlugin::run_raw_client_cmd($scfg, $storeid, $cmd, $param);
+ $options->{progress}->{'target-changed'} = 1;
PVE::ReplicationState::delete_guest_states($vmid);
if ($has_firewall_config) {
$param = [$pbs_backup_name, "fw.conf", $firewall_config_fn];
PVE::Storage::PBSPlugin::run_raw_client_cmd($scfg, $storeid, $cmd, $param);
+ $options->{progress}->{'firewall-restored'} = 1;
restore_firewall_config($vmid, $firewall_config_fn);
}
@@ -7049,9 +7072,12 @@ sub restore_proxmox_backup_archive {
rmtree $tmpdir;
if ($err) {
- $restore_destroy_volumes->($storecfg, $virtdev_hash);
+ # a volume that could not be removed stays with the VM
+ $options->{progress}->{'volumes-kept'} = 1
+ if $restore_destroy_volumes->($storecfg, $virtdev_hash);
die $err;
}
+ $options->{progress}->{'volumes-kept'} = 1;
if ($options->{live}) {
# keep lock during live-restore
@@ -7139,12 +7165,14 @@ sub restore_external_archive {
my $data = $backup_provider->archive_get_guest_config($volname)
or die "backup provider failed to extract guest configuration\n";
+ $options->{progress}->{'target-changed'} = 1;
PVE::ReplicationState::delete_guest_states($vmid);
PVE::Tools::file_set_contents($cfgfn, $data);
if ($data = $backup_provider->archive_get_firewall_config($volname)) {
PVE::Tools::file_set_contents($firewall_config_fn, $data);
+ $options->{progress}->{'firewall-restored'} = 1;
restore_firewall_config($vmid, $firewall_config_fn);
}
@@ -7223,9 +7251,12 @@ sub restore_external_archive {
rmtree($tmpdir);
if ($err) {
- $restore_destroy_volumes->($storecfg, $virtdev_hash);
+ # a volume that could not be removed stays with the VM
+ $options->{progress}->{'volumes-kept'} = 1
+ if $restore_destroy_volumes->($storecfg, $virtdev_hash);
die $err;
}
+ $options->{progress}->{'volumes-kept'} = 1;
my $new_conf = restore_merge_config($conffile, $new_conf_raw, $options->{override_conf});
check_restore_permissions($rpcenv, $user, $new_conf);
@@ -7551,6 +7582,7 @@ sub restore_vma_archive {
my $print_devmap = sub {
my $cfgfn = "$tmpdir/qemu-server.conf";
+ $opts->{progress}->{'target-changed'} = 1;
PVE::ReplicationState::delete_guest_states($vmid);
# we can read the config - that is already extracted
@@ -7558,7 +7590,10 @@ sub restore_vma_archive {
|| die "unable to read qemu-server.conf - $!\n";
my $fwcfgfn = "$tmpdir/qemu-server.fw";
- restore_firewall_config($vmid, $fwcfgfn) if -f $fwcfgfn;
+ if (-f $fwcfgfn) {
+ $opts->{progress}->{'firewall-restored'} = 1;
+ restore_firewall_config($vmid, $fwcfgfn);
+ }
$virtdev_hash = $parse_backup_hints->($rpcenv, $user, $cfg, $fh, $devinfo, $opts);
@@ -7677,9 +7712,12 @@ sub restore_vma_archive {
rmtree $tmpdir;
if ($err) {
- $restore_destroy_volumes->($cfg, $virtdev_hash);
+ # a volume that could not be removed stays with the VM
+ $opts->{progress}->{'volumes-kept'} = 1
+ if $restore_destroy_volumes->($cfg, $virtdev_hash);
die $err;
}
+ $opts->{progress}->{'volumes-kept'} = 1;
my $new_conf = restore_merge_config($conffile, $new_conf_raw, $opts->{override_conf});
check_restore_permissions($rpcenv, $user, $new_conf);
@@ -7707,6 +7745,7 @@ sub restore_tar_archive {
my $storecfg = PVE::Storage::config();
+ $opts->{progress}->{'target-changed'} = 1;
PVE::ReplicationState::delete_guest_states($vmid);
# avoid zombie disks when restoring over an existing VM -> cleanup first
@@ -7786,9 +7825,12 @@ sub restore_tar_archive {
$srcfd->close();
};
if (my $err = $@) {
- tar_restore_cleanup($storecfg, "$tmpdir/qmrestore.stat") if !$opts->{info};
+ # a volume that could not be removed stays with the VM
+ $opts->{progress}->{'volumes-kept'} = 1
+ if !$opts->{info} && tar_restore_cleanup($storecfg, "$tmpdir/qmrestore.stat");
die $err;
}
+ $opts->{progress}->{'volumes-kept'} = 1;
rmtree $tmpdir;
diff --git a/src/test/QemuServerTest/FakeCFS.pm b/src/test/QemuServerTest/FakeCFS.pm
new file mode 100644
index 00000000..99d09f87
--- /dev/null
+++ b/src/test/QemuServerTest/FakeCFS.pm
@@ -0,0 +1,186 @@
+package QemuServerTest::FakeCFS;
+
+# Fake cluster filesystem for VM config files, so that tests can use the real config class methods
+# of PVE::QemuConfig, including the guest identity handling of write_config, without pmxcfs.
+#
+# Configs are kept as raw text, indexed by VMID, and an empty file at the path that config_file()
+# returns exists for each, for checks of the file's existence. Generated guest identities are
+# deterministic: the n-th generated uid is 'a' repeated 17 times followed by the base32 encoding of
+# n, and rtime is $now_ms.
+
+use v5.36;
+
+use File::Basename qw(dirname);
+use File::Path qw(make_path remove_tree);
+use File::Temp qw(tempdir);
+use Test::MockModule;
+
+use PVE::GuestHelpers;
+use PVE::GuestID;
+use PVE::GuestIdentity;
+use PVE::QemuConfig;
+use PVE::QemuServer;
+
+our $configs = {}; # VMID => raw config, for VMs on the local node
+our $other_guests = {}; # VMID => raw 'meta' property of other guests, like on other nodes or CTs
+our $now_ms = 1700000000000;
+our $generated = 0;
+our $lock_domain_error;
+our $writes = [];
+
+my $tmpdir = tempdir('fake-cfs-XXXXXX', DIR => '.', CLEANUP => 1);
+make_path("$tmpdir/lock");
+$PVE::GuestHelpers::lockdir = "$tmpdir/lock";
+
+sub reset() {
+ remove_tree("$tmpdir/etc-pve");
+ $configs = {};
+ $other_guests = {};
+ $now_ms = 1700000000000;
+ $generated = 0;
+ $lock_domain_error = undef;
+ $writes = [];
+}
+
+my sub vmid_of_path($path) {
+ return $path =~ m|/qemu-server/(\d+)\.conf$| ? $1 : undef;
+}
+
+# Creates or removes the file at the path of the config of $vmid, matching the stored config.
+my sub sync_config_file($vmid) {
+ my $path = PVE::QemuConfig->config_file($vmid);
+ if (defined($configs->{$vmid})) {
+ make_path(dirname($path));
+ open(my $fh, '>', $path) or die "unable to create '$path' - $!\n";
+ close($fh);
+ } else {
+ unlink($path);
+ }
+}
+
+# Stores the raw config $raw for $vmid, like a manual edit of the config file.
+sub set_config($vmid, $raw) {
+ $configs->{$vmid} = $raw;
+ sync_config_file($vmid);
+}
+
+# Returns the last 'meta' property of the top-level section of the raw config $raw, like the config
+# class reads the stored identity.
+my sub top_level_meta($raw) {
+ my $meta;
+ for my $line (split(/\n/, $raw // '')) {
+ last if $line =~ m/^\[/;
+ # like pmxcfs, a line with nothing after the colon does not count
+ $meta = $1 =~ s/^\s+|\s+$//gr if $line =~ m/^meta:(.+)$/;
+ }
+
+ return $meta;
+}
+
+# Returns the identity stored in the raw config of $vmid, undef if there is none.
+sub stored_identity($vmid) {
+ return PVE::GuestIdentity::parse_identity(top_level_meta($configs->{$vmid}));
+}
+
+# Returns the uid that the $n-th generation of an identity yields.
+sub generated_uid($n) {
+ my $alphabet = 'abcdefghijklmnopqrstuvwxyz234567';
+ return ('a' x 17) . join('', map { substr($alphabet, ($n >> (5 * $_)) & 31, 1) } (2, 1, 0));
+}
+
+# the list of used guest IDs is not part of the fake cluster filesystem
+our $guest_id_module = Test::MockModule->new('PVE::GuestID');
+$guest_id_module->mock(register_used_id => sub { return; });
+
+our $guest_identity_module = Test::MockModule->new('PVE::GuestIdentity');
+$guest_identity_module->mock(
+ __random_bytes => sub {
+ my ($len) = @_;
+ die "unexpected length $len\n" if $len != 20;
+ $generated++;
+ # one byte per uid character, the last three hold 5 bits of the counter each
+ return ("\0" x 17) . pack('C3', map { ($generated >> $_) & 31 } (10, 5, 0));
+ },
+ __time_ms => sub {
+ return $now_ms;
+ },
+);
+
+our $cluster_module = Test::MockModule->new('PVE::Cluster');
+$cluster_module->mock(
+ get_config => sub {
+ my ($path) = @_;
+ my $vmid = vmid_of_path($path) // die "get_config (mocked) - unexpected path '$path'\n";
+ return $configs->{$vmid};
+ },
+ cfs_read_file => sub {
+ my ($path) = @_;
+ return {} if $path eq 'datacenter.cfg';
+ my $vmid = vmid_of_path($path)
+ // die "cfs_read_file (mocked) - unexpected path '$path'\n";
+ return PVE::QemuServer::parse_vm_config($path, $configs->{$vmid});
+ },
+ cfs_write_file => sub {
+ my ($path, $conf) = @_;
+ my $vmid = vmid_of_path($path)
+ // die "cfs_write_file (mocked) - unexpected path '$path'\n";
+ $configs->{$vmid} = PVE::QemuServer::write_vm_config($path, $conf);
+ sync_config_file($vmid);
+ push $writes->@*, $vmid;
+ return;
+ },
+ cfs_update => sub {
+ return;
+ },
+ check_vmid_unused => sub {
+ my ($vmid, $noerr) = @_;
+ return 1 if !defined($configs->{$vmid}) && !defined($other_guests->{$vmid});
+ return undef if $noerr;
+ die "VM $vmid already exists\n";
+ },
+ cfs_lock_domain => sub {
+ my ($domain, $timeout, $code, @param) = @_;
+ die "cfs_lock_domain (mocked) - unexpected domain '$domain'\n"
+ if $domain ne 'guest-identity';
+ if ($lock_domain_error) {
+ $@ = $lock_domain_error;
+ return undef;
+ }
+ my $res = eval { $code->(@param) };
+ return $res;
+ },
+ get_guest_config_properties => sub {
+ my ($properties) = @_;
+ die "get_guest_config_properties (mocked) - unexpected properties\n"
+ if join(',', $properties->@*) ne 'meta';
+
+ my $res = {};
+ for my $vmid (keys $configs->%*) {
+ my $meta = top_level_meta($configs->{$vmid});
+ $res->{$vmid} = { meta => $meta } if defined($meta);
+ }
+ $res->{$_} = { meta => $other_guests->{$_} } for keys $other_guests->%*;
+ return $res;
+ },
+);
+
+our $qemu_config_module = Test::MockModule->new('PVE::QemuConfig');
+$qemu_config_module->mock(
+ config_file => sub {
+ my ($class, $vmid, $node) = @_;
+ return "$tmpdir/etc-pve/" . $class->cfs_config_path($vmid, $node);
+ },
+ config_file_lock => sub {
+ my ($class, $vmid) = @_;
+ return "$tmpdir/lock/lock-$vmid.conf";
+ },
+ destroy_config => sub {
+ my ($class, $vmid) = @_;
+ die "failed to remove config file: No such file or directory\n"
+ if !defined($configs->{$vmid});
+ delete $configs->{$vmid};
+ sync_config_file($vmid);
+ },
+);
+
+1;
diff --git a/src/test/run_qemu_restore_config_tests.pl b/src/test/run_qemu_restore_config_tests.pl
index 20cb3df0..24abf046 100755
--- a/src/test/run_qemu_restore_config_tests.pl
+++ b/src/test/run_qemu_restore_config_tests.pl
@@ -3,35 +3,31 @@
use strict;
use warnings;
-use lib qw(..);
+use lib qw(.. .);
+use Errno qw(EACCES ENOENT);
+use Fcntl qw(:flock O_RDONLY O_NONBLOCK);
+use File::Path qw(make_path);
+use File::Temp qw(tempdir);
+use JSON;
use Test::MockModule;
use Test::More;
use File::Basename;
+use PVE::API2::Qemu;
+use PVE::QemuConfig;
use PVE::QemuServer;
-use PVE::Tools qw(dir_glob_foreach file_get_contents);
+use PVE::QemuServer::Helpers;
+use PVE::RPCEnvironment;
+use PVE::Tools qw(dir_glob_foreach file_get_contents file_set_contents);
+
+# also provides the 'datacenter.cfg' that restore_update_config_line reads
+use QemuServerTest::FakeCFS;
my $INPUT_DIR = './restore-config-input';
my $EXPECTED_DIR = './restore-config-expected';
-# NOTE update when you add/remove tests
-plan tests => 5;
-
-my $pve_cluster_module = Test::MockModule->new("PVE::Cluster");
-$pve_cluster_module->mock(
- cfs_read_file => sub {
- my ($file) = @_;
-
- if ($file eq 'datacenter.cfg') {
- return {};
- } else {
- die "'cfs_read_file' called - missing mock?\n";
- }
- },
-);
-
dir_glob_foreach(
'./restore-config-input',
'[0-9]+.conf',
@@ -78,4 +74,481 @@ dir_glob_foreach(
},
);
+# Restores through the create API and the restore paths for PBS, backup providers, VMA and legacy
+# tar archives, against the fake cluster filesystem. Storage, archive tools and other external
+# effects are mocked. The backups have no disks, so that no volumes get allocated.
+
+my $runtime_dir = tempdir('restore-run-XXXXXX', DIR => '.', CLEANUP => 1);
+$PVE::QemuServer::Helpers::var_run_tmpdir = $runtime_dir;
+
+my $rpcenv = PVE::RPCEnvironment->init('cli');
+$rpcenv->set_user('root@pam');
+
+my $ARCHIVES = {
+ pbs => 'pbs:backup/vm/100/2025-01-01T00:00:00Z',
+ provider => 'provider:backup/vm-100-2025-01-01',
+ 'vma file' => '/var/lib/vz/dump/vzdump-qemu-100-2025_01_01-00_00_00.vma',
+ 'vma stdin' => '-',
+ tar => '/var/lib/vz/dump/vzdump-qemu-100-2025_01_01-00_00_00.tar',
+};
+
+our $current_backup; # raw config of the backup
+# where the restore fails: 'read-config', 'firewall', 'data', 'after-restore' or 'live'
+our $fail_at = '';
+our $firewall_removal_errno; # errno with which removing the firewall config fails, if set
+our $firewall_existed = 0; # whether VM 100 had a firewall config before the restore
+my $calls = {}; # calls with effects on the target VM that tests check
+
+sub backup_config {
+ my ($meta) = @_;
+
+ return
+ "boot: order=scsi0\ncores: 1\nmemory: 512\n"
+ . (defined($meta) ? "meta: $meta\n" : '')
+ . "name: backup\n";
+}
+
+package TestBackupProvider {
+
+ sub new {
+ my ($class) = @_;
+ return bless {}, $class;
+ }
+
+ sub restore_get_mechanism { return ('qemu-img', 'qemu'); }
+ sub restore_vm_init { return {}; }
+
+ sub archive_get_guest_config {
+ die "unable to read the backup\n" if $main::fail_at eq 'read-config';
+ return $main::current_backup;
+ }
+ sub archive_get_firewall_config { return "[OPTIONS]\nenable: 1\n"; }
+ sub restore_vm_cleanup { return; }
+}
+
+my $replication_state_module = Test::MockModule->new('PVE::ReplicationState');
+$replication_state_module->mock(delete_guest_states => sub { $calls->{delete_guest_states}++; });
+
+my $firewall_module = Test::MockModule->new('PVE::Firewall');
+$firewall_module->mock(
+ remove_vmfw_conf => sub {
+ my ($vmid) = @_;
+ $calls->{remove_vmfw_conf}++;
+ $calls->{'remove_vmfw_conf-reserved'}++
+ if defined($QemuServerTest::FakeCFS::configs->{$vmid});
+ # like PVE::Firewall, which returns the result of unlink
+ return 1 if !$firewall_removal_errno;
+ $! = $firewall_removal_errno;
+ return 0;
+ },
+);
+
+my $tools_module = Test::MockModule->new('PVE::Tools');
+$tools_module->mock(
+ file_copy => sub {
+ my ($src, $dst) = @_;
+ die "file_copy (mocked) - unexpected destination '$dst'\n" if $dst !~ m|/firewall/|;
+ $calls->{firewall}++;
+ },
+);
+
+my $storage_module = Test::MockModule->new('PVE::Storage');
+$storage_module->mock(
+ abs_filesystem_path => sub {
+ my ($cfg, $path) = @_;
+ return $path;
+ },
+ check_volume_access => sub { return; },
+ config => sub { return {}; },
+ deactivate_volumes => sub { return 1; },
+ extract_vzdump_config => sub { return $current_backup; },
+ new_backup_provider => sub { return TestBackupProvider->new(); },
+ parse_volname => sub {
+ return ('backup', 'vm/100/2025-01-01T00:00:00Z', 100, undef, undef, undef, 'pbs-vm');
+ },
+ parse_volume_id => sub {
+ my ($volid, $noerr) = @_;
+ return ($1, $2) if $volid =~ m/^([a-z]+):(.+)$/;
+ die "unable to parse volume ID '$volid'\n" if !$noerr;
+ return;
+ },
+ path_to_volume_id => sub { return; },
+ storage_config => sub {
+ my ($cfg, $storeid) = @_;
+ return { type => 'pbs', server => 'localhost', datastore => 'ds' } if $storeid eq 'pbs';
+ return { type => 'dir', path => '/mnt/provider' };
+ },
+ storage_has_feature => sub {
+ my ($cfg, $storeid, $feature) = @_;
+ return $storeid eq 'provider' && $feature eq 'backup-provider';
+ },
+);
+
+my $pbs_plugin_module = Test::MockModule->new('PVE::Storage::PBSPlugin');
+$pbs_plugin_module->mock(
+ pbs_encryption_key_file_name => sub { return '/nonexistent/key'; },
+ pbs_get_password => sub { return 'secret'; },
+ run_raw_client_cmd => sub {
+ my ($scfg, $storeid, $cmd, $param) = @_;
+ my (undef, $file, $target) = $param->@*;
+ if ($file eq 'index.json') {
+ my $files =
+ [{ filename => 'qemu-server.conf.blob' }, { filename => 'fw.conf.blob' }];
+ file_set_contents($target, encode_json({ files => $files }));
+ } elsif ($file eq 'qemu-server.conf') {
+ die "unable to read the backup\n" if $fail_at eq 'read-config';
+ file_set_contents($target, $current_backup);
+ } elsif ($file eq 'fw.conf') {
+ file_set_contents($target, "[OPTIONS]\nenable: 1\n");
+ } else {
+ die "run_raw_client_cmd (mocked) - unexpected file '$file'\n";
+ }
+ },
+);
+
+my $network_module = Test::MockModule->new('PVE::QemuServer::Network');
+$network_module->mock(
+ create_ifaces_ipams_ips => sub {
+ die "unable to register IPs\n" if $fail_at eq 'after-restore';
+ },
+);
+
+my $qemu_server_module = Test::MockModule->new('PVE::QemuServer');
+$qemu_server_module->mock(
+ check_running => sub { return 0; },
+ pbs_live_restore => sub {
+ die "live restore failed\n" if $fail_at eq 'live';
+ $calls->{live_restore}++;
+ },
+ firewall_config_exists => sub { return $firewall_existed; },
+ rescan => sub { return; },
+ restore_firewall_config => sub {
+ die "unable to restore the firewall config\n" if $fail_at eq 'firewall';
+ $calls->{firewall}++;
+ },
+ restore_update_config_line => sub {
+ die "restoring the data failed\n" if $fail_at eq 'data';
+ return $qemu_server_module->original('restore_update_config_line')->(@_);
+ },
+ tar_archive_read_firstfile => sub { return 'qemu-server.conf'; },
+ # like 'tar' with qmextract and 'vma extract': write the config and report the VMA header
+ run_command => sub {
+ my ($cmd, %param) = @_;
+
+ if (!ref($cmd) && $cmd =~ m/^zcat -f\|tar xf /) {
+ die "unable to read the backup\n" if $fail_at eq 'read-config';
+ file_set_contents("$ENV{VZDUMP_TMPDIR}/qemu-server.conf", $current_backup);
+ return 0;
+ }
+
+ my $vma = $cmd->[-1];
+ die "run_command (mocked) - unexpected command\n"
+ if ref($vma) ne 'ARRAY' || $vma->[0] ne 'vma';
+ my ($mapfifo, $tmpdir) = $vma->@[4, 6];
+
+ die "unable to read the backup\n" if $fail_at eq 'read-config';
+
+ make_path($tmpdir);
+ file_set_contents("$tmpdir/qemu-server.conf", $current_backup);
+ file_set_contents("$tmpdir/qemu-server.fw", "[OPTIONS]\nenable: 1\n");
+
+ sysopen(my $reader, $mapfifo, O_RDONLY | O_NONBLOCK)
+ or die "unable to open fifo - $!\n";
+ $param{afterfork}->();
+ $param{outfunc}->('CTIME: Wed Jan 1 00:00:00 2025');
+ close($reader);
+
+ return 0;
+ },
+);
+
+my $task_error;
+my $rpcenv_module = Test::MockModule->new('PVE::RPCEnvironment');
+$rpcenv_module->mock(
+ check => sub { return 1; },
+ fork_worker => sub {
+ my ($self, $type, $id, $user, $code) = @_;
+ eval { $code->('UPID:test') };
+ $task_error = $@;
+ return 'UPID:test';
+ },
+);
+
+$QemuServerTest::FakeCFS::cluster_module->mock(check_cfs_quorum => sub { return 1; });
+
+# Restores a backup with the raw config $backup to VMID 100 through the create API, from an archive
+# of type $type with the additional API parameters $param. Returns the error of the API call or the
+# task and the task output.
+sub restore_api {
+ my ($type, $backup, $param) = @_;
+
+ $current_backup = $backup;
+ $calls = {};
+ $task_error = '';
+
+ my ($err, $output) = ('', '');
+ {
+ open(my $out_fh, '>', \$output);
+ local *STDERR = $out_fh;
+ my $oldfh = select($out_fh);
+
+ eval {
+ PVE::API2::Qemu->create_vm({
+ node => 'localhost',
+ vmid => 100,
+ archive => $ARCHIVES->{$type},
+ $param->%*,
+ });
+ };
+ $err = $@ || $task_error;
+
+ select($oldfh);
+ close($out_fh);
+ }
+
+ return ($err, $output);
+}
+
+my $TARGET_CONF = "memory: 1024\nmeta: creation-qemu=10.0.0,ctime=1700000000\nname: target\n";
+my $NOT_CLEANED_UP = qr/VM 100 state is NOT cleaned up/;
+
+for my $type (sort keys $ARCHIVES->%*) {
+ subtest "$type: failed restore to a free VMID" => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ $fail_at = 'data';
+ my ($err, $output) = restore_api($type, backup_config(), {});
+ is($err, "restoring the data failed\n", 'restore failed');
+ ok(!defined($QemuServerTest::FakeCFS::configs->{100}), 'reserved config removed');
+ # legacy tar archives have no firewall config
+ my $restored_firewall = $type eq 'tar' ? undef : 1;
+ is($calls->{remove_vmfw_conf}, $restored_firewall, 'restored firewall config removed');
+ is(
+ $calls->{'remove_vmfw_conf-reserved'},
+ $restored_firewall,
+ 'while the VMID was still reserved',
+ );
+ unlike($output, $NOT_CLEANED_UP, 'no warning about a VM left behind');
+
+ $fail_at = '';
+ ($err, $output) = restore_api($type, backup_config(), {});
+ is($err, '', 'retry needs no force') or diag($output);
+ like($QemuServerTest::FakeCFS::configs->{100}, qr/^name: backup$/m, 'VM restored');
+ };
+}
+
+subtest 'failed restore to a free VMID before reading the backup' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ $fail_at = 'read-config';
+ my ($err, $output) = restore_api('pbs', backup_config(), {});
+ is($err, "unable to read the backup\n", 'restore failed');
+ ok(!defined($QemuServerTest::FakeCFS::configs->{100}), 'reserved config removed');
+ ok(!$calls->{remove_vmfw_conf}, 'firewall config that the restore did not write is kept');
+ unlike($output, $NOT_CLEANED_UP, 'no warning about a VM left behind');
+};
+
+subtest 'removing the restored firewall config warns if it replaced one' => sub {
+ my $replaced =
+ qr/^WARN: removed the firewall config of VM 100, the failed restore had replaced/m;
+ for my $existed (0, 1) {
+ my $what = $existed ? 'firewall config before' : 'no firewall config before';
+ QemuServerTest::FakeCFS::reset();
+
+ local $fail_at = 'data';
+ local $firewall_existed = $existed;
+ my ($err, $output) = restore_api('pbs', backup_config(), {});
+ is($err, "restoring the data failed\n", "$what: restore failed");
+ is($calls->{remove_vmfw_conf}, 1, "$what: restored firewall config removed");
+ if ($existed) {
+ like($output, $replaced, "$what: warning about the replaced firewall config");
+ } else {
+ unlike($output, $replaced, "$what: no warning");
+ }
+ }
+};
+
+subtest 'failed removal of the restored firewall config keeps the VMID' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ local $fail_at = 'data';
+ local $firewall_removal_errno = EACCES;
+ my ($err, $output) = restore_api('pbs', backup_config(), {});
+ is($err, "restoring the data failed\n", 'restore failed');
+ like($output, qr/^unable to remove the firewall config - /m, 'failed removal is reported');
+ ok(defined($QemuServerTest::FakeCFS::configs->{100}), 'reservation is kept');
+ unlike($QemuServerTest::FakeCFS::configs->{100}, qr/^lock:/m, 'lock removed');
+ like($output, $NOT_CLEANED_UP, 'warning about the VM left behind');
+
+ # the restore can fail before the firewall config was written
+ QemuServerTest::FakeCFS::reset();
+ $fail_at = 'firewall';
+ $firewall_removal_errno = ENOENT;
+ ($err, $output) = restore_api('pbs', backup_config(), {});
+ is($err, "unable to restore the firewall config\n", 'restore failed');
+ ok(!defined($QemuServerTest::FakeCFS::configs->{100}), 'missing firewall config: removed');
+};
+
+subtest 'failed removal of a restored volume keeps the VMID' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ my $allocated = [];
+ my $run_raw_client_cmd = \&PVE::Storage::PBSPlugin::run_raw_client_cmd;
+ $pbs_plugin_module->mock(
+ run_raw_client_cmd => sub {
+ my ($scfg, $storeid, $cmd, $param) = @_;
+ my (undef, $file, $target) = $param->@*;
+ return $run_raw_client_cmd->(@_) if $file ne 'index.json';
+ my $files = [
+ { filename => 'qemu-server.conf.blob' },
+ { filename => 'drive-scsi0.img.fidx', size => 1048576 },
+ ];
+ file_set_contents($target, encode_json({ files => $files }));
+ },
+ );
+ my $storage_config = \&PVE::Storage::storage_config;
+ my @storage_functions =
+ qw(resolve_format_hint vdisk_alloc activate_volumes path volume_has_feature vdisk_free);
+ $storage_module->mock(
+ storage_config => sub {
+ my ($cfg, $storeid) = @_;
+ return { type => 'pbs', server => 'localhost', datastore => 'ds' }
+ if $storeid eq 'pbs';
+ return { type => 'dir', path => '/nonexistent', content => { images => 1 } };
+ },
+ resolve_format_hint => sub { return 'raw'; },
+ vdisk_alloc => sub {
+ my ($cfg, $storeid, $vmid) = @_;
+ push $allocated->@*, "$storeid:vm-$vmid-disk-0";
+ return "$storeid:vm-$vmid-disk-0";
+ },
+ activate_volumes => sub { return; },
+ path => sub { return '/nonexistent/vm-100-disk-0.raw'; },
+ volume_has_feature => sub { return 0; },
+ vdisk_free => sub { die "volume is busy\n"; },
+ );
+ my $run_command = \&PVE::QemuServer::run_command;
+ $qemu_server_module->mock(
+ run_command => sub {
+ my ($cmd) = @_;
+ return 0 if ref($cmd) && $cmd->[0] eq '/usr/bin/pbs-restore';
+ return $run_command->(@_);
+ },
+ );
+
+ local $fail_at = 'data';
+ my $backup = backup_config()
+ . "scsi0: target:vm-100-disk-0,size=1M\n#qmdump#map:scsi0:drive-scsi0:target:raw:\n";
+ my ($err, $output) = restore_api('pbs', $backup, {});
+ is($err, "restoring the data failed\n", 'restore failed after allocating the volume');
+ is_deeply($allocated, ['target:vm-100-disk-0'], 'volume was allocated');
+ like($output, qr/^unable to cleanup 'target:vm-100-disk-0' - volume is busy$/m, 'reported');
+ ok(defined($QemuServerTest::FakeCFS::configs->{100}), 'reservation is kept');
+ like($output, $NOT_CLEANED_UP, 'warning about the VM left behind');
+
+ $pbs_plugin_module->mock(run_raw_client_cmd => $run_raw_client_cmd);
+ $storage_module->mock(storage_config => $storage_config);
+ $storage_module->unmock($_) for @storage_functions;
+ $qemu_server_module->mock(run_command => $run_command);
+};
+
+subtest "'force' does not overwrite a VM created after the checks" => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ # the archive is checked after the target VM
+ my $check_volume_access = \&PVE::Storage::check_volume_access;
+ $storage_module->mock(
+ check_volume_access => sub { QemuServerTest::FakeCFS::set_config(100, $TARGET_CONF); },
+ );
+
+ my ($err) = restore_api('pbs', backup_config(), { force => 1 });
+ $storage_module->mock(check_volume_access => $check_volume_access);
+ is($err, "unable to restore VM 100 - VM 100 already exists\n", 'restore refused');
+ is(
+ $QemuServerTest::FakeCFS::configs->{100},
+ $TARGET_CONF,
+ 'VM created in between unchanged',
+ );
+};
+
+subtest 'another restore right after a failed restore to a free VMID' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ # run a second restore with 'force' as soon as the failed one releases the guest lock
+ my $lock_config_full = PVE::QemuConfig->can('lock_config_full');
+ my ($interposed, $lock_free, $second_err) = (0, 0, '');
+ $QemuServerTest::FakeCFS::qemu_config_module->mock(
+ lock_config_full => sub {
+ my ($class, $vmid, @param) = @_;
+ my $res = eval { $lock_config_full->($class, $vmid, @param) };
+ my $err = $@;
+ if ($err && !$interposed++) {
+ open(my $fh, '>>', PVE::QemuConfig->config_file_lock($vmid)) or die "$!\n";
+ $lock_free = flock($fh, LOCK_EX | LOCK_NB);
+ close($fh);
+ local $fail_at = '';
+ ($second_err) = restore_api('pbs', backup_config(), { force => 1 });
+ }
+ die $err if $err;
+ return $res;
+ },
+ );
+
+ $fail_at = 'data';
+ my ($err) = restore_api('vma file', backup_config(), {});
+ $QemuServerTest::FakeCFS::qemu_config_module->unmock('lock_config_full');
+
+ is($err, "restoring the data failed\n", 'first restore failed');
+ ok($lock_free, 'guest lock was free for the second restore');
+ is($second_err, '', 'second restore succeeded');
+ like($QemuServerTest::FakeCFS::configs->{100}, qr/^name: backup$/m, 'its config stays');
+ is($calls->{firewall}, 1, 'its firewall config was restored');
+ ok(!$calls->{remove_vmfw_conf}, 'and stays');
+};
+
+for my $type ('pbs', 'provider', 'vma file', 'vma stdin') {
+ subtest "$type: failed restore over an existing VM" => sub {
+ QemuServerTest::FakeCFS::reset();
+ QemuServerTest::FakeCFS::set_config(100, $TARGET_CONF);
+
+ $fail_at = 'read-config';
+ my ($err, $output) = restore_api($type, backup_config(), { force => 1 });
+ like($err, qr/unable to read the backup\n$/, 'fails before changing the VM');
+ is($QemuServerTest::FakeCFS::configs->{100}, $TARGET_CONF, 'config unchanged');
+ is_deeply($calls, {}, 'replication state and firewall config untouched');
+ unlike($output, $NOT_CLEANED_UP, 'no warning about a changed VM');
+
+ $fail_at = 'data';
+ ($err, $output) = restore_api($type, backup_config(), { force => 1 });
+ is($err, "restoring the data failed\n", 'fails while restoring');
+ like($QemuServerTest::FakeCFS::configs->{100}, qr/^memory: 128$/m, 'temporary config');
+ unlike($output, qr/no lock found/, 'temporary config had the create lock');
+ unlike($QemuServerTest::FakeCFS::configs->{100}, qr/^lock:/m, 'lock removed');
+ like($output, qr/error before or during data restore/, 'warning about the changed VM');
+ };
+}
+
+subtest 'failed live restore to a free VMID keeps the restored config' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ $fail_at = 'live';
+ my ($err, $output) = restore_api('pbs', backup_config(), { 'live-restore' => 1 });
+ is($err, "live restore failed\n", 'restore failed');
+ like($QemuServerTest::FakeCFS::configs->{100}, qr/^name: backup$/m, 'restored config kept');
+ unlike($QemuServerTest::FakeCFS::configs->{100}, qr/^lock:/m, 'lock removed');
+ like($output, $NOT_CLEANED_UP, 'warning about the VM left behind');
+};
+
+subtest 'failure after the data was restored keeps the VM' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ $fail_at = 'after-restore';
+ my ($err, $output) = restore_api('pbs', backup_config(), { unique => 1 });
+ is($err, "unable to register IPs\n", 'restore failed');
+ like($QemuServerTest::FakeCFS::configs->{100}, qr/^name: backup$/m, 'restored config kept');
+ like($output, qr/error after data was restored/, 'warning about the VM left behind');
+};
+
+$fail_at = '';
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH qemu-server v3 15/38] api: migrate, clone: refuse targets that would drop the guest identity
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (13 preceding siblings ...)
2026-11-18 4:37 ` [PATCH qemu-server v3 14/38] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 16/38] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
` (24 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Older qemu-server versions reject the identity members of 'meta' and
drop the whole property when they write the config. A guest that moves
to such a node loses its identity, and gets a new one with a later
write by an upgraded node.
Refuse online and offline migrations and clones to another node unless
the target reports a Proxmox VE version that keeps the identity. Use the
shared version threshold and treat missing version information as too
old. Report the reason and the versions involved in the migration
preconditions, so that clients can explain the refusal. Leave such nodes
out of the nodes reported for the clone and copy features, which the web
interface uses to validate the target of a clone. The snapshot feature,
which the web interface polls, keeps reporting all nodes without
looking up their versions.
Check before starting the task or handing an API migration request to
HA. This does not keep an HA resource plugin from shutting down a VM
for a relocation before it calls the migration API.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* describe the refusal reason per node, on top of the independent schema
fix, and validate complete responses in the test
* feature API: no version lookups for 'snapshot' requests
changes v1 -> v2:
* leave such nodes out of the nodes of the clone feature, which the
clone dialog uses
* use the shared schema for the precondition result
src/PVE/API2/Qemu.pm | 27 +++++-
src/test/run_api_tests.pl | 168 ++++++++++++++++++++++++++++++++++++++
2 files changed, 194 insertions(+), 1 deletion(-)
diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index 1a3f2ce9..5954dd8e 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -29,6 +29,7 @@ use PVE::RESTHandler;
use PVE::ReplicationConfig;
use PVE::GuestHelpers qw(assert_tag_permissions);
use PVE::GuestID;
+use PVE::GuestIdentity;
use PVE::GuestImport;
use PVE::QemuConfig;
use PVE::QemuServer;
@@ -4394,6 +4395,8 @@ __PACKAGE__->register_method({
properties => {
hasFeature => { type => 'boolean' },
nodes => {
+ description => "Nodes on which the storages of the VM are available, for"
+ . " 'clone' and 'copy' only those that keep the guest identity.",
type => 'array',
items => { type => 'string' },
},
@@ -4422,6 +4425,15 @@ __PACKAGE__->register_method({
my $storecfg = PVE::Storage::config();
my $nodelist = PVE::QemuServer::shared_nodes($conf, $storecfg);
+ # a clone to another node is refused if that node would drop the guest identity
+ if ($feature ne 'snapshot') {
+ my $nodename = PVE::INotify::nodename();
+ for my $other (keys $nodelist->%*) {
+ next if $other eq $nodename;
+ delete $nodelist->{$other}
+ if PVE::GuestIdentity::node_identity_unsupported($other);
+ }
+ }
my $hasFeature =
PVE::QemuConfig->has_feature($feature, $conf, $storecfg, $snapname, $running);
@@ -4554,7 +4566,10 @@ __PACKAGE__->register_method({
my $load_and_check = sub {
$rpcenv->check_pool_exist($pool) if defined($pool);
- PVE::Cluster::check_node_exists($target) if $target;
+ if ($target) {
+ PVE::Cluster::check_node_exists($target);
+ PVE::GuestIdentity::assert_node_supports_identity($target);
+ }
my $storecfg = PVE::Storage::config();
@@ -5443,6 +5458,8 @@ __PACKAGE__->register_method({
},
},
},
+ 'guest-identity-unsupported' =>
+ PVE::GuestIdentity::node_identity_unsupported_schema(),
},
},
description => "Not allowed nodes, keyed by node name, with the reasons.",
@@ -5585,6 +5602,12 @@ __PACKAGE__->register_method({
$blocking_ha_resources;
}
+ # the migration refuses nodes that would drop the guest identity
+ if (my $unsupported = PVE::GuestIdentity::node_identity_unsupported($node)) {
+ $res->{not_allowed_nodes}->{$node}->{'guest-identity-unsupported'} =
+ $unsupported;
+ }
+
# if nothing came up, add it to the allowed nodes
if (scalar($res->{not_allowed_nodes}->{$node}->%*) == 0) {
push $res->{allowed_nodes}->@*, $node;
@@ -5697,6 +5720,8 @@ __PACKAGE__->register_method({
PVE::Cluster::check_node_exists($target);
+ PVE::GuestIdentity::assert_node_supports_identity($target);
+
my $targetip = PVE::Cluster::remote_node_ip($target);
my $vmid = extract_param($param, 'vmid');
diff --git a/src/test/run_api_tests.pl b/src/test/run_api_tests.pl
index 4d5e1cdf..cc9cbadf 100755
--- a/src/test/run_api_tests.pl
+++ b/src/test/run_api_tests.pl
@@ -13,6 +13,7 @@ use Test::MockModule;
use Test::More;
use PVE::API2::Qemu;
+use PVE::GuestIdentity;
use PVE::RPCEnvironment;
my $rpcenv = PVE::RPCEnvironment->init('cli');
@@ -38,6 +39,31 @@ sub mock_unreachable_config {
}
mock_unreachable_config();
+my $qemu_server_module = Test::MockModule->new('PVE::QemuServer');
+$qemu_server_module->mock(check_running => sub { return 0; });
+
+# Proxmox VE versions the cluster nodes report, undef for none
+my $node_versions = {};
+
+my $cluster_module = Test::MockModule->new('PVE::Cluster');
+$cluster_module->mock(
+ check_cfs_quorum => sub { return 1; },
+ check_node_exists => sub { return 1; },
+ remote_node_ip => sub { return '192.0.2.2'; },
+ get_node_pvecfg_version => sub {
+ my ($node) = @_;
+ return $node_versions->{$node};
+ },
+ node_pvecfg_version_at_least => sub {
+ my ($node, @required) = @_;
+ my @version = split(/\./, $node_versions->{$node} // return 0);
+ for my $i (0 .. 2) {
+ return $version[$i] > $required[$i] if $version[$i] != $required[$i];
+ }
+ return 1;
+ },
+);
+
# Returns the error of calling the API method $method with $param, either the parameter errors as
# hash or the error message.
sub api_error {
@@ -145,4 +171,146 @@ subtest 'pending changes of meta are dropped' => sub {
mock_unreachable_config();
};
+my $min_version = PVE::GuestIdentity::MIN_PVE_MANAGER_VERSION;
+my $node_cases = [
+ [$min_version, "reached load_config\n"],
+ ['10.0.0', "reached load_config\n"],
+ [
+ '9.2.0',
+ "node 'pve2' runs Proxmox VE 9.2.0, keeping the guest identity requires $min_version or"
+ . " newer - upgrade the node first\n",
+ ],
+ [
+ undef,
+ "node 'pve2' does not report its Proxmox VE version, so it is unknown whether the node"
+ . " keeps the guest identity\n",
+ ],
+];
+
+for my $case ($node_cases->@*) {
+ my ($version, $expected) = $case->@*;
+ $node_versions->{pve2} = $version;
+ my $what = 'target node with ' . ($version // 'no') . ' version';
+
+ is(
+ api_error('migrate_vm', { target => 'pve2' }), $expected, "migrate_vm: offline, $what",
+ );
+ is(
+ api_error('migrate_vm', { target => 'pve2', online => 1 }),
+ $expected,
+ "migrate_vm: online, $what",
+ );
+ is(
+ api_error('clone_vm', { newid => 101, target => 'pve2' }), $expected, "clone_vm: $what",
+ );
+}
+
+$node_versions->{pve2} = undef;
+is(
+ api_error('clone_vm', { newid => 101 }),
+ "reached load_config\n",
+ 'clone_vm: no check without target node',
+);
+
+subtest 'migration preconditions' => sub {
+ my $nodes = ['localhost', 'pve2', 'pve3', 'pve4'];
+ $cluster_module->mock(get_nodelist => sub { return $nodes; });
+ my $inotify_module = Test::MockModule->new('PVE::INotify');
+ $inotify_module->mock(nodename => sub { return 'localhost'; });
+
+ $qemu_config_module->mock(load_config => sub { return { memory => 512 }; });
+ my $migrate_helpers_module = Test::MockModule->new('PVE::QemuMigrate::Helpers');
+ $migrate_helpers_module->mock(
+ check_local_resources => sub {
+ return ([], {}, { map { $_ => [] } $nodes->@* });
+ },
+ );
+ my $ha_config_module = Test::MockModule->new('PVE::HA::Config');
+ $ha_config_module->mock(vm_is_ha_managed => sub { return 0; });
+
+ $node_versions->@{ 'pve2', 'pve3', 'pve4' } = ($min_version, '9.2.0', undef);
+
+ my $res = PVE::API2::Qemu->migrate_vm_precondition({ node => 'localhost', vmid => 100 });
+
+ is_deeply($res->{allowed_nodes}, ['pve2'], 'only the node keeping the identity is allowed');
+ is_deeply(
+ $res->{not_allowed_nodes}->{pve3},
+ {
+ 'guest-identity-unsupported' => {
+ version => '9.2.0',
+ 'required-version' => $min_version,
+ },
+ },
+ 'node with an older version is not allowed',
+ );
+ is_deeply(
+ $res->{not_allowed_nodes}->{pve4},
+ { 'guest-identity-unsupported' => { 'required-version' => $min_version } },
+ 'node without version is not allowed',
+ );
+
+ my $returns = PVE::API2::Qemu->map_method_by_name('migrate_vm_precondition')->{returns};
+ # every reason of a node must be documented
+ my $strict = dclone($returns);
+ my $node_schema = $strict->{properties}->{not_allowed_nodes}->{additionalProperties};
+ $node_schema->{additionalProperties} = 0;
+ eval { PVE::JSONSchema::validate($res, $strict) };
+ is($@, '', 'result matches the schema, including the reasons of each node');
+ my $undocumented = { $res->%*, not_allowed_nodes => { pve3 => { 'other-reason' => 1 } } };
+ eval { PVE::JSONSchema::validate($undocumented, $strict) };
+ like($@, qr/other-reason/, 'an undocumented reason does not');
+
+ $cluster_module->unmock('get_nodelist');
+ mock_unreachable_config();
+};
+
+subtest 'clone target nodes' => sub {
+ my $inotify_module = Test::MockModule->new('PVE::INotify');
+ $inotify_module->mock(nodename => sub { return 'pve1'; });
+
+ $qemu_config_module->mock(
+ load_config => sub { return { memory => 512 }; },
+ has_feature => sub { return 1; },
+ );
+ $qemu_server_module->mock(
+ shared_nodes => sub {
+ return { map { $_ => 1 } 'pve1', 'pve2', 'pve3', 'pve4' };
+ },
+ );
+
+ # the version of the local node does not matter, a clone on the same node needs no check
+ $node_versions->@{ 'pve1', 'pve2', 'pve3', 'pve4' } = (undef, $min_version, '9.2.0', undef);
+
+ # the local node is the one the handler runs on, whatever name the API path used for it
+ for my $node ('pve1', 'localhost') {
+ my $res =
+ PVE::API2::Qemu->vm_feature({ node => $node, vmid => 100, feature => 'clone' });
+ is_deeply(
+ [sort $res->{nodes}->@*],
+ ['pve1', 'pve2'],
+ "node '$node': only nodes that keep the identity are offered as clone target",
+ );
+ }
+
+ # the web interface polls the snapshot feature, which needs no version lookups
+ my $lookups = 0;
+ my $node_identity_unsupported = \&PVE::GuestIdentity::node_identity_unsupported;
+ my $guest_identity_module = Test::MockModule->new('PVE::GuestIdentity');
+ $guest_identity_module->mock(
+ node_identity_unsupported => sub {
+ $lookups++;
+ return $node_identity_unsupported->(@_);
+ },
+ );
+ my $res =
+ PVE::API2::Qemu->vm_feature({ node => 'pve1', vmid => 100, feature => 'snapshot' });
+ is_deeply([sort $res->{nodes}->@*], ['pve1', 'pve2', 'pve3', 'pve4'], 'snapshot: nodes');
+ is($lookups, 0, 'snapshot: no version lookup');
+ $guest_identity_module->unmock('node_identity_unsupported');
+
+ $qemu_server_module->unmock('shared_nodes');
+ $qemu_config_module->unmock('has_feature');
+ mock_unreachable_config();
+};
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH qemu-server v3 16/38] fix #7846: config: add a guest identity to the 'meta' property
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (14 preceding siblings ...)
2026-11-18 4:37 ` [PATCH qemu-server v3 15/38] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 17/38] restore: take over the guest uid from the backup where possible Thomas Lamprecht
` (23 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Give VMs a guest identity in the read-only 'meta' property: a random
identifier, 'uid', and a registration time, 'rtime'. Tools can use it
to recognize a guest across restores and migrations, independently of
its VMID.
Enable the identity handling of the config class, so that each config
write keeps the identity a VM already has or assigns one if it has
none. New VMs, clones and imports get an identity when their VMID is
reserved. Existing VMs get one with their next config write. A stale
config, a clone source or a snapshot rollback cannot replace the
identity of a VM.
The other members of 'meta' behave as before. A config without creation
information thus gets a 'meta' property with only the identity. Older
qemu-server versions reject the new members and drop the whole property
on their next config write.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* group the imports of the new test script
changes v1 -> v2:
* rebase, adapt the tests
src/PVE/QemuConfig.pm | 7 +
src/PVE/QemuServer/MetaInfo.pm | 2 +
src/test/Makefile | 5 +-
.../q35-windows-pinning-identity-only.conf | 6 +
...q35-windows-pinning-identity-only.conf.cmd | 26 ++
.../meta-identity-invalid.conf | 6 +
.../meta-identity-invalid.conf.strict.error | 2 +
.../meta-identity-invalid.conf | 7 +
.../parse-config-input/meta-identity.conf | 20 ++
src/test/run_guest_identity_tests.pl | 258 ++++++++++++++++++
src/test/run_parse_config_tests.pl | 2 +-
src/test/run_qemu_restore_config_tests.pl | 3 +-
.../create/qemu-server/401.conf | 31 +++
.../create/qemu-server/402.conf | 31 +++
.../rollback/qemu-server/401.conf | 36 +++
.../rollback/qemu-server/402.conf | 35 +++
.../create/qemu-server/401.conf | 13 +
.../create/qemu-server/402.conf | 13 +
.../rollback/qemu-server/401.conf | 35 +++
.../rollback/qemu-server/402.conf | 34 +++
src/test/snapshot-test.pm | 82 +++++-
21 files changed, 640 insertions(+), 14 deletions(-)
create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf
create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf.cmd
create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf
create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf.strict.error
create mode 100644 src/test/parse-config-input/meta-identity-invalid.conf
create mode 100644 src/test/parse-config-input/meta-identity.conf
create mode 100755 src/test/run_guest_identity_tests.pl
create mode 100644 src/test/snapshot-expected/create/qemu-server/401.conf
create mode 100644 src/test/snapshot-expected/create/qemu-server/402.conf
create mode 100644 src/test/snapshot-expected/rollback/qemu-server/401.conf
create mode 100644 src/test/snapshot-expected/rollback/qemu-server/402.conf
create mode 100644 src/test/snapshot-input/create/qemu-server/401.conf
create mode 100644 src/test/snapshot-input/create/qemu-server/402.conf
create mode 100644 src/test/snapshot-input/rollback/qemu-server/401.conf
create mode 100644 src/test/snapshot-input/rollback/qemu-server/402.conf
diff --git a/src/PVE/QemuConfig.pm b/src/PVE/QemuConfig.pm
index d89a62e6..ac9e202a 100644
--- a/src/PVE/QemuConfig.pm
+++ b/src/PVE/QemuConfig.pm
@@ -644,6 +644,13 @@ sub write_config {
return $class->SUPER::write_config($vmid, $conf);
}
+# the format of the 'meta' property includes the guest identity, see PVE::QemuServer::MetaInfo
+sub __identity_enabled {
+ my ($class) = @_;
+
+ return 1;
+}
+
# END implemented abstract methods from PVE::AbstractConfig
sub has_cloudinit {
diff --git a/src/PVE/QemuServer/MetaInfo.pm b/src/PVE/QemuServer/MetaInfo.pm
index b69e5a1c..399cd920 100644
--- a/src/PVE/QemuServer/MetaInfo.pm
+++ b/src/PVE/QemuServer/MetaInfo.pm
@@ -3,6 +3,7 @@ package PVE::QemuServer::MetaInfo;
use strict;
use warnings;
+use PVE::GuestIdentity;
use PVE::JSONSchema;
use PVE::QemuServer::Helpers;
@@ -20,6 +21,7 @@ our $meta_info_fmt = {
pattern => '\d+(\.\d+)+',
optional => 1,
},
+ PVE::GuestIdentity::meta_format_properties()->%*,
};
sub parse_meta_info {
diff --git a/src/test/Makefile b/src/test/Makefile
index d8ee1d59..9d490838 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -1,6 +1,6 @@
all: test
-test: test_snapshot test_cfg_to_cmd test_cfg_to_cmd_aarch64 test_hotplug test_hotplug_aarch64 test_pci_addr_conflicts test_pci_reservation test_qemu_img_convert test_migration test_restore_config test_parse_config test_memory test_raw_config_write test_api test_api_cloudinit_dump
+test: test_snapshot test_cfg_to_cmd test_cfg_to_cmd_aarch64 test_hotplug test_hotplug_aarch64 test_pci_addr_conflicts test_pci_reservation test_qemu_img_convert test_migration test_restore_config test_parse_config test_memory test_raw_config_write test_api test_api_cloudinit_dump test_guest_identity
test_snapshot: run_snapshot_tests.pl
./run_snapshot_tests.pl
@@ -54,6 +54,9 @@ test_api: run_api_tests.pl
test_api_cloudinit_dump: run_api_cloudinit_dump_tests.pl
./run_api_cloudinit_dump_tests.pl
+test_guest_identity: run_guest_identity_tests.pl QemuServerTest/FakeCFS.pm
+ ./run_guest_identity_tests.pl
+
.PHONY: clean
clean:
rm -rf MigrationTest/run
diff --git a/src/test/cfg2cmd/q35-windows-pinning-identity-only.conf b/src/test/cfg2cmd/q35-windows-pinning-identity-only.conf
new file mode 100644
index 00000000..5242ec13
--- /dev/null
+++ b/src/test/cfg2cmd/q35-windows-pinning-identity-only.conf
@@ -0,0 +1,6 @@
+# TEST: Windows without pinned machine and with only the guest identity as meta info stays on 5.1
+machine: q35
+meta: rtime=1762300000000,uid=abcdefghijklmnopqrst
+ostype: win11
+smbios1: uuid=7b10d7af-b932-4c66-b2c3-3996152ec465
+vmgenid: c773c261-d800-4348-9f5d-167fadd53cf8
diff --git a/src/test/cfg2cmd/q35-windows-pinning-identity-only.conf.cmd b/src/test/cfg2cmd/q35-windows-pinning-identity-only.conf.cmd
new file mode 100644
index 00000000..67d20b79
--- /dev/null
+++ b/src/test/cfg2cmd/q35-windows-pinning-identity-only.conf.cmd
@@ -0,0 +1,26 @@
+/usr/bin/kvm \
+ -id 8006 \
+ -name vm8006 \
+ -no-shutdown \
+ -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server=on,wait=off' \
+ -object 'monitor-qmp,chardev=qmp,id=monitor-qmp' \
+ -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect-ms=5000' \
+ -object 'monitor-qmp,chardev=qmp-event,id=monitor-qmp-event' \
+ -pidfile /var/run/qemu-server/8006.pid \
+ -daemonize \
+ -smbios 'type=1,uuid=7b10d7af-b932-4c66-b2c3-3996152ec465' \
+ -smp '1,sockets=1,cores=1,maxcpus=1' \
+ -nodefaults \
+ -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \
+ -vnc 'unix:/var/run/qemu-server/8006.vnc,password=on' \
+ -global 'kvm-pit.lost_tick_policy=discard' \
+ -cpu 'kvm64,-cet-ibt,-cet-ss,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vpindex,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep' \
+ -m 512 \
+ -readconfig /usr/share/qemu-server/pve-q35-4.0.cfg \
+ -device 'vmgenid,guid=c773c261-d800-4348-9f5d-167fadd53cf8' \
+ -device 'usb-tablet,id=tablet,bus=ehci.0,port=1' \
+ -device 'VGA,id=vga,bus=pcie.0,addr=0x1,edid=off' \
+ -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \
+ -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \
+ -rtc 'driftfix=slew,base=localtime' \
+ -machine 'hpet=off,type=pc-q35-5.1+pve0'
diff --git a/src/test/parse-config-expected/meta-identity-invalid.conf b/src/test/parse-config-expected/meta-identity-invalid.conf
new file mode 100644
index 00000000..195514fa
--- /dev/null
+++ b/src/test/parse-config-expected/meta-identity-invalid.conf
@@ -0,0 +1,6 @@
+# VM with a guest uid that is not valid
+boot: order=scsi0
+cores: 2
+memory: 4096
+name: identity
+scsi0: local-lvm:vm-8006-disk-0,size=32G
diff --git a/src/test/parse-config-expected/meta-identity-invalid.conf.strict.error b/src/test/parse-config-expected/meta-identity-invalid.conf.strict.error
new file mode 100644
index 00000000..ae584c1f
--- /dev/null
+++ b/src/test/parse-config-expected/meta-identity-invalid.conf.strict.error
@@ -0,0 +1,2 @@
+vm 8006 - unable to parse value of 'meta' - format error
+uid: value does not match the regex pattern
diff --git a/src/test/parse-config-input/meta-identity-invalid.conf b/src/test/parse-config-input/meta-identity-invalid.conf
new file mode 100644
index 00000000..90e60197
--- /dev/null
+++ b/src/test/parse-config-input/meta-identity-invalid.conf
@@ -0,0 +1,7 @@
+# VM with a guest uid that is not valid
+boot: order=scsi0
+cores: 2
+memory: 4096
+meta: creation-qemu=10.1.2,ctime=1762000000,rtime=1762000000123,uid=ABCDEFGHIJKLMNOPQRST
+name: identity
+scsi0: local-lvm:vm-8006-disk-0,size=32G
diff --git a/src/test/parse-config-input/meta-identity.conf b/src/test/parse-config-input/meta-identity.conf
new file mode 100644
index 00000000..7a1e03be
--- /dev/null
+++ b/src/test/parse-config-input/meta-identity.conf
@@ -0,0 +1,20 @@
+# VM with guest identity
+boot: order=scsi0
+cores: 2
+memory: 4096
+meta: creation-qemu=10.1.2,ctime=1762000000,rtime=1762000000123,uid=abcdefghijklmnopqrst
+name: identity
+scsi0: local-lvm:vm-8006-disk-0,size=32G
+scsihw: virtio-scsi-single
+smbios1: uuid=7b4f2a1e-3c2d-4e5f-8a9b-0c1d2e3f4a5b
+
+[snap]
+boot: order=scsi0
+cores: 2
+memory: 4096
+meta: creation-qemu=10.1.2,ctime=1762000000,rtime=1762000000123,uid=abcdefghijklmnopqrst
+name: identity
+scsi0: local-lvm:vm-8006-disk-0,size=32G
+scsihw: virtio-scsi-single
+smbios1: uuid=7b4f2a1e-3c2d-4e5f-8a9b-0c1d2e3f4a5b
+snaptime: 1762000100
diff --git a/src/test/run_guest_identity_tests.pl b/src/test/run_guest_identity_tests.pl
new file mode 100755
index 00000000..16f012d4
--- /dev/null
+++ b/src/test/run_guest_identity_tests.pl
@@ -0,0 +1,258 @@
+#!/usr/bin/perl
+
+# Tests the guest identity of VM configs through the real config class methods, with a fake cluster
+# filesystem, see QemuServerTest::FakeCFS.
+
+use v5.36;
+
+use lib qw(.. .);
+
+use File::Temp qw(tempdir);
+use Test::MockModule;
+use Test::More;
+
+use PVE::JSONSchema;
+
+use PVE::QemuConfig;
+use PVE::QemuConfig::NoWrite;
+use PVE::QemuServer;
+use PVE::QemuServer::MetaInfo;
+
+use QemuServerTest::FakeCFS;
+
+my $UID_A = 'abcdefghijklmnopqrst';
+my $UID_B = 'bbbbbbbbbbbbbbbbbbbb';
+
+my $helpers_module = Test::MockModule->new('PVE::QemuServer::Helpers');
+$helpers_module->mock(kvm_user_version => sub { return '10.1.2'; });
+
+sub configs { return $QemuServerTest::FakeCFS::configs; }
+sub generated_uid { return QemuServerTest::FakeCFS::generated_uid(@_); }
+sub stored_identity { return QemuServerTest::FakeCFS::stored_identity(@_); }
+
+sub load_meta($vmid) {
+ return PVE::QemuServer::MetaInfo::parse_meta_info(PVE::QemuConfig->load_config($vmid)->{meta});
+}
+
+subtest 'meta format' => sub {
+ my $meta =
+ 'creation-qemu=10.1.2,ctime=1700000000,rtime=1700000000123,uid=abcdefghijklmnopqrst';
+
+ is_deeply(
+ PVE::QemuServer::MetaInfo::parse_meta_info($meta),
+ {
+ 'creation-qemu' => '10.1.2',
+ ctime => 1700000000,
+ rtime => 1700000000123,
+ uid => $UID_A,
+ },
+ 'identity members get parsed',
+ );
+ is(
+ PVE::JSONSchema::print_property_string(
+ PVE::QemuServer::MetaInfo::parse_meta_info($meta),
+ $PVE::QemuServer::MetaInfo::meta_info_fmt,
+ ),
+ $meta,
+ 'printing the parsed property gives the same string',
+ );
+
+ for my $invalid (
+ 'uid=ABCDEFGHIJKLMNOPQRST', 'uid=abcdefghijklmnopqrs', 'rtime=-1', 'rtime=1x',
+ ) {
+ eval {
+ PVE::JSONSchema::parse_property_string(
+ $PVE::QemuServer::MetaInfo::meta_info_fmt, $invalid,
+ );
+ };
+ ok($@, "'$invalid' is rejected");
+ }
+
+ # versions before the identity members only know these two, and drop the whole property
+ my $old_fmt = {
+ ctime => $PVE::QemuServer::MetaInfo::meta_info_fmt->{ctime},
+ 'creation-qemu' => $PVE::QemuServer::MetaInfo::meta_info_fmt->{'creation-qemu'},
+ };
+ eval { PVE::JSONSchema::parse_property_string($old_fmt, $meta) };
+ like(
+ $@,
+ qr/property is not defined in schema/,
+ 'format without identity members rejects it',
+ );
+};
+
+subtest 'identities are enabled, but not for writing temporary configs' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ ok(PVE::QemuConfig->__identity_enabled(), 'enabled for PVE::QemuConfig');
+
+ my $conf = { name => 'template-cmd' };
+ PVE::QemuConfig::NoWrite->mark_config($conf);
+ eval { PVE::QemuConfig->write_config(100, $conf) };
+ is($@, "refusing to write temporary configuration\n", 'NoWrite refuses');
+ eval { PVE::QemuConfig::NoWrite->write_config(100, $conf) };
+ is(
+ $@,
+ "refusing to write temporary configuration\n",
+ 'NoWrite refuses when called directly',
+ );
+ is_deeply(configs(), {}, 'nothing written');
+ is($QemuServerTest::FakeCFS::generated, 0, 'no identity generated');
+};
+
+subtest 'config writes keep the stored identity' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ PVE::QemuConfig->write_config(100, { memory => 512, meta => "rtime=1,uid=$UID_B" });
+ is_deeply(
+ stored_identity(100),
+ { uid => generated_uid(1), rtime => 1700000000000 },
+ 'first write ignores a carried identity and generates one',
+ );
+ like(configs()->{100}, qr/^meta: rtime=1700000000000,uid=\w+$/m,
+ 'written as meta property');
+
+ $QemuServerTest::FakeCFS::now_ms = 1800000000000;
+
+ my $conf = PVE::QemuConfig->load_config(100);
+ $conf->{meta} = "creation-qemu=9.2.0,ctime=1600000000,rtime=1,uid=$UID_B";
+ PVE::QemuConfig->write_config(100, $conf);
+ is_deeply(
+ load_meta(100),
+ {
+ 'creation-qemu' => '9.2.0',
+ ctime => 1600000000,
+ rtime => 1700000000000,
+ uid => generated_uid(1),
+ },
+ 'foreign identity replaced by the stored one, other members kept',
+ );
+
+ $conf = PVE::QemuConfig->load_config(100);
+ delete $conf->{meta};
+ PVE::QemuConfig->write_config(100, $conf);
+ is_deeply(
+ load_meta(100),
+ { rtime => 1700000000000, uid => generated_uid(1) },
+ 'identity kept when the property is gone',
+ );
+
+ configs()->{101} = "memory: 512\nmeta: creation-qemu=8.0.2,ctime=1500000000\n";
+ $conf = PVE::QemuConfig->load_config(101);
+ $conf->{name} = 'old';
+ PVE::QemuConfig->write_config(101, $conf);
+ is_deeply(
+ load_meta(101),
+ {
+ 'creation-qemu' => '8.0.2',
+ ctime => 1500000000,
+ rtime => 1800000000000,
+ uid => generated_uid(2),
+ },
+ 'existing guest without identity gets one on its first write',
+ );
+};
+
+subtest 'create' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ my $reservation = PVE::QemuConfig->create_and_lock_config(100);
+ is_deeply(
+ $reservation,
+ {
+ existed => 0,
+ previous => undef,
+ 'previous-pending-uid' => undef,
+ assigned => { uid => generated_uid(1), rtime => 1700000000000 },
+ },
+ 'reservation stores a new identity',
+ );
+
+ $QemuServerTest::FakeCFS::now_ms = 1700000005000;
+
+ my $conf = { memory => 512, meta => PVE::QemuServer::MetaInfo::new_meta_info_string() };
+ PVE::QemuConfig->write_config(100, $conf);
+ my $meta = load_meta(100);
+ is_deeply(
+ [$meta->@{qw(creation-qemu uid rtime)}],
+ ['10.1.2', generated_uid(1), 1700000000000],
+ 'final config keeps the reserved identity next to the creation info',
+ );
+ ok($meta->{ctime}, 'creation time set');
+};
+
+subtest 'clone' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ configs()->{100} = "memory: 512\nmeta: creation-qemu=9.2.0,ctime=1600000000,"
+ . "rtime=1600000000000,uid=$UID_A\nname: source\n";
+
+ for my $snapshot (0, 1) {
+ my $newid = 200 + $snapshot;
+ my $source = PVE::QemuConfig->load_config(100);
+ if ($snapshot) {
+ # a snapshot can carry any identity, the clone must not take it over either
+ $source = { memory => 256, meta => "rtime=1,uid=$UID_B" };
+ }
+
+ PVE::QemuConfig->create_and_lock_config($newid, 0, 'clone');
+
+ # like the clone API, which copies everything but some snapshot related keys
+ my $newconf = { $source->%*, lock => 'clone', name => "clone-$newid" };
+ delete $newconf->{digest};
+ PVE::QemuConfig->write_config($newid, $newconf);
+ delete $newconf->{lock};
+ PVE::QemuConfig->write_config($newid, $newconf);
+
+ my $uid = generated_uid(1 + $snapshot);
+ is(stored_identity($newid)->{uid}, $uid, "clone $newid has a new uid");
+ is_deeply(
+ load_meta($newid),
+ $snapshot
+ ? { rtime => 1700000000000, uid => $uid }
+ : {
+ 'creation-qemu' => '9.2.0',
+ ctime => 1600000000,
+ rtime => 1700000000000,
+ uid => $uid,
+ },
+ "clone $newid keeps the creation info of its source",
+ );
+ }
+ is(stored_identity(100)->{uid}, $UID_A, 'source keeps its uid');
+};
+
+subtest 'importovf' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ require PVE::CLI::qm;
+
+ my $tmpdir = tempdir('importovf-XXXXXX', DIR => '.', CLEANUP => 1);
+ my $manifest = "$tmpdir/test.ovf";
+ PVE::Tools::file_set_contents($manifest, "<ovf/>\n");
+
+ my $storage_module = Test::MockModule->new('PVE::Storage');
+ $storage_module->mock(
+ config => sub { return {}; },
+ storage_check_enabled => sub { return {}; },
+ );
+ my $ovf_module = Test::MockModule->new('PVE::GuestImport::OVF');
+ $ovf_module->mock(
+ parse_ovf => sub {
+ return { qm => { name => 'imported', memory => 1024, cores => 2 }, disks => [] };
+ },
+ );
+
+ PVE::CLI::qm->importovf({ vmid => 100, manifest => $manifest, storage => 'local' });
+
+ my $conf = PVE::QemuConfig->load_config(100);
+ is($conf->{name}, 'imported', 'config imported');
+ ok(!defined($conf->{lock}), 'create lock removed');
+ is_deeply(
+ stored_identity(100),
+ { uid => generated_uid(1), rtime => 1700000000000 },
+ 'imported VM got a new identity on reservation and kept it',
+ );
+};
+
+done_testing();
diff --git a/src/test/run_parse_config_tests.pl b/src/test/run_parse_config_tests.pl
index c30aad99..566eec4f 100755
--- a/src/test/run_parse_config_tests.pl
+++ b/src/test/run_parse_config_tests.pl
@@ -25,7 +25,7 @@ my $INPUT_DIR = './parse-config-input';
my $EXPECTED_DIR = './parse-config-expected';
# NOTE update when you add/remove tests
-plan tests => 2 * 11;
+plan tests => 2 * 13;
sub run_tests {
my ($strict) = @_;
diff --git a/src/test/run_qemu_restore_config_tests.pl b/src/test/run_qemu_restore_config_tests.pl
index 24abf046..1c6c6fef 100755
--- a/src/test/run_qemu_restore_config_tests.pl
+++ b/src/test/run_qemu_restore_config_tests.pl
@@ -309,7 +309,8 @@ sub restore_api {
return ($err, $output);
}
-my $TARGET_CONF = "memory: 1024\nmeta: creation-qemu=10.0.0,ctime=1700000000\nname: target\n";
+my $TARGET_CONF = "memory: 1024\nmeta: creation-qemu=10.0.0,ctime=1700000000,"
+ . "rtime=1700000000000,uid=tttttttttttttttttttt\nname: target\n";
my $NOT_CLEANED_UP = qr/VM 100 state is NOT cleaned up/;
for my $type (sort keys $ARCHIVES->%*) {
diff --git a/src/test/snapshot-expected/create/qemu-server/401.conf b/src/test/snapshot-expected/create/qemu-server/401.conf
new file mode 100644
index 00000000..87aadf80
--- /dev/null
+++ b/src/test/snapshot-expected/create/qemu-server/401.conf
@@ -0,0 +1,31 @@
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+memory: 8192
+meta: creation-qemu=10.0.0,ctime=1700000000,rtime=1700000000000,uid=abcdefghijklmnopqrst
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+parent: test
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+sockets: 1
+vga: qxl
+
+[test]
+#test comment
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+memory: 8192
+meta: creation-qemu=10.0.0,ctime=1700000000,rtime=1700000000000,uid=abcdefghijklmnopqrst
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+snaptime: 1234567890
+sockets: 1
+vga: qxl
diff --git a/src/test/snapshot-expected/create/qemu-server/402.conf b/src/test/snapshot-expected/create/qemu-server/402.conf
new file mode 100644
index 00000000..976c0a39
--- /dev/null
+++ b/src/test/snapshot-expected/create/qemu-server/402.conf
@@ -0,0 +1,31 @@
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+memory: 8192
+meta: creation-qemu=8.0.2,ctime=1600000000,rtime=1234567890123,uid=qrstuvwxyzbcdefghijk
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+parent: test
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+sockets: 1
+vga: qxl
+
+[test]
+#test comment
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+memory: 8192
+meta: creation-qemu=8.0.2,ctime=1600000000
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+snaptime: 1234567890
+sockets: 1
+vga: qxl
diff --git a/src/test/snapshot-expected/rollback/qemu-server/401.conf b/src/test/snapshot-expected/rollback/qemu-server/401.conf
new file mode 100644
index 00000000..6215ce26
--- /dev/null
+++ b/src/test/snapshot-expected/rollback/qemu-server/401.conf
@@ -0,0 +1,36 @@
+# this is a description
+agent: 1
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+memory: 8192
+meta: creation-qemu=9.2.0,ctime=1600000000,rtime=1700000000000,uid=abcdefghijklmnopqrst
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+parent: test
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+sockets: 1
+vga: qxl
+
+[test]
+#test comment
+agent: 1
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+machine: q35
+memory: 8192
+meta: creation-qemu=9.2.0,ctime=1600000000,rtime=1600000000000,uid=bbbbbbbbbbbbbbbbbbbb
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+snaptime: 1234567890
+sockets: 1
+vga: qxl
+vmstate: somestorage:state-volume
diff --git a/src/test/snapshot-expected/rollback/qemu-server/402.conf b/src/test/snapshot-expected/rollback/qemu-server/402.conf
new file mode 100644
index 00000000..a10bfa13
--- /dev/null
+++ b/src/test/snapshot-expected/rollback/qemu-server/402.conf
@@ -0,0 +1,35 @@
+# this is a description
+agent: 1
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+memory: 8192
+meta: rtime=1700000000000,uid=abcdefghijklmnopqrst
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+parent: test
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+sockets: 1
+vga: qxl
+
+[test]
+#test comment
+agent: 1
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+machine: q35
+memory: 8192
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+snaptime: 1234567890
+sockets: 1
+vga: qxl
+vmstate: somestorage:state-volume
diff --git a/src/test/snapshot-input/create/qemu-server/401.conf b/src/test/snapshot-input/create/qemu-server/401.conf
new file mode 100644
index 00000000..41ccdae5
--- /dev/null
+++ b/src/test/snapshot-input/create/qemu-server/401.conf
@@ -0,0 +1,13 @@
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+memory: 8192
+meta: creation-qemu=10.0.0,ctime=1700000000,rtime=1700000000000,uid=abcdefghijklmnopqrst
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+sockets: 1
+vga: qxl
diff --git a/src/test/snapshot-input/create/qemu-server/402.conf b/src/test/snapshot-input/create/qemu-server/402.conf
new file mode 100644
index 00000000..e03717b6
--- /dev/null
+++ b/src/test/snapshot-input/create/qemu-server/402.conf
@@ -0,0 +1,13 @@
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+memory: 8192
+meta: creation-qemu=8.0.2,ctime=1600000000
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+sockets: 1
+vga: qxl
diff --git a/src/test/snapshot-input/rollback/qemu-server/401.conf b/src/test/snapshot-input/rollback/qemu-server/401.conf
new file mode 100644
index 00000000..7895e0df
--- /dev/null
+++ b/src/test/snapshot-input/rollback/qemu-server/401.conf
@@ -0,0 +1,35 @@
+# this is a description
+agent: 1
+bootdisk: ide2
+cores: 2
+ide0: local:snapshotable-disk-1,size=32G
+ide2: none,media=cdrom
+memory: 4096
+meta: creation-qemu=10.0.0,ctime=1700000000,rtime=1700000000000,uid=abcdefghijklmnopqrst
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+parent: test
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+sockets: 2
+
+[test]
+#test comment
+agent: 1
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+machine: q35
+memory: 8192
+meta: creation-qemu=9.2.0,ctime=1600000000,rtime=1600000000000,uid=bbbbbbbbbbbbbbbbbbbb
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+snaptime: 1234567890
+sockets: 1
+vga: qxl
+vmstate: somestorage:state-volume
diff --git a/src/test/snapshot-input/rollback/qemu-server/402.conf b/src/test/snapshot-input/rollback/qemu-server/402.conf
new file mode 100644
index 00000000..0fa7fe51
--- /dev/null
+++ b/src/test/snapshot-input/rollback/qemu-server/402.conf
@@ -0,0 +1,34 @@
+# this is a description
+agent: 1
+bootdisk: ide2
+cores: 2
+ide0: local:snapshotable-disk-1,size=32G
+ide2: none,media=cdrom
+memory: 4096
+meta: creation-qemu=10.0.0,ctime=1700000000,rtime=1700000000000,uid=abcdefghijklmnopqrst
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+parent: test
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+sockets: 2
+
+[test]
+#test comment
+agent: 1
+bootdisk: ide0
+cores: 4
+ide0: local:snapshotable-disk-1,discard=on,size=32G
+ide2: none,media=cdrom
+machine: q35
+memory: 8192
+name: win
+net0: e1000=12:34:56:78:90:12,bridge=somebr0,firewall=1
+numa: 0
+ostype: win7
+smbios1: uuid=01234567-890a-bcde-f012-34567890abcd
+snaptime: 1234567890
+sockets: 1
+vga: qxl
+vmstate: somestorage:state-volume
diff --git a/src/test/snapshot-test.pm b/src/test/snapshot-test.pm
index 44e55d6a..84629f00 100644
--- a/src/test/snapshot-test.pm
+++ b/src/test/snapshot-test.pm
@@ -5,6 +5,7 @@ use warnings;
use lib qw(..);
+use PVE::GuestHelpers;
use PVE::Storage;
use PVE::Storage::Plugin;
use PVE::QemuServer;
@@ -30,6 +31,9 @@ my $freeze_possible;
my $stop_possible;
my $save_vmstate_works;
my $vm_mon = {};
+my $real_write_config;
+
+my $original_write_config = \&PVE::QemuConfig::write_config;
# Mocked methods
@@ -296,6 +300,13 @@ sub cfs_config_path {
return "snapshot-working/$node/qemu-server/$vmid.conf";
}
+# the same file that mocked_get_config reads, so that nothing looks at the host's /etc/pve
+sub config_file {
+ my ($class, $vmid, $node) = @_;
+
+ return $class->cfs_config_path($vmid, $node);
+}
+
sub load_config {
my ($class, $vmid, $node) = @_;
@@ -310,18 +321,10 @@ sub load_config {
sub write_config {
my ($class, $vmid, $conf) = @_;
- my $filename = $class->cfs_config_path($vmid);
-
- if ($conf->{snapshots}) {
- foreach my $snapname (keys %{ $conf->{snapshots} }) {
- $conf->{snapshots}->{$snapname}->{snaptime} = "1234567890"
- if $conf->{snapshots}->{$snapname}->{snaptime};
- }
- }
+ # the real method with the guest identity handling, which writes through mocked_cfs_write_file
+ return $original_write_config->($class, $vmid, $conf) if $real_write_config;
- my $raw = PVE::QemuServer::write_vm_config($filename, $conf);
-
- PVE::Tools::file_set_contents($filename, $raw);
+ mocked_cfs_write_file($class->cfs_config_path($vmid), $conf);
}
sub has_feature {
@@ -345,6 +348,31 @@ sub assert_config_exists_on_node {
}
# END mocked PVE::QemuConfig methods
+# BEGIN mocked PVE::Cluster methods
+
+sub mocked_get_config {
+ my ($path) = @_;
+
+ return -e $path ? PVE::Tools::file_get_contents($path) : undef;
+}
+
+sub mocked_cfs_write_file {
+ my ($filename, $conf) = @_;
+
+ if ($conf->{snapshots}) {
+ foreach my $snapname (keys %{ $conf->{snapshots} }) {
+ $conf->{snapshots}->{$snapname}->{snaptime} = "1234567890"
+ if $conf->{snapshots}->{$snapname}->{snaptime};
+ }
+ }
+
+ my $raw = PVE::QemuServer::write_vm_config($filename, $conf);
+
+ PVE::Tools::file_set_contents($filename, $raw);
+}
+
+# END mocked PVE::Cluster methods
+
# BEGIN mocked PVE::QemuServer::Helpers methods
sub vm_running_locally {
@@ -441,6 +469,16 @@ sub vm_stop {
PVE::Tools::run_command("rm -rf snapshot-working");
PVE::Tools::run_command("cp -a snapshot-input snapshot-working");
+$PVE::GuestHelpers::lockdir = "snapshot-working/lock";
+
+my $cluster_module = Test::MockModule->new('PVE::Cluster');
+$cluster_module->mock('get_config', \&mocked_get_config);
+$cluster_module->mock('cfs_write_file', \&mocked_cfs_write_file);
+
+my $guest_identity_module = Test::MockModule->new('PVE::GuestIdentity');
+$guest_identity_module->mock('__random_bytes', sub { return '0123456789abcdefghij'; });
+$guest_identity_module->mock('__time_ms', sub { return 1234567890123; });
+
my $qemu_migrate_helpers_module = Test::MockModule->new('PVE::QemuMigrate::Helpers');
$qemu_migrate_helpers_module->mock('set_migration_caps', \&set_migration_caps);
@@ -453,6 +491,7 @@ $qemu_monitor_module->mock('qmp_cmd', \&qmp_cmd);
my $qemu_config_module = Test::MockModule->new('PVE::QemuConfig');
$qemu_config_module->mock('config_file_lock', \&config_file_lock);
$qemu_config_module->mock('cfs_config_path', \&cfs_config_path);
+$qemu_config_module->mock('config_file', \&config_file);
$qemu_config_module->mock('load_config', \&load_config);
$qemu_config_module->mock('write_config', \&write_config);
$qemu_config_module->mock('has_feature', \&has_feature);
@@ -641,6 +680,16 @@ testcase_create("303", "test", 1, "test comment", "storage activation failed\n\n
$activate_storage_possible = 1;
+$real_write_config = 1;
+
+printf("Successful snapshot_create keeping the guest identity\n");
+testcase_create("401", "test", 0, "test comment", "", { "local:snapshotable-disk-1" => "test" });
+
+printf("Successful snapshot_create assigning a guest identity to a config without one\n");
+testcase_create("402", "test", 0, "test comment", "", { "local:snapshotable-disk-1" => "test" });
+
+$real_write_config = 0;
+
$nodename = "delete";
printf("\n");
printf("Running delete tests\n");
@@ -777,6 +826,17 @@ testcase_rollback("303", "test", "storage activation failed\n");
$activate_storage_possible = 1;
+$real_write_config = 1;
+
+printf("Successful snapshot_rollback keeping the current guest identity, not the snapshot's\n");
+testcase_rollback("401", "test", "", { "local:snapshotable-disk-1" => "test" });
+
+printf(
+ "Successful snapshot_rollback to a snapshot without meta info, keeping the guest identity\n");
+testcase_rollback("402", "test", "", { "local:snapshotable-disk-1" => "test" });
+
+$real_write_config = 0;
+
done_testing();
1;
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH qemu-server v3 17/38] restore: take over the guest uid from the backup where possible
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (15 preceding siblings ...)
2026-11-18 4:37 ` [PATCH qemu-server v3 16/38] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 18/38] api: remote migration: take over the guest uid from the source Thomas Lamprecht
` (22 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A restore to an unused VMID, or over a VM without an identity, should
keep the uid of the backup if no other guest in the cluster uses it.
Take over that uid through the cluster-wide check of pve-guest-common
after the data is restored and the final config is written, while
holding the config lock. A VM that already had an identity keeps it,
and 'unique' prevents taking over the uid. Log the outcome, and only
warn if taking over the uid fails, as the restored VM has an identity
of its own.
Refuse to restore a backup with a different uid over an existing VM
unless 'allow-identity-mismatch' is set in the create API or qmrestore.
This is independent of 'force', which permits overwriting the VM. With
the override, the VM keeps its identity, unless it has the uid of the
backup as pending uid from an unfinished restore.
For PBS, backup providers and VMA, compare the uids before resetting
the replication state, restoring the firewall config or replacing the
data of the target. For VMA this also works for standard input, through
the header callback. Read the uid from the config that the restore
itself uses, not from the best-effort extraction of the API, and ignore
unknown members of 'meta', so that the uid of a backup from a newer
version is not lost when the property fails to parse.
Take over the uid only after the live phase, the conversion to a
template and, with 'unique', the registration of the new MAC addresses.
A failed restore must not take the uid and so keep a retry at another
VMID from getting it.
For a retry over a VM left behind by an unfinished restore to work,
record the uid of the backup as a pending change of 'meta'. Record it
after the uid check if the restore could take the uid over, also with
'unique', and keep it in all configs that the restore writes. A VM
with that pending uid can be restored over without the override, and
without 'unique' the retry takes the uid over if no other guest uses
it. A pending uid does not count as in use.
Taking over the uid removes the pending uid, whatever the outcome. If
the cluster-wide lock cannot be taken, it stays for a retry. Starting
the VM for a live restore keeps it while the 'create' lock is held.
Otherwise, applying pending changes drops it, and a clone does not get
it.
Legacy tar archives predate guest identities, so their restore does not
handle a uid.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
# Conflicts:
# src/test/run_qemu_restore_config_tests.pl
---
changes v2 -> v3:
* group the imports of the test script
changes v1 -> v2:
* use the shared helpers of pve-guest-common, one check helper instead
of three
* no identity handling for legacy tar archives
* describe the effect of 'unique', shorter description and a default for
'allow-identity-mismatch' (Dominik)
* starting a VM left behind by a failed restore warns that the pending
uid gets dropped
src/PVE/API2/Qemu.pm | 22 +-
src/PVE/CLI/qmrestore.pm | 11 +-
src/PVE/QemuServer.pm | 99 +++-
src/test/QemuServerTest/FakeCFS.pm | 3 +
src/test/restore-config-expected/150.conf | 9 +
src/test/restore-config-input/150.conf | 10 +
src/test/run_api_tests.pl | 13 +
src/test/run_guest_identity_tests.pl | 78 +++
src/test/run_qemu_restore_config_tests.pl | 561 +++++++++++++++++++++-
9 files changed, 774 insertions(+), 32 deletions(-)
create mode 100644 src/test/restore-config-expected/150.conf
create mode 100644 src/test/restore-config-input/150.conf
diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index 5954dd8e..c10cc945 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -1247,7 +1247,18 @@ __PACKAGE__->register_method({
unique => {
optional => 1,
type => 'boolean',
- description => "Assign a unique random ethernet address.",
+ description =>
+ "Assign a unique random ethernet address and do not take over"
+ . " the guest uid of the backup.",
+ requires => 'archive',
+ },
+ 'allow-identity-mismatch' => {
+ optional => 1,
+ type => 'boolean',
+ default => 0,
+ description => "Allow overwriting an existing VM with a backup of another"
+ . " guest, as detected by a different 'uid' in 'meta'. The VM keeps its"
+ . " own uid.",
requires => 'archive',
},
'live-restore' => {
@@ -1318,6 +1329,7 @@ __PACKAGE__->register_method({
my $ha_managed = extract_param($param, 'ha-managed');
my $storage = extract_param($param, 'storage');
my $unique = extract_param($param, 'unique');
+ my $allow_identity_mismatch = extract_param($param, 'allow-identity-mismatch');
my $live_restore = extract_param($param, 'live-restore');
my $extraction_storage = extract_param($param, 'import-working-storage');
@@ -1453,6 +1465,8 @@ __PACKAGE__->register_method({
live => $live_restore,
override_conf => $param,
progress => $restore_progress,
+ reservation => $reservation,
+ 'allow-identity-mismatch' => $allow_identity_mismatch,
};
if (my $volid = $archive->{volid}) {
# best effort, real check is after restoring!
@@ -1508,6 +1522,8 @@ __PACKAGE__->register_method({
PVE::QemuServer::Network::create_ifaces_ipams_ips($restored_conf, $vmid)
if $unique;
+
+ PVE::QemuServer::restore_adopt_identity($vmid, $restore_options);
};
my $restore_locked = sub {
@@ -4694,6 +4710,10 @@ __PACKAGE__->register_method({
next if PVE::QemuServer::drive_is_cloudinit($drive);
push @$vollist, $volid;
}
+ } elsif ($opt eq 'pending') {
+ # a pending uid only belongs to the source, see PVE::GuestIdentity
+ $newconf->{pending} = { $value->%* };
+ PVE::GuestIdentity::set_pending_uid($newconf, undef);
} else {
# copy everything else
$newconf->{$opt} = $value;
diff --git a/src/PVE/CLI/qmrestore.pm b/src/PVE/CLI/qmrestore.pm
index abfa2de7..ab7deb8e 100755
--- a/src/PVE/CLI/qmrestore.pm
+++ b/src/PVE/CLI/qmrestore.pm
@@ -52,7 +52,16 @@ __PACKAGE__->register_method({
unique => {
optional => 1,
type => 'boolean',
- description => "Assign a unique random ethernet address.",
+ description => "Assign a unique random ethernet address and do not take over"
+ . " the guest uid of the backup.",
+ },
+ 'allow-identity-mismatch' => {
+ optional => 1,
+ type => 'boolean',
+ default => 0,
+ description => "Allow overwriting an existing VM with a backup of another"
+ . " guest, as detected by a different 'uid' in 'meta'. The VM keeps its own"
+ . " uid.",
},
pool => {
optional => 1,
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 855ede59..e6f3998f 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -32,6 +32,7 @@ use PVE::DataCenterConfig;
use PVE::Exception qw(raise raise_param_exc);
use PVE::Format qw(render_duration render_bytes);
use PVE::GuestHelpers qw(safe_string_ne safe_num_ne safe_boolean_ne);
+use PVE::GuestIdentity;
use PVE::Mapping::Dir;
use PVE::Mapping::PCI;
use PVE::Mapping::USB;
@@ -4430,19 +4431,6 @@ for my $opt (keys %$confdesc_cloudinit) {
$fast_plug_option->{$opt} = 1;
}
-# 'meta' is read-only, so only a manual edit or an older version can have queued a change of it.
-# Drop such a change instead of applying it.
-my sub drop_pending_meta {
- my ($conf) = @_;
-
- my $pending_delete_hash = PVE::QemuConfig->parse_pending_delete($conf->{pending}->{delete});
- return if !defined($conf->{pending}->{meta}) && !$pending_delete_hash->{meta};
-
- warn "ignoring pending change of read-only option 'meta'\n";
- delete $conf->{pending}->{meta};
- PVE::QemuConfig->remove_from_pending_delete($conf, 'meta');
-}
-
# hotplug changes in [PENDING]
# $selection hash can be used to only apply specified options, for
# example: { cores => 1 } (only apply changed 'cores')
@@ -4450,7 +4438,7 @@ my sub drop_pending_meta {
sub vmconfig_hotplug_pending {
my ($vmid, $conf, $storecfg, $selection, $errors) = @_;
- drop_pending_meta($conf);
+ PVE::QemuConfig->drop_pending_meta($conf);
my $defaults = load_defaults();
my $arch = PVE::QemuServer::Helpers::get_vm_arch($conf);
@@ -4850,7 +4838,13 @@ sub vmconfig_apply_pending {
return if !scalar(keys %{ $conf->{pending} });
- drop_pending_meta($conf);
+ # a restore keeps its pending uid while it holds the 'create' lock, also when it starts the VM
+ # for a live restore, and removes it itself, see restore_check_identity
+ my $restore_uid =
+ ($conf->{lock} // '') eq 'create' ? PVE::GuestIdentity::get_pending_uid($conf) : undef;
+ PVE::GuestIdentity::set_pending_uid($conf, undef) if defined($restore_uid);
+
+ PVE::QemuConfig->drop_pending_meta($conf);
my $add_apply_error = sub {
my ($opt, $msg) = @_;
@@ -4959,6 +4953,8 @@ sub vmconfig_apply_pending {
}
}
+ PVE::GuestIdentity::set_pending_uid($conf, $restore_uid) if defined($restore_uid);
+
# write all changes at once to avoid unnecessary i/o
PVE::QemuConfig->write_config($vmid, $conf);
if ($generate_cloudinit) {
@@ -6735,6 +6731,61 @@ sub restore_merge_config {
return $backup_conf;
}
+# Checks the guest identity of the backup against the one of the target VM and prepares taking over
+# its uid. Must run before anything of the target VM changes. Returns the identity of the backup.
+#
+# Restoring over a VM that has another uid is refused unless 'allow-identity-mismatch' is set, as
+# that is most likely a backup of another guest. A VM with the uid of the backup as pending uid is
+# a leftover of an earlier restore of it and matches.
+#
+# If the restore could take the uid of the backup over at its end, see
+# PVE::GuestIdentity::adoptable_uid, the uid becomes the pending uid of the VM until then, so that a
+# VM left behind by a failure or an abort of the restore still has it for a retry, see
+# restore_write_config. That holds with 'unique' too, which only keeps the restore itself from
+# taking the uid over.
+sub restore_check_identity {
+ my ($vmid, $backup_conf_raw, $opts) = @_;
+
+ my $backup = PVE::GuestIdentity::parse_identity_from_raw_config($backup_conf_raw);
+ my $uid = $backup ? $backup->{uid} : undef;
+ my $reservation = $opts->{reservation};
+
+ PVE::GuestIdentity::assert_backup_matches_target(
+ 'PVE::QemuConfig',
+ $vmid,
+ $reservation,
+ $uid,
+ $opts->{'allow-identity-mismatch'},
+ );
+
+ $opts->{'backup-uid'} = $uid;
+ $opts->{'pending-uid'} = PVE::GuestIdentity::adoptable_uid($reservation, $uid);
+ PVE::QemuConfig->record_pending_uid($vmid, $uid) if defined($opts->{'pending-uid'});
+
+ return $backup;
+}
+
+# Writes $conf, a config that the restore assembled itself, like a temporary or the restored one,
+# with the pending uid of the restore, see restore_check_identity.
+my sub restore_write_config {
+ my ($vmid, $conf, $opts) = @_;
+
+ PVE::GuestIdentity::set_pending_uid($conf, $opts->{'pending-uid'});
+ PVE::QemuConfig->write_config($vmid, $conf);
+}
+
+# Takes over the uid of the backup for the restored VM where the identity rules allow it, see
+# PVE::GuestIdentity::adopt, which also removes the pending uid. Must be the last step of a
+# successful restore, with the config lock still held, so that a failure before leaves the pending
+# uid behind. A failure only warns, as the restored VM keeps the identity of its reservation.
+sub restore_adopt_identity {
+ my ($vmid, $opts) = @_;
+
+ my $uid = $opts->{unique} ? undef : $opts->{'backup-uid'};
+
+ PVE::GuestIdentity::adopt_from_backup('PVE::QemuConfig', $vmid, $uid, $opts->{reservation});
+}
+
sub scan_volids {
my ($cfg, $vmid) = @_;
@@ -6981,6 +7032,8 @@ sub restore_proxmox_backup_archive {
$param = [$pbs_backup_name, "qemu-server.conf", $cfgfn];
PVE::Storage::PBSPlugin::run_raw_client_cmd($scfg, $storeid, $cmd, $param);
+ restore_check_identity($vmid, PVE::Tools::file_get_contents($cfgfn), $options);
+
$options->{progress}->{'target-changed'} = 1;
PVE::ReplicationState::delete_guest_states($vmid);
@@ -7001,7 +7054,7 @@ sub restore_proxmox_backup_archive {
# fixme: rate limit?
# create empty/temp config
- PVE::QemuConfig->write_config($vmid, { memory => 128, lock => 'create' });
+ restore_write_config($vmid, { memory => 128, lock => 'create' }, $options);
$restore_cleanup_oldconf->($storecfg, $vmid, $oldconf, $virtdev_hash) if $oldconf;
@@ -7086,7 +7139,7 @@ sub restore_proxmox_backup_archive {
my $new_conf = restore_merge_config($conffile, $new_conf_raw, $options->{override_conf});
check_restore_permissions($rpcenv, $user, $new_conf);
- PVE::QemuConfig->write_config($vmid, $new_conf);
+ restore_write_config($vmid, $new_conf, $options);
eval { rescan($vmid, 1); };
warn $@ if $@;
@@ -7165,6 +7218,8 @@ sub restore_external_archive {
my $data = $backup_provider->archive_get_guest_config($volname)
or die "backup provider failed to extract guest configuration\n";
+ restore_check_identity($vmid, $data, $options);
+
$options->{progress}->{'target-changed'} = 1;
PVE::ReplicationState::delete_guest_states($vmid);
@@ -7182,7 +7237,7 @@ sub restore_external_archive {
$parse_backup_hints->($rpcenv, $user, $storecfg, $fh, $devinfo, $options);
# create empty/temp config
- PVE::QemuConfig->write_config($vmid, { memory => 128, lock => 'create' });
+ restore_write_config($vmid, { memory => 128, lock => 'create' }, $options);
$restore_cleanup_oldconf->($storecfg, $vmid, $oldconf, $virtdev_hash) if $oldconf;
@@ -7260,7 +7315,7 @@ sub restore_external_archive {
my $new_conf = restore_merge_config($conffile, $new_conf_raw, $options->{override_conf});
check_restore_permissions($rpcenv, $user, $new_conf);
- PVE::QemuConfig->write_config($vmid, $new_conf);
+ restore_write_config($vmid, $new_conf, $options);
eval { rescan($vmid, 1); };
warn $@ if $@;
@@ -7582,6 +7637,8 @@ sub restore_vma_archive {
my $print_devmap = sub {
my $cfgfn = "$tmpdir/qemu-server.conf";
+ restore_check_identity($vmid, PVE::Tools::file_get_contents($cfgfn), $opts);
+
$opts->{progress}->{'target-changed'} = 1;
PVE::ReplicationState::delete_guest_states($vmid);
@@ -7613,7 +7670,7 @@ sub restore_vma_archive {
# create empty/temp config
if ($oldconf) {
- PVE::QemuConfig->write_config($vmid, { memory => 128, lock => 'create' });
+ restore_write_config($vmid, { memory => 128, lock => 'create' }, $opts);
$restore_cleanup_oldconf->($cfg, $vmid, $oldconf, $virtdev_hash);
}
@@ -7721,7 +7778,7 @@ sub restore_vma_archive {
my $new_conf = restore_merge_config($conffile, $new_conf_raw, $opts->{override_conf});
check_restore_permissions($rpcenv, $user, $new_conf);
- PVE::QemuConfig->write_config($vmid, $new_conf);
+ restore_write_config($vmid, $new_conf, $opts);
eval { rescan($vmid, 1); };
warn $@ if $@;
diff --git a/src/test/QemuServerTest/FakeCFS.pm b/src/test/QemuServerTest/FakeCFS.pm
index 99d09f87..232cfeff 100644
--- a/src/test/QemuServerTest/FakeCFS.pm
+++ b/src/test/QemuServerTest/FakeCFS.pm
@@ -27,6 +27,7 @@ our $now_ms = 1700000000000;
our $generated = 0;
our $lock_domain_error;
our $writes = [];
+our $written = {}; # VMID => the written raw configs, in the order of the writes
my $tmpdir = tempdir('fake-cfs-XXXXXX', DIR => '.', CLEANUP => 1);
make_path("$tmpdir/lock");
@@ -40,6 +41,7 @@ sub reset() {
$generated = 0;
$lock_domain_error = undef;
$writes = [];
+ $written = {};
}
my sub vmid_of_path($path) {
@@ -127,6 +129,7 @@ $cluster_module->mock(
$configs->{$vmid} = PVE::QemuServer::write_vm_config($path, $conf);
sync_config_file($vmid);
push $writes->@*, $vmid;
+ push $written->{$vmid}->@*, $configs->{$vmid};
return;
},
cfs_update => sub {
diff --git a/src/test/restore-config-expected/150.conf b/src/test/restore-config-expected/150.conf
new file mode 100644
index 00000000..a8e8909c
--- /dev/null
+++ b/src/test/restore-config-expected/150.conf
@@ -0,0 +1,9 @@
+# VM with guest identity
+boot: order=scsi0
+cores: 1
+memory: 512
+meta: creation-qemu=10.1.2,ctime=1762000000,rtime=1762000000123,uid=abcdefghijklmnopqrst
+name: identity
+scsi0: target:150/vm-150-disk-0.qcow2,size=4G
+scsihw: virtio-scsi-pci
+vmgenid: 0
diff --git a/src/test/restore-config-input/150.conf b/src/test/restore-config-input/150.conf
new file mode 100644
index 00000000..927be6dc
--- /dev/null
+++ b/src/test/restore-config-input/150.conf
@@ -0,0 +1,10 @@
+# VM with guest identity
+boot: order=scsi0
+cores: 1
+memory: 512
+meta: creation-qemu=10.1.2,ctime=1762000000,rtime=1762000000123,uid=abcdefghijklmnopqrst
+name: identity
+scsi0: mydir:150/vm-150-disk-0.qcow2,size=4G
+scsihw: virtio-scsi-pci
+vmgenid: 0
+#qmdump#map:scsi0:drive-scsi0:mydir:qcow2:
diff --git a/src/test/run_api_tests.pl b/src/test/run_api_tests.pl
index cc9cbadf..15c6acfe 100755
--- a/src/test/run_api_tests.pl
+++ b/src/test/run_api_tests.pl
@@ -13,6 +13,7 @@ use Test::MockModule;
use Test::More;
use PVE::API2::Qemu;
+use PVE::CLI::qmrestore;
use PVE::GuestIdentity;
use PVE::RPCEnvironment;
@@ -132,6 +133,18 @@ is(
'update_vm: reverting other options passes the parameter checks',
);
+is_deeply(
+ api_error('create_vm', { 'allow-identity-mismatch' => 1 }),
+ { archive => "missing property required by 'allow-identity-mismatch'" },
+ "create_vm: 'allow-identity-mismatch' only for restores",
+);
+my $qmrestore_param = PVE::CLI::qmrestore->map_method_by_name('qmrestore')->{parameters};
+is(
+ $qmrestore_param->{properties}->{'allow-identity-mismatch'}->{type},
+ 'boolean',
+ "qmrestore has 'allow-identity-mismatch'",
+);
+
subtest 'pending changes of meta are dropped' => sub {
my $meta = 'creation-qemu=10.1.2,ctime=1762000000';
my $stored;
diff --git a/src/test/run_guest_identity_tests.pl b/src/test/run_guest_identity_tests.pl
index 16f012d4..1e60cbf7 100755
--- a/src/test/run_guest_identity_tests.pl
+++ b/src/test/run_guest_identity_tests.pl
@@ -12,12 +12,15 @@ use Test::MockModule;
use Test::More;
use PVE::JSONSchema;
+use PVE::RPCEnvironment;
use PVE::QemuConfig;
use PVE::QemuConfig::NoWrite;
use PVE::QemuServer;
use PVE::QemuServer::MetaInfo;
+use PVE::API2::Qemu;
+
use QemuServerTest::FakeCFS;
my $UID_A = 'abcdefghijklmnopqrst';
@@ -26,6 +29,9 @@ my $UID_B = 'bbbbbbbbbbbbbbbbbbbb';
my $helpers_module = Test::MockModule->new('PVE::QemuServer::Helpers');
$helpers_module->mock(kvm_user_version => sub { return '10.1.2'; });
+my $rpcenv = PVE::RPCEnvironment->init('cli');
+$rpcenv->set_user('root@pam');
+
sub configs { return $QemuServerTest::FakeCFS::configs; }
sub generated_uid { return QemuServerTest::FakeCFS::generated_uid(@_); }
sub stored_identity { return QemuServerTest::FakeCFS::stored_identity(@_); }
@@ -222,6 +228,78 @@ subtest 'clone' => sub {
is(stored_identity(100)->{uid}, $UID_A, 'source keeps its uid');
};
+subtest 'clone does not carry the pending uid of its source' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ # a VM left behind by a restore of a backup with uid B, with another pending change
+ configs()->{100} =
+ "memory: 512\nmeta: rtime=1600000000000,uid=$UID_A\nname: source\n"
+ . "scsi0: local:100/vm-100-disk-0.raw,size=4G\n\n[PENDING]\nmemory: 1024\n"
+ . "meta: uid=$UID_B\n";
+
+ my $rpcenv_module = Test::MockModule->new('PVE::RPCEnvironment');
+ $rpcenv_module->mock(
+ check => sub { return 1; },
+ check_vm_perm => sub { return 1; },
+ fork_worker => sub {
+ my ($self, $type, $id, $user, $code) = @_;
+ $code->('UPID:test');
+ return 'UPID:test';
+ },
+ );
+ $QemuServerTest::FakeCFS::cluster_module->mock(check_cfs_quorum => sub { return 1; });
+ my $storage_module = Test::MockModule->new('PVE::Storage');
+ $storage_module->mock(
+ activate_volumes => sub { return; },
+ config => sub {
+ return {
+ ids => {
+ local =>
+ { type => 'dir', path => '/var/lib/vz', content => { images => 1 } },
+ },
+ };
+ },
+ deactivate_volumes => sub { return; },
+ get_bandwidth_limit => sub { return undef; },
+ volume_has_feature => sub { return 1; },
+ );
+ my $qemu_server_module = Test::MockModule->new('PVE::QemuServer');
+ $qemu_server_module->mock(
+ check_running => sub { return 0; },
+ clone_disk => sub {
+ my ($storecfg, $source, $dest) = @_;
+ return { file => "local:$dest->{vmid}/vm-$dest->{vmid}-disk-0.raw", size => '4G' };
+ },
+ );
+ my $firewall_module = Test::MockModule->new('PVE::Firewall');
+ $firewall_module->mock(clone_vmfw_conf => sub { return; });
+ my $network_module = Test::MockModule->new('PVE::QemuServer::Network');
+ $network_module->mock(create_ifaces_ipams_ips => sub { return; });
+
+ my $warnings = '';
+ {
+ local $SIG{__WARN__} = sub { $warnings .= $_[0]; };
+ open(my $out_fh, '>', \my $output);
+ my $oldfh = select($out_fh);
+ eval { PVE::API2::Qemu->clone_vm({ node => 'localhost', vmid => 100, newid => 300 }) };
+ select($oldfh);
+ is($@, '', 'clone succeeds');
+ }
+
+ my $writes = $QemuServerTest::FakeCFS::written->{300};
+ ok(scalar(grep { m/^scsi0: local:300\//m } $writes->@*) >= 2,
+ 'clone writes the disk first');
+ ok(!scalar(grep { m/uid=$UID_B/ } $writes->@*),
+ 'no write of the clone has the pending uid');
+ like(
+ $warnings,
+ qr/pending changes for 'memory', discarding for clone/,
+ 'other pending changes are discarded as before',
+ );
+ unlike(configs()->{300}, qr/^\[PENDING\]/m, 'clone has no pending section');
+ like(configs()->{100}, qr/^meta: uid=$UID_B$/m, 'source keeps its pending uid');
+};
+
subtest 'importovf' => sub {
QemuServerTest::FakeCFS::reset();
diff --git a/src/test/run_qemu_restore_config_tests.pl b/src/test/run_qemu_restore_config_tests.pl
index 1c6c6fef..10736d9e 100755
--- a/src/test/run_qemu_restore_config_tests.pl
+++ b/src/test/run_qemu_restore_config_tests.pl
@@ -16,9 +16,11 @@ use Test::More;
use File::Basename;
use PVE::API2::Qemu;
+use PVE::GuestIdentity;
use PVE::QemuConfig;
use PVE::QemuServer;
use PVE::QemuServer::Helpers;
+use PVE::QemuServer::MetaInfo;
use PVE::RPCEnvironment;
use PVE::Tools qw(dir_glob_foreach file_get_contents file_set_contents);
@@ -143,15 +145,6 @@ $firewall_module->mock(
},
);
-my $tools_module = Test::MockModule->new('PVE::Tools');
-$tools_module->mock(
- file_copy => sub {
- my ($src, $dst) = @_;
- die "file_copy (mocked) - unexpected destination '$dst'\n" if $dst !~ m|/firewall/|;
- $calls->{firewall}++;
- },
-);
-
my $storage_module = Test::MockModule->new('PVE::Storage');
$storage_module->mock(
abs_filesystem_path => sub {
@@ -217,6 +210,10 @@ my $qemu_server_module = Test::MockModule->new('PVE::QemuServer');
$qemu_server_module->mock(
check_running => sub { return 0; },
pbs_live_restore => sub {
+ my ($vmid, $conf, $storecfg) = @_;
+ # like the start of the VM for the live phase
+ PVE::QemuServer::vmconfig_apply_pending($vmid, $conf, $storecfg)
+ if scalar(keys(($conf->{pending} // {})->%*));
die "live restore failed\n" if $fail_at eq 'live';
$calls->{live_restore}++;
},
@@ -552,4 +549,550 @@ subtest 'failure after the data was restored keeps the VM' => sub {
$fail_at = '';
+# Guest identity of restored VMs
+
+my $UID_BACKUP = 'bbbbbbbbbbbbbbbbbbbb';
+my $UID_TARGET = 'tttttttttttttttttttt';
+
+my $TARGET_CONF_NO_UID = "memory: 1024\nmeta: creation-qemu=8.0.2,ctime=1500000000\nname: target\n";
+
+my $NOW = 1800000000000;
+my $BACKUP_META = 'creation-qemu=9.2.0,ctime=1600000000';
+my $REFUSED =
+ "backup belongs to another guest: its uid '$UID_BACKUP' differs from uid '$UID_TARGET' of the"
+ . " existing VM 100 - use 'allow-identity-mismatch' to overwrite the VM anyway\n";
+my $IGNORED = "guest identity: ignoring backup uid $UID_BACKUP, VM 100 keeps uid $UID_TARGET";
+# captures the uid that the VM keeps
+my $ADOPT_FAILED = "WARN: guest identity: unable to adopt backup uid $UID_BACKUP, keeping uid"
+ . " ([a-z2-7]{20}) - cfs-lock 'domain-guest-identity' error: no quorum!";
+$ADOPT_FAILED = qr/^$ADOPT_FAILED$/m;
+my $KEPT_TARGET = qr/^guest identity: kept uid $UID_TARGET$/m;
+
+# legacy tar archives get no guest identity handling
+my @IDENTITY_ARCHIVES = grep { $_ ne 'tar' } sort keys $ARCHIVES->%*;
+
+my $cases = [
+ {
+ name => 'free VMID, backup uid unused',
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP",
+ result => { uid => $UID_BACKUP, rtime => $NOW },
+ output => qr/^guest identity: adopted uid $UID_BACKUP from the backup$/m,
+ },
+ {
+ name => 'free VMID, backup uid in use by another guest',
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP",
+ other => "rtime=1,uid=$UID_BACKUP",
+ result => { uid => 'generated', rtime => $NOW },
+ output => qr/: backup uid $UID_BACKUP is in use by guest 300, keeping uid \w{20}$/m,
+ },
+ {
+ name => 'free VMID, backup without uid',
+ backup => $BACKUP_META,
+ result => { uid => 'generated', rtime => $NOW },
+ output => qr/^guest identity: new uid aaaaaaaaaaaaaaaaaaab$/m,
+ },
+ {
+ name => 'free VMID, unique',
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP",
+ param => { unique => 1 },
+ result => { uid => 'generated', rtime => $NOW },
+ output => qr/^guest identity: new uid aaaaaaaaaaaaaaaaaaab$/m,
+ },
+ {
+ name => 'free VMID, identity lock fails',
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP",
+ lock_error => "cfs-lock 'domain-guest-identity' error: no quorum!\n",
+ result => { uid => 'generated', rtime => $NOW },
+ output => $ADOPT_FAILED,
+ },
+ {
+ name => 'free VMID, backup meta with a member unknown to this version',
+ backup => "$BACKUP_META,future=1,rtime=1600000000000,uid=$UID_BACKUP",
+ result => { uid => $UID_BACKUP, rtime => $NOW },
+ # the parser drops the whole property of the backup, as it does for other members
+ meta => { rtime => $NOW, uid => $UID_BACKUP },
+ output => qr/^guest identity: adopted uid $UID_BACKUP from the backup$/m,
+ },
+ {
+ name => 'in place, backup of another guest',
+ target => $TARGET_CONF,
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP",
+ error => $REFUSED,
+ },
+ {
+ name => 'in place, backup of another guest, unique',
+ target => $TARGET_CONF,
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP",
+ param => { unique => 1 },
+ error => $REFUSED,
+ },
+ {
+ name => 'in place, backup of another guest, override',
+ target => $TARGET_CONF,
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP",
+ param => { 'allow-identity-mismatch' => 1 },
+ result => { uid => $UID_TARGET, rtime => 1700000000000 },
+ output => qr/^\Q$IGNORED\E\n(?:.*\n)*guest identity: kept uid $UID_TARGET$/m,
+ },
+ {
+ name => 'in place, backup of the same guest',
+ target => $TARGET_CONF,
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_TARGET",
+ result => { uid => $UID_TARGET, rtime => 1700000000000 },
+ output => $KEPT_TARGET,
+ },
+ {
+ name => 'in place, backup without uid',
+ target => $TARGET_CONF,
+ backup => $BACKUP_META,
+ result => { uid => $UID_TARGET, rtime => 1700000000000 },
+ output => $KEPT_TARGET,
+ },
+ {
+ name => 'in place, target without uid',
+ target => $TARGET_CONF_NO_UID,
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP",
+ result => { uid => $UID_BACKUP, rtime => $NOW },
+ output => qr/^guest identity: adopted uid $UID_BACKUP from the backup$/m,
+ },
+ {
+ name => 'in place, target without uid, unique',
+ target => $TARGET_CONF_NO_UID,
+ backup => "$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP",
+ param => { unique => 1 },
+ result => { uid => 'generated', rtime => $NOW },
+ output => qr/^guest identity: new uid aaaaaaaaaaaaaaaaaaab$/m,
+ },
+];
+
+sub meta_of {
+ my ($vmid) = @_;
+
+ my $conf = PVE::QemuConfig->load_config($vmid);
+ return PVE::QemuServer::MetaInfo::parse_meta_info($conf->{meta});
+}
+
+for my $type (@IDENTITY_ARCHIVES) {
+ for my $case ($cases->@*) {
+ subtest "$type: $case->{name}" => sub {
+ QemuServerTest::FakeCFS::reset();
+ $QemuServerTest::FakeCFS::now_ms = $NOW;
+ QemuServerTest::FakeCFS::set_config(100, $case->{target}) if $case->{target};
+ $QemuServerTest::FakeCFS::other_guests->{300} = $case->{other} if $case->{other};
+ $QemuServerTest::FakeCFS::lock_domain_error = $case->{lock_error};
+
+ my $param = { ($case->{param} // {})->%* };
+ $param->{force} = 1 if $case->{target};
+ my ($err, $output) = restore_api($type, backup_config($case->{backup}), $param);
+
+ if (my $expected_err = $case->{error}) {
+ is($err, $expected_err, 'restore refused');
+ is(
+ $QemuServerTest::FakeCFS::configs->{100},
+ $case->{target},
+ 'config unchanged',
+ );
+ is_deeply($calls, {}, 'replication state and firewall config untouched');
+ unlike($output, $NOT_CLEANED_UP, 'no warning about a changed VM');
+ return;
+ }
+
+ is($err, '', 'restore succeeded') or diag($output);
+ like($output, $case->{output}, 'task log reports the identity');
+
+ my $expected = { $case->{result}->%* };
+ $expected->{uid} = QemuServerTest::FakeCFS::generated_uid(1)
+ if $expected->{uid} eq 'generated';
+ is_deeply(
+ QemuServerTest::FakeCFS::stored_identity(100),
+ $expected,
+ 'stored identity',
+ );
+ is_deeply(
+ meta_of(100),
+ $case->{meta}
+ // { 'creation-qemu' => '9.2.0', ctime => 1600000000, $expected->%* },
+ 'other members of meta come from the backup',
+ );
+ };
+ }
+}
+
+subtest 'identity check on raw backup configs' => sub {
+ my $opts = { reservation => { previous => { uid => $UID_TARGET, rtime => 1 } } };
+
+ my $raw = "memory: 512\nmeta: uid=$UID_TARGET\nmeta: rtime=1,uid=$UID_BACKUP\n";
+ is_deeply(
+ PVE::QemuServer::restore_check_identity(100, $raw, { 'allow-identity-mismatch' => 1 }),
+ { uid => $UID_BACKUP, rtime => 1 },
+ 'last top-level meta property counts',
+ );
+ eval { PVE::QemuServer::restore_check_identity(100, $raw, $opts) };
+ is($@, $REFUSED, 'refused');
+
+ $raw = "memory: 512\nmeta: rtime=1,uid=$UID_TARGET\n\n[snap]\nmeta: uid=$UID_BACKUP\n";
+ is_deeply(
+ PVE::QemuServer::restore_check_identity(100, $raw, $opts),
+ { uid => $UID_TARGET, rtime => 1 },
+ 'snapshot sections are ignored',
+ );
+
+ is(
+ PVE::QemuServer::restore_check_identity(100, "memory: 512\n", $opts),
+ undef,
+ 'backup without meta has no identity',
+ );
+
+ QemuServerTest::FakeCFS::reset();
+ QemuServerTest::FakeCFS::set_config(100, "lock: create\nmemory: 128\n");
+ $opts = { reservation => { previous => undef } };
+ is_deeply(
+ PVE::QemuServer::restore_check_identity(100, "meta: uid=$UID_BACKUP\n", $opts),
+ { uid => $UID_BACKUP, rtime => undef },
+ 'no check without identity of the target',
+ );
+ is($opts->{'backup-uid'}, $UID_BACKUP, 'uid of the backup is remembered');
+ is(pending_uid_of(100), $UID_BACKUP, 'and recorded as pending uid');
+};
+
+subtest 'retry of a failed restore to a free VMID keeps the lineage' => sub {
+ QemuServerTest::FakeCFS::reset();
+ my $backup = backup_config("$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP");
+
+ $fail_at = 'data';
+ my ($err) = restore_api('pbs', $backup, {});
+ is($err, "restoring the data failed\n", 'first attempt failed');
+
+ # the web interface sets 'force' for a retry
+ $fail_at = '';
+ ($err) = restore_api('pbs', $backup, { force => 1 });
+ is($err, '', 'retry succeeds');
+ is(QemuServerTest::FakeCFS::stored_identity(100)->{uid}, $UID_BACKUP, 'uid adopted');
+};
+
+subtest 'live restore adopts the uid of the backup only when complete' => sub {
+ my $backup = backup_config("$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP");
+ my $template = "$backup" . "template: 1\n";
+
+ for my $failure ('before the live phase', 'during the live phase') {
+ QemuServerTest::FakeCFS::reset();
+
+ $fail_at = $failure eq 'during the live phase' ? 'live' : '';
+ my ($err, $output) = restore_api(
+ 'pbs',
+ $failure eq 'before the live phase' ? $template : $backup,
+ { 'live-restore' => 1 },
+ );
+ ok($err, "restore fails $failure");
+ isnt(
+ QemuServerTest::FakeCFS::stored_identity(100)->{uid},
+ $UID_BACKUP,
+ "leftover VM does not have the uid of the backup ($failure)",
+ );
+ unlike($output, qr/^guest identity:/m, "no identity reported ($failure)");
+
+ $fail_at = '';
+ ($err, $output) = restore_api('pbs', $backup, { vmid => 101, 'live-restore' => 1 });
+ is($err, '', "retry at another VMID succeeds ($failure)") or diag($output);
+ is($calls->{live_restore}, 1, 'live phase ran');
+ is(
+ QemuServerTest::FakeCFS::stored_identity(101)->{uid},
+ $UID_BACKUP,
+ "retry adopts the uid of the backup ($failure)",
+ );
+ like(
+ $output,
+ qr/^guest identity: adopted uid $UID_BACKUP from the backup$/m,
+ "adoption reported ($failure)",
+ );
+ }
+};
+
+# A VM that a restore leaves behind keeps the uid of the backup as pending uid, so that retrying
+# the restore over it takes the uid over.
+
+my $BACKUP_WITH_UID = backup_config("$BACKUP_META,rtime=1600000000000,uid=$UID_BACKUP");
+my $ADOPTED = qr/^guest identity: adopted uid $UID_BACKUP from the backup$/m;
+
+sub pending_uid_of {
+ my ($vmid) = @_;
+
+ return PVE::GuestIdentity::get_pending_uid(PVE::QemuConfig->load_config($vmid));
+}
+
+sub stored_uid_of {
+ my ($vmid) = @_;
+
+ my $identity = QemuServerTest::FakeCFS::stored_identity($vmid);
+ return $identity ? $identity->{uid} : undef;
+}
+
+# Fails the live restore of a backup with uid to the free VMID 100 in its live phase.
+sub failed_live_restore {
+ QemuServerTest::FakeCFS::reset();
+ $QemuServerTest::FakeCFS::now_ms = $NOW;
+
+ local $fail_at = 'live';
+ my ($err, $output) = restore_api('pbs', $BACKUP_WITH_UID, { 'live-restore' => 1 });
+ is($err, "live restore failed\n", 'live restore failed in its live phase');
+ unlike($output, qr/dropping pending uid/, 'starting the VM keeps the pending uid');
+ isnt(stored_uid_of(100), $UID_BACKUP, 'leftover VM has a uid of its own');
+ is(pending_uid_of(100), $UID_BACKUP, 'leftover VM has the uid of the backup as pending uid');
+ unlike($QemuServerTest::FakeCFS::configs->{100}, qr/^lock:/m, 'leftover VM is not locked');
+}
+
+subtest 'retry in place after a failed live restore' => sub {
+ for my $retry ({ 'live-restore' => 1 }, {}) {
+ my $what = $retry->{'live-restore'} ? 'live' : 'regular';
+
+ failed_live_restore();
+
+ # the web interface restores from the backup list of the VM with 'force'
+ my ($err, $output) = restore_api('pbs', $BACKUP_WITH_UID, { force => 1, $retry->%* });
+ is($err, '', "$what retry succeeds") or diag($output);
+ like($output, $ADOPTED, "$what retry adopts the uid of the backup");
+ is(stored_uid_of(100), $UID_BACKUP, "$what retry: stored");
+ is(pending_uid_of(100), undef, "$what retry: pending uid is removed");
+ }
+};
+
+subtest 'retry in place after a failure once the restored volumes stay' => sub {
+ QemuServerTest::FakeCFS::reset();
+ my $backup = $BACKUP_WITH_UID . "hostpci0: 0000:01:00.0\n";
+
+ # the permission check needs the restored config, so it runs once the data is restored
+ $rpcenv->set_user('user@pve');
+ my ($err, $output) = restore_api('vma file', $backup, {});
+ $rpcenv->set_user('root@pam');
+ is(
+ $err,
+ "only root can set 'hostpci0' config for non-mapped devices\n",
+ 'restore refused after restoring the data',
+ );
+ ok(defined($QemuServerTest::FakeCFS::configs->{100}), 'VM is left behind');
+ is(pending_uid_of(100), $UID_BACKUP,
+ 'leftover VM has the uid of the backup as pending uid');
+
+ ($err, $output) = restore_api('vma file', $backup, { force => 1 });
+ is($err, '', 'retry succeeds') or diag($output);
+ is(stored_uid_of(100), $UID_BACKUP, 'retry adopts the uid of the backup');
+ is(pending_uid_of(100), undef, 'retry: pending uid is removed');
+};
+
+my $FAILURES = {
+ # right after the check, before the restore writes a config itself
+ firewall => "unable to restore the firewall config\n",
+ data => "restoring the data failed\n",
+};
+
+for my $type ('pbs', 'provider', 'vma file', 'vma stdin') {
+ for my $failure (sort keys $FAILURES->%*) {
+ my $what = "$type: retry in place after a failed restore over a VM without uid";
+
+ subtest "$what ($failure)" => sub {
+ QemuServerTest::FakeCFS::reset();
+ QemuServerTest::FakeCFS::set_config(100, $TARGET_CONF_NO_UID);
+
+ local $fail_at = $failure;
+ my ($err) = restore_api($type, $BACKUP_WITH_UID, { force => 1 });
+ is($err, $FAILURES->{$failure}, 'restore failed');
+ ok(stored_uid_of(100), 'VM got a uid of its own');
+ isnt(stored_uid_of(100), $UID_BACKUP, 'not the one of the backup');
+ is(pending_uid_of(100), $UID_BACKUP, 'VM has the uid of the backup as pending uid');
+
+ $fail_at = '';
+ ($err, my $output) = restore_api($type, $BACKUP_WITH_UID, { force => 1 });
+ is($err, '', 'retry succeeds') or diag($output);
+ is(stored_uid_of(100), $UID_BACKUP, 'retry adopts the uid of the backup');
+ is(pending_uid_of(100), undef, 'retry: pending uid is removed');
+ };
+ }
+}
+
+subtest 'retry elsewhere, then over the VM left behind' => sub {
+ failed_live_restore();
+ my $leftover = stored_uid_of(100);
+
+ my ($err, $output) = restore_api('pbs', $BACKUP_WITH_UID, { vmid => 101 });
+ is($err, '', 'restore to another VMID succeeds') or diag($output);
+ is(stored_uid_of(101), $UID_BACKUP, 'pending uid of the leftover VM is not in use');
+ is(pending_uid_of(100), $UID_BACKUP, 'leftover VM keeps its pending uid');
+
+ ($err, $output) = restore_api('pbs', $BACKUP_WITH_UID, { force => 1 });
+ is($err, '', 'restore over the leftover VM is not refused') or diag($output);
+ like(
+ $output,
+ qr/^guest identity: backup uid $UID_BACKUP is in use by guest 101, keeping uid $leftover$/m,
+ 'uid of the backup is in use by now',
+ );
+ is(stored_uid_of(100), $leftover, 'leftover VM keeps its uid');
+ is(pending_uid_of(100), undef, 'pending uid is removed');
+};
+
+subtest 'using the VM left behind drops its pending uid' => sub {
+ failed_live_restore();
+ my $leftover = stored_uid_of(100);
+
+ # like a start, or a config change of the stopped VM
+ my $warnings = '';
+ {
+ local $SIG{__WARN__} = sub { $warnings .= $_[0]; };
+ my $conf = PVE::QemuConfig->load_config(100);
+ PVE::QemuServer::vmconfig_apply_pending(100, $conf, {});
+ }
+ my $dropped = "guest identity: dropping pending uid $UID_BACKUP of an unfinished restore"
+ . " - a retry now needs 'allow-identity-mismatch'";
+ like($warnings, qr/^\Q$dropped\E$/m, 'dropped');
+ is(pending_uid_of(100), undef, 'pending uid is gone');
+
+ my ($err) = restore_api('pbs', $BACKUP_WITH_UID, { force => 1 });
+ like(
+ $err,
+ qr/^backup belongs to another guest: its uid .$UID_BACKUP. differs/,
+ 'retry is refused',
+ );
+ is(stored_uid_of(100), $leftover, 'VM stays its own guest');
+};
+
+subtest 'a restore with unique keeps a pending uid, but never takes it over' => sub {
+ for my $retry ({ unique => 1 }, {}) {
+ my $what = $retry->{unique} ? 'retry with unique' : 'retry without unique';
+ QemuServerTest::FakeCFS::reset();
+
+ # registering the addresses of the new MACs is the last step before the adoption
+ local $fail_at = 'after-restore';
+ my ($err) = restore_api('pbs', $BACKUP_WITH_UID, { unique => 1 });
+ is($err, "unable to register IPs\n", 'restore with unique failed at its end');
+ my $leftover = stored_uid_of(100);
+ is(pending_uid_of(100), $UID_BACKUP, 'VM left behind has the pending uid');
+
+ $fail_at = '';
+ my $output;
+ ($err, $output) = restore_api('pbs', $BACKUP_WITH_UID, { force => 1, $retry->%* });
+ is($err, '', "$what succeeds") or diag($output);
+ is(pending_uid_of(100), undef, "$what: pending uid is removed");
+ if ($retry->{unique}) {
+ is(stored_uid_of(100), $leftover, "$what: uid of the backup is not taken over");
+ } else {
+ is(stored_uid_of(100), $UID_BACKUP, "$what: uid of the backup is taken over");
+ }
+ }
+};
+
+subtest 'a failed adoption keeps the pending uid' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ $QemuServerTest::FakeCFS::lock_domain_error =
+ "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+ my ($err, $output) = restore_api('pbs', $BACKUP_WITH_UID, {});
+ $QemuServerTest::FakeCFS::lock_domain_error = undef;
+ is($err, '', 'restore succeeds') or diag($output);
+ my ($kept) = $output =~ $ADOPT_FAILED;
+ ok(defined($kept), 'failed adoption is a warning') or diag($output);
+ isnt(stored_uid_of(100), $UID_BACKUP, 'VM has a uid of its own');
+ is($kept, stored_uid_of(100), 'warning names the uid that the VM keeps');
+ is(pending_uid_of(100), $UID_BACKUP, 'nothing decided, pending uid stays');
+
+ ($err, $output) = restore_api('pbs', $BACKUP_WITH_UID, { force => 1 });
+ is($err, '', 'restore of the backup again is not refused') or diag($output);
+ is(stored_uid_of(100), $UID_BACKUP, 'uid of the backup is taken over');
+};
+
+subtest 'config of a VM left behind shows its stored identity' => sub {
+ failed_live_restore();
+ my $meta = PVE::QemuConfig->load_config(100)->{meta};
+
+ my $conf = PVE::API2::Qemu->vm_config({ node => 'localhost', vmid => 100 });
+ is($conf->{meta}, $meta, 'config with pending changes applied: stored identity');
+ $conf = PVE::API2::Qemu->vm_config({ node => 'localhost', vmid => 100, current => 1 });
+ is($conf->{meta}, $meta, 'current config: stored identity');
+
+ my $pending = PVE::API2::Qemu->vm_pending({ node => 'localhost', vmid => 100 });
+ my ($item) = grep { $_->{key} eq 'meta' } $pending->@*;
+ is($item->{value}, $meta, 'pending view: current value');
+ is($item->{pending}, "uid=$UID_BACKUP", 'pending view: pending uid');
+};
+
+subtest 'removing the pending uid keeps an identity that the parser cannot read' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ # a member of a newer version makes the parser drop the whole property
+ my $target_meta = "creation-qemu=10.0.0,ctime=1700000000,future-member=1,"
+ . "rtime=1700000000000,uid=$UID_TARGET";
+ QemuServerTest::FakeCFS::set_config(
+ 100,
+ "memory: 512\nmeta: $target_meta\n\n[PENDING]\nmeta: uid=$UID_BACKUP\n",
+ );
+ my $reservation = {
+ existed => 1,
+ previous => { uid => $UID_TARGET, rtime => 1700000000000 },
+ 'previous-pending-uid' => $UID_BACKUP,
+ assigned => { uid => $UID_TARGET, rtime => 1700000000000 },
+ };
+ my $res;
+ {
+ local $SIG{__WARN__} = sub { };
+ $res =
+ PVE::GuestIdentity::adopt('PVE::QemuConfig', 100, 'cccccccccccccccccccc', $reservation);
+ }
+ is($res->{result}, 'kept', 'identity of another backup is not taken over');
+ is(stored_uid_of(100), $UID_TARGET, 'stored identity is kept');
+ is(pending_uid_of(100), undef, 'pending uid is removed');
+};
+
+# vzdump never archives pending changes, a crafted backup must not bring any along
+for my $type (sort keys $ARCHIVES->%*) {
+ subtest "$type: pending changes of the backup are dropped" => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ my $crafted = $BACKUP_WITH_UID
+ . "\n[PENDING]\nargs: -device foo\nhostpci0: 0000:01:00.0\nmeta: uid=$UID_TARGET\n";
+ my ($err, $output) = restore_api($type, $crafted, {});
+ is($err, '', 'restore succeeds') or diag($output);
+ my @writes = $QemuServerTest::FakeCFS::written->{100}->@*;
+ ok(!grep({ m/^(args|hostpci0):|uid=$UID_TARGET/m } @writes), 'no write has them');
+ unlike(
+ $QemuServerTest::FakeCFS::configs->{100},
+ qr/^\[PENDING\]/m,
+ 'no pending section',
+ );
+ };
+}
+
+# Once a write of the config of VMID 100 has the pending uid of the backup, all later writes of the
+# restore have it, so that the VM keeps it whenever the restore stops. Only the adoption removes it.
+for my $type (@IDENTITY_ARCHIVES, 'pbs live') {
+ for my $target (undef, $TARGET_CONF_NO_UID) {
+ my $what = $type . ($target ? ', in place over a VM without uid' : ', free VMID');
+
+ subtest "$what: every write of the restore keeps the pending uid" => sub {
+ QemuServerTest::FakeCFS::reset();
+ QemuServerTest::FakeCFS::set_config(100, $target) if $target;
+
+ my $param = { $target ? (force => 1) : () };
+ $param->{'live-restore'} = 1 if $type eq 'pbs live';
+ my ($err, $output) = restore_api($type =~ s/ live$//r, $BACKUP_WITH_UID, $param);
+ is($err, '', 'restore succeeds') or diag($output);
+ like($output, $ADOPTED, 'uid of the backup is adopted');
+
+ my @writes = $QemuServerTest::FakeCFS::written->{100}->@*;
+ my $last = pop(@writes);
+ my $first = 0;
+ $first++ while $first < @writes && $writes[$first] !~ m/^\[PENDING\]/m;
+ ok($first < @writes, 'pending uid is written') or return;
+ is(
+ scalar(
+ grep { m/^\[PENDING\]\nmeta: uid=$UID_BACKUP$/m }
+ @writes[$first .. $#writes]
+ ),
+ scalar(@writes) - $first,
+ 'all later writes keep it',
+ );
+ unlike($last, qr/^\[PENDING\]/m, 'adoption removes it');
+ };
+ }
+}
+
+$fail_at = '';
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH qemu-server v3 18/38] api: remote migration: take over the guest uid from the source
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (16 preceding siblings ...)
2026-11-18 4:37 ` [PATCH qemu-server v3 17/38] restore: take over the guest uid from the backup where possible Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 19/38] api: status: return the guest uid Thomas Lamprecht
` (21 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A VM migrated to another cluster should keep its uid. Send the uid to
the target right after the tunnel version check, before anything is
allocated or transferred there. The target takes it over through the
cluster-wide check of pve-guest-common, for the config that reserves the
VMID, and the config writes of the migration keep it. The target accepts
the uid only once and only before the config arrives. The other members
of 'meta', including the creation information, come from the source as
before. The command schema, the target handler and the sender are shared
with containers through pve-guest-common.
If another guest in the target cluster already has the uid, refuse the
migration. A migration moves a guest, so a new uid without notice would
leave the identity of the VM with the other guest. The new option
'allow-new-uid' lets the migration go on with a new uid.
A failure to take over the uid, for example because the cluster-wide
lock cannot be taken, stops the migration at that point as well. The
source then asks the target to clean up, as on a tunnel version
mismatch.
Report the result in the task logs of both sides. If a newer target
reports a result that the source does not know, the source only logs a
warning. A source with the previous tunnel version sends no uid, so the
VM keeps the uid it got when the target reserved its VMID. A VM without
uid gets one before the source sends it.
Bump the tunnel version, so that older targets are refused before disks
are allocated or transferred, and not only when their strict config
parser rejects the identity. Keep accepting the previous tunnel version
from older sources.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* use the shared helpers of pve-guest-common for the identity command
* a VM without uid gets one before the source sends it, and a stored uid
is sent even if the parser dropped 'meta'; before, source and target
got unrelated uids
* tests for targets with an older and a newer tunnel version, and for
the version the target reports
changes v1 -> v2:
* take the uid over at the start of the migration, through a new tunnel
command, instead of with the config
* refuse the migration if another guest in the target cluster has the
uid, unless the new option 'allow-new-uid' is set (Dominik)
src/PVE/API2/Qemu.pm | 19 +-
src/PVE/CLI/qm.pm | 7 +
src/PVE/QemuMigrate.pm | 18 +-
src/test/run_guest_identity_tests.pl | 371 +++++++++++++++++++++++++++
4 files changed, 407 insertions(+), 8 deletions(-)
diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index c10cc945..901c3f71 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -5897,6 +5897,13 @@ __PACKAGE__->register_method({
minimum => '0',
default => 'migrate limit from datacenter or storage config',
},
+ 'allow-new-uid' => {
+ type => 'boolean',
+ description => "Allow the migration if another guest in the target cluster"
+ . " already has the uid of this guest. The migrated guest then gets a new uid.",
+ optional => 1,
+ default => 0,
+ },
},
},
returns => {
@@ -6913,7 +6920,7 @@ __PACKAGE__->register_method({
PVE::Cluster::check_cfs_quorum();
my $lock = 'create';
- eval { PVE::QemuConfig->create_and_lock_config($vmid, 0, $lock); };
+ my $reservation = eval { PVE::QemuConfig->create_and_lock_config($vmid, 0, $lock); };
raise_param_exc({ vmid => "unable to create empty VM config - $@" })
if $@;
@@ -6943,6 +6950,7 @@ __PACKAGE__->register_method({
};
my $cmd_desc = {
+ identity => PVE::GuestIdentity::tunnel_identity_cmd_schema(),
config => {
conf => {
type => 'string',
@@ -7067,9 +7075,16 @@ __PACKAGE__->register_method({
# bump/bump for opt-in changes
return {
api => $PVE::QemuMigrate::WS_TUNNEL_VERSION,
- age => 0,
+ age => 1,
};
},
+ 'identity' => sub {
+ my ($params) = @_;
+
+ return PVE::GuestIdentity::handle_tunnel_identity(
+ 'PVE::QemuConfig', $state, $reservation, $params,
+ );
+ },
'config' => sub {
my ($params) = @_;
diff --git a/src/PVE/CLI/qm.pm b/src/PVE/CLI/qm.pm
index c8b352d6..355063a9 100755
--- a/src/PVE/CLI/qm.pm
+++ b/src/PVE/CLI/qm.pm
@@ -225,6 +225,13 @@ __PACKAGE__->register_method({
minimum => '0',
default => 'migrate limit from datacenter or storage config',
},
+ 'allow-new-uid' => {
+ type => 'boolean',
+ description => "Allow the migration if another guest in the target cluster"
+ . " already has the uid of this guest. The migrated guest then gets a new uid.",
+ optional => 1,
+ default => 0,
+ },
},
},
returns => {
diff --git a/src/PVE/QemuMigrate.pm b/src/PVE/QemuMigrate.pm
index 4477966f..e0edfd79 100644
--- a/src/PVE/QemuMigrate.pm
+++ b/src/PVE/QemuMigrate.pm
@@ -47,7 +47,8 @@ use PVE::AbstractMigrate;
use base qw(PVE::AbstractMigrate);
# compared against remote end's minimum version
-our $WS_TUNNEL_VERSION = 2;
+# 3: the 'identity' command, and 'meta' with the guest identity in the config
+our $WS_TUNNEL_VERSION = 3;
sub fork_tunnel {
my ($self, $ssh_forward_info) = @_;
@@ -350,17 +351,22 @@ sub prepare {
$self->log('info', "local WS tunnel version: $WS_TUNNEL_VERSION");
$self->log('info', "remote WS tunnel version: $tunnel->{version}");
$self->log('info', "minimum required WS tunnel version: $min_version");
- my $version_err;
- $version_err = "Remote tunnel endpoint not compatible, upgrade required\n"
+ my $err;
+ $err = "Remote tunnel endpoint not compatible, upgrade required\n"
if $WS_TUNNEL_VERSION < $min_version;
- $version_err = "Remote tunnel endpoint too old, upgrade required\n"
+ $err = "Remote tunnel endpoint too old, upgrade required\n"
if $WS_TUNNEL_VERSION > $tunnel->{version};
- if ($version_err) {
+ if (!$err) {
+ eval { $self->take_over_identity_on_target('PVE::QemuConfig', $tunnel) };
+ $err = $@;
+ }
+
+ if ($err) {
# the target already reserved the VMID with a locked config
eval { PVE::Tunnel::finish_tunnel($tunnel, 1); };
$self->log('warn', "cleanup of the target failed - $@") if $@;
- die $version_err;
+ die $err;
}
print "websocket tunnel started\n";
diff --git a/src/test/run_guest_identity_tests.pl b/src/test/run_guest_identity_tests.pl
index 1e60cbf7..1379220a 100755
--- a/src/test/run_guest_identity_tests.pl
+++ b/src/test/run_guest_identity_tests.pl
@@ -8,6 +8,9 @@ use v5.36;
use lib qw(.. .);
use File::Temp qw(tempdir);
+use IO::Handle;
+use JSON;
+use Socket;
use Test::MockModule;
use Test::More;
@@ -16,6 +19,7 @@ use PVE::RPCEnvironment;
use PVE::QemuConfig;
use PVE::QemuConfig::NoWrite;
+use PVE::QemuMigrate;
use PVE::QemuServer;
use PVE::QemuServer::MetaInfo;
@@ -333,4 +337,371 @@ subtest 'importovf' => sub {
);
};
+# Runs the migration tunnel endpoint for $vmid with the commands @$commands from the source side
+# and returns the decoded replies, the task output and the error of the task.
+sub run_mtunnel($vmid, $commands) {
+ socketpair(my $client, my $server, AF_UNIX, SOCK_STREAM, PF_UNSPEC)
+ or die "socketpair failed - $!\n";
+ $client->autoflush(1);
+
+ print $client encode_json($_) . "\n" for $commands->@*;
+ shutdown($client, 1);
+
+ my $rpcenv_module = Test::MockModule->new('PVE::RPCEnvironment');
+ $rpcenv_module->mock(
+ fork_worker => sub {
+ my ($self, $type, $id, $user, $code) = @_;
+ $code->('UPID:test');
+ return 'UPID:test';
+ },
+ );
+ my $access_control_module = Test::MockModule->new('PVE::AccessControl');
+ $access_control_module->mock(assemble_tunnel_ticket => sub { return 'ticket'; });
+ my $socket_module = Test::MockModule->new('IO::Socket::UNIX');
+ $socket_module->mock(new => sub { return bless { conn => $server }, 'TestListenSocket'; });
+
+ my ($output, $err) = ('', '');
+ {
+ open(my $out_fh, '>', \$output);
+ my $oldfh = select($out_fh);
+ eval { PVE::API2::Qemu->mtunnel({ node => 'localhost', vmid => $vmid }) };
+ $err = $@;
+ select($oldfh);
+ close($out_fh);
+ }
+ close($server);
+
+ my $replies = [];
+ while (defined(my $line = <$client>)) {
+ push $replies->@*, decode_json($line);
+ }
+
+ return ($replies, $output, $err);
+}
+
+package TestListenSocket {
+
+ sub accept($self) {
+ return $self->{conn};
+ }
+}
+
+subtest 'remote migration, target side' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ my $cluster_module = Test::MockModule->new('PVE::Cluster');
+ $cluster_module->mock(check_cfs_quorum => sub { return 1; });
+ my $storage_module = Test::MockModule->new('PVE::Storage');
+ $storage_module->mock(config => sub { return {}; });
+ my $replication_config_module = Test::MockModule->new('PVE::ReplicationConfig');
+ $replication_config_module->mock(
+ new => sub { return bless {}, 'PVE::ReplicationConfig'; },
+ check_for_existing_jobs => sub { return; },
+ );
+ my $qemu_server_module = Test::MockModule->new('PVE::QemuServer');
+ $qemu_server_module->mock(check_running => sub { return 0; });
+
+ my $source_conf = "cores: 2\nmemory: 1024\nmeta: creation-qemu=9.2.0,ctime=1600000000,"
+ . "rtime=1600000000000,uid=$UID_A\nname: source\n";
+ # like the source sends it
+ my $identity_cmd = { cmd => 'identity', uid => $UID_A, 'allow-new-uid' => 0 };
+ my $config_cmd = { cmd => 'config', conf => $source_conf };
+ my $quit_cmd = { cmd => 'quit', cleanup => 0 };
+ my $cleanup_cmd = { cmd => 'quit', cleanup => 1 };
+
+ $QemuServerTest::FakeCFS::now_ms = 1800000000000;
+ my ($replies, $output, $err) =
+ run_mtunnel(100, [$identity_cmd, $config_cmd, $config_cmd, $identity_cmd, $quit_cmd]);
+ is($err, '', 'tunnel exits normally');
+ like(
+ $replies->[3]->{msg},
+ qr/the uid can only be taken over once, before the config arrives/,
+ 'identity command after the config fails',
+ );
+
+ ok($replies->[0]->{success}, 'identity command succeeds');
+ is_deeply(
+ $replies->[0]->{identity},
+ { result => 'adopted', uid => $UID_A, requested => $UID_A },
+ 'identity command returns the adopted identity',
+ );
+ like(
+ $output,
+ qr/^guest identity: adopted uid $UID_A from the source guest$/m,
+ 'target task logs it',
+ );
+ my $writes = $QemuServerTest::FakeCFS::written->{100};
+ ok(
+ $writes->[1] =~ m/^lock: create$/m && $writes->[1] =~ m/^meta: .*uid=$UID_A$/m,
+ 'uid is taken over right after the reservation',
+ );
+ ok($replies->[1]->{success}, 'config command succeeds');
+ ok(!exists($replies->[1]->{identity}), 'config command takes over nothing');
+ ok(!$replies->[2]->{success}, 'repeated config command fails');
+ is_deeply(
+ load_meta(100),
+ {
+ 'creation-qemu' => '9.2.0',
+ ctime => 1600000000,
+ rtime => 1800000000000,
+ uid => $UID_A,
+ },
+ 'config write keeps the uid, other members come from the source',
+ );
+ ok($replies->[4]->{success}, 'tunnel quits');
+
+ QemuServerTest::FakeCFS::reset();
+ $QemuServerTest::FakeCFS::other_guests->{300} = "rtime=1,uid=$UID_A";
+ ($replies, $output, $err) = run_mtunnel(101, [$identity_cmd, $cleanup_cmd]);
+ is($err, '', 'uid in use: tunnel exits normally');
+ ok(!$replies->[0]->{success}, 'uid in use: identity command fails');
+ is(
+ $replies->[0]->{msg},
+ "failed to handle 'identity' command - guest 300 in the target cluster already has the"
+ . " uid '$UID_A' of this guest - use 'allow-new-uid' to migrate it with a new uid\n",
+ 'uid in use: error explains the refusal',
+ );
+ is(
+ scalar($QemuServerTest::FakeCFS::written->{101}->@*),
+ 1,
+ 'uid in use: nothing written besides the reservation',
+ );
+ ok($replies->[1]->{success}, 'uid in use: cleanup on quit succeeds');
+ ok(!defined(configs()->{101}), 'uid in use: reservation removed by the cleanup');
+
+ QemuServerTest::FakeCFS::reset();
+ $QemuServerTest::FakeCFS::other_guests->{300} = "rtime=1,uid=$UID_A";
+ ($replies, $output, $err) = run_mtunnel(
+ 101, [{ $identity_cmd->%*, 'allow-new-uid' => 1 }, $config_cmd, $quit_cmd],
+ );
+ is($err, '', 'uid in use, allow-new-uid: tunnel exits normally');
+ is_deeply(
+ $replies->[0]->{identity},
+ {
+ result => 'in-use',
+ uid => generated_uid(1),
+ requested => $UID_A,
+ 'in-use-by' => 300,
+ },
+ 'uid in use, allow-new-uid: result is returned',
+ );
+ my $kept = generated_uid(1);
+ like(
+ $output,
+ qr/^guest identity: source guest uid $UID_A is in use by guest 300, keeping uid $kept$/m,
+ 'uid in use, allow-new-uid: target task logs it',
+ );
+ ok($replies->[1]->{success}, 'uid in use, allow-new-uid: config command succeeds');
+ is(stored_identity(101)->{uid}, $kept, 'uid in use, allow-new-uid: new uid stays');
+
+ QemuServerTest::FakeCFS::reset();
+ ($replies) = run_mtunnel(105, [{ cmd => 'version' }, $cleanup_cmd]);
+ is($replies->[0]->{api}, $PVE::QemuMigrate::WS_TUNNEL_VERSION, 'version: tunnel version');
+ is($replies->[0]->{api} - $replies->[0]->{age}, 2, 'version: version 2 accepted');
+
+ # like a source with tunnel version 2
+ QemuServerTest::FakeCFS::reset();
+ ($replies) = run_mtunnel(102, [$config_cmd, $quit_cmd]);
+ ok($replies->[0]->{success}, 'source without identity command: config command succeeds');
+ is(
+ stored_identity(102)->{uid},
+ generated_uid(1),
+ 'source without identity command: new uid, not the one of the config',
+ );
+
+ QemuServerTest::FakeCFS::reset();
+ my $config_without_uid =
+ { cmd => 'config', conf => $source_conf =~ s/,rtime=\d+,uid=\w+//r };
+ ($replies) = run_mtunnel(102, [$config_without_uid, $quit_cmd]);
+ ok($replies->[0]->{success}, 'source without uid: config command succeeds');
+ is(stored_identity(102)->{uid}, generated_uid(1), 'source without uid: new uid');
+
+ QemuServerTest::FakeCFS::reset();
+ $QemuServerTest::FakeCFS::lock_domain_error =
+ "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+ ($replies, $output, $err) = run_mtunnel(103, [$identity_cmd, $cleanup_cmd]);
+ is($err, '', 'adopting fails: tunnel exits normally');
+ ok(!$replies->[0]->{success}, 'adopting fails: identity command fails');
+ is(
+ $replies->[0]->{msg},
+ "failed to handle 'identity' command - unable to adopt the uid of the source guest"
+ . " - cfs-lock 'domain-guest-identity' error: no quorum!\n",
+ 'adopting fails: error explains the failure',
+ );
+ ok($replies->[1]->{success}, 'adopting fails: cleanup on quit succeeds');
+ ok(!defined(configs()->{103}), 'adopting fails: reservation removed by the cleanup');
+};
+
+subtest 'remote migration, source side' => sub {
+ QemuServerTest::FakeCFS::reset();
+
+ # the source reads the firewall config of the VM from /etc/pve, use a VMID the host lacks
+ my $vmid = 999999999;
+ my $source_conf = "cores: 2\nmemory: 1024\nmeta: rtime=1600000000000,uid=$UID_A\n";
+ configs()->{$vmid} = $source_conf;
+
+ my ($commands, $reply, $cleanup, $log) = ([], undef, undef, '');
+ my $tunnel_module = Test::MockModule->new('PVE::Tunnel');
+ $tunnel_module->mock(
+ write_tunnel => sub {
+ my ($tunnel, $timeout, $command, $params) = @_;
+ push $commands->@*, { command => $command, timeout => $timeout, params => $params };
+ die $reply if !ref($reply);
+ return $reply;
+ },
+ finish_tunnel => sub {
+ (undef, $cleanup) = @_;
+ },
+ );
+ my $migrate_module = Test::MockModule->new('PVE::QemuMigrate');
+ $migrate_module->mock(fork_websocket_tunnel => sub { return { version => 3, age => 1 }; });
+ my $abstract_migrate_module = Test::MockModule->new('PVE::AbstractMigrate');
+ $abstract_migrate_module->mock(log => sub { $log .= "$_[1]: $_[2]\n"; });
+ $QemuServerTest::FakeCFS::qemu_config_module->mock(cleanup_fleecing_images => sub { });
+ my $helpers_module = Test::MockModule->new('PVE::QemuMigrate::Helpers');
+ $helpers_module->mock(check_local_resources => sub { return ([], {}, {}); });
+ my $qemu_server_module = Test::MockModule->new('PVE::QemuServer');
+ $qemu_server_module->mock(check_running => sub { return 0; });
+ my $storage_module = Test::MockModule->new('PVE::Storage');
+ $storage_module->mock(config => sub { return {}; });
+ my $replication_config_module = Test::MockModule->new('PVE::ReplicationConfig');
+ $replication_config_module->mock(
+ new => sub { return bless {}, 'PVE::ReplicationConfig'; },
+ find_local_replication_job => sub { return; },
+ check_for_existing_jobs => sub { return; },
+ );
+
+ my $new_migration = sub($opts = {}) {
+ return bless {
+ opts => { remote => { vmid => 300 }, $opts->%* },
+ node => 'remote-node',
+ vmid => $vmid,
+ local_volumes => {},
+ volume_map => {},
+ storecfg => {},
+ },
+ 'PVE::QemuMigrate';
+ };
+ my $prepare = sub($self) {
+ ($commands, $cleanup, $log) = ([], undef, '');
+ my $output = '';
+ open(my $out_fh, '>', \$output);
+ my $oldfh = select($out_fh);
+ eval { $self->prepare($vmid) };
+ my $err = $@;
+ select($oldfh);
+ close($out_fh);
+ return $err;
+ };
+
+ $reply =
+ { success => 1, identity => { result => 'adopted', uid => $UID_A, requested => $UID_A } };
+ my $self = $new_migration->();
+ is($prepare->($self), '', 'uid free on the target: prepare succeeds');
+ is_deeply(
+ $commands,
+ [{
+ command => 'identity',
+ timeout => 60,
+ params => { uid => $UID_A, 'allow-new-uid' => 0 },
+ }],
+ 'uid free on the target: only the uid is sent before anything else',
+ );
+ like(
+ $log,
+ qr/^info: guest identity: adopted uid $UID_A from the source guest$/m,
+ 'uid free on the target: source task logs the outcome',
+ );
+ ok($self->{tunnel}, 'uid free on the target: tunnel is kept for the migration');
+ ok(!defined($cleanup), 'uid free on the target: no cleanup');
+
+ $reply = {
+ success => 1,
+ identity =>
+ { result => 'in-use', uid => $UID_B, requested => $UID_A, 'in-use-by' => 300 },
+ };
+ $self = $new_migration->({ 'allow-new-uid' => 1 });
+ is($prepare->($self), '', 'allow-new-uid: prepare succeeds');
+ is($commands->[0]->{params}->{'allow-new-uid'}, 1, 'allow-new-uid: option is sent');
+ my $in_use =
+ "guest identity: source guest uid $UID_A is in use by guest 300, keeping uid $UID_B";
+ like($log, qr/^info: \Q$in_use\E$/m, 'allow-new-uid: source task logs the new uid');
+
+ $reply =
+ "error - tunnel command 'identity' failed - failed to handle 'identity' command -"
+ . " guest 300 in the target cluster already has the uid '$UID_A' of this guest - use"
+ . " 'allow-new-uid' to migrate it with a new uid\n";
+ $self = $new_migration->();
+ is($prepare->($self), $reply, 'uid in use: prepare fails with the error of the target');
+ is(scalar($commands->@*), 1, 'uid in use: nothing else is sent');
+ is($cleanup, 1, 'uid in use: target is asked to clean up');
+ ok(!$self->{tunnel}, 'uid in use: tunnel is not kept');
+
+ # a target with a newer version has accepted the uid at this point
+ $reply = { success => 1, identity => { result => 'future', uid => $UID_A } };
+ $self = $new_migration->();
+ is($prepare->($self), '', 'unknown outcome does not fail the migration');
+ like($log, qr/^warn: unknown result of guest identity adoption 'future'$/m, 'it is logged');
+
+ my $without_uid = "cores: 2\nmemory: 1024\n";
+ configs()->{$vmid} = $without_uid;
+ for my $case (
+ [{ version => 2, age => 0 }, "Remote tunnel endpoint too old, upgrade required\n"],
+ [
+ { version => 4, age => 0 },
+ "Remote tunnel endpoint not compatible, upgrade required\n",
+ ],
+ ) {
+ my ($remote, $expected) = $case->@*;
+ my $what = "target with tunnel version $remote->{version}";
+ $migrate_module->mock(fork_websocket_tunnel => sub { return $remote; });
+ $self = $new_migration->();
+ is($prepare->($self), $expected, "$what: refused");
+ is_deeply($commands, [], "$what: nothing is sent");
+ is($cleanup, 1, "$what: target is asked to clean up");
+ is(configs()->{$vmid}, $without_uid, "$what: source config is unchanged");
+ }
+ $migrate_module->mock(fork_websocket_tunnel => sub { return { version => 3, age => 1 }; });
+
+ $self = $new_migration->();
+ is($prepare->($self), '', 'source without uid: prepare succeeds');
+ my $assigned = stored_identity($vmid)->{uid};
+ is($assigned, generated_uid(1), 'source without uid: gets one first');
+ is($commands->[0]->{params}->{uid}, $assigned, 'source without uid: and sends it');
+
+ # a member of a newer version makes the parser drop the whole property
+ configs()->{$vmid} =
+ "cores: 2\nmemory: 1024\nmeta: future=1,rtime=1600000000000,uid=$UID_A\n";
+ $self = $new_migration->();
+ {
+ local $SIG{__WARN__} = sub { };
+ is($prepare->($self), '', 'meta the parser drops: prepare succeeds');
+ }
+ is($commands->[0]->{params}->{uid}, $UID_A, 'meta the parser drops: stored uid is sent');
+
+ configs()->{$vmid} = $source_conf;
+ $reply = { success => 1 };
+ $self = $new_migration->();
+ $self->{tunnel} = {};
+ ($commands, $log) = ([], '');
+ my $output = '';
+ {
+ open(my $out_fh, '>', \$output);
+ my $oldfh = select($out_fh);
+ eval { $self->phase1_remote($vmid) };
+ is($@, '', 'config sent');
+ select($oldfh);
+ close($out_fh);
+ }
+ is($commands->[0]->{command}, 'config', 'config command');
+ like(
+ $commands->[0]->{params}->{conf},
+ qr/^meta: rtime=1600000000000,uid=$UID_A$/m,
+ 'source config sent with its uid',
+ );
+ unlike($log, qr/guest identity/, 'config reply has no identity to log');
+
+ $QemuServerTest::FakeCFS::qemu_config_module->unmock('cleanup_fleecing_images');
+};
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH qemu-server v3 19/38] api: status: return the guest uid
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (17 preceding siblings ...)
2026-11-18 4:37 ` [PATCH qemu-server v3 18/38] api: remote migration: take over the guest uid from the source Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 20/38] tests: remote migration: check a failed command and an early tunnel end Thomas Lamprecht
` (20 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Tools that follow a guest across restores and migrations need its uid.
It is part of the config already, but the status of a single VM is the
more common entry point for them, so return it there as well.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
no changes since v2
changes v1 -> v2:
* rebase only
src/PVE/API2/Qemu.pm | 5 +++++
src/test/run_api_tests.pl | 25 +++++++++++++++++++++++++
2 files changed, 30 insertions(+)
diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index 901c3f71..be1eb407 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -3588,6 +3588,7 @@ __PACKAGE__->register_method({
enum => ['vnc'],
optional => 1,
},
+ uid => PVE::GuestIdentity::meta_format_properties()->{uid},
},
},
code => sub {
@@ -3610,6 +3611,10 @@ __PACKAGE__->register_method({
}
$status->{agent} = 1 if PVE::QemuServer::Agent::get_qga_key($conf, 'enabled');
+ if (my $identity = PVE::QemuConfig->get_identity($conf)) {
+ $status->{uid} = $identity->{uid};
+ }
+
return $status;
},
});
diff --git a/src/test/run_api_tests.pl b/src/test/run_api_tests.pl
index 15c6acfe..6bc8b554 100755
--- a/src/test/run_api_tests.pl
+++ b/src/test/run_api_tests.pl
@@ -326,4 +326,29 @@ subtest 'clone target nodes' => sub {
mock_unreachable_config();
};
+subtest 'status' => sub {
+ my $vm_conf;
+ $qemu_config_module->mock(load_config => sub { return $vm_conf; });
+ $qemu_server_module->mock(
+ vmstatus => sub { return { 100 => { vmid => 100, status => 'stopped' } }; });
+ my $ha_config_module = Test::MockModule->new('PVE::HA::Config');
+ $ha_config_module->mock(get_service_status => sub { return { managed => 0 }; });
+
+ my $returns = PVE::API2::Qemu->map_method_by_name('vm_status')->{returns};
+
+ $vm_conf =
+ { meta => 'creation-qemu=10.1.2,ctime=1,rtime=1700000000000,uid=abcdefghijklmnopqrst' };
+ my $status = PVE::API2::Qemu->vm_status({ node => 'localhost', vmid => 100 });
+ is($status->{uid}, 'abcdefghijklmnopqrst', 'uid returned');
+ eval { PVE::JSONSchema::validate($status, $returns) };
+ is($@, '', 'result matches the schema');
+
+ $vm_conf = { meta => 'creation-qemu=10.1.2,ctime=1' };
+ $status = PVE::API2::Qemu->vm_status({ node => 'localhost', vmid => 100 });
+ ok(!exists($status->{uid}), 'no uid for config without identity');
+
+ mock_unreachable_config();
+ $qemu_server_module->unmock('vmstatus');
+};
+
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH qemu-server v3 20/38] tests: remote migration: check a failed command and an early tunnel end
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (18 preceding siblings ...)
2026-11-18 4:37 ` [PATCH qemu-server v3 19/38] api: status: return the guest uid Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 21/38] config: add read-only meta property with the creation time Thomas Lamprecht
` (19 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
With the uid taken over at the start of a remote migration, a config
left behind on the target would keep the uid of the VM in the target
cluster. Check that the target removes its reservation if the tunnel
closes before the config arrived, keeps a config that arrived, and
logs the error of a failed tunnel command.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* follow the fixed wait for a volume transfer in the cleanup of the
target, which is bounded by time: check a transfer that exits late and
one that cannot be collected
changes v1 -> v2:
* new in v2
src/test/run_guest_identity_tests.pl | 96 ++++++++++++++++++++++++++++
1 file changed, 96 insertions(+)
diff --git a/src/test/run_guest_identity_tests.pl b/src/test/run_guest_identity_tests.pl
index 1379220a..9d97cd9b 100755
--- a/src/test/run_guest_identity_tests.pl
+++ b/src/test/run_guest_identity_tests.pl
@@ -530,6 +530,102 @@ subtest 'remote migration, target side' => sub {
);
ok($replies->[1]->{success}, 'adopting fails: cleanup on quit succeeds');
ok(!defined(configs()->{103}), 'adopting fails: reservation removed by the cleanup');
+
+ my $warnings = '';
+ local $SIG{__WARN__} = sub { $warnings .= $_[0]; };
+
+ ($replies, $output, $err) = run_mtunnel(104, [$identity_cmd, $cleanup_cmd]);
+ like(
+ $warnings,
+ qr/^failed to handle 'identity' command - unable to adopt the uid of the source guest/m,
+ 'failed command: target task logs the error',
+ );
+
+ QemuServerTest::FakeCFS::reset();
+ $warnings = '';
+ ($replies, $output, $err) = run_mtunnel(104, [$identity_cmd]);
+ like($err, qr/mtunnel exited unexpectedly/, 'end before the config: tunnel fails');
+ ok($replies->[0]->{success}, 'end before the config: uid was taken over');
+ like(
+ $warnings,
+ qr/^tunnel closed before the config arrived, cleaning up$/m,
+ 'end before the config: cleanup is logged',
+ );
+ ok(!defined(configs()->{104}),
+ 'end before the config: reservation with the uid is removed');
+
+ QemuServerTest::FakeCFS::reset();
+ $warnings = '';
+ ($replies, $output, $err) = run_mtunnel(104, [$identity_cmd, $config_cmd]);
+ like($err, qr/mtunnel exited unexpectedly/, 'end after the config: tunnel fails');
+ unlike($warnings, qr/cleaning up/, 'end after the config: no cleanup');
+ like(configs()->{104}, qr/^lock: migrate$/m, 'end after the config: config is kept');
+
+ # the volume of a transfer is only known once the transfer is done
+ my $import_cmd = {
+ cmd => 'disk-import',
+ volname => 'vm-104-disk-0',
+ format => 'raw',
+ export_formats => 'raw+size',
+ storage => 'local',
+ };
+ my ($queries, $ends_after, $query_error, $freed) = (0, 2, undef, []);
+ my $storage_tunnel_module = Test::MockModule->new('PVE::StorageTunnel');
+ $storage_tunnel_module->mock(
+ handle_disk_import => sub {
+ my ($state) = @_;
+ $state->{disk_import} = { pid => 1 };
+ return {};
+ },
+ handle_query_disk_import => sub {
+ my ($state) = @_;
+ die $query_error if $query_error;
+ return { status => 'pending' } if ++$queries < $ends_after;
+ delete $state->{disk_import};
+ $state->{cleanup}->{volumes}->{'local:104/vm-104-disk-0.raw'} = 1;
+ return { status => 'complete' };
+ },
+ );
+ $storage_module->mock(
+ storage_check_enabled => sub { return 1; },
+ storage_config => sub { return { content => { images => 1 } }; },
+ vdisk_free => sub { push $freed->@*, $_[1]; },
+ );
+ my $rpcenv_module = Test::MockModule->new('PVE::RPCEnvironment');
+ $rpcenv_module->mock(check => sub { return 1; });
+
+ QemuServerTest::FakeCFS::reset();
+ $warnings = '';
+ ($replies, $output, $err) = run_mtunnel(104, [$import_cmd]);
+ ok($replies->[0]->{success}, 'transfer that ends: it starts') or diag(explain($replies));
+ is_deeply(
+ $freed,
+ ['local:104/vm-104-disk-0.raw'],
+ 'transfer that ends: its volume is freed',
+ );
+ ok(!defined(configs()->{104}), 'transfer that ends: reservation is removed');
+
+ # a transfer that closed its output can be queried many times before it has exited
+ QemuServerTest::FakeCFS::reset();
+ ($queries, $ends_after, $freed, $warnings) = (0, 20, [], '');
+ ($replies, $output, $err) = run_mtunnel(104, [$import_cmd]);
+ is_deeply(
+ $freed,
+ ['local:104/vm-104-disk-0.raw'],
+ 'transfer that exits late: volume is freed',
+ );
+ ok(!defined(configs()->{104}), 'transfer that exits late: reservation is removed');
+
+ QemuServerTest::FakeCFS::reset();
+ ($queries, $query_error, $freed, $warnings) = (0, "unable to query the transfer\n", [], '');
+ ($replies, $output, $err) = run_mtunnel(104, [$import_cmd]);
+ like(
+ $warnings,
+ qr/^cleanup failed - a volume transfer is still running$/m,
+ 'transfer that cannot be collected: it is reported',
+ );
+ is_deeply($freed, [], 'transfer that cannot be collected: nothing is freed');
+ ok(defined(configs()->{104}), 'transfer that cannot be collected: reservation is kept');
};
subtest 'remote migration, source side' => sub {
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH container v3 21/38] config: add read-only meta property with the creation time
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (19 preceding siblings ...)
2026-11-18 4:37 ` [PATCH qemu-server v3 20/38] tests: remote migration: check a failed command and an early tunnel end Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 22/38] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
` (18 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Containers need a place for information that the system records about
them, including the guest identity. Add a read-only 'meta' property
with the same 'ctime' member as for VMs. It is set when a container
gets created; a restore takes it from the backup and a clone from the
source.
'meta' is not accepted as a parameter by the create and update API
calls, and deleting or reverting it is rejected for every user,
including root. A manually queued change gets dropped with a warning
instead of applied. The target of a remote migration takes the property
from the source as it is, bypassing the generic update path, which
rejects read-only properties.
The config API returns 'meta' as a plain string, so that members from
newer versions do not fail the result validation of 'pct config'.
Older pve-container versions drop the property on their next config
write, and their strict parser for remote migrations rejects it after
the volumes were transferred.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* description of 'meta' says what it has
* tests for the independent fixes before the series (failed overwrite
keeps the container, create over a container that appeared after the
request started)
changes v1 -> v2:
* use the shared helper of pve-guest-common to drop a pending change of
'meta'
* test script: use v5.36, mock the registration of used guest IDs
* wording of the commit message
src/PVE/API2/LXC.pm | 4 +
src/PVE/API2/LXC/Config.pm | 29 +-
src/PVE/LXC/Config.pm | 43 ++-
src/test/Makefile | 8 +-
src/test/config_tests.pl | 23 ++
src/test/lifecycle_tests.pl | 613 ++++++++++++++++++++++++++++++++++++
6 files changed, 702 insertions(+), 18 deletions(-)
create mode 100755 src/test/lifecycle_tests.pl
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index be7d64e..753988c 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -427,6 +427,7 @@ __PACKAGE__->register_method({
PVE::LXC::Config->update_pct_config($vmid, $conf, 0, $no_disk_param);
$conf->{unprivileged} = 1 if $unprivileged;
+ $conf->{meta} = PVE::LXC::Config->new_meta_info_string() if !$restore;
my $emsg = $restore ? "unable to restore CT $vmid -" : "unable to create CT $vmid -";
@@ -3317,6 +3318,8 @@ __PACKAGE__->register_method({
my $unprivileged = delete $new_conf->{unprivileged};
my $arch = delete $new_conf->{arch};
+ # read-only for the update below, the source's values are kept as they are
+ my $meta = delete $new_conf->{meta};
# TODO handle properly?
delete $new_conf->{snapshots};
@@ -3351,6 +3354,7 @@ __PACKAGE__->register_method({
$state->{vmid}, $conf, 0, $new_conf, [], [],
);
raise_param_exc($errors) if scalar(keys %$errors);
+ $conf->{meta} = $meta if defined($meta);
PVE::LXC::Config->write_config($state->{vmid}, $conf);
PVE::LXC::update_lxc_config($vmid, $conf);
};
diff --git a/src/PVE/API2/LXC/Config.pm b/src/PVE/API2/LXC/Config.pm
index 805eac0..3d7529f 100644
--- a/src/PVE/API2/LXC/Config.pm
+++ b/src/PVE/API2/LXC/Config.pm
@@ -57,20 +57,23 @@ __PACKAGE__->register_method({
},
returns => {
type => "object",
- properties => PVE::LXC::Config->json_config_properties({
- lxc => {
- description =>
- "Array of lxc low-level configurations ([[key1, value1], [key2, value2] ...]).",
- type => 'array',
- items => { type => 'array', items => { type => 'string' } },
- optional => 1,
- },
- digest => {
- type => 'string',
- description =>
- 'SHA1 digest of configuration file. This can be used to prevent concurrent modifications.',
+ properties => PVE::LXC::Config->json_config_properties(
+ {
+ lxc => {
+ description =>
+ "Array of lxc low-level configurations ([[key1, value1], [key2, value2] ...]).",
+ type => 'array',
+ items => { type => 'array', items => { type => 'string' } },
+ optional => 1,
+ },
+ digest => {
+ type => 'string',
+ description =>
+ 'SHA1 digest of configuration file. This can be used to prevent concurrent modifications.',
+ },
},
- }),
+ 1,
+ ),
},
code => sub {
my ($param) = @_;
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index b6eaf0b..c274e93 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -505,6 +505,15 @@ my $features_desc = {
},
};
+my $meta_info_fmt = {
+ ctime => {
+ type => 'integer',
+ description => "The guest creation timestamp as UNIX epoch time",
+ minimum => 0,
+ optional => 1,
+ },
+};
+
my $confdesc = {
lock => {
optional => 1,
@@ -717,6 +726,13 @@ my $confdesc = {
"Try to be more verbose. For now this only enables debug log-level on start.",
default => 0,
},
+ meta => {
+ optional => 1,
+ type => 'string',
+ format => $meta_info_fmt,
+ description =>
+ "Read-only meta-information about this container, for example its creation time.",
+ },
};
my $valid_lxc_conf_keys = {
@@ -1310,6 +1326,7 @@ sub queue_pct_config_changes {
my $storage_cfg = PVE::Storage::config();
foreach my $opt (@$revert) {
+ die "unable to revert read-only option: '$opt'\n" if $opt eq 'meta';
delete $conf->{pending}->{$opt};
$class->remove_from_pending_delete($conf, $opt); # also remove from deletion queue
}
@@ -1318,6 +1335,7 @@ sub queue_pct_config_changes {
my $modified = {}; # record modified options
foreach my $opt (@$delete) {
+ die "unable to delete read-only option: '$opt'\n" if $opt eq 'meta';
if (!defined($conf->{$opt}) && !defined($conf->{pending}->{$opt})) {
warn "cannot delete '$opt' - not set in current configuration!\n";
next;
@@ -1358,7 +1376,7 @@ sub queue_pct_config_changes {
$value = PVE::LXC::verify_nameserver_list($value);
} elsif ($opt eq 'searchdomain') {
$value = PVE::LXC::verify_searchdomain_list($value);
- } elsif ($opt eq 'unprivileged') {
+ } elsif ($opt eq 'unprivileged' || $opt eq 'meta') {
die "unable to modify read-only option: '$opt'\n";
} elsif ($opt eq 'tags') {
$value = PVE::GuestHelpers::get_unique_tags($value);
@@ -1449,19 +1467,34 @@ sub check_type {
}
}
-# add JSON properties for create and set function
+# add JSON properties for create and set function, with $with_meta also the read-only 'meta' for
+# returning the config
sub json_config_properties {
- my ($class, $prop) = @_;
+ my ($class, $prop, $with_meta) = @_;
foreach my $opt (keys %$confdesc) {
next if $opt eq 'parent' || $opt eq 'snaptime';
+ next if $opt eq 'meta' && !$with_meta;
next if $prop->{$opt};
$prop->{$opt} = $confdesc->{$opt};
}
+ if ($with_meta) {
+ # without its format, as a config written by a newer version can have unknown members
+ $prop->{meta} = { $prop->{meta}->%* };
+ delete $prop->{meta}->{format};
+ }
+
return $prop;
}
+# Returns the 'meta' property string for a newly created container.
+sub new_meta_info_string {
+ my ($class) = @_;
+
+ return PVE::JSONSchema::print_property_string({ ctime => int(time()) }, $meta_info_fmt);
+}
+
my $parse_ct_mountpoint_full = sub {
my ($class, $desc, $data, $noerr) = @_;
@@ -1653,6 +1686,8 @@ my $LXC_FASTPLUG_OPTIONS = {
sub vmconfig_hotplug_pending {
my ($class, $vmid, $conf, $storecfg, $selection, $errors) = @_;
+ $class->drop_pending_meta($conf);
+
my $pid = PVE::LXC::find_lxc_pid($vmid);
my $rootdir = "/proc/$pid/root";
@@ -1779,6 +1814,8 @@ sub vmconfig_hotplug_pending {
sub vmconfig_apply_pending {
my ($class, $vmid, $conf, $storecfg, $selection, $errors) = @_;
+ $class->drop_pending_meta($conf);
+
my $add_apply_error = sub {
my ($opt, $msg) = @_;
my $err_msg = "unable to apply pending change $opt : $msg";
diff --git a/src/test/Makefile b/src/test/Makefile
index c1ff506..456dfb2 100644
--- a/src/test/Makefile
+++ b/src/test/Makefile
@@ -2,7 +2,8 @@ RUN_USERNS := lxc-usernsexec -m "u:0:`id -u`:1" -m "g:0:`id -g`:1" --
all: test
-test: test_setup test_snapshot test_bindmount test_idmap test_config test_raw_config_write
+test: test_setup test_snapshot test_bindmount test_idmap test_config test_raw_config_write \
+ test_lifecycle
test_setup: run_setup_tests.pl
if test -e /run/lock/sbuild; then \
@@ -30,5 +31,8 @@ test_config: config_tests.pl
test_raw_config_write: raw_config_write_test.pl
./raw_config_write_test.pl
+test_lifecycle: lifecycle_tests.pl
+ ./lifecycle_tests.pl
+
clean:
- rm -rf tmprootfs
+ rm -rf tmprootfs lifecycle-working
diff --git a/src/test/config_tests.pl b/src/test/config_tests.pl
index beb7e37..a37a9ad 100755
--- a/src/test/config_tests.pl
+++ b/src/test/config_tests.pl
@@ -175,4 +175,27 @@ subtest 'restore does not take over pending changes of the backup' => sub {
}
};
+subtest 'meta property' => sub {
+ like(PVE::LXC::Config->new_meta_info_string(), qr/^ctime=\d+$/, 'creation time is set');
+
+ my $raw = "hostname: ct100\nmeta: ctime=1700000000\n";
+ my $conf = PVE::LXC::Config::parse_pct_config('/lxc/100.conf', $raw, 1);
+ is($conf->{meta}, 'ctime=1700000000', 'meta is parsed');
+ is(PVE::LXC::Config::write_pct_config('/lxc/100.conf', $conf), $raw, 'meta is written');
+
+ $raw = "hostname: ct100\nmeta: ctime=1700000000,unknown=value\n";
+ my $warnings = '';
+ {
+ local $SIG{__WARN__} = sub { $warnings .= $_[0] };
+ $conf = PVE::LXC::Config::parse_pct_config('/lxc/100.conf', $raw);
+ }
+ like($warnings, qr/unable to parse value of 'meta'/, 'unknown member causes a warning');
+ is($conf->{meta}, 'ctime=1700000000,unknown=value', 'meta with unknown member is kept');
+ eval { PVE::LXC::Config::parse_pct_config('/lxc/100.conf', $raw, 1) };
+ like($@, qr/unable to parse value of 'meta'/, 'unknown member fails strict parsing');
+
+ ok(!exists(PVE::LXC::Config->json_config_properties({})->{meta}), 'meta cannot be set');
+ ok(exists(PVE::LXC::Config->json_config_properties({}, 1)->{meta}), 'meta is returned');
+};
+
done_testing();
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
new file mode 100755
index 0000000..01e0106
--- /dev/null
+++ b/src/test/lifecycle_tests.pl
@@ -0,0 +1,613 @@
+#!/usr/bin/perl
+
+# Runs API calls that create, restore, update and migrate containers against a fake cluster
+# filesystem in a working directory, with storage and container runtime operations mocked.
+
+use v5.36;
+
+use lib qw(..);
+
+# The API modules load modules of pve-ha-manager, which is not a build dependency, so stub them. Tests
+# that call into them mock the functions they need.
+BEGIN {
+ $INC{'PVE/HA/Config.pm'} = $INC{'PVE/HA/Env/PVE2.pm'} = __FILE__;
+}
+
+use File::Path qw(make_path remove_tree);
+use JSON;
+use Socket qw(AF_UNIX SOCK_STREAM PF_UNSPEC);
+use Test::MockModule;
+use Test::More;
+
+use PVE::Cluster;
+use PVE::GuestHelpers;
+use PVE::INotify;
+use PVE::JSONSchema;
+use PVE::Tools qw(file_get_contents file_set_contents);
+
+use PVE::LXC;
+use PVE::LXC::Config;
+use PVE::LXC::Create;
+
+use PVE::API2::LXC;
+use PVE::API2::LXC::Config;
+
+my $WORKDIR = 'lifecycle-working';
+my $NODENAME = PVE::INotify::nodename();
+my $BACKUP_VOLID = 'local:backup/vzdump-lxc-100-2026_01_01-00_00_00.tar.zst';
+
+remove_tree($WORKDIR);
+make_path("$WORKDIR/lock");
+
+# Fake cluster filesystem
+
+my sub cfs_file($path) {
+ return "$WORKDIR/cfs/$path";
+}
+
+my sub ct_config_path($vmid, $node = undef) {
+ return "nodes/" . ($node // $NODENAME) . "/lxc/$vmid.conf";
+}
+
+my sub write_raw_config($vmid, $raw, $node = undef) {
+ my $file = cfs_file(ct_config_path($vmid, $node));
+ make_path($file =~ s|/[^/]+$||r);
+ file_set_contents($file, $raw);
+}
+
+my sub read_raw_config($vmid, $node = undef) {
+ my $file = cfs_file(ct_config_path($vmid, $node));
+ return -f $file ? file_get_contents($file) : undef;
+}
+
+# VMIDs whose container config still existed when their firewall config got removed
+my $vmfw_removed_reserved = {};
+
+my sub reset_configs {
+ remove_tree("$WORKDIR/cfs", "$WORKDIR/rootfs");
+ $vmfw_removed_reserved = {};
+}
+
+my $storage_cfg = {
+ ids => {
+ local => {
+ type => 'dir',
+ path => '/var/lib/vz',
+ content => { rootdir => 1, images => 1, backup => 1, vztmpl => 1 },
+ },
+ },
+};
+
+# recording used guest IDs needs the cluster filesystem and is not what gets tested here
+my $guest_id_module = Test::MockModule->new('PVE::GuestID');
+$guest_id_module->mock(register_used_id => sub { return; });
+
+my $cluster_module = Test::MockModule->new('PVE::Cluster');
+$cluster_module->mock(
+ cfs_update => sub { },
+ check_cfs_quorum => sub { },
+ check_node_exists => sub { },
+ get_config => sub {
+ my ($path) = @_;
+ my $file = cfs_file($path);
+ return -f $file ? file_get_contents($file) : undef;
+ },
+ cfs_read_file => sub {
+ my ($path) = @_;
+ die "unexpected read of '$path'\n" if $path !~ m|^nodes/[^/]+/lxc/\d+\.conf$|;
+ my $file = cfs_file($path);
+ return undef if !-f $file;
+ return PVE::LXC::Config::parse_pct_config("/etc/pve/$path", file_get_contents($file));
+ },
+ cfs_write_file => sub {
+ my ($path, $conf) = @_;
+ die "unexpected write of '$path'\n" if $path !~ m|^nodes/[^/]+/lxc/\d+\.conf$|;
+ my $raw = PVE::LXC::Config::write_pct_config("/etc/pve/$path", $conf);
+ my $file = cfs_file($path);
+ make_path($file =~ s|/[^/]+$||r);
+ file_set_contents($file, $raw);
+ },
+ check_vmid_unused => sub {
+ my ($vmid, $noerr) = @_;
+ return 1 if !-f cfs_file(ct_config_path($vmid));
+ return undef if $noerr;
+ die "CT $vmid already exists on node '$NODENAME'\n";
+ },
+);
+
+my $api_module = Test::MockModule->new('PVE::API2::LXC');
+$api_module->mock(
+ cfs_read_file => sub {
+ my ($path) = @_;
+ die "unexpected read of '$path'\n" if $path ne 'storage.cfg';
+ return $storage_cfg;
+ },
+);
+
+my $config_module = Test::MockModule->new('PVE::LXC::Config');
+$config_module->mock(
+ config_file => sub {
+ my ($class, $vmid, $node) = @_;
+ return cfs_file(ct_config_path($vmid, $node));
+ },
+ config_file_lock => sub {
+ my ($class, $vmid) = @_;
+ return "$WORKDIR/lock/pve-config-$vmid.lock";
+ },
+);
+
+my $repl_config_module = Test::MockModule->new('PVE::ReplicationConfig');
+$repl_config_module->mock(
+ new => sub { return bless {}, 'PVE::ReplicationConfig' },
+ check_for_existing_jobs => sub { return undef },
+);
+
+# Fake environment, records task output
+
+my $task_log = '';
+
+package PVE::LXC::Test::RPCEnvironment {
+
+ sub new($class, $user) {
+ return bless { user => $user, type => 'cli' }, $class;
+ }
+
+ sub get_user { return $_[0]->{user} }
+ sub check { return 1 }
+ sub check_vm_perm { return 1 }
+ sub check_pool_exist { return 1 }
+
+ sub warn($self, $msg) {
+ $task_log .= "WARN: $msg\n";
+ }
+
+ sub fork_worker($self, $type, $id, $user, $function, @) {
+ my $upid = "UPID:$NODENAME:$type:$id";
+ $function->($upid);
+ return $upid;
+ }
+}
+
+my $rpcenv = PVE::LXC::Test::RPCEnvironment->new('root@pam');
+
+my $rpcenv_module = Test::MockModule->new('PVE::RPCEnvironment');
+$rpcenv_module->mock(get => sub { return $rpcenv });
+
+my $restenv_module = Test::MockModule->new('PVE::RESTEnvironment');
+$restenv_module->mock(
+ log_warn => sub {
+ my ($msg) = @_;
+ chomp($msg);
+ $task_log .= "WARN: $msg\n";
+ },
+);
+
+# Runs $code like a task, with its output going to the task log. Returns the error, if any.
+my sub run_task($code) {
+ $task_log = '';
+ my $output = '';
+ eval {
+ local *STDOUT;
+ open(STDOUT, '>', \$output) or die "unable to capture output - $!\n";
+ local $SIG{__WARN__} = sub { $output .= "WARN: $_[0]" };
+ $code->();
+ };
+ my $err = $@;
+ $task_log = $output . $task_log;
+
+ return $err;
+}
+
+# Fake storage and container runtime
+
+my $backup_raw_config;
+
+my $storage_module = Test::MockModule->new('PVE::Storage');
+$storage_module->mock(
+ config => sub { return $storage_cfg },
+ activate_storage => sub { },
+ activate_volumes => sub { },
+ deactivate_volumes => sub { },
+ check_volume_access => sub { },
+ storage_check_enabled => sub {
+ my ($cfg, $storeid) = @_;
+ return $storage_cfg->{ids}->{$storeid};
+ },
+ storage_config => sub {
+ my ($cfg, $storeid) = @_;
+ return $storage_cfg->{ids}->{$storeid} // die "storage '$storeid' does not exist\n";
+ },
+ storage_has_feature => sub { return 0 },
+ get_bandwidth_limit => sub { return undef },
+);
+
+my $lxc_module = Test::MockModule->new('PVE::LXC');
+$lxc_module->mock(
+ check_running => sub { return 0 },
+ create_disks => sub {
+ my ($storecfg, $vmid, $settings, $conf) = @_;
+ my $vollist = [];
+ for my $ms (sort keys %$settings) {
+ my $volid = "local:$vmid/vm-$vmid-disk-$ms.raw";
+ $conf->{$ms} = "$volid,size=4G";
+ push @$vollist, $volid;
+ }
+ return $vollist;
+ },
+ destroy_disks => sub { },
+ destroy_lxc_container => sub {
+ my ($storecfg, $vmid, $conf, $replacement_conf) = @_;
+ if (defined($replacement_conf)) {
+ PVE::LXC::Config->write_config($vmid, $replacement_conf);
+ } else {
+ PVE::LXC::Config->destroy_config($vmid);
+ }
+ },
+ mount_all => sub {
+ my ($vmid) = @_;
+ my $rootdir = "$WORKDIR/rootfs/$vmid";
+ make_path($rootdir);
+ return $rootdir;
+ },
+ umount_all => sub { },
+ create_ifaces_ipams_ips => sub { },
+ delete_ifaces_ipams_ips => sub { },
+ update_lxc_config => sub { },
+);
+
+my $create_module = Test::MockModule->new('PVE::LXC::Create');
+$create_module->mock(
+ archive_is_oci_format => sub { return 0 },
+ recover_config => sub {
+ my ($storecfg, $volid, $vmid) = @_;
+ my $conf = PVE::LXC::Config::parse_pct_config("/lxc/$vmid.conf", $backup_raw_config);
+ delete $conf->{snapshots};
+ my $mp_param = {};
+ PVE::LXC::Config->foreach_volume($conf,
+ sub { $mp_param->{ $_[0] } = $conf->{ $_[0] } });
+ return wantarray ? ($conf, $mp_param) : $conf;
+ },
+ restore_archive => sub {
+ my ($storecfg, $archive, $rootdir, $conf) = @_;
+ if (defined($backup_raw_config)) {
+ make_path("$rootdir/etc/vzdump");
+ file_set_contents("$rootdir/etc/vzdump/pct.conf", $backup_raw_config);
+ }
+ },
+);
+
+my $setup_module = Test::MockModule->new('PVE::LXC::Setup');
+$setup_module->mock(new => sub { return bless {}, 'PVE::LXC::Test::Setup' });
+
+package PVE::LXC::Test::Setup {
+ sub post_create_hook { }
+ sub post_restore_hook { }
+ sub post_clone_hook { }
+}
+
+my $firewall_module = Test::MockModule->new('PVE::Firewall');
+$firewall_module->mock(
+ remove_vmfw_conf => sub {
+ my ($vmid) = @_;
+ $vmfw_removed_reserved->{$vmid}++ if defined(read_raw_config($vmid));
+ },
+);
+
+my $access_control_module = Test::MockModule->new('PVE::AccessControl');
+$access_control_module->mock(assemble_tunnel_ticket => sub { return 'ticket' });
+
+# API call helpers
+
+my sub create_ct($vmid, $param = undef) {
+ return run_task(sub {
+ PVE::API2::LXC->create_vm({
+ node => $NODENAME,
+ vmid => $vmid,
+ ostemplate => 'local:vztmpl/debian-13-standard_13.1-2_amd64.tar.zst',
+ hostname => "ct$vmid",
+ ($param // {})->%*,
+ });
+ });
+}
+
+my sub restore_ct($vmid, $param = undef) {
+ return run_task(sub {
+ PVE::API2::LXC->create_vm({
+ node => $NODENAME,
+ vmid => $vmid,
+ ostemplate => $BACKUP_VOLID,
+ restore => 1,
+ ($param // {})->%*,
+ });
+ });
+}
+
+my sub update_ct($vmid, $param) {
+ return run_task(sub {
+ PVE::API2::LXC::Config->update_vm({ node => $NODENAME, vmid => $vmid, $param->%* });
+ });
+}
+
+# Runs the migration tunnel for $vmid with the commands @commands, which are sent at once, and
+# returns the error of the task, if any, and the replies.
+my sub run_mtunnel($vmid, @commands) {
+ socketpair(my $tunnel_end, my $client_end, AF_UNIX, SOCK_STREAM, PF_UNSPEC)
+ or die "socketpair failed - $!\n";
+ for my $command (@commands) {
+ print $client_end JSON::encode_json($command) . "\n";
+ }
+ $client_end->flush();
+ shutdown($client_end, 1);
+
+ my $socket_module = Test::MockModule->new('IO::Socket::UNIX');
+ $socket_module->mock(
+ new => sub { return bless { conn => $tunnel_end }, 'PVE::LXC::Test::Socket' });
+
+ my $err = run_task(sub { PVE::API2::LXC->mtunnel({ node => $NODENAME, vmid => $vmid }) });
+
+ close($tunnel_end);
+ my $replies = [map { JSON::decode_json($_) } <$client_end>];
+ close($client_end);
+
+ return ($err, $replies);
+}
+
+package PVE::LXC::Test::Socket {
+ sub accept { return $_[0]->{conn} }
+}
+
+my $INCOMING_CONFIG = <<~'EOF';
+ arch: amd64
+ hostname: incoming
+ memory: 1024
+ meta: ctime=1600000000
+ ostype: debian
+ rootfs: local:300/vm-300-disk-0.raw,size=4G
+ unprivileged: 1
+ EOF
+
+my sub backup_config($meta = undef) {
+ my $raw = <<~'EOF';
+ arch: amd64
+ hostname: backup
+ memory: 2048
+ ostype: debian
+ rootfs: local:100/vm-100-disk-0.raw,size=8G
+ unprivileged: 1
+ EOF
+ $raw .= "meta: $meta\n" if defined($meta);
+
+ return $raw;
+}
+
+my sub existing_config($meta = undef) {
+ my $raw = <<~'EOF';
+ arch: amd64
+ hostname: existing
+ memory: 512
+ ostype: debian
+ rootfs: local:200/vm-200-disk-0.raw,size=4G
+ unprivileged: 1
+ EOF
+ $raw .= "meta: $meta\n" if defined($meta);
+
+ return $raw;
+}
+
+# Returns the members of the 'meta' property in the config of $vmid.
+my sub meta_of($vmid) {
+ my $conf = PVE::LXC::Config->load_config($vmid);
+
+ return { map { split(/=/, $_, 2) } split(/,/, $conf->{meta} // '') };
+}
+
+# Tests
+
+subtest 'meta cannot be changed through the API' => sub {
+ reset_configs();
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ my $raw = read_raw_config(200);
+
+ for my $user ('root@pam', 'admin@pve') {
+ $rpcenv = PVE::LXC::Test::RPCEnvironment->new($user);
+
+ my $err = update_ct(200, { meta => 'ctime=1' });
+ like($err, qr/meta: property is not defined in schema/, "$user cannot set meta");
+
+ $err = update_ct(200, { delete => 'meta' });
+ like($err, qr/unable to delete read-only option: 'meta'/, "$user cannot delete meta");
+
+ $err = update_ct(200, { revert => 'meta' });
+ like($err, qr/unable to revert read-only option: 'meta'/, "$user cannot revert meta");
+
+ $err = create_ct(201, { meta => 'ctime=1' });
+ like($err, qr/meta: property is not defined in schema/,
+ "$user cannot create with meta");
+
+ $err = restore_ct(200, { force => 1, meta => 'ctime=1' });
+ like(
+ $err,
+ qr/meta: property is not defined in schema/,
+ "$user cannot restore with meta",
+ );
+ }
+ $rpcenv = PVE::LXC::Test::RPCEnvironment->new('root@pam');
+
+ is(read_raw_config(200), $raw, 'config is unchanged');
+ ok(!defined(read_raw_config(201)), 'no container got created');
+
+ my $conf = PVE::LXC::Config->load_config(200);
+ eval { PVE::LXC::Config->update_pct_config(200, $conf, 0, { meta => 'ctime=1' }, [], []) };
+ like($@, qr/unable to modify read-only option: 'meta'/, 'update cannot set meta');
+
+ write_raw_config(200, existing_config());
+ my $err = update_ct(200, { delete => 'meta' });
+ like($err, qr/unable to delete read-only option: 'meta'/,
+ 'meta cannot be deleted if unset');
+};
+
+subtest 'pending changes of meta are dropped' => sub {
+ my $raw = existing_config('ctime=1500000000') . <<~'EOF';
+
+ [pve:pending]
+ delete: meta
+ meta: ctime=1
+ EOF
+
+ reset_configs();
+ write_raw_config(200, $raw);
+ my $err = update_ct(200, { hostname => 'changed' });
+ is($err, '', 'update succeeds');
+ like($task_log, qr/ignoring pending change of read-only option 'meta'/, 'drop is logged');
+ my $conf = PVE::LXC::Config->load_config(200);
+ is(meta_of(200)->{ctime}, 1500000000, 'ctime is kept');
+ is_deeply($conf->{pending}, {}, 'pending change of meta is dropped');
+
+ for my $pending ('meta: ctime=1', 'delete: meta') {
+ $conf = PVE::LXC::Config::parse_pct_config(
+ '/lxc/200.conf',
+ existing_config('ctime=1500000000') . "\n[pve:pending]\n$pending\n",
+ );
+ {
+ local $SIG{__WARN__} = sub { };
+ PVE::LXC::Config->vmconfig_apply_pending(200, $conf, $storage_cfg);
+ }
+ is($conf->{meta}, 'ctime=1500000000', "applying '$pending' keeps meta");
+ is_deeply($conf->{pending}, {}, "applying drops '$pending'");
+ }
+
+ $conf = PVE::LXC::Config::parse_pct_config('/lxc/200.conf', $raw);
+ my $cgroup_module = Test::MockModule->new('PVE::LXC::CGroup');
+ $cgroup_module->mock(new => sub { return bless {}, 'PVE::LXC::CGroup' });
+ $lxc_module->mock(find_lxc_pid => sub { return 1 });
+ {
+ local $SIG{__WARN__} = sub { };
+ PVE::LXC::Config->vmconfig_hotplug_pending(200, $conf, $storage_cfg, undef, {});
+ }
+ $lxc_module->unmock('find_lxc_pid');
+ is($conf->{meta}, 'ctime=1500000000', 'hotplugging keeps meta');
+ is_deeply($conf->{pending}, {}, 'hotplugging drops the pending change of meta');
+};
+
+subtest 'meta with members unknown to this version can be read' => sub {
+ reset_configs();
+ write_raw_config(200, existing_config('ctime=1500000000,future=value'));
+
+ my $info = PVE::API2::LXC::Config->map_method_by_name('vm_config');
+ my $conf = eval {
+ local $SIG{__WARN__} = sub { };
+ PVE::API2::LXC::Config->handle($info, { node => $NODENAME, vmid => 200 }, 1);
+ };
+ is($@, '', 'config passes the result verification');
+ is($conf->{meta}, 'ctime=1500000000,future=value', 'meta is returned as it is');
+};
+
+subtest 'create sets the creation time' => sub {
+ reset_configs();
+
+ my $before = time();
+ my $err = create_ct(201);
+ my $after = time();
+ is($err, '', 'create succeeds');
+
+ my $ctime = meta_of(201)->{ctime};
+ ok($ctime >= $before && $ctime <= $after, 'ctime is the time of the create');
+};
+
+subtest 'restore keeps the meta information of the backup' => sub {
+ reset_configs();
+
+ $backup_raw_config = backup_config('ctime=1400000000');
+ my $err = restore_ct(201);
+ is($err, '', 'restore to a free VMID succeeds');
+ is(meta_of(201)->{ctime}, 1400000000, 'backup ctime is kept');
+
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ $err = restore_ct(200, { force => 1 });
+ is($err, '', 'restore over an existing container succeeds');
+ is(meta_of(200)->{ctime}, 1400000000, 'backup ctime is used');
+};
+
+subtest 'restore over a container keeps it if replacing it fails early' => sub {
+ $backup_raw_config = backup_config('ctime=1400000000');
+
+ my $kept = sub {
+ my ($name, $expected_err) = @_;
+
+ my $err = restore_ct(200, { force => 1 });
+ like($err, $expected_err, "$name: restore fails");
+ ok(!$vmfw_removed_reserved->{200}, "$name: firewall config is kept");
+ ok(defined(read_raw_config(200)), "$name: config is kept") or return;
+ my $conf = PVE::LXC::Config->load_config(200);
+ is($conf->{hostname}, 'existing', "$name: config is the one of the container");
+ is($conf->{rootfs}, 'local:200/vm-200-disk-0.raw,size=4G', "$name: volume is kept");
+ ok(!defined($conf->{lock}), "$name: lock is removed");
+ };
+
+ my $mocked_create_disks = \&PVE::LXC::create_disks;
+ $lxc_module->mock(create_disks => sub { die "unable to allocate volume - storage full\n" });
+ reset_configs();
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ $kept->('allocation fails', qr/storage full/);
+ $lxc_module->mock(create_disks => $mocked_create_disks);
+
+ my $mocked_destroy = \&PVE::LXC::destroy_lxc_container;
+ $lxc_module->mock(
+ destroy_lxc_container => sub {
+ my ($storecfg, $vmid, $conf) = @_;
+ # like the real function, which checks this before it removes anything
+ die "base volume 'local:200/base-200-disk-0.raw' is still in use by linked clone\n"
+ if $conf->{template};
+ return $mocked_destroy->(@_);
+ },
+ );
+ reset_configs();
+ write_raw_config(200, existing_config('ctime=1500000000') . "template: 1\n");
+ $kept->('template with linked clones', qr/still in use by linked clone/);
+ $lxc_module->mock(destroy_lxc_container => $mocked_destroy);
+};
+
+subtest 'create refuses a container created after the call started' => sub {
+ my $raw = existing_config('ctime=1500000000') =~ s/^hostname: existing$/hostname: other/mr;
+ reset_configs();
+
+ my $mocked_activate = \&PVE::Storage::activate_storage;
+ $storage_module->mock(
+ activate_storage => sub {
+ # another task creates CT 200 between the checks and the reservation
+ write_raw_config(200, $raw) if !defined(read_raw_config(200));
+ return $mocked_activate->(@_);
+ },
+ );
+ # like the guest list of PVE::Cluster, which only changes with cfs_update at the call start
+ my $mocked_unused = \&PVE::Cluster::check_vmid_unused;
+ $cluster_module->mock(check_vmid_unused => sub { return 1 });
+
+ my $err = create_ct(200);
+ $cluster_module->mock(check_vmid_unused => $mocked_unused);
+ $storage_module->mock(activate_storage => $mocked_activate);
+
+ like($err, qr/CT 200 already exists/, 'create is refused');
+ is(read_raw_config(200), $raw, 'container is unchanged');
+};
+
+subtest 'remote migration keeps the meta information of the source' => sub {
+ reset_configs();
+
+ my ($err, $replies) = run_mtunnel(
+ 300,
+ { cmd => 'config', conf => $INCOMING_CONFIG },
+ { cmd => 'quit', cleanup => 0 },
+ );
+ is($err, '', 'tunnel exits normally') or diag(explain($replies), $task_log);
+ ok($replies->[0]->{success}, 'config command succeeds');
+
+ my $conf = PVE::LXC::Config->load_config(300);
+ is(meta_of(300)->{ctime}, 1600000000, 'source ctime is kept');
+ is($conf->{hostname}, 'incoming', 'other options are applied');
+ is($conf->{lock}, 'migrate', 'config is locked for the migration');
+};
+
+remove_tree($WORKDIR);
+
+done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH container v3 22/38] fix #7846: config: add a guest identity to the 'meta' property
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (20 preceding siblings ...)
2026-11-18 4:37 ` [PATCH container v3 21/38] config: add read-only meta property with the creation time Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 23/38] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
` (17 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Give containers a guest identity in the 'meta' property: a random
identifier, 'uid', and the time the container was registered in the
cluster, 'rtime'.
With identities enabled for the config class, every config write keeps
the identity a container already has, or assigns one if it has none,
whatever the written config carries. So create and clone get a new
identity when the VMID is reserved, while config changes, pending
changes applied by the post-stop hook, snapshot rollback and a restore
over an existing container keep it. A container without identity gets
one with its first config write by an upgraded node.
The post-stop hook writes the config it loaded before, without the
guest lock. As the write reads the existing identity under the
per-VMID identity lock, an identity that another writer assigned in the
meantime is kept and not replaced by a second one.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
no changes since v2
changes v1 -> v2:
* rebase, adapt the tests
src/PVE/LXC/Config.pm | 8 ++
src/test/lifecycle_tests.pl | 242 +++++++++++++++++++++++++++++++++++-
2 files changed, 248 insertions(+), 2 deletions(-)
diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index c274e93..34e8065 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -9,6 +9,7 @@ use PVE::AbstractConfig;
use PVE::Cluster qw(cfs_register_file);
use PVE::DataCenterConfig;
use PVE::GuestHelpers;
+use PVE::GuestIdentity;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
use PVE::Network::SDN::Vnets;
@@ -60,6 +61,12 @@ sub cfs_config_path {
return "nodes/$node/lxc/$vmid.conf";
}
+sub __identity_enabled {
+ my ($class) = @_;
+
+ return 1;
+}
+
sub mountpoint_backup_enabled {
my ($class, $mp_key, $mountpoint) = @_;
@@ -512,6 +519,7 @@ my $meta_info_fmt = {
minimum => 0,
optional => 1,
},
+ PVE::GuestIdentity::meta_format_properties()->%*,
};
my $confdesc = {
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
index 01e0106..6652cc8 100755
--- a/src/test/lifecycle_tests.pl
+++ b/src/test/lifecycle_tests.pl
@@ -7,10 +7,12 @@ use v5.36;
use lib qw(..);
-# The API modules load modules of pve-ha-manager, which is not a build dependency, so stub them. Tests
-# that call into them mock the functions they need.
+# The API modules load modules of pve-ha-manager and the backup plugin PVE::VZDump of pve-manager,
+# which are not build dependencies, so stub them. Tests that call into them mock the functions they
+# need.
BEGIN {
$INC{'PVE/HA/Config.pm'} = $INC{'PVE/HA/Env/PVE2.pm'} = __FILE__;
+ $INC{'PVE/VZDump.pm'} = __FILE__;
}
use File::Path qw(make_path remove_tree);
@@ -136,6 +138,26 @@ $config_module->mock(
},
);
+# Guest identities are generated deterministically, the local identity lock is taken in the working
+# directory.
+
+$PVE::GuestHelpers::lockdir = "$WORKDIR/lock";
+
+my $random_counter = 0;
+my $now_ms = 1_700_000_000_000;
+
+my $identity_module = Test::MockModule->new('PVE::GuestIdentity');
+$identity_module->mock(
+ __random_bytes => sub {
+ my ($len) = @_;
+ die "unexpected length $len\n" if $len != 20;
+ # one byte per uid character, the last three hold 5 bits of the counter each
+ my $n = ++$random_counter;
+ return ("\0" x 17) . pack('C3', map { ($n >> $_) & 31 } (10, 5, 0));
+ },
+ __time_ms => sub { return $now_ms },
+);
+
my $repl_config_module = Test::MockModule->new('PVE::ReplicationConfig');
$repl_config_module->mock(
new => sub { return bless {}, 'PVE::ReplicationConfig' },
@@ -180,6 +202,7 @@ $restenv_module->mock(
chomp($msg);
$task_log .= "WARN: $msg\n";
},
+ setup_default_cli_env => sub { },
);
# Runs $code like a task, with its output going to the task log. Returns the error, if any.
@@ -219,6 +242,8 @@ $storage_module->mock(
},
storage_has_feature => sub { return 0 },
get_bandwidth_limit => sub { return undef },
+ volume_rollback_is_possible => sub { return 1 },
+ volume_snapshot_rollback => sub { },
);
my $lxc_module = Test::MockModule->new('PVE::LXC');
@@ -250,6 +275,10 @@ $lxc_module->mock(
return $rootdir;
},
umount_all => sub { },
+ copy_volume => sub {
+ my ($mp, $newid) = @_;
+ return "local:$newid/vm-$newid-disk-0.raw";
+ },
create_ifaces_ipams_ips => sub { },
delete_ifaces_ipams_ips => sub { },
update_lxc_config => sub { },
@@ -287,6 +316,7 @@ package PVE::LXC::Test::Setup {
my $firewall_module = Test::MockModule->new('PVE::Firewall');
$firewall_module->mock(
+ clone_vmfw_conf => sub { },
remove_vmfw_conf => sub {
my ($vmid) = @_;
$vmfw_removed_reserved->{$vmid}++ if defined(read_raw_config($vmid));
@@ -328,6 +358,17 @@ my sub update_ct($vmid, $param) {
});
}
+my sub clone_ct($vmid, $newid, $param = undef) {
+ return run_task(sub {
+ PVE::API2::LXC->clone_vm({
+ node => $NODENAME,
+ vmid => $vmid,
+ newid => $newid,
+ ($param // {})->%*,
+ });
+ });
+}
+
# Runs the migration tunnel for $vmid with the commands @commands, which are sent at once, and
# returns the error of the task, if any, and the replies.
my sub run_mtunnel($vmid, @commands) {
@@ -401,6 +442,10 @@ my sub meta_of($vmid) {
return { map { split(/=/, $_, 2) } split(/,/, $conf->{meta} // '') };
}
+my sub identity_of($vmid) {
+ return PVE::LXC::Config->get_identity(PVE::LXC::Config->load_config($vmid));
+}
+
# Tests
subtest 'meta cannot be changed through the API' => sub {
@@ -608,6 +653,199 @@ subtest 'remote migration keeps the meta information of the source' => sub {
is($conf->{lock}, 'migrate', 'config is locked for the migration');
};
+subtest 'create assigns a new guest identity' => sub {
+ reset_configs();
+ $now_ms = 1_700_000_000_000;
+
+ my $err = create_ct(201);
+ is($err, '', 'create succeeds');
+
+ my $meta = meta_of(201);
+ like($meta->{uid}, qr/^[a-z2-7]{20}$/, 'uid is assigned');
+ is($meta->{rtime}, 1_700_000_000_000, 'rtime is the time of the create');
+ ok(defined($meta->{ctime}), 'ctime is kept next to the identity');
+ ok(!defined(PVE::LXC::Config->load_config(201)->{lock}), 'create lock is removed');
+
+ $err = create_ct(202);
+ isnt(identity_of(202)->{uid}, $meta->{uid}, 'another container gets another uid');
+};
+
+subtest 'config changes keep the guest identity' => sub {
+ reset_configs();
+ my $meta = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis';
+ write_raw_config(200, existing_config($meta));
+ $now_ms = 1_700_000_000_000;
+
+ my $err = update_ct(200, { hostname => 'changed', memory => 1024 });
+ is($err, '', 'update succeeds');
+ my $conf = PVE::LXC::Config->load_config(200);
+ is($conf->{hostname}, 'changed', 'change is applied');
+ is($conf->{meta}, $meta, 'meta is unchanged');
+
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ $err = update_ct(200, { hostname => 'first-write' });
+ is($err, '', 'update of a container without identity succeeds');
+ my $identity = identity_of(200);
+ like($identity->{uid}, qr/^[a-z2-7]{20}$/, 'first write assigns a uid');
+ is($identity->{rtime}, 1_700_000_000_000, 'first write sets rtime');
+ is(meta_of(200)->{ctime}, 1500000000, 'ctime is kept');
+
+ $now_ms += 1000;
+ $err = update_ct(200, { hostname => 'second-write' });
+ is_deeply(identity_of(200), $identity, 'later writes keep the identity');
+};
+
+subtest 'clone assigns a new guest identity' => sub {
+ reset_configs();
+ write_raw_config(
+ 200,
+ existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis'),
+ );
+ $now_ms = 1_700_000_000_000;
+
+ my $err = clone_ct(200, 201, { hostname => 'clone' });
+ is($err, '', 'clone succeeds') or diag($task_log);
+
+ my $meta = meta_of(201);
+ like($meta->{uid}, qr/^[a-z2-7]{20}$/, 'clone has a uid');
+ isnt($meta->{uid}, 'existingexistingexis', 'clone does not get the uid of the source');
+ is($meta->{rtime}, 1_700_000_000_000, 'rtime is the time of the clone');
+ is($meta->{ctime}, 1500000000, 'ctime of the source is kept');
+ is(identity_of(200)->{uid}, 'existingexistingexis', 'source keeps its uid');
+};
+
+subtest 'snapshot rollback keeps the current guest identity' => sub {
+ reset_configs();
+ write_raw_config(200, <<~'EOF');
+ arch: amd64
+ hostname: existing
+ memory: 512
+ meta: ctime=1500000000,rtime=1600000000000,uid=existingexistingexis
+ ostype: debian
+ parent: snap
+ rootfs: local:200/vm-200-disk-0.raw,size=4G
+ unprivileged: 1
+
+ [snap]
+ arch: amd64
+ hostname: snapshot
+ memory: 512
+ meta: ctime=1500000000,rtime=1500000000000,uid=snapshotsnapshotsnap
+ ostype: debian
+ rootfs: local:200/vm-200-disk-0.raw,size=4G
+ snaptime: 1500000000
+ unprivileged: 1
+ EOF
+
+ my $err = run_task(sub { PVE::LXC::Config->snapshot_rollback(200, 'snap') });
+ is($err, '', 'rollback succeeds') or diag($task_log);
+
+ my $conf = PVE::LXC::Config->load_config(200);
+ is($conf->{hostname}, 'snapshot', 'config of the snapshot is restored');
+ is_deeply(
+ identity_of(200),
+ { uid => 'existingexistingexis', rtime => 1600000000000 },
+ 'current identity is kept',
+ );
+};
+
+subtest 'in-place restore keeps the guest identity of the target' => sub {
+ reset_configs();
+ my $meta = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis';
+ write_raw_config(200, existing_config($meta));
+
+ for my $backup_meta (
+ undef, 'ctime=1400000000,rtime=1400000000000,uid=existingexistingexis',
+ ) {
+ $backup_raw_config = backup_config($backup_meta);
+ my $err = restore_ct(200, { force => 1 });
+ is($err, '', 'restore succeeds') or diag($task_log);
+ is_deeply(
+ identity_of(200),
+ { uid => 'existingexistingexis', rtime => 1500000000000 },
+ 'identity is kept',
+ );
+ is(PVE::LXC::Config->load_config(200)->{hostname}, 'backup', 'backup is restored');
+ }
+};
+
+subtest 'backup assigns the guest identity and includes it' => sub {
+ reset_configs();
+ write_raw_config(987, existing_config('ctime=1500000000'));
+ $now_ms = 1_700_000_000_000;
+
+ require PVE::VZDump::LXC;
+ my $plugin = bless { vzdump => { opts => { pbs => 1 } } }, 'PVE::VZDump::LXC';
+
+ $plugin->lock_vm(987);
+ my $identity = identity_of(987);
+ like($identity->{uid}, qr/^[a-z2-7]{20}$/, 'backup lock assigns a uid');
+ is($identity->{rtime}, $now_ms, 'backup lock sets rtime');
+
+ $plugin->assemble({ tmpdir => "$WORKDIR/backup" }, 987);
+ my $raw = file_get_contents("$WORKDIR/backup/etc/vzdump/pct.conf");
+ my $backup_conf = PVE::LXC::Config::parse_pct_config('/lxc/987.conf', $raw);
+ is_deeply(
+ PVE::LXC::Config->get_identity($backup_conf),
+ $identity,
+ 'backup has the identity',
+ );
+
+ $now_ms += 1000;
+ $plugin->unlock_vm(987);
+ is_deeply(identity_of(987), $identity, 'removing the backup lock keeps the identity');
+};
+
+subtest 'post-stop hook keeps an identity stored while it ran' => sub {
+ reset_configs();
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ $now_ms = 1_700_000_000_000;
+
+ my $stored;
+ $lxc_module->mock(
+ get_staging_tempfs => sub { return "$WORKDIR/staging" },
+ vm_stop_cleanup => sub {
+ # another writer assigns the identity after the hook loaded the config
+ PVE::LXC::Config->lock_config(
+ 200,
+ sub {
+ my $conf = PVE::LXC::Config->load_config(200);
+ $conf->{description} = 'written while the hook ran';
+ PVE::LXC::Config->write_config(200, $conf);
+ },
+ );
+ $stored = identity_of(200);
+ $now_ms += 1000;
+ },
+ );
+ my $tools_module = Test::MockModule->new('PVE::Tools');
+ $tools_module->mock(run_command => sub { });
+ my $guest_helpers_module = Test::MockModule->new('PVE::GuestHelpers');
+ $guest_helpers_module->mock(exec_hookscript => sub { });
+
+ local $ENV{LXC_HOOK_VERSION} = '1';
+ local $ENV{LXC_NAME} = '200';
+ local $ENV{LXC_HOOK_SECTION} = 'lxc';
+ local $ENV{LXC_HOOK_TYPE} = 'post-stop';
+ local $ENV{LXC_ROOTFS_MOUNT} = "$WORKDIR/rootfs/200";
+ local $ENV{LXC_ROOTFS_PATH} = "$WORKDIR/rootfs/200";
+ local $ENV{LXC_CONFIG_FILE} = "$WORKDIR/lxc-200.config";
+ delete local $ENV{LXC_TARGET};
+
+ my $err = run_task(sub {
+ do '../lxc-pve-poststop-hook';
+ die $@ if $@;
+ });
+ is($err, '', 'hook succeeds') or diag($task_log);
+ unlike($task_log, qr/WARN/, 'hook does not warn') or diag($task_log);
+
+ like($stored->{uid}, qr/^[a-z2-7]{20}$/, 'other writer assigned a uid');
+ ok(!defined(PVE::LXC::Config->load_config(200)->{description}), 'hook wrote its config');
+ is_deeply(identity_of(200), $stored, 'hook keeps the identity');
+
+ $lxc_module->unmock('get_staging_tempfs', 'vm_stop_cleanup');
+};
+
remove_tree($WORKDIR);
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH container v3 23/38] api: migrate, clone: refuse targets that would drop the guest identity
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (21 preceding siblings ...)
2026-11-18 4:37 ` [PATCH container v3 22/38] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 24/38] restore: take over the guest uid from the backup where possible Thomas Lamprecht
` (16 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Older pve-container versions drop the 'meta' property when they write a
config, and with it the guest identity. A later write by an upgraded
node then assigns a new identity.
Refuse restart and offline migrations and clones to another node unless
the target reports a version that keeps the identity. Check before the
operation starts, and report such nodes with the versions involved in
the migration preconditions, so that clients can explain the refusal.
Also return the nodes that keep the identity for the clone feature, so
that the web interface can validate the target of a clone. That list
does not consider the storages of the container.
The migration API also checks requests that come through HA, but an HA
relocation stops the container before it calls that API. The recovery
of guests from a fenced node moves configs without this check.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* describe the refusal reason per node, on top of the independent schema
fix, and validate complete responses in the test
* feature API: no version lookups and no 'nodes' for 'snapshot' requests
* description of 'nodes'
changes v1 -> v2:
* return the nodes that keep the identity for the clone feature, which
the clone dialog uses
* use the shared schema for the precondition result
src/PVE/API2/LXC.pm | 40 ++++++--
src/test/lifecycle_tests.pl | 195 ++++++++++++++++++++++++++++++++++++
2 files changed, 228 insertions(+), 7 deletions(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 753988c..40ec0f4 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -16,6 +16,7 @@ use PVE::Exception qw(raise raise_param_exc raise_perm_exc);
use PVE::Firewall;
use PVE::GuestHelpers;
use PVE::GuestID;
+use PVE::GuestIdentity;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
use PVE::RESTHandler;
@@ -1579,6 +1580,8 @@ __PACKAGE__->register_method({
},
},
},
+ 'guest-identity-unsupported' =>
+ PVE::GuestIdentity::node_identity_unsupported_schema(),
},
},
description => "Not allowed nodes, keyed by node name, with the reasons.",
@@ -1642,6 +1645,12 @@ __PACKAGE__->register_method({
$blocking_ha_resources;
}
+ # the migration refuses nodes that would drop the guest identity
+ if (my $unsupported = PVE::GuestIdentity::node_identity_unsupported($node)) {
+ $res->{'not-allowed-nodes'}->{$node}->{'guest-identity-unsupported'} =
+ $unsupported;
+ }
+
# if nothing came up, add it to the allowed nodes
if (!defined($res->{'not-allowed-nodes'}->{$node})) {
push $res->{'allowed-nodes'}->@*, $node;
@@ -1720,6 +1729,8 @@ __PACKAGE__->register_method({
PVE::Cluster::check_cfs_quorum();
PVE::Cluster::check_node_exists($target);
+ # an older node drops the guest identity on its next write of the config
+ PVE::GuestIdentity::assert_node_supports_identity($target);
my $targetip = PVE::Cluster::remote_node_ip($target);
@@ -1819,10 +1830,13 @@ __PACKAGE__->register_method({
type => "object",
properties => {
hasFeature => { type => 'boolean' },
- #nodes => {
- #type => 'array',
- #items => { type => 'string' },
- #}
+ nodes => {
+ description => "Nodes that pass the guest identity check for a clone. The"
+ . " storages of the container are not checked. Not set for 'snapshot'.",
+ type => 'array',
+ items => { type => 'string' },
+ optional => 1,
+ },
},
},
code => sub {
@@ -1844,14 +1858,24 @@ __PACKAGE__->register_method({
$conf = $snap;
}
my $storage_cfg = PVE::Storage::config();
- #Maybe include later
- #my $nodelist = PVE::LXC::shared_nodes($conf, $storage_cfg);
my $hasFeature =
PVE::LXC::Config->has_feature($feature, $conf, $storage_cfg, $snapname);
+ # the web interface polls the snapshot feature, which needs no target nodes
+ return { hasFeature => $hasFeature } if $feature eq 'snapshot';
+
+ # A clone to another node is refused if that node would drop the guest identity. Whether
+ # the storages of the container are available on a node is not checked.
+ my $nodename = PVE::INotify::nodename();
+ my $nodes = [];
+ for my $other (PVE::Cluster::get_nodelist()->@*) {
+ my $unsupported =
+ $other ne $nodename && PVE::GuestIdentity::node_identity_unsupported($other);
+ push $nodes->@*, $other if !$unsupported;
+ }
return {
hasFeature => $hasFeature,
- #nodes => [ keys %$nodelist ],
+ nodes => $nodes,
};
},
});
@@ -2032,6 +2056,8 @@ __PACKAGE__->register_method({
undef $target if $target && ($target eq $localnode || $target eq 'localhost');
PVE::Cluster::check_node_exists($target) if $target;
+ # an older node drops the guest identity on its next write of the config
+ PVE::GuestIdentity::assert_node_supports_identity($target) if $target;
my $storecfg = PVE::Storage::config();
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
index 6652cc8..32898fd 100755
--- a/src/test/lifecycle_tests.pl
+++ b/src/test/lifecycle_tests.pl
@@ -23,6 +23,7 @@ use Test::More;
use PVE::Cluster;
use PVE::GuestHelpers;
+use PVE::GuestIdentity;
use PVE::INotify;
use PVE::JSONSchema;
use PVE::Tools qw(file_get_contents file_set_contents);
@@ -77,9 +78,21 @@ my $storage_cfg = {
path => '/var/lib/vz',
content => { rootdir => 1, images => 1, backup => 1, vztmpl => 1 },
},
+ shared => {
+ type => 'nfs',
+ path => '/mnt/pve/shared',
+ shared => 1,
+ content => { rootdir => 1, images => 1 },
+ },
},
};
+# reported Proxmox VE versions of the other cluster nodes
+my $node_versions = {
+ 'new-node' => PVE::GuestIdentity::MIN_PVE_MANAGER_VERSION,
+ 'old-node' => '9.2.0',
+};
+
# recording used guest IDs needs the cluster filesystem and is not what gets tested here
my $guest_id_module = Test::MockModule->new('PVE::GuestID');
$guest_id_module->mock(register_used_id => sub { return; });
@@ -115,6 +128,19 @@ $cluster_module->mock(
return undef if $noerr;
die "CT $vmid already exists on node '$NODENAME'\n";
},
+ remote_node_ip => sub { return '192.0.2.1' },
+ get_node_pvecfg_version => sub {
+ my ($node) = @_;
+ return $node_versions->{$node};
+ },
+ node_pvecfg_version_at_least => sub {
+ my ($node, @required) = @_;
+ my @version = split(/\./, $node_versions->{$node} // return 0);
+ for my $i (0 .. 2) {
+ return $version[$i] > $required[$i] if $version[$i] != $required[$i];
+ }
+ return 1;
+ },
);
my $api_module = Test::MockModule->new('PVE::API2::LXC');
@@ -846,6 +872,175 @@ subtest 'post-stop hook keeps an identity stored while it ran' => sub {
$lxc_module->unmock('get_staging_tempfs', 'vm_stop_cleanup');
};
+subtest 'migration is refused to nodes that drop the guest identity' => sub {
+ reset_configs();
+ my $raw = existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis');
+ write_raw_config(200, $raw);
+
+ my $migrated_to;
+ my $migrate_module = Test::MockModule->new('PVE::LXC::Migrate');
+ $migrate_module->mock(migrate => sub { $migrated_to = $_[1] });
+ my $ha_config_module = Test::MockModule->new('PVE::HA::Config');
+ $ha_config_module->mock(vm_is_ha_managed => sub { return 0 });
+
+ my $migrate = sub {
+ my ($target, $param) = @_;
+ return run_task(sub {
+ PVE::API2::LXC->migrate_vm({
+ node => $NODENAME,
+ vmid => 200,
+ target => $target,
+ ($param // {})->%*,
+ });
+ });
+ };
+
+ my $err = $migrate->('old-node');
+ like($err, qr/node 'old-node' runs Proxmox VE 9\.2\.0/, 'node too old is refused');
+ $err = $migrate->('unknown-node');
+ like($err, qr/does not report its Proxmox VE version/, 'node without version is refused');
+
+ my $stopped = 0;
+ $lxc_module->mock(
+ check_running => sub { return 1 },
+ vm_stop => sub { $stopped = 1 },
+ );
+ $err = $migrate->('old-node', { restart => 1 });
+ $lxc_module->mock(check_running => sub { return 0 });
+ $lxc_module->unmock('vm_stop');
+ like($err, qr/node 'old-node' runs Proxmox VE 9\.2\.0/, 'restart migration is refused');
+ ok(!$stopped, 'running container is not stopped');
+
+ ok(!defined($migrated_to), 'migration does not start');
+ is(read_raw_config(200), $raw, 'config is unchanged');
+
+ $err = $migrate->('new-node');
+ is($err, '', 'migration to an upgraded node starts');
+ is($migrated_to, 'new-node', 'migration goes to the target');
+};
+
+subtest 'migration preconditions list nodes that drop the guest identity' => sub {
+ reset_configs();
+ my $raw = existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis');
+ write_raw_config(200, $raw);
+
+ my $ha_config_module = Test::MockModule->new('PVE::HA::Config');
+ $ha_config_module->mock(vm_is_ha_managed => sub { return 0 });
+ $cluster_module->mock(
+ get_nodelist => sub { return [$NODENAME, 'new-node', 'old-node', 'unknown-node'] },
+ );
+
+ my $res = PVE::API2::LXC->migrate_vm_precondition({ node => $NODENAME, vmid => 200 });
+ $cluster_module->unmock('get_nodelist');
+
+ my $required = PVE::GuestIdentity::MIN_PVE_MANAGER_VERSION;
+ is_deeply($res->{'allowed-nodes'}, ['new-node'], 'only the upgraded node is allowed');
+ is_deeply(
+ $res->{'not-allowed-nodes'},
+ {
+ 'old-node' => {
+ 'guest-identity-unsupported' => {
+ version => '9.2.0',
+ 'required-version' => $required,
+ },
+ },
+ 'unknown-node' => {
+ 'guest-identity-unsupported' => { 'required-version' => $required },
+ },
+ },
+ 'nodes that drop the identity are not allowed',
+ );
+
+ my $returns = PVE::API2::LXC->map_method_by_name('migrate_vm_precondition')->{returns};
+ eval { PVE::JSONSchema::validate($res, $returns) };
+ is($@, '', 'result matches the schema');
+ my $reasons_schema = {
+ $returns->{properties}->{'not-allowed-nodes'}->{additionalProperties}->%*,
+ additionalProperties => 0,
+ };
+ for my $node ('old-node', 'unknown-node') {
+ my $reasons = $res->{'not-allowed-nodes'}->{$node};
+ eval { PVE::JSONSchema::validate($reasons, $reasons_schema) };
+ is($@, '', "reason for $node is documented in the schema");
+ }
+};
+
+subtest 'clone feature only lists nodes that keep the guest identity' => sub {
+ reset_configs();
+ my $raw = existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis');
+ write_raw_config(200, $raw);
+
+ my $config_module = Test::MockModule->new('PVE::LXC::Config');
+ $config_module->mock(has_feature => sub { return 1 });
+ $cluster_module->mock(
+ get_nodelist => sub { return [$NODENAME, 'new-node', 'old-node', 'unknown-node'] },
+ );
+
+ my $res =
+ PVE::API2::LXC->vm_feature({ node => $NODENAME, vmid => 200, feature => 'clone' });
+ $cluster_module->unmock('get_nodelist');
+
+ # the local node reports no version here, a clone on the same node needs no check
+ is_deeply(
+ $res->{nodes},
+ [$NODENAME, 'new-node'],
+ 'nodes that drop the identity are left out',
+ );
+
+ my $returns = PVE::API2::LXC->map_method_by_name('vm_feature')->{returns};
+ eval { PVE::JSONSchema::validate($res, $returns) };
+ is($@, '', 'result matches the schema');
+
+ my $lookups = 0;
+ my $mocked_at_least = \&PVE::Cluster::node_pvecfg_version_at_least;
+ my $mocked_version = \&PVE::Cluster::get_node_pvecfg_version;
+ $cluster_module->mock(
+ get_nodelist => sub { return [$NODENAME, 'new-node', 'old-node', 'unknown-node'] },
+ node_pvecfg_version_at_least => sub { $lookups++; return $mocked_at_least->(@_) },
+ get_node_pvecfg_version => sub { $lookups++; return $mocked_version->(@_) },
+ );
+ $res =
+ PVE::API2::LXC->vm_feature({ node => $NODENAME, vmid => 200, feature => 'snapshot' });
+ $cluster_module->mock(
+ node_pvecfg_version_at_least => $mocked_at_least,
+ get_node_pvecfg_version => $mocked_version,
+ );
+ $cluster_module->unmock('get_nodelist');
+ is($lookups, 0, 'snapshot feature: no node versions are looked up');
+ ok(!exists($res->{nodes}), 'snapshot feature: no nodes are returned');
+ eval { PVE::JSONSchema::validate($res, $returns) };
+ is($@, '', 'snapshot feature: result matches the schema');
+};
+
+subtest 'clone to another node is refused if the node drops the guest identity' => sub {
+ reset_configs();
+ my $raw = <<~'EOF';
+ arch: amd64
+ hostname: existing
+ memory: 512
+ meta: ctime=1500000000,rtime=1500000000000,uid=existingexistingexis
+ ostype: debian
+ rootfs: shared:200/vm-200-disk-0.raw,size=4G
+ unprivileged: 1
+ EOF
+ write_raw_config(200, $raw);
+ make_path(cfs_file("nodes/new-node/lxc"));
+
+ my $err = clone_ct(200, 201, { target => 'old-node' });
+ like($err, qr/node 'old-node' runs Proxmox VE 9\.2\.0/, 'clone is refused');
+ is(read_raw_config(200), $raw, 'source is unchanged');
+ ok(!defined(read_raw_config(201)), 'no clone is created');
+
+ $err = clone_ct(200, 201, { target => 'new-node' });
+ is($err, '', 'clone to an upgraded node succeeds') or diag($task_log);
+ my $clone_raw = read_raw_config(201, 'new-node');
+ ok(defined($clone_raw), 'clone is moved to the target node');
+ my $identity =
+ PVE::GuestIdentity::parse_identity($clone_raw =~ m/^meta: (.*)$/m ? $1 : undef);
+ like($identity->{uid}, qr/^[a-z2-7]{20}$/, 'clone has a uid');
+ isnt($identity->{uid}, 'existingexistingexis', 'clone has a new uid');
+};
+
remove_tree($WORKDIR);
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH container v3 24/38] restore: take over the guest uid from the backup where possible
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (22 preceding siblings ...)
2026-11-18 4:37 ` [PATCH container v3 23/38] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 25/38] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
` (15 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A restore to an unused VMID, or over a container without an identity,
should keep the uid of the backup if no other guest in the cluster uses
it. Take over that uid after the data is restored and the final config
is written, while holding the config lock. With 'unique', or if the uid
is in use, the container keeps the identity assigned when its VMID was
reserved. In all these cases the registration time is the time of the
restore. A container that already had an identity keeps both its uid
and its registration time.
Taking over the uid only after the restore succeeded means that a
failed restore does not take the uid for a container it leaves behind.
If the cluster-wide lock cannot be taken, warn and do not fail the
completed restore; the container keeps its own identity. Report the
outcome in the task log.
Record the uid of the backup as a pending change of 'meta' until the
restore finishes, if the restore could take it over. A retry over a
container left behind by an unfinished restore can then take the uid
over, if no other guest uses it. Record it with 'unique' too, which only
keeps the current restore from taking the uid. If the cluster-wide lock
cannot be taken, the pending uid stays for a later attempt.
A pending uid does not count as in use, so a retry at another VMID
stays possible. Outside of a restore, applying pending changes drops
it.
For restores from a file, PBS or a backup provider, record the pending
uid as soon as the config of the backup is known and keep it in the
configs that the restore writes afterwards. For standard input, the uid
is only known from the restored config, as API parameters cannot set
'meta'. That config carries the pending uid, also if the conversion to
a template writes it first. Archives without a PVE config have no uid.
The extraction opens the archive again and merges the config in it. If
that config has another uid than the one read before, the archive got
replaced in between, so fail the restore instead of taking over a uid
that does not belong to the restored data.
Decide whether a failed restore removes the config by whether the
reservation found one, not by the check before it. A container created
in the meantime is then kept, and the reservation of a VMID whose
container got removed in the meantime is removed again. The Sys.Modify
check for a new privileged container follows the reservation for the
same reason.
A failed create or restore that removes a firewall config that existed
before it, for example one written by hand, logs a warning about that.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* the cleanup of a failed restore and the Sys.Modify check for a new
privileged container follow what the reservation found, not the check
before it
* fail the restore if the uid of the extracted config differs from the
one checked before, as the archive got opened twice
* warn if the cleanup of a failed create or restore removes a firewall
config that existed for the free VMID before
* the lookup double of the tests reads a bare 'meta:' line like pmxcfs
does
changes v1 -> v2:
* use the shared helper of pve-guest-common for the take-over
* describe the effect of 'unique'
* test the order of the cleanup after a failed restore
src/PVE/API2/LXC.pm | 71 +++-
src/test/lifecycle_tests.pl | 653 +++++++++++++++++++++++++++++++++++-
2 files changed, 714 insertions(+), 10 deletions(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 40ec0f4..d231d24 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -19,6 +19,7 @@ use PVE::GuestID;
use PVE::GuestIdentity;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
+use PVE::RESTEnvironment;
use PVE::RESTHandler;
use PVE::RPCEnvironment;
use PVE::RRD;
@@ -63,6 +64,25 @@ my sub assert_not_restore_from_external {
die "refusing to restore privileged container backup from external source\n";
}
+# Returns the uid of the backup if the restore could take it over at its end, see
+# PVE::GuestIdentity::adoptable_uid. Until then, the restore keeps it as pending uid in every config
+# it writes, so that a container left behind by a failure or an abort still has it for a retry.
+# That holds with 'unique' too, which only keeps the restore itself from taking the uid over.
+my sub restore_pending_uid {
+ my ($reservation, $backup_identity) = @_;
+
+ my $uid = $backup_identity ? $backup_identity->{uid} : undef;
+
+ return PVE::GuestIdentity::adoptable_uid($reservation, $uid);
+}
+
+# Returns whether a firewall config exists for $vmid, in a function of its own for tests to mock.
+sub __vmfw_conf_exists {
+ my ($vmid) = @_;
+
+ return -e "/etc/pve/firewall/$vmid.fw";
+}
+
my $check_storage_access_migrate = sub {
my ($rpcenv, $authuser, $storecfg, $storage, $node) = @_;
@@ -199,7 +219,8 @@ __PACKAGE__->register_method({
unique => {
optional => 1,
type => 'boolean',
- description => "Assign a unique random ethernet address.",
+ description => "Assign a unique random ethernet address and do not take over"
+ . " the guest uid of the backup.",
requires => 'restore',
},
pool => {
@@ -433,12 +454,14 @@ __PACKAGE__->register_method({
my $emsg = $restore ? "unable to restore CT $vmid -" : "unable to create CT $vmid -";
# only overwrite a container that the checks above were done for
- eval {
+ my $reservation = eval {
PVE::LXC::Config->create_and_lock_config($vmid, $force && $same_container_exists);
};
die "$emsg $@" if $@;
- my $destroy_config_on_error = !$same_container_exists;
+ my $destroy_config_on_error = !$reservation->{existed};
+ # the cleanup of a failure removes the firewall config, also one that existed before
+ my $vmfw_conf_existed = !$reservation->{existed} && __vmfw_conf_exists($vmid);
my $code = sub {
my $old_conf = PVE::LXC::Config->load_config($vmid);
@@ -446,6 +469,7 @@ __PACKAGE__->register_method({
PVE::LXC::Config->check_protection($old_conf, "unable to restore CT $vmid")
if $restore;
my $was_template;
+ my ($backup_identity, $pending_uid);
my $vollist = [];
eval {
@@ -457,6 +481,10 @@ __PACKAGE__->register_method({
print "recovering backed-up configuration from '$archive'\n";
($orig_conf, $orig_mp_param) =
PVE::LXC::Create::recover_config($storage_cfg, $archive, $vmid);
+ $backup_identity = PVE::LXC::Config->get_identity($orig_conf);
+ $pending_uid = restore_pending_uid($reservation, $backup_identity);
+ PVE::LXC::Config->record_pending_uid($vmid, $pending_uid)
+ if defined($pending_uid);
for my $opt (keys %$orig_conf) {
# early check before disks are created
@@ -484,7 +512,7 @@ __PACKAGE__->register_method({
# implicit privileged change, or creating a new privileged container is checked here
if (
- (!$same_container_exists || $old_conf->{unprivileged})
+ (!$reservation->{existed} || $old_conf->{unprivileged})
&& !$conf->{unprivileged}
) {
$rpcenv->check($authuser, '/', ['Sys.Modify']);
@@ -554,15 +582,14 @@ __PACKAGE__->register_method({
# we always have the 'create' lock so check for more than 1 entry
if (scalar(keys %$old_conf) > 1) {
+ my $replacement_conf = { lock => 'create' };
+ PVE::GuestIdentity::set_pending_uid($replacement_conf, $pending_uid);
# destroy old container volumes
PVE::LXC::destroy_lxc_container(
- $storage_cfg,
- $vmid,
- $old_conf,
- { lock => 'create' },
+ $storage_cfg, $vmid, $old_conf, $replacement_conf,
);
}
- # up until here the old container is intact, besides its lock
+ # up until here the old container is intact, besides its lock and pending uid
$destroy_config_on_error = 1;
eval {
@@ -603,6 +630,18 @@ __PACKAGE__->register_method({
$unique,
$skip_fw_config_restore,
);
+ # the API parameters cannot carry 'meta', so it comes from the backup
+ if ($archive eq '-') {
+ $backup_identity = PVE::LXC::Config->get_identity($conf);
+ $pending_uid = restore_pending_uid($reservation, $backup_identity);
+ } else {
+ # the extraction opened the archive again, it must be the checked one
+ my ($checked, $extracted) = map { $_ ? $_->{uid} : 'none' }
+ ($backup_identity, PVE::LXC::Config->get_identity($conf));
+ die "backup changed during the restore: its uid '$extracted'"
+ . " differs from uid '$checked' read before\n"
+ if $extracted ne $checked;
+ }
PVE::LXC::create_ifaces_ipams_ips($conf, $vmid) if $unique;
my $lxc_setup = PVE::LXC::Setup->new($conf, $rootdir);
$lxc_setup->post_restore_hook($password, $ssh_keys);
@@ -626,6 +665,8 @@ __PACKAGE__->register_method({
foreach my $mp (keys %$delayed_mp_param) {
$conf->{$mp} = $delayed_mp_param->{$mp};
}
+ # the template conversion writes the restored config, too
+ PVE::GuestIdentity::set_pending_uid($conf, $pending_uid);
# If the template flag was set, we try to convert again to template after restore
if ($was_template) {
print STDERR "Convert restored container to template...\n";
@@ -644,6 +685,12 @@ __PACKAGE__->register_method({
if (!$skip_fw_config_restore) { # Only if user has permission to change the fw
eval { PVE::Firewall::remove_vmfw_conf($vmid) };
warn $@ if $@;
+ if ($vmfw_conf_existed && !__vmfw_conf_exists($vmid)) {
+ my $what = $restore ? 'restore' : 'create';
+ my $msg = "removed the firewall config of CT $vmid that existed"
+ . " before the failed $what";
+ PVE::RESTEnvironment::log_warn($msg);
+ }
}
eval { PVE::LXC::Config->destroy_config($vmid) };
@@ -651,6 +698,12 @@ __PACKAGE__->register_method({
}
die "$emsg $err";
}
+ if ($restore) {
+ my $uid = $backup_identity ? $backup_identity->{uid} : undef;
+ PVE::GuestIdentity::adopt_from_backup(
+ 'PVE::LXC::Config', $vmid, $unique ? undef : $uid, $reservation,
+ );
+ }
PVE::AccessControl::add_vm_to_pool($vmid, $pool) if $pool;
};
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
index 32898fd..93d69b1 100755
--- a/src/test/lifecycle_tests.pl
+++ b/src/test/lifecycle_tests.pl
@@ -15,6 +15,7 @@ BEGIN {
$INC{'PVE/VZDump.pm'} = __FILE__;
}
+use Fcntl qw(:flock);
use File::Path qw(make_path remove_tree);
use JSON;
use Socket qw(AF_UNIX SOCK_STREAM PF_UNSPEC);
@@ -58,16 +59,25 @@ my sub write_raw_config($vmid, $raw, $node = undef) {
file_set_contents($file, $raw);
}
+my sub write_raw_vm_config($vmid, $raw) {
+ my $dir = cfs_file("nodes/$NODENAME/qemu-server");
+ make_path($dir);
+ file_set_contents("$dir/$vmid.conf", $raw);
+}
+
my sub read_raw_config($vmid, $node = undef) {
my $file = cfs_file(ct_config_path($vmid, $node));
return -f $file ? file_get_contents($file) : undef;
}
+my $written = {}; # VMID => the written raw configs, in the order of the writes
+
# VMIDs whose container config still existed when their firewall config got removed
my $vmfw_removed_reserved = {};
my sub reset_configs {
remove_tree("$WORKDIR/cfs", "$WORKDIR/rootfs");
+ $written = {};
$vmfw_removed_reserved = {};
}
@@ -87,6 +97,9 @@ my $storage_cfg = {
},
};
+# error of the cluster-wide guest identity lock, if set
+my $domain_lock_error;
+
# reported Proxmox VE versions of the other cluster nodes
my $node_versions = {
'new-node' => PVE::GuestIdentity::MIN_PVE_MANAGER_VERSION,
@@ -121,6 +134,7 @@ $cluster_module->mock(
my $file = cfs_file($path);
make_path($file =~ s|/[^/]+$||r);
file_set_contents($file, $raw);
+ push $written->{ ($path =~ m|/(\d+)\.conf$|)[0] }->@*, $raw;
},
check_vmid_unused => sub {
my ($vmid, $noerr) = @_;
@@ -128,6 +142,32 @@ $cluster_module->mock(
return undef if $noerr;
die "CT $vmid already exists on node '$NODENAME'\n";
},
+ cfs_lock_domain => sub {
+ my ($domain, $timeout, $code, @param) = @_;
+ die "unexpected lock domain '$domain'\n" if $domain ne 'guest-identity';
+ if (defined($domain_lock_error)) {
+ $@ = $domain_lock_error;
+ return undef;
+ }
+ my $res = eval { $code->(@param) };
+ return $res;
+ },
+ get_guest_config_properties => sub {
+ my ($properties) = @_;
+ my $res = {};
+ for my $file (glob("$WORKDIR/cfs/nodes/*/{lxc,qemu-server}/*.conf")) {
+ my ($vmid) = $file =~ m|/(\d+)\.conf$|;
+ for my $line (split(/\n/, file_get_contents($file))) {
+ last if $line =~ m/^\[/;
+ for my $property (@$properties) {
+ # like pmxcfs, a line with nothing after the colon does not count
+ $res->{$vmid}->{$property} = $1 =~ s/^\s+|\s+$//gr
+ if $line =~ m/^\Q$property\E:(.+)$/;
+ }
+ }
+ }
+ return $res;
+ },
remote_node_ip => sub { return '192.0.2.1' },
get_node_pvecfg_version => sub {
my ($node) = @_;
@@ -150,6 +190,8 @@ $api_module->mock(
die "unexpected read of '$path'\n" if $path ne 'storage.cfg';
return $storage_cfg;
},
+ # no guest has a firewall config, unless a test says otherwise
+ __vmfw_conf_exists => sub { return 0 },
);
my $config_module = Test::MockModule->new('PVE::LXC::Config');
@@ -201,7 +243,12 @@ package PVE::LXC::Test::RPCEnvironment {
}
sub get_user { return $_[0]->{user} }
- sub check { return 1 }
+
+ sub check($self, $user, $path, $privs, $noerr = undef) {
+ return 1 if !$self->{deny}->{$path};
+ return 0 if $noerr;
+ die "permission check failed ($path, " . join(', ', @$privs) . ")\n";
+ }
sub check_vm_perm { return 1 }
sub check_pool_exist { return 1 }
@@ -250,6 +297,7 @@ my sub run_task($code) {
# Fake storage and container runtime
my $backup_raw_config;
+my $backup_read_error;
my $storage_module = Test::MockModule->new('PVE::Storage');
$storage_module->mock(
@@ -315,6 +363,7 @@ $create_module->mock(
archive_is_oci_format => sub { return 0 },
recover_config => sub {
my ($storecfg, $volid, $vmid) = @_;
+ die $backup_read_error if defined($backup_read_error);
my $conf = PVE::LXC::Config::parse_pct_config("/lxc/$vmid.conf", $backup_raw_config);
delete $conf->{snapshots};
my $mp_param = {};
@@ -1041,6 +1090,608 @@ subtest 'clone to another node is refused if the node drops the guest identity'
isnt($identity->{uid}, 'existingexistingexis', 'clone has a new uid');
};
+subtest 'restore takes over the guest identity with the config lock held' => sub {
+ reset_configs();
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+
+ my $real_adopt = \&PVE::GuestIdentity::adopt;
+ my $locked;
+ $identity_module->mock(
+ adopt => sub {
+ my ($class, $vmid) = @_;
+ open(my $fh, '>>', "$WORKDIR/lock/pve-config-$vmid.lock")
+ or die "open failed - $!\n";
+ $locked = !flock($fh, LOCK_EX | LOCK_NB);
+ close($fh);
+ return $real_adopt->(@_);
+ },
+ );
+ my $err = restore_ct(200);
+ $identity_module->unmock('adopt');
+
+ is($err, '', 'restore succeeds') or diag($task_log);
+ ok($locked, 'config lock is held while taking over the identity');
+ is(identity_of(200)->{uid}, 'backupbackupbackupba', 'uid of the backup is taken over');
+};
+
+subtest 'restore takes over the guest identity of the backup' => sub {
+ my $backup_meta = 'ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba';
+ my $existing_meta = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis';
+ $now_ms = 1_700_000_000_000;
+
+ my $in_use_by_ct =
+ "guest identity: backup uid backupbackupbackupba is in use by guest 150, keeping uid ";
+
+ my $run = sub {
+ my ($name, $setup, $param, $expected_uid, $expected_rtime, $expected_log) = @_;
+
+ reset_configs();
+ $setup->();
+ my $err = restore_ct(200, $param);
+ is($err, '', "$name: restore succeeds") or diag($task_log);
+ my $identity = identity_of(200);
+ if (defined($expected_uid)) {
+ is($identity->{uid}, $expected_uid, "$name: uid");
+ } else {
+ like($identity->{uid}, qr/^[a-z2-7]{20}$/, "$name: uid is assigned");
+ isnt(
+ $identity->{uid},
+ 'backupbackupbackupba',
+ "$name: uid is not the one of the backup",
+ );
+ }
+ is($identity->{rtime}, $expected_rtime, "$name: rtime");
+ like($task_log, $expected_log, "$name: result is logged");
+ };
+
+ $backup_raw_config = backup_config($backup_meta);
+ $run->(
+ 'free VMID',
+ sub { },
+ {},
+ 'backupbackupbackupba',
+ $now_ms,
+ qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m,
+ );
+ $run->(
+ 'free VMID, uid used by another container',
+ sub { write_raw_config(150, existing_config('uid=backupbackupbackupba')) },
+ {},
+ undef,
+ $now_ms,
+ qr/^\Q$in_use_by_ct\E[a-z2-7]{20}$/m,
+ );
+ $run->(
+ 'free VMID, uid used by a VM',
+ sub { write_raw_vm_config(151, "memory: 512\nmeta: uid=backupbackupbackupba\n") },
+ {},
+ undef,
+ $now_ms,
+ qr/^guest identity: backup uid backupbackupbackupba is in use by guest 151, keeping uid /m,
+ );
+ $run->(
+ 'free VMID, unique',
+ sub { },
+ { unique => 1 },
+ undef,
+ $now_ms,
+ qr/^guest identity: new uid [a-z2-7]{20}$/m,
+ );
+ $run->(
+ 'in place, target without identity',
+ sub { write_raw_config(200, existing_config('ctime=1500000000')) },
+ { force => 1 },
+ 'backupbackupbackupba',
+ $now_ms,
+ qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m,
+ );
+ $run->(
+ 'in place, target without identity, unique',
+ sub { write_raw_config(200, existing_config('ctime=1500000000')) },
+ { force => 1, unique => 1 },
+ undef,
+ $now_ms,
+ qr/^guest identity: new uid [a-z2-7]{20}$/m,
+ );
+ $run->(
+ 'in place, target with the same identity',
+ sub {
+ write_raw_config(
+ 200,
+ existing_config(
+ 'ctime=1500000000,rtime=1500000000000,uid=backupbackupbackupba'),
+ );
+ },
+ { force => 1 },
+ 'backupbackupbackupba',
+ 1500000000000,
+ qr/^guest identity: kept uid backupbackupbackupba$/m,
+ );
+ $run->(
+ 'in place, target with the same identity, unique',
+ sub {
+ write_raw_config(
+ 200,
+ existing_config(
+ 'ctime=1500000000,rtime=1500000000000,uid=backupbackupbackupba'),
+ );
+ },
+ { force => 1, unique => 1 },
+ 'backupbackupbackupba',
+ 1500000000000,
+ qr/^guest identity: kept uid backupbackupbackupba$/m,
+ );
+
+ $backup_raw_config = backup_config('ctime=1400000000');
+ $run->(
+ 'free VMID, backup without identity',
+ sub { },
+ {},
+ undef,
+ $now_ms,
+ qr/^guest identity: new uid [a-z2-7]{20}$/m,
+ );
+ $run->(
+ 'in place, target with identity, backup without identity',
+ sub { write_raw_config(200, existing_config($existing_meta)) },
+ { force => 1 },
+ 'existingexistingexis',
+ 1500000000000,
+ qr/^guest identity: kept uid existingexistingexis$/m,
+ );
+
+ $backup_raw_config = backup_config($backup_meta);
+ $run->(
+ 'free VMID, from STDIN',
+ sub { },
+ { ostemplate => '-', rootfs => 'local:4' },
+ 'backupbackupbackupba',
+ $now_ms,
+ qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m,
+ );
+ $backup_raw_config = undef;
+ $run->(
+ 'free VMID, from STDIN without config',
+ sub { },
+ { ostemplate => '-', rootfs => 'local:4' },
+ undef,
+ $now_ms,
+ qr/^guest identity: new uid [a-z2-7]{20}$/m,
+ );
+};
+
+subtest 'restore keeps the new guest identity if the identity lock fails' => sub {
+ reset_configs();
+ $now_ms = 1_700_000_000_000;
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+ $domain_lock_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+
+ my $err = restore_ct(200);
+ $domain_lock_error = undef;
+ is($err, '', 'restore succeeds') or diag($task_log);
+ my $identity = identity_of(200);
+ like($identity->{uid}, qr/^[a-z2-7]{20}$/, 'uid is assigned');
+ isnt($identity->{uid}, 'backupbackupbackupba', 'uid of the backup is not taken over');
+ my $warning =
+ "WARN: guest identity: unable to adopt backup uid backupbackupbackupba, keeping"
+ . " uid $identity->{uid} - cfs-lock 'domain-guest-identity' error: no quorum!";
+ like($task_log, qr/^\Q$warning\E$/m, 'failure is logged as warning');
+};
+
+my sub pending_uid_of($vmid) {
+ return PVE::GuestIdentity::get_pending_uid(PVE::LXC::Config->load_config($vmid));
+}
+
+subtest 'failed restore keeps the uid of the backup only as pending uid' => sub {
+ reset_configs();
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+
+ my $fake_setup_module = Test::MockModule->new('PVE::LXC::Test::Setup', no_auto => 1);
+ $fake_setup_module->mock(post_restore_hook => sub { die "post-restore hook failed\n" });
+
+ my $err = restore_ct(200);
+ like($err, qr/post-restore hook failed/, 'restore fails');
+ ok(!defined(read_raw_config(200)), 'config of the free VMID is removed');
+ ok(
+ $vmfw_removed_reserved->{200},
+ 'its firewall config was removed while the VMID was still reserved',
+ );
+
+ $fake_setup_module->unmock('post_restore_hook');
+ $err = restore_ct(200);
+ is($err, '', 'retry restores to a free VMID again') or diag($task_log);
+ like(
+ $task_log,
+ qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m,
+ 'retry takes over the uid of the backup',
+ );
+ PVE::LXC::Config->destroy_config(200);
+
+ my $existing_meta = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis';
+ write_raw_config(200, existing_config($existing_meta));
+ $backup_read_error = "unable to read backup\n";
+ $err = restore_ct(200, { force => 1 });
+ $backup_read_error = undef;
+ like($err, qr/unable to read backup/, 'restore fails before changing the container');
+ my $conf = PVE::LXC::Config->load_config(200);
+ is($conf->{meta}, $existing_meta, 'target keeps its identity');
+ ok(!defined($conf->{lock}), 'create lock is removed');
+
+ # a privileged backup needs Sys.Modify, which is checked after reading the backup
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba') =~
+ s/^unprivileged: 1\n//mr;
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ $rpcenv->{deny}->{'/'} = 1;
+ $err = restore_ct(200, { force => 1 });
+ delete $rpcenv->{deny};
+ like($err, qr/permission check failed/, 'restore fails after reading the backup');
+ my $identity = identity_of(200);
+ like($identity->{uid}, qr/^[a-z2-7]{20}$/, 'target got an identity with the reservation');
+ isnt($identity->{uid}, 'backupbackupbackupba', 'uid of the backup is not taken over');
+ is(pending_uid_of(200), 'backupbackupbackupba', 'target has it as pending uid');
+ ok(!defined(PVE::LXC::Config->load_config(200)->{lock}), 'create lock is removed');
+
+ $err = restore_ct(200, { force => 1 });
+ is($err, '', 'retry of the same restore succeeds') or diag($task_log);
+ like(
+ $task_log,
+ qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m,
+ 'retry takes over the uid of the backup',
+ );
+ is(identity_of(200)->{uid}, 'backupbackupbackupba', 'stored');
+ is(pending_uid_of(200), undef, 'pending uid is removed');
+
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ my $mocked_create_disks = \&PVE::LXC::create_disks;
+ $lxc_module->mock(create_disks => sub { die "unable to allocate volume - storage full\n" });
+ $err = restore_ct(200, { force => 1 });
+ $lxc_module->mock(create_disks => $mocked_create_disks);
+ like($err, qr/storage full/, 'restore fails before replacing the container');
+ is(PVE::LXC::Config->load_config(200)->{hostname}, 'existing', 'container is kept');
+ is(pending_uid_of(200), 'backupbackupbackupba', 'container has the pending uid');
+ $err = restore_ct(200, { force => 1 });
+ is($err, '', 'retry succeeds') or diag($task_log);
+ is(
+ identity_of(200)->{uid},
+ 'backupbackupbackupba',
+ 'retry takes over the uid of the backup',
+ );
+};
+
+subtest 'restore only changes the container that the checks were done for' => sub {
+ my $existing_raw =
+ existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis');
+ my $other_raw =
+ existing_config('ctime=1500000000,rtime=1600000000000,uid=otherotherotherother') =~
+ s/^hostname: existing$/hostname: other/mr;
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+
+ my $conf_of = sub {
+ my ($raw) = @_;
+ my $conf = PVE::LXC::Config::parse_pct_config('/lxc/200.conf', $raw);
+ delete $conf->{digest};
+ return $conf;
+ };
+ my $current_conf = sub {
+ my $raw = read_raw_config(200);
+ return defined($raw) ? $conf_of->($raw) : undef;
+ };
+
+ # runs between the checks of the create API call and its reservation
+ my $in_between;
+ my $mocked_activate = \&PVE::Storage::activate_storage;
+ $storage_module->mock(
+ activate_storage => sub {
+ if (my $code = $in_between) {
+ undef $in_between;
+ $code->();
+ }
+ return $mocked_activate->(@_);
+ },
+ );
+
+ reset_configs();
+ $in_between = sub { write_raw_config(200, $other_raw) };
+ $rpcenv = PVE::LXC::Test::RPCEnvironment->new('user@pve');
+ $rpcenv->{deny}->{'/vms/200'} = 1;
+ my $err = restore_ct(200, { force => 1, pool => 'pool1' });
+ $rpcenv = PVE::LXC::Test::RPCEnvironment->new('root@pam');
+ like($err, qr/CT 200 already exists/, 'created in between, allowed by the pool: refused');
+ is_deeply(
+ $current_conf->(),
+ $conf_of->($other_raw),
+ 'created in between: config is unchanged',
+ );
+
+ reset_configs();
+ write_raw_config(200, $existing_raw);
+ $in_between = sub { write_raw_config(200, $existing_raw . "protection: 1\n") };
+ $err = restore_ct(200, { force => 1 });
+ like(
+ $err,
+ qr/^unable to restore CT 200 - protection mode enabled$/,
+ 'protection set in between: refused',
+ );
+ is_deeply(
+ $current_conf->(),
+ $conf_of->($existing_raw . "protection: 1\n"),
+ 'protection set in between: config is unchanged',
+ );
+
+ # the guest list of the reservation is the one of the start of the API call
+ my $mocked_unused = \&PVE::Cluster::check_vmid_unused;
+ $cluster_module->mock(check_vmid_unused => sub { return 1 });
+ reset_configs();
+ $in_between = sub { write_raw_config(200, $other_raw) };
+ $backup_read_error = "unable to read backup\n";
+ $err = restore_ct(200);
+ $backup_read_error = undef;
+ $cluster_module->mock(check_vmid_unused => $mocked_unused);
+ isnt($err, '', 'created in between, early failure: restore fails');
+ is_deeply($current_conf->(), $conf_of->($other_raw), 'early failure: config is unchanged');
+ ok(!$vmfw_removed_reserved->{200}, 'early failure: firewall config is kept');
+
+ reset_configs();
+ write_raw_config(200, $existing_raw);
+ $in_between = sub { PVE::LXC::Config->destroy_config(200) };
+ $backup_read_error = "unable to read backup\n";
+ $err = restore_ct(200, { force => 1 });
+ $backup_read_error = undef;
+ like($err, qr/unable to read backup/, 'removed in between, early failure: restore fails');
+ ok(!defined(read_raw_config(200)), 'removed in between: the VMID is free again');
+
+ $storage_module->mock(activate_storage => $mocked_activate);
+};
+
+subtest 'failed create or restore warns if it removes an older firewall config' => sub {
+ my $vmfw_exists = {};
+ $api_module->mock(
+ __vmfw_conf_exists => sub {
+ my ($vmid) = @_;
+ return $vmfw_exists->{$vmid};
+ },
+ );
+ my $mocked_remove_vmfw = \&PVE::Firewall::remove_vmfw_conf;
+ $firewall_module->mock(
+ remove_vmfw_conf => sub {
+ my ($vmid) = @_;
+ delete $vmfw_exists->{$vmid};
+ return $mocked_remove_vmfw->(@_);
+ },
+ );
+ my $fake_setup_module = Test::MockModule->new('PVE::LXC::Test::Setup', no_auto => 1);
+ $fake_setup_module->mock(
+ post_create_hook => sub { die "post-create hook failed\n" },
+ post_restore_hook => sub { die "post-restore hook failed\n" },
+ );
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+
+ for my $case (['restore', 0], ['restore', 1], ['create', 1]) {
+ my ($what, $existed) = $case->@*;
+ my $name = "failed $what " . ($existed ? 'with' : 'without') . ' older firewall config';
+
+ reset_configs();
+ $vmfw_exists = $existed ? { 200 => 1 } : {};
+ my $err = $what eq 'restore' ? restore_ct(200) : create_ct(200);
+ like($err, qr/post-$what hook failed/, "$name: fails");
+ ok($vmfw_removed_reserved->{200}, "$name: firewall config is removed");
+ my $warning =
+ "removed the firewall config of CT 200 that existed before the failed $what";
+ if ($existed) {
+ like($task_log, qr/^WARN: \Q$warning\E$/m, "$name: warning is logged");
+ } else {
+ unlike($task_log, qr/existed before/, "$name: no warning");
+ }
+ }
+
+ reset_configs();
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ $vmfw_exists = { 200 => 1 };
+ $backup_read_error = "unable to read backup\n";
+ my $err = restore_ct(200, { force => 1 });
+ $backup_read_error = undef;
+ like($err, qr/unable to read backup/, 'failed restore over a container: fails');
+ ok($vmfw_exists->{200}, 'failed restore over a container: firewall config is kept');
+ unlike($task_log, qr/existed before/, 'failed restore over a container: no warning');
+
+ $firewall_module->mock(remove_vmfw_conf => $mocked_remove_vmfw);
+ $api_module->mock(__vmfw_conf_exists => sub { return 0 });
+};
+
+subtest 'restore fails if the backup changed after its uid was read' => sub {
+ my $mocked_restore_archive = \&PVE::LXC::Create::restore_archive;
+ $create_module->mock(
+ restore_archive => sub {
+ my ($storecfg, $archive, $rootdir) = @_;
+ # the extraction opens the archive again and finds another backup
+ make_path("$rootdir/etc/vzdump");
+ file_set_contents(
+ "$rootdir/etc/vzdump/pct.conf",
+ backup_config('ctime=1300000000,rtime=1300000000000,uid=anotheranotheranothe'),
+ );
+ },
+ );
+ my $changed = "unable to restore CT 200 - backup changed during the restore: its uid"
+ . " 'anotheranotheranothe' differs from uid 'backupbackupbackupba' read before";
+
+ reset_configs();
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+ my $err = restore_ct(200);
+ like($err, qr/^\Q$changed\E$/, 'free VMID: restore fails');
+ ok(!defined(read_raw_config(200)), 'free VMID: nothing is left behind');
+
+ $create_module->mock(restore_archive => $mocked_restore_archive);
+};
+
+# Restores in place over a container without identity, failing after reading the backup.
+my sub failed_restore_in_place {
+ reset_configs();
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba') =~
+ s/^unprivileged: 1\n//mr;
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ $rpcenv->{deny}->{'/'} = 1;
+ my $err = restore_ct(200, { force => 1 });
+ delete $rpcenv->{deny};
+ like($err, qr/permission check failed/, 'restore fails after reading the backup');
+ is(pending_uid_of(200), 'backupbackupbackupba', 'container left behind has the pending uid');
+}
+
+subtest 'retry elsewhere, then over the container left behind' => sub {
+ failed_restore_in_place();
+ my $leftover = identity_of(200)->{uid};
+
+ my $err = restore_ct(201);
+ is($err, '', 'restore to another VMID succeeds') or diag($task_log);
+ is(identity_of(201)->{uid}, 'backupbackupbackupba', 'pending uid is not in use');
+ is(pending_uid_of(200), 'backupbackupbackupba', 'container left behind keeps it');
+
+ $err = restore_ct(200, { force => 1 });
+ is($err, '', 'restore over the container left behind succeeds') or diag($task_log);
+ like(
+ $task_log,
+ qr/: backup uid backupbackupbackupba is in use by guest 201, keeping uid $leftover$/m,
+ 'uid of the backup is in use by now',
+ );
+ is(identity_of(200)->{uid}, $leftover, 'container left behind keeps its uid');
+ is(pending_uid_of(200), undef, 'pending uid is removed');
+};
+
+subtest 'using the container left behind drops its pending uid' => sub {
+ failed_restore_in_place();
+
+ my $err = update_ct(200, { description => 'used' });
+ is($err, '', 'config change of the stopped container') or diag($task_log);
+ my $dropped = "guest identity: dropping pending uid backupbackupbackupba of an unfinished"
+ . " restore - a retry now needs 'allow-identity-mismatch'";
+ like($task_log, qr/^WARN: \Q$dropped\E$/m, 'dropped');
+ is(pending_uid_of(200), undef, 'pending uid is gone');
+};
+
+subtest 'restore with unique keeps a pending uid, but never takes it over' => sub {
+ for my $retry ({ unique => 1 }, {}) {
+ my $what = $retry->{unique} ? 'retry with unique' : 'retry without unique';
+
+ reset_configs();
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba') =~
+ s/^unprivileged: 1\n//mr;
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ $rpcenv->{deny}->{'/'} = 1;
+ my $err = restore_ct(200, { force => 1, unique => 1 });
+ delete $rpcenv->{deny};
+ like(
+ $err,
+ qr/permission check failed/,
+ 'restore with unique fails after reading the backup',
+ );
+ my $leftover = identity_of(200)->{uid};
+ is(
+ pending_uid_of(200),
+ 'backupbackupbackupba',
+ 'container left behind has the pending uid',
+ );
+
+ $err = restore_ct(200, { force => 1, $retry->%* });
+ is($err, '', "$what succeeds") or diag($task_log);
+ is(pending_uid_of(200), undef, "$what: pending uid is removed");
+ is(
+ identity_of(200)->{uid},
+ $retry->{unique} ? $leftover : 'backupbackupbackupba',
+ "$what: "
+ . ($retry->{unique} ? 'keeps its uid' : 'takes over the uid of the backup'),
+ );
+ }
+};
+
+subtest 'failed adoption keeps the pending uid' => sub {
+ reset_configs();
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+ $domain_lock_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+ my $err = restore_ct(200);
+ $domain_lock_error = undef;
+ is($err, '', 'restore succeeds') or diag($task_log);
+ like(
+ $task_log,
+ qr/unable to adopt backup uid backupbackupbackupba, keeping uid /,
+ 'warning',
+ );
+ isnt(identity_of(200)->{uid}, 'backupbackupbackupba', 'container has a uid of its own');
+ is(pending_uid_of(200), 'backupbackupbackupba', 'nothing decided, pending uid stays');
+
+ $err = restore_ct(200, { force => 1 });
+ is($err, '', 'restore of the backup again is not refused') or diag($task_log);
+ is(identity_of(200)->{uid}, 'backupbackupbackupba', 'uid of the backup is taken over');
+};
+
+subtest 'config of a container left behind shows its stored identity' => sub {
+ failed_restore_in_place();
+ my $meta = PVE::LXC::Config->load_config(200)->{meta};
+
+ my $conf = PVE::API2::LXC::Config->vm_config({ node => $NODENAME, vmid => 200 });
+ is($conf->{meta}, $meta, 'config with pending changes applied: stored identity');
+ $conf = PVE::API2::LXC::Config->vm_config({ node => $NODENAME, vmid => 200, current => 1 });
+ is($conf->{meta}, $meta, 'current config: stored identity');
+
+ my $pending = PVE::API2::LXC->vm_pending({ node => $NODENAME, vmid => 200 });
+ my ($item) = grep { $_->{key} eq 'meta' } $pending->@*;
+ is($item->{value}, $meta, 'pending view: current value');
+ is($item->{pending}, 'uid=backupbackupbackupba', 'pending view: pending uid');
+};
+
+# Once a write of the config of CT 200 has the pending uid of the backup, all later writes of the
+# restore have it, so that the container keeps it whenever the restore stops. Only the adoption
+# removes it.
+subtest 'every write of a restore keeps the pending uid' => sub {
+ my $backup_meta = 'ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba';
+ my $cases = [
+ ['free VMID', undef, {}],
+ ['in place over a container without uid', existing_config('ctime=1500000000'), {}],
+ ['free VMID, from STDIN', undef, { ostemplate => '-', rootfs => 'local:4' }],
+ ['free VMID, template', undef, {}, "template: 1\n"],
+ ];
+ $storage_module->mock(volume_has_feature => sub { return 0 });
+
+ for my $case (@$cases) {
+ my ($what, $target, $param, $extra) = @$case;
+
+ reset_configs();
+ $backup_raw_config = backup_config($backup_meta) . ($extra // '');
+ write_raw_config(200, $target) if $target;
+ my $err = restore_ct(200, { $target ? (force => 1) : (), $param->%* });
+ is($err, '', "$what: restore succeeds") or diag($task_log);
+ like(
+ $task_log,
+ qr/^guest identity: adopted uid backupbackupbackupba/m,
+ "$what: adopted",
+ );
+
+ my @writes = $written->{200}->@*;
+ my $last = pop(@writes);
+ my $first = 0;
+ $first++ while $first < @writes && $writes[$first] !~ m/^\[pve:pending\]/m;
+ ok($first < @writes, "$what: pending uid is written") or next;
+ my @later = @writes[$first .. $#writes];
+ is(
+ scalar(grep { m/^\[pve:pending\]\nmeta: uid=backupbackupbackupba$/m } @later),
+ scalar(@later),
+ "$what: all later writes keep it",
+ );
+ unlike($last, qr/^\[pve:pending\]/m, "$what: adoption removes it");
+ }
+ $storage_module->unmock('volume_has_feature');
+};
+
remove_tree($WORKDIR);
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH container v3 25/38] restore: refuse overwriting a container with another guest's backup
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (23 preceding siblings ...)
2026-11-18 4:37 ` [PATCH container v3 24/38] restore: take over the guest uid from the backup where possible Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 26/38] test: remote migration: check the cleanup of the target after failures Thomas Lamprecht
` (14 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Restoring the backup of another guest over an existing container
results in the data of the backup under the identity of the target.
Tools that follow that identity would take the result for the original
guest.
Refuse a restore if the uids differ, unless 'allow-identity-mismatch'
is set through the create API or 'pct restore'. The existing 'force'
option permits overwriting a container but does not override this
check. With the override, a container that already had an identity
keeps it.
A container that has the uid of the backup as pending uid was left
behind by an unfinished restore of that backup, so a retry may proceed.
Without 'unique', the retry takes that uid over if no other guest uses
it.
For standard input, the config of the backup is only known after the
old volumes were destroyed and the archive got extracted. So require
the override whenever the target already has an identity. A target
without identity can still be overwritten without it.
Compare the uids before the restore destroys anything. Apply the
network options only once the old container is removed. Applying them
can allocate addresses in the IPAM of SDN for their MAC addresses. The
cleanup after an earlier failure would remove such a mapping even if
the existing container still uses that MAC address, and removing the
old container removes the mappings of its MAC addresses, also of one
that an option keeps. Validate and queue the options before the worker
starts, without changing the IPAM state.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* apply the network options after the old container is removed, so that
removing it does not drop the IPAM mapping of a restored interface
with the same MAC address
* description of 'allow-identity-mismatch': any backup with the pending
uid counts
changes v1 -> v2:
* use the shared check of pve-guest-common
* 'Allow overwriting' instead of 'Allow to overwrite' and a default for
'allow-identity-mismatch' (Dominik)
src/PVE/API2/LXC.pm | 56 +++++++++-
src/test/lifecycle_tests.pl | 198 ++++++++++++++++++++++++++++++++++++
2 files changed, 253 insertions(+), 1 deletion(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index d231d24..a5e45c2 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -223,6 +223,18 @@ __PACKAGE__->register_method({
. " the guest uid of the backup.",
requires => 'restore',
},
+ 'allow-identity-mismatch' => {
+ optional => 1,
+ type => 'boolean',
+ default => 0,
+ description => "Allow overwriting an existing container with a backup of"
+ . " another guest, as detected by a different 'uid' in 'meta'. A restore from"
+ . " standard input over a container that has a uid always needs this option,"
+ . " as the backup cannot be checked beforehand. Unless the container is left"
+ . " over from an unfinished restore of a backup with the same uid, it keeps its"
+ . " own uid.",
+ requires => 'restore',
+ },
pool => {
optional => 1,
type => 'string',
@@ -285,6 +297,7 @@ __PACKAGE__->register_method({
my $unprivileged = extract_param($param, 'unprivileged');
my $restore = extract_param($param, 'restore');
my $unique = extract_param($param, 'unique');
+ my $allow_identity_mismatch = extract_param($param, 'allow-identity-mismatch');
$param->{cpuunits} = PVE::CGroup::clamp_cpu_shares($param->{cpuunits})
if defined($param->{cpuunits}); # clamp value depending on cgroup version
@@ -446,7 +459,13 @@ __PACKAGE__->register_method({
# check/activate default storage
&$check_and_activate_storage($storage) if !defined($mp_param->{rootfs});
- PVE::LXC::Config->update_pct_config($vmid, $conf, 0, $no_disk_param);
+ if ($restore) {
+ # applying network options can allocate addresses in the IPAM of SDN, so only check and
+ # queue the options here, and apply them once the restore may overwrite the container
+ PVE::LXC::Config->queue_pct_config_changes($vmid, $conf, $no_disk_param, [], []);
+ } else {
+ PVE::LXC::Config->update_pct_config($vmid, $conf, 0, $no_disk_param);
+ }
$conf->{unprivileged} = 1 if $unprivileged;
$conf->{meta} = PVE::LXC::Config->new_meta_info_string() if !$restore;
@@ -463,6 +482,32 @@ __PACKAGE__->register_method({
# the cleanup of a failure removes the firewall config, also one that existed before
my $vmfw_conf_existed = !$reservation->{existed} && __vmfw_conf_exists($vmid);
+ # Restoring over an existing container must not turn it into another guest. The identity of
+ # a backup from STDIN is only known once the container got overwritten. A container with the
+ # uid of the backup as pending uid is a leftover of an earlier restore of it.
+ my $assert_identity_matches = sub {
+ my ($backup_identity, $from_stdin) = @_;
+
+ my $target = $reservation->{previous} or return;
+
+ if ($from_stdin) {
+ if (!$allow_identity_mismatch) {
+ die "cannot verify the guest identity of a backup from standard input"
+ . " before overwriting CT $vmid (uid '$target->{uid}') - use"
+ . " 'allow-identity-mismatch' to overwrite the CT anyway\n";
+ }
+ return;
+ }
+
+ PVE::GuestIdentity::assert_backup_matches_target(
+ 'PVE::LXC::Config',
+ $vmid,
+ $reservation,
+ $backup_identity ? $backup_identity->{uid} : undef,
+ $allow_identity_mismatch,
+ );
+ };
+
my $code = sub {
my $old_conf = PVE::LXC::Config->load_config($vmid);
# the container may have changed since the protection check above
@@ -482,6 +527,7 @@ __PACKAGE__->register_method({
($orig_conf, $orig_mp_param) =
PVE::LXC::Create::recover_config($storage_cfg, $archive, $vmid);
$backup_identity = PVE::LXC::Config->get_identity($orig_conf);
+ $assert_identity_matches->($backup_identity);
$pending_uid = restore_pending_uid($reservation, $backup_identity);
PVE::LXC::Config->record_pending_uid($vmid, $pending_uid)
if defined($pending_uid);
@@ -517,6 +563,8 @@ __PACKAGE__->register_method({
) {
$rpcenv->check($authuser, '/', ['Sys.Modify']);
}
+ } else {
+ $assert_identity_matches->(undef, 1);
}
}
if ($storage_only_mode) {
@@ -592,6 +640,10 @@ __PACKAGE__->register_method({
# up until here the old container is intact, besides its lock and pending uid
$destroy_config_on_error = 1;
+ # registers addresses in the IPAM of SDN, so only once the old container is gone
+ PVE::LXC::Config->vmconfig_apply_pending($vmid, $conf, $storage_cfg)
+ if $restore;
+
eval {
my $rootdir = PVE::LXC::mount_all($vmid, $storage_cfg, $conf, 1);
$bwlimit = PVE::Storage::get_bandwidth_limit(
@@ -633,6 +685,8 @@ __PACKAGE__->register_method({
# the API parameters cannot carry 'meta', so it comes from the backup
if ($archive eq '-') {
$backup_identity = PVE::LXC::Config->get_identity($conf);
+ # only logs a mismatch, which needed the override already
+ $assert_identity_matches->($backup_identity);
$pending_uid = restore_pending_uid($reservation, $backup_identity);
} else {
# the extraction opened the archive again, it must be the checked one
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
index 93d69b1..7606d3e 100755
--- a/src/test/lifecycle_tests.pl
+++ b/src/test/lifecycle_tests.pl
@@ -29,6 +29,7 @@ use PVE::INotify;
use PVE::JSONSchema;
use PVE::Tools qw(file_get_contents file_set_contents);
+use PVE::CLI::pct;
use PVE::LXC;
use PVE::LXC::Config;
use PVE::LXC::Create;
@@ -298,6 +299,7 @@ my sub run_task($code) {
my $backup_raw_config;
my $backup_read_error;
+my $volumes_touched = 0;
my $storage_module = Test::MockModule->new('PVE::Storage');
$storage_module->mock(
@@ -325,6 +327,7 @@ $lxc_module->mock(
check_running => sub { return 0 },
create_disks => sub {
my ($storecfg, $vmid, $settings, $conf) = @_;
+ $volumes_touched++;
my $vollist = [];
for my $ms (sort keys %$settings) {
my $volid = "local:$vmid/vm-$vmid-disk-$ms.raw";
@@ -336,6 +339,7 @@ $lxc_module->mock(
destroy_disks => sub { },
destroy_lxc_container => sub {
my ($storecfg, $vmid, $conf, $replacement_conf) = @_;
+ $volumes_touched++;
if (defined($replacement_conf)) {
PVE::LXC::Config->write_config($vmid, $replacement_conf);
} else {
@@ -1438,6 +1442,17 @@ subtest 'restore only changes the container that the checks were done for' => su
is_deeply($current_conf->(), $conf_of->($other_raw), 'early failure: config is unchanged');
ok(!$vmfw_removed_reserved->{200}, 'early failure: firewall config is kept');
+ $cluster_module->mock(check_vmid_unused => sub { return 1 });
+ reset_configs();
+ $volumes_touched = 0;
+ $in_between = sub { write_raw_config(200, $other_raw) };
+ $err = restore_ct(200);
+ $cluster_module->mock(check_vmid_unused => $mocked_unused);
+ isnt($err, '', 'created in between, other uid: restore fails');
+ is_deeply($current_conf->(), $conf_of->($other_raw), 'other uid: config is unchanged');
+ ok(!$vmfw_removed_reserved->{200}, 'other uid: firewall config is kept');
+ is($volumes_touched, 0, 'other uid: volumes are not touched');
+
reset_configs();
write_raw_config(200, $existing_raw);
$in_between = sub { PVE::LXC::Config->destroy_config(200) };
@@ -1529,6 +1544,19 @@ subtest 'restore fails if the backup changed after its uid was read' => sub {
like($err, qr/^\Q$changed\E$/, 'free VMID: restore fails');
ok(!defined(read_raw_config(200)), 'free VMID: nothing is left behind');
+ # the checked backup has the uid of the container, so the restore needs no override
+ reset_configs();
+ write_raw_config(
+ 200,
+ existing_config('ctime=1500000000,rtime=1500000000000,uid=backupbackupbackupba'),
+ );
+ $err = restore_ct(200, { force => 1 });
+ like($err, qr/^\Q$changed\E$/, 'in place: restore fails');
+ ok(
+ !defined(read_raw_config(200)),
+ 'in place: the other backup is not left behind with the uid of the container',
+ );
+
$create_module->mock(restore_archive => $mocked_restore_archive);
};
@@ -1575,6 +1603,44 @@ subtest 'using the container left behind drops its pending uid' => sub {
. " restore - a retry now needs 'allow-identity-mismatch'";
like($task_log, qr/^WARN: \Q$dropped\E$/m, 'dropped');
is(pending_uid_of(200), undef, 'pending uid is gone');
+
+ $backup_raw_config =~ s/^arch: amd64$/arch: amd64\nunprivileged: 1/m;
+ $err = restore_ct(200, { force => 1 });
+ like($err, qr/backup belongs to another guest/, 'container stays a guest of its own');
+};
+
+subtest 'retry over the container left behind by a restore of another backup' => sub {
+ failed_restore_in_place();
+
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=anotheranotheranothe');
+ my $err = restore_ct(200, { force => 1 });
+ like(
+ $err,
+ qr/backup belongs to another guest: its uid .anotheranotheranothe. differs/,
+ 'refused',
+ );
+ is(pending_uid_of(200), 'backupbackupbackupba', 'pending uid is unchanged');
+};
+
+subtest 'restore from STDIN over the container left behind' => sub {
+ failed_restore_in_place();
+ my $leftover = identity_of(200)->{uid};
+
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+ my $param = { force => 1, ostemplate => '-', rootfs => 'local:4' };
+ my $err = restore_ct(200, $param);
+ like($err, qr/cannot verify the guest identity of a backup from standard input/, 'refused');
+
+ $err = restore_ct(200, { $param->%*, 'allow-identity-mismatch' => 1 });
+ is($err, '', 'restore with override succeeds') or diag($task_log);
+ is(
+ identity_of(200)->{uid},
+ 'backupbackupbackupba',
+ 'pending uid matches the backup: taken over',
+ );
+ is(pending_uid_of(200), undef, 'pending uid is removed');
};
subtest 'restore with unique keeps a pending uid, but never takes it over' => sub {
@@ -1692,6 +1758,138 @@ subtest 'every write of a restore keeps the pending uid' => sub {
$storage_module->unmock('volume_has_feature');
};
+subtest 'restore over a container of another guest needs an override' => sub {
+ my $existing_meta = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis';
+ my $existing_raw = existing_config($existing_meta);
+ my $existing_conf = PVE::LXC::Config::parse_pct_config('/lxc/200.conf', $existing_raw);
+ delete $existing_conf->{digest};
+ $backup_raw_config =
+ backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba');
+
+ # network options can change the IPAM of SDN, with the same MAC address as the existing one
+ my $net = 'name=eth0,bridge=vnet0,hwaddr=BC:24:11:00:00:01,ip=dhcp';
+ $existing_raw .= "net0: $net,type=veth\n";
+ $existing_conf = PVE::LXC::Config::parse_pct_config('/lxc/200.conf', $existing_raw);
+ delete $existing_conf->{digest};
+ my @sdn_calls;
+ my $ipam = {}; # bridge => MAC => hostname
+ my $vnets_module = Test::MockModule->new('PVE::Network::SDN::Vnets');
+ $vnets_module->mock(
+ add_next_free_cidr => sub {
+ my ($bridge, $hostname, $mac) = @_;
+ push @sdn_calls, 'add_next_free_cidr';
+ $ipam->{$bridge}->{$mac} = $hostname;
+ },
+ del_ips_from_mac => sub {
+ my ($bridge, $mac) = @_;
+ push @sdn_calls, 'del_ips_from_mac';
+ delete $ipam->{$bridge}->{$mac};
+ },
+ );
+ $lxc_module->unmock('delete_ifaces_ipams_ips');
+ my $mocked_destroy = \&PVE::LXC::destroy_lxc_container;
+ $lxc_module->mock(
+ destroy_lxc_container => sub {
+ my ($storecfg, $vmid, $conf) = @_;
+ # like the real function, which removes the addresses of the old container
+ PVE::LXC::delete_ifaces_ipams_ips($conf, $vmid);
+ return $mocked_destroy->(@_);
+ },
+ );
+
+ my $refused = sub {
+ my ($name, $param, $expected_err) = @_;
+
+ reset_configs();
+ write_raw_config(200, $existing_raw);
+ $volumes_touched = 0;
+ @sdn_calls = ();
+ my $err = restore_ct(200, { force => 1, $param->%* });
+ like($err, $expected_err, "$name: restore is refused");
+ is($volumes_touched, 0, "$name: volumes are not touched");
+ is_deeply(\@sdn_calls, [], "$name: IPAM is not touched");
+ my $conf = PVE::LXC::Config->load_config(200);
+ delete $conf->{digest};
+ is_deeply($conf, $existing_conf, "$name: config is unchanged");
+ };
+
+ my $mismatch_err =
+ "unable to restore CT 200 - backup belongs to another guest: its uid 'backupbackupbackupba'"
+ . " differs from uid 'existingexistingexis' of the existing CT 200 - use"
+ . " 'allow-identity-mismatch' to overwrite the CT anyway";
+ my $stdin_err =
+ "unable to restore CT 200 - cannot verify the guest identity of a backup from standard"
+ . " input before overwriting CT 200 (uid 'existingexistingexis') - use"
+ . " 'allow-identity-mismatch' to overwrite the CT anyway";
+ my $ignored = "guest identity: ignoring backup uid backupbackupbackupba, CT 200 keeps uid"
+ . " existingexistingexis";
+ my $stdin = { ostemplate => '-', rootfs => 'local:4' };
+ $refused->('different uid', {}, qr/^\Q$mismatch_err\E$/);
+ $refused->('different uid, unique', { unique => 1 }, qr/^\Q$mismatch_err\E$/);
+ $refused->('different uid, network option', { net0 => $net }, qr/^\Q$mismatch_err\E$/);
+ $refused->('from STDIN', $stdin, qr/^\Q$stdin_err\E$/);
+ $refused->('from STDIN, network option', { $stdin->%*, net0 => $net },
+ qr/^\Q$stdin_err\E$/);
+
+ for my $param ({}, $stdin) {
+ my $name = $param->{ostemplate} ? 'from STDIN' : 'from file';
+ reset_configs();
+ write_raw_config(200, $existing_raw);
+ @sdn_calls = ();
+ $ipam = { vnet0 => { 'BC:24:11:00:00:01' => 'existing' } };
+ my $err = restore_ct(
+ 200,
+ { force => 1, 'allow-identity-mismatch' => 1, net0 => $net, $param->%* },
+ );
+ is($err, '', "$name: restore with override succeeds") or diag($task_log);
+ is_deeply(
+ \@sdn_calls,
+ ['del_ips_from_mac', 'add_next_free_cidr'],
+ "$name: network option is applied once the old container is removed",
+ );
+ ok(
+ exists($ipam->{vnet0}->{'BC:24:11:00:00:01'}),
+ "$name: restored interface has an IPAM mapping",
+ );
+ is(
+ PVE::LXC::Config->load_config(200)->{hostname},
+ 'backup',
+ "$name: backup is restored",
+ );
+ is_deeply(
+ identity_of(200),
+ { uid => 'existingexistingexis', rtime => 1500000000000 },
+ "$name: identity of the target is kept",
+ );
+ like(
+ $task_log, qr/^\Q$ignored\E$/m, "$name: override is logged",
+ );
+ like(
+ $task_log,
+ qr/^guest identity: kept uid existingexistingexis$/m,
+ "$name: result is logged",
+ );
+ }
+
+ reset_configs();
+ write_raw_config(200, existing_config('ctime=1500000000'));
+ my $err = restore_ct(200, { force => 1, ostemplate => '-', rootfs => 'local:4' });
+ is($err, '', 'restore from STDIN over a container without identity succeeds');
+ is(identity_of(200)->{uid}, 'backupbackupbackupba', 'uid of the backup is taken over');
+
+ $err = create_ct(201, { 'allow-identity-mismatch' => 1 });
+ like($err, qr/allow-identity-mismatch/, 'override is only allowed for restores');
+
+ my ($class, $method, undef, $fixed) = $PVE::CLI::pct::cmddef->{restore}->@*;
+ is("$class->$method", 'PVE::API2::LXC->create_vm', 'pct restore uses the create API call');
+ ok($fixed->{restore}, 'pct restore sets restore, so it accepts the override');
+
+ $lxc_module->mock(
+ delete_ifaces_ipams_ips => sub { },
+ destroy_lxc_container => $mocked_destroy,
+ );
+};
+
remove_tree($WORKDIR);
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH container v3 26/38] test: remote migration: check the cleanup of the target after failures
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (24 preceding siblings ...)
2026-11-18 4:37 ` [PATCH container v3 25/38] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 27/38] api: remote migration: take over the guest uid from the source Thomas Lamprecht
` (13 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A config left behind on the target of a failed remote migration blocks
its VMID there, and with guest identities it would also keep the uid of
the container in the target cluster.
Check with a mocked tunnel that the source asks the target to clean up
if the tunnel versions do not match or the container does not stop for a
restart migration. Check on the target that it removes its reservation
if the tunnel closes before the config arrived, keeps a config that
arrived, and logs the error of a failed tunnel command.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* follow the fixed wait for a volume transfer in the cleanup of the
target, which is bounded by time: check a transfer that exits late and
one that cannot be collected
changes v1 -> v2:
* was 'migrate: remote: clean up the target on a tunnel version
mismatch', its fix got applied separately together with the cleanup
after a failed shutdown
* more tests: failed shutdown of a restart migration, tunnel that closes
before the config arrived (with and without a running volume
transfer), logging of a failed tunnel command
src/test/lifecycle_tests.pl | 175 ++++++++++++++++++++++++++++++++++++
1 file changed, 175 insertions(+)
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
index 7606d3e..ff42c5f 100755
--- a/src/test/lifecycle_tests.pl
+++ b/src/test/lifecycle_tests.pl
@@ -1890,6 +1890,181 @@ subtest 'restore over a container of another guest needs an override' => sub {
);
};
+subtest 'remote restart migration cleans up the target if the container does not stop' => sub {
+ reset_configs();
+ my $raw = existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis');
+ write_raw_config(200, $raw);
+
+ my $log = '';
+ my $migrate_module = Test::MockModule->new('PVE::AbstractMigrate');
+ $migrate_module->mock(log => sub { $log .= "$_[1]: $_[2]\n" });
+
+ my $cleanup;
+ my $tunnel_module = Test::MockModule->new('PVE::Tunnel');
+ $tunnel_module->mock(
+ fork_websocket_tunnel => sub {
+ return { version => $PVE::LXC::Migrate::WS_TUNNEL_VERSION, age => 0 };
+ },
+ write_tunnel => sub { return {} },
+ finish_tunnel => sub { (undef, $cleanup) = @_ },
+ );
+ $lxc_module->mock(
+ check_running => sub { return 1 },
+ vm_stop => sub { die "container did not stop\n" },
+ );
+
+ my $err = run_task(sub {
+ PVE::LXC::Migrate->migrate(
+ 'remote-node',
+ '192.0.2.1',
+ 200,
+ {
+ remote => { vmid => 300, conn => { host => '192.0.2.1', port => 8006 } },
+ restart => 1,
+ },
+ );
+ });
+ $lxc_module->mock(check_running => sub { return 0 });
+ $lxc_module->unmock('vm_stop');
+
+ like($err, qr/migration aborted/, 'migration fails');
+ like($log, qr/^err: .*container did not stop/m, 'failed shutdown is reported');
+ is($cleanup, 1, 'target is asked to clean up');
+ is(read_raw_config(200), $raw, 'config of the source is unchanged');
+};
+
+subtest 'migration tunnel logs failed commands and cleans up after an early end' => sub {
+ reset_configs();
+ my ($err, $replies) = run_mtunnel(300);
+ like($err, qr/mtunnel exited unexpectedly/, 'tunnel that ends without a command fails');
+ like(
+ $task_log,
+ qr/^WARN: tunnel closed before the config arrived, cleaning up$/m,
+ 'cleanup is logged',
+ );
+ ok(!defined(read_raw_config(300)), 'reserved config is removed');
+
+ reset_configs();
+ my $broken = $INCOMING_CONFIG =~ s/^rootfs: local:/rootfs: iso:/mr;
+ ($err, $replies) = run_mtunnel(300, { cmd => 'config', conf => $broken });
+ like($err, qr/mtunnel exited unexpectedly/, 'tunnel that ends after a failed config fails');
+ ok(!$replies->[0]->{success}, 'config command fails');
+ like(
+ $task_log,
+ qr/^WARN: failed to handle 'config' command - /m,
+ 'failed command is logged',
+ );
+ ok(!defined(read_raw_config(300)), 'reserved config is removed after a failed config');
+
+ reset_configs();
+ ($err, $replies) = run_mtunnel(300, { cmd => 'config', conf => $INCOMING_CONFIG });
+ like($err, qr/mtunnel exited unexpectedly/, 'tunnel that ends after the config fails');
+ unlike($task_log, qr/cleaning up/, 'no cleanup once the config arrived');
+ is(PVE::LXC::Config->load_config(300)->{lock}, 'migrate', 'config that arrived is kept');
+
+ # the volume of a transfer is only known once the transfer is done
+ my $import_cmd = {
+ cmd => 'disk-import',
+ volname => 'vm-300-disk-0',
+ format => 'raw',
+ export_formats => 'raw+size',
+ storage => 'local',
+ };
+ my ($queries, $ends_after, $query_error, $freed) = (0, 2, undef, []);
+ my $storage_tunnel_module = Test::MockModule->new('PVE::StorageTunnel');
+ $storage_tunnel_module->mock(
+ handle_disk_import => sub {
+ my ($state) = @_;
+ $state->{disk_import} = { pid => 1 };
+ return {};
+ },
+ handle_query_disk_import => sub {
+ my ($state) = @_;
+ die $query_error if $query_error;
+ return { status => 'pending' } if ++$queries < $ends_after;
+ delete $state->{disk_import};
+ $state->{cleanup}->{volumes}->{'local:300/vm-300-disk-0.raw'} = 1;
+ return { status => 'complete' };
+ },
+ );
+ $storage_module->mock(vdisk_free => sub { push $freed->@*, $_[1] });
+
+ reset_configs();
+ ($err, $replies) = run_mtunnel(300, $import_cmd);
+ ok($replies->[0]->{success}, 'volume transfer starts') or diag(explain($replies));
+ is_deeply(
+ $freed,
+ ['local:300/vm-300-disk-0.raw'],
+ 'volume of a transfer that ended is freed',
+ );
+ ok(!defined(read_raw_config(300)), 'reserved config is removed after the transfer ended');
+
+ # a transfer that closed its output can be queried many times before it has exited
+ reset_configs();
+ ($queries, $ends_after, $freed) = (0, 20, []);
+ ($err, $replies) = run_mtunnel(300, $import_cmd);
+ is_deeply($freed, ['local:300/vm-300-disk-0.raw'], 'volume of a transfer that exits late');
+ ok(!defined(read_raw_config(300)), 'reserved config is removed after a late exit');
+
+ reset_configs();
+ ($queries, $query_error, $freed) = (0, "unable to query the transfer\n", []);
+ ($err, $replies) = run_mtunnel(300, $import_cmd);
+ like(
+ $task_log,
+ qr/^WARN: cleanup failed - a volume transfer is still running$/m,
+ 'transfer that cannot be collected is reported',
+ );
+ is_deeply($freed, [], 'nothing is freed while a transfer is not collected');
+ ok(
+ defined(read_raw_config(300)),
+ 'reserved config is kept while a transfer is not collected',
+ );
+ $storage_module->unmock('vdisk_free');
+};
+
+subtest 'remote migration cleans up a target with an incompatible tunnel version' => sub {
+ reset_configs();
+ my $raw = existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis');
+ write_raw_config(200, $raw);
+
+ my $log = '';
+ my $migrate_module = Test::MockModule->new('PVE::AbstractMigrate');
+ $migrate_module->mock(log => sub { $log .= "$_[1]: $_[2]\n" });
+
+ my $remote_version = { version => $PVE::LXC::Migrate::WS_TUNNEL_VERSION - 1, age => 0 };
+ my ($commands, $cleanup) = (0);
+ my $tunnel_module = Test::MockModule->new('PVE::Tunnel');
+ $tunnel_module->mock(
+ fork_websocket_tunnel => sub { return $remote_version },
+ write_tunnel => sub { $commands++ },
+ finish_tunnel => sub { (undef, $cleanup) = @_ },
+ );
+
+ my $migrate = sub {
+ ($log, $commands, $cleanup) = ('', 0);
+ return run_task(sub {
+ PVE::LXC::Migrate->migrate(
+ 'remote-node',
+ '192.0.2.1',
+ 200,
+ { remote => { vmid => 300, conn => { host => '192.0.2.1', port => 8006 } } },
+ );
+ });
+ };
+
+ like($migrate->(), qr/migration aborted/, 'migration to an older target fails');
+ like($log, qr/^err: .*Remote tunnel endpoint too old/m, 'target is reported as too old');
+ is($cleanup, 1, 'older target is asked to clean up');
+ is($commands, 0, 'nothing else is sent to the target');
+ is(read_raw_config(200), $raw, 'config of the source is unchanged');
+
+ $remote_version = { version => $PVE::LXC::Migrate::WS_TUNNEL_VERSION + 1, age => 0 };
+ like($migrate->(), qr/migration aborted/, 'migration to an incompatible target fails');
+ like($log, qr/^err: .*Remote tunnel endpoint not compatible/m, 'target is reported');
+ is($cleanup, 1, 'incompatible target is asked to clean up');
+ is(read_raw_config(200), $raw, 'config of the source is unchanged');
+};
+
remove_tree($WORKDIR);
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH container v3 27/38] api: remote migration: take over the guest uid from the source
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (25 preceding siblings ...)
2026-11-18 4:37 ` [PATCH container v3 26/38] test: remote migration: check the cleanup of the target after failures Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 28/38] api: status: return the guest uid Thomas Lamprecht
` (12 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
A container migrated to another cluster should keep its uid. Send the
uid to the target right after the tunnel version check, before the
container gets stopped for a restart migration and before any volume is
transferred. The target takes it over for the config that reserves the
VMID, and the config writes of the migration keep it. The target accepts
the uid only once and only before the config arrives. The registration
time is the time of that take-over. The command schema, the target
handler and the sender come from pve-guest-common and are shared with
VMs.
If another guest in the target cluster already has the uid, refuse the
migration. A migration moves a guest, so a new uid without notice would
leave the identity of the container with the other guest. The new option
'allow-new-uid' lets the migration go on with a new uid.
A failure to take over the uid, for example because the cluster-wide
lock cannot be taken, stops the migration at that point as well, and the
source asks the target to clean up. It also does so if stopping the
container for a restart migration fails afterwards, as the config on the
target then has the uid already. The source waits up to 60 seconds for
the reply to the uid, to cover the wait for that lock.
Report the result in the task logs of both sides. If a newer target
reports a result that the source does not know, the source only logs a
warning. A source with the previous tunnel version sends no uid, so the
container keeps the uid it got when the target reserved its VMID. A
container without uid gets one before the source sends it.
Bump the tunnel version, so that older targets are refused before
volumes are transferred, and not only when their strict parser rejects
'meta'. Keep accepting the previous version from older sources.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* use the shared helpers of pve-guest-common for the identity command
* a container without uid gets one before the source sends it; before,
source and target got unrelated uids
changes v1 -> v2:
* take the uid over at the start of the migration, through a new tunnel
command, instead of with the config
* refuse the migration if another guest in the target cluster has the
uid, unless the new option 'allow-new-uid' is set (Dominik)
* log the result on the source without a prefix, as for VMs
src/PVE/API2/LXC.pm | 19 +-
src/PVE/CLI/pct.pm | 7 +
src/PVE/LXC/Migrate.pm | 8 +-
src/test/lifecycle_tests.pl | 350 ++++++++++++++++++++++++++++++++++++
4 files changed, 381 insertions(+), 3 deletions(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index a5e45c2..24605d9 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -1500,6 +1500,13 @@ __PACKAGE__->register_method({
minimum => '0',
default => 'migrate limit from datacenter or storage config',
},
+ 'allow-new-uid' => {
+ type => 'boolean',
+ description => "Allow the migration if another guest in the target cluster"
+ . " already has the uid of this guest. The migrated guest then gets a new uid.",
+ optional => 1,
+ default => 0,
+ },
},
},
returns => {
@@ -3329,7 +3336,7 @@ __PACKAGE__->register_method({
my $socket_addr = "/run/pve/ct-$vmid.mtunnel";
my $lock = 'create';
- eval { PVE::LXC::Config->create_and_lock_config($vmid, 0, $lock); };
+ my $reservation = eval { PVE::LXC::Config->create_and_lock_config($vmid, 0, $lock) };
raise_param_exc({ vmid => "unable to create empty CT config - $@" })
if $@;
@@ -3359,6 +3366,7 @@ __PACKAGE__->register_method({
};
my $cmd_desc = {
+ identity => PVE::GuestIdentity::tunnel_identity_cmd_schema(),
config => {
conf => {
type => 'string',
@@ -3413,9 +3421,16 @@ __PACKAGE__->register_method({
# bump/bump for opt-in changes
return {
api => $PVE::LXC::Migrate::WS_TUNNEL_VERSION,
- age => 0,
+ age => 1,
};
},
+ 'identity' => sub {
+ my ($params) = @_;
+
+ return PVE::GuestIdentity::handle_tunnel_identity(
+ 'PVE::LXC::Config', $state, $reservation, $params,
+ );
+ },
'config' => sub {
my ($params) = @_;
diff --git a/src/PVE/CLI/pct.pm b/src/PVE/CLI/pct.pm
index 792ae11..da69eed 100755
--- a/src/PVE/CLI/pct.pm
+++ b/src/PVE/CLI/pct.pm
@@ -980,6 +980,13 @@ __PACKAGE__->register_method({
minimum => '0',
default => 'migrate limit from datacenter or storage config',
},
+ 'allow-new-uid' => {
+ type => 'boolean',
+ description => "Allow the migration if another guest in the target cluster"
+ . " already has the uid of this guest. The migrated guest then gets a new uid.",
+ optional => 1,
+ default => 0,
+ },
},
},
returns => {
diff --git a/src/PVE/LXC/Migrate.pm b/src/PVE/LXC/Migrate.pm
index 84c5139..d385c80 100644
--- a/src/PVE/LXC/Migrate.pm
+++ b/src/PVE/LXC/Migrate.pm
@@ -27,7 +27,8 @@ use PVE::AbstractMigrate;
use base qw(PVE::AbstractMigrate);
# compared against remote end's minimum version
-our $WS_TUNNEL_VERSION = 2;
+# 3: the 'identity' command, and 'meta' with the guest identity in the config
+our $WS_TUNNEL_VERSION = 3;
sub lock_vm {
my ($self, $vmid, $code, @param) = @_;
@@ -155,6 +156,11 @@ sub prepare {
$err = "Remote tunnel endpoint too old, upgrade required\n"
if $WS_TUNNEL_VERSION > $tunnel->{version};
+ if (!$err) {
+ eval { $self->take_over_identity_on_target('PVE::LXC::Config', $tunnel) };
+ $err = $@;
+ }
+
if ($err) {
# the target already reserved the VMID with a locked config
eval { PVE::Tunnel::finish_tunnel($tunnel, 1); };
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
index ff42c5f..089be3a 100755
--- a/src/test/lifecycle_tests.pl
+++ b/src/test/lifecycle_tests.pl
@@ -33,6 +33,7 @@ use PVE::CLI::pct;
use PVE::LXC;
use PVE::LXC::Config;
use PVE::LXC::Create;
+use PVE::LXC::Migrate;
use PVE::API2::LXC;
use PVE::API2::LXC::Config;
@@ -95,6 +96,11 @@ my $storage_cfg = {
shared => 1,
content => { rootdir => 1, images => 1 },
},
+ iso => {
+ type => 'dir',
+ path => '/mnt/iso',
+ content => { iso => 1 },
+ },
},
};
@@ -2065,6 +2071,350 @@ subtest 'remote migration cleans up a target with an incompatible tunnel version
is(read_raw_config(200), $raw, 'config of the source is unchanged');
};
+subtest 'remote migration takes over the guest identity of the source' => sub {
+ my $source_meta = 'ctime=1600000000,rtime=1600000000000,uid=sourcesourcesourceso';
+ my $incoming = $INCOMING_CONFIG =~ s/^meta: .*$/meta: $source_meta/mr;
+ # like the source sends it
+ my $identity_cmd =
+ { cmd => 'identity', uid => 'sourcesourcesourceso', 'allow-new-uid' => 0 };
+ my $config_cmd = { cmd => 'config', conf => $incoming };
+ my $quit_cmd = { cmd => 'quit', cleanup => 0 };
+ my $cleanup_cmd = { cmd => 'quit', cleanup => 1 };
+ $now_ms = 1_700_000_000_000;
+
+ my $migrate = sub {
+ my ($name, $setup, $allow_new_uid, $expected_result, $expected_uid) = @_;
+
+ reset_configs();
+ $setup->();
+ my ($err, $replies) = run_mtunnel(
+ 300,
+ { $identity_cmd->%*, 'allow-new-uid' => $allow_new_uid },
+ $config_cmd,
+ $quit_cmd,
+ );
+ is($err, '', "$name: tunnel exits normally") or diag(explain($replies), $task_log);
+ ok($replies->[0]->{success}, "$name: identity command succeeds");
+ ok($replies->[1]->{success}, "$name: config command succeeds");
+ ok(!exists($replies->[1]->{identity}), "$name: config command takes over nothing");
+
+ my $identity = $replies->[0]->{identity};
+ is($identity->{result}, $expected_result, "$name: result is returned");
+ my $stored = identity_of(300);
+ is($identity->{uid}, $stored->{uid}, "$name: returned uid is the stored one");
+ if (defined($expected_uid)) {
+ is($stored->{uid}, $expected_uid, "$name: uid");
+ } else {
+ isnt(
+ $stored->{uid},
+ 'sourcesourcesourceso',
+ "$name: uid is not the one of the source",
+ );
+ }
+ is($stored->{rtime}, $now_ms, "$name: rtime is the time of the migration");
+ is(meta_of(300)->{ctime}, 1600000000, "$name: ctime of the source is kept");
+ like(
+ $task_log,
+ qr/^guest identity: .*$identity->{uid}/m,
+ "$name: result is logged on the target",
+ );
+
+ return $identity;
+ };
+
+ $migrate->('free uid', sub { }, 0, 'adopted', 'sourcesourcesourceso');
+ my $reserved_write = $written->{300}->[1];
+ ok(
+ $reserved_write =~ m/^lock: create$/m
+ && $reserved_write =~ m/^meta: .*uid=sourcesourcesourceso$/m,
+ 'free uid: taken over right after the reservation',
+ );
+
+ my $in_use_by_ct =
+ sub { write_raw_config(150, existing_config('uid=sourcesourcesourceso')) };
+ my $in_use_by_vm =
+ sub { write_raw_vm_config(151, "memory: 512\nmeta: uid=sourcesourcesourceso\n") };
+ $migrate->('uid used by another container, allow-new-uid', $in_use_by_ct, 1, 'in-use');
+ $migrate->('uid used by a VM, allow-new-uid', $in_use_by_vm, 1, 'in-use');
+
+ for my $case (
+ ['uid used by another container', $in_use_by_ct, 150],
+ ['uid used by a VM', $in_use_by_vm, 151],
+ ) {
+ my ($name, $setup, $holder) = $case->@*;
+ reset_configs();
+ $setup->();
+ my ($err, $replies) = run_mtunnel(300, $identity_cmd, $cleanup_cmd);
+ is($err, '', "$name: tunnel exits normally") or diag($task_log);
+ ok(!$replies->[0]->{success}, "$name: identity command fails");
+ is(
+ $replies->[0]->{msg},
+ "failed to handle 'identity' command - guest $holder in the target cluster already"
+ . " has the uid 'sourcesourcesourceso' of this guest - use 'allow-new-uid' to"
+ . " migrate it with a new uid\n",
+ "$name: migration is refused",
+ );
+ is(scalar($written->{300}->@*), 1, "$name: nothing written besides the reservation");
+ ok($replies->[1]->{success}, "$name: cleanup succeeds");
+ ok(!defined(read_raw_config(300)), "$name: reservation is removed");
+ }
+
+ for my $case (
+ ['source without identity command', $incoming],
+ ['source without identity command and uid', $INCOMING_CONFIG],
+ ) {
+ my ($name, $conf) = $case->@*;
+ reset_configs();
+ my ($err, $replies) = run_mtunnel(300, { cmd => 'config', conf => $conf }, $quit_cmd);
+ is($err, '', "$name: tunnel exits normally") or diag($task_log);
+ ok($replies->[0]->{success}, "$name: config command succeeds");
+ my $uid = identity_of(300)->{uid};
+ like($uid, qr/^[a-z2-7]{20}$/, "$name: container has a uid");
+ isnt($uid, 'sourcesourcesourceso', "$name: it is a new one");
+ }
+
+ my $real_adopt = \&PVE::GuestIdentity::adopt;
+ my ($lock, $locked);
+ $identity_module->mock(
+ adopt => sub {
+ my ($class, $vmid) = @_;
+ $lock = PVE::LXC::Config->load_config($vmid)->{lock};
+ open(my $fh, '>>', "$WORKDIR/lock/pve-config-$vmid.lock")
+ or die "open failed - $!\n";
+ $locked = !flock($fh, LOCK_EX | LOCK_NB);
+ close($fh);
+ return $real_adopt->(@_);
+ },
+ );
+ $migrate->('order of the writes', sub { }, 0, 'adopted', 'sourcesourcesourceso');
+ $identity_module->unmock('adopt');
+ is($lock, 'create', 'identity is taken over before the config of the source is written');
+ ok($locked, 'config lock is held while taking over the identity');
+
+ reset_configs();
+ $domain_lock_error = "cfs-lock 'domain-guest-identity' error: got lock request timeout\n";
+ my ($err, $replies) = run_mtunnel(300, $identity_cmd, $cleanup_cmd);
+ $domain_lock_error = undef;
+ is($err, '', 'identity lock failure: tunnel exits normally') or diag($task_log);
+ ok(!$replies->[0]->{success}, 'identity lock failure: identity command fails');
+ is(
+ $replies->[0]->{msg},
+ "failed to handle 'identity' command - unable to adopt the uid of the source guest -"
+ . " cfs-lock 'domain-guest-identity' error: got lock request timeout\n",
+ 'identity lock failure: error is returned',
+ );
+ ok($replies->[1]->{success}, 'identity lock failure: cleanup succeeds');
+ ok(!defined(read_raw_config(300)), 'identity lock failure: target config is removed');
+
+ reset_configs();
+ ($err, $replies) =
+ run_mtunnel(300, $identity_cmd, $identity_cmd, $config_cmd, $identity_cmd, $quit_cmd);
+ is($err, '', 'repeated identity: tunnel exits normally');
+ ok($replies->[0]->{success}, 'first identity command succeeds');
+ ok($replies->[2]->{success}, 'config command succeeds');
+ for my $i (1, 3) {
+ like(
+ $replies->[$i]->{msg},
+ qr/the uid can only be taken over once, before the config arrives/,
+ "identity command at position $i fails",
+ );
+ }
+
+ reset_configs();
+ my $broken = $incoming =~ s/^rootfs: local:/rootfs: iso:/mr;
+ ($err, $replies) = run_mtunnel(
+ 300,
+ $identity_cmd,
+ { cmd => 'config', conf => $broken },
+ $config_cmd,
+ $config_cmd,
+ $quit_cmd,
+ );
+ is($err, '', 'repeated config: tunnel exits normally');
+ is($replies->[0]->{identity}->{result}, 'adopted', 'uid is taken over at the start');
+ ok(!$replies->[1]->{success}, 'failing config command fails');
+ ok($replies->[2]->{success}, 'retry of the config command succeeds');
+ ok(!$replies->[3]->{success}, 'config command after success fails');
+ is_deeply(
+ identity_of(300),
+ { uid => 'sourcesourcesourceso', rtime => $now_ms },
+ 'identity taken over at the start is kept',
+ );
+};
+
+subtest 'remote migration sends the guest identity on the source' => sub {
+ my $source_raw = <<~'EOF';
+ arch: amd64
+ hostname: source
+ memory: 512
+ meta: ctime=1600000000,rtime=1600000000000,uid=sourcesourcesourceso
+ ostype: debian
+ unprivileged: 1
+ EOF
+ reset_configs();
+ write_raw_config(987, $source_raw);
+
+ my $log = '';
+ my $migrate_module = Test::MockModule->new('PVE::AbstractMigrate');
+ $migrate_module->mock(log => sub { $log .= "$_[1]: $_[2]\n" });
+
+ my ($commands, $reply) = ([]);
+ my $tunnel_module = Test::MockModule->new('PVE::Tunnel');
+ $tunnel_module->mock(
+ write_tunnel => sub {
+ my ($tunnel, $timeout, $command, $params) = @_;
+ push $commands->@*, { command => $command, timeout => $timeout, params => $params };
+ return $reply;
+ },
+ );
+
+ my $new_migration = sub($opts = {}) {
+ return bless {
+ opts => { remote => { vmid => 300 }, $opts->%* },
+ vmid => 987,
+ node => 'remote-node',
+ nodeip => '192.0.2.1',
+ storecfg => $storage_cfg,
+ vmconf => PVE::LXC::Config->load_config(987),
+ tunnel => {},
+ },
+ 'PVE::LXC::Migrate';
+ };
+ my $take_over = sub($self) {
+ ($commands, $log) = ([], '');
+ return run_task(sub { $self->take_over_identity_on_target('PVE::LXC::Config', {}) });
+ };
+
+ $reply = {
+ success => 1,
+ identity => {
+ result => 'adopted',
+ uid => 'sourcesourcesourceso',
+ requested => 'sourcesourcesourceso',
+ },
+ };
+ is($take_over->($new_migration->()), '', 'identity command succeeds');
+ is_deeply(
+ $commands,
+ [{
+ command => 'identity',
+ timeout => 60,
+ params => { uid => 'sourcesourcesourceso', 'allow-new-uid' => 0 },
+ }],
+ 'uid is sent, the source waits for the identity lock of the target',
+ );
+ my $adopted = 'guest identity: adopted uid sourcesourcesourceso from the source guest';
+ like($log, qr/^info: \Q$adopted\E$/m, 'result of the target is logged');
+
+ is($take_over->($new_migration->({ 'allow-new-uid' => 1 })), '', 'allow-new-uid');
+ is($commands->[0]->{params}->{'allow-new-uid'}, 1, 'allow-new-uid is sent');
+
+ # a target with a newer version has accepted the uid at this point
+ $reply =
+ { success => 1, identity => { result => 'future', uid => 'sourcesourcesourceso' } };
+ is($take_over->($new_migration->()), '', 'outcome unknown here does not fail');
+ like(
+ $log,
+ qr/^warn: unknown result of guest identity adoption 'future'$/m,
+ 'unknown outcome is a warning',
+ );
+
+ write_raw_config(987, $source_raw =~ s/^meta: .*\n//mr);
+ $reply = { success => 1 };
+ is($take_over->($new_migration->()), '', 'source without uid');
+ my $uid = identity_of(987)->{uid};
+ like($uid, qr/^[a-z2-7]{20}$/, 'source without uid: it gets a uid first');
+ is_deeply(
+ $commands,
+ [{
+ command => 'identity',
+ timeout => 60,
+ params => { uid => $uid, 'allow-new-uid' => 0 },
+ }],
+ 'source without uid: that uid is sent',
+ );
+ write_raw_config(987, $source_raw);
+
+ $reply = { success => 1 };
+ ($commands, $log) = ([], '');
+ is(run_task(sub { $new_migration->()->phase1(987) }), '', 'phase 1 succeeds')
+ or diag($log, $task_log);
+ is($commands->[0]->{command}, 'config', 'phase 1 sends the config');
+ like(
+ $commands->[0]->{params}->{conf},
+ qr/^meta: .*uid=sourcesourcesourceso/m,
+ 'config is sent with the identity',
+ );
+ is($commands->[0]->{timeout}, 10, 'config command does not wait for the identity lock');
+ unlike($log, qr/guest identity/, 'nothing is logged about the identity');
+ is(identity_of(987)->{uid}, 'sourcesourcesourceso', 'source keeps its uid');
+ PVE::LXC::Config->remove_lock(987, 'migrate');
+};
+
+subtest 'remote migration keeps the source if the target refuses the identity' => sub {
+ reset_configs();
+ write_raw_config(987, <<~'EOF');
+ arch: amd64
+ hostname: source
+ memory: 512
+ meta: ctime=1600000000,rtime=1600000000000,uid=sourcesourcesourceso
+ ostype: debian
+ unprivileged: 1
+ EOF
+
+ my $log = '';
+ my $migrate_module = Test::MockModule->new('PVE::AbstractMigrate');
+ $migrate_module->mock(log => sub { $log .= "$_[1]: $_[2]\n" });
+
+ my ($commands, $cleanup) = ([]);
+ my $tunnel_module = Test::MockModule->new('PVE::Tunnel');
+ $tunnel_module->mock(
+ fork_websocket_tunnel => sub { return { version => 3, age => 1 } },
+ write_tunnel => sub {
+ my ($tunnel, $timeout, $command) = @_;
+ push $commands->@*, $command;
+ die "error - tunnel command '$command' failed - failed to handle 'identity' command"
+ . " - guest 150 in the target cluster already has the uid 'sourcesourcesourceso'"
+ . " of this guest - use 'allow-new-uid' to migrate it with a new uid\n";
+ },
+ finish_tunnel => sub { (undef, $cleanup) = @_ },
+ );
+
+ my ($running, $stopped, $destroyed) = (1, 0, 0);
+ my $destroy_ct = \&PVE::LXC::destroy_lxc_container;
+ $lxc_module->mock(
+ check_running => sub { return $running },
+ vm_stop => sub { $stopped = 1 },
+ destroy_lxc_container => sub { $destroyed = 1 },
+ );
+
+ my $err = run_task(sub {
+ PVE::LXC::Migrate->migrate(
+ 'remote-node',
+ '192.0.2.1',
+ 987,
+ {
+ remote => { vmid => 300, conn => { host => '192.0.2.1', port => 8006 } },
+ restart => 1,
+ delete => 1,
+ },
+ );
+ });
+
+ $lxc_module->mock(check_running => sub { return 0 }, destroy_lxc_container => $destroy_ct);
+ $lxc_module->unmock('vm_stop');
+
+ like($err, qr/migration aborted/, 'migration fails');
+ like($log, qr/^err: .*already has the uid 'sourcesourcesourceso'/m, 'refusal is logged');
+ is_deeply($commands, ['identity'], 'nothing else is sent to the target');
+ is($cleanup, 1, 'target gets cleaned up');
+ ok(!$stopped, 'source is not stopped for the restart migration');
+ ok(!$destroyed, 'source is not destroyed');
+ my $conf = PVE::LXC::Config->load_config(987);
+ ok(!defined($conf->{lock}), 'source is not locked');
+ is(identity_of(987)->{uid}, 'sourcesourcesourceso', 'source keeps its uid');
+};
+
remove_tree($WORKDIR);
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH container v3 28/38] api: status: return the guest uid
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (26 preceding siblings ...)
2026-11-18 4:37 ` [PATCH container v3 27/38] api: remote migration: take over the guest uid from the source Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 29/38] api: cluster resources: add guest uid and registration time Thomas Lamprecht
` (11 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Return the uid with the container's current status so tools can follow
the guest across restores and migrations without parsing the 'meta'
property of its config.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* group the imports of the test script
changes v1 -> v2:
* use the shared schema of the uid
src/PVE/API2/LXC/Status.pm | 6 ++++++
src/test/lifecycle_tests.pl | 26 ++++++++++++++++++++++++++
2 files changed, 32 insertions(+)
diff --git a/src/PVE/API2/LXC/Status.pm b/src/PVE/API2/LXC/Status.pm
index c95fc3c..c8e1814 100644
--- a/src/PVE/API2/LXC/Status.pm
+++ b/src/PVE/API2/LXC/Status.pm
@@ -10,6 +10,7 @@ use PVE::INotify;
use PVE::Cluster qw(cfs_read_file);
use PVE::AccessControl;
use PVE::Firewall;
+use PVE::GuestIdentity;
use PVE::Storage;
use PVE::RESTHandler;
use PVE::RPCEnvironment;
@@ -98,6 +99,7 @@ __PACKAGE__->register_method({
description => "HA manager service status.",
type => 'object',
},
+ uid => PVE::GuestIdentity::meta_format_properties()->{uid},
},
},
code => sub {
@@ -112,6 +114,10 @@ __PACKAGE__->register_method({
$status->{ha} = PVE::HA::Config::get_service_status("ct:$vmid");
+ if (my $identity = PVE::LXC::Config->get_identity($conf)) {
+ $status->{uid} = $identity->{uid};
+ }
+
return $status;
},
});
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
index 089be3a..05712d1 100755
--- a/src/test/lifecycle_tests.pl
+++ b/src/test/lifecycle_tests.pl
@@ -37,6 +37,7 @@ use PVE::LXC::Migrate;
use PVE::API2::LXC;
use PVE::API2::LXC::Config;
+use PVE::API2::LXC::Status;
my $WORKDIR = 'lifecycle-working';
my $NODENAME = PVE::INotify::nodename();
@@ -2415,6 +2416,31 @@ subtest 'remote migration keeps the source if the target refuses the identity' =
is(identity_of(987)->{uid}, 'sourcesourcesourceso', 'source keeps its uid');
};
+subtest 'current status reports the uid' => sub {
+ reset_configs();
+ write_raw_config(
+ 200,
+ existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis'),
+ );
+ write_raw_config(201, existing_config('ctime=1500000000'));
+
+ $lxc_module->mock(
+ vmstatus => sub {
+ my ($vmid) = @_;
+ return { $vmid => { vmid => int($vmid), status => 'stopped' } };
+ },
+ );
+ my $ha_config_module = Test::MockModule->new('PVE::HA::Config');
+ $ha_config_module->mock(get_service_status => sub { return { managed => 0 } });
+
+ my $status = PVE::API2::LXC::Status->vm_status({ node => $NODENAME, vmid => 200 });
+ is($status->{uid}, 'existingexistingexis', 'uid is reported');
+ $status = PVE::API2::LXC::Status->vm_status({ node => $NODENAME, vmid => 201 });
+ ok(!exists($status->{uid}), 'no uid is reported for a container without identity');
+
+ $lxc_module->unmock('vmstatus');
+};
+
remove_tree($WORKDIR);
done_testing();
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH manager v3 29/38] api: cluster resources: add guest uid and registration time
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (27 preceding siblings ...)
2026-11-18 4:37 ` [PATCH container v3 28/38] api: status: return the guest uid Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 30/38] api: cluster status: add cluster ID Thomas Lamprecht
` (10 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Tools that follow guests across restores and migrations, for example
backup products, need the guest identity. Report the 'uid' and 'rtime'
of guests that have them on their rows, so such tools get both for all
guests with one call instead of reading each guest config. Like lock
and tags, they are only reported to users with VM.Audit on the guest.
Both come from the 'meta' line, which the existing request for lock and
tags now also returns, so there is no extra round trip to pmxcfs. The
line is parsed per guest, and a guest with a missing or malformed
identity only lacks the fields. Neither the raw line nor its other
members are exposed.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
no changes since v2
changes v1 -> v2:
* use the shared schema of pve-guest-common for uid and rtime
* note why 'meta' is not part of the property list (Dominik)
PVE/API2/Cluster.pm | 14 +-
test/ClusterResources_test.pl | 249 ++++++++++++++++++++++++++++++++++
test/Makefile | 6 +-
3 files changed, 267 insertions(+), 2 deletions(-)
create mode 100755 test/ClusterResources_test.pl
diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index aaec3ec3..d3ea77aa 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -12,6 +12,7 @@ use PVE::Exception qw(raise_param_exc);
use PVE::Firewall;
use PVE::GuestHelpers;
use PVE::GuestID;
+use PVE::GuestIdentity;
use PVE::HA::Config;
use PVE::HA::Env::PVE2;
use PVE::INotify;
@@ -260,6 +261,9 @@ my sub can_access_network {
return 0;
}
+my $guest_identity_properties = PVE::GuestIdentity::meta_format_properties();
+$_->{description} .= " (for types 'qemu' and 'lxc')" for values $guest_identity_properties->%*;
+
__PACKAGE__->register_method({
name => 'resources',
path => 'resources',
@@ -475,6 +479,7 @@ __PACKAGE__->register_method({
optional => 1,
default => 0,
},
+ $guest_identity_properties->%*,
network => {
description => "The name of a Network entity (for type 'network').",
type => "string",
@@ -552,8 +557,9 @@ __PACKAGE__->register_method({
# we try to generate 'numbers' by using "$X + 0"
if (!$param->{type} || $param->{type} eq 'vm') {
+ # copied into the entry as they are, unlike 'meta', which only provides the identity
my $prop_list = [qw(lock tags)];
- my $props = PVE::Cluster::get_guest_config_properties($prop_list);
+ my $props = PVE::Cluster::get_guest_config_properties([@$prop_list, 'meta']);
for my $vmid (sort keys %$idlist) {
@@ -594,6 +600,12 @@ __PACKAGE__->register_method({
}
}
+ my $identity = PVE::GuestIdentity::parse_identity($props->{$vmid}->{meta});
+ if ($identity) {
+ $entry->{uid} = $identity->{uid};
+ $entry->{rtime} = int($identity->{rtime}) if defined($identity->{rtime});
+ }
+
if (
defined($entry->{pool})
&& !$rpcenv->check($authuser, "/pool/$entry->{pool}", ['Pool.Audit'], 1)
diff --git a/test/ClusterResources_test.pl b/test/ClusterResources_test.pl
new file mode 100755
index 00000000..4b981dc7
--- /dev/null
+++ b/test/ClusterResources_test.pl
@@ -0,0 +1,249 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib ('.', '..');
+
+use JSON;
+use Test::MockModule;
+use Test::More;
+
+use PVE::JSONSchema;
+
+use PVE::API2::Cluster;
+
+my $uid_100 = 'k3d7q2mxa5vb4n6pw2fz';
+my $uid_101 = 'r6t2w4y7z3c5e2g4h7jm';
+my $uid_104 = 'q2w3e4r5t6y7u2i3o4pa';
+
+# top-level config values per guest as decoded from the pmxcfs reply, plus whether the user has
+# VM.Audit on it; node2 is offline
+my $guests;
+my $default_guests = {
+ 100 => {
+ type => 'qemu',
+ node => 'node1',
+ audit => 1,
+ lock => 'backup',
+ tags => 'prod;web',
+ meta => "creation-qemu=9.2.0,ctime=1700000000,rtime=1731600000123,uid=$uid_100",
+ },
+ 101 => { type => 'lxc', node => 'node2', audit => 1, meta => "uid=$uid_101" },
+ 102 => {
+ type => 'qemu',
+ node => 'node1',
+ audit => 1,
+ meta => 'rtime=1731600000123,uid=NOT-A-UID',
+ },
+ 103 => { type => 'qemu', node => 'node1', audit => 1, lock => 'migrate' },
+ 104 => { type => 'qemu', node => 'node2', audit => 1, meta => "rtime=12x,uid=$uid_104" },
+ 105 => {
+ type => 'lxc',
+ node => 'node1',
+ audit => 0,
+ meta => "rtime=1731600000123,uid=$uid_100",
+ },
+};
+
+my $default_identities = {
+ 100 => { uid => $uid_100, rtime => 1731600000123 },
+ 101 => { uid => $uid_101 },
+ 102 => {},
+ 103 => {},
+ 104 => { uid => $uid_104 },
+};
+
+my $property_requests;
+my $ipc_failures = 0;
+
+my $cluster_mock = Test::MockModule->new('PVE::Cluster');
+$cluster_mock->redefine(
+ get_nodelist => sub { return ['node1', 'node2'] },
+ get_members => sub {
+ return { node1 => { id => 1, online => 1 }, node2 => { id => 2, online => 0 } };
+ },
+ rrd_dump => sub { return {} },
+ get_node_kv => sub { return {} },
+ get_vmlist => sub {
+ my $ids = {};
+ for my $vmid (keys %$guests) {
+ my $guest = $guests->{$vmid};
+ $ids->{$vmid} = { node => $guest->{node}, type => $guest->{type}, version => 1 };
+ }
+ return { ids => $ids };
+ },
+ get_guest_config_properties => sub($properties, $vmid = undef) {
+ die "only the request for all guests is expected\n" if defined($vmid);
+ push @$property_requests, [@$properties];
+ die "ipcc_send_rec[11] failed: Connection refused\n" if $ipc_failures-- > 0;
+
+ my $res = {};
+ for my $id (keys %$guests) {
+ for my $prop (@$properties) {
+ my $value = $guests->{$id}->{$prop} // next;
+ $res->{$id}->{$prop} = $value;
+ }
+ }
+ return $res;
+ },
+);
+
+my $ha_config_mock = Test::MockModule->new('PVE::HA::Config');
+$ha_config_mock->redefine(
+ read_manager_status => sub { return {} },
+ read_resources_config => sub { return { ids => {} } },
+);
+
+my $storage_mock = Test::MockModule->new('PVE::Storage');
+$storage_mock->redefine(
+ config => sub { return { ids => {} } },
+ storage_ids => sub { return () },
+);
+
+my $rpcenv = bless { user_cfg => { pools => { p1 => {} }, vms => { 100 => 'p1' } } },
+ 'PVE::RPCEnvironment::Test';
+sub PVE::RPCEnvironment::Test::get_user { return 'auditor@pve'; }
+
+sub PVE::RPCEnvironment::Test::check($self, $user, $path, $privs, $noerr) {
+ return 1 if $path eq '/pool/p1' && grep { $_ eq 'Pool.Audit' } @$privs;
+ my ($vmid) = $path =~ m!^/vms/(\d+)$! or return 0;
+ return $guests->{$vmid}->{audit} && grep { $_ eq 'VM.Audit' } @$privs;
+}
+my $rpc_mock = Test::MockModule->new('PVE::RPCEnvironment');
+$rpc_mock->redefine(get => sub { return $rpcenv });
+
+my $method = PVE::API2::Cluster->map_method_by_name('resources');
+
+# returns the rows and their JSON encoding, which is what API clients see
+sub get_rows($param) {
+ $property_requests = [];
+
+ my $res = PVE::API2::Cluster->handle($method, $param);
+ my $json = JSON->new->utf8->canonical->encode($res);
+
+ PVE::JSONSchema::validate($res, $method->{returns}, "return schema violated\n");
+
+ return ($res, $json);
+}
+
+sub guest_rows($res) {
+ return { map { $_->{vmid} => $_ } grep { $_->{type} =~ m/^(?:qemu|lxc)$/ } @$res };
+}
+
+sub identity_view($res) {
+ my $rows = guest_rows($res);
+
+ my $view = {};
+ for my $vmid (keys %$rows) {
+ my $row = $rows->{$vmid};
+ $view->{$vmid} = { map { $_ => $row->{$_} } grep { exists($row->{$_}) } qw(uid rtime) };
+ }
+ return $view;
+}
+
+subtest 'identity of guests with valid, partial, invalid and missing meta' => sub {
+ $guests = {%$default_guests};
+
+ my ($res, $json) = get_rows({ type => 'vm' });
+ my $rows = guest_rows($res);
+
+ is_deeply(
+ $property_requests,
+ [[qw(lock tags meta)]],
+ 'meta is fetched together with lock and tags in a single request',
+ );
+
+ is_deeply(
+ identity_view($res),
+ $default_identities,
+ 'uid and rtime only where valid, guests without VM.Audit are not listed',
+ );
+
+ like($json, qr/"rtime":1731600000123[,}]/, 'rtime is encoded as JSON number');
+ like($json, qr/"uid":"$uid_100"/, 'uid is encoded as JSON string');
+
+ is($rows->{100}->{lock}, 'backup', 'lock is still reported');
+ is($rows->{100}->{tags}, 'prod;web', 'tags are still reported');
+ is($rows->{103}->{lock}, 'migrate', 'lock of a guest without meta is still reported');
+
+ for my $key (qw(meta ctime creation-qemu)) {
+ ok(!grep({ exists($_->{$key}) } @$res), "no '$key' in any row");
+ }
+};
+
+subtest 'listing of all resource types' => sub {
+ $guests = {%$default_guests};
+
+ my ($res) = get_rows({});
+
+ is(scalar(@$property_requests), 1, 'guest config properties are requested once');
+ is_deeply(identity_view($res), $default_identities, 'same identities as for type vm');
+ is(guest_rows($res)->{101}->{status}, 'unknown', 'guest on offline node is listed');
+
+ my @others = grep { $_->{type} !~ m/^(?:qemu|lxc)$/ } @$res;
+ is_deeply(
+ [sort map { $_->{id} } @others],
+ ['/pool/p1', 'node/node1', 'node/node2'],
+ 'pool and node rows are listed',
+ );
+ ok(!grep({ exists($_->{uid}) || exists($_->{rtime}) } @others),
+ 'no identity on other rows');
+};
+
+subtest 'control character or replacement character in meta' => sub {
+ # pmxcfs escapes control characters and replaces bytes that are not valid UTF-8 with U+FFFD
+ $guests = {
+ %$default_guests,
+ 106 => {
+ type => 'qemu',
+ node => 'node1',
+ audit => 1,
+ lock => 'backup',
+ meta => "rtime=1731600000123,uid=k3d7q2mx\ta5vb4n6p",
+ },
+ 107 => {
+ type => 'qemu',
+ node => 'node1',
+ audit => 1,
+ meta => "rtime=1731600000123,uid=k3d7q2mx\x{FFFD}5vb4n6p",
+ },
+ 108 => {
+ type => 'lxc',
+ node => 'node1',
+ audit => 1,
+ meta => "ctime=17000\x{1}00000,rtime=1731600000456,uid=z2y3x4w5v6u7t2s3r4qn",
+ },
+ };
+
+ my ($res) = eval { get_rows({ type => 'vm' }) };
+ is($@, '', 'listing does not fail');
+
+ is_deeply(
+ identity_view($res),
+ {
+ %$default_identities,
+ 106 => {},
+ 107 => {},
+ 108 => { uid => 'z2y3x4w5v6u7t2s3r4qn', rtime => 1731600000456 },
+ },
+ 'only the guests with a malformed uid lack the identity',
+ );
+ is(guest_rows($res)->{106}->{lock}, 'backup', 'lock of such a guest is still reported');
+};
+
+subtest 'transient IPC failure fails the listing' => sub {
+ $guests = {%$default_guests};
+ $ipc_failures = 1;
+
+ eval { get_rows({ type => 'vm' }) };
+ like(
+ $@,
+ qr/Connection refused/,
+ 'error is raised instead of listing guests without identity',
+ );
+ is(scalar(@$property_requests), 1, 'request is not repeated');
+
+ $ipc_failures = 0;
+};
+
+done_testing();
diff --git a/test/Makefile b/test/Makefile
index 91761579..a43d7da7 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -5,7 +5,7 @@ all:
export PERLLIB=..
.PHONY: check
-check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-pvesh test-report
+check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-pvesh test-report test-cluster-resources
.PHONY: test-balloon
test-balloon:
@@ -65,6 +65,10 @@ test-pvesh:
test-report:
./report_test.pl
+.PHONY: test-cluster-resources
+test-cluster-resources:
+ ./ClusterResources_test.pl
+
.PHONY: install
install:
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH manager v3 30/38] api: cluster status: add cluster ID
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (28 preceding siblings ...)
2026-11-18 4:37 ` [PATCH manager v3 29/38] api: cluster resources: add guest uid and registration time Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 31/38] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
` (9 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Tools need an identifier of the cluster next to the guest uid, to tell
apart guests with the same uid in different clusters. Cluster names are
chosen freely and standalone nodes have none, and the existing 'id'
fields only identify entries within the response.
Return the ID that pmxcfs generates as 'cluster-id' on the cluster
entry, or on the node entry for a standalone node. Omit it while no ID
exists, for example as long as the cluster member with the lowest node
ID runs an older version.
Reading the ID requires Sys.Audit on '/', like the rest of this
endpoint, while the guest uid and rtime in /cluster/resources only need
VM.Audit on the guest. The ID is not a secret; restricting it to this
endpoint is a cautious start, as access is easier to widen than to take
back. Exposing it to less privileged users remains an option, to be
weighed carefully, if a use case needs it.
A failure to read the ID only omits the field, so the status stays
available. It is not logged, as clients poll this endpoint often.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
no changes since v2
changes v1 -> v2:
* rebase only
PVE/API2/Cluster.pm | 10 ++++
test/ClusterStatus_test.pl | 114 +++++++++++++++++++++++++++++++++++++
test/Makefile | 6 +-
3 files changed, 129 insertions(+), 1 deletion(-)
create mode 100755 test/ClusterStatus_test.pl
diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index d3ea77aa..f68a6717 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -923,6 +923,13 @@ __PACKAGE__->register_method({
description =>
'[cluster] Indicates if there is a majority of nodes online to make decisions',
},
+ 'cluster-id' => {
+ type => 'string',
+ optional => 1,
+ description =>
+ '[cluster] Random identifier of the cluster. A standalone node'
+ . ' reports its own identifier on its node entry.',
+ },
nodeid => {
type => 'integer',
optional => 1,
@@ -963,6 +970,7 @@ __PACKAGE__->register_method({
my $members = PVE::Cluster::get_members();
my $nodename = PVE::INotify::nodename();
my $rrd = PVE::Cluster::rrd_dump();
+ my $cluster_id = eval { PVE::Cluster::get_cluster_id() };
if ($members) {
my $res = [];
@@ -976,6 +984,7 @@ __PACKAGE__->register_method({
version => $d->{version},
name => $d->{name},
quorate => $d->{quorate},
+ defined($cluster_id) ? ('cluster-id' => $cluster_id) : (),
};
}
@@ -1017,6 +1026,7 @@ __PACKAGE__->register_method({
nodeid => 0,
online => 1,
level => $sublevel,
+ defined($cluster_id) ? ('cluster-id' => $cluster_id) : (),
}];
}
},
diff --git a/test/ClusterStatus_test.pl b/test/ClusterStatus_test.pl
new file mode 100755
index 00000000..02e9c1d6
--- /dev/null
+++ b/test/ClusterStatus_test.pl
@@ -0,0 +1,114 @@
+#!/usr/bin/perl
+
+use v5.36;
+
+use lib ('.', '..');
+
+use Test::MockModule;
+use Test::More;
+
+use PVE::JSONSchema;
+
+use PVE::API2::Cluster;
+use PVE::API2::Subscription;
+
+my $nodename = 'node1';
+my $cluster_id = 'm4q7z2kd3x6ab5tyw3hc';
+
+my $clinfo;
+my $cluster_id_result; # code returning the result of reading the cluster ID
+
+my $cluster_mock = Test::MockModule->new('PVE::Cluster');
+$cluster_mock->redefine(
+ cfs_update => sub { },
+ get_clinfo => sub { return $clinfo },
+ get_members => sub { return $clinfo->{nodelist} },
+ rrd_dump => sub { return {} },
+ remote_node_ip => sub { return '192.0.2.1' },
+ get_cluster_id => sub { return $cluster_id_result->() },
+);
+
+my $inotify_mock = Test::MockModule->new('PVE::INotify');
+$inotify_mock->redefine(nodename => sub { return $nodename });
+
+my $subscription_mock = Test::MockModule->new('PVE::API2::Subscription');
+$subscription_mock->redefine(read_etc_subscription => sub { return { level => 'c' } });
+
+my $cluster_info = {
+ cluster => { name => 'testcluster', nodes => 2, version => 3, quorate => 1 },
+ nodelist => {
+ node1 => { id => 1, online => 1, ip => '192.0.2.1' },
+ node2 => { id => 2, online => 0, ip => '192.0.2.2' },
+ },
+};
+# what pmxcfs reports without a corosync configuration
+my $standalone_info = { version => 1 };
+
+my $method = PVE::API2::Cluster->map_method_by_name('get_status');
+
+sub get_status() {
+ my $res = PVE::API2::Cluster->handle($method, {});
+ PVE::JSONSchema::validate($res, $method->{returns}, "return schema violated\n");
+ return $res;
+}
+
+sub cluster_ids($res) {
+ return { map { $_->{id} => $_->{'cluster-id'} } grep { exists($_->{'cluster-id'}) } @$res };
+}
+
+for my $case (
+ {
+ name => 'cluster with ID',
+ clinfo => $cluster_info,
+ read => sub { return $cluster_id },
+ expected => { cluster => $cluster_id },
+ },
+ {
+ name => 'cluster without ID',
+ clinfo => $cluster_info,
+ read => sub { return undef },
+ expected => {},
+ },
+ {
+ name => 'cluster, reading the ID fails',
+ clinfo => $cluster_info,
+ read => sub { die "Connection refused\n" },
+ expected => {},
+ },
+ {
+ name => 'standalone node with ID',
+ clinfo => $standalone_info,
+ read => sub { return $cluster_id },
+ expected => { "node/$nodename" => $cluster_id },
+ },
+ {
+ name => 'standalone node without ID',
+ clinfo => $standalone_info,
+ read => sub { return undef },
+ expected => {},
+ },
+ {
+ name => 'standalone node, reading the ID fails',
+ clinfo => $standalone_info,
+ read => sub { die "Connection refused\n" },
+ expected => {},
+ },
+) {
+ subtest $case->{name} => sub {
+ $clinfo = $case->{clinfo};
+ $cluster_id_result = $case->{read};
+
+ my $res = eval { get_status() };
+ is($@, '', 'status does not fail');
+ is_deeply(cluster_ids($res), $case->{expected}, 'cluster ID on the expected entry');
+
+ my $ids = [sort map { $_->{id} } @$res];
+ if ($clinfo->{cluster}) {
+ is_deeply($ids, ['cluster', 'node/node1', 'node/node2'], 'entry IDs unchanged');
+ } else {
+ is_deeply($ids, ["node/$nodename"], 'entry IDs unchanged');
+ }
+ };
+}
+
+done_testing();
diff --git a/test/Makefile b/test/Makefile
index a43d7da7..84dd46cc 100644
--- a/test/Makefile
+++ b/test/Makefile
@@ -5,7 +5,7 @@ all:
export PERLLIB=..
.PHONY: check
-check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-pvesh test-report test-cluster-resources
+check: test-replication test-balloon test-vzdump test-osd test-ceph-health test-ceph-services test-ceph-auth test-ceph-key-migration test-ceph-lockbox-migration test-custom-cpu-models test-pvesh test-report test-cluster-resources test-cluster-status
.PHONY: test-balloon
test-balloon:
@@ -69,6 +69,10 @@ test-report:
test-cluster-resources:
./ClusterResources_test.pl
+.PHONY: test-cluster-status
+test-cluster-status:
+ ./ClusterStatus_test.pl
+
.PHONY: install
install:
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH manager v3 31/38] ui: restore: allow overwriting a guest with a backup of another guest
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (29 preceding siblings ...)
2026-11-18 4:37 ` [PATCH manager v3 30/38] api: cluster status: add cluster ID Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 32/38] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
` (8 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Restoring a backup with a different uid over an existing guest requires
'allow-identity-mismatch', including backups of a former guest with the
same VMID. The restore task can reject such a backup, but the web UI
could not pass the override, leaving no way to proceed there.
Offer an unchecked checkbox whenever the restore window overwrites an
existing guest. Keep it independent of the overwrite option so an
accidentally selected backup is still rejected by default. The restore
task decides whether the identities match; the UI does not try to
predict that result. With the override, a guest that already had an
identity keeps it.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
no changes since v1
www/manager6/window/Restore.js | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/www/manager6/window/Restore.js b/www/manager6/window/Restore.js
index 60266ec1..27df7d53 100644
--- a/www/manager6/window/Restore.js
+++ b/www/manager6/window/Restore.js
@@ -47,6 +47,9 @@ Ext.define('PVE.window.Restore', {
if (values['live-restore']) {
params['live-restore'] = 1;
}
+ if (values['allow-identity-mismatch']) {
+ params['allow-identity-mismatch'] = 1;
+ }
if (values.storage) {
params.storage = values.storage;
}
@@ -256,6 +259,21 @@ Ext.define('PVE.window.Restore', {
},
];
+ if (me.vmid) {
+ items.push({
+ xtype: 'proxmoxcheckbox',
+ name: 'allow-identity-mismatch',
+ boxLabel: gettext('Allow restoring a backup of another guest'),
+ checked: false,
+ autoEl: {
+ tag: 'div',
+ 'data-qtip': gettext(
+ 'Also required for a backup of a previous guest with the same VMID. The existing guest keeps its identity.',
+ ),
+ },
+ });
+ }
+
if (me.vmtype === 'lxc') {
items.push({
xtype: 'radiogroup',
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH manager v3 32/38] ui: migrate: refuse target nodes that would drop the guest identity
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (30 preceding siblings ...)
2026-11-18 4:37 ` [PATCH manager v3 31/38] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 33/38] ui: guest summary: show " Thomas Lamprecht
` (7 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
The migrate API refuses target nodes that run a Proxmox VE version too
old to keep the guest identity, as they drop it on their next write of
the guest config. The precondition endpoints report such nodes with the
reported and the required version, but the migrate window ignored that:
for a stopped VM, the target was only marked as not allowed without a
reason, while for a running VM and for containers the window offered
the migration and only showed the API error after pressing Migrate.
Show the reason as an error, like for other blocking preconditions, and
disable the Migrate button for such a target. The API has no override
for this check, so the window offers none either.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
no changes since v1
www/manager6/window/Migrate.js | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)
diff --git a/www/manager6/window/Migrate.js b/www/manager6/window/Migrate.js
index 23090b7a..61683921 100644
--- a/www/manager6/window/Migrate.js
+++ b/www/manager6/window/Migrate.js
@@ -214,6 +214,31 @@ Ext.define('PVE.window.Migrate', {
me.lookup('formPanel').isValid();
},
+ checkGuestIdentitySupport: function (migration, disallowed) {
+ let unsupported = disallowed['guest-identity-unsupported'];
+ if (!unsupported) {
+ return;
+ }
+
+ let text;
+ if (unsupported.version) {
+ text = Ext.String.format(
+ gettext(
+ 'The selected target node runs Proxmox VE {0}, but keeping the guest identity requires {1} or newer. Upgrade the target node first.',
+ ),
+ unsupported.version,
+ unsupported['required-version'],
+ );
+ } else {
+ text = gettext(
+ 'The selected target node does not report its Proxmox VE version, so support for guest identities cannot be verified.',
+ );
+ }
+
+ migration.possible = false;
+ migration.preconditions.push({ text, severity: 'error' });
+ },
+
checkQemuPreconditions: async function (resetMigrationPossible) {
let me = this,
vm = me.getViewModel(),
@@ -301,6 +326,8 @@ Ext.define('PVE.window.Migrate', {
migration.preconditions.push({ text, severity: 'error' });
}
+ me.checkGuestIdentitySupport(migration, disallowed);
+
let blockingResources = [];
let mappedResources = migrateStats['mapped-resource-info'] ?? {};
@@ -517,6 +544,8 @@ Ext.define('PVE.window.Migrate', {
let targetNode = me.lookup('pveNodeSelector').value;
let disallowed = migrateStats['not-allowed-nodes']?.[targetNode] ?? {};
+ me.checkGuestIdentitySupport(migration, disallowed);
+
let blockingHAResources = disallowed['blocking-ha-resources'] ?? [];
if (blockingHAResources.length) {
migration.possible = false;
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH manager v3 33/38] ui: guest summary: show the guest identity
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (31 preceding siblings ...)
2026-11-18 4:37 ` [PATCH manager v3 32/38] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 34/38] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
` (6 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
The 'meta' property of a guest now holds its identity, a unique ID that
restores and migrations from other clusters keep where possible, and
the time the guest was registered in this cluster, next to the creation
time and, for VMs, the QEMU version the VM was created with. The web UI
showed none of it, so comparing a guest with what a backup or monitoring
tool reports for it required reading the config or the API.
Show the unique ID as a quiet row in the status panel of the guest
summary, also for templates, as it is the value such tools report. A
button next to it opens a window with all these values and, for users
with Sys.Audit on '/', the cluster ID. Both IDs are in read-only text
fields, so they can be selected and copied.
The row uses the uid that the guest status already returns, so the
summary makes no extra periodic request, and the window loads the guest
config and the cluster status once when opened. A failure to load the
cluster status only leaves out the cluster ID. Guests without a uid,
for example on nodes that do not handle the identity yet, get no row.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
no changes since v1
www/manager6/Makefile | 1 +
www/manager6/panel/GuestIdentity.js | 142 +++++++++++++++++++++++
www/manager6/panel/GuestStatusView.js | 3 +
www/manager6/panel/TemplateStatusView.js | 3 +
4 files changed, 149 insertions(+)
create mode 100644 www/manager6/panel/GuestIdentity.js
diff --git a/www/manager6/Makefile b/www/manager6/Makefile
index e8e2bd0b..24ad74cb 100644
--- a/www/manager6/Makefile
+++ b/www/manager6/Makefile
@@ -113,6 +113,7 @@ JSSRC= \
panel/IPView.js \
panel/RunningChart.js \
panel/StatusPanel.js \
+ panel/GuestIdentity.js \
panel/GuestStatusView.js \
panel/GuestSummary.js \
panel/TemplateStatusView.js \
diff --git a/www/manager6/panel/GuestIdentity.js b/www/manager6/panel/GuestIdentity.js
new file mode 100644
index 00000000..d67ec7d4
--- /dev/null
+++ b/www/manager6/panel/GuestIdentity.js
@@ -0,0 +1,142 @@
+Ext.define('PVE.window.GuestIdentity', {
+ extend: 'Ext.window.Window',
+ alias: 'widget.pveGuestIdentityWindow',
+
+ title: gettext('Guest Identity'),
+ width: 450,
+ modal: true,
+ resizable: false,
+ bodyPadding: 10,
+
+ layout: 'anchor',
+ defaults: {
+ anchor: '100%',
+ labelWidth: 130,
+ },
+
+ // the parsed 'meta' property of the guest config
+ meta: {},
+ clusterId: undefined,
+
+ initComponent: function () {
+ let me = this;
+
+ let meta = me.meta;
+ let renderTime = (seconds) =>
+ Ext.isNumeric(seconds) ? PVE.Utils.render_timestamp_human_readable(seconds) : '';
+
+ let fields = [
+ {
+ xtype: 'textfield',
+ itemId: 'uid',
+ fieldLabel: gettext('Unique ID'),
+ value: meta.uid,
+ editable: false,
+ },
+ {
+ xtype: 'displayfield',
+ itemId: 'rtime',
+ fieldLabel: gettext('Registered'),
+ value: renderTime(meta.rtime / 1000),
+ },
+ {
+ xtype: 'textfield',
+ itemId: 'cluster-id',
+ fieldLabel: gettext('Cluster ID'),
+ value: me.clusterId,
+ editable: false,
+ },
+ {
+ xtype: 'displayfield',
+ itemId: 'ctime',
+ fieldLabel: gettext('Created'),
+ value: renderTime(meta.ctime),
+ },
+ {
+ xtype: 'displayfield',
+ itemId: 'creation-qemu',
+ fieldLabel: gettext('Created with QEMU'),
+ value: meta['creation-qemu'],
+ },
+ ];
+ me.items = fields.filter((field) => (field.value ?? '') !== '');
+
+ me.callParent();
+ },
+});
+
+Ext.define('PVE.panel.GuestIdentityInfo', {
+ extend: 'Ext.container.Container',
+ alias: 'widget.pveGuestIdentityInfo',
+
+ hidden: true,
+ layout: {
+ type: 'hbox',
+ align: 'middle',
+ },
+
+ items: [
+ {
+ xtype: 'pmxInfoWidget',
+ flex: 1,
+ iconCls: 'fa fa-id-card-o fa-fw',
+ title: gettext('Unique ID'),
+ printBar: false,
+ textField: 'uid',
+ // the status view only updates info widgets, so this one shows or hides the whole row
+ renderer: function (uid) {
+ this.up('pveGuestIdentityInfo').setHidden(!uid);
+ return Ext.htmlEncode(uid);
+ },
+ },
+ {
+ xtype: 'button',
+ iconCls: 'fa fa-info-circle',
+ tooltip: gettext('Details'),
+ ariaLabel: gettext('Details'),
+ ui: 'default-toolbar',
+ margin: '0 0 0 5',
+ handler: function () {
+ this.up('pveGuestIdentityInfo').showDetails();
+ },
+ },
+ ],
+
+ showDetails: async function () {
+ let me = this;
+
+ let view = me.up('pmxStatusView');
+ let { node, vmid, type } = view.pveSelNode.data;
+ let caps = Ext.state.Manager.get('GuiCap');
+
+ let [config, status] = await Promise.allSettled([
+ // not the pending values: an unfinished restore queues the uid of its backup there
+ Proxmox.Async.api2({
+ url: `/nodes/${node}/${type}/${vmid}/config`,
+ params: { current: 1 },
+ method: 'GET',
+ waitMsgTarget: view,
+ }),
+ // the cluster ID is optional, so a missing privilege or a failure only omits it
+ caps.dc['Sys.Audit']
+ ? Proxmox.Async.api2({ url: '/cluster/status', method: 'GET', waitMsgTarget: view })
+ : undefined,
+ ]);
+
+ if (config.status === 'rejected') {
+ Ext.Msg.alert(gettext('Error'), config.reason.htmlStatus);
+ return;
+ }
+
+ let meta = PVE.Parser.parsePropertyString(config.value.result.data.meta) ?? {};
+ // a standalone node reports the ID on its node entry, a cluster only on the cluster entry
+ let statusEntries = status.value?.result?.data ?? [];
+ let clusterId = statusEntries.find((entry) => entry['cluster-id'])?.['cluster-id'];
+
+ Ext.create('PVE.window.GuestIdentity', {
+ autoShow: true,
+ meta,
+ clusterId,
+ });
+ },
+});
diff --git a/www/manager6/panel/GuestStatusView.js b/www/manager6/panel/GuestStatusView.js
index 315d3420..00851b07 100644
--- a/www/manager6/panel/GuestStatusView.js
+++ b/www/manager6/panel/GuestStatusView.js
@@ -92,6 +92,9 @@ Ext.define('PVE.panel.GuestStatusView', {
hidden: '{isQemu}',
},
},
+ {
+ xtype: 'pveGuestIdentityInfo',
+ },
{
xtype: 'box',
height: 10,
diff --git a/www/manager6/panel/TemplateStatusView.js b/www/manager6/panel/TemplateStatusView.js
index 8742cfc2..26732386 100644
--- a/www/manager6/panel/TemplateStatusView.js
+++ b/www/manager6/panel/TemplateStatusView.js
@@ -30,6 +30,9 @@ Ext.define('PVE.panel.TemplateStatusView', {
iconCls: 'fa fa-fw fa-building',
title: gettext('Node'),
},
+ {
+ xtype: 'pveGuestIdentityInfo',
+ },
{
xtype: 'box',
height: 20,
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH manager v3 34/38] api: bulk migrate: refuse targets that would drop the guest identity
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (32 preceding siblings ...)
2026-11-18 4:37 ` [PATCH manager v3 33/38] ui: guest summary: show " Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 35/38] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
` (5 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
The migration of each guest refuses a target node that does not run a
version keeping the guest identity. A bulk migration to such a node thus
logged one warning per guest and only failed after it had tried all of
them. Check the target once before starting the task, for the migration
of all guests of a node and for the cluster-wide bulk migration. Check
there first that the target node exists, as the version check would
report a mistyped node as one without version information.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* also check the target of the cluster-wide bulk migration once before
the task starts, after checking that the node exists; subject changed
accordingly
changes v1 -> v2:
* new in v2
PVE/API2/Cluster/BulkAction/Guest.pm | 5 +++++
PVE/API2/Nodes.pm | 3 +++
2 files changed, 8 insertions(+)
diff --git a/PVE/API2/Cluster/BulkAction/Guest.pm b/PVE/API2/Cluster/BulkAction/Guest.pm
index a55ea2ea..8ae5e6a6 100644
--- a/PVE/API2/Cluster/BulkAction/Guest.pm
+++ b/PVE/API2/Cluster/BulkAction/Guest.pm
@@ -6,6 +6,7 @@ use warnings;
use PVE::APIClient::LWP;
use PVE::Cluster;
use PVE::Exception qw(raise raise_perm_exc raise_param_exc);
+use PVE::GuestIdentity;
use PVE::INotify;
use PVE::JSONSchema qw(get_standard_option);
use PVE::RESTEnvironment qw(log_warn);
@@ -760,6 +761,10 @@ __PACKAGE__->register_method({
check_guest_permissions($rpcenv, $authuser, $vmlist, ['VM.Migrate']);
+ PVE::Cluster::check_node_exists($param->{target});
+ # refuse once instead of failing the migration of every guest
+ PVE::GuestIdentity::assert_node_supports_identity($param->{target});
+
my $code = sub {
my $list =
PVE::API2::Nodes::Nodeinfo::get_filtered_vmlist(undef, $vmlist_string, 1, 1);
diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index 1e2a96f1..a86dcd04 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -23,6 +23,7 @@ use PVE::Cluster qw(cfs_read_file);
use PVE::DataCenterConfig;
use PVE::Exception qw(raise raise_perm_exc raise_param_exc);
use PVE::Firewall;
+use PVE::GuestIdentity;
use PVE::HA::Config;
use PVE::HA::Env::PVE2;
use PVE::INotify;
@@ -2759,6 +2760,8 @@ __PACKAGE__->register_method({
PVE::Cluster::check_cfs_quorum();
PVE::Cluster::check_node_exists($target);
+ # refuse once instead of failing the migration of every guest
+ PVE::GuestIdentity::assert_node_supports_identity($target);
my $datacenterconfig = cfs_read_file('datacenter.cfg');
# prefer parameter over datacenter cfg settings
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH docs v3 35/38] vzdump: describe the guest identity of VMs and containers
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (33 preceding siblings ...)
2026-11-18 4:37 ` [PATCH manager v3 34/38] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 36/38] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
` (4 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Since Proxmox VE 9.3, VMs and containers have a guest identity in
'meta': a random uid and a registration time. Backup software and other
tools can distinguish a guest from another that reuses its VMID [0].
Administrators need to understand why restores over a different guest
are refused, when a restore keeps the uid of the backup, and why
migrations and clones to older nodes are blocked. Those nodes drop
'meta', including VM creation information used for hardware defaults.
Document the lifecycle, failed-restore retries, and mixed-version
restrictions in the backup and restore chapter, where users are most
likely to encounter the identity. Include migration and HA behavior
there so the rules can be read together.
[0]: https://bugzilla.proxmox.com/show_bug.cgi?id=7846
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* backups are still grouped by type and VMID, also for retention
* refused remote migration: do not remove the reported guest if it is
the source itself
* own section for failed restores, with what a killed or late-failing
restore leaves behind
* remote migration of a guest without uid, HA recovery through the node
of the active CRM, and smaller corrections (the uid gets compared, not
the identity)
changes v1 -> v2:
* say that the uid has 100 random bits and how rtime differs from ctime
* describe the refusal of a migration to another cluster, its typical
cases and 'allow-new-uid'
* retry after a failed restore: scope the pending entry, name the
warning, correct the placeholder case and add the note for enforced
unique guest IDs
vzdump.adoc | 332 ++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 332 insertions(+)
diff --git a/vzdump.adoc b/vzdump.adoc
index 44db027..1d74602 100644
--- a/vzdump.adoc
+++ b/vzdump.adoc
@@ -464,6 +464,13 @@ following CLI tools:
For details see the corresponding manual pages.
+When you restore a backup over an existing guest, {pve} compares the `uid` of
+the backup and of the guest. If they differ, the restore is refused, unless you
+set the `allow-identity-mismatch` option. The existing guest then keeps its
+identity. See
+xref:guest_identity_restore[Restore Over an Existing Guest] for details and
+exceptions.
+
Bandwidth Limit
~~~~~~~~~~~~~~~
@@ -563,6 +570,331 @@ file in `/var/log/proxmox-backup/file-restore/`. The log file might contain
additional information in case an attempt to restore individual files or
accessing file systems contained in a backup archive fails.
+[[guest_identity]]
+Guest Identity
+--------------
+
+Since {pve} 9.3, each virtual machine and container gets a guest identity: a
+random identifier, `uid`, and a registration time, `rtime`. Unlike a VMID,
+which can be reused after a guest is destroyed, the `uid` lets backup software
+and other tools recognize the same logical guest across restores and
+migrations, even under a different VMID. Whether an operation keeps the `uid`
+depends on the target and the options of the operation, as described in
+xref:guest_identity_lifecycle[Lifecycle].
+
+Backups are still grouped by guest type and VMID, also for retention. If a new
+guest reuses the VMID of a removed guest of the same type, pruning handles the
+backups of both guests together.
+
+[[guest_identity_values]]
+Identity Values
+~~~~~~~~~~~~~~~
+
+The guest identity is part of the `meta` property in the guest configuration.
+This property also contains the creation time `ctime` and, for VMs, the QEMU
+version that the VM was created with, `creation-qemu`:
+
+----
+meta: creation-qemu=10.1.2,ctime=1772444467,rtime=1794320571482,uid=w5qczgiem26qnwzwsoil
+----
+
+`uid`::
+
+A random identifier of 20 characters, consisting of the lowercase letters `a` to
+`z` and the digits `2` to `7`. It encodes 100 random bits, so two guests
+practically never get the same `uid` by chance.
+
+`rtime`::
+
+The time at which the guest was registered in this cluster, in milliseconds
+since the UNIX epoch. Creating or cloning a guest, restoring it to a free VMID,
+and migrating it from another cluster register the guest in the cluster. A
+guest restored to a free VMID gets a new `rtime`, even if it keeps the backup's
+`uid`. See xref:guest_identity_lifecycle[Lifecycle] for the other operations.
+As a timestamp, the `rtime` is not guaranteed to be unique.
+
+The `rtime` differs from the creation time `ctime`. The `ctime` stays with the
+guest: a clone has the `ctime` of its source, and a restored guest or a guest
+migrated from another cluster has that of the backup or of the source guest. The
+`rtime` is set whenever the guest gets its `uid` in this cluster. A guest whose
+`uid` is unchanged but whose `rtime` is newer was therefore registered again,
+for example by a restore to a free VMID.
+
+{pve} manages the `meta` property automatically. It is read-only through the
+API, even for `root@pam`.
+
+IMPORTANT: The `uid` helps to correlate a guest with its backups and copies,
+but does not prove where a guest comes from. Whoever restores a backup or
+migrates a guest into the cluster supplies the `uid` of that guest, which {pve}
+takes over if no other guest in the cluster has it. {pve} keeps a `uid` unique
+within a cluster only for operations through its API. A configuration file
+copied by hand has the same `uid` as the original, and guests in different
+clusters can have the same `uid`.
+
+[[guest_identity_read]]
+Reading the Identity
+~~~~~~~~~~~~~~~~~~~~
+
+The following places show the guest identity. All of them require the
+`VM.Audit` privilege on the guest.
+
+* In the web interface, the 'Summary' panel of the guest shows the `uid` as
+ 'Unique ID'. The 'Details' button next to it, which has an info icon, opens
+ the 'Guest Identity' window. This window additionally shows the `rtime` as
+ 'Registered' and the creation information.
+* The guest configuration, for example with `qm config <vmid>` or
+ `pct config <vmid>`, and through the API paths
+ `/nodes/<node>/qemu/<vmid>/config` and `/nodes/<node>/lxc/<vmid>/config`.
+* The guest status at the API paths `/nodes/<node>/qemu/<vmid>/status/current`
+ and `/nodes/<node>/lxc/<vmid>/status/current`, which contains the `uid`.
+* The cluster resources at the API path `/cluster/resources`, which contain the
+ `uid` and `rtime` of each guest, for example:
++
+----
+# pvesh get /cluster/resources --type vm --output-format json-pretty
+----
+
+The configuration in a backup contains the `meta` property as well. You can
+show it with `pvesm extractconfig <volume>`, for example.
+
+[[guest_identity_lifecycle]]
+Lifecycle
+~~~~~~~~~
+
+The following table shows how operations affect the identity of a guest.
+
+[width="100%",cols="3,4,2",options="header"]
+|===========================================================================
+|Operation |`uid` |`rtime`
+|Create, including import |new |current time
+|Clone, including linked clones and clones from snapshots |new |current time
+|Restore to a free VMID |the backup's `uid` if present and not used by another
+guest in the cluster; otherwise new |current time
+|Restore to a free VMID with `unique` set |new |current time
+|Restore over an existing guest that has a `uid`, with or without `unique` set
+|kept, see xref:guest_identity_restore[Restore Over an Existing Guest] |kept
+|Restore over an existing guest without a `uid` |as for a free VMID |current
+time
+|Migration to another cluster, on the target |the source guest's `uid`; new if
+another guest in the target cluster has it and `allow-new-uid` is set, or if the
+source node runs a {pve} version older than 9.3, see
+xref:guest_identity_remote_migration[Migration to Another Cluster] |current time
+|Migration to another cluster, on the source if the guest is kept |kept |kept
+|Snapshot rollback |kept, not taken from the snapshot |kept
+|Migration within the cluster, HA recovery, configuration change, backup,
+replication, conversion to a template |kept |kept
+|Destroy |removed |removed
+|===========================================================================
+
+Restores and migrations from another cluster report the outcome in their task
+log, in a line with `guest identity:`, for example:
+
+----
+guest identity: backup uid w5qczgiem26qnwzwsoil is in use by guest 100, keeping uid er3n3wqoqgcl362ep3da
+----
+
+When a restore takes over a `uid`, the guest first has a newly assigned `uid`,
+which the restore replaces with the `uid` of the backup at its end. Tools that
+read `/cluster/resources` during a restore can therefore see a `uid` that
+disappears afterwards, also on a guest that is no longer locked. A migration
+from another cluster takes over the `uid` of the source guest right at its
+start. In both cases, a changed `uid` does not mean that the task has finished.
+
+Guests that existed before the upgrade to {pve} 9.3 get their identity the
+first time a node running {pve} 9.3 or later writes their configuration. This
+happens, for example, when their configuration changes, when they are backed
+up, or when they are migrated to such a node. Their `rtime` is the time of that
+write. Backups made before that have no `uid`.
+
+When a tool restores a guest by removing it and creating a new one through the
+API, the new guest gets a new `uid`. To keep the identity, a tool has to change
+the existing guest in place, or let {pve} restore the guest.
+
+[[guest_identity_restore]]
+Restore Over an Existing Guest
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+If the existing guest already has a guest identity, restoring a backup over it
+keeps the `uid` and `rtime` of that guest. When both the guest and the backup
+have a `uid` and the values differ, {pve} treats the backup as belonging to
+another guest. This can happen when a VMID was reused. {pve} refuses such a
+restore before it overwrites the guest:
+
+----
+backup belongs to another guest: its uid 'w5qczgiem26qnwzwsoil' differs from uid 'er3n3wqoqgcl362ep3da' of the existing VM 100 - use 'allow-identity-mismatch' to overwrite the VM anyway
+----
+
+To restore such a backup anyway, set the `allow-identity-mismatch` option. The
+existing guest then keeps its identity. In the web interface, select 'Allow
+restoring a backup of another guest' in the restore window, which you open from
+the 'Backup' panel of the guest. On the command line, use for example:
+
+----
+# qmrestore <archive> <vmid> --force 1 --allow-identity-mismatch 1
+# pct restore <vmid> <archive> --force 1 --allow-identity-mismatch 1
+----
+
+Backups without a `uid`, such as backups made with {pve} versions older than
+9.3, are not refused. Container restores from standard input
+(`pct restore <vmid> -`) are an exception: the `uid` of such a backup is only
+known once the container has been overwritten. A restore from standard input
+over a container with a `uid` therefore always requires
+`allow-identity-mismatch`.
+
+[[guest_identity_failed_restore]]
+Failed Restores
+~~~~~~~~~~~~~~~
+
+A restore that fails or is stopped can leave the guest behind, for example a
+restore over an existing guest, or a VM restore that fails after it restored the
+data or in the live phase of a live restore. A container restore over an
+existing container that fails after it removed the old volumes removes the
+container. If the restore task was killed, the guest can also stay locked.
+Remove the lock with `qm unlock <vmid>` or `pct unlock <vmid>` before you retry
+the restore or remove the guest.
+
+If the failed restore was to a free VMID or over a guest without a `uid`, and it
+had already read a backup that has a `uid`, `qm pending <vmid>` or
+`pct pending <vmid>` lists an entry for `meta` with the `uid` of the backup. As
+long as this entry exists, retrying the same restore over the guest left behind
+is accepted. Starting the guest or changing its options removes the entry and
+logs a warning that names the `uid`. The retry is then refused like a restore of
+another guest's backup.
+
+A successful retry takes over the `uid` of the backup and sets a new `rtime`, as
+a restore to a free VMID does. The guest keeps its `uid` and `rtime` instead if
+another guest in the cluster has the `uid` of the backup, or if the retry sets
+`unique`. Restores from standard input still require `allow-identity-mismatch`,
+as described above.
+
+A guest that had a `uid` and no such entry before the failed restore keeps its
+`uid` and gets no entry. A retry is checked like the first attempt: a backup
+with the same `uid` is accepted, also after the guest was started or changed,
+and a backup with another `uid` requires `allow-identity-mismatch`.
+
+A restore to a free VMID that fails before it restored the data normally
+removes the guest again. If the restore task was killed, or if that removal
+failed, an empty placeholder can stay at that VMID instead. Unlock it as
+described above if it is locked, and remove it, for example with
+`qm destroy <vmid>` or `pct destroy <vmid>`. Add
+`--destroy-unreferenced-disks 1` to also remove volumes that a killed restore
+created for that VMID. In both cases, restore again. Use
+another VMID if the xref:pvecm_next_id_range[`next-id`] setting of the
+datacenter has the `unique` and `enforce` options set, as the first VMID then
+counts as used.
+
+If a restore over an existing guest without a `uid` stopped before it read the
+backup, for example because the backup storage was unreachable, the guest now
+has a new `uid`. Retry with `allow-identity-mismatch`. The guest then keeps the
+new `uid`.
+
+A restore can also complete with a warning starting with
+`guest identity: unable to adopt backup uid`, for example if the cluster-wide
+lock for this step could not be taken in time. The guest is then restored
+completely, but keeps its new `uid`. To take over the `uid` of the backup,
+repeat the restore over the guest before you start it or change its
+configuration.
+
+[[guest_identity_remote_migration]]
+Migration to Another Cluster
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+A migration to another cluster (`qm remote-migrate`, `pct remote-migrate`) keeps
+the `uid` of the guest. The target takes it over at the start of the migration,
+before any data is transferred.
+
+If another guest in the target cluster already has that `uid`, the migration is
+refused at that point. Typical cases are:
+
+* the guest gets migrated back to a cluster that still has the source guest of
+ an earlier migration, which stays there unless `delete` was set,
+* a copy of the guest was restored in the target cluster from a backup,
+* an earlier migration of the guest broke off after it had transferred the
+ configuration and left a locked guest in the target cluster,
+* the target is the cluster that the guest is in.
+
+The error ends with:
+
+----
+guest 105 in the target cluster already has the uid 'w5qczgiem26qnwzwsoil' of this guest - use 'allow-new-uid' to migrate it with a new uid
+----
+
+If the reported guest is the source guest itself, do not remove it. To keep the
+`uid`, migrate the guest within the cluster instead, with `qm migrate` or
+`pct migrate`.
+
+Otherwise, to keep the `uid`, remove the other guest in the target cluster and
+migrate again. To migrate anyway, set the `allow-new-uid` option. The migrated
+guest then gets a new `uid`, and the other guest keeps the old one. In both
+cases, use another target VMID if the xref:pvecm_next_id_range[`next-id`]
+setting of the target cluster has the `unique` and `enforce` options set, as the
+refused migration already used the first one. For example:
+
+----
+# qm remote-migrate <vmid> <target-vmid> <target-endpoint> --target-bridge <bridge> --target-storage <storage> --allow-new-uid 1
+----
+
+[[guest_identity_mixed_versions]]
+Mixed Versions in a Cluster
+~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+While some nodes of a cluster still run a {pve} version older than 9.3,
+migrating or cloning a guest to one of these nodes is refused. The guest keeps
+running on its current node. If the target node reports its version, the error
+message shows that version and the minimum required version. If the node does
+not report its version, the operation is refused because {pve} cannot verify
+that the node supports guest identities. The 'Migrate' window of the web
+interface shows the reason for the selected target node.
+
+Moves of HA-managed guests to such nodes are refused in the same way. A running
+VM that the HA stack migrates keeps running. The HA stack stops a container, and
+a VM that it relocates, before it migrates the guest, so such a guest is stopped
+although the move is then refused. The HA stack retries automatic moves as long
+as it selects such a node, and such a guest can stay stopped in the meantime.
+This includes moves for:
+
+* the maintenance mode,
+* a node shutdown with the `migrate` shutdown policy,
+* node affinity rules,
+* the CRS load balancer.
+
+Such moves complete only after the target nodes that the HA stack selects run
+{pve} 9.3 or later.
+
+To avoid refused moves while you upgrade a cluster, do one of the following:
+
+* Move all guests away from a node before you upgrade it.
+* Upgrade the packages on all nodes, one node at a time, before you reboot any
+ node or enable the maintenance mode.
+
+The reason for these refusals is that {pve} versions older than 9.3 ignore the
+whole `meta` property of a guest that has a guest identity, including the
+creation information. A VM started on such a node can therefore get different
+virtual hardware, see xref:qm_machine_type[Machine Type]. When such a node
+writes the configuration, the property is lost.
+
+The check covers migrations and clones started on a node running {pve} 9.3 or
+later. It does not cover:
+
+* the HA recovery of guests from a fenced node,
+* configuration files moved by hand,
+* operations started on a node with an older version.
+
+If an older node writes the configuration of a guest in one of these cases, the
+guest loses its `meta` property. In an HA recovery, such a node can also be the
+node of the active HA manager (CRM), which writes the configuration when it
+removes a leftover lock, for example of a backup. The guest gets a new identity
+the next time a node running {pve} 9.3 or later writes its configuration.
+
+Backups made with {pve} 9.3 or later contain the guest identity. Restore them
+only on nodes running {pve} 9.3 or later, because older versions ignore the
+`meta` property of such backups in the same way.
+
+A migration to another cluster (`qm remote-migrate`, `pct remote-migrate`) is
+refused right after connecting to a target node with a version older than 9.3,
+before any data is transferred. The error is
+`Remote tunnel endpoint too old, upgrade required`.
+
[[vzdump_configuration]]
Configuration
-------------
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH docs v3 36/38] pvecm: document the cluster ID and how to get a new one
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (34 preceding siblings ...)
2026-11-18 4:37 ` [PATCH docs v3 35/38] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 37/38] qm: configuration: describe the read-only meta property Thomas Lamprecht
` (3 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Since Proxmox VE 9.3, clusters and standalone nodes have a random ID in
/etc/pve/cluster-id. Tools need it alongside the guest uid because
guests in different clusters can share a uid, for example after a
remote migration that keeps the source guest.
Explain where to read the ID, why generation can be delayed, and how
creating or joining a cluster affects it. pmxcfs protects the file
against ordinary changes from user space.
Cloned installations and nodes separated without reinstalling keep
their old ID. Document removal in forced local mode to get a new
one, and include it in the existing node-separation procedure. Warn
against doing this on a cluster member: the removal can propagate and
cause the entire cluster to generate a new ID.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* the file can exist without a valid ID, and forced local mode also
allows moving it away
no changes from v1 to v2
pmxcfs.adoc | 1 +
pvecm.adoc | 87 +++++++++++++++++++++++++++++++++++++++++++++++++++++
vzdump.adoc | 3 ++
3 files changed, 91 insertions(+)
diff --git a/pmxcfs.adoc b/pmxcfs.adoc
index 1128425..281dfa6 100644
--- a/pmxcfs.adoc
+++ b/pmxcfs.adoc
@@ -95,6 +95,7 @@ Files
|=======
|`authkey.pub` | Public key used by the ticket system
|`ceph.conf` | Ceph configuration file (note: /etc/ceph/ceph.conf is a symbolic link to this)
+|`cluster-id` | Random ID of the cluster, generated by pmxcfs and read-only (see xref:guest_identity_cluster_id[Cluster ID])
|`corosync.conf` | Corosync cluster configuration file (prior to {pve} 4.x, this file was called cluster.conf)
|`datacenter.cfg` | {pve} datacenter-wide configuration (keyboard layout, proxy, ...)
|`domains.cfg` | {pve} authentication domains
diff --git a/pvecm.adoc b/pvecm.adoc
index cd5a0fb..186fbb6 100644
--- a/pvecm.adoc
+++ b/pvecm.adoc
@@ -529,6 +529,14 @@ rm /etc/pve/corosync.conf
rm -r /etc/corosync/*
----
+Remove the xref:guest_identity_cluster_id[cluster ID], if the cluster has one.
+Without this step, the node keeps the ID of its former cluster instead of
+generating its own when the file system starts again as a normal service:
+[source,bash]
+----
+rm /etc/pve/cluster-id
+----
+
You can now start the file system again as a normal service:
[source,bash]
----
@@ -572,6 +580,85 @@ authentication. You should fix this by removing the respective keys from the
'/etc/pve/priv/authorized_keys' file.
+[[guest_identity_cluster_id]]
+Cluster ID
+----------
+
+Since {pve} 9.3, each cluster has a random cluster ID in the same format as the
+`uid` of a xref:guest_identity[guest identity]. A node that is not part of a
+cluster has an ID of its own. Guests in different clusters can have the same
+`uid`, for example after a migration to another cluster that kept the source
+guest. The cluster ID tells tools to which cluster the `uid` of a guest belongs.
+
+The cluster ID is stored in `/etc/pve/cluster-id`. Through the API,
+`/cluster/status` returns it in the `cluster-id` field of the entry of type
+`cluster`. On a node without a cluster, the field is on the node entry instead.
+Reading the cluster ID through the API requires the `Sys.Audit` privilege on
+`/`. Users with this privilege also see it in the 'Guest Identity' window of a
+guest in the web interface.
+
+----
+# cat /etc/pve/cluster-id
+----
+
+{pve} generates the cluster ID and keeps it unchanged. Creating, changing,
+moving or removing `/etc/pve/cluster-id` fails with `Permission denied`. The
+only exception is removing or moving it away in local mode (`pmxcfs -l`), as
+described below. Nodes running a {pve} version older than 9.3 do not protect the
+file.
+
+A node without a cluster generates its ID when its cluster file system starts.
+In a cluster, the online node with the lowest node ID generates the ID once all
+of the following apply:
+
+* The cluster is quorate.
+* More than half of the configured nodes are online.
+* The cluster file systems of the online nodes are in sync.
+
+That node must run {pve} 9.3 or later. While a node with an older version has
+the lowest online node ID, the cluster gets no ID. Until an ID exists, the API
+omits the field, and the file is missing or holds no valid ID. If the node with
+the lowest online node ID runs {pve} 9.3 or later but cannot generate the ID
+yet, its `pve-cluster` service logs the reason in a message starting with
+`not generating a cluster ID yet`.
+
+Creating a cluster with `pvecm create` keeps the ID of the node, which then
+identifies the new cluster. A node that joins a cluster takes over the ID of the
+cluster, because joining replaces the configuration of the node in `/etc/pve`.
+A copy of an installation, such as a cloned disk or a copy of the cluster file
+system database, has the same ID as the original.
+
+To give a node that is not part of a cluster a new ID, for example after cloning
+an installation, follow the steps below. To separate a node from its cluster,
+follow xref:pvecm_separate_node_without_reinstall[Separate a Node Without Reinstalling]
+instead. That procedure includes the removal of the cluster ID.
+
+WARNING: Do not remove `/etc/pve/cluster-id` on a node that is still part of a
+cluster. When that node rejoins the cluster, the removal is either undone, or it
+reaches all nodes and the whole cluster gets a new ID.
+
+. Stop the cluster file system and start it again in local mode:
++
+----
+# systemctl stop pve-cluster
+# pmxcfs -l
+----
+
+. Remove the cluster ID file:
++
+----
+# rm /etc/pve/cluster-id
+----
+
+. Stop the cluster file system and start it again as a normal service, which
+ generates a new ID:
++
+----
+# killall pmxcfs
+# systemctl start pve-cluster
+----
+
+
Quorum
------
diff --git a/vzdump.adoc b/vzdump.adoc
index 1d74602..f247edc 100644
--- a/vzdump.adoc
+++ b/vzdump.adoc
@@ -657,6 +657,9 @@ The following places show the guest identity. All of them require the
The configuration in a backup contains the `meta` property as well. You can
show it with `pvesm extractconfig <volume>`, for example.
+To tell apart guests with the same `uid` in different clusters, combine the
+`uid` with the xref:guest_identity_cluster_id[cluster ID].
+
[[guest_identity_lifecycle]]
Lifecycle
~~~~~~~~~
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH docs v3 37/38] qm: configuration: describe the read-only meta property
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (35 preceding siblings ...)
2026-11-18 4:37 ` [PATCH docs v3 36/38] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 38/38] pct: " Thomas Lamprecht
` (2 subsequent siblings)
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
The 'meta' property of VMs was not documented anywhere, as the generated
option reference leaves it out for being read-only. Since Proxmox VE
9.3, it also holds the guest identity, and it matters for upgrades:
older versions drop the whole property, including 'creation-qemu',
which keeps the machine version of Windows VMs and some other hardware
defaults stable.
Describe its members next to the snapshot properties, refer to the guest
identity section for 'uid' and 'rtime', and state why migrations and
clones to nodes with older versions are refused. Point to the
description from qm.conf(5), whose option list lacks the property, too.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* point to the description from qm.conf(5)
no changes from v1 to v2
qm.adoc | 38 ++++++++++++++++++++++++++++++++++++++
qm.conf.adoc | 3 +++
2 files changed, 41 insertions(+)
diff --git a/qm.adoc b/qm.adoc
index 350c6ef..487673b 100644
--- a/qm.adoc
+++ b/qm.adoc
@@ -2441,6 +2441,44 @@ For details about how the target storage gets chosen for the VM state, see
xref:qm_vmstatestorage[State storage selection] in the chapter
xref:qm_hibernate[Hibernation].
+[[qm_meta_info]]
+Meta Information
+~~~~~~~~~~~~~~~~
+
+The read-only `meta` property contains information that {pve} records
+automatically about the VM:
+
+`ctime`::
+
+The time the VM was created, in seconds since the UNIX epoch.
+
+`creation-qemu`::
+
+The QEMU version that the VM was created with. For VMs without a fixed machine
+version, {pve} uses it to keep parts of the virtual hardware as they were at
+creation, for example the machine version of Windows VMs, see
+xref:qm_machine_type[Machine Type].
+
+`uid` and `rtime`::
+
+The guest identity of the VM, available since {pve} 9.3, see
+xref:guest_identity[Guest Identity].
+
+.Example `meta` property of a VM
+----
+meta: creation-qemu=10.1.2,ctime=1772444467,rtime=1794320571482,uid=w5qczgiem26qnwzwsoil
+----
+
+A clone keeps its source VM's `ctime` and `creation-qemu` values. A restored VM
+uses the values from the backup. VMs created with older versions of {pve} can
+lack these values.
+
+{pve} versions older than 9.3 ignore the whole `meta` property of a VM with a
+guest identity, including `creation-qemu`, and drop it when they write the
+configuration. A VM started on a node running such a version can therefore get
+different virtual hardware. Migrating or cloning a VM to such a node is refused,
+see xref:guest_identity_mixed_versions[Mixed Versions in a Cluster].
+
[[qm_options]]
Options
~~~~~~~
diff --git a/qm.conf.adoc b/qm.conf.adoc
index 237ebbb..369675a 100644
--- a/qm.conf.adoc
+++ b/qm.conf.adoc
@@ -51,6 +51,9 @@ One can use the `qm` command to generate and modify those files.
Options
-------
+The read-only `meta` property is described in
+xref:qm_meta_info[Meta Information].
+
include::generated/qm.conf.5-opts.adoc[]
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* [PATCH docs v3 38/38] pct: configuration: describe the read-only meta property
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (36 preceding siblings ...)
2026-11-18 4:37 ` [PATCH docs v3 37/38] qm: configuration: describe the read-only meta property Thomas Lamprecht
@ 2026-11-18 4:37 ` Thomas Lamprecht
2026-11-18 13:49 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
2026-11-19 10:38 ` superseded: " Thomas Lamprecht
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 4:37 UTC (permalink / raw)
To: pve-devel
Since Proxmox VE 9.3, containers have a 'meta' property with their
creation time and guest identity. The generated option reference omits
this read-only property, so document its members alongside the other
configuration metadata, and point to them from pct.conf(5).
Explain that containers created with older versions have no recorded
creation time and that migration or cloning to older nodes is refused
to avoid losing the identity.
Also note that restoring from standard input over a container that
already has an identity requires 'allow-identity-mismatch'. The backup
uid is only known after the container has been overwritten, too late
for the normal mismatch check.
Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
changes v2 -> v3:
* point to the description from pct.conf(5)
no changes from v1 to v2
pct.adoc | 35 +++++++++++++++++++++++++++++++++++
pct.conf.adoc | 3 +++
2 files changed, 38 insertions(+)
diff --git a/pct.adoc b/pct.adoc
index 25dc1fe..b36086b 100644
--- a/pct.adoc
+++ b/pct.adoc
@@ -970,6 +970,10 @@ command line (see the `pct` manual page for details).
NOTE: `pvesm extractconfig` can be used to view the backed up configuration
contained in a vzdump archive.
+A restore from standard input over an existing container with a guest identity
+always needs the `allow-identity-mismatch` option, see
+xref:guest_identity_restore[Restore Over an Existing Guest].
+
There are two basic restore modes, only differing by their handling of mount
points:
@@ -1249,6 +1253,37 @@ There are a few snapshot related properties like `parent` and `snaptime`. The
snapshots. `snaptime` is the snapshot creation time stamp (Unix epoch).
+[[pct_meta_info]]
+Meta Information
+~~~~~~~~~~~~~~~~
+
+The read-only `meta` property contains information that {pve} records
+automatically about the container:
+
+`ctime`::
+
+The time the container was created, in seconds since the UNIX epoch.
+
+`uid` and `rtime`::
+
+The guest identity of the container, available since {pve} 9.3, see
+xref:guest_identity[Guest Identity].
+
+.Example `meta` property of a container
+----
+meta: ctime=1772444467,rtime=1794320571482,uid=w5qczgiem26qnwzwsoil
+----
+
+A clone keeps its source container's `ctime`. A restored container uses the
+`ctime` from the backup. Containers created with {pve} versions older than 9.3
+have no `ctime`.
+
+{pve} versions older than 9.3 do not know the `meta` property of containers and
+drop it when they write the configuration. Therefore, migrating or cloning a
+container to a node running such a version is refused, see
+xref:guest_identity_mixed_versions[Mixed Versions in a Cluster].
+
+
[[pct_options]]
Options
~~~~~~~
diff --git a/pct.conf.adoc b/pct.conf.adoc
index 103b1ef..eaf568d 100644
--- a/pct.conf.adoc
+++ b/pct.conf.adoc
@@ -62,6 +62,9 @@ Those settings are directly passed to the LXC low-level tools.
Options
-------
+The read-only `meta` property is described in
+xref:pct_meta_info[Meta Information].
+
include::generated/pct.conf.5-opts.adoc[]
--
2.47.3
^ permalink raw reply related [flat|nested] 42+ messages in thread
* Re: [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (37 preceding siblings ...)
2026-11-18 4:37 ` [PATCH docs v3 38/38] pct: " Thomas Lamprecht
@ 2026-11-18 13:49 ` Dominik Csapak
2026-11-18 14:13 ` Thomas Lamprecht
2026-11-19 10:38 ` superseded: " Thomas Lamprecht
39 siblings, 1 reply; 42+ messages in thread
From: Dominik Csapak @ 2026-11-18 13:49 UTC (permalink / raw)
To: Thomas Lamprecht, pve-devel
Read through it again (though mostly via diff from the v1)
and it looks good to me now:
* the rationale is given
* the flow is better visible (and with better wording)
* remote migrate is now blocked by default on conflicting uids
the biggest issue to discuss maybe still is if we want to allow
restore with a new uid in case the same uid exists already,
same argument as for the remote migration:
if the same vm already exists, restoring it again can have
bad implications like duplicate ips, etc.
but IMHO it's not a blocker, maybe even something the admin
would want to configure globally? (datacenter.cfg?)
one thing i noticed during testing was if i modified a guest
to have an invalid uid, on the next config write, the
ctime and creation-qemu vanished from the config and
only a new rtime and uid remained.
(though i had to manually edit the config, so not a problem IMO)
there are still some nits here and there, but they are mostly
taste questions and nothing that can't be improved further down the line
for example: when restoring a vm over another another, the failure i get is:
TASK ERROR: command 'set -o pipefail && zstd -q -d -c --
/var/lib/vz/dump/vzdump-qemu-9999-2026_11_18-14_37_58.vma.zst | vma
extract -v -r /var/tmp/vzdumptmp8272.fifo - /var/tmp/vzdumptmp8272'
failed: backup belongs to another guest: its uid '2xuof4nw6rnowo5mpgs2'
differs from uid 'itomggi4ofm33szeb3h2' of the existing VM 102 - use
'allow-identity-mismatch' to overwrite the VM anyway
which is not very readable in the task log window (still functional though)
also at least a few things in the ui i'm missing:
a way to show the cluster/node id (probably in the datacenter summary)
using the uid (maybe rtime) as variables in the templating for the notes
Tested it on a test cluster with some backup/restore cycles,
adopting of uid, manually editing configs, etc.
Since nothing of these blocks this series from my POV, consider it:
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
On 11/18/26 5:39 AM, Thomas Lamprecht wrote:
> A VMID can be reused for another guest, and a guest can get another
> VMID or move to another cluster through a restore or a remote
> migration. So backup software and other tools cannot tell whether they
> still look at the same guest, see [0].
>
> This series gives every VM and container a guest identity in the
> read-only 'meta' property: a random uid (100 bits as 20 base32
> characters) and the time the guest was registered in the cluster. Create
> and clone get a new uid. Config changes, migration, rollback, backup and
> restore keep it, also under another VMID or in another cluster, as long
> as no other guest there has it. Each cluster or standalone node also
> gets a cluster ID in the same format, generated and protected by pmxcfs.
> The docs patches describe the details.
>
> The uid is a plain random nonce: 100 bits make a duplicate practically
> impossible and fit into a 128-bit value with room to spare, and 20
> base32 characters stay short and need no quoting. The registration time
> is separate from the creation time, which clones, restores and remote
> migrations copy along. It changes when a guest gets registered again, so
> tools can tell such a guest from the one they knew.
>
> Three behavior changes come with it:
>
> * Restoring over an existing guest whose uid differs from the one of
> the backup is refused unless 'allow-identity-mismatch' is set. 'force'
> alone does not override this.
> * A migration to another cluster is refused if another guest there
> already has the uid, unless 'allow-new-uid' is set. The check runs
> before anything is transferred. PDM does not know the option yet.
> * Older versions drop the whole 'meta' property of a guest that has an
> identity. So migration and clone to such nodes is refused, and remote
> migration bumps the tunnel version. This also hits moves by the HA
> stack, which stops a container or a relocated VM before the migration
> gets refused. A target check in the LRM that avoids this will follow
> separately.
>
> The patches are in dependency order: guest-common needs the pve-cluster
> of this series, qemu-server and container need guest-common, and manager
> needs all of them. manager also has to depend on the qemu-server and
> container versions with this series, as the migration guard takes the
> pve-manager version of a node as proof that the node keeps the identity.
> MIN_PVE_MANAGER_VERSION in guest-common is a stand-in for that version.
>
> Tested with the affected tests of the repositories for every patch and
> their full suites at the end, pmxcfs with corosync in network
> namespaces, and end to end on a nested three-node cluster that was
> upgraded node by node, with remote migrations to a standalone node.
>
> changes v2 -> v3:
> * remote migration: a guest without uid gets one before the source
> sends it, so source and target no longer end up with unrelated uids;
> the identity command moved to pve-guest-common for both guest types
> (new patch 13)
> * restore: keep the guest if a restored volume or firewall config
> cannot be removed, fail if a container backup changes between its
> check and its extraction, and let the cleanup follow the reservation
> * reserving a VMID refuses an existing config that fails to load
> instead of replacing it
> * cluster ID: generate it right away when the configured nodes change
> (new patch 5)
> * feature API without version lookups for snapshot requests, and the
> cluster-wide bulk migration refuses its target up front
> * docs: failed restores, uid conflicts on migration, qm.conf(5) and
> pct.conf(5)
> * more independent fixes that the series builds on were applied on
> their own: failed and concurrent restores and creates, the schema of
> the migration preconditions, the lock of a VMA restore, a node list
> cut short in pmxcfs
>
> changes v1 -> v2, thanks to Dominik for the review:
> * rebase onto current master (tracking of used guest IDs, dfsm rework)
> * remote migration takes the uid over at its start and refuses a uid
> that is in use in the target cluster, unless 'allow-new-uid' is set
> * qemu-server and container share the restore checks and messages
> through pve-guest-common, with fewer special cases
> * IDs take each character from one random byte, without an encoder
> * the clone dialog does not offer nodes that would drop the identity,
> and a bulk migration checks its target once (patch 34)
> * independent fixes moved out of the series and got applied on their
> own: the IPC client error, the order of removing firewall and guest
> config, and the cleanup of a remote migration target after failures
> * comments, commit messages and docs give the reasons for the format,
> for rtime next to ctime and for the separate lock
> Details are in the notes below the "---" of each patch.
>
> v2: https://lore.proxmox.com/pve-devel/20261117223945.862914-1-t.lamprecht@proxmox.com/
> v1: https://lore.proxmox.com/pve-devel/20261116004703.2908294-1-t.lamprecht@proxmox.com/
>
> [0]: https://bugzilla.proxmox.com/show_bug.cgi?id=7846
>
> cluster:
>
> Thomas Lamprecht (8):
> pmxcfs: reject changes to the cluster ID file from user space
> pmxcfs: generate a cluster ID on standalone nodes
> pmxcfs: dfsm: add a callback that runs once all members are in sync
> pmxcfs: quorum: pass the corosync membership to an optional listener
> pmxcfs: confdb: call an optional listener when the nodes get published
> pmxcfs: dcdb: generate the cluster ID once the cluster is in sync
> cluster: add helper to read the cluster ID
> pmxcfs: allow removing the cluster ID file in forced local mode
>
> src/PVE/Cluster.pm | 16 +
> src/pmxcfs/Makefile | 12 +-
> src/pmxcfs/cfs-plug-memdb.c | 46 +++
> src/pmxcfs/cfs-utils.c | 8 +
> src/pmxcfs/cfs-utils.h | 5 +
> src/pmxcfs/check_dcdb.c | 563 ++++++++++++++++++++++++++++++++++++
> src/pmxcfs/check_memdb.c | 283 ++++++++++++++++++
> src/pmxcfs/cluster-id.c | 142 +++++++++
> src/pmxcfs/cluster-id.h | 50 ++++
> src/pmxcfs/confdb.c | 18 +-
> src/pmxcfs/confdb.h | 5 +-
> src/pmxcfs/dcdb.c | 153 ++++++++++
> src/pmxcfs/dcdb.h | 8 +
> src/pmxcfs/dfsm.c | 59 ++++
> src/pmxcfs/dfsm.h | 14 +
> src/pmxcfs/pmxcfs.c | 12 +-
> src/pmxcfs/quorum.c | 26 +-
> src/pmxcfs/quorum.h | 9 +-
> src/pmxcfs/status.c | 12 +
> src/pmxcfs/status.h | 3 +
> src/test/Makefile | 6 +-
> src/test/test_cluster_id.pl | 46 +++
> 22 files changed, 1478 insertions(+), 18 deletions(-)
> create mode 100644 src/pmxcfs/check_dcdb.c
> create mode 100644 src/pmxcfs/cluster-id.c
> create mode 100644 src/pmxcfs/cluster-id.h
> create mode 100644 src/test/test_cluster_id.pl
>
>
> guest-common:
>
> Thomas Lamprecht (5):
> guest identity: add helpers for the guest uid and registration time
> abstract config: assign a guest identity and keep it on config writes
> abstract config: let the VMID reservation return the previous state
> guest identity: allow taking over a uid that is unused in the cluster
> migrate: add shared helpers for the identity command of the migration tunnel
>
> src/Makefile | 1 +
> src/PVE/AbstractConfig.pm | 202 +++-
> src/PVE/AbstractMigrate.pm | 30 +
> src/PVE/GuestIdentity.pm | 510 ++++++++++
> src/tests/Makefile | 10 +-
> src/tests/config-identity-tests.pl | 1391 ++++++++++++++++++++++++++++
> src/tests/guest-identity-tests.pl | 530 +++++++++++
> 7 files changed, 2668 insertions(+), 6 deletions(-)
> create mode 100644 src/PVE/GuestIdentity.pm
> create mode 100755 src/tests/config-identity-tests.pl
> create mode 100755 src/tests/guest-identity-tests.pl
>
>
> qemu-server:
>
> Thomas Lamprecht (7):
> api: restore: release the VMID if a restore to an unused VMID fails
> api: migrate, clone: refuse targets that would drop the guest identity
> fix #7846: config: add a guest identity to the 'meta' property
> restore: take over the guest uid from the backup where possible
> api: remote migration: take over the guest uid from the source
> api: status: return the guest uid
> tests: remote migration: check a failed command and an early tunnel end
>
> src/PVE/API2/Qemu.pm | 118 +-
> src/PVE/CLI/qm.pm | 7 +
> src/PVE/CLI/qmrestore.pm | 11 +-
> src/PVE/QemuConfig.pm | 7 +
> src/PVE/QemuMigrate.pm | 18 +-
> src/PVE/QemuServer.pm | 155 ++-
> src/PVE/QemuServer/MetaInfo.pm | 2 +
> src/test/Makefile | 5 +-
> src/test/QemuServerTest/FakeCFS.pm | 189 +++
> .../q35-windows-pinning-identity-only.conf | 6 +
> ...q35-windows-pinning-identity-only.conf.cmd | 26 +
> .../meta-identity-invalid.conf | 6 +
> .../meta-identity-invalid.conf.strict.error | 2 +
> .../meta-identity-invalid.conf | 7 +
> .../parse-config-input/meta-identity.conf | 20 +
> src/test/restore-config-expected/150.conf | 9 +
> src/test/restore-config-input/150.conf | 10 +
> src/test/run_api_tests.pl | 206 ++++
> src/test/run_guest_identity_tests.pl | 803 +++++++++++++
> src/test/run_parse_config_tests.pl | 2 +-
> src/test/run_qemu_restore_config_tests.pl | 1053 ++++++++++++++++-
> .../create/qemu-server/401.conf | 31 +
> .../create/qemu-server/402.conf | 31 +
> .../rollback/qemu-server/401.conf | 36 +
> .../rollback/qemu-server/402.conf | 35 +
> .../create/qemu-server/401.conf | 13 +
> .../create/qemu-server/402.conf | 13 +
> .../rollback/qemu-server/401.conf | 35 +
> .../rollback/qemu-server/402.conf | 34 +
> src/test/snapshot-test.pm | 82 +-
> 30 files changed, 2899 insertions(+), 73 deletions(-)
> create mode 100644 src/test/QemuServerTest/FakeCFS.pm
> create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf
> create mode 100644 src/test/cfg2cmd/q35-windows-pinning-identity-only.conf.cmd
> create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf
> create mode 100644 src/test/parse-config-expected/meta-identity-invalid.conf.strict.error
> create mode 100644 src/test/parse-config-input/meta-identity-invalid.conf
> create mode 100644 src/test/parse-config-input/meta-identity.conf
> create mode 100644 src/test/restore-config-expected/150.conf
> create mode 100644 src/test/restore-config-input/150.conf
> create mode 100755 src/test/run_guest_identity_tests.pl
> create mode 100644 src/test/snapshot-expected/create/qemu-server/401.conf
> create mode 100644 src/test/snapshot-expected/create/qemu-server/402.conf
> create mode 100644 src/test/snapshot-expected/rollback/qemu-server/401.conf
> create mode 100644 src/test/snapshot-expected/rollback/qemu-server/402.conf
> create mode 100644 src/test/snapshot-input/create/qemu-server/401.conf
> create mode 100644 src/test/snapshot-input/create/qemu-server/402.conf
> create mode 100644 src/test/snapshot-input/rollback/qemu-server/401.conf
> create mode 100644 src/test/snapshot-input/rollback/qemu-server/402.conf
>
>
> container:
>
> Thomas Lamprecht (8):
> config: add read-only meta property with the creation time
> fix #7846: config: add a guest identity to the 'meta' property
> api: migrate, clone: refuse targets that would drop the guest identity
> restore: take over the guest uid from the backup where possible
> restore: refuse overwriting a container with another guest's backup
> test: remote migration: check the cleanup of the target after failures
> api: remote migration: take over the guest uid from the source
> api: status: return the guest uid
>
> src/PVE/API2/LXC.pm | 190 ++-
> src/PVE/API2/LXC/Config.pm | 29 +-
> src/PVE/API2/LXC/Status.pm | 6 +
> src/PVE/CLI/pct.pm | 7 +
> src/PVE/LXC/Config.pm | 51 +-
> src/PVE/LXC/Migrate.pm | 8 +-
> src/test/Makefile | 8 +-
> src/test/config_tests.pl | 23 +
> src/test/lifecycle_tests.pl | 2446 +++++++++++++++++++++++++++++++++++
> 9 files changed, 2730 insertions(+), 38 deletions(-)
> create mode 100755 src/test/lifecycle_tests.pl
>
>
> manager:
>
> Thomas Lamprecht (6):
> api: cluster resources: add guest uid and registration time
> api: cluster status: add cluster ID
> ui: restore: allow overwriting a guest with a backup of another guest
> ui: migrate: refuse target nodes that would drop the guest identity
> ui: guest summary: show the guest identity
> api: bulk migrate: refuse targets that would drop the guest identity
>
> PVE/API2/Cluster.pm | 24 ++-
> PVE/API2/Cluster/BulkAction/Guest.pm | 5 +
> PVE/API2/Nodes.pm | 3 +
> test/ClusterResources_test.pl | 249 +++++++++++++++++++++++
> test/ClusterStatus_test.pl | 114 +++++++++++
> test/Makefile | 10 +-
> www/manager6/Makefile | 1 +
> www/manager6/panel/GuestIdentity.js | 142 +++++++++++++
> www/manager6/panel/GuestStatusView.js | 3 +
> www/manager6/panel/TemplateStatusView.js | 3 +
> www/manager6/window/Migrate.js | 29 +++
> www/manager6/window/Restore.js | 18 ++
> 12 files changed, 599 insertions(+), 2 deletions(-)
> create mode 100755 test/ClusterResources_test.pl
> create mode 100755 test/ClusterStatus_test.pl
> create mode 100644 www/manager6/panel/GuestIdentity.js
>
>
> docs:
>
> Thomas Lamprecht (4):
> vzdump: describe the guest identity of VMs and containers
> pvecm: document the cluster ID and how to get a new one
> qm: configuration: describe the read-only meta property
> pct: configuration: describe the read-only meta property
>
> pct.adoc | 35 ++++++
> pct.conf.adoc | 3 +
> pmxcfs.adoc | 1 +
> pvecm.adoc | 87 +++++++++++++
> qm.adoc | 38 ++++++
> qm.conf.adoc | 3 +
> vzdump.adoc | 335 ++++++++++++++++++++++++++++++++++++++++++++++++++
> 7 files changed, 502 insertions(+)
>
>
> Summary over all repositories:
> 87 files changed, 10876 insertions(+), 137 deletions(-)
>
^ permalink raw reply [flat|nested] 42+ messages in thread
* Re: [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID
2026-11-18 13:49 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
@ 2026-11-18 14:13 ` Thomas Lamprecht
0 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-18 14:13 UTC (permalink / raw)
To: Dominik Csapak, pve-devel
Am 18.11.26 um 14:49 schrieb Dominik Csapak:
> Read through it again (though mostly via diff from the v1)
> and it looks good to me now:
>
> * the rationale is given
> * the flow is better visible (and with better wording)
> * remote migrate is now blocked by default on conflicting uids
>
> the biggest issue to discuss maybe still is if we want to allow
> restore with a new uid in case the same uid exists already,
> same argument as for the remote migration:
>
> if the same vm already exists, restoring it again can have
> bad implications like duplicate ips, etc.
>
> but IMHO it's not a blocker, maybe even something the admin
> would want to configure globally? (datacenter.cfg?)
IMO such restores can be - while potentially not highly frequent
– part of standard procedure, like test copy, restore test or partial
recovery.
It certainly is a valid point, but as we can only do a likely
correlation that same uid == same network and thus likely bad,
one can also have the situation with a different guest uid but
same network address in both the guests. Also, I'd rather have
this improved midterm by offering to restore to a isolated or
nat'ed or so vnet, and then show a warning hint (and adapt
the defaults to make that isolated-network-env-restore) if a guest
with the uid exist.
Might come out for free for a general slightly overdue improvement
for providing some convenience foundations for easier backup
restore testing.
> one thing i noticed during testing was if i modified a guest
> to have an invalid uid, on the next config write, the
> ctime and creation-qemu vanished from the config and
> only a new rtime and uid remained.
> (though i had to manually edit the config, so not a problem IMO)
will double check, but yeah, such manual edits are a bit hard
to handle correctly; maybe we can harden some simpler/specific
cases.
>
> there are still some nits here and there, but they are mostly
> taste questions and nothing that can't be improved further down the line
>
> for example: when restoring a vm over another another, the failure i get is:
>
> TASK ERROR: command 'set -o pipefail && zstd -q -d -c -- /var/lib/vz/dump/vzdump-qemu-9999-2026_11_18-14_37_58.vma.zst | vma extract -v -r /var/tmp/vzdumptmp8272.fifo - /var/tmp/vzdumptmp8272' failed: backup belongs to another guest: its uid '2xuof4nw6rnowo5mpgs2' differs from uid 'itomggi4ofm33szeb3h2' of the existing VM 102 - use 'allow-identity-mismatch' to overwrite the VM anyway
>
> which is not very readable in the task log window (still functional though)
true, multi-line errors would help here a lot, we support them in
rust side, need to recheck if in perl side too.
>
> also at least a few things in the ui i'm missing:
>
> a way to show the cluster/node id (probably in the datacenter summary)
Or cluster status view, while not coming from corosync or so
directly at least naming wise it fits. I'll see if one seems clearly
better (summary one is probably easier), else keep that out for now.
> using the uid (maybe rtime) as variables in the templating for the notes
good idea.
> Tested it on a test cluster with some backup/restore cycles,
> adopting of uid, manually editing configs, etc.
>
> Since nothing of these blocks this series from my POV, consider it:
>
> Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
> Tested-by: Dominik Csapak <d.csapak@proxmox.com>
big thanks!
^ permalink raw reply [flat|nested] 42+ messages in thread
* superseded: [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
` (38 preceding siblings ...)
2026-11-18 13:49 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
@ 2026-11-19 10:38 ` Thomas Lamprecht
39 siblings, 0 replies; 42+ messages in thread
From: Thomas Lamprecht @ 2026-11-19 10:38 UTC (permalink / raw)
To: pve-devel
Superseded-by: https://lore.proxmox.com/all/20261119103551.1666964-1-t.lamprecht@proxmox.com/
^ permalink raw reply [flat|nested] 42+ messages in thread
end of thread, other threads:[~2026-11-19 10:43 UTC | newest]
Thread overview: 42+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 4:37 [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 01/38] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 02/38] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 03/38] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 04/38] pmxcfs: quorum: pass the corosync membership to an optional listener Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 05/38] pmxcfs: confdb: call an optional listener when the nodes get published Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 06/38] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 07/38] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-18 4:37 ` [PATCH cluster v3 08/38] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 09/38] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 10/38] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 11/38] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 12/38] guest identity: allow taking over a uid that is unused in the cluster Thomas Lamprecht
2026-11-18 4:37 ` [PATCH guest-common v3 13/38] migrate: add shared helpers for the identity command of the migration tunnel Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 14/38] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 15/38] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 16/38] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 17/38] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 18/38] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 19/38] api: status: return the guest uid Thomas Lamprecht
2026-11-18 4:37 ` [PATCH qemu-server v3 20/38] tests: remote migration: check a failed command and an early tunnel end Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 21/38] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 22/38] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 23/38] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 24/38] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 25/38] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 26/38] test: remote migration: check the cleanup of the target after failures Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 27/38] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-18 4:37 ` [PATCH container v3 28/38] api: status: return the guest uid Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 29/38] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 30/38] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 31/38] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 32/38] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 33/38] ui: guest summary: show " Thomas Lamprecht
2026-11-18 4:37 ` [PATCH manager v3 34/38] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 35/38] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 36/38] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 37/38] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-18 4:37 ` [PATCH docs v3 38/38] pct: " Thomas Lamprecht
2026-11-18 13:49 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v3 00/38] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
2026-11-18 14:13 ` Thomas Lamprecht
2026-11-19 10:38 ` superseded: " Thomas Lamprecht
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox