public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH guest-common v4 11/41] guest identity: allow adopting a uid that is unused in the cluster
Date: Thu, 19 Nov 2026 11:33:57 +0100	[thread overview]
Message-ID: <20261119103551.1666964-12-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20261119103551.1666964-1-t.lamprecht@proxmox.com>

A restore should keep the uid of the backup, and a remote migration the
uid of the source guest. Neither may lead to two guests with the same
uid in one cluster. Config writes always keep the identity that a guest
already has, so adopting another uid needs a separate step.

That step takes a cluster-wide lock, which VMs and containers share.
Under this lock, it checks the top-level 'meta' property of all guests
and writes the uid if no other guest has it. A guest that had an
identity before the operation keeps it. Otherwise, the uid replaces the
identity that the guest got when the operation reserved its VMID. If a
guest adopts the uid that it already has, its registration time stays
unchanged.

A restore adopts the uid at its end. If the restore fails or gets
aborted before that, the guest that it leaves behind only has the
identity from the reservation. To allow a retry, the restore records the
uid that it wants to adopt as a pending change of 'meta'. A retry over
such a guest is allowed if the uid of the backup matches the pending
uid, and it adopts the uid under the same cluster-wide check. A pending
uid does not count as in use, so a retry at another VMID stays possible,
too.

The adopt step removes the pending uid when it is done, whatever the
result. Only if the cluster-wide lock cannot be taken, the pending uid
stays for a later operation. The config view with pending changes
applied shows the identity of the guest, while the list of pending
changes still shows the pending uid. Applying pending changes drops a
pending uid with a warning, because a retry of the restore then needs
the override.

Both guest types share the two identity steps of a restore and their
messages. At its start, a restore refuses a backup over a guest with
another uid and records the pending uid. At its end, it adopts the uid
and logs the result. The override for the refusal is a standard option,
so that both guest types describe it the same way.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---

changes v3 -> v4:
* 'adopt' instead of 'take over' everywhere, the code used both words
  for the same step; subject changed accordingly
* new prepare_restore for the start of a restore: it checks the backup
  against the target and records the pending uid, which both guest types
  did with the same calls; adoptable_uid is private now and
  assert_backup_matches_target is gone
* 'allow-identity-mismatch' is a standard option, so that both guest
  types describe it the same way
* adopt_from_source and remote_adopt_result_log are gone: the tunnel
  handler of the next patch refuses a uid in use itself, and the source
  logs the result in its own words
* comment: the order of the locks
* tests: a table of cases for prepare_restore, and the log lines of
  adopt_from_backup
* commit message and comments: plainer wording, same content

changes v2 -> v3:
* the lookup double of the tests reads a 'meta:' line with nothing after
  the colon like pmxcfs does, with a test case
* comments: when the pending uid gets stored, rewrap

changes v1 -> v2:
* add shared helpers for the check of a backup against its target, for
  the take-over at the end of a restore and for the take-over at the
  start of a remote migration, so that both guest types share checks and
  messages
* drop the result 'changed' with its check, which only a manual edit
  during the operation could trigger (Dominik found the name confusing)
* drop a pending uid through one shared helper with a warning of its own
* add an overview of the steps and rework the comment of adopt (Dominik)
* add a test with the real registration of used guest IDs

 src/PVE/AbstractConfig.pm          |  78 +++-
 src/PVE/GuestIdentity.pm           | 215 ++++++++++
 src/tests/config-identity-tests.pl | 647 ++++++++++++++++++++++++++++-
 src/tests/guest-identity-tests.pl  | 128 ++++++
 4 files changed, 1059 insertions(+), 9 deletions(-)

diff --git a/src/PVE/AbstractConfig.pm b/src/PVE/AbstractConfig.pm
index ebdbde6..60cac03 100644
--- a/src/PVE/AbstractConfig.pm
+++ b/src/PVE/AbstractConfig.pm
@@ -119,7 +119,7 @@ sub __identity_read_stored {
 #
 # With guest identities enabled for the class, the identity stored in the existing config file
 # gets set in $conf, whatever $conf contains, or a new one if none is stored. So a config write
-# never changes a stored identity.
+# never changes the uid of a guest, only PVE::GuestIdentity::adopt does.
 sub write_config {
     my ($class, $vmid, $conf) = @_;
 
@@ -150,6 +150,49 @@ sub write_config {
     return;
 }
 
+# Replaces the stored identity of the config of $vmid with $identity and removes its pending uid
+# in the same write. Only for PVE::GuestIdentity::adopt, which checks that no other guest has the
+# uid.
+sub __identity_replace {
+    my ($class, $vmid, $identity) = @_;
+
+    die "guest identities are not enabled for this guest type\n" if !$class->__identity_enabled();
+
+    $class->__identity_lock(
+        $vmid,
+        sub {
+            # the file cache only notices writes of other processes after an update
+            PVE::Cluster::cfs_update();
+            my $conf = $class->load_config($vmid);
+
+            $conf->{meta} = PVE::GuestIdentity::merge_identity($conf->{meta}, $identity);
+            PVE::GuestIdentity::set_pending_uid($conf, undef);
+            PVE::Cluster::cfs_write_file($class->cfs_config_path($vmid), $conf);
+        },
+    );
+
+    return;
+}
+
+# Writes $uid as pending uid into the config of $vmid, or removes the pending uid if $uid is undef,
+# keeping everything else, see PVE::GuestIdentity::set_pending_uid. Does not write if the config
+# already has it that way. The caller must hold the config lock.
+sub record_pending_uid {
+    my ($class, $vmid, $uid) = @_;
+
+    # the file cache only notices writes of other processes after an update
+    PVE::Cluster::cfs_update();
+    my $conf = $class->load_config($vmid);
+    return if ($uid // '') eq (PVE::GuestIdentity::get_pending_uid($conf) // '');
+
+    PVE::GuestIdentity::set_pending_uid($conf, $uid);
+    # the parsed config lacks the stored identity if the parser could not read the 'meta'
+    # property, but write_config puts it back
+    $class->write_config($vmid, $conf);
+
+    return;
+}
+
 # Pending changes related
 
 sub parse_pending_delete {
@@ -314,14 +357,19 @@ sub load_current_config {
 
     # take pending changes in
     if (!$current) {
+        # a guest type with identities never applies a pending change of 'meta', which can hold
+        # the pending uid, see PVE::GuestIdentity::set_pending_uid
+        my $skip = $class->__identity_enabled() ? { meta => 1 } : {};
+
         foreach my $opt (keys %{ $conf->{pending} }) {
-            next if $opt eq 'delete';
+            next if $opt eq 'delete' || $skip->{$opt};
             my $value = $conf->{pending}->{$opt};
             next if ref($value); # just to be sure
             $conf->{$opt} = $value;
         }
         my $pending_delete_hash = $class->parse_pending_delete($conf->{pending}->{delete});
         foreach my $opt (keys %$pending_delete_hash) {
+            next if $skip->{$opt};
             delete $conf->{$opt} if $conf->{$opt};
         }
     }
@@ -332,6 +380,29 @@ sub load_current_config {
     return $conf;
 }
 
+# 'meta' is read-only, so only a manual edit, an older version, or an unfinished restore, which
+# keeps the uid of its backup as pending uid, see PVE::GuestIdentity::set_pending_uid, can have
+# queued a change of it. Drops such a change from the parsed config $conf instead of applying it.
+sub drop_pending_meta {
+    my ($class, $conf) = @_;
+
+    my $pending_delete_hash = $class->parse_pending_delete($conf->{pending}->{delete});
+    return if !defined($conf->{pending}->{meta}) && !$pending_delete_hash->{meta};
+
+    my $uid = PVE::GuestIdentity::get_pending_uid($conf);
+    if (defined($uid)) {
+        warn "guest identity: dropping pending uid $uid of an unfinished restore - a retry now"
+            . " needs 'allow-identity-mismatch'\n";
+    } else {
+        warn "ignoring pending change of read-only option 'meta'\n";
+    }
+
+    delete $conf->{pending}->{meta};
+    $class->remove_from_pending_delete($conf, 'meta');
+
+    return;
+}
+
 # Reserves $vmid for an operation by writing its config with the lock $lock, 'create' by default.
 # An existing config is only allowed with $allow_existing, and required if that is 'require'. It
 # must not be locked. With identities enabled, one that fails to load is refused as well.
@@ -340,6 +411,7 @@ sub load_current_config {
 # as the write can assign an identity to a config without one. Adopting a uid later needs it.
 #   existed - whether a config existed before
 #   previous - the guest identity of that config, undef if it had none or did not exist
+#   previous-pending-uid - the pending uid of that config, see PVE::GuestIdentity::set_pending_uid
 #   assigned - the guest identity of the config after the write, undef with identities
 #     disabled
 sub create_and_lock_config {
@@ -361,6 +433,7 @@ sub create_and_lock_config {
             die $load_err if !defined($conf) && ($allow_existing // '') eq 'require';
             my $existed = defined($conf);
             my $previous = $existed ? $class->get_identity($conf) : undef;
+            my $pending = $existed ? PVE::GuestIdentity::get_pending_uid($conf) : undef;
 
             $conf //= {};
             $class->check_lock($conf);
@@ -391,6 +464,7 @@ sub create_and_lock_config {
             return {
                 existed => $existed ? 1 : 0,
                 previous => $previous,
+                'previous-pending-uid' => $pending,
                 assigned => $class->__identity_enabled() ? $class->get_identity($conf) : undef,
             };
         },
diff --git a/src/PVE/GuestIdentity.pm b/src/PVE/GuestIdentity.pm
index 5621a35..7763ac7 100644
--- a/src/PVE/GuestIdentity.pm
+++ b/src/PVE/GuestIdentity.pm
@@ -5,6 +5,8 @@ use v5.36;
 use Time::HiRes;
 
 use PVE::Cluster;
+use PVE::JSONSchema;
+use PVE::RESTEnvironment;
 
 # The identity of a guest, VM or container alike, consists of two members of the 'meta' property
 # string in the top-level section of its config: 'uid', a random identifier of the guest's lineage
@@ -172,6 +174,219 @@ sub parse_identity_from_raw_config($raw) {
     return parse_identity($meta);
 }
 
+# A restore adopts a uid in three steps. An incoming remote migration does the same without step 2:
+# it calls adopt right after the reservation, before anything of the guest gets transferred, so that
+# a uid in use by another guest can still stop the migration. Its later config writes keep the
+# identity, as every config write does, see PVE::AbstractConfig::write_config.
+#
+# 1. Reserve: create_and_lock_config of the config class writes the config. The guest then has an
+#    identity, the one it had before or a new one. The returned reservation records which
+#    identity the guest had before, if any, and which one it has now.
+# 2. Remember: when the restore knows the uid of the backup, it calls prepare_restore, which stores
+#    that uid as pending uid, if step 3 could adopt it without 'unique'. The pending uid is a
+#    pending change of 'meta' with only the uid member. It does not count as in use, and no config
+#    write applies it. If the restore fails, the guest that it leaves behind still has the pending
+#    uid, so a retry can adopt the uid.
+# 3. Adopt: at its end, the restore calls adopt. The guest gets the uid if it had no identity
+#    before step 1, or had the same uid as pending uid, and if no other guest in the cluster has
+#    that uid. Otherwise, the guest keeps the identity it has after step 1.
+
+# Returns the pending uid in the parsed config $conf, undef if there is none.
+sub get_pending_uid($conf) {
+    my $pending = $conf->{pending} ? parse_identity($conf->{pending}->{meta}) : undef;
+
+    return $pending ? $pending->{uid} : undef;
+}
+
+# Sets $uid as pending uid in the parsed config $conf, replacing any pending change of 'meta', or
+# removes such a change if $uid is undef.
+sub set_pending_uid($conf, $uid) {
+    if (defined($uid)) {
+        die "invalid guest identity uid '$uid'\n" if !is_valid_uid($uid);
+        $conf->{pending}->{meta} = "uid=$uid";
+    } elsif ($conf->{pending}) {
+        delete $conf->{pending}->{meta};
+    }
+
+    return;
+}
+
+# Returns $uid if adopt would try to make it the uid of the guest that the operation reserved with
+# $reservation. That is the case if the guest had no identity before the operation, or had $uid as
+# pending uid. Returns undef otherwise, or if $uid is undef.
+my sub adoptable_uid($reservation, $uid) {
+    return undef if !defined($uid);
+    return $uid
+        if !$reservation->{previous} || ($reservation->{'previous-pending-uid'} // '') eq $uid;
+
+    return undef;
+}
+
+PVE::JSONSchema::register_standard_option(
+    'pve-guest-allow-identity-mismatch',
+    {
+        type => 'boolean',
+        description => "Allow overwriting an existing guest with a backup of another guest, as"
+            . " detected by a different 'uid' in 'meta'. The guest then keeps its own uid. A guest"
+            . " left behind by an unfinished restore also counts as the guest of that backup.",
+        optional => 1,
+        default => 0,
+    },
+);
+
+# Step 2 of a restore, see above. $backup_uid is the uid of the backup, or undef if it has none. The
+# restore overwrites the guest $vmid of the config class $class and reserved it with $reservation.
+# This checks $backup_uid against that guest and stores it as pending uid if adopt could make it the
+# uid of the guest. The caller must hold the config lock of $vmid. Without $allow_mismatch, this
+# must run before the restore changes anything else of the guest.
+#
+# If the guest had another uid before the restore, and did not have $backup_uid as pending uid
+# either, the backup most likely belongs to another guest. This then dies. With $allow_mismatch, it
+# only logs that and the guest keeps its uid.
+#
+# Returns the pending uid, undef if there is none. Until the restore calls adopt, it has to keep the
+# pending uid in every config that it writes, see set_pending_uid, also with 'unique', which only
+# keeps the restore itself from adopting the uid.
+sub prepare_restore($class, $vmid, $reservation, $backup_uid, $allow_mismatch) {
+    return undef if !$reservation || !defined($backup_uid);
+
+    my $target = $reservation->{previous};
+    my $pending_uid = adoptable_uid($reservation, $backup_uid);
+
+    if ($target && !defined($pending_uid) && $target->{uid} ne $backup_uid) {
+        my $guest_type = $class->guest_type();
+        if (!$allow_mismatch) {
+            die "backup belongs to another guest: its uid '$backup_uid' differs from uid"
+                . " '$target->{uid}' of the existing $guest_type $vmid - use"
+                . " 'allow-identity-mismatch' to overwrite the $guest_type anyway\n";
+        }
+
+        print "guest identity: ignoring backup uid $backup_uid, $guest_type $vmid keeps uid"
+            . " $target->{uid}\n";
+    }
+
+    $class->record_pending_uid($vmid, $pending_uid) if defined($pending_uid);
+
+    return $pending_uid;
+}
+
+my sub try_adopt($class, $vmid, $uid, $reservation) {
+    my $outcome = sub($result, $current_uid, @extra) {
+        return { result => $result, uid => $current_uid, requested => $uid, @extra };
+    };
+
+    my $previous = $reservation->{previous};
+    return $outcome->('kept', $previous->{uid}) if $previous && !adoptable_uid($reservation, $uid);
+
+    my $assigned_uid = $reservation->{assigned}->{uid};
+    return $outcome->('skipped', $assigned_uid) if !defined($uid);
+
+    # Shared by all guest types, as they share the VMID and uid namespaces. Lock order: the config
+    # lock of the guest, this lock, then the local identity lock of the config write.
+    my $res = PVE::Cluster::cfs_lock_domain(
+        'guest-identity',
+        undef,
+        sub {
+            my $metas = PVE::Cluster::get_guest_config_properties(['meta']);
+
+            my $current = parse_identity($metas->{$vmid} ? $metas->{$vmid}->{meta} : undef);
+            return $outcome->('kept', $uid) if $current && $current->{uid} eq $uid;
+
+            for my $other (sort { $a <=> $b } keys %$metas) {
+                my $identity = parse_identity($metas->{$other}->{meta});
+                next if !$identity || $identity->{uid} ne $uid;
+
+                my $current_uid = $current ? $current->{uid} : undef;
+                return $outcome->('in-use', $current_uid, 'in-use-by' => $other);
+            }
+
+            $class->__identity_replace($vmid, new_identity($uid));
+
+            return $outcome->('adopted', $uid);
+        },
+    );
+    die $@ if $@;
+
+    return $res;
+}
+
+# Makes $uid, for example the one of a restored backup or of the source guest of a remote
+# migration, the uid of the guest $vmid, with the current time as rtime, where the rules below
+# allow that.
+#
+# $class is the config class of the guest and must have guest identities enabled. $reservation is
+# what create_and_lock_config returned for the operation. The caller must still hold the config
+# lock of $vmid. A restore calls this after its final config write, as its writes keep the pending
+# uid until then. $uid can be undef if there is nothing to adopt, for example for a backup without
+# uid.
+#
+# A guest that had an identity before the operation keeps it, unless it had $uid as pending uid.
+# Otherwise, $uid replaces the identity that the guest has since the reservation, but only if no
+# other guest in the cluster has $uid.
+#
+# Each result listed below is final, so adopt then removes the pending uid from the config. If the
+# cluster-wide identity lock cannot be taken, for example without quorum, there is no result: adopt
+# dies and the pending uid stays, so that a later operation can still adopt it.
+#
+# Returns a hash with the keys 'result', 'uid' (the uid of $vmid afterwards), 'requested' ($uid)
+# and, for the result 'in-use', 'in-use-by' (the lowest VMID of the other guests with $uid). The
+# result is one of:
+#   kept     - the guest had an identity before the operation and keeps it, or it already has $uid
+#   adopted  - the guest now has $uid
+#   in-use   - another guest has $uid, so the guest keeps the identity of the reservation
+#   skipped  - $uid is undef, so the guest keeps the identity of the reservation
+sub adopt($class, $vmid, $uid, $reservation) {
+    die "guest identities are not enabled for this guest type\n" if !$class->__identity_enabled();
+    die "missing guest identity reservation\n"
+        if ref($reservation) ne 'HASH' || !$reservation->{assigned};
+    die "invalid guest identity uid '$uid'\n" if defined($uid) && !is_valid_uid($uid);
+
+    my $res = try_adopt($class, $vmid, $uid, $reservation);
+
+    # the config write that adopts a uid removes the pending uid, too
+    if ($res->{result} ne 'adopted') {
+        eval { $class->record_pending_uid($vmid, undef) };
+        die "unable to remove the pending uid - $@" if $@;
+    }
+
+    return $res;
+}
+
+# Returns the task log line for the result $res of adopt, so that all guest types use the same
+# wording. $origin names where the requested uid comes from.
+my sub adopt_result_message($res, $origin) {
+    my ($result, $requested) = $res->@{qw(result requested)};
+    my $uid = $res->{uid} // 'none';
+
+    return "guest identity: kept uid $uid" if $result eq 'kept';
+    return "guest identity: adopted uid $uid from the $origin" if $result eq 'adopted';
+    return "guest identity: new uid $uid" if $result eq 'skipped';
+
+    return "guest identity: $origin uid $requested is in use by guest $res->{'in-use-by'}, keeping"
+        . " uid $uid";
+}
+
+# Calls adopt for $uid, the uid of the backup or undef if the restore does not adopt one, as the
+# last step of a restore, and logs the result. A failure only warns, as the guest itself got
+# restored and keeps the identity of its reservation.
+sub adopt_from_backup($class, $vmid, $uid, $reservation) {
+    my $res = eval { adopt($class, $vmid, $uid, $reservation) };
+    if (my $err = $@) {
+        chomp($err);
+        my $kept = $reservation->{assigned}->{uid} // 'none';
+        my $msg =
+            defined($uid)
+            ? "unable to adopt backup uid $uid, keeping uid $kept"
+            : "keeping uid $kept";
+        PVE::RESTEnvironment::log_warn("guest identity: $msg - $err");
+        return;
+    }
+
+    print adopt_result_message($res, 'backup') . "\n";
+
+    return;
+}
+
 # Returns undef if $node runs a version of Proxmox VE that keeps the identity when writing guest
 # configs, see MIN_PVE_MANAGER_VERSION. Missing or invalid version information counts as too old.
 # Otherwise returns a hash with 'required-version', the minimum version, and 'version', the version
diff --git a/src/tests/config-identity-tests.pl b/src/tests/config-identity-tests.pl
index bf67e9c..8172fc4 100755
--- a/src/tests/config-identity-tests.pl
+++ b/src/tests/config-identity-tests.pl
@@ -76,16 +76,22 @@ my $cache = {};
 
 our $get_config_error;
 our $check_identity_lock = 1;
+our $lock_domain_error;
+our $domain_locked = 0;
+our $on_lookup;
 
 my $reads = 0;
 my $writes = 0;
+my $domain_writes = 0;
 
 sub parse_test_config($raw) {
     my $conf = { snapshots => {} };
     my $section = $conf;
     for my $line (split(/\n/, $raw)) {
         next if $line =~ m/^\s*$/;
-        if ($line =~ m/^\[([^\]]+)\]$/) {
+        if ($line =~ m/^\[PENDING\]$/) {
+            $section = $conf->{pending} = {};
+        } elsif ($line =~ m/^\[([^\]]+)\]$/) {
             $section = $conf->{snapshots}->{$1} = {};
         } elsif ($line =~ m/^([a-z][a-z0-9_\-]*):\s*(.*?)\s*$/) {
             $section->{$1} = $2;
@@ -105,6 +111,8 @@ sub write_test_config($conf) {
     };
 
     my $raw = $print_section->($conf);
+    $raw .= "\n[PENDING]\n" . $print_section->($conf->{pending})
+        if scalar(keys(($conf->{pending} // {})->%*));
     for my $snapname (sort keys(($conf->{snapshots} // {})->%*)) {
         $raw .= "\n[$snapname]\n" . $print_section->($conf->{snapshots}->{$snapname});
     }
@@ -147,6 +155,7 @@ $cluster_module->redefine(
         die "test error: config of $vmid written without identity lock\n"
             if $check_identity_lock && !identity_lock_held($vmid);
         $writes++;
+        $domain_writes++ if $domain_locked;
         $files->{$path} = write_test_config($conf);
         $versions->{$path} = ++$version_counter;
         $cache->{$path}->{version} = undef if $cache->{$path};
@@ -157,6 +166,39 @@ $cluster_module->redefine(
         return undef if $noerr;
         die "guest $vmid already exists\n";
     },
+    get_guest_config_properties => sub {
+        my ($properties, $vmid) = @_;
+        die "test error: unexpected arguments\n" if defined($vmid) || "@$properties" ne 'meta';
+
+        my $res = {};
+        for my $path (grep { defined($files->{$_}) } keys %$files) {
+            for my $line (split(/\n/, $files->{$path})) {
+                last if $line =~ m/^\[/;
+                # like pmxcfs, a line with nothing after the colon does not count
+                $res->{ vmid_of_path($path) }->{meta} = $1 =~ s/^\s+|\s+$//gr
+                    if $line =~ m/^meta:(.+)$/;
+            }
+        }
+
+        $on_lookup->() if $on_lookup;
+
+        return $res;
+    },
+    cfs_lock_domain => sub {
+        my ($domain, $timeout, $code, @param) = @_;
+        die "test error: unexpected lock domain '$domain'\n" if $domain ne 'guest-identity';
+        die "test error: nested domain lock\n" if $domain_locked;
+
+        if ($lock_domain_error) {
+            $@ = $lock_domain_error;
+            return undef;
+        }
+
+        local $domain_locked = 1;
+        PVE::Cluster::cfs_update();
+        # like the real one, errors of $code are passed on in $@
+        return eval { $code->(@param) };
+    },
     get_config => sub {
         my ($path) = @_;
         my $vmid = vmid_of_path($path);
@@ -394,7 +436,12 @@ subtest 'create and lock config' => sub {
     ok($reserved, 'free VMID: reservation gets an identity');
     is_deeply(
         $state,
-        { existed => 0, previous => undef, assigned => $reserved },
+        {
+            existed => 0,
+            previous => undef,
+            'previous-pending-uid' => undef,
+            assigned => $reserved,
+        },
         'free VMID: no previous identity',
     );
     is(conf_of($class, 150)->{lock}, 'create', 'free VMID: reserved');
@@ -408,7 +455,12 @@ subtest 'create and lock config' => sub {
     $state = $class->create_and_lock_config(151, 1, 'restore');
     is_deeply(
         $state,
-        { existed => 1, previous => $stored, assigned => $stored },
+        {
+            existed => 1,
+            previous => $stored,
+            'previous-pending-uid' => undef,
+            assigned => $stored,
+        },
         'in place: previous identity',
     );
     my $conf = conf_of($class, 151);
@@ -428,7 +480,12 @@ subtest 'create and lock config' => sub {
     $state = $class->create_and_lock_config(152, 1);
     is_deeply(
         $state,
-        { existed => 1, previous => undef, assigned => identity_of($class, 152) },
+        {
+            existed => 1,
+            previous => undef,
+            'previous-pending-uid' => undef,
+            assigned => identity_of($class, 152),
+        },
         'in place without identity: reservation gets one',
     );
     ok($state->{assigned}, 'in place without identity: reserved identity');
@@ -437,7 +494,12 @@ subtest 'create and lock config' => sub {
     $state = $class->create_and_lock_config(157, 1);
     is_deeply(
         $state,
-        { existed => 1, previous => undef, assigned => identity_of($class, 157) },
+        {
+            existed => 1,
+            previous => undef,
+            'previous-pending-uid' => undef,
+            assigned => identity_of($class, 157),
+        },
         'in place with empty config: reservation gets an identity',
     );
     ok($state->{assigned}, 'in place with empty config: reserved identity');
@@ -474,7 +536,7 @@ subtest 'create and lock config' => sub {
     $state = $class->create_and_lock_config(154);
     is_deeply(
         $state,
-        { existed => 0, previous => undef, assigned => undef },
+        { existed => 0, previous => undef, 'previous-pending-uid' => undef, assigned => undef },
         'disabled, free VMID: no previous identity',
     );
     is(file_of($class, 154), "lock: create\n", 'disabled, free VMID: no identity assigned');
@@ -483,7 +545,12 @@ subtest 'create and lock config' => sub {
     $state = $class->create_and_lock_config(155, 1);
     is_deeply(
         $state,
-        { existed => 1, previous => { uid => $foreign_uid, rtime => 5 }, assigned => undef },
+        {
+            existed => 1,
+            previous => { uid => $foreign_uid, rtime => 5 },
+            'previous-pending-uid' => undef,
+            assigned => undef,
+        },
         'disabled, in place: identity from the config',
     );
     is(
@@ -493,4 +560,570 @@ subtest 'create and lock config' => sub {
     );
 };
 
+subtest 'create and lock config with unique and enforced guest IDs' => sub {
+    my $class = 'PVE::TestConfig';
+
+    # PVE::GuestID imported the functions it uses to access its files, so fake them there, with the
+    # parser and writer it registered for the list of used guest IDs
+    my (undef, $id_list) = PVE::Cluster::cfs_file_version('virtual-guest/used-guest-ids');
+    my $used_ids = "500\n";
+    my $dc_conf = { 'next-id' => { unique => 1, enforce => 1 } };
+    $guest_id_module->unmock('register_used_id');
+    $guest_id_module->redefine(
+        cfs_read_file => sub($filename) {
+            return $dc_conf if $filename eq 'datacenter.cfg';
+            die "test error: unexpected read of '$filename'\n"
+                if $filename ne 'virtual-guest/used-guest-ids';
+            return $id_list->{parser}->("/etc/pve/$filename", $used_ids);
+        },
+        cfs_write_file => sub($filename, $ranges) {
+            $used_ids = $id_list->{writer}->("/etc/pve/$filename", $ranges);
+        },
+        cfs_lock_file => sub($filename, $timeout, $code) {
+            return eval { $code->() };
+        },
+    );
+
+    eval { $class->create_and_lock_config(500) };
+    is($@, "guest ID 500 was used before\n", 'used VMID is refused');
+    ok(!defined(file_of($class, 500)), 'refused VMID: no config is left behind');
+
+    my $created = $class->create_and_lock_config(510);
+    $class->write_config(510, { name => 'existing' });
+    my $state = $class->create_and_lock_config(510, 1, 'restore');
+    is_deeply(
+        $state,
+        {
+            existed => 1,
+            previous => $created->{assigned},
+            'previous-pending-uid' => undef,
+            assigned => $created->{assigned},
+        },
+        'existing guest: reservation is accepted and returns its identity',
+    );
+    is(conf_of($class, 510)->{lock}, 'restore', 'existing guest: locked');
+
+    $class->create_and_lock_config(520);
+    is($used_ids, "500\n510\n520\n", 'new reservations are recorded as used');
+
+    # like a failed restore that releases the VMID it reserved
+    my $path = $class->cfs_config_path(520);
+    delete $files->{$path};
+    $versions->{$path} = ++$version_counter;
+
+    for my $force (0, 1) {
+        eval { $class->create_and_lock_config(520, $force) };
+        is($@, "guest ID 520 was used before\n", "released VMID is refused (force: $force)");
+    }
+    ok(!defined(file_of($class, 520)), 'released VMID: no config is left behind');
+
+    $guest_id_module->unmock(qw(cfs_read_file cfs_write_file cfs_lock_file));
+    $guest_id_module->redefine(register_used_id => sub { });
+};
+
+subtest 'adopt' => sub {
+    my $class = 'PVE::TestConfig';
+    my $ct_class = 'PVE::TestConfig::CT';
+
+    my $backup_uid = new_uid();
+
+    # restore to a free VMID, the final write carries the identity of the backup
+    my $reservation = $class->create_and_lock_config(200);
+    my $restored = { name => 'restored', meta => "ctime=1,rtime=5,uid=$backup_uid" };
+    $class->write_config(200, { $restored->%*, lock => 'create' });
+    my $generated = identity_of($class, 200);
+    is_deeply($generated, $reservation->{assigned}, 'final write keeps the reserved identity');
+
+    $now += 1000;
+    my $res = PVE::GuestIdentity::adopt($class, 200, $backup_uid, $reservation);
+    is_deeply(
+        $res,
+        { result => 'adopted', uid => $backup_uid, requested => $backup_uid },
+        'free uid is adopted',
+    );
+    is_deeply(
+        identity_of($class, 200),
+        { uid => $backup_uid, rtime => $now },
+        'adopted uid is stored with the current time',
+    );
+    like(
+        file_of($class, 200),
+        qr/^meta: ctime=1,rtime=$now,uid=$backup_uid$/m,
+        'other members are kept',
+    );
+    is(conf_of($class, 200)->{lock}, 'create', 'rest of the config is kept');
+    is($domain_writes, 1, 'stored under the cluster-wide lock');
+
+    my $adopted_rtime = $now;
+    $now += 1000;
+
+    my $stale = { $restored->%*, meta => "ctime=1,rtime=5,uid=$generated->{uid}" };
+    $class->write_config(200, $stale);
+    is_deeply(
+        identity_of($class, 200),
+        { uid => $backup_uid, rtime => $adopted_rtime },
+        'later write of a stale hash keeps the adopted identity',
+    );
+
+    my $writes_before = $writes;
+    $res = PVE::GuestIdentity::adopt($class, 200, $backup_uid, $reservation);
+    is_deeply(
+        $res,
+        { result => 'kept', uid => $backup_uid, requested => $backup_uid },
+        'repeated adoption keeps it',
+    );
+    is($writes, $writes_before, 'repeated adoption writes nothing');
+    is(identity_of($class, 200)->{rtime}, $adopted_rtime, 'repeated adoption keeps rtime');
+
+    # restore in place onto a guest with identity
+    $class->write_config(210, { name => 'target' });
+    my $target = identity_of($class, 210);
+    $reservation = $class->create_and_lock_config(210, 1, 'restore');
+    my $other_uid = new_uid();
+    {
+        local $lock_domain_error = "test error: unexpected lock\n";
+        $res = PVE::GuestIdentity::adopt($class, 210, $other_uid, $reservation);
+    }
+    is_deeply(
+        $res,
+        { result => 'kept', uid => $target->{uid}, requested => $other_uid },
+        'in place: identity from before the operation is kept',
+    );
+    is_deeply(identity_of($class, 210), $target, 'in place: identity is unchanged');
+
+    set_file($class, 211, "meta: ctime=1\nname: old\n");
+    $reservation = $class->create_and_lock_config(211, 1);
+    $other_uid = new_uid();
+    $res = PVE::GuestIdentity::adopt($class, 211, $other_uid, $reservation);
+    is($res->{result}, 'adopted', 'in place without identity: uid is adopted');
+    is(identity_of($class, 211)->{uid}, $other_uid, 'in place without identity: stored');
+
+    $reservation = $class->create_and_lock_config(212);
+    {
+        local $lock_domain_error = "test error: unexpected lock\n";
+        $res = PVE::GuestIdentity::adopt($class, 212, undef, $reservation);
+    }
+    is_deeply(
+        $res,
+        { result => 'skipped', uid => $reservation->{assigned}->{uid}, requested => undef },
+        'nothing to adopt',
+    );
+
+    $reservation = $class->create_and_lock_config(201);
+    $class->write_config(201, { name => 'second restore', lock => 'create' });
+    my $second = identity_of($class, 201);
+    $writes_before = $writes;
+    $res = PVE::GuestIdentity::adopt($class, 201, $backup_uid, $reservation);
+    is_deeply(
+        $res,
+        {
+            result => 'in-use',
+            uid => $second->{uid},
+            requested => $backup_uid,
+            'in-use-by' => 200,
+        },
+        'uid of another VM is not adopted',
+    );
+    is_deeply(identity_of($class, 201), $second, 'reserved identity stays');
+    is($writes, $writes_before, 'nothing is written');
+
+    # manual copies of a config
+    my $copied_uid = new_uid();
+    set_file($class, 220, "meta: rtime=1,uid=$copied_uid\n");
+    set_file($ct_class, 205, "meta: rtime=1,uid=$copied_uid\n");
+    $res = PVE::GuestIdentity::adopt($class, 201, $copied_uid, $reservation);
+    is($res->{'in-use-by'}, 205, 'lowest VMID of several holders is reported');
+
+    my $hidden_uid = new_uid();
+    set_file($class, 221, "meta: rtime=1,uid=$hidden_uid\nmeta:\n");
+    $res = PVE::GuestIdentity::adopt($class, 201, $hidden_uid, $reservation);
+    is($res->{'in-use-by'}, 221, 'meta line without value does not hide the uid before it');
+
+    my $ct_uid = new_uid();
+    my $ct_reservation = $ct_class->create_and_lock_config(300);
+    $res = PVE::GuestIdentity::adopt($ct_class, 300, $ct_uid, $ct_reservation);
+    is($res->{result}, 'adopted', 'container adopts a free uid');
+    like(file_of($ct_class, 300), qr/^meta: rtime=$now,uid=$ct_uid$/m, 'container config');
+
+    $res = PVE::GuestIdentity::adopt($class, 201, $ct_uid, $reservation);
+    is($res->{'in-use-by'}, 300, 'uid of a container is not adopted for a VM');
+
+    $ct_reservation = $ct_class->create_and_lock_config(301);
+    $res = PVE::GuestIdentity::adopt($ct_class, 301, $backup_uid, $ct_reservation);
+    is($res->{'in-use-by'}, 200, 'uid of a VM is not adopted for a container');
+
+    # another process writes the config after the cluster-wide lookup
+    $reservation = $class->create_and_lock_config(230);
+    $class->write_config(230, { name => 'restored', lock => 'create' });
+    PVE::Cluster::cfs_update();
+    $class->load_config(230);
+    {
+        local $on_lookup = sub {
+            set_file($class, 230, file_of($class, 230) =~ s/^name: .*$/name: hook/mr);
+        };
+        $res = PVE::GuestIdentity::adopt($class, 230, new_uid(), $reservation);
+    }
+    is($res->{result}, 'adopted', 'concurrent write: uid is adopted');
+    is(conf_of($class, 230)->{name}, 'hook', 'concurrent write: its change is kept');
+
+    $reservation = $class->create_and_lock_config(202);
+    my $before = identity_of($class, 202);
+    {
+        local $lock_domain_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+        eval { PVE::GuestIdentity::adopt($class, 202, new_uid(), $reservation) };
+        is($@, $lock_domain_error, 'lock error is passed on');
+        is_deeply(identity_of($class, 202), $before, 'lock error: identity is unchanged');
+    }
+
+    my $missing =
+        { existed => 0, previous => undef, assigned => PVE::GuestIdentity::new_identity() };
+    eval { PVE::GuestIdentity::adopt($class, 299, new_uid(), $missing) };
+    like($@, qr/does not exist/, 'missing config is an error');
+
+    eval { PVE::GuestIdentity::adopt($class, 202, 'GAYTEMZUGU3DOOBZMFRG', $reservation) };
+    like($@, qr/invalid guest identity uid/, 'invalid uid is refused');
+
+    for my $invalid (undef, {}, { existed => 0, previous => undef, assigned => undef }) {
+        eval { PVE::GuestIdentity::adopt($class, 202, new_uid(), $invalid) };
+        like($@, qr/missing guest identity reservation/, 'missing reservation is refused');
+    }
+
+    local $check_identity_lock = 0;
+    eval {
+        PVE::GuestIdentity::adopt('PVE::TestConfig::NoIdentity', 202, new_uid(), $reservation);
+    };
+    like($@, qr/not enabled/, 'guest type without identities is refused');
+};
+
+subtest 'adopt from backup' => sub {
+    my $class = 'PVE::TestConfig';
+
+    my $warnings = [];
+    my $restenv_module = Test::MockModule->new('PVE::RESTEnvironment');
+    $restenv_module->redefine(log_warn => sub($msg) { push $warnings->@*, $msg; });
+
+    my $adopt_from_backup = sub($vmid, $uid, $reservation) {
+        my $output = '';
+        open(my $fh, '>', \$output) or die "unable to capture output - $!\n";
+        my $previous_fh = select($fh);
+        eval { PVE::GuestIdentity::adopt_from_backup($class, $vmid, $uid, $reservation) };
+        my $err = $@;
+        select($previous_fh);
+        close($fh);
+        die $err if $err;
+        return $output;
+    };
+
+    my $backup_uid = new_uid();
+    my $reservation = $class->create_and_lock_config(240);
+    is(
+        $adopt_from_backup->(240, $backup_uid, $reservation),
+        "guest identity: adopted uid $backup_uid from the backup\n",
+        'result is logged',
+    );
+    is(identity_of($class, 240)->{uid}, $backup_uid, 'uid is adopted');
+
+    $reservation = $class->create_and_lock_config(242);
+    my $assigned_uid = $reservation->{assigned}->{uid};
+    is(
+        $adopt_from_backup->(242, $backup_uid, $reservation),
+        "guest identity: backup uid $backup_uid is in use by guest 240, keeping uid"
+            . " $assigned_uid\n",
+        'uid in use: logged',
+    );
+    is(
+        $adopt_from_backup->(242, undef, $reservation),
+        "guest identity: new uid $assigned_uid\n",
+        'nothing to adopt: logged',
+    );
+
+    $class->write_config(243, { name => 'target' });
+    $reservation = $class->create_and_lock_config(243, 1);
+    is(
+        $adopt_from_backup->(243, new_uid(), $reservation),
+        "guest identity: kept uid $reservation->{previous}->{uid}\n",
+        'target with identity: logged',
+    );
+
+    $reservation = $class->create_and_lock_config(241);
+    $assigned_uid = $reservation->{assigned}->{uid};
+    $backup_uid = new_uid();
+    $class->record_pending_uid(241, $backup_uid);
+    {
+        local $lock_domain_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+        is($adopt_from_backup->(241, $backup_uid, $reservation), '', 'failure: no result');
+    }
+    is_deeply(
+        $warnings,
+        [
+            "guest identity: unable to adopt backup uid $backup_uid, keeping uid $assigned_uid"
+                . " - cfs-lock 'domain-guest-identity' error: no quorum!",
+        ],
+        'failure only warns',
+    );
+
+    $warnings = [];
+    {
+        local $get_config_error = "Connection refused\n";
+        is($adopt_from_backup->(241, undef, $reservation), '', 'nothing to adopt, failure');
+    }
+    is_deeply(
+        $warnings,
+        [
+            "guest identity: keeping uid $assigned_uid - unable to remove the pending uid -"
+                . " Connection refused",
+        ],
+        'nothing to adopt, failure: warning without requested uid',
+    );
+};
+
+# Returns the pending uid of a restore of a backup with the uid $uid over the guest $vmid, without
+# 'allow-identity-mismatch'.
+my sub prepare_restore($class, $vmid, $reservation, $uid) {
+    return PVE::GuestIdentity::prepare_restore($class, $vmid, $reservation, $uid, 0);
+}
+
+# like the guest that a failed operation leaves behind, with the lock removed by its error handling
+my sub unlock_config($class, $vmid) {
+    my $conf = conf_of($class, $vmid);
+    delete $conf->{lock};
+    $class->write_config($vmid, $conf);
+}
+
+my sub pending_of($class, $vmid) {
+    my $conf = conf_of($class, $vmid);
+    return $conf->{pending} ? $conf->{pending}->{meta} : undef;
+}
+
+subtest 'pending uid' => sub {
+    my $class = 'PVE::TestConfig';
+
+    # a restore to a free VMID records the uid of the backup, then fails and leaves the guest
+    my $backup_uid = new_uid();
+    my $first = $class->create_and_lock_config(400);
+    my $leftover = $first->{assigned};
+    is(prepare_restore($class, 400, $first, $backup_uid), $backup_uid, 'free VMID: adoptable');
+    is(pending_of($class, 400), "uid=$backup_uid", 'pending uid is recorded');
+    is_deeply(identity_of($class, 400), $leftover, 'stored identity is kept');
+    is(conf_of($class, 400)->{lock}, 'create', 'rest of the config is kept');
+    unlock_config($class, 400);
+    is(pending_of($class, 400), "uid=$backup_uid", 'write of the loaded config keeps it');
+
+    # a retry at another VMID adopts the uid, as a pending uid is not in use
+    my $elsewhere = $class->create_and_lock_config(401);
+    my $res = PVE::GuestIdentity::adopt($class, 401, $backup_uid, $elsewhere);
+    is($res->{result}, 'adopted',
+        'retry elsewhere: pending uid of another guest is not in use');
+    is(pending_of($class, 400), "uid=$backup_uid", 'retry elsewhere: leftover keeps it');
+
+    # a retry over the leftover is a restore of the same backup, but the uid is in use by now
+    my $retry = $class->create_and_lock_config(400, 1);
+    is_deeply(
+        $retry,
+        {
+            existed => 1,
+            previous => $leftover,
+            'previous-pending-uid' => $backup_uid,
+            assigned => $leftover,
+        },
+        'reservation reports the pending uid',
+    );
+    my $writes_before = $writes;
+    is(prepare_restore($class, 400, $retry, $backup_uid), $backup_uid,
+        'pending uid: adoptable');
+    is(prepare_restore($class, 400, $retry, $leftover->{uid}), undef, 'stored uid matches');
+    eval { prepare_restore($class, 400, $retry, new_uid()) };
+    like($@, qr/^backup belongs to another guest/, 'other uid does not match');
+    is($writes, $writes_before, 'pending uid is not written again');
+    $res = PVE::GuestIdentity::adopt($class, 400, $backup_uid, $retry);
+    is_deeply(
+        $res,
+        {
+            result => 'in-use',
+            uid => $leftover->{uid},
+            requested => $backup_uid,
+            'in-use-by' => 401,
+        },
+        'retry in place after the retry elsewhere: in use',
+    );
+    is_deeply(identity_of($class, 400), $leftover, 'in use: identity is kept');
+    ok(!defined(pending_of($class, 400)), 'in use: pending uid is removed');
+
+    # a failed restore over a guest without identity, then retried in place
+    set_file($class, 410, "name: old\n");
+    $first = $class->create_and_lock_config(410, 1);
+    $backup_uid = new_uid();
+    is(prepare_restore($class, 410, $first, $backup_uid), $backup_uid,
+        'no identity: adoptable');
+    unlock_config($class, 410);
+
+    $retry = $class->create_and_lock_config(410, 1);
+    is($retry->{'previous-pending-uid'}, $backup_uid, 'retry in place: pending uid reported');
+    $class->write_config(
+        410,
+        { name => 'restored', lock => 'create', pending => { meta => "uid=$backup_uid" } },
+    );
+    $now += 1000;
+    $writes_before = $writes;
+    $res = PVE::GuestIdentity::adopt($class, 410, $backup_uid, $retry);
+    is_deeply(
+        $res,
+        { result => 'adopted', uid => $backup_uid, requested => $backup_uid },
+        'retry in place: pending uid is adopted',
+    );
+    is_deeply(identity_of($class, 410), { uid => $backup_uid, rtime => $now }, 'stored');
+    ok(!defined(pending_of($class, 410)), 'retry in place: pending uid is removed');
+    is($writes, $writes_before + 1, 'in the same write');
+    is(conf_of($class, 410)->{name}, 'restored', 'rest of the config is kept');
+
+    # an in-place restore of another backup keeps the identity, with a pending uid of the earlier
+    # backup
+    set_file($class, 430, "name: old\n");
+    $first = $class->create_and_lock_config(430, 1);
+    my $earlier_uid = new_uid();
+    $class->record_pending_uid(430, $earlier_uid);
+    unlock_config($class, 430);
+    $retry = $class->create_and_lock_config(430, 1);
+    my $target = identity_of($class, 430);
+    my $other_uid = new_uid();
+    eval { prepare_restore($class, 430, $retry, $other_uid) };
+    like($@, qr/^backup belongs to another guest/, 'other backup does not match');
+    is(pending_of($class, 430), "uid=$earlier_uid", 'other backup: pending uid is unchanged');
+    {
+        local $lock_domain_error = "test error: unexpected lock\n";
+        $res = PVE::GuestIdentity::adopt($class, 430, $other_uid, $retry);
+    }
+    is($res->{result}, 'kept', 'other backup: identity is kept');
+    is_deeply(identity_of($class, 430), $target, 'other backup: identity is unchanged');
+    ok(!defined(pending_of($class, 430)), 'other backup: pending uid is removed');
+
+    # nothing to adopt, for example with 'unique'
+    $first = $class->create_and_lock_config(440);
+    $class->record_pending_uid(440, new_uid());
+    $res = PVE::GuestIdentity::adopt($class, 440, undef, $first);
+    is($res->{result}, 'skipped', 'nothing to adopt: skipped');
+    ok(!defined(pending_of($class, 440)), 'nothing to adopt: pending uid is removed');
+
+    # adopting fails
+    $first = $class->create_and_lock_config(450);
+    $backup_uid = new_uid();
+    $class->record_pending_uid(450, $backup_uid);
+    {
+        local $lock_domain_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+        eval { PVE::GuestIdentity::adopt($class, 450, $backup_uid, $first) };
+        is($@, $lock_domain_error, 'lock error is passed on');
+    }
+    is_deeply(identity_of($class, 450), $first->{assigned},
+        'lock error: identity is unchanged');
+    is(
+        pending_of($class, 450),
+        "uid=$backup_uid",
+        'lock error: nothing decided, pending uid stays',
+    );
+
+    unlock_config($class, 450);
+    $retry = $class->create_and_lock_config(450, 1);
+    $res = PVE::GuestIdentity::adopt($class, 450, $backup_uid, $retry);
+    is($res->{result}, 'adopted', 'lock error: a later operation adopts the pending uid');
+    ok(!defined(pending_of($class, 450)), 'lock error: then the pending uid is removed');
+
+    # removing the pending uid keeps a stored identity that the parser could not read
+    my $config_module = Test::MockModule->new('PVE::TestConfig', no_auto => 1);
+    $config_module->mock(
+        load_config => sub {
+            my $conf = $config_module->original('load_config')->(@_);
+            delete $conf->{meta};
+            return $conf;
+        },
+    );
+    $class->write_config(470, { name => 'target', meta => 'ctime=1' });
+    $target = identity_of($class, 470);
+    $retry = $class->create_and_lock_config(470, 1);
+    $class->record_pending_uid(470, new_uid());
+    {
+        local $lock_domain_error = "test error: unexpected lock\n";
+        $res = PVE::GuestIdentity::adopt($class, 470, new_uid(), $retry);
+    }
+    $config_module->unmock('load_config');
+    is($res->{result}, 'kept', 'unreadable meta: identity is kept');
+    ok(!defined(pending_of($class, 470)), 'unreadable meta: pending uid is removed');
+    is_deeply(identity_of($class, 470), $target, 'unreadable meta: stored identity is kept');
+
+    # removing the pending uid fails
+    $class->write_config(460, { name => 'target' });
+    $retry = $class->create_and_lock_config(460, 1);
+    $class->record_pending_uid(460, new_uid());
+    {
+        local $get_config_error = "Connection refused\n";
+        eval { PVE::GuestIdentity::adopt($class, 460, new_uid(), $retry) };
+        is($@, "unable to remove the pending uid - Connection refused\n", 'error is passed on');
+    }
+    ok(defined(pending_of($class, 460)), 'pending uid stays');
+
+    eval { $class->record_pending_uid(460, 'GAYTEMZUGU3DOOBZMFRG') };
+    like($@, qr/invalid guest identity uid/, 'invalid pending uid is refused');
+};
+
+subtest 'config view with pending changes' => sub {
+    my $class = 'PVE::TestConfig';
+
+    my $pending_uid = new_uid();
+    $class->write_config(480, { name => 'old', meta => 'ctime=1' });
+    my $meta = conf_of($class, 480)->{meta};
+    $class->record_pending_uid(480, $pending_uid);
+    my $conf = conf_of($class, 480);
+    $conf->{pending}->{name} = 'new';
+    $class->write_config(480, $conf);
+
+    PVE::Cluster::cfs_update();
+    $conf = $class->load_current_config(480);
+    is($conf->{name}, 'new', 'other pending change is shown');
+    is($conf->{meta}, $meta, 'pending uid is not shown as meta');
+    ok(!exists($conf->{pending}), 'no pending section');
+    is($class->load_current_config(480, 1)->{name}, 'old', 'current values only');
+
+    set_file($class, 481, "meta: $meta\nname: old\n\n[PENDING]\ndelete: meta\n");
+    PVE::Cluster::cfs_update();
+    is($class->load_current_config(481)->{meta}, $meta, 'pending deletion of meta not shown');
+
+    # a guest type without identities may apply pending changes of 'meta'
+    local $check_identity_lock = 0;
+    $class = 'PVE::TestConfig::NoIdentity';
+    set_file($class, 482, "meta: ctime=1\nname: old\n\n[PENDING]\nmeta: ctime=2\n");
+    PVE::Cluster::cfs_update();
+    is($class->load_current_config(482)->{meta}, 'ctime=2', 'without identities: shown');
+};
+
+subtest 'drop pending meta' => sub {
+    my $class = 'PVE::TestConfig';
+    my $uid = new_uid();
+
+    my $uid_dropped = "guest identity: dropping pending uid $uid of an unfinished restore - a"
+        . " retry now needs 'allow-identity-mismatch'\n";
+    my $ignored = "ignoring pending change of read-only option 'meta'\n";
+    my $tests = [
+        ['pending uid', { meta => "uid=$uid", name => 'new' }, { name => 'new' }, $uid_dropped],
+        ['other change', { meta => 'ctime=2', name => 'new' }, { name => 'new' }, $ignored],
+        ['deletion', { delete => 'meta,name' }, { delete => 'name' }, $ignored],
+        ['uid and deletion', { meta => "uid=$uid", delete => 'meta' }, {}, $uid_dropped],
+        ['no change of meta', { name => 'new' }, { name => 'new' }, ''],
+    ];
+
+    for my $test (@$tests) {
+        my ($name, $pending, $expected, $expected_warning) = @$test;
+        my $conf = { meta => 'ctime=1', name => 'old', pending => { $pending->%* } };
+
+        my $warnings = '';
+        local $SIG{__WARN__} = sub { $warnings .= $_[0]; };
+        $class->drop_pending_meta($conf);
+
+        is_deeply(
+            $conf,
+            { meta => 'ctime=1', name => 'old', pending => $expected },
+            "$name: only a change of meta is dropped",
+        );
+        is($warnings, $expected_warning, "$name: warning");
+    }
+};
+
 done_testing();
diff --git a/src/tests/guest-identity-tests.pl b/src/tests/guest-identity-tests.pl
index b6a7c85..19635a5 100755
--- a/src/tests/guest-identity-tests.pl
+++ b/src/tests/guest-identity-tests.pl
@@ -13,6 +13,25 @@ use PVE::Tools;
 
 use PVE::GuestIdentity;
 
+# records the pending uids that get written, as [$vmid, $uid]
+my $recorded = [];
+
+package PVE::TestGuest::VM {
+    sub guest_type { return 'VM'; }
+
+    sub record_pending_uid($class, $vmid, $uid) {
+        push $recorded->@*, [$vmid, $uid];
+    }
+}
+
+package PVE::TestGuest::CT {
+    use base qw(PVE::TestGuest::VM);
+
+    sub guest_type { return 'CT'; }
+}
+
+package main;
+
 subtest 'generation with fixed random bytes and time' => sub {
     my $random_bytes = [];
     my $time_ms = 1_700_000_000_123;
@@ -343,4 +362,113 @@ subtest 'node version check' => sub {
     }
 };
 
+subtest 'pending uid' => sub {
+    my $uid = 'gaytemzugu3doobzmfrg';
+    my $other = 'aaaaaaaaaaaaaaaaaaaa';
+
+    is(PVE::GuestIdentity::get_pending_uid({}), undef, 'none without pending section');
+    is(PVE::GuestIdentity::get_pending_uid({ pending => { memory => 1 } }), undef, 'none');
+    is(
+        PVE::GuestIdentity::get_pending_uid({ pending => { meta => 'uid=' . uc($uid) } }),
+        undef,
+        'invalid pending uid is none',
+    );
+    is(
+        PVE::GuestIdentity::get_pending_uid({ pending => { meta => "ctime=1,uid=$uid" } }),
+        $uid,
+        'other members are ignored',
+    );
+
+    my $conf = { meta => "rtime=1,uid=$other", pending => { memory => 1 } };
+    PVE::GuestIdentity::set_pending_uid($conf, $uid);
+    is_deeply(
+        $conf,
+        { meta => "rtime=1,uid=$other", pending => { memory => 1, meta => "uid=$uid" } },
+        'set: only the uid member, other pending changes are kept',
+    );
+    is(PVE::GuestIdentity::get_pending_uid($conf), $uid, 'set: read back');
+
+    PVE::GuestIdentity::set_pending_uid($conf, undef);
+    is_deeply(
+        $conf,
+        { meta => "rtime=1,uid=$other", pending => { memory => 1 } },
+        'remove: other pending changes are kept',
+    );
+    $conf = {};
+    PVE::GuestIdentity::set_pending_uid($conf, undef);
+    is_deeply($conf, {}, 'remove without pending section changes nothing');
+
+    eval { PVE::GuestIdentity::set_pending_uid({}, 'GAYTEMZUGU3DOOBZMFRG') };
+    like($@, qr/invalid guest identity uid/, 'invalid uid is refused');
+};
+
+subtest 'prepare restore' => sub {
+    my $uid = 'gaytemzugu3doobzmfrg';
+    my $other = 'aaaaaaaaaaaaaaaaaaaa';
+
+    my $previous = { uid => $other, rtime => 1 };
+    my $without_identity = { previous => undef, 'previous-pending-uid' => undef };
+    my $with_identity = { previous => $previous, 'previous-pending-uid' => undef };
+    my $leftover = { previous => $previous, 'previous-pending-uid' => $uid };
+    my $other_leftover = { previous => $previous, 'previous-pending-uid' => 'b' x 20 };
+
+    # reservation, uid of the backup, 'allow-identity-mismatch', then what is expected: whether
+    # the restore is refused or logs that it ignores the uid of the backup, and the pending uid
+    my $tests = [
+        [$without_identity, $uid, 0, undef, $uid, 'target without identity'],
+        [$without_identity, undef, 0, undef, undef, 'backup without uid'],
+        [undef, $uid, 0, undef, undef, 'no reservation'],
+        [$with_identity, $other, 0, undef, undef, 'backup of the target'],
+        [$with_identity, $uid, 0, 'refused', undef, 'backup of another guest'],
+        [$with_identity, $uid, 1, 'ignored', undef, 'backup of another guest, allowed'],
+        [$leftover, $uid, 0, undef, $uid, 'leftover of the backup'],
+        [$leftover, $other, 0, undef, undef, 'leftover of the backup, backup of the target'],
+        [$other_leftover, $uid, 0, 'refused', undef, 'leftover of another backup'],
+        [$other_leftover, $uid, 1, 'ignored', undef, 'leftover of another backup, allowed'],
+    ];
+
+    for my $class (qw(PVE::TestGuest::VM PVE::TestGuest::CT)) {
+        my $type = $class->guest_type();
+        my $messages = {
+            refused =>
+                "backup belongs to another guest: its uid '$uid' differs from uid '$other' of"
+                . " the existing $type 100 - use 'allow-identity-mismatch' to overwrite the $type"
+                . " anyway\n",
+            ignored => "guest identity: ignoring backup uid $uid, $type 100 keeps uid $other\n",
+        };
+
+        for my $test (@$tests) {
+            my ($reservation, $backup_uid, $allow_mismatch, $outcome, $pending_uid, $name) =
+                @$test;
+            $recorded = [];
+
+            my $output = '';
+            open(my $fh, '>', \$output) or die "unable to capture output - $!\n";
+            my $previous_fh = select($fh);
+            my $res = eval {
+                PVE::GuestIdentity::prepare_restore(
+                    $class, 100, $reservation, $backup_uid, $allow_mismatch,
+                );
+            };
+            my $err = $@;
+            select($previous_fh);
+            close($fh);
+
+            is_deeply(
+                [$err, $output, $res, $recorded],
+                [
+                    ($outcome // '') eq 'refused' ? $messages->{refused} : '',
+                    ($outcome // '') eq 'ignored' ? $messages->{ignored} : '',
+                    $pending_uid,
+                    defined($pending_uid) ? [[100, $pending_uid]] : [],
+                ],
+                "$type: $name",
+            );
+        }
+    }
+
+    my $option = PVE::JSONSchema::get_standard_option('pve-guest-allow-identity-mismatch');
+    is($option->{type}, 'boolean', 'the override is a standard option');
+};
+
 done_testing();
-- 
2.47.3





  parent reply	other threads:[~2026-11-19 10:42 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 10:33 [PATCH cluster/guest-common/qemu-server/container/manager/docs v4 00/41] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 01/41] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 02/41] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 03/41] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 04/41] pmxcfs: quorum: record the corosync membership Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 05/41] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 06/41] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 07/41] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 08/41] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 09/41] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 10/41] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-19 10:33 ` Thomas Lamprecht [this message]
2026-11-19 10:33 ` [PATCH guest-common v4 12/41] migrate: add shared helpers for the identity command of the migration tunnel Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 13/41] vzdump: notes template: list the uid variable Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 14/41] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 15/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 16/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 17/41] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 18/41] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 19/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 20/41] restore: decide by the reservation whether the container is new Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 21/41] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 22/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 23/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 24/41] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 25/41] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 26/41] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 27/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 28/41] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 29/41] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-19 12:32   ` Lukas Wagner
2026-11-19 12:51     ` Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 30/41] ui: datacenter summary: show the " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 31/41] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 32/41] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 33/41] ui: guest summary: show " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 34/41] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 35/41] vzdump: notes template: add the guest uid as variable Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 36/41] vzdump: restore: describe what a failed restore can leave behind Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 37/41] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 38/41] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 39/41] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 40/41] pct: " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 41/41] vzdump: notes: document the uid template variable Thomas Lamprecht
2026-11-19 14:46 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v4 00/41] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
2026-11-19 19:49 ` superseded: " Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119103551.1666964-12-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal