public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH container v4 24/41] restore: take over the guest uid from the backup where possible
Date: Thu, 19 Nov 2026 11:34:10 +0100	[thread overview]
Message-ID: <20261119103551.1666964-25-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20261119103551.1666964-1-t.lamprecht@proxmox.com>

A restore to an unused VMID, or over a container without an identity,
should keep the uid of the backup if no other guest in the cluster uses
it. Take over that uid after the data is restored and the final config
is written, while holding the config lock. With 'unique', or if the uid
is in use, the container keeps the identity that it got when its VMID
was reserved. In all these cases the registration time is the time of
the restore. A container that already had an identity keeps both its uid
and its registration time.

As the uid is only taken over after the restore succeeded, a failed
restore does not take the uid for a container that it leaves behind. If
the cluster-wide lock cannot be taken, warn, but do not fail the
completed restore. The container then keeps its own identity. Report the
result in the task log.

If the restore is one that could take the uid over, record the uid of
the backup as a pending change of 'meta' until the restore finishes. A
retry over a container that an unfinished restore left behind can then
take the uid over, if no other guest uses it. Record it with 'unique'
too, as that option only keeps the current restore from taking the uid.
If the cluster-wide lock cannot be taken, the pending uid stays for a
later attempt.

A pending uid does not count as in use, so a retry at another VMID stays
possible. Outside of a restore, applying pending changes drops it.

For restores from a file, PBS or a backup provider, record the pending
uid as soon as the config of the backup is known, and keep it in the
configs that the restore writes afterwards. For standard input, the uid
is only known from the restored config, because API parameters cannot
set 'meta'. That config has the pending uid, also if the conversion to a
template is what writes it first. Archives without a PVE config have no
uid.

The uid that gets checked and taken over must be the uid of the data
that gets restored. An archive file could get replaced between reading
its config and extracting it. So open the file once, when reading its
config, and extract it through that handle, which a replaced file does
not affect. The config of a backup on a Proxmox Backup Server or of a
backup provider is read a second time for the merge. Fail the restore if
its uid then differs from the one read first.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---

changes v3 -> v4:
* open an archive file once and extract it through that handle, so that
  the checked uid is the one of the restored data even if the file gets
  replaced in between; the uid check after the merge stays for PBS and
  backup providers
* the cleanup and the Sys.Modify check that follow the reservation moved
  to a patch of their own before this one
* the warning about a removed firewall config is gone: an independent
  fix before the series puts the previous firewall config back instead
* use prepare_restore of pve-guest-common for the pending uid
* tests: extraction through the handle; the cases about the reservation
  and the firewall config went with their code
* commit message: plainer wording, same content

changes v2 -> v3:
* the cleanup of a failed restore and the Sys.Modify check for a new
  privileged container follow what the reservation found, not the check
  before it
* fail the restore if the uid of the extracted config differs from the
  one checked before, as the archive got opened twice
* warn if the cleanup of a failed create or restore removes a firewall
  config that existed for the free VMID before
* the lookup double of the tests reads a bare 'meta:' line like pmxcfs
  does

changes v1 -> v2:
* use the shared helper of pve-guest-common for the take-over
* describe the effect of 'unique'
* test the order of the cleanup after a failed restore

 src/PVE/API2/LXC.pm         |  44 +++-
 src/PVE/LXC/Create.pm       |  26 ++-
 src/test/lifecycle_tests.pl | 424 ++++++++++++++++++++++++++++++++++++
 3 files changed, 481 insertions(+), 13 deletions(-)

diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 91e6678..99f3d01 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -257,7 +257,8 @@ __PACKAGE__->register_method({
             unique => {
                 optional => 1,
                 type => 'boolean',
-                description => "Assign a unique random ethernet address.",
+                description => "Assign a unique random ethernet address and do not take over"
+                    . " the guest uid of the backup.",
                 requires => 'restore',
             },
             pool => {
@@ -515,6 +516,8 @@ __PACKAGE__->register_method({
             my $was_template;
             # for the cleanup after a failure, a restore can replace the firewall config
             my $old_firewall_config = PVE::GuestHelpers::get_firewall_config_raw($vmid);
+            my ($backup_uid, $pending_uid);
+            my $archive_fh;
 
             my $vollist = [];
             my $old_volumes_left = [];
@@ -526,8 +529,14 @@ __PACKAGE__->register_method({
                     if ($archive ne '-') {
                         my $orig_conf;
                         print "recovering backed-up configuration from '$archive'\n";
-                        ($orig_conf, $orig_mp_param) =
+                        # the handle is for extracting the archive file that the config is from
+                        ($orig_conf, $orig_mp_param, $archive_fh) =
                             PVE::LXC::Create::recover_config($storage_cfg, $archive, $vmid);
+                        $backup_uid = (PVE::LXC::Config->get_identity($orig_conf) // {})->{uid};
+                        # a container with another uid keeps it, which only gets logged
+                        $pending_uid = PVE::GuestIdentity::prepare_restore(
+                            'PVE::LXC::Config', $vmid, $reservation, $backup_uid, 1,
+                        );
 
                         for my $opt (keys %$orig_conf) {
                             # early check before disks are created
@@ -625,12 +634,11 @@ __PACKAGE__->register_method({
 
                 # we always have the 'create' lock so check for more than 1 entry
                 if (scalar(keys %$old_conf) > 1) {
+                    my $replacement_conf = { lock => 'create' };
+                    PVE::GuestIdentity::set_pending_uid($replacement_conf, $pending_uid);
                     # destroy old container volumes
                     $old_volumes_left = PVE::LXC::destroy_lxc_container(
-                        $storage_cfg,
-                        $vmid,
-                        $old_conf,
-                        { lock => 'create' },
+                        $storage_cfg, $vmid, $old_conf, $replacement_conf,
                     );
                 }
                 # up until here the old container is intact, besides its lock
@@ -662,6 +670,7 @@ __PACKAGE__->register_method({
                             $conf,
                             $ignore_unpack_errors,
                             $bwlimit,
+                            $archive_fh,
                         );
                     }
 
@@ -677,6 +686,22 @@ __PACKAGE__->register_method({
                             $unique,
                             $overwrite_only,
                         );
+                        # the API parameters cannot carry 'meta', so it comes from the backup
+                        my $restored_uid = (PVE::LXC::Config->get_identity($conf) // {})->{uid};
+                        if ($archive eq '-') {
+                            $backup_uid = $restored_uid;
+                            $pending_uid = PVE::GuestIdentity::prepare_restore(
+                                'PVE::LXC::Config', $vmid, $reservation, $backup_uid, 1,
+                            );
+                        } else {
+                            # the merge read the config of the backup again, it must have the uid
+                            # that was checked
+                            my ($checked, $restored) =
+                                map { $_ // 'none' } ($backup_uid, $restored_uid);
+                            die "backup changed during the restore: its uid '$restored' differs"
+                                . " from uid '$checked' read before\n"
+                                if $restored ne $checked;
+                        }
                         PVE::LXC::create_ifaces_ipams_ips($conf, $vmid) if $unique;
                         my $lxc_setup = PVE::LXC::Setup->new($conf, $rootdir);
                         $lxc_setup->post_restore_hook($password, $ssh_keys);
@@ -700,6 +725,8 @@ __PACKAGE__->register_method({
                 foreach my $mp (keys %$delayed_mp_param) {
                     $conf->{$mp} = $delayed_mp_param->{$mp};
                 }
+                # the template conversion writes the restored config, too
+                PVE::GuestIdentity::set_pending_uid($conf, $pending_uid);
                 # If the template flag was set, we try to convert again to template after restore
                 if ($was_template) {
                     print STDERR "Convert restored container to template...\n";
@@ -739,6 +766,11 @@ __PACKAGE__->register_method({
                 }
                 die "$emsg $err";
             }
+            if ($restore) {
+                PVE::GuestIdentity::adopt_from_backup(
+                    'PVE::LXC::Config', $vmid, $unique ? undef : $backup_uid, $reservation,
+                );
+            }
             PVE::AccessControl::add_vm_to_pool($vmid, $pool) if $pool;
         };
 
diff --git a/src/PVE/LXC/Create.pm b/src/PVE/LXC/Create.pm
index a517af3..472e19b 100644
--- a/src/PVE/LXC/Create.pm
+++ b/src/PVE/LXC/Create.pm
@@ -22,8 +22,9 @@ use PVE::VZDump::ConvertOVZ;
 use PVE::Tools;
 use POSIX;
 
+# $archive_fh is the handle that recover_config returned for $archive, if any.
 sub restore_archive {
-    my ($storage_cfg, $archive, $rootdir, $conf, $no_unpack_error, $bwlimit) = @_;
+    my ($storage_cfg, $archive, $rootdir, $conf, $no_unpack_error, $bwlimit, $archive_fh) = @_;
 
     my ($storeid, $volname) = PVE::Storage::parse_volume_id($archive, 1);
     if (defined($storeid)) {
@@ -48,7 +49,7 @@ sub restore_archive {
     }
 
     $archive = PVE::Storage::abs_filesystem_path($storage_cfg, $archive) if $archive ne '-';
-    restore_tar_archive($archive, $rootdir, $conf, $no_unpack_error, $bwlimit);
+    restore_tar_archive($archive, $rootdir, $conf, $no_unpack_error, $bwlimit, 0, $archive_fh);
 }
 
 sub restore_proxmox_backup_archive {
@@ -193,8 +194,9 @@ my sub restore_tar_archive_command {
     return $cmd;
 }
 
+# Extracts $archive, through the handle $opened_fh of it if given.
 sub restore_tar_archive {
-    my ($archive, $rootdir, $conf, $no_unpack_error, $bwlimit, $untrusted) = @_;
+    my ($archive, $rootdir, $conf, $no_unpack_error, $bwlimit, $untrusted, $opened_fh) = @_;
 
     my $archive_fh;
     my $tar_input = '<&STDIN';
@@ -202,8 +204,11 @@ sub restore_tar_archive {
     if ($archive ne '-') {
         # GNU tar refuses to autodetect this... *sigh*
         $compression_opt = tar_compression_option($archive);
-        sysopen($archive_fh, $archive, O_RDONLY)
-            or die "failed to open '$archive': $!\n";
+        $archive_fh = $opened_fh;
+        if (!$archive_fh) {
+            sysopen($archive_fh, $archive, O_RDONLY)
+                or die "failed to open '$archive': $!\n";
+        }
         my $flags = $archive_fh->fcntl(Fcntl::F_GETFD(), 0);
         $archive_fh->fcntl(Fcntl::F_SETFD(), $flags & ~(Fcntl::FD_CLOEXEC()));
         $tar_input = '<&' . fileno($archive_fh);
@@ -223,7 +228,7 @@ sub restore_tar_archive {
     }
     eval { PVE::Tools::run_command($cmd, input => $tar_input); };
     my $err = $@;
-    close($archive_fh) if defined $archive_fh;
+    close($archive_fh) if defined($archive_fh) && !$opened_fh;
     die $err if $err && !$no_unpack_error;
 }
 
@@ -285,6 +290,9 @@ sub restore_external_archive {
 
 }
 
+# For an archive file, the third return value is a handle of it. Extracting the archive through that
+# handle with restore_archive gets the data that the config belongs to, even if the path names
+# another file by then.
 sub recover_config {
     my ($storage_cfg, $volid, $vmid) = @_;
 
@@ -299,7 +307,11 @@ sub recover_config {
     }
 
     my $archive = PVE::Storage::abs_filesystem_path($storage_cfg, $volid);
-    recover_config_from_tar($archive, $vmid);
+    sysopen(my $fh, $archive, O_RDONLY) or die "failed to open '$archive': $!\n";
+    # this path opens the file of the handle again, also for child processes
+    my ($conf, $mp_param) = recover_config_from_tar("/proc/$$/fd/" . fileno($fh), $vmid);
+
+    return wantarray ? ($conf, $mp_param, $fh) : $conf;
 }
 
 sub recover_config_from_proxmox_backup {
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
index c10a3b9..a172393 100755
--- a/src/test/lifecycle_tests.pl
+++ b/src/test/lifecycle_tests.pl
@@ -14,6 +14,7 @@ BEGIN {
     $INC{'PVE/VZDump.pm'} = __FILE__;
 }
 
+use Fcntl qw(:flock);
 use File::Path qw(make_path remove_tree);
 use JSON;
 use Socket qw(AF_UNIX SOCK_STREAM PF_UNSPEC);
@@ -39,6 +40,8 @@ my $WORKDIR = 'lifecycle-working';
 my $NODENAME = PVE::INotify::nodename();
 my $BACKUP_VOLID = 'local:backup/vzdump-lxc-100-2026_01_01-00_00_00.tar.zst';
 my $PBS_VOLID = 'pbs:backup/ct/100/2026-01-01T00:00:00Z';
+my $BACKUP_META = 'ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba';
+my $EXISTING_META = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis';
 
 remove_tree($WORKDIR);
 make_path("$WORKDIR/lock");
@@ -103,6 +106,9 @@ my $storage_cfg = {
     },
 };
 
+# error of the cluster-wide guest identity lock, if set
+my $domain_lock_error;
+
 # reported Proxmox VE versions of the other cluster nodes
 my $node_versions = {
     'new-node' => PVE::GuestIdentity::MIN_PVE_MANAGER_VERSION,
@@ -145,6 +151,32 @@ $cluster_module->mock(
         return undef if $noerr;
         die "CT $vmid already exists on node '$NODENAME'\n";
     },
+    cfs_lock_domain => sub {
+        my ($domain, $timeout, $code, @param) = @_;
+        die "unexpected lock domain '$domain'\n" if $domain ne 'guest-identity';
+        if (defined($domain_lock_error)) {
+            $@ = $domain_lock_error;
+            return undef;
+        }
+        my $res = eval { $code->(@param) };
+        return $res;
+    },
+    get_guest_config_properties => sub {
+        my ($properties) = @_;
+        my $res = {};
+        for my $file (glob("$WORKDIR/cfs/nodes/*/{lxc,qemu-server}/*.conf")) {
+            my ($vmid) = $file =~ m|/(\d+)\.conf$|;
+            for my $line (split(/\n/, file_get_contents($file))) {
+                last if $line =~ m/^\[/;
+                for my $property (@$properties) {
+                    # like pmxcfs, a line with nothing after the colon does not count
+                    $res->{$vmid}->{$property} = $1 =~ s/^\s+|\s+$//gr
+                        if $line =~ m/^\Q$property\E:(.+)$/;
+                }
+            }
+        }
+        return $res;
+    },
     remote_node_ip => sub { return '192.0.2.1' },
     get_node_pvecfg_version => sub {
         my ($node) = @_;
@@ -496,6 +528,7 @@ my sub reset_state {
     ($backup_read_error, $hook_error, $in_between) = ();
     $backup_fw_config = undef;
     $now_ms = 1_700_000_000_000;
+    $domain_lock_error = undef;
 }
 
 # API call helpers
@@ -612,6 +645,17 @@ my sub identity_of($vmid) {
     return PVE::LXC::Config->get_identity(PVE::LXC::Config->load_config($vmid));
 }
 
+my sub pending_uid_of($vmid) {
+    return PVE::GuestIdentity::get_pending_uid(PVE::LXC::Config->load_config($vmid));
+}
+
+# Returns whether the config lock of $vmid is held, for mocks of functions that need it.
+my sub config_lock_held($vmid) {
+    open(my $fh, '>>', "$WORKDIR/lock/pve-config-$vmid.lock") or die "open failed - $!\n";
+
+    return !flock($fh, LOCK_EX | LOCK_NB);
+}
+
 # Tests
 
 subtest 'create and restore write the config of the container' => sub {
@@ -1753,6 +1797,386 @@ subtest 'clone to another node is refused if the node drops the guest identity'
     isnt($identity->{uid}, 'existingexistingexis', 'clone has a new uid');
 };
 
+subtest 'restore takes over the guest identity of the backup' => sub {
+    my $in_use =
+        "guest identity: backup uid backupbackupbackupba is in use by guest 150, keeping uid ";
+    my $adopted = qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m;
+    my $new = qr/^guest identity: new uid [a-z2-7]{20}$/m;
+
+    my $now = 1_700_000_000_000;
+
+    my $locked = 1;
+    my $adopt = PVE::GuestIdentity->can('adopt');
+    my $probe_lock = {
+        adopt => sub {
+            my ($class, $vmid) = @_;
+            $locked &&= config_lock_held($vmid);
+            return $adopt->(@_);
+        },
+    };
+
+    my $run = sub($name, $setup, $param, $expected_uid, $expected_rtime, $expected_log) {
+        reset_state();
+        $backup_raw_config = backup_config($BACKUP_META);
+        $setup->();
+        my $err = with_mock($identity_module, $probe_lock, sub { restore_ct(200, $param) });
+        is($err, '', "$name: restore succeeds") or diag($task_log);
+        my $identity = identity_of(200);
+        if (defined($expected_uid)) {
+            is($identity->{uid}, $expected_uid, "$name: uid");
+        } else {
+            like($identity->{uid}, qr/^[a-z2-7]{20}$/, "$name: uid is assigned");
+            isnt(
+                $identity->{uid},
+                'backupbackupbackupba',
+                "$name: uid is not the one of the backup",
+            );
+        }
+        is($identity->{rtime}, $expected_rtime, "$name: rtime");
+        like($task_log, $expected_log, "$name: result is logged");
+    };
+    my $without_identity = sub { write_raw_config(200, existing_config('ctime=1500000000')) };
+    my $same_identity = sub {
+        write_raw_config(
+            200,
+            existing_config($EXISTING_META =~ s/uid=.*/uid=backupbackupbackupba/r),
+        );
+    };
+    my $kept = qr/^guest identity: kept uid backupbackupbackupba$/m;
+    my $stdin = { ostemplate => '-', rootfs => 'local:4' };
+
+    $run->('unused VMID', sub { }, {}, 'backupbackupbackupba', $now, $adopted);
+    $run->(
+        'unused VMID, uid used by another container',
+        sub { write_raw_config(150, existing_config('uid=backupbackupbackupba')) },
+        {},
+        undef,
+        $now,
+        qr/^\Q$in_use\E[a-z2-7]{20}$/m,
+    );
+    $run->('unused VMID, unique', sub { }, { unique => 1 }, undef, $now, $new);
+    $run->(
+        'unused VMID, backup without identity',
+        sub { $backup_raw_config = backup_config('ctime=1400000000') },
+        {},
+        undef,
+        $now,
+        $new,
+    );
+    $run->(
+        'unused VMID, from STDIN', sub { }, $stdin, 'backupbackupbackupba', $now, $adopted,
+    );
+    $run->(
+        'unused VMID, from STDIN without config',
+        sub { $backup_raw_config = undef },
+        $stdin,
+        undef,
+        $now,
+        $new,
+    );
+    $run->(
+        'in place, container without identity',
+        $without_identity,
+        { force => 1 },
+        'backupbackupbackupba',
+        $now,
+        $adopted,
+    );
+    $run->(
+        'in place, container without identity, unique',
+        $without_identity,
+        { force => 1, unique => 1 },
+        undef,
+        $now,
+        $new,
+    );
+    $run->(
+        'in place, container with the same identity',
+        $same_identity,
+        { force => 1 },
+        'backupbackupbackupba',
+        1500000000000,
+        $kept,
+    );
+    $run->(
+        'in place, container with the same identity, unique',
+        $same_identity,
+        { force => 1, unique => 1 },
+        'backupbackupbackupba',
+        1500000000000,
+        $kept,
+    );
+    $run->(
+        'in place, container with identity, backup without identity',
+        sub {
+            write_raw_config(200, existing_config($EXISTING_META));
+            $backup_raw_config = backup_config('ctime=1400000000');
+        },
+        { force => 1 },
+        'existingexistingexis',
+        1500000000000,
+        qr/^guest identity: kept uid existingexistingexis$/m,
+    );
+    $run->(
+        'in place, by a user with only VM.Backup',
+        sub {
+            $without_identity->();
+            $rpcenv = PVE::LXC::Test::RPCEnvironment->new('user@pve');
+            $rpcenv->{deny} = { '/vms/200' => { 'VM.Allocate' => 1 } };
+        },
+        { force => 1 },
+        'backupbackupbackupba',
+        $now,
+        $adopted,
+    );
+
+    ok($locked, 'config lock is held while taking over the identity');
+};
+
+# Restores over a container without identity and fails after reading the backup. That leaves the
+# container behind with the uid of the backup as pending uid.
+my sub failed_restore_in_place($param = {}) {
+    reset_state();
+    $backup_raw_config = backup_config($BACKUP_META);
+    write_raw_config(200, existing_config('ctime=1500000000'));
+    my $full = { create_disks => sub { die "unable to allocate volume - storage full\n" } };
+    my $err = with_mock($lxc_module, $full, sub { restore_ct(200, { force => 1, $param->%* }) });
+    like($err, qr/storage full/, 'restore fails after reading the backup');
+    is(PVE::LXC::Config->load_config(200)->{hostname}, 'existing', 'container is kept');
+    is(pending_uid_of(200), 'backupbackupbackupba', 'it has the uid of the backup as pending uid');
+}
+
+subtest 'retry of a failed restore takes over the uid of the backup' => sub {
+    reset_state();
+    $backup_raw_config = backup_config($BACKUP_META);
+    $hook_error = "setup hook failed\n";
+    like(restore_ct(200), qr/setup hook failed/, 'restore to an unused VMID fails');
+    ok(!defined(read_raw_config(200)), 'its config is removed');
+    $hook_error = undef;
+    is(restore_ct(200), '', 'retry succeeds') or diag($task_log);
+    is(identity_of(200)->{uid}, 'backupbackupbackupba', 'retry takes over the uid');
+
+    failed_restore_in_place();
+    isnt(identity_of(200)->{uid}, 'backupbackupbackupba', 'container has a uid of its own');
+    is(restore_ct(200, { force => 1 }), '', 'retry over the container succeeds')
+        or diag($task_log);
+    is(identity_of(200)->{uid}, 'backupbackupbackupba', 'retry takes over the uid');
+    is(pending_uid_of(200), undef, 'pending uid is removed');
+};
+
+subtest 'retry elsewhere, then over the container left behind' => sub {
+    failed_restore_in_place();
+    my $leftover = identity_of(200)->{uid};
+
+    my $err = restore_ct(201);
+    is($err, '', 'restore to another VMID succeeds') or diag($task_log);
+    is(identity_of(201)->{uid}, 'backupbackupbackupba', 'pending uid is not in use');
+    is(pending_uid_of(200), 'backupbackupbackupba', 'container left behind keeps it');
+
+    $err = restore_ct(200, { force => 1 });
+    is($err, '', 'restore over the container left behind succeeds') or diag($task_log);
+    like(
+        $task_log,
+        qr/: backup uid backupbackupbackupba is in use by guest 201, keeping uid $leftover$/m,
+        'uid of the backup is in use by now',
+    );
+    is(identity_of(200)->{uid}, $leftover, 'container left behind keeps its uid');
+    is(pending_uid_of(200), undef, 'pending uid is removed');
+};
+
+subtest 'using the container left behind drops its pending uid' => sub {
+    failed_restore_in_place();
+
+    my $err = update_ct(200, { description => 'used' });
+    is($err, '', 'config change of the stopped container') or diag($task_log);
+    my $dropped = "guest identity: dropping pending uid backupbackupbackupba of an unfinished"
+        . " restore - a retry now needs 'allow-identity-mismatch'";
+    like($task_log, qr/^WARN: \Q$dropped\E$/m, 'dropped');
+    is(pending_uid_of(200), undef, 'pending uid is gone');
+};
+
+subtest 'restore with unique keeps a pending uid, but never takes it over' => sub {
+    for my $retry ({ unique => 1 }, {}) {
+        my $what = $retry->{unique} ? 'retry with unique' : 'retry without unique';
+
+        failed_restore_in_place({ unique => 1 });
+        my $leftover = identity_of(200)->{uid};
+
+        my $err = restore_ct(200, { force => 1, $retry->%* });
+        is($err, '', "$what succeeds") or diag($task_log);
+        is(pending_uid_of(200), undef, "$what: pending uid is removed");
+        is(
+            identity_of(200)->{uid},
+            $retry->{unique} ? $leftover : 'backupbackupbackupba',
+            "$what: "
+                . ($retry->{unique} ? 'keeps its uid' : 'takes over the uid of the backup'),
+        );
+    }
+};
+
+subtest 'failed adoption keeps the pending uid' => sub {
+    reset_state();
+    $backup_raw_config = backup_config($BACKUP_META);
+    $domain_lock_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+    my $err = restore_ct(200);
+    $domain_lock_error = undef;
+    is($err, '', 'restore succeeds') or diag($task_log);
+    my $uid = identity_of(200)->{uid};
+    isnt($uid, 'backupbackupbackupba', 'container has a uid of its own');
+    my $warning =
+        "WARN: guest identity: unable to adopt backup uid backupbackupbackupba, keeping"
+        . " uid $uid - cfs-lock 'domain-guest-identity' error: no quorum!";
+    like($task_log, qr/^\Q$warning\E$/m, 'failure is logged as warning');
+    is(pending_uid_of(200), 'backupbackupbackupba', 'nothing decided, pending uid stays');
+
+    $err = restore_ct(200, { force => 1 });
+    is($err, '', 'restore of the backup again succeeds') or diag($task_log);
+    is(identity_of(200)->{uid}, 'backupbackupbackupba', 'uid of the backup is taken over');
+};
+
+subtest 'config of a container left behind shows its stored identity' => sub {
+    failed_restore_in_place();
+    my $meta = PVE::LXC::Config->load_config(200)->{meta};
+
+    my $conf = PVE::API2::LXC::Config->vm_config({ node => $NODENAME, vmid => 200 });
+    is($conf->{meta}, $meta, 'config with pending changes applied: stored identity');
+    $conf = PVE::API2::LXC::Config->vm_config({ node => $NODENAME, vmid => 200, current => 1 });
+    is($conf->{meta}, $meta, 'current config: stored identity');
+
+    my $pending = PVE::API2::LXC->vm_pending({ node => $NODENAME, vmid => 200 });
+    my ($item) = grep { $_->{key} eq 'meta' } $pending->@*;
+    is($item->{value}, $meta, 'pending view: current value');
+    is($item->{pending}, 'uid=backupbackupbackupba', 'pending view: pending uid');
+};
+
+# Once a write of the config of CT 200 has the pending uid of the backup, all later writes of the
+# restore have it, so that the container keeps it whenever the restore stops. Only the adoption
+# removes it.
+subtest 'every write of a restore keeps the pending uid' => sub {
+    my $cases = [
+        ['unused VMID', undef, {}],
+        ['in place over a container without uid', existing_config('ctime=1500000000'), {}],
+        ['unused VMID, from STDIN', undef, { ostemplate => '-', rootfs => 'local:4' }],
+        ['unused VMID, template', undef, {}, "template: 1\n"],
+    ];
+
+    for my $case (@$cases) {
+        my ($what, $target, $param, $extra) = @$case;
+
+        reset_state();
+        $backup_raw_config = backup_config($BACKUP_META) . ($extra // '');
+        write_raw_config(200, $target) if $target;
+        my $restore = sub { restore_ct(200, { $target ? (force => 1) : (), $param->%* }) };
+        my $err =
+            with_mock($storage_module, { volume_has_feature => sub { return 0 } }, $restore);
+        is($err, '', "$what: restore succeeds") or diag($task_log);
+        like(
+            $task_log,
+            qr/^guest identity: adopted uid backupbackupbackupba/m,
+            "$what: adopted",
+        );
+
+        my @writes = $written->{200}->@*;
+        my $last = pop(@writes);
+        my $first = 0;
+        $first++ while $first < @writes && $writes[$first] !~ m/^\[pve:pending\]/m;
+        ok($first < @writes, "$what: pending uid is written") or next;
+        my @later = @writes[$first .. $#writes];
+        is(
+            scalar(grep { m/^\[pve:pending\]\nmeta: uid=backupbackupbackupba$/m } @later),
+            scalar(@later),
+            "$what: all later writes keep it",
+        );
+        unlike($last, qr/^\[pve:pending\]/m, "$what: adoption removes it");
+    }
+};
+
+subtest 'an archive file is extracted through the handle that its config was read from' => sub {
+    my $dir = "$WORKDIR/dump";
+    my $path = "$dir/vzdump-lxc-100-2026_01_01-00_00_00.tar.gz";
+    my $make_archive = sub($target, $uid) {
+        my $source = "$WORKDIR/archive-source";
+        remove_tree($source);
+        make_path("$source/etc/vzdump", $dir);
+        file_set_contents("$source/etc/vzdump/pct.conf", backup_config("uid=$uid"));
+        PVE::Tools::run_command(['tar', '-czf', $target, '-C', $source, './etc']);
+    };
+    $make_archive->($path, 'backupbackupbackupba');
+    $make_archive->("$dir/other.tar.gz", 'anotheranotheranothe');
+
+    # the path gets to name another archive once the file is open, before its config is read
+    my $extract_config = PVE::Storage->can('extract_vzdump_config_tar');
+    my $replace = {
+        extract_vzdump_config_tar => sub {
+            rename("$dir/other.tar.gz", $path) or die "rename failed - $!\n";
+            return $extract_config->(@_);
+        },
+    };
+    my $recover_config = $create_module->original('recover_config');
+    my ($conf, undef, $fh) =
+        with_mock($storage_module, $replace, sub { [$recover_config->($storage_cfg, $path, 200)] })
+        ->@*;
+    is($conf->{meta}, 'uid=backupbackupbackupba', 'config is read from the opened file');
+    ok($fh, 'the handle of the file is returned');
+
+    reset_state();
+    my $rootdir = "$WORKDIR/rootfs/200";
+    make_path($rootdir);
+    my $err = run_task(sub {
+        $create_module->original('restore_archive')
+            ->($storage_cfg, $path, $rootdir, {}, 0, undef, $fh);
+    });
+    is($err, '', 'extraction succeeds') or diag($task_log);
+    like(
+        file_get_contents("$rootdir/etc/vzdump/pct.conf"),
+        qr/^meta: uid=backupbackupbackupba$/m,
+        'data is extracted from the opened file',
+    );
+    close($fh);
+
+    $conf = $recover_config->($storage_cfg, $path, 200);
+    is($conf->{meta}, 'uid=anotheranotheranothe', 'the path names another archive by now');
+};
+
+subtest 'restore fails if the backup changed after its uid was read' => sub {
+    # the merge of the config reads the backup again and finds another one
+    my $mocked_restore_archive = PVE::LXC::Create->can('restore_archive');
+    my $changes = {
+        restore_archive => sub {
+            $backup_raw_config =
+                backup_config($BACKUP_META =~ s/uid=.*/uid=anotheranotheranothe/r);
+            return $mocked_restore_archive->(@_);
+        },
+    };
+    my $changed = "unable to restore CT 200 - backup changed during the restore: its uid"
+        . " 'anotheranotheranothe' differs from uid 'backupbackupbackupba' read before";
+
+    for my $archive ($BACKUP_VOLID, $PBS_VOLID) {
+        my $name = $archive eq $PBS_VOLID ? 'backup server' : 'archive file';
+        my $restore = sub($param = {}) {
+            $backup_raw_config = backup_config($BACKUP_META);
+            my $call = sub { restore_ct(200, { ostemplate => $archive, $param->%* }) };
+            return with_mock($create_module, $changes, $call);
+        };
+
+        reset_state();
+        like($restore->(), qr/^\Q$changed\E$/, "$name, unused VMID: restore fails");
+        ok(!defined(read_raw_config(200)), "$name, unused VMID: nothing is left behind");
+
+        # the checked backup has the uid of the container
+        reset_state();
+        write_raw_config(
+            200,
+            existing_config($EXISTING_META =~ s/uid=.*/uid=backupbackupbackupba/r),
+        );
+        like($restore->({ force => 1 }), qr/^\Q$changed\E$/, "$name, in place: restore fails");
+        ok(
+            !defined(read_raw_config(200)),
+            "$name, in place: the other backup is not left behind with the uid of the container",
+        );
+    }
+};
+
 remove_tree($WORKDIR);
 
 done_testing();
-- 
2.47.3





  parent reply	other threads:[~2026-11-19 10:42 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 10:33 [PATCH cluster/guest-common/qemu-server/container/manager/docs v4 00/41] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 01/41] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 02/41] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 03/41] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 04/41] pmxcfs: quorum: record the corosync membership Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 05/41] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 06/41] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 07/41] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 08/41] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 09/41] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 10/41] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 11/41] guest identity: allow adopting a uid that is unused in the cluster Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 12/41] migrate: add shared helpers for the identity command of the migration tunnel Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 13/41] vzdump: notes template: list the uid variable Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 14/41] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 15/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 16/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 17/41] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 18/41] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 19/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 20/41] restore: decide by the reservation whether the container is new Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 21/41] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 22/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 23/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` Thomas Lamprecht [this message]
2026-11-19 10:34 ` [PATCH container v4 25/41] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 26/41] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 27/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 28/41] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 29/41] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-19 12:32   ` Lukas Wagner
2026-11-19 12:51     ` Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 30/41] ui: datacenter summary: show the " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 31/41] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 32/41] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 33/41] ui: guest summary: show " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 34/41] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 35/41] vzdump: notes template: add the guest uid as variable Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 36/41] vzdump: restore: describe what a failed restore can leave behind Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 37/41] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 38/41] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 39/41] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 40/41] pct: " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 41/41] vzdump: notes: document the uid template variable Thomas Lamprecht
2026-11-19 14:46 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v4 00/41] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
2026-11-19 19:49 ` superseded: " Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119103551.1666964-25-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal