From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id C24D61FF0C0 for ; Wed, 18 Nov 2026 05:44:37 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id E28C321BE5; Wed, 18 Nov 2026 05:39:39 +0100 (CET) From: Thomas Lamprecht To: pve-devel@lists.proxmox.com Subject: [PATCH container v3 24/38] restore: take over the guest uid from the backup where possible Date: Wed, 18 Nov 2026 05:37:36 +0100 Message-ID: <20261118043817.1659212-25-t.lamprecht@proxmox.com> X-Mailer: git-send-email 2.47.3 In-Reply-To: <20261118043817.1659212-1-t.lamprecht@proxmox.com> References: <20261118043817.1659212-1-t.lamprecht@proxmox.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1794976703071 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.185 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: XYO5NL54DQQCVGD6CQU2GFOX4SS5QLDQ X-Message-ID-Hash: XYO5NL54DQQCVGD6CQU2GFOX4SS5QLDQ X-MailFrom: t.lamprecht@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: A restore to an unused VMID, or over a container without an identity, should keep the uid of the backup if no other guest in the cluster uses it. Take over that uid after the data is restored and the final config is written, while holding the config lock. With 'unique', or if the uid is in use, the container keeps the identity assigned when its VMID was reserved. In all these cases the registration time is the time of the restore. A container that already had an identity keeps both its uid and its registration time. Taking over the uid only after the restore succeeded means that a failed restore does not take the uid for a container it leaves behind. If the cluster-wide lock cannot be taken, warn and do not fail the completed restore; the container keeps its own identity. Report the outcome in the task log. Record the uid of the backup as a pending change of 'meta' until the restore finishes, if the restore could take it over. A retry over a container left behind by an unfinished restore can then take the uid over, if no other guest uses it. Record it with 'unique' too, which only keeps the current restore from taking the uid. If the cluster-wide lock cannot be taken, the pending uid stays for a later attempt. A pending uid does not count as in use, so a retry at another VMID stays possible. Outside of a restore, applying pending changes drops it. For restores from a file, PBS or a backup provider, record the pending uid as soon as the config of the backup is known and keep it in the configs that the restore writes afterwards. For standard input, the uid is only known from the restored config, as API parameters cannot set 'meta'. That config carries the pending uid, also if the conversion to a template writes it first. Archives without a PVE config have no uid. The extraction opens the archive again and merges the config in it. If that config has another uid than the one read before, the archive got replaced in between, so fail the restore instead of taking over a uid that does not belong to the restored data. Decide whether a failed restore removes the config by whether the reservation found one, not by the check before it. A container created in the meantime is then kept, and the reservation of a VMID whose container got removed in the meantime is removed again. The Sys.Modify check for a new privileged container follows the reservation for the same reason. A failed create or restore that removes a firewall config that existed before it, for example one written by hand, logs a warning about that. Signed-off-by: Thomas Lamprecht --- changes v2 -> v3: * the cleanup of a failed restore and the Sys.Modify check for a new privileged container follow what the reservation found, not the check before it * fail the restore if the uid of the extracted config differs from the one checked before, as the archive got opened twice * warn if the cleanup of a failed create or restore removes a firewall config that existed for the free VMID before * the lookup double of the tests reads a bare 'meta:' line like pmxcfs does changes v1 -> v2: * use the shared helper of pve-guest-common for the take-over * describe the effect of 'unique' * test the order of the cleanup after a failed restore src/PVE/API2/LXC.pm | 71 +++- src/test/lifecycle_tests.pl | 653 +++++++++++++++++++++++++++++++++++- 2 files changed, 714 insertions(+), 10 deletions(-) diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm index 40ec0f4..d231d24 100644 --- a/src/PVE/API2/LXC.pm +++ b/src/PVE/API2/LXC.pm @@ -19,6 +19,7 @@ use PVE::GuestID; use PVE::GuestIdentity; use PVE::INotify; use PVE::JSONSchema qw(get_standard_option); +use PVE::RESTEnvironment; use PVE::RESTHandler; use PVE::RPCEnvironment; use PVE::RRD; @@ -63,6 +64,25 @@ my sub assert_not_restore_from_external { die "refusing to restore privileged container backup from external source\n"; } +# Returns the uid of the backup if the restore could take it over at its end, see +# PVE::GuestIdentity::adoptable_uid. Until then, the restore keeps it as pending uid in every config +# it writes, so that a container left behind by a failure or an abort still has it for a retry. +# That holds with 'unique' too, which only keeps the restore itself from taking the uid over. +my sub restore_pending_uid { + my ($reservation, $backup_identity) = @_; + + my $uid = $backup_identity ? $backup_identity->{uid} : undef; + + return PVE::GuestIdentity::adoptable_uid($reservation, $uid); +} + +# Returns whether a firewall config exists for $vmid, in a function of its own for tests to mock. +sub __vmfw_conf_exists { + my ($vmid) = @_; + + return -e "/etc/pve/firewall/$vmid.fw"; +} + my $check_storage_access_migrate = sub { my ($rpcenv, $authuser, $storecfg, $storage, $node) = @_; @@ -199,7 +219,8 @@ __PACKAGE__->register_method({ unique => { optional => 1, type => 'boolean', - description => "Assign a unique random ethernet address.", + description => "Assign a unique random ethernet address and do not take over" + . " the guest uid of the backup.", requires => 'restore', }, pool => { @@ -433,12 +454,14 @@ __PACKAGE__->register_method({ my $emsg = $restore ? "unable to restore CT $vmid -" : "unable to create CT $vmid -"; # only overwrite a container that the checks above were done for - eval { + my $reservation = eval { PVE::LXC::Config->create_and_lock_config($vmid, $force && $same_container_exists); }; die "$emsg $@" if $@; - my $destroy_config_on_error = !$same_container_exists; + my $destroy_config_on_error = !$reservation->{existed}; + # the cleanup of a failure removes the firewall config, also one that existed before + my $vmfw_conf_existed = !$reservation->{existed} && __vmfw_conf_exists($vmid); my $code = sub { my $old_conf = PVE::LXC::Config->load_config($vmid); @@ -446,6 +469,7 @@ __PACKAGE__->register_method({ PVE::LXC::Config->check_protection($old_conf, "unable to restore CT $vmid") if $restore; my $was_template; + my ($backup_identity, $pending_uid); my $vollist = []; eval { @@ -457,6 +481,10 @@ __PACKAGE__->register_method({ print "recovering backed-up configuration from '$archive'\n"; ($orig_conf, $orig_mp_param) = PVE::LXC::Create::recover_config($storage_cfg, $archive, $vmid); + $backup_identity = PVE::LXC::Config->get_identity($orig_conf); + $pending_uid = restore_pending_uid($reservation, $backup_identity); + PVE::LXC::Config->record_pending_uid($vmid, $pending_uid) + if defined($pending_uid); for my $opt (keys %$orig_conf) { # early check before disks are created @@ -484,7 +512,7 @@ __PACKAGE__->register_method({ # implicit privileged change, or creating a new privileged container is checked here if ( - (!$same_container_exists || $old_conf->{unprivileged}) + (!$reservation->{existed} || $old_conf->{unprivileged}) && !$conf->{unprivileged} ) { $rpcenv->check($authuser, '/', ['Sys.Modify']); @@ -554,15 +582,14 @@ __PACKAGE__->register_method({ # we always have the 'create' lock so check for more than 1 entry if (scalar(keys %$old_conf) > 1) { + my $replacement_conf = { lock => 'create' }; + PVE::GuestIdentity::set_pending_uid($replacement_conf, $pending_uid); # destroy old container volumes PVE::LXC::destroy_lxc_container( - $storage_cfg, - $vmid, - $old_conf, - { lock => 'create' }, + $storage_cfg, $vmid, $old_conf, $replacement_conf, ); } - # up until here the old container is intact, besides its lock + # up until here the old container is intact, besides its lock and pending uid $destroy_config_on_error = 1; eval { @@ -603,6 +630,18 @@ __PACKAGE__->register_method({ $unique, $skip_fw_config_restore, ); + # the API parameters cannot carry 'meta', so it comes from the backup + if ($archive eq '-') { + $backup_identity = PVE::LXC::Config->get_identity($conf); + $pending_uid = restore_pending_uid($reservation, $backup_identity); + } else { + # the extraction opened the archive again, it must be the checked one + my ($checked, $extracted) = map { $_ ? $_->{uid} : 'none' } + ($backup_identity, PVE::LXC::Config->get_identity($conf)); + die "backup changed during the restore: its uid '$extracted'" + . " differs from uid '$checked' read before\n" + if $extracted ne $checked; + } PVE::LXC::create_ifaces_ipams_ips($conf, $vmid) if $unique; my $lxc_setup = PVE::LXC::Setup->new($conf, $rootdir); $lxc_setup->post_restore_hook($password, $ssh_keys); @@ -626,6 +665,8 @@ __PACKAGE__->register_method({ foreach my $mp (keys %$delayed_mp_param) { $conf->{$mp} = $delayed_mp_param->{$mp}; } + # the template conversion writes the restored config, too + PVE::GuestIdentity::set_pending_uid($conf, $pending_uid); # If the template flag was set, we try to convert again to template after restore if ($was_template) { print STDERR "Convert restored container to template...\n"; @@ -644,6 +685,12 @@ __PACKAGE__->register_method({ if (!$skip_fw_config_restore) { # Only if user has permission to change the fw eval { PVE::Firewall::remove_vmfw_conf($vmid) }; warn $@ if $@; + if ($vmfw_conf_existed && !__vmfw_conf_exists($vmid)) { + my $what = $restore ? 'restore' : 'create'; + my $msg = "removed the firewall config of CT $vmid that existed" + . " before the failed $what"; + PVE::RESTEnvironment::log_warn($msg); + } } eval { PVE::LXC::Config->destroy_config($vmid) }; @@ -651,6 +698,12 @@ __PACKAGE__->register_method({ } die "$emsg $err"; } + if ($restore) { + my $uid = $backup_identity ? $backup_identity->{uid} : undef; + PVE::GuestIdentity::adopt_from_backup( + 'PVE::LXC::Config', $vmid, $unique ? undef : $uid, $reservation, + ); + } PVE::AccessControl::add_vm_to_pool($vmid, $pool) if $pool; }; diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl index 32898fd..93d69b1 100755 --- a/src/test/lifecycle_tests.pl +++ b/src/test/lifecycle_tests.pl @@ -15,6 +15,7 @@ BEGIN { $INC{'PVE/VZDump.pm'} = __FILE__; } +use Fcntl qw(:flock); use File::Path qw(make_path remove_tree); use JSON; use Socket qw(AF_UNIX SOCK_STREAM PF_UNSPEC); @@ -58,16 +59,25 @@ my sub write_raw_config($vmid, $raw, $node = undef) { file_set_contents($file, $raw); } +my sub write_raw_vm_config($vmid, $raw) { + my $dir = cfs_file("nodes/$NODENAME/qemu-server"); + make_path($dir); + file_set_contents("$dir/$vmid.conf", $raw); +} + my sub read_raw_config($vmid, $node = undef) { my $file = cfs_file(ct_config_path($vmid, $node)); return -f $file ? file_get_contents($file) : undef; } +my $written = {}; # VMID => the written raw configs, in the order of the writes + # VMIDs whose container config still existed when their firewall config got removed my $vmfw_removed_reserved = {}; my sub reset_configs { remove_tree("$WORKDIR/cfs", "$WORKDIR/rootfs"); + $written = {}; $vmfw_removed_reserved = {}; } @@ -87,6 +97,9 @@ my $storage_cfg = { }, }; +# error of the cluster-wide guest identity lock, if set +my $domain_lock_error; + # reported Proxmox VE versions of the other cluster nodes my $node_versions = { 'new-node' => PVE::GuestIdentity::MIN_PVE_MANAGER_VERSION, @@ -121,6 +134,7 @@ $cluster_module->mock( my $file = cfs_file($path); make_path($file =~ s|/[^/]+$||r); file_set_contents($file, $raw); + push $written->{ ($path =~ m|/(\d+)\.conf$|)[0] }->@*, $raw; }, check_vmid_unused => sub { my ($vmid, $noerr) = @_; @@ -128,6 +142,32 @@ $cluster_module->mock( return undef if $noerr; die "CT $vmid already exists on node '$NODENAME'\n"; }, + cfs_lock_domain => sub { + my ($domain, $timeout, $code, @param) = @_; + die "unexpected lock domain '$domain'\n" if $domain ne 'guest-identity'; + if (defined($domain_lock_error)) { + $@ = $domain_lock_error; + return undef; + } + my $res = eval { $code->(@param) }; + return $res; + }, + get_guest_config_properties => sub { + my ($properties) = @_; + my $res = {}; + for my $file (glob("$WORKDIR/cfs/nodes/*/{lxc,qemu-server}/*.conf")) { + my ($vmid) = $file =~ m|/(\d+)\.conf$|; + for my $line (split(/\n/, file_get_contents($file))) { + last if $line =~ m/^\[/; + for my $property (@$properties) { + # like pmxcfs, a line with nothing after the colon does not count + $res->{$vmid}->{$property} = $1 =~ s/^\s+|\s+$//gr + if $line =~ m/^\Q$property\E:(.+)$/; + } + } + } + return $res; + }, remote_node_ip => sub { return '192.0.2.1' }, get_node_pvecfg_version => sub { my ($node) = @_; @@ -150,6 +190,8 @@ $api_module->mock( die "unexpected read of '$path'\n" if $path ne 'storage.cfg'; return $storage_cfg; }, + # no guest has a firewall config, unless a test says otherwise + __vmfw_conf_exists => sub { return 0 }, ); my $config_module = Test::MockModule->new('PVE::LXC::Config'); @@ -201,7 +243,12 @@ package PVE::LXC::Test::RPCEnvironment { } sub get_user { return $_[0]->{user} } - sub check { return 1 } + + sub check($self, $user, $path, $privs, $noerr = undef) { + return 1 if !$self->{deny}->{$path}; + return 0 if $noerr; + die "permission check failed ($path, " . join(', ', @$privs) . ")\n"; + } sub check_vm_perm { return 1 } sub check_pool_exist { return 1 } @@ -250,6 +297,7 @@ my sub run_task($code) { # Fake storage and container runtime my $backup_raw_config; +my $backup_read_error; my $storage_module = Test::MockModule->new('PVE::Storage'); $storage_module->mock( @@ -315,6 +363,7 @@ $create_module->mock( archive_is_oci_format => sub { return 0 }, recover_config => sub { my ($storecfg, $volid, $vmid) = @_; + die $backup_read_error if defined($backup_read_error); my $conf = PVE::LXC::Config::parse_pct_config("/lxc/$vmid.conf", $backup_raw_config); delete $conf->{snapshots}; my $mp_param = {}; @@ -1041,6 +1090,608 @@ subtest 'clone to another node is refused if the node drops the guest identity' isnt($identity->{uid}, 'existingexistingexis', 'clone has a new uid'); }; +subtest 'restore takes over the guest identity with the config lock held' => sub { + reset_configs(); + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'); + + my $real_adopt = \&PVE::GuestIdentity::adopt; + my $locked; + $identity_module->mock( + adopt => sub { + my ($class, $vmid) = @_; + open(my $fh, '>>', "$WORKDIR/lock/pve-config-$vmid.lock") + or die "open failed - $!\n"; + $locked = !flock($fh, LOCK_EX | LOCK_NB); + close($fh); + return $real_adopt->(@_); + }, + ); + my $err = restore_ct(200); + $identity_module->unmock('adopt'); + + is($err, '', 'restore succeeds') or diag($task_log); + ok($locked, 'config lock is held while taking over the identity'); + is(identity_of(200)->{uid}, 'backupbackupbackupba', 'uid of the backup is taken over'); +}; + +subtest 'restore takes over the guest identity of the backup' => sub { + my $backup_meta = 'ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'; + my $existing_meta = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis'; + $now_ms = 1_700_000_000_000; + + my $in_use_by_ct = + "guest identity: backup uid backupbackupbackupba is in use by guest 150, keeping uid "; + + my $run = sub { + my ($name, $setup, $param, $expected_uid, $expected_rtime, $expected_log) = @_; + + reset_configs(); + $setup->(); + my $err = restore_ct(200, $param); + is($err, '', "$name: restore succeeds") or diag($task_log); + my $identity = identity_of(200); + if (defined($expected_uid)) { + is($identity->{uid}, $expected_uid, "$name: uid"); + } else { + like($identity->{uid}, qr/^[a-z2-7]{20}$/, "$name: uid is assigned"); + isnt( + $identity->{uid}, + 'backupbackupbackupba', + "$name: uid is not the one of the backup", + ); + } + is($identity->{rtime}, $expected_rtime, "$name: rtime"); + like($task_log, $expected_log, "$name: result is logged"); + }; + + $backup_raw_config = backup_config($backup_meta); + $run->( + 'free VMID', + sub { }, + {}, + 'backupbackupbackupba', + $now_ms, + qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m, + ); + $run->( + 'free VMID, uid used by another container', + sub { write_raw_config(150, existing_config('uid=backupbackupbackupba')) }, + {}, + undef, + $now_ms, + qr/^\Q$in_use_by_ct\E[a-z2-7]{20}$/m, + ); + $run->( + 'free VMID, uid used by a VM', + sub { write_raw_vm_config(151, "memory: 512\nmeta: uid=backupbackupbackupba\n") }, + {}, + undef, + $now_ms, + qr/^guest identity: backup uid backupbackupbackupba is in use by guest 151, keeping uid /m, + ); + $run->( + 'free VMID, unique', + sub { }, + { unique => 1 }, + undef, + $now_ms, + qr/^guest identity: new uid [a-z2-7]{20}$/m, + ); + $run->( + 'in place, target without identity', + sub { write_raw_config(200, existing_config('ctime=1500000000')) }, + { force => 1 }, + 'backupbackupbackupba', + $now_ms, + qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m, + ); + $run->( + 'in place, target without identity, unique', + sub { write_raw_config(200, existing_config('ctime=1500000000')) }, + { force => 1, unique => 1 }, + undef, + $now_ms, + qr/^guest identity: new uid [a-z2-7]{20}$/m, + ); + $run->( + 'in place, target with the same identity', + sub { + write_raw_config( + 200, + existing_config( + 'ctime=1500000000,rtime=1500000000000,uid=backupbackupbackupba'), + ); + }, + { force => 1 }, + 'backupbackupbackupba', + 1500000000000, + qr/^guest identity: kept uid backupbackupbackupba$/m, + ); + $run->( + 'in place, target with the same identity, unique', + sub { + write_raw_config( + 200, + existing_config( + 'ctime=1500000000,rtime=1500000000000,uid=backupbackupbackupba'), + ); + }, + { force => 1, unique => 1 }, + 'backupbackupbackupba', + 1500000000000, + qr/^guest identity: kept uid backupbackupbackupba$/m, + ); + + $backup_raw_config = backup_config('ctime=1400000000'); + $run->( + 'free VMID, backup without identity', + sub { }, + {}, + undef, + $now_ms, + qr/^guest identity: new uid [a-z2-7]{20}$/m, + ); + $run->( + 'in place, target with identity, backup without identity', + sub { write_raw_config(200, existing_config($existing_meta)) }, + { force => 1 }, + 'existingexistingexis', + 1500000000000, + qr/^guest identity: kept uid existingexistingexis$/m, + ); + + $backup_raw_config = backup_config($backup_meta); + $run->( + 'free VMID, from STDIN', + sub { }, + { ostemplate => '-', rootfs => 'local:4' }, + 'backupbackupbackupba', + $now_ms, + qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m, + ); + $backup_raw_config = undef; + $run->( + 'free VMID, from STDIN without config', + sub { }, + { ostemplate => '-', rootfs => 'local:4' }, + undef, + $now_ms, + qr/^guest identity: new uid [a-z2-7]{20}$/m, + ); +}; + +subtest 'restore keeps the new guest identity if the identity lock fails' => sub { + reset_configs(); + $now_ms = 1_700_000_000_000; + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'); + $domain_lock_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n"; + + my $err = restore_ct(200); + $domain_lock_error = undef; + is($err, '', 'restore succeeds') or diag($task_log); + my $identity = identity_of(200); + like($identity->{uid}, qr/^[a-z2-7]{20}$/, 'uid is assigned'); + isnt($identity->{uid}, 'backupbackupbackupba', 'uid of the backup is not taken over'); + my $warning = + "WARN: guest identity: unable to adopt backup uid backupbackupbackupba, keeping" + . " uid $identity->{uid} - cfs-lock 'domain-guest-identity' error: no quorum!"; + like($task_log, qr/^\Q$warning\E$/m, 'failure is logged as warning'); +}; + +my sub pending_uid_of($vmid) { + return PVE::GuestIdentity::get_pending_uid(PVE::LXC::Config->load_config($vmid)); +} + +subtest 'failed restore keeps the uid of the backup only as pending uid' => sub { + reset_configs(); + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'); + + my $fake_setup_module = Test::MockModule->new('PVE::LXC::Test::Setup', no_auto => 1); + $fake_setup_module->mock(post_restore_hook => sub { die "post-restore hook failed\n" }); + + my $err = restore_ct(200); + like($err, qr/post-restore hook failed/, 'restore fails'); + ok(!defined(read_raw_config(200)), 'config of the free VMID is removed'); + ok( + $vmfw_removed_reserved->{200}, + 'its firewall config was removed while the VMID was still reserved', + ); + + $fake_setup_module->unmock('post_restore_hook'); + $err = restore_ct(200); + is($err, '', 'retry restores to a free VMID again') or diag($task_log); + like( + $task_log, + qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m, + 'retry takes over the uid of the backup', + ); + PVE::LXC::Config->destroy_config(200); + + my $existing_meta = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis'; + write_raw_config(200, existing_config($existing_meta)); + $backup_read_error = "unable to read backup\n"; + $err = restore_ct(200, { force => 1 }); + $backup_read_error = undef; + like($err, qr/unable to read backup/, 'restore fails before changing the container'); + my $conf = PVE::LXC::Config->load_config(200); + is($conf->{meta}, $existing_meta, 'target keeps its identity'); + ok(!defined($conf->{lock}), 'create lock is removed'); + + # a privileged backup needs Sys.Modify, which is checked after reading the backup + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba') =~ + s/^unprivileged: 1\n//mr; + write_raw_config(200, existing_config('ctime=1500000000')); + $rpcenv->{deny}->{'/'} = 1; + $err = restore_ct(200, { force => 1 }); + delete $rpcenv->{deny}; + like($err, qr/permission check failed/, 'restore fails after reading the backup'); + my $identity = identity_of(200); + like($identity->{uid}, qr/^[a-z2-7]{20}$/, 'target got an identity with the reservation'); + isnt($identity->{uid}, 'backupbackupbackupba', 'uid of the backup is not taken over'); + is(pending_uid_of(200), 'backupbackupbackupba', 'target has it as pending uid'); + ok(!defined(PVE::LXC::Config->load_config(200)->{lock}), 'create lock is removed'); + + $err = restore_ct(200, { force => 1 }); + is($err, '', 'retry of the same restore succeeds') or diag($task_log); + like( + $task_log, + qr/^guest identity: adopted uid backupbackupbackupba from the backup$/m, + 'retry takes over the uid of the backup', + ); + is(identity_of(200)->{uid}, 'backupbackupbackupba', 'stored'); + is(pending_uid_of(200), undef, 'pending uid is removed'); + + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'); + write_raw_config(200, existing_config('ctime=1500000000')); + my $mocked_create_disks = \&PVE::LXC::create_disks; + $lxc_module->mock(create_disks => sub { die "unable to allocate volume - storage full\n" }); + $err = restore_ct(200, { force => 1 }); + $lxc_module->mock(create_disks => $mocked_create_disks); + like($err, qr/storage full/, 'restore fails before replacing the container'); + is(PVE::LXC::Config->load_config(200)->{hostname}, 'existing', 'container is kept'); + is(pending_uid_of(200), 'backupbackupbackupba', 'container has the pending uid'); + $err = restore_ct(200, { force => 1 }); + is($err, '', 'retry succeeds') or diag($task_log); + is( + identity_of(200)->{uid}, + 'backupbackupbackupba', + 'retry takes over the uid of the backup', + ); +}; + +subtest 'restore only changes the container that the checks were done for' => sub { + my $existing_raw = + existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis'); + my $other_raw = + existing_config('ctime=1500000000,rtime=1600000000000,uid=otherotherotherother') =~ + s/^hostname: existing$/hostname: other/mr; + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'); + + my $conf_of = sub { + my ($raw) = @_; + my $conf = PVE::LXC::Config::parse_pct_config('/lxc/200.conf', $raw); + delete $conf->{digest}; + return $conf; + }; + my $current_conf = sub { + my $raw = read_raw_config(200); + return defined($raw) ? $conf_of->($raw) : undef; + }; + + # runs between the checks of the create API call and its reservation + my $in_between; + my $mocked_activate = \&PVE::Storage::activate_storage; + $storage_module->mock( + activate_storage => sub { + if (my $code = $in_between) { + undef $in_between; + $code->(); + } + return $mocked_activate->(@_); + }, + ); + + reset_configs(); + $in_between = sub { write_raw_config(200, $other_raw) }; + $rpcenv = PVE::LXC::Test::RPCEnvironment->new('user@pve'); + $rpcenv->{deny}->{'/vms/200'} = 1; + my $err = restore_ct(200, { force => 1, pool => 'pool1' }); + $rpcenv = PVE::LXC::Test::RPCEnvironment->new('root@pam'); + like($err, qr/CT 200 already exists/, 'created in between, allowed by the pool: refused'); + is_deeply( + $current_conf->(), + $conf_of->($other_raw), + 'created in between: config is unchanged', + ); + + reset_configs(); + write_raw_config(200, $existing_raw); + $in_between = sub { write_raw_config(200, $existing_raw . "protection: 1\n") }; + $err = restore_ct(200, { force => 1 }); + like( + $err, + qr/^unable to restore CT 200 - protection mode enabled$/, + 'protection set in between: refused', + ); + is_deeply( + $current_conf->(), + $conf_of->($existing_raw . "protection: 1\n"), + 'protection set in between: config is unchanged', + ); + + # the guest list of the reservation is the one of the start of the API call + my $mocked_unused = \&PVE::Cluster::check_vmid_unused; + $cluster_module->mock(check_vmid_unused => sub { return 1 }); + reset_configs(); + $in_between = sub { write_raw_config(200, $other_raw) }; + $backup_read_error = "unable to read backup\n"; + $err = restore_ct(200); + $backup_read_error = undef; + $cluster_module->mock(check_vmid_unused => $mocked_unused); + isnt($err, '', 'created in between, early failure: restore fails'); + is_deeply($current_conf->(), $conf_of->($other_raw), 'early failure: config is unchanged'); + ok(!$vmfw_removed_reserved->{200}, 'early failure: firewall config is kept'); + + reset_configs(); + write_raw_config(200, $existing_raw); + $in_between = sub { PVE::LXC::Config->destroy_config(200) }; + $backup_read_error = "unable to read backup\n"; + $err = restore_ct(200, { force => 1 }); + $backup_read_error = undef; + like($err, qr/unable to read backup/, 'removed in between, early failure: restore fails'); + ok(!defined(read_raw_config(200)), 'removed in between: the VMID is free again'); + + $storage_module->mock(activate_storage => $mocked_activate); +}; + +subtest 'failed create or restore warns if it removes an older firewall config' => sub { + my $vmfw_exists = {}; + $api_module->mock( + __vmfw_conf_exists => sub { + my ($vmid) = @_; + return $vmfw_exists->{$vmid}; + }, + ); + my $mocked_remove_vmfw = \&PVE::Firewall::remove_vmfw_conf; + $firewall_module->mock( + remove_vmfw_conf => sub { + my ($vmid) = @_; + delete $vmfw_exists->{$vmid}; + return $mocked_remove_vmfw->(@_); + }, + ); + my $fake_setup_module = Test::MockModule->new('PVE::LXC::Test::Setup', no_auto => 1); + $fake_setup_module->mock( + post_create_hook => sub { die "post-create hook failed\n" }, + post_restore_hook => sub { die "post-restore hook failed\n" }, + ); + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'); + + for my $case (['restore', 0], ['restore', 1], ['create', 1]) { + my ($what, $existed) = $case->@*; + my $name = "failed $what " . ($existed ? 'with' : 'without') . ' older firewall config'; + + reset_configs(); + $vmfw_exists = $existed ? { 200 => 1 } : {}; + my $err = $what eq 'restore' ? restore_ct(200) : create_ct(200); + like($err, qr/post-$what hook failed/, "$name: fails"); + ok($vmfw_removed_reserved->{200}, "$name: firewall config is removed"); + my $warning = + "removed the firewall config of CT 200 that existed before the failed $what"; + if ($existed) { + like($task_log, qr/^WARN: \Q$warning\E$/m, "$name: warning is logged"); + } else { + unlike($task_log, qr/existed before/, "$name: no warning"); + } + } + + reset_configs(); + write_raw_config(200, existing_config('ctime=1500000000')); + $vmfw_exists = { 200 => 1 }; + $backup_read_error = "unable to read backup\n"; + my $err = restore_ct(200, { force => 1 }); + $backup_read_error = undef; + like($err, qr/unable to read backup/, 'failed restore over a container: fails'); + ok($vmfw_exists->{200}, 'failed restore over a container: firewall config is kept'); + unlike($task_log, qr/existed before/, 'failed restore over a container: no warning'); + + $firewall_module->mock(remove_vmfw_conf => $mocked_remove_vmfw); + $api_module->mock(__vmfw_conf_exists => sub { return 0 }); +}; + +subtest 'restore fails if the backup changed after its uid was read' => sub { + my $mocked_restore_archive = \&PVE::LXC::Create::restore_archive; + $create_module->mock( + restore_archive => sub { + my ($storecfg, $archive, $rootdir) = @_; + # the extraction opens the archive again and finds another backup + make_path("$rootdir/etc/vzdump"); + file_set_contents( + "$rootdir/etc/vzdump/pct.conf", + backup_config('ctime=1300000000,rtime=1300000000000,uid=anotheranotheranothe'), + ); + }, + ); + my $changed = "unable to restore CT 200 - backup changed during the restore: its uid" + . " 'anotheranotheranothe' differs from uid 'backupbackupbackupba' read before"; + + reset_configs(); + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'); + my $err = restore_ct(200); + like($err, qr/^\Q$changed\E$/, 'free VMID: restore fails'); + ok(!defined(read_raw_config(200)), 'free VMID: nothing is left behind'); + + $create_module->mock(restore_archive => $mocked_restore_archive); +}; + +# Restores in place over a container without identity, failing after reading the backup. +my sub failed_restore_in_place { + reset_configs(); + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba') =~ + s/^unprivileged: 1\n//mr; + write_raw_config(200, existing_config('ctime=1500000000')); + $rpcenv->{deny}->{'/'} = 1; + my $err = restore_ct(200, { force => 1 }); + delete $rpcenv->{deny}; + like($err, qr/permission check failed/, 'restore fails after reading the backup'); + is(pending_uid_of(200), 'backupbackupbackupba', 'container left behind has the pending uid'); +} + +subtest 'retry elsewhere, then over the container left behind' => sub { + failed_restore_in_place(); + my $leftover = identity_of(200)->{uid}; + + my $err = restore_ct(201); + is($err, '', 'restore to another VMID succeeds') or diag($task_log); + is(identity_of(201)->{uid}, 'backupbackupbackupba', 'pending uid is not in use'); + is(pending_uid_of(200), 'backupbackupbackupba', 'container left behind keeps it'); + + $err = restore_ct(200, { force => 1 }); + is($err, '', 'restore over the container left behind succeeds') or diag($task_log); + like( + $task_log, + qr/: backup uid backupbackupbackupba is in use by guest 201, keeping uid $leftover$/m, + 'uid of the backup is in use by now', + ); + is(identity_of(200)->{uid}, $leftover, 'container left behind keeps its uid'); + is(pending_uid_of(200), undef, 'pending uid is removed'); +}; + +subtest 'using the container left behind drops its pending uid' => sub { + failed_restore_in_place(); + + my $err = update_ct(200, { description => 'used' }); + is($err, '', 'config change of the stopped container') or diag($task_log); + my $dropped = "guest identity: dropping pending uid backupbackupbackupba of an unfinished" + . " restore - a retry now needs 'allow-identity-mismatch'"; + like($task_log, qr/^WARN: \Q$dropped\E$/m, 'dropped'); + is(pending_uid_of(200), undef, 'pending uid is gone'); +}; + +subtest 'restore with unique keeps a pending uid, but never takes it over' => sub { + for my $retry ({ unique => 1 }, {}) { + my $what = $retry->{unique} ? 'retry with unique' : 'retry without unique'; + + reset_configs(); + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba') =~ + s/^unprivileged: 1\n//mr; + write_raw_config(200, existing_config('ctime=1500000000')); + $rpcenv->{deny}->{'/'} = 1; + my $err = restore_ct(200, { force => 1, unique => 1 }); + delete $rpcenv->{deny}; + like( + $err, + qr/permission check failed/, + 'restore with unique fails after reading the backup', + ); + my $leftover = identity_of(200)->{uid}; + is( + pending_uid_of(200), + 'backupbackupbackupba', + 'container left behind has the pending uid', + ); + + $err = restore_ct(200, { force => 1, $retry->%* }); + is($err, '', "$what succeeds") or diag($task_log); + is(pending_uid_of(200), undef, "$what: pending uid is removed"); + is( + identity_of(200)->{uid}, + $retry->{unique} ? $leftover : 'backupbackupbackupba', + "$what: " + . ($retry->{unique} ? 'keeps its uid' : 'takes over the uid of the backup'), + ); + } +}; + +subtest 'failed adoption keeps the pending uid' => sub { + reset_configs(); + $backup_raw_config = + backup_config('ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'); + $domain_lock_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n"; + my $err = restore_ct(200); + $domain_lock_error = undef; + is($err, '', 'restore succeeds') or diag($task_log); + like( + $task_log, + qr/unable to adopt backup uid backupbackupbackupba, keeping uid /, + 'warning', + ); + isnt(identity_of(200)->{uid}, 'backupbackupbackupba', 'container has a uid of its own'); + is(pending_uid_of(200), 'backupbackupbackupba', 'nothing decided, pending uid stays'); + + $err = restore_ct(200, { force => 1 }); + is($err, '', 'restore of the backup again is not refused') or diag($task_log); + is(identity_of(200)->{uid}, 'backupbackupbackupba', 'uid of the backup is taken over'); +}; + +subtest 'config of a container left behind shows its stored identity' => sub { + failed_restore_in_place(); + my $meta = PVE::LXC::Config->load_config(200)->{meta}; + + my $conf = PVE::API2::LXC::Config->vm_config({ node => $NODENAME, vmid => 200 }); + is($conf->{meta}, $meta, 'config with pending changes applied: stored identity'); + $conf = PVE::API2::LXC::Config->vm_config({ node => $NODENAME, vmid => 200, current => 1 }); + is($conf->{meta}, $meta, 'current config: stored identity'); + + my $pending = PVE::API2::LXC->vm_pending({ node => $NODENAME, vmid => 200 }); + my ($item) = grep { $_->{key} eq 'meta' } $pending->@*; + is($item->{value}, $meta, 'pending view: current value'); + is($item->{pending}, 'uid=backupbackupbackupba', 'pending view: pending uid'); +}; + +# Once a write of the config of CT 200 has the pending uid of the backup, all later writes of the +# restore have it, so that the container keeps it whenever the restore stops. Only the adoption +# removes it. +subtest 'every write of a restore keeps the pending uid' => sub { + my $backup_meta = 'ctime=1400000000,rtime=1400000000000,uid=backupbackupbackupba'; + my $cases = [ + ['free VMID', undef, {}], + ['in place over a container without uid', existing_config('ctime=1500000000'), {}], + ['free VMID, from STDIN', undef, { ostemplate => '-', rootfs => 'local:4' }], + ['free VMID, template', undef, {}, "template: 1\n"], + ]; + $storage_module->mock(volume_has_feature => sub { return 0 }); + + for my $case (@$cases) { + my ($what, $target, $param, $extra) = @$case; + + reset_configs(); + $backup_raw_config = backup_config($backup_meta) . ($extra // ''); + write_raw_config(200, $target) if $target; + my $err = restore_ct(200, { $target ? (force => 1) : (), $param->%* }); + is($err, '', "$what: restore succeeds") or diag($task_log); + like( + $task_log, + qr/^guest identity: adopted uid backupbackupbackupba/m, + "$what: adopted", + ); + + my @writes = $written->{200}->@*; + my $last = pop(@writes); + my $first = 0; + $first++ while $first < @writes && $writes[$first] !~ m/^\[pve:pending\]/m; + ok($first < @writes, "$what: pending uid is written") or next; + my @later = @writes[$first .. $#writes]; + is( + scalar(grep { m/^\[pve:pending\]\nmeta: uid=backupbackupbackupba$/m } @later), + scalar(@later), + "$what: all later writes keep it", + ); + unlike($last, qr/^\[pve:pending\]/m, "$what: adoption removes it"); + } + $storage_module->unmock('volume_has_feature'); +}; + remove_tree($WORKDIR); done_testing(); -- 2.47.3