public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH qemu-server v4 14/41] api: restore: release the VMID if a restore to an unused VMID fails
Date: Thu, 19 Nov 2026 11:34:00 +0100	[thread overview]
Message-ID: <20261119103551.1666964-15-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20261119103551.1666964-1-t.lamprecht@proxmox.com>

If a restore to a previously unused VMID failed, its temporary or empty
config stayed, even if the restore had removed the volumes that it
allocated. The VMID stayed taken and a retry needed 'force'. Containers
already remove their config in this case.

Remove the VM config if the VMID was unused before the operation and the
restore fails before the point from which it keeps the restored volumes.
Later failures, for example in the live phase of a live restore, keep
the VM as before. Also remove the reservation if the restore could not
start at all, for example because it did not get the guest lock in time.
If the next-id setting enforces unique guest IDs, the released VMID
counts as used. A retry then needs another VMID, as for containers.

The cleanup can be incomplete: a volume that the restore allocated
cannot be removed, or the previous firewall config cannot be put back.
Keep the VM with its 'create' lock in that case, so that what is left
behind keeps its VMID, and log what is left. For a legacy tar archive, a
record of the allocated volumes that cannot be read counts like a volume
that cannot be removed. Containers keep their locked config in this case
as well. The lock ensures that the leftover does not get started or
restored over before an admin has looked at it and removed the lock.

Remove the config before the guest config lock is released. Otherwise,
another operation could take the VMID in between and lose its config to
this cleanup. Put the firewall config back even before that, for a
similar reason: the guest config lock only holds on this node, so
another node can take the VMID as soon as the VM config is gone.

Only warn that the VM was not cleaned up if the restore changed the
target and left it behind. A failure to read the backup, for example,
should not produce that warning.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---

changes v3 -> v4:
* the firewall part moved to an independent fix before the series, which
  puts the previous firewall config back after a failed restore, instead
  of removing the one of the backup with a warning; here, a config that
  cannot be put back keeps the VM
* a VM that stays at a previously unused VMID because its cleanup was
  incomplete keeps its 'create' lock, like a container, and the task log
  says what is left; v3 unlocked it
* remove the reservation also if the restore could not take its guest
  lock
* tar archives: an allocation record that cannot be read or parsed keeps
  the VMID
* the fake cluster filesystem and the tests of unchanged behavior moved
  to an independent test commit before the series
* commit message and comments: plainer wording, same content

changes v2 -> v3:
* keep the VM if a restored volume or the restored firewall config
  cannot be removed
* only remove a firewall config that the restore wrote, and warn if that
  one had replaced a firewall config that existed for the free VMID
  before
* tests: retry without force, and cases for the independent fixes before
  the series ('force' and a VM created in between, lock of the temporary
  VMA config)

changes v1 -> v2:
* remove the firewall config before the VM config, with a test for the
  order
* note the effect with enforced unique guest IDs in the commit message
* test helper: use v5.36, mock the registration of used guest IDs

 src/PVE/API2/Qemu.pm                      |  45 ++++-
 src/PVE/QemuServer.pm                     |  79 ++++++--
 src/test/run_qemu_restore_config_tests.pl | 234 +++++++++++++++++++++-
 3 files changed, 332 insertions(+), 26 deletions(-)

diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index b798a590..16779440 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -1509,12 +1509,15 @@ __PACKAGE__->register_method({
         # 'force' only covers the VM that the checks above saw, not one created in the meantime, and
         # the permission to overwrite a VM does not cover creating it anew if it is gone by now
         my $allow_existing = $overwrite_only ? 'require' : $force && $vm_exists;
-        eval { PVE::QemuConfig->create_and_lock_config($vmid, $allow_existing) };
+        my $reservation =
+            eval { PVE::QemuConfig->create_and_lock_config($vmid, $allow_existing) };
         die "$emsg $@" if $@;
 
         my $restored_data = 0;
-        # the restore paths record whether the restored volumes stay with the VM
+        # the restore paths record whether they changed the VM, whether the restored volumes stay
+        # with it, and whether they could not remove all volumes after a failure
         my $restore_progress = {};
+        my $restore_started; # whether the restore got its guest lock
         my $restorefn = sub {
             my $conf = PVE::QemuConfig->load_config($vmid);
 
@@ -1590,15 +1593,36 @@ __PACKAGE__->register_method({
 
             my $restore_locked = sub {
                 my $firewall_conf = PVE::GuestHelpers::get_firewall_config_raw($vmid);
+                $restore_started = 1;
                 eval { $realcmd->() };
                 my $err = $@ or return;
 
                 # the VM that keeps its restored volumes also keeps the firewall config of the backup
                 die $err if $restore_progress->{'volumes-kept'};
 
+                my @left =
+                    $restore_progress->{'volumes-left'} ? ('volumes of the restore') : ();
                 # any other failed restore must not leave that firewall config behind
                 eval { PVE::GuestHelpers::set_firewall_config_raw($vmid, $firewall_conf) };
-                warn "unable to reset the firewall config - $@" if $@;
+                if (my $firewall_err = $@) {
+                    warn "unable to reset the firewall config - $firewall_err";
+                    push @left, 'the firewall config';
+                }
+
+                if (!$reservation->{existed}) {
+                    if (@left) {
+                        # what is left keeps the VMID, so that no other guest gets it
+                        log_warn("keeping VM $vmid locked, the cleanup left behind "
+                            . join(', ', @left));
+                    } else {
+                        # Release the VMID while the guest lock is still held, so that no other
+                        # operation can take the VMID in between and then lose its config here.
+                        # The firewall config is settled before that, as another node can take
+                        # the VMID as soon as the config is gone.
+                        eval { PVE::QemuConfig->destroy_config($vmid) };
+                        warn $@ if $@;
+                    }
+                }
                 die $err;
             };
 
@@ -1735,13 +1759,26 @@ __PACKAGE__->register_method({
             $code = sub {
                 eval { $restorefn->() };
                 if (my $err = $@) {
+                    # For a VMID that was free before, the restore already removed the config, or
+                    # kept it locked because its cleanup left something behind. Only unlock a VM
+                    # that existed before or that keeps its restored volumes.
+                    if (!$reservation->{existed} && !$restore_progress->{'volumes-kept'}) {
+                        # the reservation of a restore that did not start is untouched, for example
+                        # if taking its guest lock timed out
+                        if (!$restore_started) {
+                            eval { PVE::QemuConfig->remove_unused_reservation($vmid) };
+                            warn $@ if $@;
+                        }
+                        die $err;
+                    }
+
                     eval { PVE::QemuConfig->remove_lock($vmid, 'create') };
                     warn $@ if $@;
                     if ($restored_data) {
                         warn
                             "error after data was restored, VM disks should be OK but config may "
                             . "require adaptions. VM $vmid state is NOT cleaned up.\n";
-                    } else {
+                    } elsif ($restore_progress->{'target-changed'}) {
                         warn "error before or during data restore, some or all disks were not "
                             . "completely restored. VM $vmid state is NOT cleaned up.\n";
                     }
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 3104fa73..d08dc33e 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -6422,30 +6422,48 @@ sub tar_archive_read_firstfile {
     return $firstfile;
 }
 
+# Returns the number of volumes that could not be removed. qmextract creates the stat file before it
+# allocates a volume, so a stat file or a line of it that cannot be read counts as one.
 sub tar_restore_cleanup {
     my ($storecfg, $statfile) = @_;
 
     print STDERR "starting cleanup\n";
 
-    if (my $fd = IO::File->new($statfile, "r")) {
-        while (defined(my $line = <$fd>)) {
-            if ($line =~ m/vzdump:([^\s:]*):(\S+)$/) {
-                my $volid = $2;
-                eval {
-                    if (classify_drive_file($volid) eq "absolute") {
-                        unlink($volid) || $!{ENOENT} || die "unlink failed - $!\n";
-                    } else {
-                        PVE::Storage::vdisk_free($storecfg, $volid);
-                    }
-                    print STDERR "temporary volume '$volid' successfully removed\n";
-                };
-                print STDERR "unable to cleanup '$volid' - $@" if $@;
-            } else {
-                print STDERR "unable to parse line in statfile - $line";
+    my $fd = IO::File->new($statfile, "r");
+    if (!$fd) {
+        return 0 if $!{ENOENT};
+        print STDERR "unable to read '$statfile' - $!\n";
+        return 1;
+    }
+
+    my $left = 0;
+    while (defined(my $line = <$fd>)) {
+        if ($line =~ m/vzdump:([^\s:]*):(\S+)$/) {
+            my $volid = $2;
+            eval {
+                if (classify_drive_file($volid) eq "absolute") {
+                    unlink($volid) || $!{ENOENT} || die "unlink failed - $!\n";
+                } else {
+                    PVE::Storage::vdisk_free($storecfg, $volid);
+                }
+                print STDERR "temporary volume '$volid' successfully removed\n";
+            };
+            if (my $err = $@) {
+                print STDERR "unable to cleanup '$volid' - $err";
+                $left++;
             }
+        } else {
+            print STDERR "unable to parse line in statfile - $line";
+            $left++;
         }
-        $fd->close();
     }
+    if ($fd->error()) {
+        print STDERR "unable to read '$statfile'\n";
+        $left++;
+    }
+    $fd->close();
+
+    return $left;
 }
 
 sub restore_file_archive {
@@ -6712,17 +6730,24 @@ my $restore_deactivate_volumes = sub {
     print STDERR $@ if $@;
 };
 
+# Returns the number of volumes that could not be removed.
 my $restore_destroy_volumes = sub {
     my ($storecfg, $virtdev_hash) = @_;
 
+    my $left = 0;
     for my $dev (values $virtdev_hash->%*) {
         my $volid = $dev->{volid} or next;
         eval {
             PVE::Storage::vdisk_free($storecfg, $volid);
             print STDERR "temporary volume '$volid' successfully removed\n";
         };
-        print STDERR "unable to cleanup '$volid' - $@" if $@;
+        if (my $err = $@) {
+            print STDERR "unable to cleanup '$volid' - $err";
+            $left++;
+        }
     }
+
+    return $left;
 };
 
 sub restore_merge_config {
@@ -6972,6 +6997,7 @@ sub restore_proxmox_backup_archive {
         $param = [$pbs_backup_name, "qemu-server.conf", $cfgfn];
         PVE::Storage::PBSPlugin::run_raw_client_cmd($scfg, $storeid, $cmd, $param);
 
+        $options->{progress}->{'target-changed'} = 1;
         PVE::ReplicationState::delete_guest_states($vmid);
 
         if ($has_firewall_config) {
@@ -7061,7 +7087,9 @@ sub restore_proxmox_backup_archive {
     rmtree $tmpdir;
 
     if ($err) {
-        $restore_destroy_volumes->($storecfg, $virtdev_hash);
+        # what could not be removed keeps the VMID taken
+        $options->{progress}->{'volumes-left'} = 1
+            if $restore_destroy_volumes->($storecfg, $virtdev_hash);
         die $err;
     }
     $options->{progress}->{'volumes-kept'} = 1;
@@ -7151,6 +7179,7 @@ sub restore_external_archive {
         my $data = $backup_provider->archive_get_guest_config($volname)
             or die "backup provider failed to extract guest configuration\n";
 
+        $options->{progress}->{'target-changed'} = 1;
         PVE::ReplicationState::delete_guest_states($vmid);
 
         PVE::Tools::file_set_contents($cfgfn, $data);
@@ -7235,7 +7264,9 @@ sub restore_external_archive {
     rmtree($tmpdir);
 
     if ($err) {
-        $restore_destroy_volumes->($storecfg, $virtdev_hash);
+        # what could not be removed keeps the VMID taken
+        $options->{progress}->{'volumes-left'} = 1
+            if $restore_destroy_volumes->($storecfg, $virtdev_hash);
         die $err;
     }
     $options->{progress}->{'volumes-kept'} = 1;
@@ -7563,6 +7594,7 @@ sub restore_vma_archive {
     my $print_devmap = sub {
         my $cfgfn = "$tmpdir/qemu-server.conf";
 
+        $opts->{progress}->{'target-changed'} = 1;
         PVE::ReplicationState::delete_guest_states($vmid);
 
         # we can read the config - that is already extracted
@@ -7699,7 +7731,9 @@ sub restore_vma_archive {
     rmtree $tmpdir;
 
     if ($err) {
-        $restore_destroy_volumes->($cfg, $virtdev_hash);
+        # what could not be removed keeps the VMID taken
+        $opts->{progress}->{'volumes-left'} = 1
+            if $restore_destroy_volumes->($cfg, $virtdev_hash);
         die $err;
     }
     $opts->{progress}->{'volumes-kept'} = 1;
@@ -7730,6 +7764,7 @@ sub restore_tar_archive {
 
     my $storecfg = PVE::Storage::config();
 
+    $opts->{progress}->{'target-changed'} = 1;
     PVE::ReplicationState::delete_guest_states($vmid);
 
     # avoid zombie disks when restoring over an existing VM -> cleanup first
@@ -7809,7 +7844,9 @@ sub restore_tar_archive {
         $srcfd->close();
     };
     if (my $err = $@) {
-        tar_restore_cleanup($storecfg, "$tmpdir/qmrestore.stat") if !$opts->{info};
+        # what could not be removed keeps the VMID taken
+        $opts->{progress}->{'volumes-left'} = 1
+            if !$opts->{info} && tar_restore_cleanup($storecfg, "$tmpdir/qmrestore.stat");
         die $err;
     }
     $opts->{progress}->{'volumes-kept'} = 1;
diff --git a/src/test/run_qemu_restore_config_tests.pl b/src/test/run_qemu_restore_config_tests.pl
index d91b46b5..f5f9e026 100755
--- a/src/test/run_qemu_restore_config_tests.pl
+++ b/src/test/run_qemu_restore_config_tests.pl
@@ -5,7 +5,7 @@ use warnings;
 
 use lib qw(.. .);
 
-use Fcntl qw(O_RDONLY O_NONBLOCK);
+use Fcntl qw(:flock O_RDONLY O_NONBLOCK);
 use File::Path qw(make_path remove_tree);
 use File::Temp qw(tempdir);
 use JSON;
@@ -98,6 +98,9 @@ my $BACKUP_FIREWALL = "[OPTIONS]\nenable: 1\n";
 our $current_backup; # raw config of the backup
 # where the restore fails: 'read-config', 'command', 'data', 'after-restore' or 'live'
 our $fail_at = '';
+# the stat file that qmextract leaves: undef for none, 'directory' for one that cannot be read, or
+# its content
+our $tar_stat_file;
 my $calls = {}; # calls with effects on the target VM that tests check
 
 sub backup_config {
@@ -216,6 +219,16 @@ $qemu_server_module->mock(
         if (!ref($cmd) && $cmd =~ m/^zcat -f\|tar xf /) {
             die "unable to read the backup\n" if $fail_at eq 'read-config';
             file_set_contents("$ENV{VZDUMP_TMPDIR}/qemu-server.conf", $current_backup);
+            # a failed restore leaves its temporary directory behind
+            my $statfile = "$ENV{VZDUMP_TMPDIR}/qmrestore.stat";
+            remove_tree($statfile);
+            if (defined($tar_stat_file)) {
+                if ($tar_stat_file eq 'directory') {
+                    make_path($statfile);
+                } else {
+                    file_set_contents($statfile, $tar_stat_file);
+                }
+            }
             return 0;
         }
 
@@ -301,6 +314,7 @@ sub restore_api {
 
 my $TARGET_CONF = "memory: 1024\nmeta: creation-qemu=10.0.0,ctime=1700000000\nname: target\n";
 my $NOT_CLEANED_UP = qr/VM 100 state is NOT cleaned up/;
+my $KEPT_LOCKED = 'WARN: keeping VM 100 locked, the cleanup left behind';
 
 for my $type (sort keys $ARCHIVES->%*) {
     subtest "$type: restore to a free VMID" => sub {
@@ -319,6 +333,34 @@ for my $type (sort keys $ARCHIVES->%*) {
     };
 }
 
+for my $type (sort keys $ARCHIVES->%*) {
+    subtest "$type: failed restore to a free VMID" => sub {
+        reset_cluster();
+
+        local $fail_at = 'data';
+        my ($err, $output) = restore_api($type, backup_config(), {});
+        is($err, "restoring the data failed\n", 'restore failed');
+        ok(!defined(stored_config(100)), 'reserved config removed');
+        is(PVE::GuestHelpers::get_firewall_config_raw(100), undef, 'no firewall config left');
+        unlike($output, $NOT_CLEANED_UP, 'no warning about a VM left behind');
+
+        $fail_at = '';
+        ($err, $output) = restore_api($type, backup_config(), {});
+        is($err, '', 'retry needs no force') or diag($output);
+        like(stored_config(100), qr/^name: backup$/m, 'VM restored');
+    };
+}
+
+subtest 'failed restore to a free VMID before reading the backup' => sub {
+    reset_cluster();
+
+    local $fail_at = 'read-config';
+    my ($err, $output) = restore_api('pbs', backup_config(), {});
+    is($err, "unable to read the backup\n", 'restore failed');
+    ok(!defined(stored_config(100)), 'reserved config removed');
+    unlike($output, $NOT_CLEANED_UP, 'no warning about a VM left behind');
+};
+
 subtest 'vma: an error of the restore setup is reported without the command line' => sub {
     reset_cluster();
     local $fail_at = 'data';
@@ -380,6 +422,128 @@ subtest 'failure once the data is restored keeps the firewall config of the back
     }
 };
 
+subtest 'firewall config that cannot be put back keeps the VMID' => sub {
+    reset_cluster();
+
+    # only the removal of the firewall config of the backup fails
+    my $set_firewall_config_raw = \&PVE::GuestHelpers::set_firewall_config_raw;
+    local *PVE::GuestHelpers::set_firewall_config_raw = sub {
+        my ($vmid, $raw) = @_;
+        die "unable to remove the file - Permission denied\n" if !defined($raw);
+        return $set_firewall_config_raw->($vmid, $raw);
+    };
+
+    local $fail_at = 'data';
+    my ($err, $output) = restore_api('pbs', backup_config(), {});
+    is($err, "restoring the data failed\n", 'restore failed');
+    my $reported = 'unable to reset the firewall config - unable to remove the file';
+    like($output, qr/^\Q$reported\E - Permission denied$/m, 'failure is reported');
+    like($output, qr/^\Q$KEPT_LOCKED\E the firewall config$/m, 'what is left is reported');
+    like(stored_config(100), qr/^lock: create$/m, 'reservation is kept, locked');
+    unlike($output, $NOT_CLEANED_UP, 'no other warning about the VM left behind');
+};
+
+subtest 'firewall config is put back before the VMID is released' => sub {
+    reset_cluster();
+
+    my $reserved;
+    my $set_firewall_config_raw = \&PVE::GuestHelpers::set_firewall_config_raw;
+    local *PVE::GuestHelpers::set_firewall_config_raw = sub {
+        $reserved = defined(stored_config(100));
+        return $set_firewall_config_raw->(@_);
+    };
+
+    local $fail_at = 'data';
+    restore_api('pbs', backup_config(), {});
+    ok($reserved, 'VMID was still reserved');
+    ok(!defined(stored_config(100)), 'and is released afterwards');
+};
+
+subtest 'tar: an allocation record that cannot be read keeps the VMID' => sub {
+    for my $case (
+        ['line that cannot be parsed', "garbage\n", qr/^unable to parse line in statfile/m],
+        [
+            'stat file that cannot be read',
+            'directory',
+            qr/^unable to read '.*qmrestore\.stat'/m,
+        ],
+    ) {
+        my ($what, $stat_file, $reported) = $case->@*;
+        reset_cluster();
+
+        local $fail_at = 'data';
+        local $tar_stat_file = $stat_file;
+        my ($err, $output) = restore_api('tar', backup_config(), {});
+        is($err, "restoring the data failed\n", "$what: restore failed");
+        like($output, $reported, "$what: reported");
+        like(stored_config(100), qr/^lock: create$/m, "$what: reservation is kept, locked");
+        like(
+            $output,
+            qr/^\Q$KEPT_LOCKED\E volumes of the restore$/m,
+            "$what: what is left is reported",
+        );
+    }
+};
+
+subtest 'failed removal of a restored volume keeps the VMID' => sub {
+    reset_cluster();
+
+    my $allocated = [];
+    my $run_raw_client_cmd = \&PVE::Storage::PBSPlugin::run_raw_client_cmd;
+    local *PVE::Storage::PBSPlugin::run_raw_client_cmd = sub {
+        my ($scfg, $storeid, $cmd, $param) = @_;
+        my (undef, $file, $target) = $param->@*;
+        return $run_raw_client_cmd->(@_) if $file ne 'index.json';
+        my $files = [
+            { filename => 'qemu-server.conf.blob' },
+            { filename => 'drive-scsi0.img.fidx', size => 1048576 },
+        ];
+        file_set_contents($target, encode_json({ files => $files }));
+    };
+    local *PVE::Storage::storage_config = sub {
+        my ($cfg, $storeid) = @_;
+        return { type => 'pbs', server => 'localhost', datastore => 'ds' } if $storeid eq 'pbs';
+        return { type => 'dir', path => '/nonexistent', content => { images => 1 } };
+    };
+    local *PVE::Storage::resolve_format_hint = sub { return 'raw'; };
+    local *PVE::Storage::vdisk_alloc = sub {
+        my ($cfg, $storeid, $vmid) = @_;
+        push $allocated->@*, "$storeid:vm-$vmid-disk-0";
+        return "$storeid:vm-$vmid-disk-0";
+    };
+    local *PVE::Storage::activate_volumes = sub { return; };
+    local *PVE::Storage::path = sub { return '/nonexistent/vm-100-disk-0.raw'; };
+    local *PVE::Storage::volume_has_feature = sub { return 0; };
+    local *PVE::Storage::vdisk_free = sub { die "volume is busy\n"; };
+    my $run_command = \&PVE::QemuServer::run_command;
+    local *PVE::QemuServer::run_command = sub {
+        my ($cmd) = @_;
+        return 0 if ref($cmd) && $cmd->[0] eq '/usr/bin/pbs-restore';
+        return $run_command->(@_);
+    };
+
+    local $fail_at = 'data';
+    my $backup = backup_config()
+        . "scsi0: target:vm-100-disk-0,size=1M\n#qmdump#map:scsi0:drive-scsi0:target:raw:\n";
+    my ($err, $output) = restore_api('pbs', $backup, {});
+    is($err, "restoring the data failed\n", 'restore failed after allocating the volume');
+    is_deeply($allocated, ['target:vm-100-disk-0'], 'volume was allocated');
+    like($output, qr/^unable to cleanup 'target:vm-100-disk-0' - volume is busy$/m, 'reported');
+    like(stored_config(100), qr/^lock: create$/m, 'reservation is kept, locked');
+    like(
+        $output, qr/^\Q$KEPT_LOCKED\E volumes of the restore$/m, 'what is left is reported',
+    );
+
+    # the leftover is no VM to restore over
+    $fail_at = '';
+    ($err) = restore_api('pbs', backup_config(), { force => 1 });
+    is(
+        $err,
+        "unable to restore VM 100 - VM is locked (create)\n",
+        'retry needs an unlock first',
+    );
+};
+
 subtest "'force' does not overwrite a VM created after the checks" => sub {
     reset_cluster();
 
@@ -392,6 +556,38 @@ subtest "'force' does not overwrite a VM created after the checks" => sub {
     is(stored_config(100), $TARGET_CONF, 'VM created in between unchanged');
 };
 
+subtest 'a restore that cannot take its guest lock removes the reservation' => sub {
+    my $lock_config_full = PVE::QemuConfig->can('lock_config_full');
+    my $before_timeout = sub { };
+    local *PVE::QemuConfig::lock_config_full = sub {
+        my ($class, $vmid, $timeout, @param) = @_;
+        # the restore waits one second for the lock
+        if ($timeout == 1) {
+            $before_timeout->();
+            die "can't lock file 'lock-$vmid.conf' - got timeout\n";
+        }
+        return $lock_config_full->($class, $vmid, $timeout, @param);
+    };
+
+    reset_cluster();
+    my ($err, $output) = restore_api('pbs', backup_config(), {});
+    like($err, qr/got timeout/, 'free VMID: restore failed');
+    ok(!defined(stored_config(100)), 'free VMID: reservation removed');
+
+    reset_cluster($TARGET_CONF);
+    ($err, $output) = restore_api('pbs', backup_config(), { force => 1 });
+    like($err, qr/got timeout/, 'existing VM: restore failed');
+    like(stored_config(100), qr/^name: target$/m, 'existing VM: kept');
+    unlike(stored_config(100), qr/^lock:/m, 'existing VM: lock removed');
+
+    # without the lock of the reservation, the config is not one of this restore anymore
+    reset_cluster();
+    $before_timeout = sub { QemuServerTest::FakeCFS::set_config(100, "memory: 512\n"); };
+    ($err, $output) = restore_api('pbs', backup_config(), {});
+    like($err, qr/got timeout/, 'config changed meanwhile: restore failed');
+    is(stored_config(100), "memory: 512\n", 'config changed meanwhile: kept');
+};
+
 subtest 'the permission to overwrite a VM does not cover creating it' => sub {
     # the VM goes away after the permission checks
     local *PVE::Storage::check_volume_access =
@@ -418,6 +614,41 @@ subtest 'the permission to overwrite a VM does not cover creating it' => sub {
     is_deeply($QemuServerTest::FakeCFS::written, {}, 'with VM.Backup only: nothing written');
 };
 
+subtest 'another restore right after a failed restore to a free VMID' => sub {
+    reset_cluster();
+
+    # run a second restore with 'force' as soon as the failed one releases the guest lock
+    my $lock_config_full = PVE::QemuConfig->can('lock_config_full');
+    my ($interposed, $lock_free, $second_err) = (0, 0, '');
+    local *PVE::QemuConfig::lock_config_full = sub {
+        my ($class, $vmid, @param) = @_;
+        my $res = eval { $lock_config_full->($class, $vmid, @param) };
+        my $err = $@;
+        if ($err && !$interposed++) {
+            open(my $fh, '>>', PVE::QemuConfig->config_file_lock($vmid)) or die "$!\n";
+            $lock_free = flock($fh, LOCK_EX | LOCK_NB);
+            close($fh);
+            local $fail_at = '';
+            ($second_err) = restore_api('pbs', backup_config(), { force => 1 });
+        }
+        die $err if $err;
+        return $res;
+    };
+
+    local $fail_at = 'data';
+    my ($err) = restore_api('vma file', backup_config(), {});
+
+    is($err, "restoring the data failed\n", 'first restore failed');
+    ok($lock_free, 'guest lock was free for the second restore');
+    is($second_err, '', 'second restore succeeded');
+    like(stored_config(100), qr/^name: backup$/m, 'its config stays');
+    is(
+        PVE::GuestHelpers::get_firewall_config_raw(100),
+        $BACKUP_FIREWALL,
+        'and its firewall config',
+    );
+};
+
 for my $type ('pbs', 'provider', 'vma file', 'vma stdin') {
     subtest "$type: failed restore over an existing VM" => sub {
         reset_cluster($TARGET_CONF);
@@ -427,6 +658,7 @@ for my $type ('pbs', 'provider', 'vma file', 'vma stdin') {
         like($err, qr/unable to read the backup\n$/, 'fails before changing the VM');
         is(stored_config(100), $TARGET_CONF, 'config unchanged');
         is_deeply($calls, {}, 'replication state and firewall config untouched');
+        unlike($output, $NOT_CLEANED_UP, 'no warning about a changed VM');
 
         $fail_at = 'data';
         ($err, $output) = restore_api($type, backup_config(), { force => 1 });
-- 
2.47.3





  parent reply	other threads:[~2026-11-19 10:39 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 10:33 [PATCH cluster/guest-common/qemu-server/container/manager/docs v4 00/41] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 01/41] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 02/41] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 03/41] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 04/41] pmxcfs: quorum: record the corosync membership Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 05/41] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 06/41] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 07/41] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 08/41] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 09/41] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 10/41] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 11/41] guest identity: allow adopting a uid that is unused in the cluster Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 12/41] migrate: add shared helpers for the identity command of the migration tunnel Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 13/41] vzdump: notes template: list the uid variable Thomas Lamprecht
2026-11-19 10:34 ` Thomas Lamprecht [this message]
2026-11-19 10:34 ` [PATCH qemu-server v4 15/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 16/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 17/41] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 18/41] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 19/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 20/41] restore: decide by the reservation whether the container is new Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 21/41] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 22/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 23/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 24/41] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 25/41] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 26/41] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 27/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 28/41] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 29/41] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-19 12:32   ` Lukas Wagner
2026-11-19 12:51     ` Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 30/41] ui: datacenter summary: show the " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 31/41] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 32/41] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 33/41] ui: guest summary: show " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 34/41] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 35/41] vzdump: notes template: add the guest uid as variable Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 36/41] vzdump: restore: describe what a failed restore can leave behind Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 37/41] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 38/41] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 39/41] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 40/41] pct: " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 41/41] vzdump: notes: document the uid template variable Thomas Lamprecht
2026-11-19 14:46 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v4 00/41] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
2026-11-19 19:49 ` superseded: " Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119103551.1666964-15-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal