all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH qemu-server 12/34] api: restore: release the VMID if a restore to an unused VMID fails
Date: Mon, 16 Nov 2026 01:46:24 +0100	[thread overview]
Message-ID: <20261116004703.2908294-13-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20261116004703.2908294-1-t.lamprecht@proxmox.com>

A failed restore to a previously unused VMID left its temporary or
empty config behind, even when the restore cleaned up the volumes it
had allocated. The VMID stayed taken and a retry needed 'force'.
Containers already remove their config in this case.

Remove the VM config and the restored firewall config if the VMID was
unused before the operation and the restore has not reached the point
from which it keeps the restored volumes. Keep the VM after later
failures, such as a failure in the live phase of a live restore.

Remove the config before releasing the guest config lock. The
temporary config of a VMA restore has no lock property, so removing it
later could delete the config of another restore that took the VMID in
the meantime.

Only warn that the VM was not cleaned up if the restore changed the
target and left it behind. A failure to read the backup should not
produce that warning.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---
 src/PVE/API2/Qemu.pm                      |  28 +-
 src/PVE/QemuServer.pm                     |   8 +
 src/test/QemuServerTest/FakeCFS.pm        | 194 ++++++++++++
 src/test/run_qemu_restore_config_tests.pl | 358 ++++++++++++++++++++--
 4 files changed, 567 insertions(+), 21 deletions(-)
 create mode 100644 src/test/QemuServerTest/FakeCFS.pm

diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index b9634d20..504c9768 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -1413,10 +1413,12 @@ __PACKAGE__->register_method({
 
         my $emsg = $is_restore ? "unable to restore VM $vmid -" : "unable to create VM $vmid -";
 
-        eval { PVE::QemuConfig->create_and_lock_config($vmid, $force) };
+        my $reservation = eval { PVE::QemuConfig->create_and_lock_config($vmid, $force) };
         die "$emsg $@" if $@;
 
         my $restored_data = 0;
+        # the restore paths record whether they changed the VM and whether restored volumes stay
+        my $restore_progress = {};
         my $restorefn = sub {
             my $conf = PVE::QemuConfig->load_config($vmid);
 
@@ -1432,6 +1434,7 @@ __PACKAGE__->register_method({
                     bwlimit => $bwlimit,
                     live => $live_restore,
                     override_conf => $param,
+                    progress => $restore_progress,
                 };
                 if (my $volid = $archive->{volid}) {
                     # best effort, real check is after restoring!
@@ -1489,7 +1492,24 @@ __PACKAGE__->register_method({
                     if $unique;
             };
 
-            PVE::QemuConfig->lock_config_full($vmid, 1, $realcmd);
+            my $restore_locked = sub {
+                eval { $realcmd->() };
+                my $err = $@ or return;
+
+                if (!$reservation->{existed} && !$restore_progress->{'volumes-kept'}) {
+                    # Remove the reservation before releasing the guest lock, so another
+                    # operation cannot take ownership first.
+                    eval {
+                        PVE::QemuConfig->destroy_config($vmid);
+                        $restore_progress->{'reservation-removed'} = 1;
+                        PVE::Firewall::remove_vmfw_conf($vmid);
+                    };
+                    warn $@ if $@;
+                }
+                die $err;
+            };
+
+            PVE::QemuConfig->lock_config_full($vmid, 1, $restore_locked);
 
             if ($start_after_create && !$live_restore) {
                 print "Execute autostart\n";
@@ -1622,13 +1642,15 @@ __PACKAGE__->register_method({
             $code = sub {
                 eval { $restorefn->() };
                 if (my $err = $@) {
+                    die $err if $restore_progress->{'reservation-removed'};
+
                     eval { PVE::QemuConfig->remove_lock($vmid, 'create') };
                     warn $@ if $@;
                     if ($restored_data) {
                         warn
                             "error after data was restored, VM disks should be OK but config may "
                             . "require adaptions. VM $vmid state is NOT cleaned up.\n";
-                    } else {
+                    } elsif ($restore_progress->{'target-changed'}) {
                         warn "error before or during data restore, some or all disks were not "
                             . "completely restored. VM $vmid state is NOT cleaned up.\n";
                     }
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index dcd439f3..9ffc11e8 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -6948,6 +6948,7 @@ sub restore_proxmox_backup_archive {
         $param = [$pbs_backup_name, "qemu-server.conf", $cfgfn];
         PVE::Storage::PBSPlugin::run_raw_client_cmd($scfg, $storeid, $cmd, $param);
 
+        $options->{progress}->{'target-changed'} = 1;
         PVE::ReplicationState::delete_guest_states($vmid);
 
         if ($has_firewall_config) {
@@ -7042,6 +7043,7 @@ sub restore_proxmox_backup_archive {
         $restore_destroy_volumes->($storecfg, $virtdev_hash);
         die $err;
     }
+    $options->{progress}->{'volumes-kept'} = 1;
 
     if ($options->{live}) {
         # keep lock during live-restore
@@ -7129,6 +7131,7 @@ sub restore_external_archive {
         my $data = $backup_provider->archive_get_guest_config($volname)
             or die "backup provider failed to extract guest configuration\n";
 
+        $options->{progress}->{'target-changed'} = 1;
         PVE::ReplicationState::delete_guest_states($vmid);
 
         PVE::Tools::file_set_contents($cfgfn, $data);
@@ -7218,6 +7221,7 @@ sub restore_external_archive {
         $restore_destroy_volumes->($storecfg, $virtdev_hash);
         die $err;
     }
+    $options->{progress}->{'volumes-kept'} = 1;
 
     my $new_conf = restore_merge_config($conffile, $new_conf_raw, $options->{override_conf});
     check_restore_permissions($rpcenv, $user, $new_conf);
@@ -7543,6 +7547,7 @@ sub restore_vma_archive {
     my $print_devmap = sub {
         my $cfgfn = "$tmpdir/qemu-server.conf";
 
+        $opts->{progress}->{'target-changed'} = 1;
         PVE::ReplicationState::delete_guest_states($vmid);
 
         # we can read the config - that is already extracted
@@ -7676,6 +7681,7 @@ sub restore_vma_archive {
         $restore_destroy_volumes->($cfg, $virtdev_hash);
         die $err;
     }
+    $opts->{progress}->{'volumes-kept'} = 1;
 
     my $new_conf = restore_merge_config($conffile, $new_conf_raw, $opts->{override_conf});
     check_restore_permissions($rpcenv, $user, $new_conf);
@@ -7703,6 +7709,7 @@ sub restore_tar_archive {
 
     my $storecfg = PVE::Storage::config();
 
+    $opts->{progress}->{'target-changed'} = 1;
     PVE::ReplicationState::delete_guest_states($vmid);
 
     # avoid zombie disks when restoring over an existing VM -> cleanup first
@@ -7785,6 +7792,7 @@ sub restore_tar_archive {
         tar_restore_cleanup($storecfg, "$tmpdir/qmrestore.stat") if !$opts->{info};
         die $err;
     }
+    $opts->{progress}->{'volumes-kept'} = 1;
 
     rmtree $tmpdir;
 
diff --git a/src/test/QemuServerTest/FakeCFS.pm b/src/test/QemuServerTest/FakeCFS.pm
new file mode 100644
index 00000000..dd9dddec
--- /dev/null
+++ b/src/test/QemuServerTest/FakeCFS.pm
@@ -0,0 +1,194 @@
+package QemuServerTest::FakeCFS;
+
+# Fake cluster filesystem for VM config files, so that tests can use the real config class methods
+# of PVE::QemuConfig, including the guest identity handling of write_config, without pmxcfs.
+#
+# Configs are kept as raw text, indexed by VMID, and an empty file at the path that config_file()
+# returns exists for each, for checks of the file's existence. Generated guest identities are
+# deterministic: the n-th generated uid is 'a' repeated 13 times followed by the base32 encoding of
+# n, and rtime is $now_ms.
+
+use strict;
+use warnings;
+
+use File::Basename qw(dirname);
+use File::Path qw(make_path remove_tree);
+use File::Temp qw(tempdir);
+use Test::MockModule;
+
+use PVE::GuestHelpers;
+use PVE::GuestIdentity;
+use PVE::QemuConfig;
+use PVE::QemuServer;
+
+our $configs = {}; # VMID => raw config, for VMs on the local node
+our $other_guests = {}; # VMID => raw 'meta' property of other guests, like on other nodes or CTs
+our $now_ms = 1700000000000;
+our $generated = 0;
+our $lock_domain_error;
+our $writes = [];
+
+my $tmpdir = tempdir('fake-cfs-XXXXXX', DIR => '.', CLEANUP => 1);
+make_path("$tmpdir/lock");
+$PVE::GuestHelpers::lockdir = "$tmpdir/lock";
+
+sub reset {
+    remove_tree("$tmpdir/etc-pve");
+    $configs = {};
+    $other_guests = {};
+    $now_ms = 1700000000000;
+    $generated = 0;
+    $lock_domain_error = undef;
+    $writes = [];
+}
+
+my sub vmid_of_path {
+    my ($path) = @_;
+
+    return $path =~ m|/qemu-server/(\d+)\.conf$| ? $1 : undef;
+}
+
+# Creates or removes the file at the path of the config of $vmid, matching the stored config.
+my sub sync_config_file {
+    my ($vmid) = @_;
+
+    my $path = PVE::QemuConfig->config_file($vmid);
+    if (defined($configs->{$vmid})) {
+        make_path(dirname($path));
+        open(my $fh, '>', $path) or die "unable to create '$path' - $!\n";
+        close($fh);
+    } else {
+        unlink($path);
+    }
+}
+
+# Stores the raw config $raw for $vmid, like a manual edit of the config file.
+sub set_config {
+    my ($vmid, $raw) = @_;
+
+    $configs->{$vmid} = $raw;
+    sync_config_file($vmid);
+}
+
+# Returns the last 'meta' property of the top-level section of the raw config $raw, like the config
+# class reads the published identity.
+my sub top_level_meta {
+    my ($raw) = @_;
+
+    my $meta;
+    for my $line (split(/\n/, $raw // '')) {
+        last if $line =~ m/^\[/;
+        $meta = $1 if $line =~ m/^meta:\s*(.*?)\s*$/;
+    }
+
+    return $meta;
+}
+
+# Returns the identity published in the raw config of $vmid, undef if there is none.
+sub published_identity {
+    my ($vmid) = @_;
+
+    return PVE::GuestIdentity::parse_identity(top_level_meta($configs->{$vmid}));
+}
+
+# Returns the uid that the $n-th generation of an identity yields.
+sub generated_uid {
+    my ($n) = @_;
+
+    my $alphabet = 'abcdefghijklmnopqrstuvwxyz234567';
+    return ('a' x 17) . join('', map { substr($alphabet, ($n >> (5 * $_)) & 31, 1) } (2, 1, 0));
+}
+
+our $guest_identity_module = Test::MockModule->new('PVE::GuestIdentity');
+$guest_identity_module->mock(
+    __random_bytes => sub {
+        my ($len) = @_;
+        die "unexpected length $len\n" if $len != 13;
+        $generated++;
+        # 85 zero bits encode as 17 'a', then the 15 bits of the counter, the last 4 bits are unused
+        my $counter = substr(unpack('B32', pack('N', $generated)), -15);
+        return pack('B104', ('0' x 85) . $counter . '0000');
+    },
+    __time_ms => sub {
+        return $now_ms;
+    },
+);
+
+our $cluster_module = Test::MockModule->new('PVE::Cluster');
+$cluster_module->mock(
+    get_config => sub {
+        my ($path) = @_;
+        my $vmid = vmid_of_path($path) // die "get_config (mocked) - unexpected path '$path'\n";
+        return $configs->{$vmid};
+    },
+    cfs_read_file => sub {
+        my ($path) = @_;
+        return {} if $path eq 'datacenter.cfg';
+        my $vmid = vmid_of_path($path)
+            // die "cfs_read_file (mocked) - unexpected path '$path'\n";
+        return PVE::QemuServer::parse_vm_config($path, $configs->{$vmid});
+    },
+    cfs_write_file => sub {
+        my ($path, $conf) = @_;
+        my $vmid = vmid_of_path($path)
+            // die "cfs_write_file (mocked) - unexpected path '$path'\n";
+        $configs->{$vmid} = PVE::QemuServer::write_vm_config($path, $conf);
+        sync_config_file($vmid);
+        push $writes->@*, $vmid;
+        return;
+    },
+    cfs_update => sub {
+        return;
+    },
+    check_vmid_unused => sub {
+        my ($vmid, $noerr) = @_;
+        return 1 if !defined($configs->{$vmid}) && !defined($other_guests->{$vmid});
+        return undef if $noerr;
+        die "VM $vmid already exists\n";
+    },
+    cfs_lock_domain => sub {
+        my ($domain, $timeout, $code, @param) = @_;
+        die "cfs_lock_domain (mocked) - unexpected domain '$domain'\n"
+            if $domain ne 'guest-identity';
+        if ($lock_domain_error) {
+            $@ = $lock_domain_error;
+            return undef;
+        }
+        my $res = eval { $code->(@param) };
+        return $res;
+    },
+    get_guest_config_properties => sub {
+        my ($properties) = @_;
+        die "get_guest_config_properties (mocked) - unexpected properties\n"
+            if join(',', $properties->@*) ne 'meta';
+
+        my $res = {};
+        for my $vmid (keys $configs->%*) {
+            my $meta = top_level_meta($configs->{$vmid});
+            $res->{$vmid} = { meta => $meta } if defined($meta);
+        }
+        $res->{$_} = { meta => $other_guests->{$_} } for keys $other_guests->%*;
+        return $res;
+    },
+);
+
+our $qemu_config_module = Test::MockModule->new('PVE::QemuConfig');
+$qemu_config_module->mock(
+    config_file => sub {
+        my ($class, $vmid, $node) = @_;
+        return "$tmpdir/etc-pve/" . $class->cfs_config_path($vmid, $node);
+    },
+    config_file_lock => sub {
+        my ($class, $vmid) = @_;
+        return "$tmpdir/lock/lock-$vmid.conf";
+    },
+    destroy_config => sub {
+        my ($class, $vmid) = @_;
+        die "failed to remove config file: No such file or directory\n"
+            if !defined($configs->{$vmid});
+        delete $configs->{$vmid};
+        sync_config_file($vmid);
+    },
+);
+
+1;
diff --git a/src/test/run_qemu_restore_config_tests.pl b/src/test/run_qemu_restore_config_tests.pl
index 20cb3df0..e6739d66 100755
--- a/src/test/run_qemu_restore_config_tests.pl
+++ b/src/test/run_qemu_restore_config_tests.pl
@@ -3,35 +3,30 @@
 use strict;
 use warnings;
 
-use lib qw(..);
+use lib qw(.. .);
 
+use Fcntl qw(:flock O_RDONLY O_NONBLOCK);
+use File::Path qw(make_path);
+use File::Temp qw(tempdir);
+use JSON;
 use Test::MockModule;
 use Test::More;
 
 use File::Basename;
 
+use PVE::API2::Qemu;
+use PVE::QemuConfig;
 use PVE::QemuServer;
-use PVE::Tools qw(dir_glob_foreach file_get_contents);
+use PVE::QemuServer::Helpers;
+use PVE::RPCEnvironment;
+use PVE::Tools qw(dir_glob_foreach file_get_contents file_set_contents);
+
+# also provides the 'datacenter.cfg' that restore_update_config_line reads
+use QemuServerTest::FakeCFS;
 
 my $INPUT_DIR = './restore-config-input';
 my $EXPECTED_DIR = './restore-config-expected';
 
-# NOTE update when you add/remove tests
-plan tests => 5;
-
-my $pve_cluster_module = Test::MockModule->new("PVE::Cluster");
-$pve_cluster_module->mock(
-    cfs_read_file => sub {
-        my ($file) = @_;
-
-        if ($file eq 'datacenter.cfg') {
-            return {};
-        } else {
-            die "'cfs_read_file' called - missing mock?\n";
-        }
-    },
-);
-
 dir_glob_foreach(
     './restore-config-input',
     '[0-9]+.conf',
@@ -78,4 +73,331 @@ dir_glob_foreach(
     },
 );
 
+# Restores through the create API and the restore paths for PBS, backup providers, VMA and legacy
+# tar archives, against the fake cluster filesystem. Storage, archive tools and other external
+# effects are mocked. The backups have no disks, so that no volumes get allocated.
+
+my $runtime_dir = tempdir('restore-run-XXXXXX', DIR => '.', CLEANUP => 1);
+$PVE::QemuServer::Helpers::var_run_tmpdir = $runtime_dir;
+
+my $rpcenv = PVE::RPCEnvironment->init('cli');
+$rpcenv->set_user('root@pam');
+
+my $ARCHIVES = {
+    pbs => 'pbs:backup/vm/100/2025-01-01T00:00:00Z',
+    provider => 'provider:backup/vm-100-2025-01-01',
+    'vma file' => '/var/lib/vz/dump/vzdump-qemu-100-2025_01_01-00_00_00.vma',
+    'vma stdin' => '-',
+    tar => '/var/lib/vz/dump/vzdump-qemu-100-2025_01_01-00_00_00.tar',
+};
+
+our $current_backup; # raw config of the backup
+our $fail_at = ''; # where the restore fails: 'read-config', 'data', 'after-restore' or 'live'
+my $calls = {}; # calls with effects on the target VM that tests check
+
+sub backup_config {
+    my ($meta) = @_;
+
+    return
+        "boot: order=scsi0\ncores: 1\nmemory: 512\n"
+        . (defined($meta) ? "meta: $meta\n" : '')
+        . "name: backup\n";
+}
+
+package TestBackupProvider {
+
+    sub new {
+        my ($class) = @_;
+        return bless {}, $class;
+    }
+
+    sub restore_get_mechanism { return ('qemu-img', 'qemu'); }
+    sub restore_vm_init { return {}; }
+
+    sub archive_get_guest_config {
+        die "unable to read the backup\n" if $main::fail_at eq 'read-config';
+        return $main::current_backup;
+    }
+    sub archive_get_firewall_config { return "[OPTIONS]\nenable: 1\n"; }
+    sub restore_vm_cleanup { return; }
+}
+
+my $replication_state_module = Test::MockModule->new('PVE::ReplicationState');
+$replication_state_module->mock(delete_guest_states => sub { $calls->{delete_guest_states}++; });
+
+my $firewall_module = Test::MockModule->new('PVE::Firewall');
+$firewall_module->mock(remove_vmfw_conf => sub { $calls->{remove_vmfw_conf}++; });
+
+my $tools_module = Test::MockModule->new('PVE::Tools');
+$tools_module->mock(
+    file_copy => sub {
+        my ($src, $dst) = @_;
+        die "file_copy (mocked) - unexpected destination '$dst'\n" if $dst !~ m|/firewall/|;
+        $calls->{firewall}++;
+    },
+);
+
+my $storage_module = Test::MockModule->new('PVE::Storage');
+$storage_module->mock(
+    abs_filesystem_path => sub {
+        my ($cfg, $path) = @_;
+        return $path;
+    },
+    check_volume_access => sub { return; },
+    config => sub { return {}; },
+    deactivate_volumes => sub { return 1; },
+    extract_vzdump_config => sub { return $current_backup; },
+    new_backup_provider => sub { return TestBackupProvider->new(); },
+    parse_volname => sub {
+        return ('backup', 'vm/100/2025-01-01T00:00:00Z', 100, undef, undef, undef, 'pbs-vm');
+    },
+    parse_volume_id => sub {
+        my ($volid, $noerr) = @_;
+        return ($1, $2) if $volid =~ m/^([a-z]+):(.+)$/;
+        die "unable to parse volume ID '$volid'\n" if !$noerr;
+        return;
+    },
+    path_to_volume_id => sub { return; },
+    storage_config => sub {
+        my ($cfg, $storeid) = @_;
+        return { type => 'pbs', server => 'localhost', datastore => 'ds' } if $storeid eq 'pbs';
+        return { type => 'dir', path => '/mnt/provider' };
+    },
+    storage_has_feature => sub {
+        my ($cfg, $storeid, $feature) = @_;
+        return $storeid eq 'provider' && $feature eq 'backup-provider';
+    },
+);
+
+my $pbs_plugin_module = Test::MockModule->new('PVE::Storage::PBSPlugin');
+$pbs_plugin_module->mock(
+    pbs_encryption_key_file_name => sub { return '/nonexistent/key'; },
+    pbs_get_password => sub { return 'secret'; },
+    run_raw_client_cmd => sub {
+        my ($scfg, $storeid, $cmd, $param) = @_;
+        my (undef, $file, $target) = $param->@*;
+        if ($file eq 'index.json') {
+            my $files =
+                [{ filename => 'qemu-server.conf.blob' }, { filename => 'fw.conf.blob' }];
+            file_set_contents($target, encode_json({ files => $files }));
+        } elsif ($file eq 'qemu-server.conf') {
+            die "unable to read the backup\n" if $fail_at eq 'read-config';
+            file_set_contents($target, $current_backup);
+        } elsif ($file eq 'fw.conf') {
+            file_set_contents($target, "[OPTIONS]\nenable: 1\n");
+        } else {
+            die "run_raw_client_cmd (mocked) - unexpected file '$file'\n";
+        }
+    },
+);
+
+my $network_module = Test::MockModule->new('PVE::QemuServer::Network');
+$network_module->mock(
+    create_ifaces_ipams_ips => sub {
+        die "unable to register IPs\n" if $fail_at eq 'after-restore';
+    },
+);
+
+my $qemu_server_module = Test::MockModule->new('PVE::QemuServer');
+$qemu_server_module->mock(
+    check_running => sub { return 0; },
+    pbs_live_restore => sub {
+        die "live restore failed\n" if $fail_at eq 'live';
+        $calls->{live_restore}++;
+    },
+    rescan => sub { return; },
+    restore_update_config_line => sub {
+        die "restoring the data failed\n" if $fail_at eq 'data';
+        return $qemu_server_module->original('restore_update_config_line')->(@_);
+    },
+    tar_archive_read_firstfile => sub { return 'qemu-server.conf'; },
+    # like 'tar' with qmextract and 'vma extract': write the config and report the VMA header
+    run_command => sub {
+        my ($cmd, %param) = @_;
+
+        if (!ref($cmd) && $cmd =~ m/^zcat -f\|tar xf /) {
+            die "unable to read the backup\n" if $fail_at eq 'read-config';
+            file_set_contents("$ENV{VZDUMP_TMPDIR}/qemu-server.conf", $current_backup);
+            return 0;
+        }
+
+        my $vma = $cmd->[-1];
+        die "run_command (mocked) - unexpected command\n"
+            if ref($vma) ne 'ARRAY' || $vma->[0] ne 'vma';
+        my ($mapfifo, $tmpdir) = $vma->@[4, 6];
+
+        die "unable to read the backup\n" if $fail_at eq 'read-config';
+
+        make_path($tmpdir);
+        file_set_contents("$tmpdir/qemu-server.conf", $current_backup);
+        file_set_contents("$tmpdir/qemu-server.fw", "[OPTIONS]\nenable: 1\n");
+
+        sysopen(my $reader, $mapfifo, O_RDONLY | O_NONBLOCK)
+            or die "unable to open fifo - $!\n";
+        $param{afterfork}->();
+        $param{outfunc}->('CTIME: Wed Jan  1 00:00:00 2025');
+        close($reader);
+
+        return 0;
+    },
+);
+
+my $task_error;
+my $rpcenv_module = Test::MockModule->new('PVE::RPCEnvironment');
+$rpcenv_module->mock(
+    check => sub { return 1; },
+    fork_worker => sub {
+        my ($self, $type, $id, $user, $code) = @_;
+        eval { $code->('UPID:test') };
+        $task_error = $@;
+        return 'UPID:test';
+    },
+);
+
+$QemuServerTest::FakeCFS::cluster_module->mock(check_cfs_quorum => sub { return 1; });
+
+# Restores a backup with the raw config $backup to VMID 100 through the create API, from an archive
+# of type $type with the additional API parameters $param. Returns the error of the API call or the
+# task and the task output.
+sub restore_api {
+    my ($type, $backup, $param) = @_;
+
+    $current_backup = $backup;
+    $calls = {};
+    $task_error = '';
+
+    my ($err, $output) = ('', '');
+    {
+        open(my $out_fh, '>', \$output);
+        local *STDERR = $out_fh;
+        my $oldfh = select($out_fh);
+
+        eval {
+            PVE::API2::Qemu->create_vm({
+                node => 'localhost',
+                vmid => 100,
+                archive => $ARCHIVES->{$type},
+                $param->%*,
+            });
+        };
+        $err = $@ || $task_error;
+
+        select($oldfh);
+        close($out_fh);
+    }
+
+    return ($err, $output);
+}
+
+my $TARGET_CONF = "memory: 1024\nmeta: creation-qemu=10.0.0,ctime=1700000000\nname: target\n";
+my $NOT_CLEANED_UP = qr/VM 100 state is NOT cleaned up/;
+
+for my $type (sort keys $ARCHIVES->%*) {
+    subtest "$type: failed restore to a free VMID" => sub {
+        QemuServerTest::FakeCFS::reset();
+
+        $fail_at = 'data';
+        my ($err, $output) = restore_api($type, backup_config(), {});
+        is($err, "restoring the data failed\n", 'restore failed');
+        ok(!defined($QemuServerTest::FakeCFS::configs->{100}), 'reserved config removed');
+        is($calls->{remove_vmfw_conf}, 1, 'restored firewall config removed');
+        unlike($output, $NOT_CLEANED_UP, 'no warning about a VM left behind');
+
+        $fail_at = '';
+        ($err, $output) = restore_api($type, backup_config(), { force => 1 });
+        is($err, '', 'retry succeeds') or diag($output);
+        like($QemuServerTest::FakeCFS::configs->{100}, qr/^name: backup$/m, 'VM restored');
+    };
+}
+
+subtest 'failed restore to a free VMID before reading the backup' => sub {
+    QemuServerTest::FakeCFS::reset();
+
+    $fail_at = 'read-config';
+    my ($err, $output) = restore_api('pbs', backup_config(), {});
+    is($err, "unable to read the backup\n", 'restore failed');
+    ok(!defined($QemuServerTest::FakeCFS::configs->{100}), 'reserved config removed');
+    unlike($output, $NOT_CLEANED_UP, 'no warning about a VM left behind');
+};
+
+subtest 'another restore right after a failed restore to a free VMID' => sub {
+    QemuServerTest::FakeCFS::reset();
+
+    # run a second restore with 'force' as soon as the failed one releases the guest lock
+    my $lock_config_full = PVE::QemuConfig->can('lock_config_full');
+    my ($interposed, $lock_free, $second_err) = (0, 0, '');
+    $QemuServerTest::FakeCFS::qemu_config_module->mock(
+        lock_config_full => sub {
+            my ($class, $vmid, @param) = @_;
+            my $res = eval { $lock_config_full->($class, $vmid, @param) };
+            my $err = $@;
+            if ($err && !$interposed++) {
+                open(my $fh, '>>', PVE::QemuConfig->config_file_lock($vmid)) or die "$!\n";
+                $lock_free = flock($fh, LOCK_EX | LOCK_NB);
+                close($fh);
+                local $fail_at = '';
+                ($second_err) = restore_api('pbs', backup_config(), { force => 1 });
+            }
+            die $err if $err;
+            return $res;
+        },
+    );
+
+    # the temporary config of a VMA restore has no lock that would stop the second restore
+    $fail_at = 'data';
+    my ($err) = restore_api('vma file', backup_config(), {});
+    $QemuServerTest::FakeCFS::qemu_config_module->unmock('lock_config_full');
+
+    is($err, "restoring the data failed\n", 'first restore failed');
+    ok($lock_free, 'guest lock was free for the second restore');
+    is($second_err, '', 'second restore succeeded');
+    like($QemuServerTest::FakeCFS::configs->{100}, qr/^name: backup$/m, 'its config stays');
+    is($calls->{firewall}, 1, 'its firewall config was restored');
+    ok(!$calls->{remove_vmfw_conf}, 'and stays');
+};
+
+for my $type ('pbs', 'provider', 'vma file', 'vma stdin') {
+    subtest "$type: failed restore over an existing VM" => sub {
+        QemuServerTest::FakeCFS::reset();
+        QemuServerTest::FakeCFS::set_config(100, $TARGET_CONF);
+
+        $fail_at = 'read-config';
+        my ($err, $output) = restore_api($type, backup_config(), { force => 1 });
+        like($err, qr/unable to read the backup\n$/, 'fails before changing the VM');
+        is($QemuServerTest::FakeCFS::configs->{100}, $TARGET_CONF, 'config unchanged');
+        is_deeply($calls, {}, 'replication state and firewall config untouched');
+        unlike($output, $NOT_CLEANED_UP, 'no warning about a changed VM');
+
+        $fail_at = 'data';
+        ($err, $output) = restore_api($type, backup_config(), { force => 1 });
+        is($err, "restoring the data failed\n", 'fails while restoring');
+        like($QemuServerTest::FakeCFS::configs->{100}, qr/^memory: 128$/m, 'temporary config');
+        unlike($QemuServerTest::FakeCFS::configs->{100}, qr/^lock:/m, 'lock removed');
+        like($output, qr/error before or during data restore/, 'warning about the changed VM');
+    };
+}
+
+subtest 'failed live restore to a free VMID keeps the restored config' => sub {
+    QemuServerTest::FakeCFS::reset();
+
+    $fail_at = 'live';
+    my ($err, $output) = restore_api('pbs', backup_config(), { 'live-restore' => 1 });
+    is($err, "live restore failed\n", 'restore failed');
+    like($QemuServerTest::FakeCFS::configs->{100}, qr/^name: backup$/m, 'restored config kept');
+    unlike($QemuServerTest::FakeCFS::configs->{100}, qr/^lock:/m, 'lock removed');
+    like($output, $NOT_CLEANED_UP, 'warning about the VM left behind');
+};
+
+subtest 'failure after the data was restored keeps the VM' => sub {
+    QemuServerTest::FakeCFS::reset();
+
+    $fail_at = 'after-restore';
+    my ($err, $output) = restore_api('pbs', backup_config(), { unique => 1 });
+    is($err, "unable to register IPs\n", 'restore failed');
+    like($QemuServerTest::FakeCFS::configs->{100}, qr/^name: backup$/m, 'restored config kept');
+    like($output, qr/error after data was restored/, 'warning about the VM left behind');
+};
+
+$fail_at = '';
+
 done_testing();
-- 
2.47.3





  parent reply	other threads:[~2026-11-16  0:51 UTC|newest]

Thread overview: 49+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-16  0:46 [PATCH cluster/guest-common/qemu-server/container/manager/docs 00/34] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 01/34] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 02/34] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 03/34] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 04/34] pmxcfs: quorum: pass the corosync membership to an optional listener Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 05/34] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 06/34] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-16  0:46 ` [PATCH cluster 07/34] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
2026-11-16  0:46 ` [PATCH guest-common 08/34] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-17 10:47   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH guest-common 09/34] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-17 10:47   ` Dominik Csapak
2026-11-17 12:47     ` Dominik Csapak
2026-11-16  0:46 ` [PATCH guest-common 10/34] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-16  0:46 ` [PATCH guest-common 11/34] guest identity: allow taking over a uid that is unused in the cluster Thomas Lamprecht
2026-11-17 13:14   ` Dominik Csapak
2026-11-16  0:46 ` Thomas Lamprecht [this message]
2026-11-16  0:46 ` [PATCH qemu-server 13/34] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-16  0:46 ` [PATCH qemu-server 14/34] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-16  0:46 ` [PATCH qemu-server 15/34] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-17 15:17   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH qemu-server 16/34] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-17 15:26   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH qemu-server 17/34] api: status: return the guest uid Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 18/34] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 19/34] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 20/34] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 21/34] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 22/34] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 23/34] migrate: remote: clean up the target on a tunnel version mismatch Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 24/34] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-16  0:46 ` [PATCH container 25/34] api: status: return the guest uid Thomas Lamprecht
2026-11-16  0:46 ` [PATCH manager 26/34] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-17 15:30   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH manager 27/34] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-16  0:46 ` [PATCH manager 28/34] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-16  0:46 ` [PATCH manager 29/34] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-16  0:46 ` [PATCH manager 30/34] ui: guest summary: show " Thomas Lamprecht
2026-11-17 15:44   ` Dominik Csapak
2026-11-16  0:46 ` [PATCH docs 31/34] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-16  0:46 ` [PATCH docs 32/34] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-16  0:46 ` [PATCH docs 33/34] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-16  0:46 ` [PATCH docs 34/34] pct: " Thomas Lamprecht
2026-11-17 10:47 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs 00/34] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
2026-11-17 11:29   ` Thomas Lamprecht
2026-11-17 11:55     ` Dominik Csapak
2026-11-17 12:47       ` Thomas Lamprecht
2026-11-17 13:54         ` Dominik Csapak
2026-11-17 15:55 ` Dominik Csapak

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261116004703.2908294-13-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal