public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH container v5 22/41] fix #7846: config: add a guest identity to the 'meta' property
Date: Thu, 19 Nov 2026 20:18:54 +0100	[thread overview]
Message-ID: <20261119191914.2336934-23-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20261119191914.2336934-1-t.lamprecht@proxmox.com>

Give containers a guest identity in the 'meta' property: a random
identifier, 'uid', and the time the container was registered in the
cluster, 'rtime'.

With identities enabled for the config class, every config write keeps
the identity a container already has, or assigns one if it has none,
whatever the written config carries. So create and clone get a new
identity when the VMID is reserved, while config changes, applying
pending changes, snapshot rollback and a restore over an existing
container keep it. A container without identity gets one with its first
config write by an upgraded node.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
Reviewed-by: Dominik Csapak <d.csapak@proxmox.com>
Tested-by: Dominik Csapak <d.csapak@proxmox.com>
---

changes v4 -> v5:
* dropped the test and the commit message paragraph about the post-stop
  hook writing a config that it loaded earlier: the hook only changes
  the config under its lock now, through an independent patch

changes v3 -> v4:
* tests only: rebase onto the test script of the independent test commit
  before the series, the mocks that the identity tests need come with
  this patch

changes v1 -> v2:
* rebase, adapt the tests

 src/PVE/LXC/Config.pm       |   8 ++
 src/test/lifecycle_tests.pl | 198 ++++++++++++++++++++++++++++++++++++
 2 files changed, 206 insertions(+)

diff --git a/src/PVE/LXC/Config.pm b/src/PVE/LXC/Config.pm
index 73272bb..26d9d83 100644
--- a/src/PVE/LXC/Config.pm
+++ b/src/PVE/LXC/Config.pm
@@ -9,6 +9,7 @@ use PVE::AbstractConfig;
 use PVE::Cluster qw(cfs_register_file);
 use PVE::DataCenterConfig;
 use PVE::GuestHelpers;
+use PVE::GuestIdentity;
 use PVE::INotify;
 use PVE::JSONSchema qw(get_standard_option);
 use PVE::Network::SDN::Vnets;
@@ -60,6 +61,12 @@ sub cfs_config_path {
     return "nodes/$node/lxc/$vmid.conf";
 }
 
+sub __identity_enabled {
+    my ($class) = @_;
+
+    return 1;
+}
+
 sub mountpoint_backup_enabled {
     my ($class, $mp_key, $mountpoint) = @_;
 
@@ -512,6 +519,7 @@ my $meta_info_fmt = {
         minimum => 0,
         optional => 1,
     },
+    PVE::GuestIdentity::meta_format_properties()->%*,
 };
 
 my $confdesc = {
diff --git a/src/test/lifecycle_tests.pl b/src/test/lifecycle_tests.pl
index 341732a..a9c857d 100755
--- a/src/test/lifecycle_tests.pl
+++ b/src/test/lifecycle_tests.pl
@@ -11,6 +11,7 @@ use lib qw(..);
 # tested modules load. Tests that call into them mock the functions they need.
 BEGIN {
     $INC{'PVE/HA/Config.pm'} = $INC{'PVE/HA/Env/PVE2.pm'} = __FILE__;
+    $INC{'PVE/VZDump.pm'} = __FILE__;
 }
 
 use Fcntl qw(:flock);
@@ -22,6 +23,7 @@ use Test::More;
 
 use PVE::Cluster;
 use PVE::GuestHelpers;
+use PVE::GuestIdentity;
 use PVE::INotify;
 use PVE::Tools qw(file_get_contents file_set_contents);
 
@@ -92,6 +94,12 @@ my $storage_cfg = {
             type => 'pbs',
             content => { backup => 1 },
         },
+        shared => {
+            type => 'nfs',
+            path => '/mnt/pve/shared',
+            shared => 1,
+            content => { rootdir => 1, images => 1 },
+        },
     },
 };
 
@@ -157,6 +165,28 @@ $config_module->mock(
     },
 );
 
+# guest identities are generated deterministically
+my $random_counter = 0;
+my $now_ms = 1_700_000_000_000;
+
+my $identity_module = Test::MockModule->new('PVE::GuestIdentity');
+$identity_module->mock(
+    __random_bytes => sub {
+        my ($len) = @_;
+        die "unexpected length $len\n" if $len != 20;
+        # one byte per uid character, the last three hold 5 bits of the counter each
+        my $n = ++$random_counter;
+        return ("\0" x 17) . pack('C3', map { ($n >> $_) & 31 } (10, 5, 0));
+    },
+    __time_ms => sub { return $now_ms },
+);
+
+my $repl_config_module = Test::MockModule->new('PVE::ReplicationConfig');
+$repl_config_module->mock(
+    new => sub { return bless {}, 'PVE::ReplicationConfig' },
+    check_for_existing_jobs => sub { return undef },
+);
+
 # Fake environment, records task output
 
 my $task_log = '';
@@ -202,6 +232,7 @@ $restenv_module->mock(
         chomp($msg);
         $task_log .= "WARN: $msg\n";
     },
+    setup_default_cli_env => sub { },
 );
 
 # Runs $code as $user, who lacks what $deny lists, see the check method above.
@@ -286,6 +317,8 @@ $storage_module->mock(
         die "unexpected volume '$volid'\n" if $volid ne $PBS_VOLID;
         return ('backup', 'ct/100/2026-01-01T00:00:00Z', 100, undef, undef, undef, 'pbs-ct');
     },
+    volume_rollback_is_possible => sub { return 1 },
+    volume_snapshot_rollback => sub { },
 );
 
 my $lxc_module = Test::MockModule->new('PVE::LXC');
@@ -318,6 +351,10 @@ $lxc_module->mock(
         return $rootdir;
     },
     umount_all => sub { },
+    copy_volume => sub {
+        my ($mp, $newid) = @_;
+        return "local:$newid/vm-$newid-disk-0.raw";
+    },
     create_ifaces_ipams_ips => sub { },
     delete_ifaces_ipams_ips => sub { },
     update_lxc_config => sub { },
@@ -383,10 +420,12 @@ $setup_module->mock(new => sub { return bless {}, 'PVE::LXC::Test::Setup' });
 package PVE::LXC::Test::Setup {
     sub post_create_hook { die $hook_error if defined($hook_error) }
     sub post_restore_hook { die $hook_error if defined($hook_error) }
+    sub post_clone_hook { }
 }
 
 my $firewall_module = Test::MockModule->new('PVE::Firewall');
 $firewall_module->mock(
+    clone_vmfw_conf => sub { },
     remove_vmfw_conf => sub {
         my ($vmid) = @_;
         PVE::GuestHelpers::set_firewall_config_raw($vmid, undef);
@@ -437,6 +476,7 @@ my sub reset_state {
     $backup_raw_config = backup_config();
     ($backup_read_error, $hook_error, $in_between) = ();
     $backup_fw_config = undef;
+    $now_ms = 1_700_000_000_000;
 }
 
 # API call helpers
@@ -471,6 +511,17 @@ my sub update_ct($vmid, $param) {
     });
 }
 
+my sub clone_ct($vmid, $newid, $param = undef) {
+    return run_task(sub {
+        PVE::API2::LXC->clone_vm({
+            node => $NODENAME,
+            vmid => $vmid,
+            newid => $newid,
+            ($param // {})->%*,
+        });
+    });
+}
+
 # Runs the migration tunnel for $vmid with the commands @commands, which are sent at once, and
 # returns the error of the task, if any, and the replies.
 my sub run_mtunnel($vmid, @commands) {
@@ -538,6 +589,10 @@ my sub meta_of($vmid) {
     return { map { split(/=/, $_, 2) } split(/,/, $conf->{meta} // '') };
 }
 
+my sub identity_of($vmid) {
+    return PVE::LXC::Config->get_identity(PVE::LXC::Config->load_config($vmid));
+}
+
 # Tests
 
 subtest 'create and restore write the config of the container' => sub {
@@ -1409,6 +1464,149 @@ subtest 'create sets the creation time, restore and remote migration keep it' =>
     is(PVE::LXC::Config->load_config(300)->{hostname}, 'incoming', 'with its other options');
 };
 
+subtest 'create assigns a new guest identity' => sub {
+    reset_configs();
+    $now_ms = 1_700_000_000_000;
+
+    my $err = create_ct(201);
+    is($err, '', 'create succeeds');
+
+    my $meta = meta_of(201);
+    like($meta->{uid}, qr/^[a-z2-7]{20}$/, 'uid is assigned');
+    is($meta->{rtime}, 1_700_000_000_000, 'rtime is the time of the create');
+    ok(defined($meta->{ctime}), 'ctime is kept next to the identity');
+    ok(!defined(PVE::LXC::Config->load_config(201)->{lock}), 'create lock is removed');
+
+    $err = create_ct(202);
+    isnt(identity_of(202)->{uid}, $meta->{uid}, 'another container gets another uid');
+};
+
+subtest 'config changes keep the guest identity' => sub {
+    reset_configs();
+    my $meta = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis';
+    write_raw_config(200, existing_config($meta));
+    $now_ms = 1_700_000_000_000;
+
+    my $err = update_ct(200, { hostname => 'changed', memory => 1024 });
+    is($err, '', 'update succeeds');
+    my $conf = PVE::LXC::Config->load_config(200);
+    is($conf->{hostname}, 'changed', 'change is applied');
+    is($conf->{meta}, $meta, 'meta is unchanged');
+
+    write_raw_config(200, existing_config('ctime=1500000000'));
+    $err = update_ct(200, { hostname => 'first-write' });
+    is($err, '', 'update of a container without identity succeeds');
+    my $identity = identity_of(200);
+    like($identity->{uid}, qr/^[a-z2-7]{20}$/, 'first write assigns a uid');
+    is($identity->{rtime}, 1_700_000_000_000, 'first write sets rtime');
+    is(meta_of(200)->{ctime}, 1500000000, 'ctime is kept');
+
+    $now_ms += 1000;
+    $err = update_ct(200, { hostname => 'second-write' });
+    is_deeply(identity_of(200), $identity, 'later writes keep the identity');
+};
+
+subtest 'clone assigns a new guest identity' => sub {
+    reset_configs();
+    write_raw_config(
+        200,
+        existing_config('ctime=1500000000,rtime=1500000000000,uid=existingexistingexis'),
+    );
+    $now_ms = 1_700_000_000_000;
+
+    my $err = clone_ct(200, 201, { hostname => 'clone' });
+    is($err, '', 'clone succeeds') or diag($task_log);
+
+    my $meta = meta_of(201);
+    like($meta->{uid}, qr/^[a-z2-7]{20}$/, 'clone has a uid');
+    isnt($meta->{uid}, 'existingexistingexis', 'clone does not get the uid of the source');
+    is($meta->{rtime}, 1_700_000_000_000, 'rtime is the time of the clone');
+    is($meta->{ctime}, 1500000000, 'ctime of the source is kept');
+    is(identity_of(200)->{uid}, 'existingexistingexis', 'source keeps its uid');
+};
+
+subtest 'snapshot rollback keeps the current guest identity' => sub {
+    reset_configs();
+    write_raw_config(200, <<~'EOF');
+        arch: amd64
+        hostname: existing
+        memory: 512
+        meta: ctime=1500000000,rtime=1600000000000,uid=existingexistingexis
+        ostype: debian
+        parent: snap
+        rootfs: local:200/vm-200-disk-0.raw,size=4G
+        unprivileged: 1
+
+        [snap]
+        arch: amd64
+        hostname: snapshot
+        memory: 512
+        meta: ctime=1500000000,rtime=1500000000000,uid=snapshotsnapshotsnap
+        ostype: debian
+        rootfs: local:200/vm-200-disk-0.raw,size=4G
+        snaptime: 1500000000
+        unprivileged: 1
+        EOF
+
+    my $err = run_task(sub { PVE::LXC::Config->snapshot_rollback(200, 'snap') });
+    is($err, '', 'rollback succeeds') or diag($task_log);
+
+    my $conf = PVE::LXC::Config->load_config(200);
+    is($conf->{hostname}, 'snapshot', 'config of the snapshot is restored');
+    is_deeply(
+        identity_of(200),
+        { uid => 'existingexistingexis', rtime => 1600000000000 },
+        'current identity is kept',
+    );
+};
+
+subtest 'in-place restore keeps the guest identity of the target' => sub {
+    reset_configs();
+    my $meta = 'ctime=1500000000,rtime=1500000000000,uid=existingexistingexis';
+    write_raw_config(200, existing_config($meta));
+
+    for my $backup_meta (
+        undef, 'ctime=1400000000,rtime=1400000000000,uid=existingexistingexis',
+    ) {
+        $backup_raw_config = backup_config($backup_meta);
+        my $err = restore_ct(200, { force => 1 });
+        is($err, '', 'restore succeeds') or diag($task_log);
+        is_deeply(
+            identity_of(200),
+            { uid => 'existingexistingexis', rtime => 1500000000000 },
+            'identity is kept',
+        );
+        is(PVE::LXC::Config->load_config(200)->{hostname}, 'backup', 'backup is restored');
+    }
+};
+
+subtest 'backup assigns the guest identity and includes it' => sub {
+    reset_configs();
+    write_raw_config(987, existing_config('ctime=1500000000'));
+    $now_ms = 1_700_000_000_000;
+
+    require PVE::VZDump::LXC;
+    my $plugin = bless { vzdump => { opts => { pbs => 1 } } }, 'PVE::VZDump::LXC';
+
+    $plugin->lock_vm(987);
+    my $identity = identity_of(987);
+    like($identity->{uid}, qr/^[a-z2-7]{20}$/, 'backup lock assigns a uid');
+    is($identity->{rtime}, $now_ms, 'backup lock sets rtime');
+
+    $plugin->assemble({ tmpdir => "$WORKDIR/backup" }, 987);
+    my $raw = file_get_contents("$WORKDIR/backup/etc/vzdump/pct.conf");
+    my $backup_conf = PVE::LXC::Config::parse_pct_config('/lxc/987.conf', $raw);
+    is_deeply(
+        PVE::LXC::Config->get_identity($backup_conf),
+        $identity,
+        'backup has the identity',
+    );
+
+    $now_ms += 1000;
+    $plugin->unlock_vm(987);
+    is_deeply(identity_of(987), $identity, 'removing the backup lock keeps the identity');
+};
+
 remove_tree($WORKDIR);
 
 done_testing();
-- 
2.47.3





  parent reply	other threads:[~2026-11-19 19:22 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 19:18 [PATCH cluster/guest-common/qemu-server/container/manager/docs v5 00/41] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 01/41] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 02/41] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 03/41] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 04/41] pmxcfs: quorum: record the corosync membership Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 05/41] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 06/41] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-19 19:18 ` [PATCH cluster v5 07/41] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 08/41] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 09/41] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 10/41] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 11/41] guest identity: allow adopting a uid that is unused in the cluster Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 12/41] migrate: add shared helpers for the identity command of the migration tunnel Thomas Lamprecht
2026-11-19 19:18 ` [PATCH guest-common v5 13/41] vzdump: notes template: list the uid variable Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 14/41] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 15/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 16/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 17/41] restore: adopt the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 18/41] api: remote migration: adopt the guest uid from the source Thomas Lamprecht
2026-11-19 19:18 ` [PATCH qemu-server v5 19/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 20/41] restore: decide by the reservation whether the container is new Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 21/41] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-19 19:18 ` Thomas Lamprecht [this message]
2026-11-19 19:18 ` [PATCH container v5 23/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 24/41] restore: adopt the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 25/41] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 26/41] api: remote migration: adopt the guest uid from the source Thomas Lamprecht
2026-11-19 19:18 ` [PATCH container v5 27/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 28/41] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 29/41] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 30/41] ui: datacenter summary: show the " Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 31/41] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 32/41] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 33/41] ui: guest summary: show " Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 34/41] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
2026-11-19 19:19 ` [PATCH manager v5 35/41] vzdump: notes template: add the guest uid as variable Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 36/41] vzdump: restore: describe what a failed restore can leave behind Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 37/41] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 38/41] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 39/41] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 40/41] pct: " Thomas Lamprecht
2026-11-19 19:19 ` [PATCH docs v5 41/41] vzdump: notes: document the uid template variable Thomas Lamprecht
2026-11-20  3:07 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v5 00/41] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119191914.2336934-23-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal