all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Thomas Lamprecht <t.lamprecht@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH guest-common v4 12/41] migrate: add shared helpers for the identity command of the migration tunnel
Date: Thu, 19 Nov 2026 11:33:58 +0100	[thread overview]
Message-ID: <20261119103551.1666964-13-t.lamprecht@proxmox.com> (raw)
In-Reply-To: <20261119103551.1666964-1-t.lamprecht@proxmox.com>

In a remote migration, the source sends the uid of the guest to the
target with the 'identity' tunnel command. Without shared code, VMs and
containers would each define three things: the command, the rule that
the target accepts it only once and only before the config arrives, and
the sender with its timeout and log line. All of these would have to be
kept in sync by hand with every change of the tunnel.

Add the parameter schema and the handler of the target to
PVE::GuestIdentity, as PVE::StorageTunnel does for the storage commands.
Add the sender to PVE::AbstractMigrate, next to the sender of the
'bwlimit' command.

The target adopts the uid right after it reserved the VMID, before
anything is transferred. If another guest in the target cluster has the
uid, the handler refuses the migration, unless the caller allows a new
uid for the migrated guest. That override is a standard option, so that
the migration commands of both guest types describe it the same way.

The reply of the target is about the guest on the target. The guest on
the source keeps its uid in any case, so the source logs the reply in
its own words. If a newer target reports a result that the source does
not know, the source only logs a warning.

If the parsed config of the source has no identity, the sender writes
the config first. A guest without a uid gets one with that write. This
way, the source keeps the uid that it sends and does not get another one
with a later write. The write also brings back a stored uid that the
parser of the guest type dropped, for example because of a 'meta' member
that it does not know. Callers send the command only after the version
check of the tunnel. A target that this check refuses then leaves the
source config unchanged.

Signed-off-by: Thomas Lamprecht <t.lamprecht@proxmox.com>
---

changes v3 -> v4:
* the source logs the reply in its own words (the migrated guest keeps
  its uid, or gets a new one), not in those of the target, which are
  about the guest there
* the handler adopts the uid and refuses one in use itself, without
  adopt_from_source of the previous patch
* 'allow-new-uid' is a standard option, for the command schema and for
  the migration commands of both guest types
* the sender is called adopt_identity_on_target
* comment and commit message: only a target refused by the tunnel
  version check leaves the source config unchanged
* tests: the log lines of both sides, and a failure to take the identity
  lock on the target
* commit message: plainer wording, same content

changes v2 -> v3:
* new in v3: command schema, target handler and sender moved here from
  qemu-server and container
* the sender gives a guest without uid one before it sends it, and sends
  the stored uid if the parser dropped 'meta'

 src/PVE/AbstractMigrate.pm         |  42 ++++++
 src/PVE/GuestIdentity.pm           |  51 +++++++
 src/tests/config-identity-tests.pl | 229 +++++++++++++++++++++++++++++
 3 files changed, 322 insertions(+)

diff --git a/src/PVE/AbstractMigrate.pm b/src/PVE/AbstractMigrate.pm
index 5d74f53..3c67a50 100644
--- a/src/PVE/AbstractMigrate.pm
+++ b/src/PVE/AbstractMigrate.pm
@@ -443,6 +443,48 @@ sub get_bwlimit {
     return $bwlimit;
 }
 
+# Lets the target of a remote migration adopt the uid of the guest, whose config class is
+# $config_class, through the tunnel $tunnel, before anything gets transferred there. The target
+# refuses a uid that another guest there has, unless the migration allows a new uid, see
+# PVE::GuestIdentity::handle_tunnel_identity. Call it only after the version check of the tunnel,
+# so that a target refused by that check leaves the config of the source unchanged.
+sub adopt_identity_on_target {
+    my ($self, $config_class, $tunnel) = @_;
+
+    my $conf = $self->{vmconf};
+    my $identity = $config_class->get_identity($conf);
+    if (!$identity) {
+        # A guest without uid gets one first, so the source keeps the uid that it sends. The write
+        # also restores a stored uid that the parser of the guest type dropped.
+        $config_class->write_config($self->{vmid}, $conf);
+        $identity = $config_class->get_identity($conf) or return;
+    }
+    my $uid = $identity->{uid};
+
+    my $params = { uid => $uid, 'allow-new-uid' => $self->{opts}->{'allow-new-uid'} ? 1 : 0 };
+
+    # the target waits for the cluster-wide guest identity lock
+    my $res = PVE::Tunnel::write_tunnel($tunnel, 60, 'identity', $params);
+    my $reply = $res->{identity} or return;
+
+    # the guest here keeps its uid in any case, the result is about the guest on the target
+    my $result = $reply->{result} // '';
+    if ($result eq 'adopted' || $result eq 'kept') {
+        $self->log('info', "guest identity: the migrated guest keeps uid $uid");
+    } elsif ($result eq 'in-use') {
+        $self->log(
+            'info',
+            "guest identity: guest $reply->{'in-use-by'} in the target cluster has uid $uid, the"
+                . " migrated guest gets the new uid $reply->{uid}",
+        );
+    } else {
+        # from a newer target, which accepted the uid by then
+        $self->log('warn', "guest identity: unknown result '$result' of the target");
+    }
+
+    return;
+}
+
 sub map_storage {
     my ($self, $scfg, $storeid) = @_;
 
diff --git a/src/PVE/GuestIdentity.pm b/src/PVE/GuestIdentity.pm
index 7763ac7..531a7b7 100644
--- a/src/PVE/GuestIdentity.pm
+++ b/src/PVE/GuestIdentity.pm
@@ -387,6 +387,57 @@ sub adopt_from_backup($class, $vmid, $uid, $reservation) {
     return;
 }
 
+PVE::JSONSchema::register_standard_option(
+    'pve-guest-allow-new-uid',
+    {
+        type => 'boolean',
+        description => "Allow the migration if another guest in the target cluster already has the"
+            . " uid of this guest. The migrated guest then gets a new uid.",
+        optional => 1,
+        default => 0,
+    },
+);
+
+# Returns the properties of the parameters of the 'identity' command of the migration tunnel, see
+# handle_tunnel_identity. Every call returns a new copy.
+sub tunnel_identity_cmd_schema() {
+    return {
+        uid => {
+            type => 'string',
+            description => 'uid of the source guest',
+        },
+        'allow-new-uid' => PVE::JSONSchema::get_standard_option('pve-guest-allow-new-uid'),
+    };
+}
+
+# Handles the 'identity' command with the parameters $params on the target of a remote migration of
+# a guest of the config class $class: adopts the uid of the source guest for the guest that the
+# tunnel reserved with $reservation, logs the result, and returns the reply for the source. $state
+# is the state of the tunnel, with the VMID in 'vmid' and the lock of the guest in 'lock'.
+#
+# The command is only accepted once, and only before the config arrives, while the guest still has
+# the 'create' lock of the reservation. If another guest in the cluster has the uid, the command
+# fails, which refuses the migration, unless the parameter 'allow-new-uid' is set, in which case the
+# guest keeps the uid of its reservation.
+sub handle_tunnel_identity($class, $state, $reservation, $params) {
+    die "the uid can only be adopted once, before the config arrives\n"
+        if ($state->{lock} // '') ne 'create' || $state->{'identity-handled'};
+    $state->{'identity-handled'} = 1;
+
+    my $uid = $params->{uid};
+    my $res = eval { adopt($class, $state->{vmid}, $uid, $reservation) };
+    die "unable to adopt the uid of the source guest - $@" if $@;
+
+    if ($res->{result} eq 'in-use' && !$params->{'allow-new-uid'}) {
+        die "guest $res->{'in-use-by'} in the target cluster already has the uid '$uid' of this"
+            . " guest - use 'allow-new-uid' to migrate it with a new uid\n";
+    }
+
+    print adopt_result_message($res, 'source guest') . "\n";
+
+    return { identity => $res };
+}
+
 # Returns undef if $node runs a version of Proxmox VE that keeps the identity when writing guest
 # configs, see MIN_PVE_MANAGER_VERSION. Missing or invalid version information counts as too old.
 # Otherwise returns a hash with 'required-version', the minimum version, and 'version', the version
diff --git a/src/tests/config-identity-tests.pl b/src/tests/config-identity-tests.pl
index 8172fc4..df01f53 100755
--- a/src/tests/config-identity-tests.pl
+++ b/src/tests/config-identity-tests.pl
@@ -11,8 +11,10 @@ use Test::MockModule;
 use Test::More;
 
 use PVE::AbstractConfig;
+use PVE::AbstractMigrate;
 use PVE::GuestHelpers;
 use PVE::GuestIdentity;
+use PVE::JSONSchema;
 
 my $tmpdir = tempdir('config-identity-XXXXXX', DIR => '.', CLEANUP => 1);
 $PVE::GuestHelpers::lockdir = "$tmpdir/lock";
@@ -1126,4 +1128,231 @@ subtest 'drop pending meta' => sub {
     }
 };
 
+# Runs $code with the output to STDOUT captured, returns the error and the output.
+my sub capture_output($code) {
+    my $output = '';
+    open(my $fh, '>', \$output) or die "unable to capture output - $!\n";
+    my $previous_fh = select($fh);
+    eval { $code->() };
+    my $err = $@;
+    select($previous_fh);
+    close($fh);
+
+    return ($err, $output);
+}
+
+subtest 'tunnel identity command on the target' => sub {
+    my $class = 'PVE::TestConfig';
+
+    # validates the parameters like the tunnel endpoints do, returns the error, output and reply
+    my $handle = sub($state, $reservation, $params) {
+        my $schema = {
+            type => 'object',
+            properties => PVE::GuestIdentity::tunnel_identity_cmd_schema(),
+            additionalProperties => 0,
+        };
+        my $reply;
+        my ($err, $output) = capture_output(sub {
+            PVE::JSONSchema::validate($params, $schema);
+            $reply =
+                PVE::GuestIdentity::handle_tunnel_identity($class, $state, $reservation, $params);
+        });
+        return ($err, $output, $reply);
+    };
+    my $once = "the uid can only be adopted once, before the config arrives\n";
+
+    my $uid = new_uid();
+    my $reservation = $class->create_and_lock_config(600);
+    my $state = { vmid => 600, lock => 'create' };
+    my ($err, $output, $reply) = $handle->($state, $reservation, { uid => $uid });
+    is($err, '', 'free uid: accepted');
+    is_deeply(
+        $reply,
+        { identity => { result => 'adopted', uid => $uid, requested => $uid } },
+        'free uid: reply',
+    );
+    is($output, "guest identity: adopted uid $uid from the source guest\n", 'free uid: logged');
+    is(identity_of($class, 600)->{uid}, $uid, 'free uid: stored');
+
+    ($err) = $handle->($state, $reservation, { uid => new_uid() });
+    is($err, $once, 'second command is refused');
+    is(identity_of($class, 600)->{uid}, $uid, 'second command: uid is unchanged');
+
+    $reservation = $class->create_and_lock_config(601);
+    my $reserved = identity_of($class, 601);
+    ($err) = $handle->({ vmid => 601, lock => 'migrate' }, $reservation, { uid => new_uid() });
+    is($err, $once, 'command after the config is refused');
+    is_deeply(identity_of($class, 601), $reserved, 'after the config: identity is kept');
+
+    ($err, $output) =
+        $handle->({ vmid => 601, lock => 'create' }, $reservation, { uid => $uid });
+    is(
+        $err,
+        "guest 600 in the target cluster already has the uid '$uid' of this guest - use"
+            . " 'allow-new-uid' to migrate it with a new uid\n",
+        'uid in use: refused',
+    );
+    is($output, '', 'uid in use: no result logged');
+    is_deeply(identity_of($class, 601), $reserved, 'uid in use: reserved identity stays');
+
+    ($err, $output, $reply) = $handle->(
+        { vmid => 601, lock => 'create' },
+        $reservation,
+        { uid => $uid, 'allow-new-uid' => 1 },
+    );
+    is($err, '', 'uid in use with allow-new-uid: accepted');
+    is_deeply(
+        $reply,
+        {
+            identity => {
+                result => 'in-use',
+                uid => $reserved->{uid},
+                requested => $uid,
+                'in-use-by' => 600,
+            },
+        },
+        'uid in use with allow-new-uid: reply',
+    );
+    is(
+        $output,
+        "guest identity: source guest uid $uid is in use by guest 600, keeping uid"
+            . " $reserved->{uid}\n",
+        'uid in use with allow-new-uid: logged',
+    );
+    is_deeply(identity_of($class, 601), $reserved, 'uid in use with allow-new-uid: kept');
+
+    $reservation = $class->create_and_lock_config(602);
+    {
+        local $lock_domain_error = "cfs-lock 'domain-guest-identity' error: no quorum!\n";
+        ($err) =
+            $handle->({ vmid => 602, lock => 'create' }, $reservation, { uid => new_uid() });
+    }
+    is(
+        $err,
+        "unable to adopt the uid of the source guest - cfs-lock 'domain-guest-identity' error:"
+            . " no quorum!\n",
+        'lock error is passed on',
+    );
+
+    $reservation = $class->create_and_lock_config(603);
+    ($err) = $handle->({ vmid => 603, lock => 'create' }, $reservation, {});
+    like($err, qr/^uid: property is missing/m, 'missing uid is refused');
+    ($err) =
+        $handle->({ vmid => 603, lock => 'create' }, $reservation, { uid => $uid, x => 1 });
+    like($err, qr/^x: property is not defined in schema/m, 'unknown parameter is refused');
+
+    delete PVE::GuestIdentity::tunnel_identity_cmd_schema()->{uid};
+    ok(PVE::GuestIdentity::tunnel_identity_cmd_schema()->{uid}, 'schema is a new copy');
+
+    my $option = PVE::JSONSchema::get_standard_option('pve-guest-allow-new-uid');
+    is($option->{type}, 'boolean', 'the override is a standard option');
+};
+
+subtest 'tunnel identity command on the source' => sub {
+    my $class = 'PVE::TestConfig';
+
+    my $tunnel = { version => 3 };
+    my ($commands, $reply);
+    local *PVE::Tunnel::write_tunnel = sub {
+        my ($used_tunnel, $timeout, $command, $params) = @_;
+        push $commands->@*, [$used_tunnel, $timeout, $command, $params];
+        die $reply if !ref($reply);
+        return $reply;
+    };
+
+    # like the migration of the guest $vmid with the options $opts after its preparation loaded
+    # the config
+    my $migration = sub($vmid, $opts = {}) {
+        PVE::Cluster::cfs_update();
+        my $conf = $class->load_config($vmid);
+        return bless { vmid => $vmid, vmconf => $conf, opts => $opts }, 'PVE::AbstractMigrate';
+    };
+    # returns the error, the output, and the sent parameters
+    my $send = sub($name, $migrate) {
+        $commands = [];
+        my ($err, $output) =
+            capture_output(sub { $migrate->adopt_identity_on_target($class, $tunnel) });
+        my @sent = map { [$_->@[0 .. 2]] } $commands->@*;
+        is_deeply(\@sent, [[$tunnel, 60, 'identity']], "$name: one command, up to a minute");
+        return ($err, $output, $commands->[0]->[3]);
+    };
+
+    $class->write_config(610, { name => 'source' });
+    my $source = identity_of($class, 610);
+    $reply =
+        { identity => { result => 'adopted', uid => $source->{uid}, requested => $source->{uid} } };
+    my $writes_before = $writes;
+    my ($err, $output, $params) = $send->('source with uid', $migration->(610));
+    is($err, '', 'source with uid: no error');
+    is_deeply($params, { uid => $source->{uid}, 'allow-new-uid' => 0 },
+        'source with uid: sent');
+    is($writes, $writes_before, 'source with uid: nothing is written');
+    like(
+        $output,
+        qr/^\S+ \S+ guest identity: the migrated guest keeps uid $source->{uid}\n\z/,
+        'source with uid: result is logged',
+    );
+
+    $reply = {
+        identity => {
+            result => 'in-use',
+            uid => 'targettargettargetta',
+            requested => $source->{uid},
+            'in-use-by' => 7,
+        },
+    };
+    ($err, $output, $params) =
+        $send->('allow-new-uid', $migration->(610, { 'allow-new-uid' => 1 }));
+    is($params->{'allow-new-uid'}, 1, 'allow-new-uid is sent');
+    my $line =
+        "guest identity: guest 7 in the target cluster has uid $source->{uid}, the migrated"
+        . " guest gets the new uid targettargettargetta";
+    like($output, qr/^\S+ \S+ \Q$line\E\n\z/, 'uid in use on the target: result is logged');
+
+    set_file($class, 611, "name: source\n");
+    my $migrate = $migration->(611);
+    $reply = { success => 1 };
+    $writes_before = $writes;
+    ($err, $output, $params) = $send->('source without uid', $migrate);
+    my $assigned = identity_of($class, 611);
+    ok($assigned, 'source without uid: gets one');
+    is($writes, $writes_before + 1, 'source without uid: written once');
+    is($params->{uid}, $assigned->{uid}, 'source without uid: sends the uid it got');
+    is_deeply(
+        $class->get_identity($migrate->{vmconf}),
+        $assigned,
+        'source without uid: loaded config has it',
+    );
+    is($output, '', 'source without uid: reply without result is not logged');
+
+    my $stored_uid = new_uid();
+    set_file($class, 612, "meta: ctime=1,rtime=5,uid=$stored_uid\nname: source\n");
+    $migrate = $migration->(612);
+    # like a parser that drops a 'meta' property with a member it does not know
+    delete $migrate->{vmconf}->{meta};
+    ($err, $output, $params) = $send->('stored uid only', $migrate);
+    is(
+        $params->{uid}, $stored_uid, 'stored uid only: sends the stored uid',
+    );
+    is_deeply(
+        identity_of($class, 612),
+        { uid => $stored_uid, rtime => 5 },
+        'stored uid only: stored identity is kept',
+    );
+
+    $reply =
+        { identity => { result => 'future', uid => $stored_uid, requested => $stored_uid } };
+    ($err, $output) = $send->('unknown result', $migration->(612));
+    is($err, '', 'unknown result: no error');
+    like(
+        $output,
+        qr/ guest identity: unknown result 'future' of the target$/m,
+        'unknown result: warning',
+    );
+
+    $reply = "error - tunnel command 'identity' failed - refused\n";
+    ($err, $output) = $send->('failed command', $migration->(612));
+    is($err, $reply, 'failed command: error is passed on');
+};
+
 done_testing();
-- 
2.47.3





  parent reply	other threads:[~2026-11-19 10:37 UTC|newest]

Thread overview: 46+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-19 10:33 [PATCH cluster/guest-common/qemu-server/container/manager/docs v4 00/41] fix #7846: guest identity (uid) and cluster ID Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 01/41] pmxcfs: reject changes to the cluster ID file from user space Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 02/41] pmxcfs: generate a cluster ID on standalone nodes Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 03/41] pmxcfs: dfsm: add a callback that runs once all members are in sync Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 04/41] pmxcfs: quorum: record the corosync membership Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 05/41] pmxcfs: dcdb: generate the cluster ID once the cluster is in sync Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 06/41] cluster: add helper to read the cluster ID Thomas Lamprecht
2026-11-19 10:33 ` [PATCH cluster v4 07/41] pmxcfs: allow removing the cluster ID file in forced local mode Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 08/41] guest identity: add helpers for the guest uid and registration time Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 09/41] abstract config: assign a guest identity and keep it on config writes Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 10/41] abstract config: let the VMID reservation return the previous state Thomas Lamprecht
2026-11-19 10:33 ` [PATCH guest-common v4 11/41] guest identity: allow adopting a uid that is unused in the cluster Thomas Lamprecht
2026-11-19 10:33 ` Thomas Lamprecht [this message]
2026-11-19 10:33 ` [PATCH guest-common v4 13/41] vzdump: notes template: list the uid variable Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 14/41] api: restore: release the VMID if a restore to an unused VMID fails Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 15/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 16/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 17/41] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 18/41] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-19 10:34 ` [PATCH qemu-server v4 19/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 20/41] restore: decide by the reservation whether the container is new Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 21/41] config: add read-only meta property with the creation time Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 22/41] fix #7846: config: add a guest identity to the 'meta' property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 23/41] api: migrate, clone: refuse targets that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 24/41] restore: take over the guest uid from the backup where possible Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 25/41] restore: refuse overwriting a container with another guest's backup Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 26/41] api: remote migration: take over the guest uid from the source Thomas Lamprecht
2026-11-19 10:34 ` [PATCH container v4 27/41] api: status: return the guest uid Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 28/41] api: cluster resources: add guest uid and registration time Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 29/41] api: cluster status: add cluster ID Thomas Lamprecht
2026-11-19 12:32   ` Lukas Wagner
2026-11-19 12:51     ` Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 30/41] ui: datacenter summary: show the " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 31/41] ui: restore: allow overwriting a guest with a backup of another guest Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 32/41] ui: migrate: refuse target nodes that would drop the guest identity Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 33/41] ui: guest summary: show " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 34/41] api: bulk migrate: refuse targets that would drop " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH manager v4 35/41] vzdump: notes template: add the guest uid as variable Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 36/41] vzdump: restore: describe what a failed restore can leave behind Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 37/41] vzdump: describe the guest identity of VMs and containers Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 38/41] pvecm: document the cluster ID and how to get a new one Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 39/41] qm: configuration: describe the read-only meta property Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 40/41] pct: " Thomas Lamprecht
2026-11-19 10:34 ` [PATCH docs v4 41/41] vzdump: notes: document the uid template variable Thomas Lamprecht
2026-11-19 14:46 ` [PATCH cluster/guest-common/qemu-server/container/manager/docs v4 00/41] fix #7846: guest identity (uid) and cluster ID Dominik Csapak
2026-11-19 19:49 ` superseded: " Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261119103551.1666964-13-t.lamprecht@proxmox.com \
    --to=t.lamprecht@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal