all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Jonas Theisen <j.theisen@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH qemu-server 02/15] migrate: block live migration of host CPU VMs between CPU vendors
Date: Wed, 18 Nov 2026 16:40:33 +0100	[thread overview]
Message-ID: <20261118154207.424260-3-j.theisen@proxmox.com> (raw)
In-Reply-To: <20261118154207.424260-1-j.theisen@proxmox.com>

The 'host' and 'max' CPU models pass the features of the host CPU
through to the guest, including vendor-specific ones. A guest that was
started on an AMD CPU is likely to crash after a live migration to an
Intel CPU, or vice versa, as the CPU it detected during boot changed
underneath it.

Check this before the migration, based on CPU information that each
node broadcasts to the cluster as the 'host-cpu-info' node key-value
entry, which pvestatd needs to do. Report such nodes as not allowed in
the migration precondition API, so they can be shown as such before
the migration is started, and refuse to migrate to them. Nodes that do
not broadcast the information, for example because they run an older
version, are not blocked, as it is unknown whether they are
compatible, but a warning is logged. It counts as a task warning, so
that the task status shows it, as the task log is often not read when
a migration succeeds.

The checks are defined in a list, so that further combinations of VM
CPU settings and node CPUs that cannot work, or are known to be prone
to problems, can be added easily. They are only done for running VMs,
as stopped VMs are started with a virtual CPU based on the target
node, and operate on the QEMU CPU of the running instance, so they
also apply to settings from custom CPU models. With TCG, the 'max'
model is fully emulated and therefore not affected.

Signed-off-by: Jonas Theisen <j.theisen@proxmox.com>
---
 src/PVE/API2/Qemu.pm                      |  41 ++++++-
 src/PVE/QemuMigrate.pm                    |  19 +++
 src/PVE/QemuMigrate/Helpers.pm            | 141 ++++++++++++++++++++++
 src/PVE/QemuServer/Helpers.pm             |  31 +++++
 src/test/MigrationTest/QemuMigrateMock.pm |  32 +++++
 src/test/run_qemu_migrate_tests.pl        | 114 +++++++++++++++++
 6 files changed, 377 insertions(+), 1 deletion(-)

diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index e0661738..0210dd1c 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -5508,6 +5508,27 @@ __PACKAGE__->register_method({
                                 description => 'The config key of a mapped resource',
                             },
                         },
+                        'incompatible-cpu' => {
+                            description =>
+                                "Reasons why the node's CPU cannot run the running VM"
+                                . " after a live migration.",
+                            type => 'array',
+                            optional => 1,
+                            items => {
+                                type => 'object',
+                                properties => {
+                                    type => {
+                                        type => 'string',
+                                        description => "The check that failed.",
+                                        enum => ['cpu-vendor'],
+                                    },
+                                    message => {
+                                        type => 'string',
+                                        description => "Why the node's CPU is incompatible.",
+                                    },
+                                },
+                            },
+                        },
                         'blocking-ha-resources' => {
                             description => "HA resources, which are blocking the"
                                 . " VM from being migrated to the node.",
@@ -5623,7 +5644,8 @@ __PACKAGE__->register_method({
         # try to detect errors early
         PVE::QemuConfig->check_lock($vmconf);
 
-        $res->{running} = PVE::QemuServer::check_running($vmid) ? 1 : 0;
+        my $pid = PVE::QemuServer::check_running($vmid);
+        $res->{running} = $pid ? 1 : 0;
 
         my ($local_resources, $mapped_resources, $missing_mappings_by_node) =
             PVE::QemuMigrate::Helpers::check_local_resources($vmconf, $res->{running}, 1);
@@ -5651,6 +5673,16 @@ __PACKAGE__->register_method({
         }
 
         my $nodelist = PVE::Cluster::get_nodelist();
+
+        my $cpu_issues_by_node = eval {
+            my $targets = [grep { $_ ne $localnode } $nodelist->@*];
+            my ($issues) = PVE::QemuMigrate::Helpers::check_target_cpu_compatibility(
+                $vmconf, $pid, $targets,
+            );
+            return $issues;
+        } // {};
+        warn "unable to check CPU compatibility of target nodes - $@" if $@;
+
         for my $node ($nodelist->@*) {
             next if $node eq $localnode;
 
@@ -5666,6 +5698,13 @@ __PACKAGE__->register_method({
                     $missing_mappings;
             }
 
+            my @cpu_errors =
+                grep { $_->{severity} eq 'error' } ($cpu_issues_by_node->{$node} // [])->@*;
+            if (scalar(@cpu_errors)) {
+                $res->{not_allowed_nodes}->{$node}->{'incompatible-cpu'} =
+                    [map { { type => $_->{type}, message => $_->{message} } } @cpu_errors];
+            }
+
             # extracting blocking resources for current node
             if (my $blocking_ha_resources = $blocking_ha_resources_by_node->{$node}) {
                 $res->{not_allowed_nodes}->{$node}->{'blocking-ha-resources'} =
diff --git a/src/PVE/QemuMigrate.pm b/src/PVE/QemuMigrate.pm
index 2d362140..cf01b2eb 100644
--- a/src/PVE/QemuMigrate.pm
+++ b/src/PVE/QemuMigrate.pm
@@ -17,6 +17,7 @@ use PVE::GuestHelpers qw(safe_boolean_ne safe_string_ne);
 use PVE::GuestID;
 use PVE::INotify;
 use PVE::JSONSchema;
+use PVE::RESTEnvironment qw(log_warn);
 use PVE::RPCEnvironment;
 use PVE::Replication;
 use PVE::ReplicationConfig;
@@ -315,6 +316,24 @@ sub prepare {
         }
     }
 
+    # nodes of other clusters do not broadcast their CPU information to this cluster
+    if (!$self->{opts}->{remote}) {
+        my $node = $self->{node};
+        my ($cpu_issues, $unchecked_nodes) =
+            PVE::QemuMigrate::Helpers::check_target_cpu_compatibility($conf, $running, [$node]);
+
+        my @errors = grep { $_->{severity} eq 'error' } ($cpu_issues->{$node} // [])->@*;
+        if (scalar(@errors)) {
+            die "can't migrate VM to '$node' - incompatible CPU: "
+                . join('; ', map { $_->{message} } @errors) . "\n";
+        }
+
+        if (scalar($unchecked_nodes->@*)) {
+            log_warn("unable to check CPU compatibility of node '$node' - CPU information not"
+                . " available");
+        }
+    }
+
     my $vollist = PVE::QemuServer::get_vm_volumes($conf);
 
     my $storages = {};
diff --git a/src/PVE/QemuMigrate/Helpers.pm b/src/PVE/QemuMigrate/Helpers.pm
index 466517da..f4f15b96 100644
--- a/src/PVE/QemuMigrate/Helpers.pm
+++ b/src/PVE/QemuMigrate/Helpers.pm
@@ -6,11 +6,15 @@ use warnings;
 use JSON;
 
 use PVE::Cluster;
+use PVE::INotify;
 use PVE::JSONSchema qw(parse_property_string);
 use PVE::Mapping::Dir;
 use PVE::Mapping::PCI;
 use PVE::Mapping::USB;
 
+use PVE::QemuServer::CPUConfig;
+use PVE::QemuServer::CPUFlags;
+use PVE::QemuServer::Helpers;
 use PVE::QemuServer::Monitor qw(mon_cmd);
 use PVE::QemuServer::Virtiofs;
 
@@ -113,6 +117,143 @@ sub check_local_resources {
     return wantarray ? (\@loc_res, $mapped_res, $missing_mappings_by_node) : \@loc_res;
 }
 
+# Returns the CPU information the node broadcasts, see get_host_cpu_info() in
+# PVE::QemuServer::Helpers. Information the node does not broadcast is missing.
+my sub get_node_cpu_info {
+    my ($node) = @_;
+
+    my $raw = PVE::Cluster::get_node_kv('host-cpu-info', $node)->{$node};
+    my $info = defined($raw) ? eval { decode_json($raw) } : undef;
+
+    return ref($info) eq 'HASH' ? $info : {};
+}
+
+# Parses a QEMU '-cpu' argument into the CPU model and its properties. Flags given without a value
+# are stored as 'on' or 'off'.
+my sub parse_qemu_cpu_string {
+    my ($cpu_str) = @_;
+
+    my ($model, @options) = split(/,/, $cpu_str);
+
+    my $props = {};
+    for my $option (@options) {
+        my ($op, $name, $value) = $option =~ m/^([+-]?)([^=]+)(?:=(.*))?$/ or next;
+        # QEMU accepts both underscores and dashes in property names, and some aliases
+        $name =~ s/_/-/g;
+        $name = PVE::QemuServer::CPUFlags::normalize_cpu_flag($name);
+        $value = $op eq '-' ? 'off' : 'on' if !defined($value);
+        $props->{$name} = $value;
+    }
+
+    return { model => $model, props => $props };
+}
+
+my sub is_host_cpu_model {
+    my ($cpu) = @_;
+    # with TCG, 'max' is fully emulated instead of passing through the host CPU
+    return $cpu->{model} eq 'host' || ($cpu->{model} eq 'max' && $cpu->{kvm});
+}
+
+my sub vendors_differ {
+    my ($source, $target) = @_;
+
+    my ($source_vendor, $target_vendor) = ($source->{vendor}, $target->{vendor});
+    return defined($source_vendor) && defined($target_vendor) && $source_vendor ne $target_vendor;
+}
+
+# Checks whether the CPU of a target node can run a running VM after a live migration.
+#
+# 'applies' returns true if the check is relevant for the VM's CPU, as parsed by
+# parse_qemu_cpu_string(), with 'kvm' set if the VM uses KVM. 'needs' lists the CPU information
+# of the target node the check requires, nodes lacking any of it are skipped. 'check' returns why
+# the target node is incompatible, if it is. With the 'error' severity, an incompatibility
+# prevents the migration, with 'warning' it is only reported.
+my $target_cpu_checks = [
+    {
+        type => 'cpu-vendor',
+        severity => 'error',
+        needs => ['vendor'],
+        applies => sub {
+            my ($cpu) = @_;
+            # these models pass through the host CPU, including vendor-specific features
+            return is_host_cpu_model($cpu);
+        },
+        check => sub {
+            my ($cpu, $source, $target) = @_;
+
+            return if !vendors_differ($source, $target);
+
+            return "CPU model '$cpu->{model}' requires the same CPU vendor as on the source"
+                . " node ($source->{vendor}), but the node has $target->{vendor}";
+        },
+    },
+];
+
+# Returns the issues of the target node, and whether all checks that can prevent the migration
+# were done.
+my sub get_target_cpu_issues {
+    my ($cpu, $checks, $source_info, $target_info) = @_;
+
+    my ($issues, $complete) = ([], 1);
+    for my $check ($checks->@*) {
+        if (grep { !defined($target_info->{$_}) } $check->{needs}->@*) {
+            $complete = 0 if $check->{severity} eq 'error';
+            next;
+        }
+
+        my $message = $check->{check}->($cpu, $source_info, $target_info) or next;
+        push $issues->@*,
+            { type => $check->{type}, severity => $check->{severity}, message => $message };
+    }
+
+    return ($issues, $complete);
+}
+
+=head3 check_target_cpu_compatibility
+
+    my ($issues_by_node, $unchecked_nodes) = check_target_cpu_compatibility($conf, $pid, $nodes);
+
+Checks whether the VM with the configuration C<$conf>, running with the process ID C<$pid>, can
+run with its current virtual CPU on each node of C<$nodes>, based on the CPU information the
+nodes broadcast. Only applies to running VMs, as stopped VMs are started with a virtual CPU
+matching the target node.
+
+Returns a hash reference with the list of issues for each node that has any, and an array
+reference with the nodes that could not be checked for issues preventing the migration, because
+they lack the required CPU information, for example as they run an older version. Each issue is
+a hash with a C<type> identifying the check, a C<severity> of C<error> if the VM cannot run on
+the node or C<warning> if it might not, and a human-readable C<message>.
+
+=cut
+
+sub check_target_cpu_compatibility {
+    my ($conf, $pid, $nodes) = @_;
+
+    return ({}, []) if !$pid;
+
+    my $cpu = parse_qemu_cpu_string(PVE::QemuServer::CPUConfig::get_cpu_from_running_vm($pid));
+    $cpu->{kvm} = $conf->{kvm} // 1;
+
+    my $checks = [grep { $_->{applies}->($cpu) } $target_cpu_checks->@*];
+    return ({}, []) if !scalar($checks->@*);
+
+    # Use what this node broadcasts, as for the target nodes. The local pvestatd might not
+    # broadcast it yet, for example if it was not restarted after an upgrade.
+    my $source_info = get_node_cpu_info(PVE::INotify::nodename());
+    $source_info = PVE::QemuServer::Helpers::get_host_cpu_info() if !scalar(keys $source_info->%*);
+
+    my ($issues_by_node, $unchecked_nodes) = ({}, []);
+    for my $node ($nodes->@*) {
+        my $target_info = get_node_cpu_info($node);
+
+        my ($issues, $complete) = get_target_cpu_issues($cpu, $checks, $source_info, $target_info);
+        $issues_by_node->{$node} = $issues if scalar($issues->@*);
+        push $unchecked_nodes->@*, $node if !$complete;
+    }
+
+    return ($issues_by_node, $unchecked_nodes);
+}
+
 sub set_migration_caps {
     my ($vmid, $savevm) = @_;
 
diff --git a/src/PVE/QemuServer/Helpers.pm b/src/PVE/QemuServer/Helpers.pm
index 5efe7736..14f822cc 100644
--- a/src/PVE/QemuServer/Helpers.pm
+++ b/src/PVE/QemuServer/Helpers.pm
@@ -359,4 +359,35 @@ sub get_host_phys_address_bits {
     return; # undef, cannot really do anything..
 }
 
+=head3 get_host_cpu_info
+
+    my $info = get_host_cpu_info();
+
+Returns the properties of this host's CPU that decide whether a running VM can be live-migrated
+to it. pvestatd broadcasts them to the cluster as the C<host-cpu-info> node key-value entry,
+encoded as JSON. The returned hash contains:
+
+=over
+
+=item C<vendor>: the CPU vendor ID, for example C<GenuineIntel> or C<AuthenticAMD>
+
+=back
+
+Properties that cannot be determined are missing. Consumers of the broadcast information must
+also handle properties that nodes running an older version do not provide yet.
+
+=cut
+
+sub get_host_cpu_info {
+    my $cpuinfo = PVE::ProcFSTools::read_cpuinfo();
+
+    # read_cpuinfo() reports values it cannot determine as 'unknown'
+    my $known = sub { defined($_[0]) && $_[0] ne 'unknown' };
+
+    my $info = {};
+    $info->{vendor} = $cpuinfo->{vendor} if $known->($cpuinfo->{vendor});
+
+    return $info;
+}
+
 1;
diff --git a/src/test/MigrationTest/QemuMigrateMock.pm b/src/test/MigrationTest/QemuMigrateMock.pm
index cf1fde38..459a29a1 100644
--- a/src/test/MigrationTest/QemuMigrateMock.pm
+++ b/src/test/MigrationTest/QemuMigrateMock.pm
@@ -31,6 +31,7 @@ my $migrate_params = decode_json(file_get_contents("${RUN_DIR_PATH}/migrate_para
 my $test_vmid = $migrate_params->{vmid};
 my $test_target = $migrate_params->{target};
 my $test_opts = $migrate_params->{opts};
+my $test_cpu_info = $migrate_params->{cpu_info} // {};
 my $current_log = '';
 
 my $vm_stop_executed = 0;
@@ -110,6 +111,10 @@ $qemu_migrate_module->mock(
         my ($self, $level, $message) = @_;
         $current_log .= "$level: $message\n";
     },
+    log_warn => sub {
+        my ($message) = @_;
+        $current_log .= "warn: $message\n";
+    },
     mon_cmd => sub {
         my ($vmid, $command, %params) = @_;
 
@@ -219,8 +224,35 @@ $qemu_server_cpuconfig_module->mock(
     },
 );
 
+$MigrationTest::Shared::qemu_migrate_helpers_module->mock(
+    check_target_cpu_compatibility => sub {
+        my ($conf, $pid, $nodes) = @_;
+        # checking requires the running CPU, which most tests do not need to specify
+        return ({}, []) if $pid && !defined($vm_status->{runningcpu});
+        return $MigrationTest::Shared::qemu_migrate_helpers_module->original(
+            'check_target_cpu_compatibility')->(@_);
+    },
+);
+
+$MigrationTest::Shared::cluster_module->mock(
+    get_node_kv => sub {
+        my ($key, $node) = @_;
+        if ($key eq 'host-cpu-info') {
+            # the source node is 'pve0', without 'source' get_host_cpu_info() is used
+            my $info =
+                $node eq 'pve0' ? $test_cpu_info->{source} : $test_cpu_info->{nodes}->{$node};
+            return {} if !$info;
+            return { $node => encode_json($info) };
+        }
+        return {};
+    },
+);
+
 my $qemu_server_helpers_module = Test::MockModule->new("PVE::QemuServer::Helpers");
 $qemu_server_helpers_module->mock(
+    get_host_cpu_info => sub {
+        return $test_cpu_info->{source} // { vendor => 'GenuineIntel' };
+    },
     vm_running_locally => sub {
         return $vm_status->{running} && !$vm_stop_executed;
     },
diff --git a/src/test/run_qemu_migrate_tests.pl b/src/test/run_qemu_migrate_tests.pl
index 9e27d10e..1084123b 100755
--- a/src/test/run_qemu_migrate_tests.pl
+++ b/src/test/run_qemu_migrate_tests.pl
@@ -530,6 +530,8 @@ my $tests = [
     # opts           - options for the migrate() call
     # target_volids  - hash of volids on the target at the beginning
     # vm_status      - hash with running, runningmachine and optionally runningcpu
+    # cpu_info       - hash with the CPU information of the 'source' node and of each
+    #                  other node in 'nodes'
     # expected_calls - hash whose keys are calls which are required
     #                  to be made if the migration gets far enough
     # expect_die     - expect the migration call to fail, and an error message
@@ -1454,6 +1456,117 @@ my $tests = [
             },
         },
     },
+    {
+        name => '1033_running_host_cpu_other_vendor',
+        target => 'pve1',
+        vmid => 1033,
+        vm_status => {
+            running => 1,
+            runningmachine => 'pc-q35-5.0+pve0',
+            runningcpu => 'host,+kvm_pv_eoi,+kvm_pv_unhalt',
+        },
+        cpu_info => {
+            source => { vendor => 'AuthenticAMD' },
+            nodes => {
+                pve1 => { vendor => 'GenuineIntel' },
+            },
+        },
+        opts => {
+            online => 1,
+        },
+        config_patch => {
+            cpu => 'host',
+        },
+        expected_calls => {},
+        expect_die => "can't migrate VM to 'pve1' - incompatible CPU: CPU model 'host' requires"
+            . " the same CPU vendor as on the source node (AuthenticAMD), but the node has"
+            . " GenuineIntel",
+        expected => {
+            source_volids => {},
+            target_volids => {},
+            vm_config => get_patched_config(1033, {
+                    cpu => 'host',
+            }),
+            vm_status => {
+                running => 1,
+                runningmachine => 'pc-q35-5.0+pve0',
+                runningcpu => 'host,+kvm_pv_eoi,+kvm_pv_unhalt',
+            },
+        },
+    },
+    {
+        name => '1033_running_host_cpu_target_without_cpu_info',
+        target => 'pve1',
+        vmid => 1033,
+        vm_status => {
+            running => 1,
+            runningmachine => 'pc-q35-5.0+pve0',
+            runningcpu => 'host,+kvm_pv_eoi,+kvm_pv_unhalt',
+        },
+        cpu_info => {
+            source => { vendor => 'AuthenticAMD' },
+        },
+        opts => {
+            online => 1,
+        },
+        config_patch => {
+            cpu => 'host',
+        },
+        expected_calls => $default_expected_calls_online,
+        expect_log => [
+            "unable to check CPU compatibility of node 'pve1' - CPU information not available",
+        ],
+        expected => {
+            source_volids => {},
+            target_volids => {},
+            vm_config => get_patched_config(1033, {
+                    cpu => 'host',
+            }),
+            vm_status => {
+                running => 1,
+                runningmachine => 'pc-q35-5.0+pve0',
+            },
+        },
+    },
+    {
+        name => '1033_running_max_cpu_tcg_other_vendor',
+        target => 'pve1',
+        vmid => 1033,
+        vm_status => {
+            running => 1,
+            runningmachine => 'pc-q35-5.0+pve0',
+            runningcpu => 'max',
+        },
+        cpu_info => {
+            source => { vendor => 'AuthenticAMD' },
+            nodes => {
+                pve1 => { vendor => 'GenuineIntel' },
+            },
+        },
+        opts => {
+            online => 1,
+        },
+        config_patch => {
+            cpu => 'max',
+            kvm => 0,
+        },
+        expected_calls => $default_expected_calls_online,
+        expected => {
+            source_volids => {},
+            target_volids => {},
+            vm_config => get_patched_config(
+                1033,
+                {
+                    cpu => 'max',
+                    kvm => 0,
+                },
+            ),
+            vm_status => {
+                running => 1,
+                runningmachine => 'pc-q35-5.0+pve0',
+            },
+        },
+    },
     {
         name => '105_replicated_to_non_replication_target',
         target => 'pve1',
@@ -1875,6 +1988,7 @@ foreach my $test (@{$tests}) {
         target => $test->{target},
         vmid => $test->{vmid},
         opts => $test->{opts},
+        cpu_info => $test->{cpu_info},
     };
 
     file_set_contents("${run_dir}/nbd_info", to_json({}));
-- 
2.47.3





  parent reply	other threads:[~2026-11-18 15:43 UTC|newest]

Thread overview: 16+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-11-18 15:40 [PATCH manager/qemu-server 00/15] check the target node's CPU before live migrating a VM Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 01/15] test: migration: allow checking for warnings and errors in the log Jonas Theisen
2026-11-18 15:40 ` Jonas Theisen [this message]
2026-11-18 15:40 ` [PATCH qemu-server 03/15] fix #6535: migrate: warn about live migration between CPU vendors Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 04/15] cpu flags: factor out parsing the broadcast CPU flags of a node Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 05/15] fix #2924: migrate: block live migration of host CPU VMs to nodes lacking flags Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 06/15] migrate: warn about live migration of host CPU VMs between CPU models Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 07/15] migrate: factor out whether the running virtual CPU is kept Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 08/15] migrate: block migration if the target cannot run the kept virtual CPU Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 09/15] helpers: add helper to query the host's TSC frequency and scaling support Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 10/15] cpu config: add hv-reenlightenment flag and set TSC frequency for it Jonas Theisen
2026-11-18 15:40 ` [PATCH qemu-server 11/15] migrate: block migration to nodes with incompatible TSC for re-enlightenment Jonas Theisen
2026-11-18 15:40 ` [PATCH manager 12/15] pvestatd: broadcast host CPU information relevant for live migration Jonas Theisen
2026-11-18 15:40 ` [PATCH manager 13/15] ui: migrate: show why the CPU of the target node is incompatible Jonas Theisen
2026-11-18 15:40 ` [PATCH manager 14/15] fix #6535: ui: migrate: show CPU warnings for the target node Jonas Theisen
2026-11-18 15:40 ` [PATCH manager 15/15] api: nodes: migrateall: skip VMs whose CPU is incompatible with the target Jonas Theisen

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20261118154207.424260-3-j.theisen@proxmox.com \
    --to=j.theisen@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal