From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
by lore.proxmox.com (Postfix) with ESMTPS id C1FC81FF15E
for <inbox@lore.proxmox.com>; Tue, 3 Jun 2025 09:58:34 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id E9CFA116A1;
Tue, 3 Jun 2025 09:57:12 +0200 (CEST)
To: pve-devel@lists.proxmox.com
Date: Tue, 3 Jun 2025 09:55:53 +0200
In-Reply-To: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>
References: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>
MIME-Version: 1.0
Message-ID: <mailman.218.1748937431.395.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: [pve-devel] [PATCH qemu-server 09/13] blockdev: add blockdev_mirror
Content-Type: multipart/mixed; boundary="===============7094199694399177231=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
--===============7094199694399177231==
Content-Type: message/rfc822
Content-Disposition: inline
Return-Path: <root@formationkvm1.odiso.net>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 5F59DCB715
for <pve-devel@lists.proxmox.com>; Tue, 3 Jun 2025 09:57:11 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id A386710CB0
for <pve-devel@lists.proxmox.com>; Tue, 3 Jun 2025 09:56:20 +0200 (CEST)
Received: from bastiontest.odiso.net (unknown [185.151.190.228])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Tue, 3 Jun 2025 09:56:14 +0200 (CEST)
Received: from formationkvm1.odiso.net (unknown [10.11.201.57])
by bastiontest.odiso.net (Postfix) with ESMTP id 634CC862E58;
Tue, 3 Jun 2025 09:56:02 +0200 (CEST)
Received: by formationkvm1.odiso.net (Postfix, from userid 0)
id 91DE3110DBD3; Tue, 3 Jun 2025 09:56:01 +0200 (CEST)
From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH qemu-server 09/13] blockdev: add blockdev_mirror
Date: Tue, 3 Jun 2025 09:55:53 +0200
Message-Id: <20250603075558.627850-19-alexandre.derumier@groupe-cyllene.com>
X-Mailer: git-send-email 2.39.5
In-Reply-To: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>
References: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL 0.033 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_NONE 0.1 DMARC none policy
HEADER_FROM_DIFFERENT_DOMAINS 0.001 From and EnvelopeFrom 2nd level mail domains are different
KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods
RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_NONE 0.001 SPF: sender does not publish an SPF Record
FIXME:
port qemu bitmap drive-mirror patch to blockdev-mirror
Signed-off-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
---
PVE/QemuMigrate.pm | 2 +-
PVE/QemuServer.pm | 115 ++++++++++++++++++++++----
test/MigrationTest/QemuMigrateMock.pm | 10 ++-
3 files changed, 104 insertions(+), 23 deletions(-)
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 5a54f173..ca9f59fb 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -1149,7 +1149,7 @@ sub phase2 {
my $bitmap = $target->{bitmap};
$self->log('info', "$drive: start migration to $nbd_uri");
- PVE::QemuServer::qemu_drive_mirror($vmid, $drive, $nbd_uri, $vmid, undef, $self->{storage_migration_jobs}, 'skip', undef, $bwlimit, $bitmap);
+ PVE::QemuServer::qemu_drive_mirror($vmid, $source_drive, $nbd_uri, $vmid, undef, $self->{storage_migration_jobs}, 'skip', undef, $bwlimit, $bitmap);
}
if (PVE::QemuServer::QMPHelpers::runs_at_least_qemu_version($vmid, 8, 2)) {
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 8e48f52b..f68c5bf2 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -8153,60 +8153,139 @@ sub qemu_img_convert {
die "copy failed: $err" if $err;
}
+
sub qemu_drive_mirror {
my ($vmid, $drive, $dst_volid, $vmiddst, $is_zero_initialized, $jobs, $completion, $qga, $bwlimit, $src_bitmap) = @_;
$jobs = {} if !$jobs;
+ my $machine_type = PVE::QemuServer::Machine::get_current_qemu_machine($vmid);
+ if (PVE::QemuServer::Machine::is_machine_version_at_least($machine_type, 10, 0)) {
+ return qemu_blockdev_mirror($vmid, $drive, $dst_volid, $vmiddst, $is_zero_initialized, $jobs, $completion, $qga, $bwlimit, $src_bitmap);
+ }
+
my $qemu_target;
my $format;
- $jobs->{"drive-$drive"} = {};
+
+ my $drive_id = PVE::QemuServer::Drive::get_drive_id($drive);
+ my $deviceid = "drive-$drive_id";
+
+ $jobs->{$deviceid} = {};
if ($dst_volid =~ /^nbd:/) {
- $qemu_target = $dst_volid;
- $format = "nbd";
+ $qemu_target = $dst_volid;
+ $format = "nbd";
} else {
- my $storecfg = PVE::Storage::config();
+ my $storecfg = PVE::Storage::config();
+
+ $format = checked_volume_format($storecfg, $dst_volid);
- $format = checked_volume_format($storecfg, $dst_volid);
+ my $dst_path = PVE::Storage::path($storecfg, $dst_volid);
- my $dst_path = PVE::Storage::path($storecfg, $dst_volid);
+ $qemu_target = $is_zero_initialized ? "zeroinit:$dst_path" : $dst_path;
+ }
- $qemu_target = $is_zero_initialized ? "zeroinit:$dst_path" : $dst_path;
+ my $opts = {
+ timeout => 10,
+ device => $deviceid,
+ mode => "existing",
+ sync => "full",
+ target => $qemu_target,
+ 'auto-dismiss' => JSON::false,
+ };
+ $opts->{format} = $format if $format;
+
+ if (defined($src_bitmap)) {
+ $opts->{sync} = 'incremental';
+ $opts->{bitmap} = $src_bitmap;
+ print "drive mirror re-using dirty bitmap '$src_bitmap'\n";
+ }
+
+ if (defined($bwlimit)) {
+ $opts->{speed} = $bwlimit * 1024;
+ print "drive mirror is starting for $deviceid with bandwidth limit: ${bwlimit} KB/s\n";
+ } else {
+ print "drive mirror is starting for $deviceid\n";
+ }
+
+ # if a job already runs for this device we get an error, catch it for cleanup
+ eval { mon_cmd($vmid, "drive-mirror", %$opts); };
+ if (my $err = $@) {
+ eval { PVE::QemuServer::qemu_blockjobs_cancel($vmid, $jobs) };
+ warn "$@\n" if $@;
+ die "mirroring error: $err\n";
}
+ qemu_drive_mirror_monitor ($vmid, $vmiddst, $jobs, $completion, $qga);
+}
+
+sub qemu_blockdev_mirror {
+ my ($vmid, $drive, $dst_volid, $vmiddst, $is_zero_initialized, $jobs, $completion, $qga, $bwlimit, $src_bitmap) = @_;
+
+ my $storecfg = PVE::Storage::config();
+
+ # copy original drive config (aio,cache,discard,...)
+ my $dst_drive = dclone($drive);
+ $dst_drive->{file} = $dst_volid;
+ $dst_drive->{zeroinit} = 1 if $is_zero_initialized;
+ #improve: if target storage don't support aio uring,change it to default native
+ #and remove clone_disk_check_io_uring()
+
+ #add new block device
+ my $target_drive_blockdev = generate_drive_blockdev($storecfg, $dst_drive);
+ my $target_blockdev = $target_drive_blockdev->{file}; #can be fmt or only file for nbd
+ PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-add', %$target_blockdev);
+ my $target_nodename = $target_blockdev->{'node-name'};
+
+ # we replace the original src_fmt node in the blockdev graph.
+ # need to be defined, or if not, it'll replace the root throttle-filter
+ my $src_drive_blockdev = generate_drive_blockdev($storecfg, $drive);
+ my $src_fmt_nodename = $src_drive_blockdev->{file}->{'node-name'};
+
+
+ my $drive_id = PVE::QemuServer::Drive::get_drive_id($drive);
+ my $deviceid = "drive-$drive_id";
+
+ $jobs = {} if !$jobs;
+ my $jobid = "mirror-$deviceid";
+ $jobs->{$jobid} = {};
+
my $opts = {
+ 'job-id' => $jobid,
timeout => 10,
- device => "drive-$drive",
- mode => "existing",
+ device => $deviceid,
+ replaces => $src_fmt_nodename,
sync => "full",
- target => $qemu_target,
+ target => $target_nodename,
'auto-dismiss' => JSON::false,
};
- $opts->{format} = $format if $format;
if (defined($src_bitmap)) {
$opts->{sync} = 'incremental';
- $opts->{bitmap} = $src_bitmap;
+ $opts->{bitmap} = $src_bitmap; ##FIXME: port qemu bitmap drive-mirror patch to blockdev-mirror
print "drive mirror re-using dirty bitmap '$src_bitmap'\n";
}
if (defined($bwlimit)) {
$opts->{speed} = $bwlimit * 1024;
- print "drive mirror is starting for drive-$drive with bandwidth limit: ${bwlimit} KB/s\n";
+ print "drive mirror is starting for $deviceid with bandwidth limit: ${bwlimit} KB/s\n";
} else {
- print "drive mirror is starting for drive-$drive\n";
+ print "drive mirror is starting for $deviceid\n";
}
# if a job already runs for this device we get an error, catch it for cleanup
- eval { mon_cmd($vmid, "drive-mirror", %$opts); };
+ eval { mon_cmd($vmid, "blockdev-mirror", %$opts); };
+
if (my $err = $@) {
eval { PVE::QemuServer::qemu_blockjobs_cancel($vmid, $jobs) };
warn "$@\n" if $@;
+ eval { mon_cmd($vmid, 'blockdev-del', 'node-name' => $target_blockdev->{file}->{'node-name'}) };
+ warn "$@\n" if $@;
+ eval { mon_cmd($vmid, 'blockdev-del', 'node-name' => $target_nodename) };
+ warn "$@\n" if $@;
die "mirroring error: $err\n";
}
-
- qemu_drive_mirror_monitor ($vmid, $vmiddst, $jobs, $completion, $qga);
+ qemu_drive_mirror_monitor ($vmid, $vmiddst, $jobs, $completion, $qga, 'mirror');
}
# $completion can be either
@@ -8565,7 +8644,7 @@ sub clone_disk {
my $sparseinit = PVE::Storage::volume_has_feature($storecfg, 'sparseinit', $newvolid);
if ($use_drive_mirror) {
- qemu_drive_mirror($vmid, $src_drivename, $newvolid, $newvmid, $sparseinit, $jobs,
+ qemu_drive_mirror($vmid, $drive, $newvolid, $newvmid, $sparseinit, $jobs,
$completion, $qga, $bwlimit);
} else {
if ($dst_drivename eq 'efidisk0') {
diff --git a/test/MigrationTest/QemuMigrateMock.pm b/test/MigrationTest/QemuMigrateMock.pm
index 11c58c08..d156ff1b 100644
--- a/test/MigrationTest/QemuMigrateMock.pm
+++ b/test/MigrationTest/QemuMigrateMock.pm
@@ -132,14 +132,16 @@ $MigrationTest::Shared::qemu_server_module->mock(
qemu_drive_mirror => sub {
my ($vmid, $drive, $dst_volid, $vmiddst, $is_zero_initialized, $jobs, $completion, $qga, $bwlimit, $src_bitmap) = @_;
+ my $drive_id = "$drive->{interface}$drive->{index}";
+
die "drive_mirror with wrong vmid: '$vmid'\n" if $vmid ne $test_vmid;
- die "qemu_drive_mirror '$drive' error\n"
- if $fail_config->{qemu_drive_mirror} && $fail_config->{qemu_drive_mirror} eq $drive;
+ die "qemu_drive_mirror '$drive_id' error\n"
+ if $fail_config->{qemu_drive_mirror} && $fail_config->{qemu_drive_mirror} eq $drive_id;
my $nbd_info = decode_json(file_get_contents("${RUN_DIR_PATH}/nbd_info"));
die "target does not expect drive mirror for '$drive'\n"
- if !defined($nbd_info->{$drive});
- delete $nbd_info->{$drive};
+ if !defined($nbd_info->{$drive_id});
+ delete $nbd_info->{$drive_id};
file_set_contents("${RUN_DIR_PATH}/nbd_info", to_json($nbd_info));
},
qemu_drive_mirror_monitor => sub {
--
2.39.5
--===============7094199694399177231==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--===============7094199694399177231==--