From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
	by lore.proxmox.com (Postfix) with ESMTPS id 79B121FF15E
	for <inbox@lore.proxmox.com>; Tue,  3 Jun 2025 09:57:49 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id E7B3B112ED;
	Tue,  3 Jun 2025 09:56:49 +0200 (CEST)
To: pve-devel@lists.proxmox.com
Date: Tue,  3 Jun 2025 09:55:43 +0200
In-Reply-To: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>
References: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>
MIME-Version: 1.0
Message-ID: <mailman.210.1748937408.395.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: [pve-devel] [PATCH qemu-server 04/13] blockdev: add support to
 qemu_driveadd && qemu_drivedel
Content-Type: multipart/mixed; boundary="===============1303236272194480119=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>

--===============1303236272194480119==
Content-Type: message/rfc822
Content-Disposition: inline

Return-Path: <root@formationkvm1.odiso.net>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits))
	(No client certificate requested)
	by lists.proxmox.com (Postfix) with ESMTPS id 92890CB573
	for <pve-devel@lists.proxmox.com>; Tue,  3 Jun 2025 09:56:48 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id 8510310B6F
	for <pve-devel@lists.proxmox.com>; Tue,  3 Jun 2025 09:56:16 +0200 (CEST)
Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits))
	(No client certificate requested)
	by firstgate.proxmox.com (Proxmox) with ESMTPS
	for <pve-devel@lists.proxmox.com>; Tue,  3 Jun 2025 09:56:12 +0200 (CEST)
Received: from formationkvm1.odiso.net (unknown [10.11.201.57])
	by bastiontest.odiso.net (Postfix) with ESMTP id 30B2B862E44;
	Tue,  3 Jun 2025 09:56:02 +0200 (CEST)
Received: by formationkvm1.odiso.net (Postfix, from userid 0)
	id 5F5E4110DBD4; Tue,  3 Jun 2025 09:56:01 +0200 (CEST)
From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH qemu-server 04/13] blockdev: add support to qemu_driveadd && qemu_drivedel
Date: Tue,  3 Jun 2025 09:55:43 +0200
Message-Id: <20250603075558.627850-9-alexandre.derumier@groupe-cyllene.com>
X-Mailer: git-send-email 2.39.5
In-Reply-To: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>
References: <20250603075558.627850-1-alexandre.derumier@groupe-cyllene.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results:  0
	AWL                     0.108 Adjusted score from AWL reputation of From: address
	BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
	DMARC_NONE                0.1 DMARC none policy
	HEADER_FROM_DIFFERENT_DOMAINS  0.001 From and EnvelopeFrom 2nd level mail domains are different
	KAM_DMARC_NONE           0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy
	KAM_DMARC_STATUS         0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
	KAM_LAZY_DOMAIN_SECURITY      1 Sending domain does not have any anti-forgery methods
	RDNS_NONE               0.793 Delivered to internal network by a host with no rDNS
	SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
	SPF_NONE                0.001 SPF: sender does not publish an SPF Record

fixme:
- backup seem to use a tpmstate0-backup drive. Not sure how it's works, but
 I think it could be converted to blockdev too

- verify drivedel with fleecing temp image

Signed-off-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
---
 PVE/QemuServer.pm          | 93 ++++++++++++++++++++++++++++++--------
 PVE/QemuServer/Blockdev.pm |  1 +
 2 files changed, 74 insertions(+), 20 deletions(-)

diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index b15b05aa..fa072fca 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -52,7 +52,7 @@ use PVE::QMPClient;
 use PVE::QemuConfig;
 use PVE::QemuConfig::NoWrite;
 use PVE::QemuServer::Helpers qw(config_aware_timeout min_version kvm_user_version windows_version);
-use PVE::QemuServer::Blockdev qw(generate_drive_blockdev);
+use PVE::QemuServer::Blockdev qw(generate_drive_blockdev generate_throttle_group);
 use PVE::QemuServer::Cloudinit;
 use PVE::QemuServer::CGroup;
 use PVE::QemuServer::CPUConfig qw(print_cpu_device get_cpu_options get_cpu_bitness is_native_arch get_amd_sev_object get_amd_sev_type);
@@ -4130,7 +4130,7 @@ sub vm_deviceplug {
 	qemu_deviceadd($vmid, $devicefull);
 	eval { qemu_deviceaddverify($vmid, $deviceid); };
 	if (my $err = $@) {
-	    eval { qemu_drivedel($vmid, $deviceid); };
+	    eval { qemu_drivedel($vmid, $deviceid, $device); };
 	    warn $@ if $@;
 	    die $err;
         }
@@ -4159,7 +4159,7 @@ sub vm_deviceplug {
 	my $devicefull = print_drivedevice_full($storecfg, $conf, $vmid, $device, undef, $arch, $machine_type);
 	eval { qemu_deviceadd($vmid, $devicefull); };
 	if (my $err = $@) {
-	    eval { qemu_drivedel($vmid, $deviceid); };
+	    eval { qemu_drivedel($vmid, $deviceid, $device); };
 	    warn $@ if $@;
 	    die $err;
         }
@@ -4220,7 +4220,7 @@ sub vm_deviceunplug {
 
 	qemu_devicedel($vmid, $deviceid);
 	qemu_devicedelverify($vmid, $deviceid);
-	qemu_drivedel($vmid, $deviceid);
+	qemu_drivedel($vmid, $deviceid, $device);
 	qemu_iothread_del($vmid, $deviceid, $device);
     } elsif ($deviceid =~ m/^(virtioscsi|scsihw)(\d+)$/) {
 	qemu_devicedel($vmid, $deviceid);
@@ -4230,7 +4230,7 @@ sub vm_deviceunplug {
 
 	qemu_devicedel($vmid, $deviceid);
 	qemu_devicedelverify($vmid, $deviceid);
-	qemu_drivedel($vmid, $deviceid);
+	qemu_drivedel($vmid, $deviceid, $device);
 	qemu_deletescsihw($conf, $vmid, $deviceid);
 
 	qemu_iothread_del($vmid, "virtioscsi$device->{index}", $device)
@@ -4281,30 +4281,83 @@ sub qemu_iothread_del {
 sub qemu_driveadd {
     my ($storecfg, $vmid, $device) = @_;
 
-    my $kvmver = get_running_qemu_version($vmid);
-    my $io_uring = min_version($kvmver, 6, 0);
-    my $drive = print_drive_commandline_full($storecfg, $vmid, $device, undef, $io_uring);
-    $drive =~ s/\\/\\\\/g;
-    my $ret = PVE::QemuServer::Monitor::hmp_cmd($vmid, "drive_add auto \"$drive\"", 60);
+    my $machine_type = PVE::QemuServer::Machine::get_current_qemu_machine($vmid);
+
+    if (PVE::QemuServer::Machine::is_machine_version_at_least($machine_type, 10, 0)) {
+
+	my $drive_id = PVE::QemuServer::Drive::get_drive_id($device);
+	# always add a throttle-group, as it's mandatory for the throttle-filter root node.
+	my $throttle_group = generate_throttle_group($device);
+	mon_cmd($vmid, 'object-add', "qom-type" => "throttle-group", %$throttle_group);
+
+	# The throttle filter is the root node with a stable name attached to the device,
+	# and currently it's not possible to insert it later
+	my $blockdev = generate_drive_blockdev($storecfg, $device);
+	mon_cmd($vmid, 'blockdev-add', %$blockdev, timeout => 10 * 60);
+	return 1;
+
+    } else {
+
+	my $kvmver = get_running_qemu_version($vmid);
+	my $io_uring = min_version($kvmver, 6, 0);
+	my $drive = print_drive_commandline_full($storecfg, $vmid, $device, undef, $io_uring);
+	$drive =~ s/\\/\\\\/g;
+	my $ret = PVE::QemuServer::Monitor::hmp_cmd($vmid, "drive_add auto \"$drive\"", 60);
+
+	# If the command succeeds qemu prints: "OK"
+	return 1 if $ret =~ m/OK/s;
+
+	die "adding drive failed: $ret\n";
+    }
+}
 
-    # If the command succeeds qemu prints: "OK"
-    return 1 if $ret =~ m/OK/s;
 
-    die "adding drive failed: $ret\n";
+my sub qemu_drivedel_backingchain;
+sub qemu_drivedel_backingchain {
+    my ($fmt_node, $vmid) = @_;
+
+    qemu_drivedel_backingchain($fmt_node->{backing}, $vmid) if $fmt_node->{backing};
+
+    eval { mon_cmd($vmid, 'blockdev-del', 'node-name' => $fmt_node->{'node-name'}); };
+    my $file_node = $fmt_node->{file};
+    eval { mon_cmd($vmid, 'blockdev-del', 'node-name' => $file_node->{'node-name'}); };
 }
 
 sub qemu_drivedel {
-    my ($vmid, $deviceid) = @_;
+    my ($vmid, $deviceid, $device) = @_;
+
+    my $machine_type = PVE::QemuServer::Machine::get_current_qemu_machine($vmid);
 
-    my $ret = PVE::QemuServer::Monitor::hmp_cmd($vmid, "drive_del drive-$deviceid", 10 * 60);
-    $ret =~ s/^\s+//;
+    if (PVE::QemuServer::Machine::is_machine_version_at_least($machine_type, 10, 0)) {
+	#remove top-node
+	eval { mon_cmd($vmid, 'blockdev-del', 'node-name' => "drive-$deviceid", timeout => 10 * 60); };
+	die "deleting blockdev $deviceid failed : $@\n" if $@;
 
-    return 1 if $ret eq "";
+	eval { mon_cmd($vmid, 'object-del', id => "throttle-drive-$deviceid"); };
+	die "deleting throttle group throttle-drive-$deviceid failed : $@\n" if $@;
 
-    # NB: device not found errors mean the drive was auto-deleted and we ignore the error
-    return 1 if $ret =~ m/Device \'.*?\' not found/s;
+	#qemu auto-remove fmt && file-node without backing chain
+	#fixme: backup|fleecing temp drivedel don't pass $device currently, maybe it's enough
+        #as they don't have backing chain
+	if($device) {
+            my $storecfg = PVE::Storage::config();
+	    my $blockdev = generate_drive_blockdev($storecfg, $device);
+	    my $fmt_node = $blockdev->{file};
+	    qemu_drivedel_backingchain($fmt_node, $vmid);
+	}
+ 
+    } else {
+
+	my $ret = PVE::QemuServer::Monitor::hmp_cmd($vmid, "drive_del drive-$deviceid", 10 * 60);
+	$ret =~ s/^\s+//;
 
-    die "deleting drive $deviceid failed : $ret\n";
+	return 1 if $ret eq "";
+
+	# NB: device not found errors mean the drive was auto-deleted and we ignore the error
+	return 1 if $ret =~ m/Device \'.*?\' not found/s;
+
+	die "deleting drive $deviceid failed : $ret\n";
+    }
 }
 
 sub qemu_deviceaddverify {
diff --git a/PVE/QemuServer/Blockdev.pm b/PVE/QemuServer/Blockdev.pm
index b4dd1ef5..01fef66d 100644
--- a/PVE/QemuServer/Blockdev.pm
+++ b/PVE/QemuServer/Blockdev.pm
@@ -10,6 +10,7 @@ use base qw(Exporter);
 
 our @EXPORT_OK = qw(
 generate_drive_blockdev
+generate_throttle_group
 );
 
 sub encode_base62 {
-- 
2.39.5



--===============1303236272194480119==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

--===============1303236272194480119==--