From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
by lore.proxmox.com (Postfix) with ESMTPS id C93A81FF15F
for <inbox@lore.proxmox.com>; Mon, 26 Aug 2024 13:00:30 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 7FAE710DDE;
Mon, 26 Aug 2024 13:00:44 +0200 (CEST)
To: pve-devel@lists.proxmox.com
Date: Mon, 26 Aug 2024 13:00:20 +0200
In-Reply-To: <20240826110030.1744732-1-alexandre.derumier@groupe-cyllene.com>
References: <20240826110030.1744732-1-alexandre.derumier@groupe-cyllene.com>
MIME-Version: 1.0
Message-ID: <mailman.399.1724670042.302.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: [pve-devel] [PATCH qemu-server 1/6] lvmqcow2: set disk write
threshold
Content-Type: multipart/mixed; boundary="===============2884228040788766035=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>
--===============2884228040788766035==
Content-Type: message/rfc822
Content-Disposition: inline
Return-Path: <root@formationkvm1.odiso.net>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 5ED39C68B0
for <pve-devel@lists.proxmox.com>; Mon, 26 Aug 2024 13:00:42 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id D93F310C70
for <pve-devel@lists.proxmox.com>; Mon, 26 Aug 2024 13:00:41 +0200 (CEST)
Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Mon, 26 Aug 2024 13:00:40 +0200 (CEST)
Received: from formationkvm1.odiso.net (unknown [10.11.201.57])
by bastiontest.odiso.net (Postfix) with ESMTP id 333BD86AEC7;
Mon, 26 Aug 2024 13:00:33 +0200 (CEST)
Received: by formationkvm1.odiso.net (Postfix, from userid 0)
id F01C2102034F; Mon, 26 Aug 2024 13:00:31 +0200 (CEST)
From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH qemu-server 1/6] lvmqcow2: set disk write threshold
Date: Mon, 26 Aug 2024 13:00:20 +0200
Message-Id: <20240826110030.1744732-3-alexandre.derumier@groupe-cyllene.com>
X-Mailer: git-send-email 2.39.2
In-Reply-To: <20240826110030.1744732-1-alexandre.derumier@groupe-cyllene.com>
References: <20240826110030.1744732-1-alexandre.derumier@groupe-cyllene.com>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.006 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_NONE 0.1 DMARC none policy
HEADER_FROM_DIFFERENT_DOMAINS 0.249 From and EnvelopeFrom 2nd level mail domains are different
KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods
RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_NONE 0.001 SPF: sender does not publish an SPF Record
T_SCC_BODY_TEXT_LINE -0.01 -
on vm start or when we hotplug a disk, we add a write threshold.
The threshold is: size of the lvm - (chunk_usage_percent * chunksize)
qemu will emit an event when the vm write on an offset higher than the threshold,
and the counter is reset to 0.
(So, we'll need to set threshold again when we extend the disk)
Signed-off-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
---
PVE/QemuServer.pm | 97 +++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 97 insertions(+)
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 88c274d..3acb32e 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -4260,6 +4260,7 @@ sub vm_deviceplug {
warn $@ if $@;
die $err;
}
+ eval { set_disks_write_threshold($storecfg, $conf->{pending}, $vmid, $deviceid) };
} elsif ($deviceid =~ m/^(virtioscsi|scsihw)(\d+)$/) {
my $scsihw = defined($conf->{scsihw}) ? $conf->{scsihw} : "lsi";
my $pciaddr = print_pci_addr($deviceid, undef, $arch, $machine_type);
@@ -4289,6 +4290,7 @@ sub vm_deviceplug {
warn $@ if $@;
die $err;
}
+ eval { PVE::QemuServer::set_disks_write_threshold($storecfg, $conf->{pending}, $vmid, $deviceid); };
} elsif ($deviceid =~ m/^(net)(\d+)$/) {
return if !qemu_netdevadd($vmid, $conf, $arch, $device, $deviceid);
@@ -6069,6 +6071,7 @@ sub vm_start_nolock {
qemu_set_link_status($vmid, $opt, 0) if $nicconf->{link_down};
}
add_nets_bridge_fdb($conf, $vmid);
+ PVE::QemuServer::set_disks_write_threshold($storecfg, $conf, $vmid);
}
if (!defined($conf->{balloon}) || $conf->{balloon}) {
@@ -8829,4 +8832,98 @@ sub delete_ifaces_ipams_ips {
}
}
+sub qemu_block_set_write_threshold {
+ my ($vmid, $nodename, $threshold) = @_;
+
+ print"set threshold $nodename $threshold\n";
+
+ PVE::QemuServer::mon_cmd(
+ $vmid,
+ "block-set-write-threshold",
+ 'node-name' => $nodename,
+ 'write-threshold' => int($threshold),
+ );
+}
+
+sub get_block_info {
+ my ($vmid, $disk, $block_info) = @_;
+
+ my $res = { deviceid => undef, blocknodeid => undef, wr_highest_offset => 0};
+
+ if($disk =~ m/block/) {
+ $res->{blocknodeid} = $disk;
+ for my $id (keys %$block_info) {
+ if($block_info->{$id}->{parent}->{'node-name'} eq $disk) {
+ $res->{deviceid} = $id;
+ $res->{deviceid} =~ s/^drive-//;
+ $res->{wr_highest_offset} = $block_info->{$id}->{parent}->{stats}->{wr_highest_offset};
+ last;
+ } elsif($block_info->{$id}->{parent}->{parent}->{'node-name'} eq $disk) {
+ $res->{deviceid} = $id;
+ $res->{deviceid} =~ s/^drive-//;
+ $res->{wr_highest_offset} = $block_info->{$id}->{parent}->{parent}->{stats}->{wr_highest_offset};
+ last;
+ }
+ }
+ } else {
+ $res->{deviceid} = $disk;
+ #when backup is running, the chain of image is different
+ if($block_info->{"drive-$disk"}->{parent}->{parent}->{'node-name'} &&
+ $block_info->{"drive-$disk"}->{parent}->{parent}->{'driver-specific'}->{driver} eq 'host_device'
+ ) {
+ $res->{blocknodeid} = $block_info->{"drive-$disk"}->{parent}->{parent}->{'node-name'};
+ $res->{wr_highest_offset} = $block_info->{"drive-$disk"}->{parent}->{parent}->{stats}->{wr_highest_offset};
+ } elsif($block_info->{"drive-$disk"}->{parent}->{'node-name'} &&
+ $block_info->{"drive-$disk"}->{parent}->{'driver-specific'}->{driver} eq 'host_device'
+ ) {
+ $res->{blocknodeid} = $block_info->{"drive-$disk"}->{parent}->{'node-name'};
+ $res->{wr_highest_offset} = $block_info->{"drive-$disk"}->{parent}->{stats}->{wr_highest_offset};
+ }
+ }
+ die "can't find blocknodeid" if !$res->{blocknodeid};
+ die "can't find devicedeid" if !$res->{deviceid};
+ return $res;
+}
+
+sub compute_write_threshold {
+ my ($size, $scfg) = @_;
+
+ my $chunksize = $scfg->{chunksize} // 1024 * 1024 * 1024;
+ my $chunk_pct_extension = $scfg->{chunk_pct_extension} // 0.5;
+
+ my $threshold = $size - ($chunksize * $chunk_pct_extension);
+
+ return $threshold;
+}
+
+sub set_disks_write_threshold {
+ my ($storecfg, $conf, $vmid, $deviceid) = @_;
+
+ $deviceid =~ s/^drive-// if $deviceid;
+
+ my $blockstats = mon_cmd($vmid, "query-blockstats");
+ $blockstats = { map { $_->{device} => $_ } $blockstats->@* };
+
+ PVE::QemuConfig->foreach_volume($conf, sub {
+ my ($ds, $drive) = @_;
+
+ return if $deviceid && $ds ne $deviceid;
+
+ my $volid = $drive->{file};
+ return if !$volid;
+
+ my ($sid, $volname) = PVE::Storage::parse_volume_id($volid, 1);
+ return if !$sid;
+
+ my $scfg = PVE::Storage::storage_config($storecfg, $sid);
+ return if $scfg->{type} ne 'lvmqcow2';
+
+ my $size = PVE::Storage::volume_size_info($storecfg, $volid, 5);
+ my $threshold = compute_write_threshold($size, $scfg);
+
+ my $blockinfo = get_block_info($vmid, $ds, $blockstats);
+ qemu_block_set_write_threshold($vmid, $blockinfo->{blocknodeid}, $threshold);
+ });
+}
+
1;
--
2.39.2
--===============2884228040788766035==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--===============2884228040788766035==--