From mboxrd@z Thu Jan  1 00:00:00 1970
Return-Path: <pve-devel-bounces@lists.proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9])
	by lore.proxmox.com (Postfix) with ESMTPS id 2A27E1FF2AA
	for <inbox@lore.proxmox.com>; Wed,  3 Jul 2024 16:24:52 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id 0716FA016;
	Wed,  3 Jul 2024 16:25:09 +0200 (CEST)
To: pve-devel@lists.proxmox.com
Date: Wed,  3 Jul 2024 16:24:47 +0200
MIME-Version: 1.0
Message-ID: <mailman.249.1720016708.331.pve-devel@lists.proxmox.com>
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Post: <mailto:pve-devel@lists.proxmox.com>
From: Alexandre Derumier via pve-devel <pve-devel@lists.proxmox.com>
Precedence: list
Cc: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
X-Mailman-Version: 2.1.29
X-BeenThere: pve-devel@lists.proxmox.com
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>, 
 <mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
Reply-To: Proxmox VE development discussion <pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
Subject: [pve-devel] [PATCH pve-storage] qcow2 format: enable subcluster
 allocation by default
Content-Type: multipart/mixed; boundary="===============2840988664926410319=="
Errors-To: pve-devel-bounces@lists.proxmox.com
Sender: "pve-devel" <pve-devel-bounces@lists.proxmox.com>

--===============2840988664926410319==
Content-Type: message/rfc822
Content-Disposition: inline

Return-Path: <root@formationkvm1.odiso.net>
X-Original-To: pve-devel@lists.proxmox.com
Delivered-To: pve-devel@lists.proxmox.com
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits))
	(No client certificate requested)
	by lists.proxmox.com (Postfix) with ESMTPS id 5FF0BBC357
	for <pve-devel@lists.proxmox.com>; Wed,  3 Jul 2024 16:25:07 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
	by firstgate.proxmox.com (Proxmox) with ESMTP id 488409FEB
	for <pve-devel@lists.proxmox.com>; Wed,  3 Jul 2024 16:25:07 +0200 (CEST)
Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
	(No client certificate requested)
	by firstgate.proxmox.com (Proxmox) with ESMTPS
	for <pve-devel@lists.proxmox.com>; Wed,  3 Jul 2024 16:25:05 +0200 (CEST)
Received: from formationkvm1.odiso.net (unknown [10.11.201.57])
	by bastiontest.odiso.net (Postfix) with ESMTP id BA309856500;
	Wed,  3 Jul 2024 16:24:56 +0200 (CEST)
Received: by formationkvm1.odiso.net (Postfix, from userid 0)
	id 06E831020B36; Wed,  3 Jul 2024 16:24:53 +0200 (CEST)
From: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
To: pve-devel@lists.proxmox.com
Subject: [PATCH pve-storage] qcow2 format: enable subcluster allocation by default
Date: Wed,  3 Jul 2024 16:24:47 +0200
Message-Id: <20240703142447.602210-1-alexandre.derumier@groupe-cyllene.com>
X-Mailer: git-send-email 2.39.2
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results:  0
	AWL                     0.124 Adjusted score from AWL reputation of From: address
	BAYES_00                 -1.9 Bayes spam probability is 0 to 1%
	DMARC_NONE                0.1 DMARC none policy
	HEADER_FROM_DIFFERENT_DOMAINS  0.001 From and EnvelopeFrom 2nd level mail domains are different
	KAM_DMARC_NONE           0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy
	KAM_DMARC_STATUS         0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
	KAM_LAZY_DOMAIN_SECURITY      1 Sending domain does not have any anti-forgery methods
	RDNS_NONE               0.793 Delivered to internal network by a host with no rDNS
	SPF_HELO_NONE           0.001 SPF: HELO does not publish an SPF Record
	SPF_NONE                0.001 SPF: sender does not publish an SPF Record
	URIBL_BLOCKED           0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked.  See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [igalia.com,plugin.pm,sched.com]

extended_l2 is an optimisation to reduce write amplification.
Currently,without it, when a vm write 4k, a full 64k cluster
need to be writen.

When enabled, the cluster is splitted in 32 subclusters.

We use a 128k cluster by default, to have 32 * 4k subclusters

https://blogs.igalia.com/berto/2020/12/03/subcluster-allocation-for-qcow2-images/
https://static.sched.com/hosted_files/kvmforum2020/d9/qcow2-subcluster-allocation.pdf

some stats for 4k randwrite benchmark

Cluster size   Without subclusters     With subclusters
16 KB          5859 IOPS               8063 IOPS
32 KB          5674 IOPS               11107 IOPS
64 KB          2527 IOPS               12731 IOPS
128 KB         1576 IOPS               11808 IOPS
256 KB         976 IOPS                 9195 IOPS
512 KB         510 IOPS                 7079 IOPS
1 MB           448 IOPS                 3306 IOPS
2 MB           262 IOPS                 2269 IOPS

Signed-off-by: Alexandre Derumier <alexandre.derumier@groupe-cyllene.com>
---
 src/PVE/Storage/Plugin.pm | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/PVE/Storage/Plugin.pm b/src/PVE/Storage/Plugin.pm
index 6444390..31b20fe 100644
--- a/src/PVE/Storage/Plugin.pm
+++ b/src/PVE/Storage/Plugin.pm
@@ -561,7 +561,7 @@ sub preallocation_cmd_option {
 	die "preallocation mode '$prealloc' not supported by format '$fmt'\n"
 	    if !$QCOW2_PREALLOCATION->{$prealloc};
 
-	return "preallocation=$prealloc";
+	return "preallocation=$prealloc,extended_l2=on,cluster_size=128k";
     } elsif ($fmt eq 'raw') {
 	$prealloc = $prealloc // 'off';
 	$prealloc = 'off' if $prealloc eq 'metadata';
-- 
2.39.2



--===============2840988664926410319==
Content-Type: text/plain; charset="us-ascii"
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

--===============2840988664926410319==--