From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 03E031FF15F for ; Mon, 26 Aug 2024 13:01:30 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id BCD3E11503; Mon, 26 Aug 2024 13:01:13 +0200 (CEST) To: pve-devel@lists.proxmox.com Date: Mon, 26 Aug 2024 13:00:18 +0200 MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Alexandre Derumier via pve-devel Precedence: list Cc: Alexandre Derumier X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH SERIES storage/qemu-server/-manager] RFC : add lvmqcow2 storage support Content-Type: multipart/mixed; boundary="===============6957573126435079227==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============6957573126435079227== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id BE961C6951 for ; Mon, 26 Aug 2024 13:01:11 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 333A710C68 for ; Mon, 26 Aug 2024 13:00:41 +0200 (CEST) Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 26 Aug 2024 13:00:39 +0200 (CEST) Received: from formationkvm1.odiso.net (unknown [10.11.201.57]) by bastiontest.odiso.net (Postfix) with ESMTP id 316D6856500; Mon, 26 Aug 2024 13:00:33 +0200 (CEST) Received: by formationkvm1.odiso.net (Postfix, from userid 0) id E93D2102034B; Mon, 26 Aug 2024 13:00:31 +0200 (CEST) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Subject: [PATCH SERIES storage/qemu-server/-manager] RFC : add lvmqcow2 storage support Date: Mon, 26 Aug 2024 13:00:18 +0200 Message-Id: <20240826110030.1744732-1-alexandre.derumier@groupe-cyllene.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.007 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_NONE 0.1 DMARC none policy HEADER_FROM_DIFFERENT_DOMAINS 0.249 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record T_SCC_BODY_TEXT_LINE -0.01 - This patch series add support for a new lvmqcow2 storage format. Currently, we can't do snasphot && thin provisionning on shared block devices because lvm thin can't share his metavolume. I have a lot of onprem vmware customers where it's really blocking the proxmox migration. (and they are looking for ovirt/oracle virtualisation where it's working fine). It's possible to format a block device without filesystem with qcow2 format directly. This is used by redhat rhev/ovirt since almost 10year in their vsdm daemon. For thin provisiniong or to handle extra size of snapshot, we need to be able to resize the lvm volume dynamically. The volume is increased by chunk of 1GB by default (can be changed). Qemu implement events to sent an alert when the write usage is reaching a threshold. (Threshold is 50% of last chunk, so when vm have 500MB free) The resize is async (around 2s), so user need to choose a correct chunk size && threshold, if the storage is really fast (nvme for example, where you can write more than 500MB in 2ss) If the resize is not enough fast, the vm will pause in io-error. pvestatd is looking for this error, and try to extend again if needed and resume the vm pve-storage: Alexandre Derumier (5): add lvmqcow2 plugin vdisk_alloc: add underlay_size option add volume_underlay_resize add refresh volume add volume_underlay_shrink src/PVE/Storage.pm | 52 +++++- src/PVE/Storage/LVMQcow2Plugin.pm | 272 ++++++++++++++++++++++++++++++ src/PVE/Storage/Makefile | 3 +- src/PVE/Storage/Plugin.pm | 20 +++ 4 files changed, 344 insertions(+), 3 deletions(-) create mode 100644 src/PVE/Storage/LVMQcow2Plugin.pm qemu-server: Alexandre Derumier (6): lvmqcow2: set disk write threshold qm cli: add blockextend qmevent: call qm disk blockextend when write_threshold event is received migration: refresh remote disk size before resume qemu_img_format: lvmqcow2 is a path_storage clone: allocate && shrink lvmcow2 underlay PVE/CLI/qm.pm | 57 ++++++++++ PVE/QemuMigrate.pm | 13 +++ PVE/QemuServer.pm | 154 +++++++++++++++++++++++++- qmeventd/qmeventd.c | 27 +++++ test/MigrationTest/QemuMigrateMock.pm | 2 + 5 files changed, 251 insertions(+), 2 deletions(-) pve-manager: Alexandre Derumier (1): pvestatd: lvmqcow2 : extend disk on io-error PVE/Service/pvestatd.pm | 62 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) -- 2.39.2 --===============6957573126435079227== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============6957573126435079227==--