From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 5321E1FF15F for ; Mon, 26 Aug 2024 13:00:36 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id E270310E2A; Mon, 26 Aug 2024 13:00:44 +0200 (CEST) To: pve-devel@lists.proxmox.com Date: Mon, 26 Aug 2024 13:00:19 +0200 In-Reply-To: <20240826110030.1744732-1-alexandre.derumier@groupe-cyllene.com> References: <20240826110030.1744732-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Alexandre Derumier via pve-devel Precedence: list Cc: Alexandre Derumier X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH pve-storage 1/5] add lvmqcow2 plugin Content-Type: multipart/mixed; boundary="===============6086003212006118671==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============6086003212006118671== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 6EA72C68B2 for ; Mon, 26 Aug 2024 13:00:42 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 4310C10CDA for ; Mon, 26 Aug 2024 13:00:42 +0200 (CEST) Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 26 Aug 2024 13:00:40 +0200 (CEST) Received: from formationkvm1.odiso.net (unknown [10.11.201.57]) by bastiontest.odiso.net (Postfix) with ESMTP id 32F7486AEC4; Mon, 26 Aug 2024 13:00:33 +0200 (CEST) Received: by formationkvm1.odiso.net (Postfix, from userid 0) id EBFF91020364; Mon, 26 Aug 2024 13:00:31 +0200 (CEST) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Subject: [PATCH pve-storage 1/5] add lvmqcow2 plugin Date: Mon, 26 Aug 2024 13:00:19 +0200 Message-Id: <20240826110030.1744732-2-alexandre.derumier@groupe-cyllene.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240826110030.1744732-1-alexandre.derumier@groupe-cyllene.com> References: <20240826110030.1744732-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.305 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_NONE 0.1 DMARC none policy HEADER_FROM_DIFFERENT_DOMAINS 0.249 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods PROLO_LEO1 0.1 Meta Catches all Leo drug variations so far RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record T_SCC_BODY_TEXT_LINE -0.01 - URI_NOVOWEL 0.5 URI hostname has long non-vowel sequence Signed-off-by: Alexandre Derumier --- src/PVE/Storage.pm | 2 + src/PVE/Storage/LVMQcow2Plugin.pm | 218 ++++++++++++++++++++++++++++++ src/PVE/Storage/Makefile | 3 +- 3 files changed, 222 insertions(+), 1 deletion(-) create mode 100644 src/PVE/Storage/LVMQcow2Plugin.pm diff --git a/src/PVE/Storage.pm b/src/PVE/Storage.pm index 57b2038..97e77c8 100755 --- a/src/PVE/Storage.pm +++ b/src/PVE/Storage.pm @@ -40,6 +40,7 @@ use PVE::Storage::ZFSPlugin; use PVE::Storage::PBSPlugin; use PVE::Storage::BTRFSPlugin; use PVE::Storage::ESXiPlugin; +use PVE::Storage::LVMQcow2Plugin; # Storage API version. Increment it on changes in storage API interface. use constant APIVER => 10; @@ -66,6 +67,7 @@ PVE::Storage::ZFSPlugin->register(); PVE::Storage::PBSPlugin->register(); PVE::Storage::BTRFSPlugin->register(); PVE::Storage::ESXiPlugin->register(); +PVE::Storage::LVMQcow2Plugin->register(); # load third-party plugins if ( -d '/usr/share/perl5/PVE/Storage/Custom' ) { diff --git a/src/PVE/Storage/LVMQcow2Plugin.pm b/src/PVE/Storage/LVMQcow2Plugin.pm new file mode 100644 index 0000000..bdb21cc --- /dev/null +++ b/src/PVE/Storage/LVMQcow2Plugin.pm @@ -0,0 +1,218 @@ +package PVE::Storage::LVMQcow2Plugin; + +use strict; +use warnings; + +use IO::File; + +use PVE::Tools qw(run_command trim); +use PVE::Storage::Plugin; +use PVE::Storage::LVMPlugin; +use PVE::JSONSchema qw(get_standard_option); + +use base qw(PVE::Storage::LVMPlugin); + +sub type { + return 'lvmqcow2'; +} + +sub plugindata { + return { + content => [ {images => 1, rootdir => 1}, { images => 1, rootdir => 1}], + }; +} + +sub properties { + + return { + chunksize => { + description => "Size of extension chunk in Gigabytes. (default 1GB)", + type => 'integer', + default => 1, + minimum => 1, + optional => 1, + }, + chunk_pct_extension => { + description => "Percentage usage of the last chunk before trigger extent. (default 50%)", + type => 'integer', + default => 50, + minimum => 1, + optional => 1, + } + + }; +} + +sub options { + return { + vgname => { fixed => 1 }, + nodes => { optional => 1 }, + disable => { optional => 1 }, + content => { optional => 1 }, + bwlimit => { optional => 1 }, + preallocation => { optional => 1 }, + shared => { optional => 1 }, + chunksize => { optional => 1 }, + sparse => { optional => 1 }, + }; +} + + +sub default_format { + my ($scfg) = @_; + + return 'qcow2'; +} + +sub find_free_diskname { + my ($class, $storeid, $scfg, $vmid, $fmt, $add_fmt_suffix) = @_; + + my $vg = $scfg->{vgname}; + + my $lvs = PVE::Storage::LVMPlugin::lvm_list_volumes($vg); + + my $disk_list = [ keys %{$lvs->{$vg}} ]; + + return PVE::Storage::Plugin::get_next_vm_diskname($disk_list, $storeid, $vmid, 'qcow2', $scfg, 1); +} + +sub alloc_image { + my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_; + + my $sparse = $scfg->{sparse}; + + my $chunksize = $scfg->{chunksize} // 1; + $chunksize *= 1024000; + + #set minimum size to chunksize + my $lv_size = $chunksize; + + #if no provisioning, $lv_size = $size + a extra chunk for metadatas overhead + $lv_size = $size + $chunksize if !$sparse; + + die "illegal name '$name' - should be 'vm-$vmid-*'\n" + if $name && $name !~ m/^vm-$vmid-/; + + my $vgs = PVE::Storage::LVMPlugin::lvm_vgs(); + + my $vg = $scfg->{vgname}; + + die "no such volume group '$vg'\n" if !defined ($vgs->{$vg}); + + my $free = int($vgs->{$vg}->{free}); + + die "not enough free space ($free < $lv_size)\n" if $free < $lv_size; + + $name = $class->find_free_diskname($storeid, $scfg, $vmid) + if !$name; + + PVE::Storage::LVMPlugin::lvcreate($vg, $name, $lv_size, ["pve-vm-$vmid"]); + + #activate + $class->activate_volume($storeid, $scfg, $name, undef, {}); + + #format the lv with qcow2 with virtualsize (we can't preallocate for sparse image) + my $path = $class->path($scfg, $name); + my $cmd = ['/usr/bin/qemu-img', 'create']; + + my $options = "extended_l2=on,cluster_size=128k"; + #only metadata can be preallocated qcow2 on block device + $options .= ",preallocation=metadata" if !$sparse; + + push @$cmd, '-o', $options; + + push @$cmd, '-f', 'qcow2', $path, "${size}K"; + run_command($cmd, errmsg => "unable to format image"); + + #desactivate + $class->deactivate_volume($storeid, $scfg, $name, undef, {}); + + return $name; +} + +sub volume_size_info { + my ($class, $scfg, $storeid, $volname, $timeout) = @_; + + my $path = $class->filesystem_path($scfg, $volname); + + my $json = eval { + my $json = ''; + run_command(['/usr/bin/qemu-img', 'measure', $path, '-O', 'qcow2', '--output=json'], + timeout => 5, + outfunc => sub { $json .= $_[0]; }, + errfunc => sub { warn "$_[0]\n"; } + ); + from_json($json) + }; + die $@ if $@; + + my $used = $json->{required}; + my $size = $json->{'fully-allocated'}; + + return wantarray ? ($size, 'raw', $used, undef) : $size; +} + +sub volume_resize { + my ($class, $scfg, $storeid, $volname, $size, $running) = @_; + + return PVE::Storage::Plugin::volume_resize($class, $scfg, $storeid, $volname, $size, $running); +} + +sub clone_image { + my ($class, $scfg, $storeid, $volname, $vmid, $snap) = @_; + + die "not implemented"; +} + +sub create_base { + my ($class, $storeid, $scfg, $volname) = @_; + + die "not implemented"; +} + +sub volume_snapshot { + my ($class, $scfg, $storeid, $volname, $snap) = @_; + + return PVE::Storage::Plugin::volume_snapshot($class, $scfg, $storeid, $volname, $snap); +} + +sub volume_snapshot_rollback { + my ($class, $scfg, $storeid, $volname, $snap) = @_; + + return PVE::Storage::Plugin::volume_snapshot_rollback($class, $scfg, $storeid, $volname, $snap); +} + +sub volume_snapshot_delete { + my ($class, $scfg, $storeid, $volname, $snap, $running) = @_; + + return PVE::Storage::Plugin::volume_snapshot_delete($class, $scfg, $storeid, $volname, $snap, $running); +} + +sub volume_has_feature { + my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running) = @_; + + my $features = { + snapshot => { current => 1 }, + clone => { base => 1, snap => 1}, + template => { current => 1}, + copy => { base => 1, current => 1, snap => 1}, + sparseinit => { base => 1, current => 1}, + rename => {current => 1}, + }; + + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = + $class->parse_volname($volname); + + my $key = undef; + if($snapname){ + $key = 'snap'; + }else{ + $key = $isBase ? 'base' : 'current'; + } + return 1 if $features->{$feature}->{$key}; + + return undef; +} + +1; + diff --git a/src/PVE/Storage/Makefile b/src/PVE/Storage/Makefile index d5cc942..f5d40bf 100644 --- a/src/PVE/Storage/Makefile +++ b/src/PVE/Storage/Makefile @@ -14,7 +14,8 @@ SOURCES= \ PBSPlugin.pm \ BTRFSPlugin.pm \ LvmThinPlugin.pm \ - ESXiPlugin.pm + ESXiPlugin.pm \ + LVMQcow2Plugin.pm .PHONY: install install: -- 2.39.2 --===============6086003212006118671== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============6086003212006118671==--