From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) by lore.proxmox.com (Postfix) with ESMTPS id 24E671FF15F for ; Mon, 16 Dec 2024 10:14:01 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 55CB4761D; Mon, 16 Dec 2024 10:13:20 +0100 (CET) To: pve-devel@lists.proxmox.com Date: Mon, 16 Dec 2024 10:12:19 +0100 In-Reply-To: <20241216091229.3142660-1-alexandre.derumier@groupe-cyllene.com> References: <20241216091229.3142660-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Alexandre Derumier via pve-devel Precedence: list Cc: Alexandre Derumier X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH v3 pve-storage 2/3] lvmplugin: add qcow2 snapshot Content-Type: multipart/mixed; boundary="===============5189342349026847023==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============5189342349026847023== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 5C36EC327D for ; Mon, 16 Dec 2024 10:13:18 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 0E5CC6E3B for ; Mon, 16 Dec 2024 10:12:46 +0100 (CET) Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 16 Dec 2024 10:12:41 +0100 (CET) Received: from formationkvm1.odiso.net (unknown [10.11.201.57]) by bastiontest.odiso.net (Postfix) with ESMTP id 212E2860940; Mon, 16 Dec 2024 10:12:32 +0100 (CET) Received: by formationkvm1.odiso.net (Postfix, from userid 0) id 622551084C6E; Mon, 16 Dec 2024 10:12:31 +0100 (CET) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Subject: [PATCH v3 pve-storage 2/3] lvmplugin: add qcow2 snapshot Date: Mon, 16 Dec 2024 10:12:19 +0100 Message-Id: <20241216091229.3142660-6-alexandre.derumier@groupe-cyllene.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241216091229.3142660-1-alexandre.derumier@groupe-cyllene.com> References: <20241216091229.3142660-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.033 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_NONE 0.1 DMARC none policy HEADER_FROM_DIFFERENT_DOMAINS 0.248 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Signed-off-by: Alexandre Derumier --- src/PVE/Storage/LVMPlugin.pm | 231 ++++++++++++++++++++++++++++++++--- 1 file changed, 213 insertions(+), 18 deletions(-) diff --git a/src/PVE/Storage/LVMPlugin.pm b/src/PVE/Storage/LVMPlugin.pm index 88fd612..1257cd3 100644 --- a/src/PVE/Storage/LVMPlugin.pm +++ b/src/PVE/Storage/LVMPlugin.pm @@ -4,6 +4,7 @@ use strict; use warnings; use IO::File; +use POSIX qw/ceil/; use PVE::Tools qw(run_command trim); use PVE::Storage::Plugin; @@ -216,6 +217,7 @@ sub type { sub plugindata { return { content => [ {images => 1, rootdir => 1}, { images => 1 }], + format => [ { raw => 1, qcow2 => 1 } , 'raw' ], }; } @@ -291,7 +293,10 @@ sub parse_volname { PVE::Storage::Plugin::parse_lvm_name($volname); if ($volname =~ m/^(vm-(\d+)-\S+)$/) { - return ('images', $1, $2, undef, undef, undef, 'raw'); + my $name = $1; + my $vmid = $2; + my $format = $volname =~ m/\.qcow2$/ ? 'qcow2' : 'raw'; + return ('images', $name, $vmid, undef, undef, undef, $format); } die "unable to parse lvm volume name '$volname'\n"; @@ -300,11 +305,13 @@ sub parse_volname { sub filesystem_path { my ($class, $scfg, $volname, $snapname) = @_; - die "lvm snapshot is not implemented"if defined($snapname); + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = + $class->parse_volname($volname); - my ($vtype, $name, $vmid) = $class->parse_volname($volname); + die "snapshot is working with qcow2 format only" if defined($snapname) && $format ne 'qcow2'; my $vg = $scfg->{vgname}; + $name = $class->get_snap_volname($volname, $snapname) if $snapname; my $path = "/dev/$vg/$name"; @@ -332,7 +339,9 @@ sub find_free_diskname { my $disk_list = [ keys %{$lvs->{$vg}} ]; - return PVE::Storage::Plugin::get_next_vm_diskname($disk_list, $storeid, $vmid, undef, $scfg); + $add_fmt_suffix = $fmt eq 'qcow2' ? 1 : undef; + + return PVE::Storage::Plugin::get_next_vm_diskname($disk_list, $storeid, $vmid, $fmt, $scfg, $add_fmt_suffix); } sub lvcreate { @@ -363,7 +372,15 @@ sub lvrename { sub alloc_image { my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_; - die "unsupported format '$fmt'" if $fmt ne 'raw'; + die "unsupported format '$fmt'" if $fmt !~ m/(raw|qcow2)/; + + $name = $class->alloc_new_image($storeid, $scfg, $vmid, $fmt, $name, $size); + $class->format_qcow2($storeid, $scfg, $name, $size) if $fmt eq 'qcow2'; + return $name; +} + +sub alloc_new_image { + my ($class, $storeid, $scfg, $vmid, $fmt, $name, $size) = @_; die "illegal name '$name' - should be 'vm-$vmid-*'\n" if $name && $name !~ m/^vm-$vmid-/; @@ -376,16 +393,45 @@ sub alloc_image { my $free = int($vgs->{$vg}->{free}); + + #add extra space for qcow2 metadatas + #without sub-allocated clusters : For 1TB storage : l2_size = disk_size × 8 / cluster_size + #with sub-allocated clusters : For 1TB storage : l2_size = disk_size × 8 / cluster_size / 16 + #4MB overhead for 1TB with extented l2 clustersize=128k + + my $qcow2_overhead = ceil($size/1024/1024/1024) * 4096; + + my $lvmsize = $size; + $lvmsize += $qcow2_overhead if $fmt eq 'qcow2'; + die "not enough free space ($free < $size)\n" if $free < $size; - $name = $class->find_free_diskname($storeid, $scfg, $vmid) + $name = $class->find_free_diskname($storeid, $scfg, $vmid, $fmt) if !$name; - lvcreate($vg, $name, $size, ["pve-vm-$vmid"]); - + my $tags = ["pve-vm-$vmid"]; + push @$tags, "\@pve-$name" if $fmt eq 'qcow2'; + lvcreate($vg, $name, $lvmsize, $tags); return $name; } +sub format_qcow2 { + my ($class, $storeid, $scfg, $name, $size, $backing_file) = @_; + + # activate volume + $class->activate_volume($storeid, $scfg, $name, undef, {}); + my $path = $class->path($scfg, $name, $storeid); + # create the qcow2 fs + my $cmd = ['/usr/bin/qemu-img', 'create']; + push @$cmd, '-b', $backing_file, '-F', 'qcow2' if $backing_file; + push @$cmd, '-f', 'qcow2', $path; + push @$cmd, "${size}K" if $size; + my $options = "extended_l2=on,"; + $options .= PVE::Storage::Plugin::preallocation_cmd_option($scfg, 'qcow2'); + push @$cmd, '-o', $options; + run_command($cmd); +} + sub free_image { my ($class, $storeid, $scfg, $volname, $isBase) = @_; @@ -536,6 +582,12 @@ sub activate_volume { my $lvm_activate_mode = 'ey'; + #activate volume && all snapshots volumes by tag + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = + $class->parse_volname($volname); + + $path = "\@pve-$name" if $format eq 'qcow2'; + my $cmd = ['/sbin/lvchange', "-a$lvm_activate_mode", $path]; run_command($cmd, errmsg => "can't activate LV '$path'"); $cmd = ['/sbin/lvchange', '--refresh', $path]; @@ -548,6 +600,10 @@ sub deactivate_volume { my $path = $class->path($scfg, $volname, $storeid, $snapname); return if ! -b $path; + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = + $class->parse_volname($volname); + $path = "\@pve-$name" if $format eq 'qcow2'; + my $cmd = ['/sbin/lvchange', '-aln', $path]; run_command($cmd, errmsg => "can't deactivate LV '$path'"); } @@ -555,15 +611,27 @@ sub deactivate_volume { sub volume_resize { my ($class, $scfg, $storeid, $volname, $size, $running) = @_; - $size = ($size/1024/1024) . "M"; + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = + $class->parse_volname($volname); + + my $lvmsize = $size / 1024; + my $qcow2_overhead = ceil($size/1024/1024/1024/1024) * 4096; + $lvmsize += $qcow2_overhead if $format eq 'qcow2'; + $lvmsize = "${lvmsize}k"; my $path = $class->path($scfg, $volname); - my $cmd = ['/sbin/lvextend', '-L', $size, $path]; + my $cmd = ['/sbin/lvextend', '-L', $lvmsize, $path]; $class->cluster_lock_storage($storeid, $scfg->{shared}, undef, sub { run_command($cmd, errmsg => "error resizing volume '$path'"); }); + if(!$running && $format eq 'qcow2') { + my $prealloc_opt = PVE::Storage::Plugin::preallocation_cmd_option($scfg, $format); + my $cmd = ['/usr/bin/qemu-img', 'resize', "--$prealloc_opt", '-f', $format, $path , $size]; + run_command($cmd, timeout => 10); + } + return 1; } @@ -585,30 +653,149 @@ sub volume_size_info { sub volume_snapshot { my ($class, $scfg, $storeid, $volname, $snap) = @_; - die "lvm snapshot is not implemented"; + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = + $class->parse_volname($volname); + + die "can't snapshot this image format\n" if $format ne 'qcow2'; + + $class->activate_volume($storeid, $scfg, $volname, undef, {}); + + my $snap_volname = $class->get_snap_volname($volname, $snap); + my $snap_path = $class->path($scfg, $volname, $storeid, $snap); + + my $size = $class->volume_size_info($scfg, $storeid, $volname, 5); + + #rename current lvm volume to snap volume + my $vg = $scfg->{vgname}; + print"rename $volname to $snap_volname\n"; + eval { lvrename($vg, $volname, $snap_volname) } ; + + + #allocate a new lvm volume + $class->alloc_new_image($storeid, $scfg, $vmid, 'qcow2', $volname, $size/1024); + eval { + $class->format_qcow2($storeid, $scfg, $volname, undef, $snap_path); + }; + + if ($@) { + eval { $class->free_image($storeid, $scfg, $volname, 0) }; + warn $@ if $@; + } +} + +sub volume_rollback_is_possible { + my ($class, $scfg, $storeid, $volname, $snap, $blockers) = @_; + + my $snap_path = $class->path($scfg, $volname, $storeid, $snap); + + my $snapshots = $class->volume_snapshot_info($scfg, $storeid, $volname); + my $parent_snap = $snapshots->{current}->{parent}; + + return 1 if !-e $snap_path || $snapshots->{$parent_snap}->{file} eq $snap_path; + die "can't rollback, '$snap' is not most recent snapshot on '$volname'\n"; + + return 1; } + sub volume_snapshot_rollback { my ($class, $scfg, $storeid, $volname, $snap) = @_; - die "lvm snapshot rollback is not implemented"; + die "can't rollback snapshot this image format\n" if $volname !~ m/\.(qcow2|qed)$/; + + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = + $class->parse_volname($volname); + + $class->activate_volume($storeid, $scfg, $volname, undef, {}); + my $size = $class->volume_size_info($scfg, $storeid, $volname, 5); + my $snap_path = $class->path($scfg, $volname, $storeid, $snap); + + #simply delete the current snapshot and recreate it + $class->free_image($storeid, $scfg, $volname, 0); + $class->alloc_new_image($storeid, $scfg, $vmid, 'qcow2', $volname, $size/1024); + $class->format_qcow2($storeid, $scfg, $volname, undef, $snap_path); + + return undef; } sub volume_snapshot_delete { - my ($class, $scfg, $storeid, $volname, $snap) = @_; + my ($class, $scfg, $storeid, $volname, $snap, $running) = @_; + + die "can't delete snapshot for this image format\n" if $volname !~ m/\.(qcow2|qed)$/; + + return 1 if $running; + + my $cmd = ""; + my $path = $class->filesystem_path($scfg, $volname); + + + my $snapshots = $class->volume_snapshot_info($scfg, $storeid, $volname); + my $snap_path = $snapshots->{$snap}->{file}; + my $snap_volname = $snapshots->{$snap}->{volname}; + return if !-e $snap_path; #already deleted ? + + my $parent_snap = $snapshots->{$snap}->{parent}; + my $child_snap = $snapshots->{$snap}->{child}; + + my $parent_path = $snapshots->{$parent_snap}->{file} if $parent_snap; + my $child_path = $snapshots->{$child_snap}->{file} if $child_snap; + my $child_volname = $snapshots->{$child_snap}->{volname} if $child_snap; + + + #if first snapshot, we merge child, and rename the snapshot to child + if(!$parent_snap) { + #we use commit here, as it's faster than rebase + #https://lists.gnu.org/archive/html/qemu-discuss/2019-08/msg00041.html + print"commit $child_path\n"; + $cmd = ['/usr/bin/qemu-img', 'commit', $child_path]; + run_command($cmd); + print"delete $child_volname\n"; + $class->free_image($storeid, $scfg, $child_volname, 0); + + print"rename $snap_volname to $child_volname\n"; + my $vg = $scfg->{vgname}; + lvrename($vg, $snap_volname, $child_volname); + } else { + print"commit $snap_path\n"; + $cmd = ['/usr/bin/qemu-img', 'commit', $snap_path]; + #if we delete an intermediate snapshot, we need to link upper snapshot to base snapshot + die "missing parentsnap snapshot to rebase child $child_path\n" if !$parent_path; + print "link $child_snap to $parent_snap\n"; + $cmd = ['/usr/bin/qemu-img', 'rebase', '-u', '-b', $parent_path, '-F', 'qcow2', '-f', 'qcow2', $child_path]; + run_command($cmd); + #delete the snapshot + $class->free_image($storeid, $scfg, $snap_volname, 0); + } - die "lvm snapshot delete is not implemented"; } sub volume_has_feature { my ($class, $scfg, $feature, $storeid, $volname, $snapname, $running) = @_; my $features = { - copy => { base => 1, current => 1}, - rename => {current => 1}, + copy => { + base => { qcow2 => 1, raw => 1}, + current => { qcow2 => 1, raw => 1}, + snap => { qcow2 => 1 }, + }, + 'rename' => { + current => { qcow2 => 1, raw => 1}, + }, + snapshot => { + current => { qcow2 => 1 }, + snap => { qcow2 => 1 }, + }, + template => { + current => { qcow2 => 1, raw => 1}, + }, +# don't allow to clone as we can't activate the base on multiple host at the same time +# clone => { +# base => { qcow2 => 1, raw => 1}, +# }, }; - my ($vtype, $name, $vmid, $basename, $basevmid, $isBase) = + + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = $class->parse_volname($volname); my $key = undef; @@ -617,7 +804,7 @@ sub volume_has_feature { }else{ $key = $isBase ? 'base' : 'current'; } - return 1 if $features->{$feature}->{$key}; + return 1 if defined($features->{$feature}->{$key}->{$format}); return undef; } @@ -738,4 +925,12 @@ sub rename_volume { return "${storeid}:${target_volname}"; } +sub get_snap_volname { + my ($class, $volname, $snapname) = @_; + + my ($vtype, $name, $vmid, $basename, $basevmid, $isBase, $format) = $class->parse_volname($volname); + $name = !$snapname || $snapname eq 'current' ? $volname : "snap-$snapname-$name"; + return $name; +} + 1; -- 2.39.5 --===============5189342349026847023== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============5189342349026847023==--