From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 98B7F1FF16B for ; Mon, 30 Sep 2024 13:32:18 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 029E31562C; Mon, 30 Sep 2024 13:32:37 +0200 (CEST) To: pve-devel@lists.proxmox.com Date: Mon, 30 Sep 2024 13:31:52 +0200 In-Reply-To: <20240930113153.2896648-1-alexandre.derumier@groupe-cyllene.com> References: <20240930113153.2896648-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Alexandre Derumier via pve-devel Precedence: list Cc: Alexandre Derumier X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH v2 qemu-server 1/1] implement external snapshot Content-Type: multipart/mixed; boundary="===============6921607183869474320==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============6921607183869474320== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 174CAC1BCC for ; Mon, 30 Sep 2024 13:32:36 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id ED43F15306 for ; Mon, 30 Sep 2024 13:32:05 +0200 (CEST) Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 30 Sep 2024 13:32:04 +0200 (CEST) Received: from formationkvm1.odiso.net (unknown [10.11.201.57]) by bastiontest.odiso.net (Postfix) with ESMTP id E08DA82E03D; Mon, 30 Sep 2024 13:31:56 +0200 (CEST) Received: by formationkvm1.odiso.net (Postfix, from userid 0) id 4A6041020376; Mon, 30 Sep 2024 13:31:54 +0200 (CEST) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Subject: [PATCH v2 qemu-server 1/1] implement external snapshot Date: Mon, 30 Sep 2024 13:31:52 +0200 Message-Id: <20240930113153.2896648-3-alexandre.derumier@groupe-cyllene.com> X-Mailer: git-send-email 2.39.2 In-Reply-To: <20240930113153.2896648-1-alexandre.derumier@groupe-cyllene.com> References: <20240930113153.2896648-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.038 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_NONE 0.1 DMARC none policy HEADER_FROM_DIFFERENT_DOMAINS 0.248 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Signed-off-by: Alexandre Derumier --- PVE/QemuServer.pm | 108 ++++++++++++++++++++++++++++++++++++++++------ 1 file changed, 95 insertions(+), 13 deletions(-) diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index b26da505..1523df15 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -1549,7 +1549,11 @@ sub print_drive_commandline_full { } else { if ($storeid) { $path = PVE::Storage::path($storecfg, $volid); - $format //= qemu_img_format($scfg, $volname); + if ($scfg->{snapext}) { + $format //= qemu_img_format($scfg, $path); + } else { + $format //= qemu_img_format($scfg, $volname); + } } else { $path = $volid; $format //= "raw"; @@ -4713,9 +4717,31 @@ sub qemu_volume_snapshot { my ($vmid, $deviceid, $storecfg, $volid, $snap) = @_; my $running = check_running($vmid); + my $do_snapshots_with_qemu = do_snapshots_with_qemu($storecfg, $volid, $deviceid) if $running; + if ($do_snapshots_with_qemu) { + if($do_snapshots_with_qemu == 2) { + my $snapshot_file = PVE::Storage::path($storecfg, $volid, $snap); + #allocate volume is external snapshot is a block device + my $snap_volid = undef; + if ($snapshot_file =~ m|^/dev/.+|) { + my ($storeid, $volname) = PVE::Storage::parse_volume_id($volid, 1); + my $size = PVE::Storage::volume_size_info($storecfg, $volid, 5); + #add 100M for qcow2 headers + $size = int($size/1024) + (100*1024); + my $snap_volname = $volname."-snap-$snap"; + $snap_volid = PVE::Storage::vdisk_alloc($storecfg, $storeid, $vmid, 'raw', $snap_volname, $size); + PVE::Storage::activate_volumes($storecfg, [$snap_volid]); + } + + eval { mon_cmd($vmid, 'blockdev-snapshot-sync', device => $deviceid, 'snapshot-file' => $snapshot_file, format => 'qcow2') }; - if ($running && do_snapshots_with_qemu($storecfg, $volid, $deviceid)) { - mon_cmd($vmid, 'blockdev-snapshot-internal-sync', device => $deviceid, name => $snap); + if ($@) { + PVE::Storage::vdisk_free($storecfg, $snap_volid) if $snapshot_file =~ m|^/dev/.+|; + die $@; + } + } else { + mon_cmd($vmid, 'blockdev-snapshot-internal-sync', device => $deviceid, name => $snap); + } } else { PVE::Storage::volume_snapshot($storecfg, $volid, $snap); } @@ -4735,13 +4761,52 @@ sub qemu_volume_snapshot_delete { }); } - if ($attached_deviceid && do_snapshots_with_qemu($storecfg, $volid, $attached_deviceid)) { - mon_cmd( - $vmid, - 'blockdev-snapshot-delete-internal-sync', - device => $attached_deviceid, - name => $snap, - ); + my $do_snapshots_with_qemu = do_snapshots_with_qemu($storecfg, $volid, $attached_deviceid) if $running; + if ($attached_deviceid && $do_snapshots_with_qemu) { + + if ($do_snapshots_with_qemu == 2) { + + my $snapshots = PVE::Storage::volume_snapshot_info($storecfg, $volid); + + my $currentpath = $snapshots->{current}->{file}; + my $snappath = $snapshots->{$snap}->{file}; + my $snapvolid = $snapshots->{$snap}->{volid}; + return if !$snappath; #already delete + + my $parentsnap = $snapshots->{$snap}->{parent}; + die "error: we can't find a parent for this snapshot" if !$parentsnap; + + my $parentpath = $snapshots->{$parentsnap}->{file}; + my $parentformat = $snapshots->{$parentsnap}->{'format'} if $parentsnap; + + print "block-commit top:$snappath base:$parentpath\n"; + + my $job_id = "commit-$attached_deviceid"; + my $jobs = {}; + mon_cmd( + $vmid, + 'block-commit', + 'job-id' => $job_id, + device => $attached_deviceid, + top => $snappath, + base => $parentpath, + ); + $jobs->{$job_id} = {}; + + #if we delete the current, block-job-complete to finish + my $completion = $currentpath eq $snappath ? 'complete' : 'auto'; + qemu_drive_mirror_monitor($vmid, undef, $jobs, $completion, 0, 'commit'); + #fixme. delete the disks when all jobs are ok ? + #delete the lvm volume + PVE::Storage::vdisk_free($storecfg, $snapvolid); + } else { + mon_cmd( + $vmid, + 'blockdev-snapshot-delete-internal-sync', + device => $attached_deviceid, + name => $snap, + ); + } } else { PVE::Storage::volume_snapshot_delete( $storecfg, $volid, $snap, $attached_deviceid ? 1 : undef); @@ -7776,6 +7841,8 @@ sub do_snapshots_with_qemu { return 1; } + return 2 if $scfg->{snapext}; + if ($volid =~ m/\.(qcow2|qed)$/){ return 1; } @@ -7849,8 +7916,23 @@ sub qemu_img_convert { if ($src_storeid) { PVE::Storage::activate_volumes($storecfg, [$src_volid], $snapname); my $src_scfg = PVE::Storage::storage_config($storecfg, $src_storeid); - $src_format = qemu_img_format($src_scfg, $src_volname); - $src_path = PVE::Storage::path($storecfg, $src_volid, $snapname); + if($src_scfg->{snapext}) { + my $snapshots = PVE::Storage::volume_snapshot_info($storecfg, $src_volid); + $snapname = 'current' if !$snapname; + #if we don't clone the current image + #need to use the parent if available, as it's the readonly image view + #at the time of the snapshot + my $parentsnap = $snapshots->{$snapname}->{parent}; + $snapname = $parentsnap if($parentsnap && $snapname ne 'current'); + $src_format = $snapshots->{$snapname}->{format}; + $src_path = $snapshots->{$snapname}->{file}; + $src_volid = $snapshots->{$snapname}->{volid}; + $snapname = undef; + PVE::Storage::activate_volumes($storecfg, [$src_volid], $snapname); + } else { + $src_format = qemu_img_format($src_scfg, $src_volname); + $src_path = PVE::Storage::path($storecfg, $src_volid, $snapname); + } $src_is_iscsi = ($src_path =~ m|^iscsi://|); $cachemode = 'none' if $src_scfg->{type} eq 'zfspool'; } elsif (-f $src_volid || -b $src_volid) { @@ -7920,7 +8002,7 @@ sub qemu_img_format { # FIXME: this entire function is kind of weird given that `parse_volname` # also already gives us a format? - my $is_path_storage = $scfg->{path} || $scfg->{type} eq 'esxi'; + my $is_path_storage = $scfg->{path} || $scfg->{type} eq 'esxi' || $scfg->{snapext}; if ($is_path_storage && $volname =~ m/\.($PVE::QemuServer::Drive::QEMU_FORMAT_RE)$/) { return $1; -- 2.39.2 --===============6921607183869474320== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============6921607183869474320==--