From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 6F3A31FF15F for ; Mon, 16 Dec 2024 10:13:23 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 9C12A7150; Mon, 16 Dec 2024 10:12:52 +0100 (CET) To: pve-devel@lists.proxmox.com Date: Mon, 16 Dec 2024 10:12:29 +0100 In-Reply-To: <20241216091229.3142660-1-alexandre.derumier@groupe-cyllene.com> References: <20241216091229.3142660-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Message-ID: List-Id: Proxmox VE development discussion List-Post: From: Alexandre Derumier via pve-devel Precedence: list Cc: Alexandre Derumier X-Mailman-Version: 2.1.29 X-BeenThere: pve-devel@lists.proxmox.com List-Subscribe: , List-Unsubscribe: , List-Archive: Reply-To: Proxmox VE development discussion List-Help: Subject: [pve-devel] [PATCH v3 qemu-server 11/11] qcow2: add external snapshot support Content-Type: multipart/mixed; boundary="===============4786539684125128183==" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" --===============4786539684125128183== Content-Type: message/rfc822 Content-Disposition: inline Return-Path: X-Original-To: pve-devel@lists.proxmox.com Delivered-To: pve-devel@lists.proxmox.com Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 8DE7CC30AA for ; Mon, 16 Dec 2024 10:12:50 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 5CE526EAA for ; Mon, 16 Dec 2024 10:12:48 +0100 (CET) Received: from bastiontest.odiso.net (unknown [IPv6:2a0a:1580:2000:6700::14]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Mon, 16 Dec 2024 10:12:45 +0100 (CET) Received: from formationkvm1.odiso.net (unknown [10.11.201.57]) by bastiontest.odiso.net (Postfix) with ESMTP id 57B99860F14; Mon, 16 Dec 2024 10:12:32 +0100 (CET) Received: by formationkvm1.odiso.net (Postfix, from userid 0) id 988831084C68; Mon, 16 Dec 2024 10:12:31 +0100 (CET) From: Alexandre Derumier To: pve-devel@lists.proxmox.com Subject: [PATCH v3 qemu-server 11/11] qcow2: add external snapshot support Date: Mon, 16 Dec 2024 10:12:29 +0100 Message-Id: <20241216091229.3142660-16-alexandre.derumier@groupe-cyllene.com> X-Mailer: git-send-email 2.39.5 In-Reply-To: <20241216091229.3142660-1-alexandre.derumier@groupe-cyllene.com> References: <20241216091229.3142660-1-alexandre.derumier@groupe-cyllene.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.028 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_NONE 0.1 DMARC none policy HEADER_FROM_DIFFERENT_DOMAINS 0.248 From and EnvelopeFrom 2nd level mail domains are different KAM_DMARC_NONE 0.25 DKIM has Failed or SPF has failed on the message and the domain has no DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment KAM_LAZY_DOMAIN_SECURITY 1 Sending domain does not have any anti-forgery methods RDNS_NONE 0.793 Delivered to internal network by a host with no rDNS SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_NONE 0.001 SPF: sender does not publish an SPF Record Signed-off-by: Alexandre Derumier --- PVE/QemuConfig.pm | 4 +- PVE/QemuServer.pm | 345 ++++++++++++++++++++++++++++++++++++++++++++-- 2 files changed, 335 insertions(+), 14 deletions(-) diff --git a/PVE/QemuConfig.pm b/PVE/QemuConfig.pm index ffdf9f03..c17edb46 100644 --- a/PVE/QemuConfig.pm +++ b/PVE/QemuConfig.pm @@ -375,7 +375,7 @@ sub __snapshot_create_vol_snapshot { print "snapshotting '$device' ($drive->{file})\n"; - PVE::QemuServer::qemu_volume_snapshot($vmid, $device, $storecfg, $volid, $snapname); + PVE::QemuServer::qemu_volume_snapshot($vmid, $device, $storecfg, $drive, $snapname); } sub __snapshot_delete_remove_drive { @@ -412,7 +412,7 @@ sub __snapshot_delete_vol_snapshot { my $storecfg = PVE::Storage::config(); my $volid = $drive->{file}; - PVE::QemuServer::qemu_volume_snapshot_delete($vmid, $storecfg, $volid, $snapname); + PVE::QemuServer::qemu_volume_snapshot_delete($vmid, $storecfg, $drive, $snapname); push @$unused, $volid; } diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 3a3feadf..f29a8449 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -4959,20 +4959,269 @@ sub qemu_block_resize { } sub qemu_volume_snapshot { - my ($vmid, $deviceid, $storecfg, $volid, $snap) = @_; + my ($vmid, $deviceid, $storecfg, $drive, $snap) = @_; + my $volid = $drive->{file}; my $running = check_running($vmid); - - if ($running && do_snapshots_with_qemu($storecfg, $volid, $deviceid)) { - mon_cmd($vmid, 'blockdev-snapshot-internal-sync', device => $deviceid, name => $snap); + my $do_snapshots_with_qemu = do_snapshots_with_qemu($storecfg, $volid, $deviceid) if $running; + if ($do_snapshots_with_qemu) { + if($do_snapshots_with_qemu == 2) { + my $snap_path = PVE::Storage::path($storecfg, $volid, $snap); + my $path = PVE::Storage::path($storecfg, $volid); + blockdev_current_rename($storecfg, $vmid, $deviceid, $drive, $path, $snap_path, 1); + blockdev_external_snapshot($storecfg, $vmid, $deviceid, $drive, $snap); + } else { + mon_cmd($vmid, 'blockdev-snapshot-internal-sync', device => $deviceid, name => $snap); + } } else { PVE::Storage::volume_snapshot($storecfg, $volid, $snap); } } +sub blockdev_external_snapshot { + my ($storecfg, $vmid, $deviceid, $drive, $snap) = @_; + + my $nodes = get_blockdev_nodes($vmid); + my $volid = $drive->{file}; + my $path = PVE::Storage::path($storecfg, $volid, $snap); + my $format_node = find_blockdev_node($nodes, $path, 'fmt'); + my $format_nodename = $format_node->{'node-name'}; + + #preallocate add a new current file + my $new_current_fmt_nodename = get_blockdev_nextid("fmt-$deviceid", $nodes); + my $new_current_file_nodename = get_blockdev_nextid("file-$deviceid", $nodes); + PVE::Storage::volume_snapshot($storecfg, $volid, $snap); + my $new_file_blockdev = generate_file_blockdev($storecfg, $drive, $new_current_file_nodename); + my $new_fmt_blockdev = generate_format_blockdev($storecfg, $drive, $new_current_fmt_nodename, $new_file_blockdev); + + $new_fmt_blockdev->{backing} = undef; + PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-add', %$new_fmt_blockdev); + mon_cmd($vmid, 'blockdev-snapshot', node => $format_nodename, overlay => $new_current_fmt_nodename); +} + +sub blockdev_snap_rename { + my ($storecfg, $vmid, $deviceid, $drive, $src_path, $target_path) = @_; + + my $nodes = get_blockdev_nodes($vmid); + my $volid = $drive->{file}; + + #copy the original drive param and change target file + my $target_fmt_nodename = get_blockdev_nextid("fmt-$deviceid", $nodes); + my $target_file_nodename = get_blockdev_nextid("file-$deviceid", $nodes); + + my $src_fmt_node = find_blockdev_node($nodes, $src_path, 'fmt'); + my $src_fmt_nodename = $src_fmt_node->{'node-name'}; + my $src_file_node = find_blockdev_node($nodes, $src_path, 'file'); + my $src_file_nodename = $src_file_node->{'node-name'}; + + #untaint + if ($src_path =~ m/^(\S+)$/) { + $src_path = $1; + } + if ($target_path =~ m/^(\S+)$/) { + $target_path = $1; + } + + #create a hardlink + link($src_path, $target_path); + + #add new format blockdev + my $read_only = 1; + my $target_file_blockdev = generate_file_blockdev($storecfg, $drive, $target_file_nodename); + $target_file_blockdev->{filename} = $target_path; + my $target_fmt_blockdev = generate_format_blockdev($storecfg, $drive, $target_fmt_nodename, $target_file_blockdev, $read_only); + + PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-add', %$target_fmt_blockdev); + + #reopen the parent node with different backing file + my $parent_fmt_node = find_parent_node($nodes, $src_path); + my $parent_fmt_nodename = $parent_fmt_node->{'node-name'}; + my $parent_path = $parent_fmt_node->{file}; + my $parent_file_node = find_blockdev_node($nodes, $parent_path, 'file'); + my $parent_file_nodename = $parent_file_node->{'node-name'}; + my $filenode_exist = 1; + $read_only = $parent_fmt_node->{ro}; + my $parent_fmt_blockdev = generate_format_blockdev($storecfg, $drive, $parent_fmt_nodename, $parent_file_nodename, $read_only); + $parent_fmt_blockdev->{backing} = $target_fmt_nodename; + PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-reopen', options => [$parent_fmt_blockdev]); + + #change backing-file in qcow2 metadatas + PVE::QemuServer::Monitor::mon_cmd($vmid, 'change-backing-file', device => $deviceid, 'image-node-name' => $parent_fmt_nodename, 'backing-file' => $target_path); + + # fileblockdev seem to be autoremoved, if it have been created online, but not if they are created at start with command line + eval { PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-del', 'node-name' => $src_file_nodename) }; + eval { PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-del', 'node-name' => $src_fmt_nodename) }; + + #delete old $path link + unlink($src_path); + + #rename underlay + my $storage_name = PVE::Storage::parse_volume_id($volid); + my $scfg = $storecfg->{ids}->{$storage_name}; + if ($scfg->{type} eq 'lvm') { + print"lvrename $src_path to $target_path\n"; + run_command( + ['/sbin/lvrename', $src_path, $target_path], + errmsg => "lvrename $src_path to $target_path error", + ); + } +} + +sub blockdev_current_rename { + my ($storecfg, $vmid, $deviceid, $drive, $path, $target_path, $skip_underlay) = @_; + ## rename current running image + + my $nodes = get_blockdev_nodes($vmid); + my $volid = $drive->{file}; + my $target_file_nodename = get_blockdev_nextid("file-$deviceid", $nodes); + + my $file_blockdev = generate_file_blockdev($storecfg, $drive, $target_file_nodename); + $file_blockdev->{filename} = $target_path; + + my $format_node = find_blockdev_node($nodes, $path, 'fmt'); + my $format_nodename = $format_node->{'node-name'}; + + my $file_node = find_blockdev_node($nodes, $path, 'file'); + my $file_nodename = $file_node->{'node-name'}; + + my $backingfile = $format_node->{image}->{'backing-filename'}; + my $backing_node = $backingfile ? find_blockdev_node($nodes, $backingfile, 'fmt') : undef; + + #create a hardlink + link($path, $target_path); + #add new file blockdev + PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-add', %$file_blockdev); + + #reopen the current fmt nodename with a new file nodename + my $reopen_blockdev = generate_format_blockdev($storecfg, $drive, $format_nodename, $target_file_nodename); + $reopen_blockdev->{backing} = $backing_node->{'node-name'}; + PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-reopen', options => [$reopen_blockdev]); + + # delete old file blockdev + # seem that the old file block is autoremoved after reopen if the file nodename is autogenerad with #block ? + eval { PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-del', 'node-name' => $file_nodename) }; + + unlink($path); + + #skip_underlay: lvm will be renamed later in Storage::volume_snaphot + return if $skip_underlay; + + #rename underlay + my $storage_name = PVE::Storage::parse_volume_id($volid); + my $scfg = $storecfg->{ids}->{$storage_name}; + if ($scfg->{type} eq 'lvm') { + print"lvrename $path to $target_path\n"; + run_command( + ['/sbin/lvrename', $path, $target_path], + errmsg => "lvrename $path to $target_path error", + ); + } +} + +sub blockdev_commit { + my ($storecfg, $vmid, $deviceid, $drive, $top_path, $base_path) = @_; + + my $nodes = get_blockdev_nodes($vmid); + my $volid = $drive->{file}; + + #untaint + if ($top_path =~ m/^(\S+)$/) { + $top_path = $1; + } + + print "block-commit top:$top_path to base:$base_path\n"; + my $job_id = "commit-$deviceid"; + my $jobs = {}; + + my $base_node = find_blockdev_node($nodes, $base_path, 'fmt'); + my $top_node = find_blockdev_node($nodes, $top_path, 'fmt'); + + my $options = { 'job-id' => $job_id, device => $deviceid }; + $options->{'top-node'} = $top_node->{'node-name'}; + $options->{'base-node'} = $base_node->{'node-name'}; + + + mon_cmd($vmid, 'block-commit', %$options); + $jobs->{$job_id} = {}; + + qemu_drive_mirror_monitor($vmid, undef, $jobs, 'auto', 0, 'commit'); + + #remove fmt-blockdev, file-blockdev && file + my $fmt_node = find_blockdev_node($nodes, $top_path, 'fmt'); + my $fmt_nodename = $fmt_node->{'node-name'}; + eval { PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-del', 'node-name' => $fmt_nodename) }; + + my $file_node = find_blockdev_node($nodes, $top_path, 'file'); + my $file_nodename = $file_node->{'node-name'}; + eval { PVE::QemuServer::Monitor::mon_cmd($vmid, 'blockdev-del', 'node-name' => $file_nodename) }; + + + + my $storage_name = PVE::Storage::parse_volume_id($volid); + my $scfg = $storecfg->{ids}->{$storage_name}; + if ($scfg->{type} eq 'lvm') { + print"lvremove $top_path\n"; + run_command( + ['/sbin/lvremove', '-f', $top_path], + errmsg => "lvremove $top_path", + ); + } else { + unlink($top_path); + } + +} + +sub blockdev_live_commit { + my ($storecfg, $vmid, $deviceid, $drive, $current_path, $snapshot_path) = @_; + + my $nodes = get_blockdev_nodes($vmid); + my $volid = $drive->{file}; + + #untaint + if ($current_path =~ m/^(\S+)$/) { + $current_path = $1; + } + + print "live block-commit top:$current_path to base:$snapshot_path\n"; + my $job_id = "commit-$deviceid"; + my $jobs = {}; + + my $snapshot_node = find_blockdev_node($nodes, $snapshot_path, 'fmt'); + my $snapshot_file_node = find_blockdev_node($nodes, $current_path, 'file'); + my $current_node = find_blockdev_node($nodes, $current_path, 'fmt'); + + my $opts = { 'job-id' => $job_id, + device => $deviceid, + 'base-node' => $snapshot_node->{'node-name'}, + replaces => $current_node->{'node-name'} + }; + mon_cmd($vmid, "block-commit", %$opts); + $jobs->{$job_id} = {}; + + qemu_drive_mirror_monitor ($vmid, undef, $jobs, 'complete', 0, 'commit'); + + eval { mon_cmd($vmid, 'blockdev-del', 'node-name' => $current_node->{'node-name'}) }; + + my $storage_name = PVE::Storage::parse_volume_id($volid); + my $scfg = $storecfg->{ids}->{$storage_name}; + if ($scfg->{type} eq 'lvm') { + print"lvremove $current_path\n"; + run_command( + ['/sbin/lvremove', '-f', $current_path], + errmsg => "lvremove $current_path", + ); + } else { + unlink($current_path); + } + + return; + +} + sub qemu_volume_snapshot_delete { - my ($vmid, $storecfg, $volid, $snap) = @_; + my ($vmid, $storecfg, $drive, $snap) = @_; + my $volid = $drive->{file}; my $running = check_running($vmid); my $attached_deviceid; @@ -4984,13 +5233,51 @@ sub qemu_volume_snapshot_delete { }); } - if ($attached_deviceid && do_snapshots_with_qemu($storecfg, $volid, $attached_deviceid)) { - mon_cmd( - $vmid, - 'blockdev-snapshot-delete-internal-sync', - device => $attached_deviceid, - name => $snap, - ); + my $do_snapshots_with_qemu = do_snapshots_with_qemu($storecfg, $volid, $attached_deviceid) if $running; + if ($attached_deviceid && $do_snapshots_with_qemu) { + + if ($do_snapshots_with_qemu == 2) { + + my $path = PVE::Storage::path($storecfg, $volid); + my $snapshots = PVE::Storage::volume_snapshot_info($storecfg, $volid); + + my $snappath = $snapshots->{$snap}->{file}; + return if !-e $snappath; #already deleted ? + + my $parentsnap = $snapshots->{$snap}->{parent}; + my $childsnap = $snapshots->{$snap}->{child}; + + my $parentpath = $snapshots->{$parentsnap}->{file} if $parentsnap; + my $childpath = $snapshots->{$childsnap}->{file} if $childsnap; + + #if first snapshot + if(!$parentsnap) { + print"delete first snapshot $childpath\n"; + if($childpath eq $path) { + #if child is the current (last snapshot), we need to a live active-commit + print"commit first snapshot $snappath to current $path\n"; + blockdev_live_commit($storecfg, $vmid, $attached_deviceid, $drive, $childpath, $snappath); + print" rename $snappath to $path\n"; + blockdev_current_rename($storecfg, $vmid, $attached_deviceid, $drive, $snappath, $path); + } else { + print"commit first snapshot $snappath to $childpath path\n"; + blockdev_commit($storecfg, $vmid, $attached_deviceid, $drive, $childpath, $snappath); + print" rename $snappath to $childpath\n"; + blockdev_snap_rename($storecfg, $vmid, $attached_deviceid, $drive, $snappath, $childpath); + } + } else { + #intermediate snapshot, we just need to commit the snapshot + print"commit intermediate snapshot $snappath to $parentpath\n"; + blockdev_commit($storecfg, $vmid, $attached_deviceid, $drive, $snappath, $parentpath, 'auto'); + } + } else { + mon_cmd( + $vmid, + 'blockdev-snapshot-delete-internal-sync', + device => $attached_deviceid, + name => $snap, + ); + } } else { PVE::Storage::volume_snapshot_delete( $storecfg, $volid, $snap, $attached_deviceid ? 1 : undef); @@ -8066,6 +8353,8 @@ sub do_snapshots_with_qemu { return 1; } + return 2 if $scfg->{snapext} || $scfg->{type} eq 'lvm' && $volid =~ m/\.(qcow2)/; + if ($volid =~ m/\.(qcow2|qed)$/){ return 1; } @@ -9169,6 +9458,38 @@ sub delete_ifaces_ipams_ips { } } +sub find_blockdev_node { + my ($nodes, $path, $type) = @_; + + my $found_nodeid = undef; + my $found_node = undef; + for my $nodeid (keys %$nodes) { + my $node = $nodes->{$nodeid}; + if ($nodeid =~ m/^$type-(\S+)$/ && $node->{file} eq $path ) { + $found_node = $node; + last; + } + } + die "can't found nodeid for file $path\n" if !$found_node; + return $found_node; +} + +sub find_parent_node { + my ($nodes, $backing_path) = @_; + + my $found_nodeid = undef; + my $found_node = undef; + for my $nodeid (keys %$nodes) { + my $node = $nodes->{$nodeid}; + if ($nodeid =~ m/^fmt-(\S+)$/ && $node->{backing_file} && $node->{backing_file} eq $backing_path) { + $found_node = $node; + last; + } + } + die "can't found nodeid for file $backing_path\n" if !$found_node; + return $found_node; +} + sub find_fmt_nodename_drive { my ($storecfg, $vmid, $drive, $nodes) = @_; -- 2.39.5 --===============4786539684125128183== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel --===============4786539684125128183==--