[pve-devel] [PATCH-SERIES qemu-server/manager/container] resize endpoint improvements

[pve-devel] [PATCH-SERIES qemu-server/manager/container] resize endpoint improvements

[Fiona Ebner]

Note that qemu-server path 2/2 is a breaking API change.

qemu-server:

Fiona Ebner (2):
  fix #517: api: allow resizing qcow2 disk with snapshots
  fix #2315: api: have resize endpoint spawn a worker task

 PVE/API2/Qemu.pm | 15 +++++++++------
 1 file changed, 9 insertions(+), 6 deletions(-)


manager:

Fiona Ebner (1):
  ui: override description for resize task

 www/manager6/Utils.js | 1 +
 1 file changed, 1 insertion(+)


container:

Fiona Ebner (2):
  api: resize: drop error that can never apply
  api: resize: fork before locking

 src/PVE/API2/LXC.pm | 127 +++++++++++++++++++++++---------------------
 1 file changed, 66 insertions(+), 61 deletions(-)

-- 
2.39.2

[pve-devel] [PATCH qemu-server 1/2] fix #517: api: allow resizing qcow2 disk with snapshots

[Fiona Ebner]

Support for this was added in QEMU 5.1 by commit 7fa140abf6 ("qcow2:
Allow resize of images with internal snapshots").

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 PVE/API2/Qemu.pm | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 587bb222..a80eefc9 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4719,9 +4719,6 @@ __PACKAGE__->register_method({
 	    my (undef, undef, undef, undef, undef, undef, $format) =
 		PVE::Storage::parse_volname($storecfg, $drive->{file});
 
-	    die "can't resize volume: $disk if snapshot exists\n"
-		if %{$conf->{snapshots}} && $format eq 'qcow2';
-
 	    my $volid = $drive->{file};
 
 	    die "disk '$disk' has no associated volume\n" if !$volid;
-- 
2.39.2

[pve-devel] [PATCH qemu-server 2/2] fix #2315: api: have resize endpoint spawn a worker task

[Fiona Ebner]

Similar to the corresponding endpoint for containers. Because disks
are involved, this can be a longer running operation, as is also
indicated by the 60 seconds timeout used in qemu_block_resize() which
is called by this endpoint.

This is a breaking API change.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

I just went with calling the task 'resize' too like for containers.
Should I rather use a different name, to be able to distinguish them?

 PVE/API2/Qemu.pm | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index a80eefc9..84fc1fdb 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -4680,7 +4680,10 @@ __PACKAGE__->register_method({
 	    },
 	},
     },
-    returns => { type => 'null'},
+    returns => {
+	type => 'string',
+	description => "the task ID.",
+    },
     code => sub {
         my ($param) = @_;
 
@@ -4764,8 +4767,11 @@ __PACKAGE__->register_method({
 	    PVE::QemuConfig->write_config($vmid, $conf);
 	};
 
-        PVE::QemuConfig->lock_config($vmid, $updatefn);
-        return;
+	my $worker = sub {
+	    PVE::QemuConfig->lock_config($vmid, $updatefn);
+	};
+
+	return $rpcenv->fork_worker('resize', $vmid, $authuser, $worker);
     }});
 
 __PACKAGE__->register_method({
-- 
2.39.2

[pve-devel] [PATCH manager 1/1] ui: override description for resize task

[Fiona Ebner]

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 www/manager6/Utils.js | 1 +
 1 file changed, 1 insertion(+)

diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js
index d5dd2999..8ec00d3b 100644
--- a/www/manager6/Utils.js
+++ b/www/manager6/Utils.js
@@ -1972,6 +1972,7 @@ Ext.define('PVE.Utils', {
 	    qmstop: ['VM', gettext('Stop')],
 	    qmsuspend: ['VM', gettext('Hibernate')],
 	    qmtemplate: ['VM', gettext('Convert to template')],
+	    resize: ['VM/CT', gettext('Resize')],
 	    spiceproxy: ['VM/CT', gettext('Console') + ' (Spice)'],
 	    spiceshell: ['', gettext('Shell') + ' (Spice)'],
 	    startall: ['', gettext('Start all VMs and Containers')],
-- 
2.39.2

[pve-devel] [PATCH container 1/2] api: resize: drop error that can never apply

[Fiona Ebner]

Because container images are never qcow2.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/API2/LXC.pm | 3 ---
 1 file changed, 3 deletions(-)

diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 50c9eaf..f69b44b 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -1955,9 +1955,6 @@ __PACKAGE__->register_method({
 	    die "can't resize mount point owned by another container ($owner)"
 		if $vmid != $owner;
 
-	    die "can't resize volume: $disk if snapshot exists\n"
-		if %{$conf->{snapshots}} && $format eq 'qcow2';
-
 	    my ($storeid, $volname) = PVE::Storage::parse_volume_id($volid);
 
 	    $rpcenv->check($authuser, "/storage/$storeid", ['Datastore.AllocateSpace']);
-- 
2.39.2

[pve-devel] [PATCH container 2/2] api: resize: fork before locking

[Fiona Ebner]

making sure the early checks are done once before the expensive
forking and locking and once after locking, because the state might
have changed.

The size calculation had to be adapted a bit, to ensure the original
size is not added twice when it's a request with a leading '+'.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---

Best viewed with -w.

 src/PVE/API2/LXC.pm | 124 +++++++++++++++++++++++---------------------
 1 file changed, 66 insertions(+), 58 deletions(-)

diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index f69b44b..2d67997 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -1926,15 +1926,14 @@ __PACKAGE__->register_method({
 
 	my $sizestr = extract_param($param, 'size');
 	my $ext = ($sizestr =~ s/^\+//);
-	my $newsize = PVE::JSONSchema::parse_size($sizestr);
-	die "invalid size string" if !defined($newsize);
+	my $request_size = PVE::JSONSchema::parse_size($sizestr);
+	die "invalid size string" if !defined($request_size);
 
 	die "no options specified\n" if !scalar(keys %$param);
 
 	my $storage_cfg = cfs_read_file("storage.cfg");
 
-	my $code = sub {
-
+	my $load_and_check = sub {
 	    my $conf = PVE::LXC::Config->load_config($vmid);
 	    PVE::LXC::Config->check_lock($conf);
 
@@ -1942,8 +1941,6 @@ __PACKAGE__->register_method({
 
 	    PVE::Tools::assert_if_modified($digest, $conf->{digest});
 
-	    my $running = PVE::LXC::check_running($vmid);
-
 	    my $disk = $param->{disk};
 	    my $mp = PVE::LXC::Config->parse_volume($disk, $conf->{$disk});
 
@@ -1965,66 +1962,77 @@ __PACKAGE__->register_method({
 
 	    die "Could not determine current size of volume '$volid'\n" if !defined($size);
 
-	    $newsize += $size if $ext;
+	    my $newsize = $ext ? $size + $request_size : $request_size;
 	    $newsize = int($newsize);
 
 	    die "unable to shrink disk size\n" if $newsize < $size;
 
 	    die "disk is already at specified size\n" if $size == $newsize;
 
-	    PVE::Cluster::log_msg('info', $authuser, "update CT $vmid: resize --disk $disk --size $sizestr");
-	    my $realcmd = sub {
-		# Note: PVE::Storage::volume_resize doesn't do anything if $running=1, so
-		# we pass 0 here (parameter only makes sense for qemu)
-		PVE::Storage::volume_resize($storage_cfg, $volid, $newsize, 0);
-
-		$mp->{size} = $newsize;
-		$conf->{$disk} = PVE::LXC::Config->print_ct_mountpoint($mp, $disk eq 'rootfs');
-
-		PVE::LXC::Config->write_config($vmid, $conf);
-
-		if ($format eq 'raw') {
-		    # we need to ensure that the volume is mapped, if not needed this is a NOP
-		    my $path = PVE::Storage::map_volume($storage_cfg, $volid);
-		    $path = PVE::Storage::path($storage_cfg, $volid) if !defined($path);
-		    if ($running) {
-
-			$mp->{mp} = '/';
-			my $use_loopdev = (PVE::LXC::mountpoint_mount_path($mp, $storage_cfg))[1];
-			$path = PVE::LXC::query_loopdev($path) if $use_loopdev;
-			die "internal error: CT running but mount point not attached to a loop device"
-			    if !$path;
-			PVE::Tools::run_command(['losetup', '--set-capacity', $path]) if $use_loopdev;
-
-			# In order for resize2fs to know that we need online-resizing a mountpoint needs
-			# to be visible to it in its namespace.
-			# To not interfere with the rest of the system we unshare the current mount namespace,
-			# mount over /tmp and then run resize2fs.
-
-			# interestingly we don't need to e2fsck on mounted systems...
-			my $quoted = PVE::Tools::shellquote($path);
-			my $cmd = "mount --make-rprivate / && mount $quoted /tmp && resize2fs $quoted";
-			eval {
-			    PVE::Tools::run_command(['unshare', '-m', '--', 'sh', '-c', $cmd]);
-			};
-			warn "Failed to update the container's filesystem: $@\n" if $@;
-		    } else {
-			eval {
-			    PVE::Tools::run_command(['e2fsck', '-f', '-y', $path]);
-			    PVE::Tools::run_command(['resize2fs', $path]);
-			};
-			warn "Failed to update the container's filesystem: $@\n" if $@;
-
-			# always un-map if not running, this is a NOP if not needed
-			PVE::Storage::unmap_volume($storage_cfg, $volid);
-		    }
-		}
-	    };
-
-	    return $rpcenv->fork_worker('resize', $vmid, $authuser, $realcmd);
+	    return ($conf, $disk, $mp, $volid, $format, $newsize);
 	};
 
-	return PVE::LXC::Config->lock_config($vmid, $code);;
+	my $code = sub {
+	    my ($conf, $disk, $mp, $volid, $format, $newsize) = $load_and_check->();
+
+	    my $running = PVE::LXC::check_running($vmid);
+
+	    PVE::Cluster::log_msg('info', $authuser, "update CT $vmid: resize --disk $disk --size $sizestr");
+
+	    # Note: PVE::Storage::volume_resize doesn't do anything if $running=1, so
+	    # we pass 0 here (parameter only makes sense for qemu)
+	    PVE::Storage::volume_resize($storage_cfg, $volid, $newsize, 0);
+
+	    $mp->{size} = $newsize;
+	    $conf->{$disk} = PVE::LXC::Config->print_ct_mountpoint($mp, $disk eq 'rootfs');
+
+	    PVE::LXC::Config->write_config($vmid, $conf);
+
+	    if ($format eq 'raw') {
+		# we need to ensure that the volume is mapped, if not needed this is a NOP
+		my $path = PVE::Storage::map_volume($storage_cfg, $volid);
+		$path = PVE::Storage::path($storage_cfg, $volid) if !defined($path);
+		if ($running) {
+
+		    $mp->{mp} = '/';
+		    my $use_loopdev = (PVE::LXC::mountpoint_mount_path($mp, $storage_cfg))[1];
+		    $path = PVE::LXC::query_loopdev($path) if $use_loopdev;
+		    die "internal error: CT running but mount point not attached to a loop device"
+			if !$path;
+		    PVE::Tools::run_command(['losetup', '--set-capacity', $path]) if $use_loopdev;
+
+		    # In order for resize2fs to know that we need online-resizing a mountpoint needs
+		    # to be visible to it in its namespace.
+		    # To not interfere with the rest of the system we unshare the current mount namespace,
+		    # mount over /tmp and then run resize2fs.
+
+		    # interestingly we don't need to e2fsck on mounted systems...
+		    my $quoted = PVE::Tools::shellquote($path);
+		    my $cmd = "mount --make-rprivate / && mount $quoted /tmp && resize2fs $quoted";
+		    eval {
+			PVE::Tools::run_command(['unshare', '-m', '--', 'sh', '-c', $cmd]);
+		    };
+		    warn "Failed to update the container's filesystem: $@\n" if $@;
+		} else {
+		    eval {
+			PVE::Tools::run_command(['e2fsck', '-f', '-y', $path]);
+			PVE::Tools::run_command(['resize2fs', $path]);
+		    };
+		    warn "Failed to update the container's filesystem: $@\n" if $@;
+
+		    # always un-map if not running, this is a NOP if not needed
+		    PVE::Storage::unmap_volume($storage_cfg, $volid);
+		}
+	    }
+	};
+
+	my $worker = sub {
+	    PVE::LXC::Config->lock_config($vmid, $code);;
+	};
+
+	$load_and_check->(); # early checks before forking+locking
+
+	return $rpcenv->fork_worker('resize', $vmid, $authuser, $worker);
     }});
 
 __PACKAGE__->register_method({
-- 
2.39.2

[pve-devel] applied-series: [PATCH-SERIES qemu-server/manager/container] resize endpoint improvements

[Thomas Lamprecht]

Am 30/05/2023 um 15:52 schrieb Fiona Ebner:
> Note that qemu-server path 2/2 is a breaking API change.
> 
> qemu-server:
> 
> Fiona Ebner (2):
>   fix #517: api: allow resizing qcow2 disk with snapshots
>   fix #2315: api: have resize endpoint spawn a worker task
> 
>  PVE/API2/Qemu.pm | 15 +++++++++------
>  1 file changed, 9 insertions(+), 6 deletions(-)
> 
> 
> manager:
> 
> Fiona Ebner (1):
>   ui: override description for resize task
> 
>  www/manager6/Utils.js | 1 +
>  1 file changed, 1 insertion(+)
> 
> 
> container:
> 
> Fiona Ebner (2):
>   api: resize: drop error that can never apply
>   api: resize: fork before locking
> 
>  src/PVE/API2/LXC.pm | 127 +++++++++++++++++++++++---------------------
>  1 file changed, 66 insertions(+), 61 deletions(-)
> 


applied series, thanks!