* [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path
@ 2021-03-26 12:32 Dominic Jäger
2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Dominic Jäger @ 2021-03-26 12:32 UTC (permalink / raw)
To: pve-devel
This is required to import from LVM storages, for example
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
---
v6->v7: Feedback Fabian G
- Variables with _ instead of camelCase
- single if instead of if/else
PVE/Storage.pm | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index 18c03ec..38dffdb 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -609,22 +609,22 @@ sub path {
}
sub abs_filesystem_path {
- my ($cfg, $volid) = @_;
+ my ($cfg, $volid, $allow_blockdev) = @_;
my $path;
if (parse_volume_id ($volid, 1)) {
activate_volumes($cfg, [ $volid ]);
$path = PVE::Storage::path($cfg, $volid);
} else {
- if (-f $volid) {
+ if (-f $volid || ($allow_blockdev && -b $volid)) {
my $abspath = abs_path($volid);
if ($abspath && $abspath =~ m|^(/.+)$|) {
$path = $1; # untaint any path
}
}
}
-
- die "can't find file '$volid'\n" if !($path && -f $path);
+ die "can't find file '$volid'\n"
+ if !($path && (-f $path || ($allow_blockdev && -b $path)));
return $path;
}
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [pve-devel] [PATCH v7 qemu-server] Add API for import wizards 2021-03-26 12:32 [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Dominic Jäger @ 2021-03-26 12:32 ` Dominic Jäger 2021-03-31 15:12 ` Fabian Grünbichler 2021-03-26 12:32 ` [pve-devel] [PATCH v7 manager] gui: Add import for disk & VM Dominic Jäger 2021-04-01 13:40 ` [pve-devel] applied: [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Thomas Lamprecht 2 siblings, 1 reply; 7+ messages in thread From: Dominic Jäger @ 2021-03-26 12:32 UTC (permalink / raw) To: pve-devel Extend qm importdisk/importovf functionality to the API. Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com> --- v6->v7: Feedback by Fabian G - Introduce a regex for the import syntax <storeid>:0 - Use parameter list instead of hash for import helper - More parsing, less string magic - More VM config digest checking - Create a schema format for diskimage source mapping - Preliminarily remove some boot parameter handling - Dare to really edit schema format subs for a cleaner solution - Whitespace, variable names, ... PVE/API2/Qemu.pm | 383 ++++++++++++++++++++++++++++++++++++++++- PVE/API2/Qemu/Makefile | 2 +- PVE/API2/Qemu/OVF.pm | 68 ++++++++ PVE/QemuServer.pm | 52 +++++- PVE/QemuServer/OVF.pm | 10 +- 5 files changed, 502 insertions(+), 13 deletions(-) create mode 100644 PVE/API2/Qemu/OVF.pm diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index e95ab13..2f50f38 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -45,7 +45,6 @@ BEGIN { } } -use Data::Dumper; # fixme: remove use base qw(PVE::RESTHandler); @@ -62,6 +61,7 @@ my $resolve_cdrom_alias = sub { }; my $NEW_DISK_RE = qr!^(([^/:\s]+):)?(\d+(\.\d+)?)$!; +my $IMPORT_DISK_RE = qr!^([^/:\s]+):0$!; my $check_storage_access = sub { my ($rpcenv, $authuser, $storecfg, $vmid, $settings, $default_storage) = @_; @@ -4377,4 +4377,385 @@ __PACKAGE__->register_method({ return PVE::QemuServer::Cloudinit::dump_cloudinit_config($conf, $param->{vmid}, $param->{type}); }}); +# Raise exception if $format is not supported by $storeid +my $check_format_is_supported = sub { + my ($format, $storeid, $storecfg) = @_; + die "storage ID parameter must be passed to the sub" if !$storeid; + die "storage configuration must be passed to the sub" if !$storecfg; + + return if !$format; + + my (undef, $valid_formats) = PVE::Storage::storage_default_format($storecfg, $storeid); + my $supported = grep { $_ eq $format } @$valid_formats; + + die "format '$format' is not supported on storage $storeid" if !$supported; +}; + +# storecfg ... PVE::Storage::config() +# vmid ... target VM ID +# vmconf ... target VM configuration +# source ... source image (volid or absolute path) +# target ... hash with +# storeid => storage ID +# format => disk format (optional) +# options => hash with device options (may or may not contain <storeid>:0) +# device => device where the disk is attached (for example, scsi3) (optional) +# +# returns ... volid of the allocated disk image (e.g. local-lvm:vm-100-disk-2) +my $import_disk_image = sub { + my ($storecfg, $vmid, $vmconf, $source, $target) = @_; + my $requested_format = $target->{format}; + my $storeid = $target->{storeid}; + + die "Source parameter is undefined!" if !defined $source; + $source = PVE::Storage::abs_filesystem_path($storecfg, $source, 1); + + eval { PVE::Storage::storage_config($storecfg, $storeid) }; + die "Error while importing disk image $source: $@\n" if $@; + + my $src_size = PVE::Storage::file_size_info($source); + die "Could not get file size of $source" if !defined($src_size); + + $check_format_is_supported->($requested_format, $storeid, $storecfg); + + my $dst_format = PVE::QemuServer::resolve_dst_disk_format( + $storecfg, + $storeid, + undef, + $requested_format, + ); + my $dst_volid = PVE::Storage::vdisk_alloc( + $storecfg, + $storeid, + $vmid, + $dst_format, + undef, + $src_size / 102, + ); + + print "Importing disk image '$source' as '$dst_volid'...\n"; + eval { + local $SIG{INT} = + local $SIG{TERM} = + local $SIG{QUIT} = + local $SIG{HUP} = + local $SIG{PIPE} = sub { die "Interrupted by signal $!\n"; }; + + my $zeroinit = PVE::Storage::volume_has_feature( + $storecfg, + 'sparseinit', + $dst_volid, + ); + PVE::Storage::activate_volumes($storecfg, [$dst_volid]); + PVE::QemuServer::qemu_img_convert( + $source, + $dst_volid, + $src_size, + undef, + $zeroinit, + ); + PVE::Storage::deactivate_volumes($storecfg, [$dst_volid]); + + }; + if (my $err = $@) { + eval { PVE::Storage::vdisk_free($storecfg, $dst_volid) }; + warn "Cleanup of $dst_volid failed: $@ \n" if $@; + + die "Importing disk '$source' failed: $err\n" if $err; + } + + $target->{options}->{file} = $dst_volid; + my $options_string = PVE::QemuServer::print_drive($target->{options}); + $target->{device} = PVE::QemuConfig->add_unused_volume($vmconf, $dst_volid) + if !$target->{device}; + + $update_vm_api->( + { + vmid => $vmid, + $target->{device} => $options_string, + skiplock => 1, + digest => $vmconf->{digest}, + }, + 1, + ); + + return $dst_volid; +}; + +__PACKAGE__->register_method ({ + name => 'importdisk', + path => '{vmid}/importdisk', + method => 'POST', + proxyto => 'node', + protected => 1, + description => "Import an external disk image into a VM. The image format ". + "has to be supported by qemu-img.", + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + vmid => get_standard_option('pve-vmid', + {completion => \&PVE::QemuServer::complete_vmid}), + source => { + description => "Disk image to import. Can be a volid ". + "(local:99/imageToImport.raw) or an absolute path on the server.", + type => 'string', + }, + device => { + type => 'string', + description => "Bus/Device type of the new disk (e.g. 'ide0', ". + "'scsi2'). Will add the image as unused disk if omitted.", + enum => [PVE::QemuServer::Drive::valid_drive_names()], + optional => 1, + }, + device_options => { + type => 'string', + description => "Options to set for the new disk (e.g. 'discard=on,backup=0')", + optional => 1, + requires => 'device', + }, + storage => get_standard_option('pve-storage-id', { + description => "The storage to which the image will be imported to.", + completion => \&PVE::QemuServer::complete_storage, + }), + format => { + type => 'string', + description => 'Target format.', + enum => [ 'raw', 'qcow2', 'vmdk' ], + optional => 1, + }, + digest => get_standard_option('pve-config-digest'), + }, + }, + returns => { type => 'string'}, + code => sub { + my ($param) = @_; + my $vmid = extract_param($param, 'vmid'); + my $node = extract_param($param, 'node'); + my $source = extract_param($param, 'source'); + my $digest = extract_param($param, 'digest'); + my $device_options = extract_param($param, 'device_options'); + my $device = extract_param($param, 'device'); + my $storeid = extract_param($param, 'storage'); + + my $rpcenv = PVE::RPCEnvironment::get(); + my $authuser = $rpcenv->get_user(); + my $storecfg = PVE::Storage::config(); + PVE::Storage::storage_config($storecfg, $storeid); + + + if ($device_options) { + # $device_options may or may not contain <storeid>:0 + my $parsed = PVE::QemuServer::Drive::parse_drive($device, $device_options); + if ($parsed) { + raise_param_exc({$device_options => "Invalid import syntax"}) + if !($parsed->{file} =~ $IMPORT_DISK_RE); + } else { + my $fake = "$storeid:0,$device_options"; + $parsed = PVE::QemuServer::Drive::parse_drive($device, $fake); + } + delete $parsed->{file}; + delete $parsed->{interface}; + delete $parsed->{index}; + $device_options = $parsed; + } + + # Format can be set explicitly "--format vmdk" + # or as part of device options "--device_options discard=on,format=vmdk" + my $format = extract_param($param, 'format'); + if ($device_options) { + raise_param_exc({format => "Format already specified in device_options!"}) + if $format && $device_options->{format}; + $format = $format || $device_options->{format}; # may be undefined + } + $check_format_is_supported->($format, $storeid, $storecfg); + + # quick checks before fork + lock + my $conf = PVE::QemuConfig->load_config($vmid); + PVE::QemuConfig->check_lock($conf); + PVE::Tools::assert_if_modified($conf->{digest}, $digest); + if ($device && $conf->{$device}) { + die "Could not import because device $device is already in ". + "use in VM $vmid. Choose a different device!"; + } + + my $worker = sub { + PVE::QemuConfig->lock_config($vmid, sub { + $conf = PVE::QemuConfig->load_config($vmid); + PVE::QemuConfig->check_lock($conf); + + PVE::Tools::assert_if_modified($conf->{digest}, $digest); + PVE::QemuConfig->set_lock($vmid, 'import'); + $conf = PVE::QemuConfig->load_config($vmid); + }); + + my $target = { + node => $node, + storeid => $storeid, + }; + $target->{format} = $format; + $target->{device} = $device; + $target->{options} = $device_options; + eval { $import_disk_image->($storecfg, $vmid, $conf, $source, $target) }; + my $err = $@; + eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; + warn $@ if $@; + die $err if $err; + }; + return $rpcenv->fork_worker('importdisk', $vmid, $authuser, $worker); + }}); + +__PACKAGE__->register_method({ + name => 'importvm', + path => '{vmid}/importvm', + method => 'POST', + description => "Import a VM from existing disk images.", + protected => 1, + proxyto => 'node', + parameters => { + additionalProperties => 0, + properties => PVE::QemuServer::json_config_properties( + { + node => get_standard_option('pve-node'), + vmid => get_standard_option('pve-vmid', { completion => + \&PVE::Cluster::complete_next_vmid }), + diskimage => { + description => "\\0 delimited mapping of devices to disk images. For " . + "example, scsi0=/mnt/nfs/image1.vmdk", + type => 'string', + format => 'device-image-pair-alist', + }, + start => { + optional => 1, + type => 'boolean', + default => 0, + description => "Start VM after it was imported successfully.", + }, + }), + }, + returns => { + type => 'string', + }, + code => sub { + my ($param) = @_; + my $node = extract_param($param, 'node'); + my $vmid = extract_param($param, 'vmid'); + my $diskimages_string = extract_param($param, 'diskimage'); + my $boot = extract_param($param, 'boot'); + my $start = extract_param($param, 'start'); + + my $rpcenv = PVE::RPCEnvironment::get(); + my $authuser = $rpcenv->get_user(); + my $storecfg = PVE::Storage::config(); + + PVE::Cluster::check_cfs_quorum(); + + my $import_param = {}; + foreach my $opt (keys %$param) { + next if $opt eq 'efidisk0'; + raise_param_exc({bootdisk => "Deprecated: Use --boot order= instead"}) + if $opt eq 'bootdisk'; + + if (PVE::QemuServer::Drive::is_valid_drivename($opt)) { + my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); + if ($drive->{file} =~ $IMPORT_DISK_RE) { + $import_param->{$opt} = $drive; + delete $param->{$opt}; + } + } + } + + my $diskimages = {}; + foreach my $pair (PVE::Tools::split_list($diskimages_string)) { + my ($device, $diskimage) = split('=', $pair); + $diskimages->{$device} = $diskimage; + raise_param_exc({ + $device => "Device '$device' not marked for import, " . + "but import source '$diskimage' specified", + }) if !defined($import_param->{$device}); + PVE::Storage::abs_filesystem_path($storecfg, $diskimage, 1); + } + + foreach my $device (keys %$import_param) { + raise_param_exc({ + $device => "Device '$device' marked for import, but no source given\n", + }) if !defined($diskimages->{$device}); + } + + eval { PVE::QemuConfig->create_and_lock_config($vmid, 0, 'import') }; + die "Unable to create config for VM import: $@" if $@; + + my $worker = sub { + my $reload_conf = sub { + my ($vmid) = @_; + my $conf = PVE::QemuConfig->load_config($vmid); + return $conf if PVE::QemuConfig->has_lock($conf, 'import'); + die "import lock in VM $vmid config file missing!"; + }; + + my $conf = $reload_conf->($vmid); + $update_vm_api->( + { + %$param, + node => $node, + vmid => $vmid, + skiplock => 1, + digest => $conf->{digest}, + }, + 1 + ); + + eval { + foreach my $device (keys %$import_param) { + $conf = $reload_conf->($vmid); + my $drive = $import_param->{$device}; + my $storeid = PVE::Storage::parse_volume_id($drive->{file}); + my $imported = $import_disk_image->( + $storecfg, + $vmid, + $conf, + $diskimages->{$device}, + { + storeid => $storeid, + format => $drive->{format}, + options => $drive, + device => $device, + }, + ); + } + }; + my $err = $@; + if ($err) { + eval { PVE::QemuServer::destroy_vm($storecfg, $vmid, 1) }; + warn "Could not destroy VM $vmid: $@" if "$@"; + + die "Import failed: $err"; + } + + $conf = $reload_conf->($vmid); + if (!$boot) { + my $bootdevs = PVE::QemuServer::get_default_bootdevices($conf); + $boot = PVE::QemuServer::print_bootorder($bootdevs); + } + $update_vm_api->( + { + node => $node, + vmid => $vmid, + boot => $boot, + skiplock => 1, + digest => $conf->{digest}, + }, + 1, + ); + + eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; + warn $@ if $@; + + PVE::QemuServer::vm_start($storecfg, $vmid) if $start; + }; + + return $rpcenv->fork_worker('importvm', $vmid, $authuser, $worker); + }}); + + 1; diff --git a/PVE/API2/Qemu/Makefile b/PVE/API2/Qemu/Makefile index 5d4abda..bdd4762 100644 --- a/PVE/API2/Qemu/Makefile +++ b/PVE/API2/Qemu/Makefile @@ -1,4 +1,4 @@ -SOURCES=Agent.pm CPU.pm Machine.pm +SOURCES=Agent.pm CPU.pm Machine.pm OVF.pm .PHONY: install install: diff --git a/PVE/API2/Qemu/OVF.pm b/PVE/API2/Qemu/OVF.pm new file mode 100644 index 0000000..bd6e90b --- /dev/null +++ b/PVE/API2/Qemu/OVF.pm @@ -0,0 +1,68 @@ +package PVE::API2::Qemu::OVF; + +use strict; +use warnings; + +use PVE::RESTHandler; +use PVE::JSONSchema qw(get_standard_option); +use PVE::QemuServer::OVF; + +use base qw(PVE::RESTHandler); + +__PACKAGE__->register_method ({ + name => 'index', + path => '', + method => 'GET', + proxyto => 'node', + description => "Read an .ovf manifest.", + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + manifest => { + description => ".ovf manifest", + type => 'string', + }, + }, + }, + returns => { + description => "VM config according to .ovf manifest and digest of manifest", + type => "object", + }, + returns => { + type => 'object', + additionalProperties => 1, + properties => PVE::QemuServer::json_ovf_properties({ + name => { + type => 'string', + optional => 1, + }, + cores => { + type => 'integer', + optional => 1, + }, + memory => { + type => 'integer', + optional => 1, + }, + }), + }, + code => sub { + my ($param) = @_; + + my $manifest = $param->{manifest}; + die "$manifest: non-existent or non-regular file\n" if (! -f $manifest); + + my $parsed = PVE::QemuServer::OVF::parse_ovf($manifest, 0, 1); + my $result; + $result->{cores} = $parsed->{qm}->{cores}; + $result->{name} = $parsed->{qm}->{name}; + $result->{memory} = $parsed->{qm}->{memory}; + my $disks = $parsed->{disks}; + foreach my $disk (@$disks) { + $result->{$disk->{disk_address}} = $disk->{backing_file}; + } + return $result; + }}); + +1; \ No newline at end of file diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 1c0b5c2..131c0b6 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -300,7 +300,7 @@ my $confdesc = { optional => 1, type => 'string', description => "Lock/unlock the VM.", - enum => [qw(backup clone create migrate rollback snapshot snapshot-delete suspending suspended)], + enum => [qw(backup clone create migrate rollback snapshot snapshot-delete suspending suspended import)], }, cpulimit => { optional => 1, @@ -985,19 +985,41 @@ PVE::JSONSchema::register_format('pve-volume-id-or-qm-path', \&verify_volume_id_ sub verify_volume_id_or_qm_path { my ($volid, $noerr) = @_; - if ($volid eq 'none' || $volid eq 'cdrom' || $volid =~ m|^/|) { - return $volid; - } + return $volid eq 'none' || $volid eq 'cdrom' ? + $volid : + verify_volume_id_or_absolute_path($volid, $noerr); +} + +PVE::JSONSchema::register_format('pve-volume-id-or-absolute-path', \&verify_volume_id_or_absolute_path); +sub verify_volume_id_or_absolute_path { + my ($volid, $noerr) = @_; + + return $volid if $volid =~ m|^/|; - # if its neither 'none' nor 'cdrom' nor a path, check if its a volume-id $volid = eval { PVE::JSONSchema::check_format('pve-volume-id', $volid, '') }; if ($@) { - return if $noerr; + return undef if $noerr; die $@; } return $volid; } +PVE::JSONSchema::register_format('device-image-pair', \&verify_device_image_pair); +sub verify_device_image_pair { + my ($pair, $noerr) = @_; + + my $error = sub { + return undef if $noerr; + die $@; + }; + + my ($device, $image) = split('=', $pair); + $error->("Invalid device '$device'") if !PVE::QemuServer::Drive::is_valid_drivename($device); + $error->("Invalid image '$image'") if !verify_volume_id_or_absolute_path($image); + + return $pair; +} + my $usb_fmt = { host => { default_key => 1, @@ -2030,6 +2052,22 @@ sub json_config_properties { return $prop; } +# Properties that we can read from an OVF file +sub json_ovf_properties { + my $prop = shift; + + foreach my $device ( PVE::QemuServer::Drive::valid_drive_names()) { + $prop->{$device} = { + type => 'string', + format => 'pve-volume-id-or-absolute-path', + description => "Disk image that gets imported to $device", + optional => 1, + }; + } + + return $prop; +} + # return copy of $confdesc_cloudinit to generate documentation sub cloudinit_config_properties { @@ -6748,7 +6786,7 @@ sub qemu_img_convert { $src_path = PVE::Storage::path($storecfg, $src_volid, $snapname); $src_is_iscsi = ($src_path =~ m|^iscsi://|); $cachemode = 'none' if $src_scfg->{type} eq 'zfspool'; - } elsif (-f $src_volid) { + } elsif (-f $src_volid || -b $src_volid) { $src_path = $src_volid; if ($src_path =~ m/\.($PVE::QemuServer::Drive::QEMU_FORMAT_RE)$/) { $src_format = $1; diff --git a/PVE/QemuServer/OVF.pm b/PVE/QemuServer/OVF.pm index c76c199..48146e9 100644 --- a/PVE/QemuServer/OVF.pm +++ b/PVE/QemuServer/OVF.pm @@ -87,7 +87,7 @@ sub id_to_pve { # returns two references, $qm which holds qm.conf style key/values, and \@disks sub parse_ovf { - my ($ovf, $debug) = @_; + my ($ovf, $debug, $manifest_only) = @_; my $dom = XML::LibXML->load_xml(location => $ovf, no_blanks => 1); @@ -220,9 +220,11 @@ ovf:Item[rasd:InstanceID='%s']/rasd:ResourceType", $controller_id); die "error parsing $filepath, file seems not to exist at $backing_file_path\n"; } - my $virtual_size; - if ( !($virtual_size = PVE::Storage::file_size_info($backing_file_path)) ) { - die "error parsing $backing_file_path, size seems to be $virtual_size\n"; + my $virtual_size = undef; + if (!$manifest_only) { # Not possible if manifest is uploaded in web gui + if ( !($virtual_size = PVE::Storage::file_size_info($backing_file_path)) ) { + die "error parsing $backing_file_path: Could not get file size info: $@\n"; + } } $pve_disk = { -- 2.20.1 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pve-devel] [PATCH v7 qemu-server] Add API for import wizards 2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger @ 2021-03-31 15:12 ` Fabian Grünbichler 2021-04-01 10:19 ` Dominic Jäger 0 siblings, 1 reply; 7+ messages in thread From: Fabian Grünbichler @ 2021-03-31 15:12 UTC (permalink / raw) To: Proxmox VE development discussion this is starting to shape up nicely. as promised, I now took a stab at (roughly!) integrating this into our regular flow (see diff below): - IMPORT_DISK_RE now uses -1, as 0 actually can be mismatched by NEW_DISK_RE - the actual import happens in create_disks - only the basic checks (match of "marked for import" with "import sources") happen early on - the target storage and source volume are permission checked now - importvm is dropped in favor of calling create_vm with import_sources - update_vm_async now also supports import_sources the last two IMHO have some nice benefits: - we can now easily mix and match importing and newly allocating blank disks in a single create call (allowing us to use a single GUI wizard as well if we want) - create_vm and importvm don't have to duplicate all the surrounding logic (or have strange contortions to make one call the other) - we could likely drop the separate import_disk API call, and let the `importdisk` CLI command prepare parameters for a regular VM config update I'm sure I've missed some corner cases, as I've only tested create_vm with importing, and not the other newly exposed APIs. stat shows how much boilerplace/duplication this removes, although there is probably even more potential here since `create_vm` partly duplicates the `update_vm_api` sub that ends up calling `create_disks`: PVE/API2/Qemu.pm | 481 ++++++++++++++++++++----------------------------------- 1 file changed, 171 insertions(+), 310 deletions(-) the original patch was +381 for that file, so in total we are now at +242 instead. could you take a look and see if I missed anything fundamental? -----8<----- diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 2f50f38..41e1ab7 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -60,8 +60,19 @@ my $resolve_cdrom_alias = sub { } }; +my $parse_import_sources = sub { + my $param = shift; + my $import = {}; + foreach my $pair (PVE::Tools::split_list($param)) { + my ($device, $diskimage) = split('=', $pair); + $import->{$device} = $diskimage; + } + + return $import; +}; + my $NEW_DISK_RE = qr!^(([^/:\s]+):)?(\d+(\.\d+)?)$!; -my $IMPORT_DISK_RE = qr!^([^/:\s]+):0$!; +my $IMPORT_DISK_RE = qr!^([^/:\s]+):-1$!; my $check_storage_access = sub { my ($rpcenv, $authuser, $storecfg, $vmid, $settings, $default_storage) = @_; @@ -84,6 +95,12 @@ my $check_storage_access = sub { my $scfg = PVE::Storage::storage_config($storecfg, $storeid); raise_param_exc({ storage => "storage '$storeid' does not support vm images"}) if !$scfg->{content}->{images}; + } elsif ($volid =~ $IMPORT_DISK_RE) { + my $storeid = $1; + $rpcenv->check($authuser, "/storage/$storeid", ['Datastore.AllocateSpace']); + my $scfg = PVE::Storage::storage_config($storecfg, $storeid); + raise_param_exc({ storage => "storage '$storeid' does not support vm images"}) + if !$scfg->{content}->{images}; } else { PVE::Storage::check_volume_access($rpcenv, $authuser, $storecfg, $vmid, $volid); } @@ -91,6 +108,13 @@ my $check_storage_access = sub { $rpcenv->check($authuser, "/storage/$settings->{vmstatestorage}", ['Datastore.AllocateSpace']) if defined($settings->{vmstatestorage}); + + if (defined($settings->{import_sources})) { + my $images = $parse_import_sources->($settings->{import_sources}); + foreach my $source_image (values %$images) { + PVE::Storage::check_volume_access($rpcenv, $authuser, $storecfg, $vmid, $source_image); + } + } }; my $check_storage_access_clone = sub { @@ -133,10 +157,25 @@ my $check_storage_access_clone = sub { return $sharedvm; }; +# Raise exception if $format is not supported by $storeid +my $check_format_is_supported = sub { + my ($format, $storeid, $storecfg) = @_; + die "storage ID parameter must be passed to the sub" if !$storeid; + die "storage configuration must be passed to the sub" if !$storecfg; + + return if !$format; + + my (undef, $valid_formats) = PVE::Storage::storage_default_format($storecfg, $storeid); + my $supported = grep { $_ eq $format } @$valid_formats; + + die "format '$format' is not supported on storage $storeid" if !$supported; +}; + + # Note: $pool is only needed when creating a VM, because pool permissions # are automatically inherited if VM already exists inside a pool. my $create_disks = sub { - my ($rpcenv, $authuser, $conf, $arch, $storecfg, $vmid, $pool, $settings, $default_storage) = @_; + my ($rpcenv, $authuser, $conf, $arch, $storecfg, $vmid, $pool, $settings, $default_storage, $import) = @_; my $vollist = []; @@ -192,6 +231,69 @@ my $create_disks = sub { $disk->{size} = PVE::Tools::convert_size($size, 'kb' => 'b'); delete $disk->{format}; # no longer needed $res->{$ds} = PVE::QemuServer::print_drive($disk); + } elsif ($volid =~ $IMPORT_DISK_RE) { + my $target_storage = $1; + + my $source = $import->{$ds}; + die "cannot import '$ds', no import source defined\n" if !$source; + $source = PVE::Storage::abs_filesystem_path($storecfg, $source, 1); + my $src_size = PVE::Storage::file_size_info($source); + die "Could not get file size of $source" if !defined($src_size); + + $check_format_is_supported->($disk->{format}, $storeid, $storecfg); + + my $dst_format = PVE::QemuServer::resolve_dst_disk_format( + $storecfg, + $storeid, + undef, + $disk->{format}, + ); + my $dst_volid = PVE::Storage::vdisk_alloc( + $storecfg, + $storeid, + $vmid, + $dst_format, + undef, + PVE::Tools::convert_size($src_size, 'b' => 'kb'), + ); + + print "Importing disk image '$source' as '$dst_volid'...\n"; + eval { + local $SIG{INT} = + local $SIG{TERM} = + local $SIG{QUIT} = + local $SIG{HUP} = + local $SIG{PIPE} = sub { die "Interrupted by signal $!\n"; }; + + my $zeroinit = PVE::Storage::volume_has_feature( + $storecfg, + 'sparseinit', + $dst_volid, + ); + PVE::Storage::activate_volumes($storecfg, [$dst_volid]); + PVE::QemuServer::qemu_img_convert( + $source, + $dst_volid, + $src_size, + undef, + $zeroinit, + ); + PVE::Storage::deactivate_volumes($storecfg, [$dst_volid]); + + }; + if (my $err = $@) { + eval { PVE::Storage::vdisk_free($storecfg, $dst_volid) }; + warn "Cleanup of $dst_volid failed: $@ \n" if $@; + + die "Importing disk '$source' failed: $err\n" if $err; + } + push @$vollist, $dst_volid; + $disk->{file} = $dst_volid; + if ($ds !~ /^unused\d+$/) { + $disk->{size} = $src_size; + delete $disk->{format}; # no longer needed + } + $res->{$ds} = PVE::QemuServer::print_drive($disk); } else { PVE::Storage::check_volume_access($rpcenv, $authuser, $storecfg, $vmid, $volid); @@ -218,7 +320,7 @@ my $create_disks = sub { } }; - eval { PVE::QemuConfig->foreach_volume($settings, $code); }; + eval { PVE::QemuConfig->foreach_volume_full($settings, { include_unused => 1 }, $code); }; # free allocated images on error if (my $err = $@) { @@ -544,6 +646,13 @@ __PACKAGE__->register_method({ default => 0, description => "Start VM after it was created successfully.", }, + import_sources => { + description => "\\0 delimited mapping of devices to disk images to import." . + "For example, scsi0=/mnt/nfs/image1.vmdk", + type => 'string', + format => 'device-image-pair-alist', + optional => 1, + }, }), }, returns => { @@ -608,21 +717,34 @@ __PACKAGE__->register_method({ &$check_cpu_model_access($rpcenv, $authuser, $param); + my $import_devices = $parse_import_sources->($param->{import_sources}); + foreach my $opt (keys %$param) { if (PVE::QemuServer::is_valid_drivename($opt)) { my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); raise_param_exc({ $opt => "unable to parse drive options" }) if !$drive; + raise_param_exc({ $opt => "not marked for import, but import source defined" }) + if $drive->{file} !~ $IMPORT_DISK_RE && $import_devices->{$opt}; + raise_param_exc({ $opt => "marked for import, but no import source defined" }) + if $drive->{file} =~ $IMPORT_DISK_RE && !$import_devices->{$opt}; PVE::QemuServer::cleanup_drive_path($opt, $storecfg, $drive); $param->{$opt} = PVE::QemuServer::print_drive($drive); } } + foreach my $opt (keys %$import_devices) { + raise_param_exc({ import_sources => "$opt not marked for import, but import source defined" }) + if !defined($param->{$opt}); + + } PVE::QemuServer::add_random_macs($param); } else { my $keystr = join(' ', keys %$param); raise_param_exc({ archive => "option conflicts with other options ($keystr)"}) if $keystr; + raise_param_exc({ import_sources => "cannot import existing disk and restore backup." }) if $param->{import_sources}; + if ($archive eq '-') { die "pipe requires cli environment\n" if $rpcenv->{type} ne 'cli'; @@ -690,10 +812,11 @@ __PACKAGE__->register_method({ my $realcmd = sub { my $conf = $param; my $arch = PVE::QemuServer::get_vm_arch($conf); + my $import = $parse_import_sources->(extract_param($param, "import_sources")); my $vollist = []; eval { - $vollist = &$create_disks($rpcenv, $authuser, $conf, $arch, $storecfg, $vmid, $pool, $param, $storage); + $vollist = &$create_disks($rpcenv, $authuser, $conf, $arch, $storecfg, $vmid, $pool, $param, $storage, $import); if (!$conf->{boot}) { my $devs = PVE::QemuServer::get_default_bootdevices($conf); @@ -1163,11 +1286,17 @@ my $update_vm_api = sub { die "cannot add non-replicatable volume to a replicated VM\n"; }; + my $import_devices = $parse_import_sources->($param->{import_sources}); + foreach my $opt (keys %$param) { if (PVE::QemuServer::is_valid_drivename($opt)) { # cleanup drive path my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); raise_param_exc({ $opt => "unable to parse drive options" }) if !$drive; + raise_param_exc({ $opt => "not marked for import, but import source defined" }) + if $drive->{file} !~ $IMPORT_DISK_RE && $import_devices->{$opt}; + raise_param_exc({ $opt => "marked for import, but no import source defined" }) + if $drive->{file} =~ $IMPORT_DISK_RE && !$import_devices->{$opt}; PVE::QemuServer::cleanup_drive_path($opt, $storecfg, $drive); $check_replication->($drive); $param->{$opt} = PVE::QemuServer::print_drive($drive); @@ -1185,12 +1314,20 @@ my $update_vm_api = sub { } } + foreach my $opt (keys %$import_devices) { + raise_param_exc({ import_sources => "$opt not marked for import, but import source defined" }) + if !defined($param->{$opt}); + + } + &$check_vm_modify_config_perm($rpcenv, $authuser, $vmid, undef, [@delete]); &$check_vm_modify_config_perm($rpcenv, $authuser, $vmid, undef, [keys %$param]); &$check_storage_access($rpcenv, $authuser, $storecfg, $vmid, $param); + delete $param->{import_sources}; + my $updatefn = sub { my $conf = PVE::QemuConfig->load_config($vmid); @@ -1332,7 +1469,7 @@ my $update_vm_api = sub { PVE::QemuServer::vmconfig_register_unused_drive($storecfg, $vmid, $conf, PVE::QemuServer::parse_drive($opt, $conf->{pending}->{$opt})) if defined($conf->{pending}->{$opt}); - &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}); + &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, {$opt => $import_devices->{$opt}}); } elsif ($opt =~ m/^serial\d+/) { if ((!defined($conf->{$opt}) || $conf->{$opt} eq 'socket') && $param->{$opt} eq 'socket') { $rpcenv->check_vm_perm($authuser, $vmid, undef, ['VM.Config.HWType']); @@ -1476,6 +1613,13 @@ __PACKAGE__->register_method({ optional => 1, requires => 'delete', }, + import_sources => { + description => "\\0 delimited mapping of devices to disk images to import." . + "For example, scsi0=/mnt/nfs/image1.vmdk", + type => 'string', + format => 'device-image-pair-alist', + optional => 1, + }, digest => { type => 'string', description => 'Prevent changes if current configuration file has different SHA1 digest. This can be used to prevent concurrent modifications.', @@ -4377,111 +4521,6 @@ __PACKAGE__->register_method({ return PVE::QemuServer::Cloudinit::dump_cloudinit_config($conf, $param->{vmid}, $param->{type}); }}); -# Raise exception if $format is not supported by $storeid -my $check_format_is_supported = sub { - my ($format, $storeid, $storecfg) = @_; - die "storage ID parameter must be passed to the sub" if !$storeid; - die "storage configuration must be passed to the sub" if !$storecfg; - - return if !$format; - - my (undef, $valid_formats) = PVE::Storage::storage_default_format($storecfg, $storeid); - my $supported = grep { $_ eq $format } @$valid_formats; - - die "format '$format' is not supported on storage $storeid" if !$supported; -}; - -# storecfg ... PVE::Storage::config() -# vmid ... target VM ID -# vmconf ... target VM configuration -# source ... source image (volid or absolute path) -# target ... hash with -# storeid => storage ID -# format => disk format (optional) -# options => hash with device options (may or may not contain <storeid>:0) -# device => device where the disk is attached (for example, scsi3) (optional) -# -# returns ... volid of the allocated disk image (e.g. local-lvm:vm-100-disk-2) -my $import_disk_image = sub { - my ($storecfg, $vmid, $vmconf, $source, $target) = @_; - my $requested_format = $target->{format}; - my $storeid = $target->{storeid}; - - die "Source parameter is undefined!" if !defined $source; - $source = PVE::Storage::abs_filesystem_path($storecfg, $source, 1); - - eval { PVE::Storage::storage_config($storecfg, $storeid) }; - die "Error while importing disk image $source: $@\n" if $@; - - my $src_size = PVE::Storage::file_size_info($source); - die "Could not get file size of $source" if !defined($src_size); - - $check_format_is_supported->($requested_format, $storeid, $storecfg); - - my $dst_format = PVE::QemuServer::resolve_dst_disk_format( - $storecfg, - $storeid, - undef, - $requested_format, - ); - my $dst_volid = PVE::Storage::vdisk_alloc( - $storecfg, - $storeid, - $vmid, - $dst_format, - undef, - $src_size / 102, - ); - - print "Importing disk image '$source' as '$dst_volid'...\n"; - eval { - local $SIG{INT} = - local $SIG{TERM} = - local $SIG{QUIT} = - local $SIG{HUP} = - local $SIG{PIPE} = sub { die "Interrupted by signal $!\n"; }; - - my $zeroinit = PVE::Storage::volume_has_feature( - $storecfg, - 'sparseinit', - $dst_volid, - ); - PVE::Storage::activate_volumes($storecfg, [$dst_volid]); - PVE::QemuServer::qemu_img_convert( - $source, - $dst_volid, - $src_size, - undef, - $zeroinit, - ); - PVE::Storage::deactivate_volumes($storecfg, [$dst_volid]); - - }; - if (my $err = $@) { - eval { PVE::Storage::vdisk_free($storecfg, $dst_volid) }; - warn "Cleanup of $dst_volid failed: $@ \n" if $@; - - die "Importing disk '$source' failed: $err\n" if $err; - } - - $target->{options}->{file} = $dst_volid; - my $options_string = PVE::QemuServer::print_drive($target->{options}); - $target->{device} = PVE::QemuConfig->add_unused_volume($vmconf, $dst_volid) - if !$target->{device}; - - $update_vm_api->( - { - vmid => $vmid, - $target->{device} => $options_string, - skiplock => 1, - digest => $vmconf->{digest}, - }, - 1, - ); - - return $dst_volid; -}; - __PACKAGE__->register_method ({ name => 'importdisk', path => '{vmid}/importdisk', @@ -4544,22 +4583,25 @@ __PACKAGE__->register_method ({ PVE::Storage::storage_config($storecfg, $storeid); - if ($device_options) { - # $device_options may or may not contain <storeid>:0 - my $parsed = PVE::QemuServer::Drive::parse_drive($device, $device_options); - if ($parsed) { - raise_param_exc({$device_options => "Invalid import syntax"}) - if !($parsed->{file} =~ $IMPORT_DISK_RE); - } else { - my $fake = "$storeid:0,$device_options"; - $parsed = PVE::QemuServer::Drive::parse_drive($device, $fake); - } - delete $parsed->{file}; - delete $parsed->{interface}; - delete $parsed->{index}; - $device_options = $parsed; + if (!$device_options) { + $device_options = "$storeid:0"; + } + + # $device_options may or may not contain <storeid>:0 + my $parsed = PVE::QemuServer::Drive::parse_drive($device, $device_options); + + if ($parsed) { + raise_param_exc({$device_options => "Invalid import syntax"}) + if !($parsed->{file} =~ $IMPORT_DISK_RE); + } else { + my $fake = "$storeid:0,$device_options"; + $parsed = PVE::QemuServer::Drive::parse_drive($device, $fake); } + delete $parsed->{interface}; + delete $parsed->{index}; + $device_options = $parsed; + # Format can be set explicitly "--format vmdk" # or as part of device options "--device_options discard=on,format=vmdk" my $format = extract_param($param, 'format'); @@ -4570,192 +4612,11 @@ __PACKAGE__->register_method ({ } $check_format_is_supported->($format, $storeid, $storecfg); - # quick checks before fork + lock - my $conf = PVE::QemuConfig->load_config($vmid); - PVE::QemuConfig->check_lock($conf); - PVE::Tools::assert_if_modified($conf->{digest}, $digest); - if ($device && $conf->{$device}) { - die "Could not import because device $device is already in ". - "use in VM $vmid. Choose a different device!"; - } - - my $worker = sub { - PVE::QemuConfig->lock_config($vmid, sub { - $conf = PVE::QemuConfig->load_config($vmid); - PVE::QemuConfig->check_lock($conf); - - PVE::Tools::assert_if_modified($conf->{digest}, $digest); - PVE::QemuConfig->set_lock($vmid, 'import'); - $conf = PVE::QemuConfig->load_config($vmid); - }); - - my $target = { - node => $node, - storeid => $storeid, - }; - $target->{format} = $format; - $target->{device} = $device; - $target->{options} = $device_options; - eval { $import_disk_image->($storecfg, $vmid, $conf, $source, $target) }; - my $err = $@; - eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; - warn $@ if $@; - die $err if $err; - }; - return $rpcenv->fork_worker('importdisk', $vmid, $authuser, $worker); + return $update_vm_api->({ + $device => PVE::QemuServer::Drive::print_drive($device_options), + import_sources => "$device=$source", + digest => $digest, + }); }}); -__PACKAGE__->register_method({ - name => 'importvm', - path => '{vmid}/importvm', - method => 'POST', - description => "Import a VM from existing disk images.", - protected => 1, - proxyto => 'node', - parameters => { - additionalProperties => 0, - properties => PVE::QemuServer::json_config_properties( - { - node => get_standard_option('pve-node'), - vmid => get_standard_option('pve-vmid', { completion => - \&PVE::Cluster::complete_next_vmid }), - diskimage => { - description => "\\0 delimited mapping of devices to disk images. For " . - "example, scsi0=/mnt/nfs/image1.vmdk", - type => 'string', - format => 'device-image-pair-alist', - }, - start => { - optional => 1, - type => 'boolean', - default => 0, - description => "Start VM after it was imported successfully.", - }, - }), - }, - returns => { - type => 'string', - }, - code => sub { - my ($param) = @_; - my $node = extract_param($param, 'node'); - my $vmid = extract_param($param, 'vmid'); - my $diskimages_string = extract_param($param, 'diskimage'); - my $boot = extract_param($param, 'boot'); - my $start = extract_param($param, 'start'); - - my $rpcenv = PVE::RPCEnvironment::get(); - my $authuser = $rpcenv->get_user(); - my $storecfg = PVE::Storage::config(); - - PVE::Cluster::check_cfs_quorum(); - - my $import_param = {}; - foreach my $opt (keys %$param) { - next if $opt eq 'efidisk0'; - raise_param_exc({bootdisk => "Deprecated: Use --boot order= instead"}) - if $opt eq 'bootdisk'; - - if (PVE::QemuServer::Drive::is_valid_drivename($opt)) { - my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); - if ($drive->{file} =~ $IMPORT_DISK_RE) { - $import_param->{$opt} = $drive; - delete $param->{$opt}; - } - } - } - - my $diskimages = {}; - foreach my $pair (PVE::Tools::split_list($diskimages_string)) { - my ($device, $diskimage) = split('=', $pair); - $diskimages->{$device} = $diskimage; - raise_param_exc({ - $device => "Device '$device' not marked for import, " . - "but import source '$diskimage' specified", - }) if !defined($import_param->{$device}); - PVE::Storage::abs_filesystem_path($storecfg, $diskimage, 1); - } - - foreach my $device (keys %$import_param) { - raise_param_exc({ - $device => "Device '$device' marked for import, but no source given\n", - }) if !defined($diskimages->{$device}); - } - - eval { PVE::QemuConfig->create_and_lock_config($vmid, 0, 'import') }; - die "Unable to create config for VM import: $@" if $@; - - my $worker = sub { - my $reload_conf = sub { - my ($vmid) = @_; - my $conf = PVE::QemuConfig->load_config($vmid); - return $conf if PVE::QemuConfig->has_lock($conf, 'import'); - die "import lock in VM $vmid config file missing!"; - }; - - my $conf = $reload_conf->($vmid); - $update_vm_api->( - { - %$param, - node => $node, - vmid => $vmid, - skiplock => 1, - digest => $conf->{digest}, - }, - 1 - ); - - eval { - foreach my $device (keys %$import_param) { - $conf = $reload_conf->($vmid); - my $drive = $import_param->{$device}; - my $storeid = PVE::Storage::parse_volume_id($drive->{file}); - my $imported = $import_disk_image->( - $storecfg, - $vmid, - $conf, - $diskimages->{$device}, - { - storeid => $storeid, - format => $drive->{format}, - options => $drive, - device => $device, - }, - ); - } - }; - my $err = $@; - if ($err) { - eval { PVE::QemuServer::destroy_vm($storecfg, $vmid, 1) }; - warn "Could not destroy VM $vmid: $@" if "$@"; - - die "Import failed: $err"; - } - - $conf = $reload_conf->($vmid); - if (!$boot) { - my $bootdevs = PVE::QemuServer::get_default_bootdevices($conf); - $boot = PVE::QemuServer::print_bootorder($bootdevs); - } - $update_vm_api->( - { - node => $node, - vmid => $vmid, - boot => $boot, - skiplock => 1, - digest => $conf->{digest}, - }, - 1, - ); - - eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; - warn $@ if $@; - - PVE::QemuServer::vm_start($storecfg, $vmid) if $start; - }; - - return $rpcenv->fork_worker('importvm', $vmid, $authuser, $worker); - }}); - - 1; ----->8----- On March 26, 2021 1:32 pm, Dominic Jäger wrote: > Extend qm importdisk/importovf functionality to the API. > > Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com> > > --- > v6->v7: Feedback by Fabian G > - Introduce a regex for the import syntax <storeid>:0 > - Use parameter list instead of hash for import helper > - More parsing, less string magic > - More VM config digest checking > - Create a schema format for diskimage source mapping > - Preliminarily remove some boot parameter handling > - Dare to really edit schema format subs for a cleaner solution > - Whitespace, variable names, ... > > PVE/API2/Qemu.pm | 383 ++++++++++++++++++++++++++++++++++++++++- > PVE/API2/Qemu/Makefile | 2 +- > PVE/API2/Qemu/OVF.pm | 68 ++++++++ > PVE/QemuServer.pm | 52 +++++- > PVE/QemuServer/OVF.pm | 10 +- > 5 files changed, 502 insertions(+), 13 deletions(-) > create mode 100644 PVE/API2/Qemu/OVF.pm > > diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm > index e95ab13..2f50f38 100644 > --- a/PVE/API2/Qemu.pm > +++ b/PVE/API2/Qemu.pm > @@ -45,7 +45,6 @@ BEGIN { > } > } > > -use Data::Dumper; # fixme: remove > > use base qw(PVE::RESTHandler); > > @@ -62,6 +61,7 @@ my $resolve_cdrom_alias = sub { > }; > > my $NEW_DISK_RE = qr!^(([^/:\s]+):)?(\d+(\.\d+)?)$!; > +my $IMPORT_DISK_RE = qr!^([^/:\s]+):0$!; > my $check_storage_access = sub { > my ($rpcenv, $authuser, $storecfg, $vmid, $settings, $default_storage) = @_; > > @@ -4377,4 +4377,385 @@ __PACKAGE__->register_method({ > return PVE::QemuServer::Cloudinit::dump_cloudinit_config($conf, $param->{vmid}, $param->{type}); > }}); > > +# Raise exception if $format is not supported by $storeid > +my $check_format_is_supported = sub { > + my ($format, $storeid, $storecfg) = @_; > + die "storage ID parameter must be passed to the sub" if !$storeid; > + die "storage configuration must be passed to the sub" if !$storecfg; > + > + return if !$format; > + > + my (undef, $valid_formats) = PVE::Storage::storage_default_format($storecfg, $storeid); > + my $supported = grep { $_ eq $format } @$valid_formats; > + > + die "format '$format' is not supported on storage $storeid" if !$supported; > +}; > + > +# storecfg ... PVE::Storage::config() > +# vmid ... target VM ID > +# vmconf ... target VM configuration > +# source ... source image (volid or absolute path) > +# target ... hash with > +# storeid => storage ID > +# format => disk format (optional) > +# options => hash with device options (may or may not contain <storeid>:0) > +# device => device where the disk is attached (for example, scsi3) (optional) > +# > +# returns ... volid of the allocated disk image (e.g. local-lvm:vm-100-disk-2) > +my $import_disk_image = sub { > + my ($storecfg, $vmid, $vmconf, $source, $target) = @_; > + my $requested_format = $target->{format}; > + my $storeid = $target->{storeid}; > + > + die "Source parameter is undefined!" if !defined $source; > + $source = PVE::Storage::abs_filesystem_path($storecfg, $source, 1); > + > + eval { PVE::Storage::storage_config($storecfg, $storeid) }; > + die "Error while importing disk image $source: $@\n" if $@; > + > + my $src_size = PVE::Storage::file_size_info($source); > + die "Could not get file size of $source" if !defined($src_size); > + > + $check_format_is_supported->($requested_format, $storeid, $storecfg); > + > + my $dst_format = PVE::QemuServer::resolve_dst_disk_format( > + $storecfg, > + $storeid, > + undef, > + $requested_format, > + ); > + my $dst_volid = PVE::Storage::vdisk_alloc( > + $storecfg, > + $storeid, > + $vmid, > + $dst_format, > + undef, > + $src_size / 102, > + ); > + > + print "Importing disk image '$source' as '$dst_volid'...\n"; > + eval { > + local $SIG{INT} = > + local $SIG{TERM} = > + local $SIG{QUIT} = > + local $SIG{HUP} = > + local $SIG{PIPE} = sub { die "Interrupted by signal $!\n"; }; > + > + my $zeroinit = PVE::Storage::volume_has_feature( > + $storecfg, > + 'sparseinit', > + $dst_volid, > + ); > + PVE::Storage::activate_volumes($storecfg, [$dst_volid]); > + PVE::QemuServer::qemu_img_convert( > + $source, > + $dst_volid, > + $src_size, > + undef, > + $zeroinit, > + ); > + PVE::Storage::deactivate_volumes($storecfg, [$dst_volid]); > + > + }; > + if (my $err = $@) { > + eval { PVE::Storage::vdisk_free($storecfg, $dst_volid) }; > + warn "Cleanup of $dst_volid failed: $@ \n" if $@; > + > + die "Importing disk '$source' failed: $err\n" if $err; > + } > + > + $target->{options}->{file} = $dst_volid; > + my $options_string = PVE::QemuServer::print_drive($target->{options}); > + $target->{device} = PVE::QemuConfig->add_unused_volume($vmconf, $dst_volid) > + if !$target->{device}; > + > + $update_vm_api->( > + { > + vmid => $vmid, > + $target->{device} => $options_string, > + skiplock => 1, > + digest => $vmconf->{digest}, > + }, > + 1, > + ); > + > + return $dst_volid; > +}; > + > +__PACKAGE__->register_method ({ > + name => 'importdisk', > + path => '{vmid}/importdisk', > + method => 'POST', > + proxyto => 'node', > + protected => 1, > + description => "Import an external disk image into a VM. The image format ". > + "has to be supported by qemu-img.", > + parameters => { > + additionalProperties => 0, > + properties => { > + node => get_standard_option('pve-node'), > + vmid => get_standard_option('pve-vmid', > + {completion => \&PVE::QemuServer::complete_vmid}), > + source => { > + description => "Disk image to import. Can be a volid ". > + "(local:99/imageToImport.raw) or an absolute path on the server.", > + type => 'string', > + }, > + device => { > + type => 'string', > + description => "Bus/Device type of the new disk (e.g. 'ide0', ". > + "'scsi2'). Will add the image as unused disk if omitted.", > + enum => [PVE::QemuServer::Drive::valid_drive_names()], > + optional => 1, > + }, > + device_options => { > + type => 'string', > + description => "Options to set for the new disk (e.g. 'discard=on,backup=0')", > + optional => 1, > + requires => 'device', > + }, > + storage => get_standard_option('pve-storage-id', { > + description => "The storage to which the image will be imported to.", > + completion => \&PVE::QemuServer::complete_storage, > + }), > + format => { > + type => 'string', > + description => 'Target format.', > + enum => [ 'raw', 'qcow2', 'vmdk' ], > + optional => 1, > + }, > + digest => get_standard_option('pve-config-digest'), > + }, > + }, > + returns => { type => 'string'}, > + code => sub { > + my ($param) = @_; > + my $vmid = extract_param($param, 'vmid'); > + my $node = extract_param($param, 'node'); > + my $source = extract_param($param, 'source'); > + my $digest = extract_param($param, 'digest'); > + my $device_options = extract_param($param, 'device_options'); > + my $device = extract_param($param, 'device'); > + my $storeid = extract_param($param, 'storage'); > + > + my $rpcenv = PVE::RPCEnvironment::get(); > + my $authuser = $rpcenv->get_user(); > + my $storecfg = PVE::Storage::config(); > + PVE::Storage::storage_config($storecfg, $storeid); > + > + > + if ($device_options) { > + # $device_options may or may not contain <storeid>:0 > + my $parsed = PVE::QemuServer::Drive::parse_drive($device, $device_options); > + if ($parsed) { > + raise_param_exc({$device_options => "Invalid import syntax"}) > + if !($parsed->{file} =~ $IMPORT_DISK_RE); > + } else { > + my $fake = "$storeid:0,$device_options"; > + $parsed = PVE::QemuServer::Drive::parse_drive($device, $fake); > + } > + delete $parsed->{file}; > + delete $parsed->{interface}; > + delete $parsed->{index}; > + $device_options = $parsed; > + } > + > + # Format can be set explicitly "--format vmdk" > + # or as part of device options "--device_options discard=on,format=vmdk" > + my $format = extract_param($param, 'format'); > + if ($device_options) { > + raise_param_exc({format => "Format already specified in device_options!"}) > + if $format && $device_options->{format}; > + $format = $format || $device_options->{format}; # may be undefined > + } > + $check_format_is_supported->($format, $storeid, $storecfg); > + > + # quick checks before fork + lock > + my $conf = PVE::QemuConfig->load_config($vmid); > + PVE::QemuConfig->check_lock($conf); > + PVE::Tools::assert_if_modified($conf->{digest}, $digest); > + if ($device && $conf->{$device}) { > + die "Could not import because device $device is already in ". > + "use in VM $vmid. Choose a different device!"; > + } > + > + my $worker = sub { > + PVE::QemuConfig->lock_config($vmid, sub { > + $conf = PVE::QemuConfig->load_config($vmid); > + PVE::QemuConfig->check_lock($conf); > + > + PVE::Tools::assert_if_modified($conf->{digest}, $digest); > + PVE::QemuConfig->set_lock($vmid, 'import'); > + $conf = PVE::QemuConfig->load_config($vmid); > + }); > + > + my $target = { > + node => $node, > + storeid => $storeid, > + }; > + $target->{format} = $format; > + $target->{device} = $device; > + $target->{options} = $device_options; > + eval { $import_disk_image->($storecfg, $vmid, $conf, $source, $target) }; > + my $err = $@; > + eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; > + warn $@ if $@; > + die $err if $err; > + }; > + return $rpcenv->fork_worker('importdisk', $vmid, $authuser, $worker); > + }}); > + > +__PACKAGE__->register_method({ > + name => 'importvm', > + path => '{vmid}/importvm', > + method => 'POST', > + description => "Import a VM from existing disk images.", > + protected => 1, > + proxyto => 'node', > + parameters => { > + additionalProperties => 0, > + properties => PVE::QemuServer::json_config_properties( > + { > + node => get_standard_option('pve-node'), > + vmid => get_standard_option('pve-vmid', { completion => > + \&PVE::Cluster::complete_next_vmid }), > + diskimage => { > + description => "\\0 delimited mapping of devices to disk images. For " . > + "example, scsi0=/mnt/nfs/image1.vmdk", > + type => 'string', > + format => 'device-image-pair-alist', > + }, > + start => { > + optional => 1, > + type => 'boolean', > + default => 0, > + description => "Start VM after it was imported successfully.", > + }, > + }), > + }, > + returns => { > + type => 'string', > + }, > + code => sub { > + my ($param) = @_; > + my $node = extract_param($param, 'node'); > + my $vmid = extract_param($param, 'vmid'); > + my $diskimages_string = extract_param($param, 'diskimage'); > + my $boot = extract_param($param, 'boot'); > + my $start = extract_param($param, 'start'); > + > + my $rpcenv = PVE::RPCEnvironment::get(); > + my $authuser = $rpcenv->get_user(); > + my $storecfg = PVE::Storage::config(); > + > + PVE::Cluster::check_cfs_quorum(); > + > + my $import_param = {}; > + foreach my $opt (keys %$param) { > + next if $opt eq 'efidisk0'; > + raise_param_exc({bootdisk => "Deprecated: Use --boot order= instead"}) > + if $opt eq 'bootdisk'; > + > + if (PVE::QemuServer::Drive::is_valid_drivename($opt)) { > + my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); > + if ($drive->{file} =~ $IMPORT_DISK_RE) { > + $import_param->{$opt} = $drive; > + delete $param->{$opt}; > + } > + } > + } > + > + my $diskimages = {}; > + foreach my $pair (PVE::Tools::split_list($diskimages_string)) { > + my ($device, $diskimage) = split('=', $pair); > + $diskimages->{$device} = $diskimage; > + raise_param_exc({ > + $device => "Device '$device' not marked for import, " . > + "but import source '$diskimage' specified", > + }) if !defined($import_param->{$device}); > + PVE::Storage::abs_filesystem_path($storecfg, $diskimage, 1); > + } > + > + foreach my $device (keys %$import_param) { > + raise_param_exc({ > + $device => "Device '$device' marked for import, but no source given\n", > + }) if !defined($diskimages->{$device}); > + } > + > + eval { PVE::QemuConfig->create_and_lock_config($vmid, 0, 'import') }; > + die "Unable to create config for VM import: $@" if $@; > + > + my $worker = sub { > + my $reload_conf = sub { > + my ($vmid) = @_; > + my $conf = PVE::QemuConfig->load_config($vmid); > + return $conf if PVE::QemuConfig->has_lock($conf, 'import'); > + die "import lock in VM $vmid config file missing!"; > + }; > + > + my $conf = $reload_conf->($vmid); > + $update_vm_api->( > + { > + %$param, > + node => $node, > + vmid => $vmid, > + skiplock => 1, > + digest => $conf->{digest}, > + }, > + 1 > + ); > + > + eval { > + foreach my $device (keys %$import_param) { > + $conf = $reload_conf->($vmid); > + my $drive = $import_param->{$device}; > + my $storeid = PVE::Storage::parse_volume_id($drive->{file}); > + my $imported = $import_disk_image->( > + $storecfg, > + $vmid, > + $conf, > + $diskimages->{$device}, > + { > + storeid => $storeid, > + format => $drive->{format}, > + options => $drive, > + device => $device, > + }, > + ); > + } > + }; > + my $err = $@; > + if ($err) { > + eval { PVE::QemuServer::destroy_vm($storecfg, $vmid, 1) }; > + warn "Could not destroy VM $vmid: $@" if "$@"; > + > + die "Import failed: $err"; > + } > + > + $conf = $reload_conf->($vmid); > + if (!$boot) { > + my $bootdevs = PVE::QemuServer::get_default_bootdevices($conf); > + $boot = PVE::QemuServer::print_bootorder($bootdevs); > + } > + $update_vm_api->( > + { > + node => $node, > + vmid => $vmid, > + boot => $boot, > + skiplock => 1, > + digest => $conf->{digest}, > + }, > + 1, > + ); > + > + eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; > + warn $@ if $@; > + > + PVE::QemuServer::vm_start($storecfg, $vmid) if $start; > + }; > + > + return $rpcenv->fork_worker('importvm', $vmid, $authuser, $worker); > + }}); > + > + > 1; > diff --git a/PVE/API2/Qemu/Makefile b/PVE/API2/Qemu/Makefile > index 5d4abda..bdd4762 100644 > --- a/PVE/API2/Qemu/Makefile > +++ b/PVE/API2/Qemu/Makefile > @@ -1,4 +1,4 @@ > -SOURCES=Agent.pm CPU.pm Machine.pm > +SOURCES=Agent.pm CPU.pm Machine.pm OVF.pm > > .PHONY: install > install: > diff --git a/PVE/API2/Qemu/OVF.pm b/PVE/API2/Qemu/OVF.pm > new file mode 100644 > index 0000000..bd6e90b > --- /dev/null > +++ b/PVE/API2/Qemu/OVF.pm > @@ -0,0 +1,68 @@ > +package PVE::API2::Qemu::OVF; > + > +use strict; > +use warnings; > + > +use PVE::RESTHandler; > +use PVE::JSONSchema qw(get_standard_option); > +use PVE::QemuServer::OVF; > + > +use base qw(PVE::RESTHandler); > + > +__PACKAGE__->register_method ({ > + name => 'index', > + path => '', > + method => 'GET', > + proxyto => 'node', > + description => "Read an .ovf manifest.", > + parameters => { > + additionalProperties => 0, > + properties => { > + node => get_standard_option('pve-node'), > + manifest => { > + description => ".ovf manifest", > + type => 'string', > + }, > + }, > + }, > + returns => { > + description => "VM config according to .ovf manifest and digest of manifest", > + type => "object", > + }, > + returns => { > + type => 'object', > + additionalProperties => 1, > + properties => PVE::QemuServer::json_ovf_properties({ > + name => { > + type => 'string', > + optional => 1, > + }, > + cores => { > + type => 'integer', > + optional => 1, > + }, > + memory => { > + type => 'integer', > + optional => 1, > + }, > + }), > + }, > + code => sub { > + my ($param) = @_; > + > + my $manifest = $param->{manifest}; > + die "$manifest: non-existent or non-regular file\n" if (! -f $manifest); > + > + my $parsed = PVE::QemuServer::OVF::parse_ovf($manifest, 0, 1); > + my $result; > + $result->{cores} = $parsed->{qm}->{cores}; > + $result->{name} = $parsed->{qm}->{name}; > + $result->{memory} = $parsed->{qm}->{memory}; > + my $disks = $parsed->{disks}; > + foreach my $disk (@$disks) { > + $result->{$disk->{disk_address}} = $disk->{backing_file}; > + } > + return $result; > + }}); > + > +1; > \ No newline at end of file > diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm > index 1c0b5c2..131c0b6 100644 > --- a/PVE/QemuServer.pm > +++ b/PVE/QemuServer.pm > @@ -300,7 +300,7 @@ my $confdesc = { > optional => 1, > type => 'string', > description => "Lock/unlock the VM.", > - enum => [qw(backup clone create migrate rollback snapshot snapshot-delete suspending suspended)], > + enum => [qw(backup clone create migrate rollback snapshot snapshot-delete suspending suspended import)], > }, > cpulimit => { > optional => 1, > @@ -985,19 +985,41 @@ PVE::JSONSchema::register_format('pve-volume-id-or-qm-path', \&verify_volume_id_ > sub verify_volume_id_or_qm_path { > my ($volid, $noerr) = @_; > > - if ($volid eq 'none' || $volid eq 'cdrom' || $volid =~ m|^/|) { > - return $volid; > - } > + return $volid eq 'none' || $volid eq 'cdrom' ? > + $volid : > + verify_volume_id_or_absolute_path($volid, $noerr); > +} > + > +PVE::JSONSchema::register_format('pve-volume-id-or-absolute-path', \&verify_volume_id_or_absolute_path); > +sub verify_volume_id_or_absolute_path { > + my ($volid, $noerr) = @_; > + > + return $volid if $volid =~ m|^/|; > > - # if its neither 'none' nor 'cdrom' nor a path, check if its a volume-id > $volid = eval { PVE::JSONSchema::check_format('pve-volume-id', $volid, '') }; > if ($@) { > - return if $noerr; > + return undef if $noerr; > die $@; > } > return $volid; > } > > +PVE::JSONSchema::register_format('device-image-pair', \&verify_device_image_pair); > +sub verify_device_image_pair { > + my ($pair, $noerr) = @_; > + > + my $error = sub { > + return undef if $noerr; > + die $@; > + }; > + > + my ($device, $image) = split('=', $pair); > + $error->("Invalid device '$device'") if !PVE::QemuServer::Drive::is_valid_drivename($device); > + $error->("Invalid image '$image'") if !verify_volume_id_or_absolute_path($image); > + > + return $pair; > +} > + > my $usb_fmt = { > host => { > default_key => 1, > @@ -2030,6 +2052,22 @@ sub json_config_properties { > return $prop; > } > > +# Properties that we can read from an OVF file > +sub json_ovf_properties { > + my $prop = shift; > + > + foreach my $device ( PVE::QemuServer::Drive::valid_drive_names()) { > + $prop->{$device} = { > + type => 'string', > + format => 'pve-volume-id-or-absolute-path', > + description => "Disk image that gets imported to $device", > + optional => 1, > + }; > + } > + > + return $prop; > +} > + > # return copy of $confdesc_cloudinit to generate documentation > sub cloudinit_config_properties { > > @@ -6748,7 +6786,7 @@ sub qemu_img_convert { > $src_path = PVE::Storage::path($storecfg, $src_volid, $snapname); > $src_is_iscsi = ($src_path =~ m|^iscsi://|); > $cachemode = 'none' if $src_scfg->{type} eq 'zfspool'; > - } elsif (-f $src_volid) { > + } elsif (-f $src_volid || -b $src_volid) { > $src_path = $src_volid; > if ($src_path =~ m/\.($PVE::QemuServer::Drive::QEMU_FORMAT_RE)$/) { > $src_format = $1; > diff --git a/PVE/QemuServer/OVF.pm b/PVE/QemuServer/OVF.pm > index c76c199..48146e9 100644 > --- a/PVE/QemuServer/OVF.pm > +++ b/PVE/QemuServer/OVF.pm > @@ -87,7 +87,7 @@ sub id_to_pve { > > # returns two references, $qm which holds qm.conf style key/values, and \@disks > sub parse_ovf { > - my ($ovf, $debug) = @_; > + my ($ovf, $debug, $manifest_only) = @_; > > my $dom = XML::LibXML->load_xml(location => $ovf, no_blanks => 1); > > @@ -220,9 +220,11 @@ ovf:Item[rasd:InstanceID='%s']/rasd:ResourceType", $controller_id); > die "error parsing $filepath, file seems not to exist at $backing_file_path\n"; > } > > - my $virtual_size; > - if ( !($virtual_size = PVE::Storage::file_size_info($backing_file_path)) ) { > - die "error parsing $backing_file_path, size seems to be $virtual_size\n"; > + my $virtual_size = undef; > + if (!$manifest_only) { # Not possible if manifest is uploaded in web gui > + if ( !($virtual_size = PVE::Storage::file_size_info($backing_file_path)) ) { > + die "error parsing $backing_file_path: Could not get file size info: $@\n"; > + } > } > > $pve_disk = { > -- > 2.20.1 > > > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pve-devel] [PATCH v7 qemu-server] Add API for import wizards 2021-03-31 15:12 ` Fabian Grünbichler @ 2021-04-01 10:19 ` Dominic Jäger 2021-04-01 11:30 ` Fabian Grünbichler 0 siblings, 1 reply; 7+ messages in thread From: Dominic Jäger @ 2021-04-01 10:19 UTC (permalink / raw) To: Proxmox VE development discussion On Wed, Mar 31, 2021 at 05:12:28PM +0200, Fabian Grünbichler wrote: > this is starting to shape up nicely. as promised, I now took a stab at > (roughly!) integrating this into our regular flow (see diff below): > .... > > - we could likely drop the separate import_disk API call, and let the > `importdisk` CLI command prepare parameters for a regular VM config > update I think dropping importdisk is a good idea, too. > I'm sure I've missed some corner cases, as I've only tested create_vm > with importing, and not the other newly exposed APIs. I didn't get the importdisk API to work yet. But I'd look closer at stuff like that if (!$device_options) { $device_options = "$storeid:0"; <=== should be -1 } only if we keep it? One thing is important: That import parameter was at the place of the default storage PVE/API2/Qemu.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 41e1ab7..73af497 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1469,7 +1469,7 @@ my $update_vm_api = sub { PVE::QemuServer::vmconfig_register_unused_drive($storecfg, $vmid, $conf, PVE::QemuServer::parse_drive($opt, $conf->{pending}->{$opt})) if defined($conf->{pending}->{$opt}); - &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, {$opt => $import_devices->{$opt}}); + &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, undef, {$opt => $import_devices->{$opt}}); } elsif ($opt =~ m/^serial\d+/) { if ((!defined($conf->{$opt}) || $conf->{$opt} eq 'socket') && $param->{$opt} eq 'socket') { $rpcenv->check_vm_perm($authuser, $vmid, undef, ['VM.Config.HWType']); > stat shows how much boilerplace/duplication this removes, although there > is probably even more potential here since `create_vm` partly duplicates > the `update_vm_api` sub that ends up calling `create_disks`: Is that urgent? ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pve-devel] [PATCH v7 qemu-server] Add API for import wizards 2021-04-01 10:19 ` Dominic Jäger @ 2021-04-01 11:30 ` Fabian Grünbichler 0 siblings, 0 replies; 7+ messages in thread From: Fabian Grünbichler @ 2021-04-01 11:30 UTC (permalink / raw) To: Dominic Jäger, Proxmox VE development discussion On April 1, 2021 12:19 pm, Dominic Jäger wrote: > On Wed, Mar 31, 2021 at 05:12:28PM +0200, Fabian Grünbichler wrote: >> this is starting to shape up nicely. as promised, I now took a stab at >> (roughly!) integrating this into our regular flow (see diff below): >> .... >> >> - we could likely drop the separate import_disk API call, and let the >> `importdisk` CLI command prepare parameters for a regular VM config >> update > I think dropping importdisk is a good idea, too. > >> I'm sure I've missed some corner cases, as I've only tested create_vm >> with importing, and not the other newly exposed APIs. > > I didn't get the importdisk API to work yet. > But I'd look closer at stuff like that > if (!$device_options) { > $device_options = "$storeid:0"; <=== should be -1 > } > only if we keep it? > > One thing is important: That import parameter was at the place of the default storage yes, I just roughly moved the code to see whether the approach works out and then tested a single entry point. if there are no objections into merging it like that, it would definitely need a cleanup and more testing :) > > PVE/API2/Qemu.pm | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm > index 41e1ab7..73af497 100644 > --- a/PVE/API2/Qemu.pm > +++ b/PVE/API2/Qemu.pm > @@ -1469,7 +1469,7 @@ my $update_vm_api = sub { > PVE::QemuServer::vmconfig_register_unused_drive($storecfg, $vmid, $conf, PVE::QemuServer::parse_drive($opt, $conf->{pending}->{$opt})) > if defined($conf->{pending}->{$opt}); > > - &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, {$opt => $import_devices->{$opt}}); > + &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, undef, {$opt => $import_devices->{$opt}}); > } elsif ($opt =~ m/^serial\d+/) { > if ((!defined($conf->{$opt}) || $conf->{$opt} eq 'socket') && $param->{$opt} eq 'socket') { > $rpcenv->check_vm_perm($authuser, $vmid, undef, ['VM.Config.HWType']); > > >> stat shows how much boilerplace/duplication this removes, although there >> is probably even more potential here since `create_vm` partly duplicates >> the `update_vm_api` sub that ends up calling `create_disks`: > Is that urgent? no, it's just something I noticed since I added the same code in the same context in two places ;) ^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] [PATCH v7 manager] gui: Add import for disk & VM 2021-03-26 12:32 [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Dominic Jäger 2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger @ 2021-03-26 12:32 ` Dominic Jäger 2021-04-01 13:40 ` [pve-devel] applied: [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Thomas Lamprecht 2 siblings, 0 replies; 7+ messages in thread From: Dominic Jäger @ 2021-03-26 12:32 UTC (permalink / raw) To: pve-devel Add GUI wizard to import whole VMs and a window to import single disks in Hardware View. Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com> --- v6->v7: - Update to API changes - Add helpers to Utils - Whitespace & line breaks according to style guide - Making conditional branches in HDEdit easier to read PVE/API2/Nodes.pm | 7 + www/manager6/Makefile | 2 + www/manager6/Utils.js | 12 + www/manager6/Workspace.js | 15 ++ www/manager6/form/ControllerSelector.js | 15 ++ www/manager6/node/CmdMenu.js | 13 + www/manager6/qemu/HDEdit.js | 169 +++++++++++- www/manager6/qemu/HardwareView.js | 25 ++ www/manager6/qemu/ImportWizard.js | 332 ++++++++++++++++++++++++ www/manager6/qemu/MultiHDEdit.js | 277 ++++++++++++++++++++ www/manager6/window/Wizard.js | 2 + 11 files changed, 856 insertions(+), 13 deletions(-) create mode 100644 www/manager6/qemu/ImportWizard.js create mode 100644 www/manager6/qemu/MultiHDEdit.js diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm index ba6621c6..1cee6cb5 100644 --- a/PVE/API2/Nodes.pm +++ b/PVE/API2/Nodes.pm @@ -48,6 +48,7 @@ use PVE::API2::LXC; use PVE::API2::Network; use PVE::API2::NodeConfig; use PVE::API2::Qemu::CPU; +use PVE::API2::Qemu::OVF; use PVE::API2::Qemu; use PVE::API2::Replication; use PVE::API2::Services; @@ -76,6 +77,11 @@ __PACKAGE__->register_method ({ path => 'cpu', }); +__PACKAGE__->register_method ({ + subclass => "PVE::API2::Qemu::OVF", + path => 'readovf', +}); + __PACKAGE__->register_method ({ subclass => "PVE::API2::LXC", path => 'lxc', @@ -2183,6 +2189,7 @@ __PACKAGE__->register_method ({ return undef; }}); + # bash completion helper sub complete_templet_repo { diff --git a/www/manager6/Makefile b/www/manager6/Makefile index a2f7be6d..753cd1c0 100644 --- a/www/manager6/Makefile +++ b/www/manager6/Makefile @@ -196,8 +196,10 @@ JSSRC= \ qemu/CmdMenu.js \ qemu/Config.js \ qemu/CreateWizard.js \ + qemu/ImportWizard.js \ qemu/DisplayEdit.js \ qemu/HDEdit.js \ + qemu/MultiHDEdit.js \ qemu/HDEfi.js \ qemu/HDMove.js \ qemu/HDResize.js \ diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js index f502950f..dbfd65ce 100644 --- a/www/manager6/Utils.js +++ b/www/manager6/Utils.js @@ -1708,6 +1708,16 @@ Ext.define('PVE.Utils', { }); }, + // collection ... collection of strings of a subset of the descendants of container + // visible ... true to show and enable, false to hide and disable + setDescendantsVisible: function(container, collection, visible = 1) { + const hide = (element, value) => { + element.setHidden(value); + element.setDisabled(value); + }; + collection.map(e => container.down(e)).forEach(e => hide(e, !visible)); + }, + cpu_vendor_map: { 'default': 'QEMU', 'AuthenticAMD': 'AMD', @@ -1787,6 +1797,8 @@ Ext.define('PVE.Utils', { hastop: ['HA', gettext('Stop')], imgcopy: ['', gettext('Copy data')], imgdel: ['', gettext('Erase data')], + importdisk: ['VM', gettext('Import disk')], + importvm: ['VM', gettext('Import VM')], lvmcreate: [gettext('LVM Storage'), gettext('Create')], lvmthincreate: [gettext('LVM-Thin Storage'), gettext('Create')], migrateall: ['', gettext('Migrate all VMs and Containers')], diff --git a/www/manager6/Workspace.js b/www/manager6/Workspace.js index 0c1b9e0c..631739a0 100644 --- a/www/manager6/Workspace.js +++ b/www/manager6/Workspace.js @@ -280,11 +280,25 @@ Ext.define('PVE.StdWorkspace', { }, }); + var importVM = Ext.createWidget('button', { + pack: 'end', + margin: '3 5 0 0', + baseCls: 'x-btn', + iconCls: 'fa fa-desktop', + text: gettext("Import VM"), + hidden: Proxmox.UserName !== 'root@pam', + handler: function() { + var wiz = Ext.create('PVE.qemu.ImportWizard', {}); + wiz.show(); + }, + }); + sprovider.on('statechange', function(sp, key, value) { if (key === 'GuiCap' && value) { caps = value; createVM.setDisabled(!caps.vms['VM.Allocate']); createCT.setDisabled(!caps.vms['VM.Allocate']); + importVM.setDisabled(!caps.vms['VM.Allocate']); } }); @@ -332,6 +346,7 @@ Ext.define('PVE.StdWorkspace', { }, createVM, createCT, + importVM, { pack: 'end', margin: '0 5 0 0', diff --git a/www/manager6/form/ControllerSelector.js b/www/manager6/form/ControllerSelector.js index 23c61159..8e9aee98 100644 --- a/www/manager6/form/ControllerSelector.js +++ b/www/manager6/form/ControllerSelector.js @@ -68,6 +68,21 @@ clist_loop: deviceid.validate(); }, + getValues: function() { + return this.query('field').map(x => x.getValue()); + }, + + getValuesAsString: function() { + return this.getValues().join(''); + }, + + setValue: function(value) { + let regex = /([a-z]+)(\d+)/; + let [_, controller, deviceid] = regex.exec(value); + this.down('field[name=controller]').setValue(controller); + this.down('field[name=deviceid]').setValue(deviceid); + }, + initComponent: function() { var me = this; diff --git a/www/manager6/node/CmdMenu.js b/www/manager6/node/CmdMenu.js index b650bfa0..407cf2d0 100644 --- a/www/manager6/node/CmdMenu.js +++ b/www/manager6/node/CmdMenu.js @@ -29,6 +29,19 @@ Ext.define('PVE.node.CmdMenu', { wiz.show(); }, }, + { + text: gettext("Import VM"), + hidden: Proxmox.UserName !== 'root@pam', + itemId: 'importvm', + iconCls: 'fa fa-cube', + handler: function() { + var me = this.up('menu'); + var wiz = Ext.create('PVE.qemu.ImportWizard', { + nodename: me.nodename, + }); + wiz.show(); + }, + }, { xtype: 'menuseparator' }, { text: gettext('Bulk Start'), diff --git a/www/manager6/qemu/HDEdit.js b/www/manager6/qemu/HDEdit.js index e22111bf..a2f6c95a 100644 --- a/www/manager6/qemu/HDEdit.js +++ b/www/manager6/qemu/HDEdit.js @@ -58,6 +58,21 @@ Ext.define('PVE.qemu.HDInputPanel', { }, }, + isImport: function() { + return this.isImportVM || this.isImportDisk; + }, + + /* + All radiofields (esp. sourceRadioPath and sourceRadioStorage) have the + same scope for name. But we need a different scope for each HDInputPanel in + a MultiHDInputPanel to get the selection for each HDInputPanel => Make + names so that those within one HDInputPanel are equal, but different from other + HDInputPanels + */ + getSourceTypeID() { + return 'sourceType_' + this.id; + }, + onGetValues: function(values) { var me = this; @@ -70,6 +85,8 @@ Ext.define('PVE.qemu.HDInputPanel', { } else if (me.isCreate) { if (values.hdimage) { me.drive.file = values.hdimage; + } else if (me.isImport()) { + me.drive.file = `${values.hdstorage}:0`; } else { me.drive.file = values.hdstorage + ":" + values.disksize; } @@ -83,15 +100,33 @@ Ext.define('PVE.qemu.HDInputPanel', { PVE.Utils.propertyStringSet(me.drive, values.iothread, 'iothread', 'on'); PVE.Utils.propertyStringSet(me.drive, values.cache, 'cache'); - var names = ['mbps_rd', 'mbps_wr', 'iops_rd', 'iops_wr']; - Ext.Array.each(names, function(name) { - var burst_name = name + '_max'; + var names = ['mbps_rd', 'mbps_wr', 'iops_rd', 'iops_wr']; + Ext.Array.each(names, function(name) { + var burst_name = name + '_max'; PVE.Utils.propertyStringSet(me.drive, values[name], name); PVE.Utils.propertyStringSet(me.drive, values[burst_name], burst_name); - }); + }); + + const getSourceImageLocation = function() { + const type = values[me.getSourceTypeID()]; + return type === 'storage' ? values.sourceVolid : values.sourcePath; + }; + if (me.isImportVM) { + params.diskimage = `${confid}=${getSourceImageLocation()}`; + } + + const options = PVE.Parser.printQemuDrive(me.drive); - params[confid] = PVE.Parser.printQemuDrive(me.drive); + if (me.isImportDisk) { + params.device = confid; + params.device_options = options; + params.source = getSourceImageLocation(); + params.device = values.controller + values.deviceid; + params.storage = values.hdstorage; + } else { + params[confid] = options; + } return params; }, @@ -149,6 +184,10 @@ Ext.define('PVE.qemu.HDInputPanel', { me.setValues(values); }, + getDevice: function() { + return this.bussel.getValuesAsString(); + }, + setNodename: function(nodename) { var me = this; me.down('#hdstorage').setNodename(nodename); @@ -169,10 +208,15 @@ Ext.define('PVE.qemu.HDInputPanel', { me.advancedColumn2 = []; if (!me.confid || me.unused) { + let controllerColumn = me.isImport() ? me.column2 : me.column1; me.bussel = Ext.create('PVE.form.ControllerSelector', { + itemId: 'bussel', vmconfig: me.insideWizard ? { ide2: 'cdrom' } : {}, }); - me.column1.push(me.bussel); + if (me.isImport()) { + me.bussel.fieldLabel = 'Target Device'; + } + controllerColumn.push(me.bussel); me.scsiController = Ext.create('Ext.form.field.Display', { fieldLabel: gettext('SCSI Controller'), @@ -184,7 +228,7 @@ Ext.define('PVE.qemu.HDInputPanel', { submitValue: false, hidden: true, }); - me.column1.push(me.scsiController); + controllerColumn.push(me.scsiController); } if (me.unused) { @@ -199,14 +243,21 @@ Ext.define('PVE.qemu.HDInputPanel', { allowBlank: false, }); me.column1.push(me.unusedDisks); - } else if (me.isCreate) { - me.column1.push({ + } else if (me.isCreate || me.isImport()) { + let selector = { xtype: 'pveDiskStorageSelector', storageContent: 'images', name: 'disk', nodename: me.nodename, - autoSelect: me.insideWizard, - }); + hideSize: me.isImport(), + autoSelect: me.insideWizard || me.isImport(), + }; + if (me.isImport()) { + selector.storageLabel = gettext('Target storage'); + me.column2.push(selector); + } else { + me.column1.push(selector); + } } else { me.column1.push({ xtype: 'textfield', @@ -217,6 +268,12 @@ Ext.define('PVE.qemu.HDInputPanel', { }); } + if (me.isImport()) { + me.column2.push({ + xtype: 'box', + autoEl: { tag: 'hr' }, + }); + } me.column2.push( { xtype: 'CacheTypeSelector', @@ -231,6 +288,84 @@ Ext.define('PVE.qemu.HDInputPanel', { name: 'discard', }, ); + if (me.isImport()) { + me.column1.unshift( + { + xtype: 'radiofield', + itemId: 'sourceRadioStorage', + name: me.getSourceTypeID(), + inputValue: 'storage', + boxLabel: gettext('Use a storage as source'), + hidden: Proxmox.UserName !== 'root@pam', + checked: true, + listeners: { + change: (_, newValue) => { + const selectors = [ + '#sourceStorageSelector', + '#sourceFileSelector', + ]; + PVE.Utils.setDescendantsVisible(me, selectors, newValue); + }, + }, + }, { + xtype: 'pveStorageSelector', + itemId: 'sourceStorageSelector', + name: 'inputImageStorage', + nodename: me.nodename, + fieldLabel: gettext('Source Storage'), + storageContent: 'images', + autoSelect: me.insideWizard, + hidden: true, + disabled: true, + listeners: { + change: function(_, selectedStorage) { + me.down('#sourceFileSelector').setStorage(selectedStorage); + }, + }, + }, { + xtype: 'pveFileSelector', + itemId: 'sourceFileSelector', + name: 'sourceVolid', + nodename: me.nodename, + storageContent: 'images', + hidden: true, + disabled: true, + fieldLabel: gettext('Source Image'), + autoEl: { + tag: 'div', + 'data-qtip': gettext("Place your source images into a new folder <storageRoot>/images/<newVMID>, for example /var/lib/vz/images/999"), + }, + }, { + xtype: 'radiofield', + itemId: 'sourceRadioPath', + name: me.getSourceTypeID(), + inputValue: 'path', + boxLabel: gettext('Use an absolute path as source'), + hidden: Proxmox.UserName !== 'root@pam', + listeners: { + change: (_, newValue) => { + PVE.Utils.setDescendantsVisible(me, ['#sourcePathTextfield'], newValue); + }, + }, + }, { + xtype: 'textfield', + itemId: 'sourcePathTextfield', + fieldLabel: gettext('Source Path'), + name: 'sourcePath', + autoEl: { + tag: 'div', + 'data-qtip': gettext('Absolute path to the source disk image, for example: /home/user/somedisk.qcow2'), + }, + hidden: true, + disabled: true, + validator: function(insertedText) { + return insertedText.startsWith('/') || + insertedText.startsWith('http') || + gettext('Must be an absolute path or URL'); + }, + }, + ); + } me.advancedColumn1.push( { @@ -373,13 +508,18 @@ Ext.define('PVE.qemu.HDEdit', { nodename: nodename, unused: unused, isCreate: me.isCreate, + isImportVM: me.isImportVM, + isImportDisk: me.isImportDisk, }); - var subject; if (unused) { me.subject = gettext('Unused Disk'); + } else if (me.isImportDisk) { + me.subject = gettext('Import Disk'); + me.submitText = 'Import'; + me.backgroundDelay = undefined; } else if (me.isCreate) { - me.subject = gettext('Hard Disk'); + me.subject = gettext('Hard Disk'); } else { me.subject = gettext('Hard Disk') + ' (' + me.confid + ')'; } @@ -404,6 +544,9 @@ Ext.define('PVE.qemu.HDEdit', { ipanel.setDrive(drive); me.isValid(); // trigger validation } + if (me.isImportDisk) { + me.url = me.url.replace(/\/config$/, "/importdisk"); + } }, }); }, diff --git a/www/manager6/qemu/HardwareView.js b/www/manager6/qemu/HardwareView.js index 98352e3f..4fbf0e5e 100644 --- a/www/manager6/qemu/HardwareView.js +++ b/www/manager6/qemu/HardwareView.js @@ -431,6 +431,30 @@ Ext.define('PVE.qemu.HardwareView', { handler: run_move, }); + var import_btn = new Proxmox.button.Button({ + text: gettext('Import disk'), + hidden: Proxmox.UserName !== 'root@pam', + handler: function() { + var win = Ext.create('PVE.qemu.HDEdit', { + method: 'POST', + url: `/api2/extjs/${baseurl}`, + pveSelNode: me.pveSelNode, + isImportDisk: true, + listeners: { + add: function(_, component) { + const selectors = [ + '#sourceStorageSelector', + '#sourceFileSelector', + ]; + PVE.Utils.setDescendantsVisible(component, selectors); + }, + }, + }); + win.on('destroy', me.reload, me); + win.show(); + }, + }); + var remove_btn = new Proxmox.button.Button({ text: gettext('Remove'), defaultText: gettext('Remove'), @@ -759,6 +783,7 @@ Ext.define('PVE.qemu.HardwareView', { edit_btn, resize_btn, move_btn, + import_btn, revert_btn, ], rows: rows, diff --git a/www/manager6/qemu/ImportWizard.js b/www/manager6/qemu/ImportWizard.js new file mode 100644 index 00000000..0066adc4 --- /dev/null +++ b/www/manager6/qemu/ImportWizard.js @@ -0,0 +1,332 @@ +/*jslint confusion: true*/ +Ext.define('PVE.qemu.ImportWizard', { + extend: 'PVE.window.Wizard', + alias: 'widget.pveQemuImportWizard', + mixins: ['Proxmox.Mixin.CBind'], + + viewModel: { + data: { + nodename: '', + current: { + scsihw: '', + }, + }, + }, + + cbindData: { + nodename: undefined, + }, + + subject: gettext('Import Virtual Machine'), + + isImportVM: true, + + addDiskFunction: function() { + let me = this; + let wizard; + if (me.xtype === 'button') { + wizard = me.up('window'); + } else if (me.xtype === 'pveQemuImportWizard') { + wizard = me; + } + let multihd = wizard.down('pveQemuMultiHDInputPanel'); + multihd.addDiskFunction(); + }, + + items: [ + { + xtype: 'inputpanel', + title: gettext('Import'), + itemId: 'importInputpanel', + column1: [ + { + xtype: 'pveNodeSelector', + name: 'nodename', + cbind: { + selectCurNode: '{!nodename}', + preferredValue: '{nodename}', + }, + bind: { + value: '{nodename}', + }, + fieldLabel: gettext('Node'), + allowBlank: false, + onlineValidator: true, + }, + { + xtype: 'pveGuestIDSelector', + name: 'vmid', + guestType: 'qemu', + value: '', + loadNextFreeID: true, + validateExists: false, + }, + ], + column2: [ + { + xtype: 'label', + itemId: 'successTextfield', + hidden: true, + html: gettext('Manifest successfully uploaded'), + margin: '0 0 0 10', + }, + { + xtype: 'textfield', + itemId: 'server_ovf_manifest', + name: 'ovf_textfield', + emptyText: '/mnt/nfs/exported.ovf', + fieldLabel: 'Absolute path to .ovf manifest on your PVE host', + listeners: { + validitychange: function(_, isValid) { + let button = Ext.ComponentQuery.query('#load_remote_manifest_button').pop(); + button.setDisabled(!isValid); + }, + }, + validator: function(value) { + return (value && value.startsWith('/')) || gettext("Must start with /"); + }, + }, + { + xtype: 'proxmoxButton', + itemId: 'load_remote_manifest_button', + text: gettext('Load remote manifest'), + disabled: true, + handler: function() { + let inputpanel = this.up('#importInputpanel'); + let nodename = inputpanel.down('pveNodeSelector').getValue(); + // independent of onGetValues(), so that value of + // ovf_textfield can be removed for submit + let ovf_textfield_value = inputpanel.down('textfield[name=ovf_textfield]').getValue(); + let wizard = this.up('window'); + Proxmox.Utils.API2Request({ + url: '/nodes/' + nodename + '/readovf', + method: 'GET', + params: { + manifest: ovf_textfield_value, + }, + success: function(response) { + let ovfdata = response.result.data; + wizard.down('#vmNameTextfield').setValue(ovfdata.name); + wizard.down('#cpupanel').getViewModel().set('coreCount', ovfdata.cores); + wizard.down('#memorypanel').down('pveMemoryField').setValue(ovfdata.memory); + delete ovfdata.name; + delete ovfdata.cores; + delete ovfdata.memory; + delete ovfdata.digest; + let devices = Object.keys(ovfdata); // e.g. ide0, sata2 + let multihd = wizard.down('pveQemuMultiHDInputPanel'); + if (devices.length > 0) { + multihd.removeAllDisks(); + } + for (var device of devices) { + multihd.addDiskFunction(device, ovfdata[device]); + } + }, + failure: function(response, opts) { + console.warn("Failure of load manifest button"); + console.warn(response); + }, + }); + }, + }, + ], + onGetValues: function(values) { + delete values.server_ovf_manifest; + delete values.ovf_textfield; + return values; + }, + }, + { + xtype: 'inputpanel', + title: gettext('General'), + onlineHelp: 'qm_general_settings', + column1: [ + { + xtype: 'textfield', + name: 'name', + itemId: 'vmNameTextfield', + vtype: 'DnsName', + value: '', + fieldLabel: gettext('Name'), + allowBlank: true, + }, + ], + column2: [ + { + xtype: 'pvePoolSelector', + fieldLabel: gettext('Resource Pool'), + name: 'pool', + value: '', + allowBlank: true, + }, + ], + advancedColumn1: [ + { + xtype: 'proxmoxcheckbox', + name: 'onboot', + uncheckedValue: 0, + defaultValue: 0, + deleteDefaultValue: true, + fieldLabel: gettext('Start at boot'), + }, + ], + advancedColumn2: [ + { + xtype: 'textfield', + name: 'order', + defaultValue: '', + emptyText: 'any', + labelWidth: 120, + fieldLabel: gettext('Start/Shutdown order'), + }, + { + xtype: 'textfield', + name: 'up', + defaultValue: '', + emptyText: 'default', + labelWidth: 120, + fieldLabel: gettext('Startup delay'), + }, + { + xtype: 'textfield', + name: 'down', + defaultValue: '', + emptyText: 'default', + labelWidth: 120, + fieldLabel: gettext('Shutdown timeout'), + }, + ], + onGetValues: function(values) { + ['name', 'pool', 'onboot', 'agent'].forEach(function(field) { + if (!values[field]) { + delete values[field]; + } + }); + + var res = PVE.Parser.printStartup({ + order: values.order, + up: values.up, + down: values.down, + }); + + if (res) { + values.startup = res; + } + + delete values.order; + delete values.up; + delete values.down; + + return values; + }, + }, + { + xtype: 'pveQemuSystemPanel', + title: gettext('System'), + isCreate: true, + insideWizard: true, + }, + { + xtype: 'pveQemuMultiHDInputPanel', + title: gettext('Hard Disk'), + bind: { + nodename: '{nodename}', + }, + isCreate: true, + insideWizard: true, + }, + { + itemId: 'cpupanel', + xtype: 'pveQemuProcessorPanel', + insideWizard: true, + title: gettext('CPU'), + }, + { + itemId: 'memorypanel', + xtype: 'pveQemuMemoryPanel', + insideWizard: true, + title: gettext('Memory'), + }, + { + xtype: 'pveQemuNetworkInputPanel', + bind: { + nodename: '{nodename}', + }, + title: gettext('Network'), + insideWizard: true, + }, + { + title: gettext('Confirm'), + layout: 'fit', + items: [ + { + xtype: 'grid', + store: { + model: 'KeyValue', + sorters: [{ + property: 'key', + direction: 'ASC', + }], + }, + columns: [ + { header: 'Key', width: 150, dataIndex: 'key' }, + { header: 'Value', flex: 1, dataIndex: 'value' }, + ], + }, + ], + dockedItems: [ + { + xtype: 'proxmoxcheckbox', + name: 'start', + dock: 'bottom', + margin: '5 0 0 0', + boxLabel: gettext('Start after created'), + }, + ], + listeners: { + show: function(panel) { + var kv = this.up('window').getValues(); + var data = []; + Ext.Object.each(kv, function(key, value) { + if (key === 'delete') { // ignore + return; + } + data.push({ key: key, value: value }); + }); + + var summarystore = panel.down('grid').getStore(); + summarystore.suspendEvents(); + summarystore.removeAll(); + summarystore.add(data); + summarystore.sort(); + summarystore.resumeEvents(); + summarystore.fireEvent('refresh'); + }, + }, + onSubmit: function() { + var wizard = this.up('window'); + var params = wizard.getValues(); + + var nodename = params.nodename; + delete params.nodename; + delete params.delete; + if (Array.isArray(params.diskimages)) { + params.diskimages = params.diskimages.join(','); + } + + Proxmox.Utils.API2Request({ + url: `/nodes/${nodename}/qemu/${params.vmid}/importvm`, + waitMsgTarget: wizard, + method: 'POST', + params: params, + success: function() { + wizard.close(); + }, + failure: function(response) { + Ext.Msg.alert(gettext('Error'), response.htmlStatus); + }, + }); + }, + }, + ], +}); diff --git a/www/manager6/qemu/MultiHDEdit.js b/www/manager6/qemu/MultiHDEdit.js new file mode 100644 index 00000000..641a802f --- /dev/null +++ b/www/manager6/qemu/MultiHDEdit.js @@ -0,0 +1,277 @@ +Ext.define('PVE.qemu.MultiHDInputPanel', { + extend: 'Proxmox.panel.InputPanel', + alias: 'widget.pveQemuMultiHDInputPanel', + + insideWizard: false, + + hiddenDisks: [], + + leftColumnRatio: 0.25, + + column1: [ + { + // Adding to the HDInputPanelContainer below automatically adds + // items to this store + xtype: 'gridpanel', + scrollable: true, + store: { + xtype: 'store', + storeId: 'importwizard_diskstorage', + // Use the panel as id + // Panels have are objects and therefore unique + // E.g. while adding new panels 'device' is ambiguous + fields: ['device', 'panel'], + removeByPanel: function(panel) { + let recordIndex = this.findBy(record => record.data.panel === panel); + this.removeAt(recordIndex); + return recordIndex; + }, + }, + columns: [ + { + text: gettext('Target device'), + dataIndex: 'device', + flex: 1, + resizable: false, + }, + ], + listeners: { + select: function(_, record) { + this.up('pveQemuMultiHDInputPanel') + .down('#HDInputPanelContainer') + .setActiveItem(record.data.panel); + }, + }, + anchor: '100% 90%', + }, { + xtype: 'container', + layout: 'hbox', + center: true, + defaults: { + margin: '5', + xtype: 'button', + }, + items: [ + { + iconCls: 'fa fa-plus-circle', + itemId: 'addDisk', + handler: function(button) { + button.up('pveQemuMultiHDInputPanel').addDiskFunction(); + }, + }, { + iconCls: 'fa fa-trash-o', + itemId: 'removeDisk', + handler: function(button) { + button.up('pveQemuMultiHDInputPanel').removeCurrentDisk(); + }, + }, + ], + }, + ], + column2: [ + { + itemId: 'HDInputPanelContainer', + xtype: 'container', + layout: 'card', + items: [], + listeners: { + beforeRender: function() { + // Initial disk if none have been added by manifest yet + if (this.items.items.length === 0) { + this.addDiskFunction(); + } + }, + add: function(container, newPanel, index) { + let store = Ext.getStore('importwizard_diskstorage'); + store.add({ device: newPanel.getDevice(), panel: newPanel }); + container.setActiveItem(newPanel); + }, + remove: function(HDInputPanelContainer, HDInputPanel, eOpts) { + let store = Ext.getStore('importwizard_diskstorage'); + let indexOfRemoved = store.removeByPanel(HDInputPanel); + if (HDInputPanelContainer.items.getCount() > 0) { + HDInputPanelContainer.setActiveItem(indexOfRemoved - 1); + } + }, + }, + defaultItem: { + xtype: 'pveQemuHDInputPanel', + bind: { + nodename: '{nodename}', + }, + isCreate: true, + isImportVM: true, + returnSingleKey: true, + insideWizard: true, + setNodename: function(nodename) { + this.down('#hdstorage').setNodename(nodename); + this.down('#hdimage').setStorage(undefined, nodename); + this.down('#sourceStorageSelector').setNodename(nodename); + this.down('#sourceFileSelector').setNodename(nodename); + }, + listeners: { + // newHDInputPanel ... the defaultItem that has just been + // cloned and added into HDInputPnaleContainer parameter + // HDInputPanelContainer ... the container from column2 + // where all the new panels go into + added: function(newHDInputPanel, HDInputPanelContainer, pos) { + // The listeners cannot be added earlier, because its fields don't exist earlier + Ext.Array.each(this.down('pveControllerSelector') + .query('field'), function(field) { + field.on('change', function() { + // Note that one setValues in a controller + // selector makes one setValue in each of + // the two fields, so this listener fires + // two times in a row so to say e.g. + // changing controller selector from ide0 to + // sata1 makes ide0->sata0 and then + // sata0->sata1 + let store = Ext.getStore('importwizard_diskstorage'); + let controllerSelector = field.up('pveQemuHDInputPanel') + .down('pveControllerSelector'); + /* + * controller+device (ide0) might be + * ambiguous during creation => find by + * panel object instead + * + * There is no function that takes a + * function and returns the model directly + * => index & getAt + */ + let recordIndex = store.findBy(record => + record.data.panel === field.up('pveQemuHDInputPanel'), + ); + let newControllerAndId = controllerSelector.getValuesAsString(); + store.getAt(recordIndex).set('device', newControllerAndId); + }); + }, + ); + let wizard = this.up('pveQemuImportWizard'); + Ext.Array.each(this.query('field'), function(field) { + field.on('change', wizard.validcheck); + field.on('validitychange', wizard.validcheck); + }); + }, + }, + validator: function() { + var valid = true; + var fields = this.query('field, fieldcontainer'); + Ext.Array.each(fields, function(field) { + // Note: not all fielcontainer have isValid() + if (Ext.isFunction(field.isValid) && !field.isValid()) { + valid = false; + } + }); + return valid; + }, + }, + + // device ... device that the new disk should be assigned to, e.g. + // ide0, sata2 + // path ... if this is set to x then the disk will + // backed/imported from the path x, that is, the textfield will + // contain the value x + addDiskFunction(device, path) { + // creating directly removes binding => no storage found? + let item = Ext.clone(this.defaultItem); + let added = this.add(item); + // At this point the 'added' listener has fired and the fields + // in the variable added have the change listeners that update + // the store Therefore we can now set values only on the field + // and they will be updated in the store + if (path) { + added.down('#sourceRadioPath').setValue(true); + added.down('#sourcePathTextfield').setValue(path); + } else { + added.down('#sourceRadioStorage').setValue(true); + added.down('#sourceStorageSelector').setHidden(false); + added.down('#sourceFileSelector').setHidden(false); + added.down('#sourceFileSelector').enable(); + added.down('#sourceStorageSelector').enable(); + } + + let sp = Ext.state.Manager.getProvider(); + let advanced_checkbox = sp.get('proxmox-advanced-cb'); + added.setAdvancedVisible(advanced_checkbox); + + if (device) { + // This happens after the 'add' and 'added' listeners of the + // item/defaultItem clone/pveQemuHDInputPanel/added have fired + added.down('pveControllerSelector').setValue(device); + } + }, + removeCurrentDisk: function() { + let activePanel = this.getLayout().activeItem; // panel = disk + if (activePanel) { + this.remove(activePanel); + } + }, + }, + ], + + addDiskFunction: function(device, path) { + this.down('#HDInputPanelContainer').addDiskFunction(device, path); + }, + removeCurrentDisk: function() { + this.down('#HDInputPanelContainer').removeCurrentDisk(); + }, + removeAllDisks: function() { + let container = this.down('#HDInputPanelContainer'); + while (container.items.items.length > 0) { + container.removeCurrentDisk(); + } + }, + + beforeRender: function() { + let leftColumnPanel = this.items.get(0).items.get(0); + leftColumnPanel.setFlex(this.leftColumnRatio); + // any other panel because this has no height yet + let panelHeight = this.up('tabpanel').items.items[0].getHeight(); + leftColumnPanel.setHeight(panelHeight); + }, + + setNodename: function(nodename) { + this.nodename = nodename; + }, + + // Call with defined parameter or without + hasDuplicateDevices: function(values) { + if (!values) { + values = this.up('form').getValues(); + } + if (!Array.isArray(values.controller)) { + return false; + } + for (let i = 0; i < values.controller.length - 1; i++) { + for (let j = i+1; j < values.controller.length; j++) { + if (values.controller[i] === values.controller[j]) { + if (values.deviceid[i] === values.deviceid[j]) { + return true; + } + } + } + } + return false; + }, + + onGetValues: function(values) { + // Returning anything here would give wrong data in the form at the end + // of the wizrad Each HDInputPanel in this MultiHD panel already has a + // sufficient onGetValues() function for the form at the end of the + // wizard + if (this.hasDuplicateDevices(values)) { + Ext.Msg.alert(gettext('Error'), 'Equal target devices are forbidden. Make all unique!'); + } + }, + + validator: function() { + let inputpanels = this.down('#HDInputPanelContainer').items.getRange(); + if (inputpanels.some(panel => !panel.validator())) { + return false; + } + if (this.hasDuplicateDevices()) { + return false; + } + return true; + }, +}); diff --git a/www/manager6/window/Wizard.js b/www/manager6/window/Wizard.js index 8b930bbd..a3e3b690 100644 --- a/www/manager6/window/Wizard.js +++ b/www/manager6/window/Wizard.js @@ -261,6 +261,8 @@ Ext.define('PVE.window.Wizard', { }; field.on('change', validcheck); field.on('validitychange', validcheck); + // Make available for fields that get added later + me.validcheck = validcheck; }); }, }); -- 2.20.1 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] applied: [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path 2021-03-26 12:32 [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Dominic Jäger 2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger 2021-03-26 12:32 ` [pve-devel] [PATCH v7 manager] gui: Add import for disk & VM Dominic Jäger @ 2021-04-01 13:40 ` Thomas Lamprecht 2 siblings, 0 replies; 7+ messages in thread From: Thomas Lamprecht @ 2021-04-01 13:40 UTC (permalink / raw) To: Proxmox VE development discussion, Dominic Jäger On 26.03.21 13:32, Dominic Jäger wrote: > This is required to import from LVM storages, for example > > Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com> > --- > v6->v7: Feedback Fabian G > - Variables with _ instead of camelCase > - single if instead of if/else > > PVE/Storage.pm | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > applied, thanks! ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-04-01 13:40 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-03-26 12:32 [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Dominic Jäger 2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger 2021-03-31 15:12 ` Fabian Grünbichler 2021-04-01 10:19 ` Dominic Jäger 2021-04-01 11:30 ` Fabian Grünbichler 2021-03-26 12:32 ` [pve-devel] [PATCH v7 manager] gui: Add import for disk & VM Dominic Jäger 2021-04-01 13:40 ` [pve-devel] applied: [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Thomas Lamprecht
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox