* [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path
@ 2021-03-26 12:32 Dominic Jäger
2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger
` (2 more replies)
0 siblings, 3 replies; 7+ messages in thread
From: Dominic Jäger @ 2021-03-26 12:32 UTC (permalink / raw)
To: pve-devel
This is required to import from LVM storages, for example
Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com>
---
v6->v7: Feedback Fabian G
- Variables with _ instead of camelCase
- single if instead of if/else
PVE/Storage.pm | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/PVE/Storage.pm b/PVE/Storage.pm
index 18c03ec..38dffdb 100755
--- a/PVE/Storage.pm
+++ b/PVE/Storage.pm
@@ -609,22 +609,22 @@ sub path {
}
sub abs_filesystem_path {
- my ($cfg, $volid) = @_;
+ my ($cfg, $volid, $allow_blockdev) = @_;
my $path;
if (parse_volume_id ($volid, 1)) {
activate_volumes($cfg, [ $volid ]);
$path = PVE::Storage::path($cfg, $volid);
} else {
- if (-f $volid) {
+ if (-f $volid || ($allow_blockdev && -b $volid)) {
my $abspath = abs_path($volid);
if ($abspath && $abspath =~ m|^(/.+)$|) {
$path = $1; # untaint any path
}
}
}
-
- die "can't find file '$volid'\n" if !($path && -f $path);
+ die "can't find file '$volid'\n"
+ if !($path && (-f $path || ($allow_blockdev && -b $path)));
return $path;
}
--
2.20.1
^ permalink raw reply [flat|nested] 7+ messages in thread* [pve-devel] [PATCH v7 qemu-server] Add API for import wizards 2021-03-26 12:32 [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Dominic Jäger @ 2021-03-26 12:32 ` Dominic Jäger 2021-03-31 15:12 ` Fabian Grünbichler 2021-03-26 12:32 ` [pve-devel] [PATCH v7 manager] gui: Add import for disk & VM Dominic Jäger 2021-04-01 13:40 ` [pve-devel] applied: [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Thomas Lamprecht 2 siblings, 1 reply; 7+ messages in thread From: Dominic Jäger @ 2021-03-26 12:32 UTC (permalink / raw) To: pve-devel Extend qm importdisk/importovf functionality to the API. Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com> --- v6->v7: Feedback by Fabian G - Introduce a regex for the import syntax <storeid>:0 - Use parameter list instead of hash for import helper - More parsing, less string magic - More VM config digest checking - Create a schema format for diskimage source mapping - Preliminarily remove some boot parameter handling - Dare to really edit schema format subs for a cleaner solution - Whitespace, variable names, ... PVE/API2/Qemu.pm | 383 ++++++++++++++++++++++++++++++++++++++++- PVE/API2/Qemu/Makefile | 2 +- PVE/API2/Qemu/OVF.pm | 68 ++++++++ PVE/QemuServer.pm | 52 +++++- PVE/QemuServer/OVF.pm | 10 +- 5 files changed, 502 insertions(+), 13 deletions(-) create mode 100644 PVE/API2/Qemu/OVF.pm diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index e95ab13..2f50f38 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -45,7 +45,6 @@ BEGIN { } } -use Data::Dumper; # fixme: remove use base qw(PVE::RESTHandler); @@ -62,6 +61,7 @@ my $resolve_cdrom_alias = sub { }; my $NEW_DISK_RE = qr!^(([^/:\s]+):)?(\d+(\.\d+)?)$!; +my $IMPORT_DISK_RE = qr!^([^/:\s]+):0$!; my $check_storage_access = sub { my ($rpcenv, $authuser, $storecfg, $vmid, $settings, $default_storage) = @_; @@ -4377,4 +4377,385 @@ __PACKAGE__->register_method({ return PVE::QemuServer::Cloudinit::dump_cloudinit_config($conf, $param->{vmid}, $param->{type}); }}); +# Raise exception if $format is not supported by $storeid +my $check_format_is_supported = sub { + my ($format, $storeid, $storecfg) = @_; + die "storage ID parameter must be passed to the sub" if !$storeid; + die "storage configuration must be passed to the sub" if !$storecfg; + + return if !$format; + + my (undef, $valid_formats) = PVE::Storage::storage_default_format($storecfg, $storeid); + my $supported = grep { $_ eq $format } @$valid_formats; + + die "format '$format' is not supported on storage $storeid" if !$supported; +}; + +# storecfg ... PVE::Storage::config() +# vmid ... target VM ID +# vmconf ... target VM configuration +# source ... source image (volid or absolute path) +# target ... hash with +# storeid => storage ID +# format => disk format (optional) +# options => hash with device options (may or may not contain <storeid>:0) +# device => device where the disk is attached (for example, scsi3) (optional) +# +# returns ... volid of the allocated disk image (e.g. local-lvm:vm-100-disk-2) +my $import_disk_image = sub { + my ($storecfg, $vmid, $vmconf, $source, $target) = @_; + my $requested_format = $target->{format}; + my $storeid = $target->{storeid}; + + die "Source parameter is undefined!" if !defined $source; + $source = PVE::Storage::abs_filesystem_path($storecfg, $source, 1); + + eval { PVE::Storage::storage_config($storecfg, $storeid) }; + die "Error while importing disk image $source: $@\n" if $@; + + my $src_size = PVE::Storage::file_size_info($source); + die "Could not get file size of $source" if !defined($src_size); + + $check_format_is_supported->($requested_format, $storeid, $storecfg); + + my $dst_format = PVE::QemuServer::resolve_dst_disk_format( + $storecfg, + $storeid, + undef, + $requested_format, + ); + my $dst_volid = PVE::Storage::vdisk_alloc( + $storecfg, + $storeid, + $vmid, + $dst_format, + undef, + $src_size / 102, + ); + + print "Importing disk image '$source' as '$dst_volid'...\n"; + eval { + local $SIG{INT} = + local $SIG{TERM} = + local $SIG{QUIT} = + local $SIG{HUP} = + local $SIG{PIPE} = sub { die "Interrupted by signal $!\n"; }; + + my $zeroinit = PVE::Storage::volume_has_feature( + $storecfg, + 'sparseinit', + $dst_volid, + ); + PVE::Storage::activate_volumes($storecfg, [$dst_volid]); + PVE::QemuServer::qemu_img_convert( + $source, + $dst_volid, + $src_size, + undef, + $zeroinit, + ); + PVE::Storage::deactivate_volumes($storecfg, [$dst_volid]); + + }; + if (my $err = $@) { + eval { PVE::Storage::vdisk_free($storecfg, $dst_volid) }; + warn "Cleanup of $dst_volid failed: $@ \n" if $@; + + die "Importing disk '$source' failed: $err\n" if $err; + } + + $target->{options}->{file} = $dst_volid; + my $options_string = PVE::QemuServer::print_drive($target->{options}); + $target->{device} = PVE::QemuConfig->add_unused_volume($vmconf, $dst_volid) + if !$target->{device}; + + $update_vm_api->( + { + vmid => $vmid, + $target->{device} => $options_string, + skiplock => 1, + digest => $vmconf->{digest}, + }, + 1, + ); + + return $dst_volid; +}; + +__PACKAGE__->register_method ({ + name => 'importdisk', + path => '{vmid}/importdisk', + method => 'POST', + proxyto => 'node', + protected => 1, + description => "Import an external disk image into a VM. The image format ". + "has to be supported by qemu-img.", + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + vmid => get_standard_option('pve-vmid', + {completion => \&PVE::QemuServer::complete_vmid}), + source => { + description => "Disk image to import. Can be a volid ". + "(local:99/imageToImport.raw) or an absolute path on the server.", + type => 'string', + }, + device => { + type => 'string', + description => "Bus/Device type of the new disk (e.g. 'ide0', ". + "'scsi2'). Will add the image as unused disk if omitted.", + enum => [PVE::QemuServer::Drive::valid_drive_names()], + optional => 1, + }, + device_options => { + type => 'string', + description => "Options to set for the new disk (e.g. 'discard=on,backup=0')", + optional => 1, + requires => 'device', + }, + storage => get_standard_option('pve-storage-id', { + description => "The storage to which the image will be imported to.", + completion => \&PVE::QemuServer::complete_storage, + }), + format => { + type => 'string', + description => 'Target format.', + enum => [ 'raw', 'qcow2', 'vmdk' ], + optional => 1, + }, + digest => get_standard_option('pve-config-digest'), + }, + }, + returns => { type => 'string'}, + code => sub { + my ($param) = @_; + my $vmid = extract_param($param, 'vmid'); + my $node = extract_param($param, 'node'); + my $source = extract_param($param, 'source'); + my $digest = extract_param($param, 'digest'); + my $device_options = extract_param($param, 'device_options'); + my $device = extract_param($param, 'device'); + my $storeid = extract_param($param, 'storage'); + + my $rpcenv = PVE::RPCEnvironment::get(); + my $authuser = $rpcenv->get_user(); + my $storecfg = PVE::Storage::config(); + PVE::Storage::storage_config($storecfg, $storeid); + + + if ($device_options) { + # $device_options may or may not contain <storeid>:0 + my $parsed = PVE::QemuServer::Drive::parse_drive($device, $device_options); + if ($parsed) { + raise_param_exc({$device_options => "Invalid import syntax"}) + if !($parsed->{file} =~ $IMPORT_DISK_RE); + } else { + my $fake = "$storeid:0,$device_options"; + $parsed = PVE::QemuServer::Drive::parse_drive($device, $fake); + } + delete $parsed->{file}; + delete $parsed->{interface}; + delete $parsed->{index}; + $device_options = $parsed; + } + + # Format can be set explicitly "--format vmdk" + # or as part of device options "--device_options discard=on,format=vmdk" + my $format = extract_param($param, 'format'); + if ($device_options) { + raise_param_exc({format => "Format already specified in device_options!"}) + if $format && $device_options->{format}; + $format = $format || $device_options->{format}; # may be undefined + } + $check_format_is_supported->($format, $storeid, $storecfg); + + # quick checks before fork + lock + my $conf = PVE::QemuConfig->load_config($vmid); + PVE::QemuConfig->check_lock($conf); + PVE::Tools::assert_if_modified($conf->{digest}, $digest); + if ($device && $conf->{$device}) { + die "Could not import because device $device is already in ". + "use in VM $vmid. Choose a different device!"; + } + + my $worker = sub { + PVE::QemuConfig->lock_config($vmid, sub { + $conf = PVE::QemuConfig->load_config($vmid); + PVE::QemuConfig->check_lock($conf); + + PVE::Tools::assert_if_modified($conf->{digest}, $digest); + PVE::QemuConfig->set_lock($vmid, 'import'); + $conf = PVE::QemuConfig->load_config($vmid); + }); + + my $target = { + node => $node, + storeid => $storeid, + }; + $target->{format} = $format; + $target->{device} = $device; + $target->{options} = $device_options; + eval { $import_disk_image->($storecfg, $vmid, $conf, $source, $target) }; + my $err = $@; + eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; + warn $@ if $@; + die $err if $err; + }; + return $rpcenv->fork_worker('importdisk', $vmid, $authuser, $worker); + }}); + +__PACKAGE__->register_method({ + name => 'importvm', + path => '{vmid}/importvm', + method => 'POST', + description => "Import a VM from existing disk images.", + protected => 1, + proxyto => 'node', + parameters => { + additionalProperties => 0, + properties => PVE::QemuServer::json_config_properties( + { + node => get_standard_option('pve-node'), + vmid => get_standard_option('pve-vmid', { completion => + \&PVE::Cluster::complete_next_vmid }), + diskimage => { + description => "\\0 delimited mapping of devices to disk images. For " . + "example, scsi0=/mnt/nfs/image1.vmdk", + type => 'string', + format => 'device-image-pair-alist', + }, + start => { + optional => 1, + type => 'boolean', + default => 0, + description => "Start VM after it was imported successfully.", + }, + }), + }, + returns => { + type => 'string', + }, + code => sub { + my ($param) = @_; + my $node = extract_param($param, 'node'); + my $vmid = extract_param($param, 'vmid'); + my $diskimages_string = extract_param($param, 'diskimage'); + my $boot = extract_param($param, 'boot'); + my $start = extract_param($param, 'start'); + + my $rpcenv = PVE::RPCEnvironment::get(); + my $authuser = $rpcenv->get_user(); + my $storecfg = PVE::Storage::config(); + + PVE::Cluster::check_cfs_quorum(); + + my $import_param = {}; + foreach my $opt (keys %$param) { + next if $opt eq 'efidisk0'; + raise_param_exc({bootdisk => "Deprecated: Use --boot order= instead"}) + if $opt eq 'bootdisk'; + + if (PVE::QemuServer::Drive::is_valid_drivename($opt)) { + my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); + if ($drive->{file} =~ $IMPORT_DISK_RE) { + $import_param->{$opt} = $drive; + delete $param->{$opt}; + } + } + } + + my $diskimages = {}; + foreach my $pair (PVE::Tools::split_list($diskimages_string)) { + my ($device, $diskimage) = split('=', $pair); + $diskimages->{$device} = $diskimage; + raise_param_exc({ + $device => "Device '$device' not marked for import, " . + "but import source '$diskimage' specified", + }) if !defined($import_param->{$device}); + PVE::Storage::abs_filesystem_path($storecfg, $diskimage, 1); + } + + foreach my $device (keys %$import_param) { + raise_param_exc({ + $device => "Device '$device' marked for import, but no source given\n", + }) if !defined($diskimages->{$device}); + } + + eval { PVE::QemuConfig->create_and_lock_config($vmid, 0, 'import') }; + die "Unable to create config for VM import: $@" if $@; + + my $worker = sub { + my $reload_conf = sub { + my ($vmid) = @_; + my $conf = PVE::QemuConfig->load_config($vmid); + return $conf if PVE::QemuConfig->has_lock($conf, 'import'); + die "import lock in VM $vmid config file missing!"; + }; + + my $conf = $reload_conf->($vmid); + $update_vm_api->( + { + %$param, + node => $node, + vmid => $vmid, + skiplock => 1, + digest => $conf->{digest}, + }, + 1 + ); + + eval { + foreach my $device (keys %$import_param) { + $conf = $reload_conf->($vmid); + my $drive = $import_param->{$device}; + my $storeid = PVE::Storage::parse_volume_id($drive->{file}); + my $imported = $import_disk_image->( + $storecfg, + $vmid, + $conf, + $diskimages->{$device}, + { + storeid => $storeid, + format => $drive->{format}, + options => $drive, + device => $device, + }, + ); + } + }; + my $err = $@; + if ($err) { + eval { PVE::QemuServer::destroy_vm($storecfg, $vmid, 1) }; + warn "Could not destroy VM $vmid: $@" if "$@"; + + die "Import failed: $err"; + } + + $conf = $reload_conf->($vmid); + if (!$boot) { + my $bootdevs = PVE::QemuServer::get_default_bootdevices($conf); + $boot = PVE::QemuServer::print_bootorder($bootdevs); + } + $update_vm_api->( + { + node => $node, + vmid => $vmid, + boot => $boot, + skiplock => 1, + digest => $conf->{digest}, + }, + 1, + ); + + eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; + warn $@ if $@; + + PVE::QemuServer::vm_start($storecfg, $vmid) if $start; + }; + + return $rpcenv->fork_worker('importvm', $vmid, $authuser, $worker); + }}); + + 1; diff --git a/PVE/API2/Qemu/Makefile b/PVE/API2/Qemu/Makefile index 5d4abda..bdd4762 100644 --- a/PVE/API2/Qemu/Makefile +++ b/PVE/API2/Qemu/Makefile @@ -1,4 +1,4 @@ -SOURCES=Agent.pm CPU.pm Machine.pm +SOURCES=Agent.pm CPU.pm Machine.pm OVF.pm .PHONY: install install: diff --git a/PVE/API2/Qemu/OVF.pm b/PVE/API2/Qemu/OVF.pm new file mode 100644 index 0000000..bd6e90b --- /dev/null +++ b/PVE/API2/Qemu/OVF.pm @@ -0,0 +1,68 @@ +package PVE::API2::Qemu::OVF; + +use strict; +use warnings; + +use PVE::RESTHandler; +use PVE::JSONSchema qw(get_standard_option); +use PVE::QemuServer::OVF; + +use base qw(PVE::RESTHandler); + +__PACKAGE__->register_method ({ + name => 'index', + path => '', + method => 'GET', + proxyto => 'node', + description => "Read an .ovf manifest.", + parameters => { + additionalProperties => 0, + properties => { + node => get_standard_option('pve-node'), + manifest => { + description => ".ovf manifest", + type => 'string', + }, + }, + }, + returns => { + description => "VM config according to .ovf manifest and digest of manifest", + type => "object", + }, + returns => { + type => 'object', + additionalProperties => 1, + properties => PVE::QemuServer::json_ovf_properties({ + name => { + type => 'string', + optional => 1, + }, + cores => { + type => 'integer', + optional => 1, + }, + memory => { + type => 'integer', + optional => 1, + }, + }), + }, + code => sub { + my ($param) = @_; + + my $manifest = $param->{manifest}; + die "$manifest: non-existent or non-regular file\n" if (! -f $manifest); + + my $parsed = PVE::QemuServer::OVF::parse_ovf($manifest, 0, 1); + my $result; + $result->{cores} = $parsed->{qm}->{cores}; + $result->{name} = $parsed->{qm}->{name}; + $result->{memory} = $parsed->{qm}->{memory}; + my $disks = $parsed->{disks}; + foreach my $disk (@$disks) { + $result->{$disk->{disk_address}} = $disk->{backing_file}; + } + return $result; + }}); + +1; \ No newline at end of file diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm index 1c0b5c2..131c0b6 100644 --- a/PVE/QemuServer.pm +++ b/PVE/QemuServer.pm @@ -300,7 +300,7 @@ my $confdesc = { optional => 1, type => 'string', description => "Lock/unlock the VM.", - enum => [qw(backup clone create migrate rollback snapshot snapshot-delete suspending suspended)], + enum => [qw(backup clone create migrate rollback snapshot snapshot-delete suspending suspended import)], }, cpulimit => { optional => 1, @@ -985,19 +985,41 @@ PVE::JSONSchema::register_format('pve-volume-id-or-qm-path', \&verify_volume_id_ sub verify_volume_id_or_qm_path { my ($volid, $noerr) = @_; - if ($volid eq 'none' || $volid eq 'cdrom' || $volid =~ m|^/|) { - return $volid; - } + return $volid eq 'none' || $volid eq 'cdrom' ? + $volid : + verify_volume_id_or_absolute_path($volid, $noerr); +} + +PVE::JSONSchema::register_format('pve-volume-id-or-absolute-path', \&verify_volume_id_or_absolute_path); +sub verify_volume_id_or_absolute_path { + my ($volid, $noerr) = @_; + + return $volid if $volid =~ m|^/|; - # if its neither 'none' nor 'cdrom' nor a path, check if its a volume-id $volid = eval { PVE::JSONSchema::check_format('pve-volume-id', $volid, '') }; if ($@) { - return if $noerr; + return undef if $noerr; die $@; } return $volid; } +PVE::JSONSchema::register_format('device-image-pair', \&verify_device_image_pair); +sub verify_device_image_pair { + my ($pair, $noerr) = @_; + + my $error = sub { + return undef if $noerr; + die $@; + }; + + my ($device, $image) = split('=', $pair); + $error->("Invalid device '$device'") if !PVE::QemuServer::Drive::is_valid_drivename($device); + $error->("Invalid image '$image'") if !verify_volume_id_or_absolute_path($image); + + return $pair; +} + my $usb_fmt = { host => { default_key => 1, @@ -2030,6 +2052,22 @@ sub json_config_properties { return $prop; } +# Properties that we can read from an OVF file +sub json_ovf_properties { + my $prop = shift; + + foreach my $device ( PVE::QemuServer::Drive::valid_drive_names()) { + $prop->{$device} = { + type => 'string', + format => 'pve-volume-id-or-absolute-path', + description => "Disk image that gets imported to $device", + optional => 1, + }; + } + + return $prop; +} + # return copy of $confdesc_cloudinit to generate documentation sub cloudinit_config_properties { @@ -6748,7 +6786,7 @@ sub qemu_img_convert { $src_path = PVE::Storage::path($storecfg, $src_volid, $snapname); $src_is_iscsi = ($src_path =~ m|^iscsi://|); $cachemode = 'none' if $src_scfg->{type} eq 'zfspool'; - } elsif (-f $src_volid) { + } elsif (-f $src_volid || -b $src_volid) { $src_path = $src_volid; if ($src_path =~ m/\.($PVE::QemuServer::Drive::QEMU_FORMAT_RE)$/) { $src_format = $1; diff --git a/PVE/QemuServer/OVF.pm b/PVE/QemuServer/OVF.pm index c76c199..48146e9 100644 --- a/PVE/QemuServer/OVF.pm +++ b/PVE/QemuServer/OVF.pm @@ -87,7 +87,7 @@ sub id_to_pve { # returns two references, $qm which holds qm.conf style key/values, and \@disks sub parse_ovf { - my ($ovf, $debug) = @_; + my ($ovf, $debug, $manifest_only) = @_; my $dom = XML::LibXML->load_xml(location => $ovf, no_blanks => 1); @@ -220,9 +220,11 @@ ovf:Item[rasd:InstanceID='%s']/rasd:ResourceType", $controller_id); die "error parsing $filepath, file seems not to exist at $backing_file_path\n"; } - my $virtual_size; - if ( !($virtual_size = PVE::Storage::file_size_info($backing_file_path)) ) { - die "error parsing $backing_file_path, size seems to be $virtual_size\n"; + my $virtual_size = undef; + if (!$manifest_only) { # Not possible if manifest is uploaded in web gui + if ( !($virtual_size = PVE::Storage::file_size_info($backing_file_path)) ) { + die "error parsing $backing_file_path: Could not get file size info: $@\n"; + } } $pve_disk = { -- 2.20.1 ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pve-devel] [PATCH v7 qemu-server] Add API for import wizards 2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger @ 2021-03-31 15:12 ` Fabian Grünbichler 2021-04-01 10:19 ` Dominic Jäger 0 siblings, 1 reply; 7+ messages in thread From: Fabian Grünbichler @ 2021-03-31 15:12 UTC (permalink / raw) To: Proxmox VE development discussion this is starting to shape up nicely. as promised, I now took a stab at (roughly!) integrating this into our regular flow (see diff below): - IMPORT_DISK_RE now uses -1, as 0 actually can be mismatched by NEW_DISK_RE - the actual import happens in create_disks - only the basic checks (match of "marked for import" with "import sources") happen early on - the target storage and source volume are permission checked now - importvm is dropped in favor of calling create_vm with import_sources - update_vm_async now also supports import_sources the last two IMHO have some nice benefits: - we can now easily mix and match importing and newly allocating blank disks in a single create call (allowing us to use a single GUI wizard as well if we want) - create_vm and importvm don't have to duplicate all the surrounding logic (or have strange contortions to make one call the other) - we could likely drop the separate import_disk API call, and let the `importdisk` CLI command prepare parameters for a regular VM config update I'm sure I've missed some corner cases, as I've only tested create_vm with importing, and not the other newly exposed APIs. stat shows how much boilerplace/duplication this removes, although there is probably even more potential here since `create_vm` partly duplicates the `update_vm_api` sub that ends up calling `create_disks`: PVE/API2/Qemu.pm | 481 ++++++++++++++++++++----------------------------------- 1 file changed, 171 insertions(+), 310 deletions(-) the original patch was +381 for that file, so in total we are now at +242 instead. could you take a look and see if I missed anything fundamental? -----8<----- diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 2f50f38..41e1ab7 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -60,8 +60,19 @@ my $resolve_cdrom_alias = sub { } }; +my $parse_import_sources = sub { + my $param = shift; + my $import = {}; + foreach my $pair (PVE::Tools::split_list($param)) { + my ($device, $diskimage) = split('=', $pair); + $import->{$device} = $diskimage; + } + + return $import; +}; + my $NEW_DISK_RE = qr!^(([^/:\s]+):)?(\d+(\.\d+)?)$!; -my $IMPORT_DISK_RE = qr!^([^/:\s]+):0$!; +my $IMPORT_DISK_RE = qr!^([^/:\s]+):-1$!; my $check_storage_access = sub { my ($rpcenv, $authuser, $storecfg, $vmid, $settings, $default_storage) = @_; @@ -84,6 +95,12 @@ my $check_storage_access = sub { my $scfg = PVE::Storage::storage_config($storecfg, $storeid); raise_param_exc({ storage => "storage '$storeid' does not support vm images"}) if !$scfg->{content}->{images}; + } elsif ($volid =~ $IMPORT_DISK_RE) { + my $storeid = $1; + $rpcenv->check($authuser, "/storage/$storeid", ['Datastore.AllocateSpace']); + my $scfg = PVE::Storage::storage_config($storecfg, $storeid); + raise_param_exc({ storage => "storage '$storeid' does not support vm images"}) + if !$scfg->{content}->{images}; } else { PVE::Storage::check_volume_access($rpcenv, $authuser, $storecfg, $vmid, $volid); } @@ -91,6 +108,13 @@ my $check_storage_access = sub { $rpcenv->check($authuser, "/storage/$settings->{vmstatestorage}", ['Datastore.AllocateSpace']) if defined($settings->{vmstatestorage}); + + if (defined($settings->{import_sources})) { + my $images = $parse_import_sources->($settings->{import_sources}); + foreach my $source_image (values %$images) { + PVE::Storage::check_volume_access($rpcenv, $authuser, $storecfg, $vmid, $source_image); + } + } }; my $check_storage_access_clone = sub { @@ -133,10 +157,25 @@ my $check_storage_access_clone = sub { return $sharedvm; }; +# Raise exception if $format is not supported by $storeid +my $check_format_is_supported = sub { + my ($format, $storeid, $storecfg) = @_; + die "storage ID parameter must be passed to the sub" if !$storeid; + die "storage configuration must be passed to the sub" if !$storecfg; + + return if !$format; + + my (undef, $valid_formats) = PVE::Storage::storage_default_format($storecfg, $storeid); + my $supported = grep { $_ eq $format } @$valid_formats; + + die "format '$format' is not supported on storage $storeid" if !$supported; +}; + + # Note: $pool is only needed when creating a VM, because pool permissions # are automatically inherited if VM already exists inside a pool. my $create_disks = sub { - my ($rpcenv, $authuser, $conf, $arch, $storecfg, $vmid, $pool, $settings, $default_storage) = @_; + my ($rpcenv, $authuser, $conf, $arch, $storecfg, $vmid, $pool, $settings, $default_storage, $import) = @_; my $vollist = []; @@ -192,6 +231,69 @@ my $create_disks = sub { $disk->{size} = PVE::Tools::convert_size($size, 'kb' => 'b'); delete $disk->{format}; # no longer needed $res->{$ds} = PVE::QemuServer::print_drive($disk); + } elsif ($volid =~ $IMPORT_DISK_RE) { + my $target_storage = $1; + + my $source = $import->{$ds}; + die "cannot import '$ds', no import source defined\n" if !$source; + $source = PVE::Storage::abs_filesystem_path($storecfg, $source, 1); + my $src_size = PVE::Storage::file_size_info($source); + die "Could not get file size of $source" if !defined($src_size); + + $check_format_is_supported->($disk->{format}, $storeid, $storecfg); + + my $dst_format = PVE::QemuServer::resolve_dst_disk_format( + $storecfg, + $storeid, + undef, + $disk->{format}, + ); + my $dst_volid = PVE::Storage::vdisk_alloc( + $storecfg, + $storeid, + $vmid, + $dst_format, + undef, + PVE::Tools::convert_size($src_size, 'b' => 'kb'), + ); + + print "Importing disk image '$source' as '$dst_volid'...\n"; + eval { + local $SIG{INT} = + local $SIG{TERM} = + local $SIG{QUIT} = + local $SIG{HUP} = + local $SIG{PIPE} = sub { die "Interrupted by signal $!\n"; }; + + my $zeroinit = PVE::Storage::volume_has_feature( + $storecfg, + 'sparseinit', + $dst_volid, + ); + PVE::Storage::activate_volumes($storecfg, [$dst_volid]); + PVE::QemuServer::qemu_img_convert( + $source, + $dst_volid, + $src_size, + undef, + $zeroinit, + ); + PVE::Storage::deactivate_volumes($storecfg, [$dst_volid]); + + }; + if (my $err = $@) { + eval { PVE::Storage::vdisk_free($storecfg, $dst_volid) }; + warn "Cleanup of $dst_volid failed: $@ \n" if $@; + + die "Importing disk '$source' failed: $err\n" if $err; + } + push @$vollist, $dst_volid; + $disk->{file} = $dst_volid; + if ($ds !~ /^unused\d+$/) { + $disk->{size} = $src_size; + delete $disk->{format}; # no longer needed + } + $res->{$ds} = PVE::QemuServer::print_drive($disk); } else { PVE::Storage::check_volume_access($rpcenv, $authuser, $storecfg, $vmid, $volid); @@ -218,7 +320,7 @@ my $create_disks = sub { } }; - eval { PVE::QemuConfig->foreach_volume($settings, $code); }; + eval { PVE::QemuConfig->foreach_volume_full($settings, { include_unused => 1 }, $code); }; # free allocated images on error if (my $err = $@) { @@ -544,6 +646,13 @@ __PACKAGE__->register_method({ default => 0, description => "Start VM after it was created successfully.", }, + import_sources => { + description => "\\0 delimited mapping of devices to disk images to import." . + "For example, scsi0=/mnt/nfs/image1.vmdk", + type => 'string', + format => 'device-image-pair-alist', + optional => 1, + }, }), }, returns => { @@ -608,21 +717,34 @@ __PACKAGE__->register_method({ &$check_cpu_model_access($rpcenv, $authuser, $param); + my $import_devices = $parse_import_sources->($param->{import_sources}); + foreach my $opt (keys %$param) { if (PVE::QemuServer::is_valid_drivename($opt)) { my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); raise_param_exc({ $opt => "unable to parse drive options" }) if !$drive; + raise_param_exc({ $opt => "not marked for import, but import source defined" }) + if $drive->{file} !~ $IMPORT_DISK_RE && $import_devices->{$opt}; + raise_param_exc({ $opt => "marked for import, but no import source defined" }) + if $drive->{file} =~ $IMPORT_DISK_RE && !$import_devices->{$opt}; PVE::QemuServer::cleanup_drive_path($opt, $storecfg, $drive); $param->{$opt} = PVE::QemuServer::print_drive($drive); } } + foreach my $opt (keys %$import_devices) { + raise_param_exc({ import_sources => "$opt not marked for import, but import source defined" }) + if !defined($param->{$opt}); + + } PVE::QemuServer::add_random_macs($param); } else { my $keystr = join(' ', keys %$param); raise_param_exc({ archive => "option conflicts with other options ($keystr)"}) if $keystr; + raise_param_exc({ import_sources => "cannot import existing disk and restore backup." }) if $param->{import_sources}; + if ($archive eq '-') { die "pipe requires cli environment\n" if $rpcenv->{type} ne 'cli'; @@ -690,10 +812,11 @@ __PACKAGE__->register_method({ my $realcmd = sub { my $conf = $param; my $arch = PVE::QemuServer::get_vm_arch($conf); + my $import = $parse_import_sources->(extract_param($param, "import_sources")); my $vollist = []; eval { - $vollist = &$create_disks($rpcenv, $authuser, $conf, $arch, $storecfg, $vmid, $pool, $param, $storage); + $vollist = &$create_disks($rpcenv, $authuser, $conf, $arch, $storecfg, $vmid, $pool, $param, $storage, $import); if (!$conf->{boot}) { my $devs = PVE::QemuServer::get_default_bootdevices($conf); @@ -1163,11 +1286,17 @@ my $update_vm_api = sub { die "cannot add non-replicatable volume to a replicated VM\n"; }; + my $import_devices = $parse_import_sources->($param->{import_sources}); + foreach my $opt (keys %$param) { if (PVE::QemuServer::is_valid_drivename($opt)) { # cleanup drive path my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); raise_param_exc({ $opt => "unable to parse drive options" }) if !$drive; + raise_param_exc({ $opt => "not marked for import, but import source defined" }) + if $drive->{file} !~ $IMPORT_DISK_RE && $import_devices->{$opt}; + raise_param_exc({ $opt => "marked for import, but no import source defined" }) + if $drive->{file} =~ $IMPORT_DISK_RE && !$import_devices->{$opt}; PVE::QemuServer::cleanup_drive_path($opt, $storecfg, $drive); $check_replication->($drive); $param->{$opt} = PVE::QemuServer::print_drive($drive); @@ -1185,12 +1314,20 @@ my $update_vm_api = sub { } } + foreach my $opt (keys %$import_devices) { + raise_param_exc({ import_sources => "$opt not marked for import, but import source defined" }) + if !defined($param->{$opt}); + + } + &$check_vm_modify_config_perm($rpcenv, $authuser, $vmid, undef, [@delete]); &$check_vm_modify_config_perm($rpcenv, $authuser, $vmid, undef, [keys %$param]); &$check_storage_access($rpcenv, $authuser, $storecfg, $vmid, $param); + delete $param->{import_sources}; + my $updatefn = sub { my $conf = PVE::QemuConfig->load_config($vmid); @@ -1332,7 +1469,7 @@ my $update_vm_api = sub { PVE::QemuServer::vmconfig_register_unused_drive($storecfg, $vmid, $conf, PVE::QemuServer::parse_drive($opt, $conf->{pending}->{$opt})) if defined($conf->{pending}->{$opt}); - &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}); + &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, {$opt => $import_devices->{$opt}}); } elsif ($opt =~ m/^serial\d+/) { if ((!defined($conf->{$opt}) || $conf->{$opt} eq 'socket') && $param->{$opt} eq 'socket') { $rpcenv->check_vm_perm($authuser, $vmid, undef, ['VM.Config.HWType']); @@ -1476,6 +1613,13 @@ __PACKAGE__->register_method({ optional => 1, requires => 'delete', }, + import_sources => { + description => "\\0 delimited mapping of devices to disk images to import." . + "For example, scsi0=/mnt/nfs/image1.vmdk", + type => 'string', + format => 'device-image-pair-alist', + optional => 1, + }, digest => { type => 'string', description => 'Prevent changes if current configuration file has different SHA1 digest. This can be used to prevent concurrent modifications.', @@ -4377,111 +4521,6 @@ __PACKAGE__->register_method({ return PVE::QemuServer::Cloudinit::dump_cloudinit_config($conf, $param->{vmid}, $param->{type}); }}); -# Raise exception if $format is not supported by $storeid -my $check_format_is_supported = sub { - my ($format, $storeid, $storecfg) = @_; - die "storage ID parameter must be passed to the sub" if !$storeid; - die "storage configuration must be passed to the sub" if !$storecfg; - - return if !$format; - - my (undef, $valid_formats) = PVE::Storage::storage_default_format($storecfg, $storeid); - my $supported = grep { $_ eq $format } @$valid_formats; - - die "format '$format' is not supported on storage $storeid" if !$supported; -}; - -# storecfg ... PVE::Storage::config() -# vmid ... target VM ID -# vmconf ... target VM configuration -# source ... source image (volid or absolute path) -# target ... hash with -# storeid => storage ID -# format => disk format (optional) -# options => hash with device options (may or may not contain <storeid>:0) -# device => device where the disk is attached (for example, scsi3) (optional) -# -# returns ... volid of the allocated disk image (e.g. local-lvm:vm-100-disk-2) -my $import_disk_image = sub { - my ($storecfg, $vmid, $vmconf, $source, $target) = @_; - my $requested_format = $target->{format}; - my $storeid = $target->{storeid}; - - die "Source parameter is undefined!" if !defined $source; - $source = PVE::Storage::abs_filesystem_path($storecfg, $source, 1); - - eval { PVE::Storage::storage_config($storecfg, $storeid) }; - die "Error while importing disk image $source: $@\n" if $@; - - my $src_size = PVE::Storage::file_size_info($source); - die "Could not get file size of $source" if !defined($src_size); - - $check_format_is_supported->($requested_format, $storeid, $storecfg); - - my $dst_format = PVE::QemuServer::resolve_dst_disk_format( - $storecfg, - $storeid, - undef, - $requested_format, - ); - my $dst_volid = PVE::Storage::vdisk_alloc( - $storecfg, - $storeid, - $vmid, - $dst_format, - undef, - $src_size / 102, - ); - - print "Importing disk image '$source' as '$dst_volid'...\n"; - eval { - local $SIG{INT} = - local $SIG{TERM} = - local $SIG{QUIT} = - local $SIG{HUP} = - local $SIG{PIPE} = sub { die "Interrupted by signal $!\n"; }; - - my $zeroinit = PVE::Storage::volume_has_feature( - $storecfg, - 'sparseinit', - $dst_volid, - ); - PVE::Storage::activate_volumes($storecfg, [$dst_volid]); - PVE::QemuServer::qemu_img_convert( - $source, - $dst_volid, - $src_size, - undef, - $zeroinit, - ); - PVE::Storage::deactivate_volumes($storecfg, [$dst_volid]); - - }; - if (my $err = $@) { - eval { PVE::Storage::vdisk_free($storecfg, $dst_volid) }; - warn "Cleanup of $dst_volid failed: $@ \n" if $@; - - die "Importing disk '$source' failed: $err\n" if $err; - } - - $target->{options}->{file} = $dst_volid; - my $options_string = PVE::QemuServer::print_drive($target->{options}); - $target->{device} = PVE::QemuConfig->add_unused_volume($vmconf, $dst_volid) - if !$target->{device}; - - $update_vm_api->( - { - vmid => $vmid, - $target->{device} => $options_string, - skiplock => 1, - digest => $vmconf->{digest}, - }, - 1, - ); - - return $dst_volid; -}; - __PACKAGE__->register_method ({ name => 'importdisk', path => '{vmid}/importdisk', @@ -4544,22 +4583,25 @@ __PACKAGE__->register_method ({ PVE::Storage::storage_config($storecfg, $storeid); - if ($device_options) { - # $device_options may or may not contain <storeid>:0 - my $parsed = PVE::QemuServer::Drive::parse_drive($device, $device_options); - if ($parsed) { - raise_param_exc({$device_options => "Invalid import syntax"}) - if !($parsed->{file} =~ $IMPORT_DISK_RE); - } else { - my $fake = "$storeid:0,$device_options"; - $parsed = PVE::QemuServer::Drive::parse_drive($device, $fake); - } - delete $parsed->{file}; - delete $parsed->{interface}; - delete $parsed->{index}; - $device_options = $parsed; + if (!$device_options) { + $device_options = "$storeid:0"; + } + + # $device_options may or may not contain <storeid>:0 + my $parsed = PVE::QemuServer::Drive::parse_drive($device, $device_options); + + if ($parsed) { + raise_param_exc({$device_options => "Invalid import syntax"}) + if !($parsed->{file} =~ $IMPORT_DISK_RE); + } else { + my $fake = "$storeid:0,$device_options"; + $parsed = PVE::QemuServer::Drive::parse_drive($device, $fake); } + delete $parsed->{interface}; + delete $parsed->{index}; + $device_options = $parsed; + # Format can be set explicitly "--format vmdk" # or as part of device options "--device_options discard=on,format=vmdk" my $format = extract_param($param, 'format'); @@ -4570,192 +4612,11 @@ __PACKAGE__->register_method ({ } $check_format_is_supported->($format, $storeid, $storecfg); - # quick checks before fork + lock - my $conf = PVE::QemuConfig->load_config($vmid); - PVE::QemuConfig->check_lock($conf); - PVE::Tools::assert_if_modified($conf->{digest}, $digest); - if ($device && $conf->{$device}) { - die "Could not import because device $device is already in ". - "use in VM $vmid. Choose a different device!"; - } - - my $worker = sub { - PVE::QemuConfig->lock_config($vmid, sub { - $conf = PVE::QemuConfig->load_config($vmid); - PVE::QemuConfig->check_lock($conf); - - PVE::Tools::assert_if_modified($conf->{digest}, $digest); - PVE::QemuConfig->set_lock($vmid, 'import'); - $conf = PVE::QemuConfig->load_config($vmid); - }); - - my $target = { - node => $node, - storeid => $storeid, - }; - $target->{format} = $format; - $target->{device} = $device; - $target->{options} = $device_options; - eval { $import_disk_image->($storecfg, $vmid, $conf, $source, $target) }; - my $err = $@; - eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; - warn $@ if $@; - die $err if $err; - }; - return $rpcenv->fork_worker('importdisk', $vmid, $authuser, $worker); + return $update_vm_api->({ + $device => PVE::QemuServer::Drive::print_drive($device_options), + import_sources => "$device=$source", + digest => $digest, + }); }}); -__PACKAGE__->register_method({ - name => 'importvm', - path => '{vmid}/importvm', - method => 'POST', - description => "Import a VM from existing disk images.", - protected => 1, - proxyto => 'node', - parameters => { - additionalProperties => 0, - properties => PVE::QemuServer::json_config_properties( - { - node => get_standard_option('pve-node'), - vmid => get_standard_option('pve-vmid', { completion => - \&PVE::Cluster::complete_next_vmid }), - diskimage => { - description => "\\0 delimited mapping of devices to disk images. For " . - "example, scsi0=/mnt/nfs/image1.vmdk", - type => 'string', - format => 'device-image-pair-alist', - }, - start => { - optional => 1, - type => 'boolean', - default => 0, - description => "Start VM after it was imported successfully.", - }, - }), - }, - returns => { - type => 'string', - }, - code => sub { - my ($param) = @_; - my $node = extract_param($param, 'node'); - my $vmid = extract_param($param, 'vmid'); - my $diskimages_string = extract_param($param, 'diskimage'); - my $boot = extract_param($param, 'boot'); - my $start = extract_param($param, 'start'); - - my $rpcenv = PVE::RPCEnvironment::get(); - my $authuser = $rpcenv->get_user(); - my $storecfg = PVE::Storage::config(); - - PVE::Cluster::check_cfs_quorum(); - - my $import_param = {}; - foreach my $opt (keys %$param) { - next if $opt eq 'efidisk0'; - raise_param_exc({bootdisk => "Deprecated: Use --boot order= instead"}) - if $opt eq 'bootdisk'; - - if (PVE::QemuServer::Drive::is_valid_drivename($opt)) { - my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); - if ($drive->{file} =~ $IMPORT_DISK_RE) { - $import_param->{$opt} = $drive; - delete $param->{$opt}; - } - } - } - - my $diskimages = {}; - foreach my $pair (PVE::Tools::split_list($diskimages_string)) { - my ($device, $diskimage) = split('=', $pair); - $diskimages->{$device} = $diskimage; - raise_param_exc({ - $device => "Device '$device' not marked for import, " . - "but import source '$diskimage' specified", - }) if !defined($import_param->{$device}); - PVE::Storage::abs_filesystem_path($storecfg, $diskimage, 1); - } - - foreach my $device (keys %$import_param) { - raise_param_exc({ - $device => "Device '$device' marked for import, but no source given\n", - }) if !defined($diskimages->{$device}); - } - - eval { PVE::QemuConfig->create_and_lock_config($vmid, 0, 'import') }; - die "Unable to create config for VM import: $@" if $@; - - my $worker = sub { - my $reload_conf = sub { - my ($vmid) = @_; - my $conf = PVE::QemuConfig->load_config($vmid); - return $conf if PVE::QemuConfig->has_lock($conf, 'import'); - die "import lock in VM $vmid config file missing!"; - }; - - my $conf = $reload_conf->($vmid); - $update_vm_api->( - { - %$param, - node => $node, - vmid => $vmid, - skiplock => 1, - digest => $conf->{digest}, - }, - 1 - ); - - eval { - foreach my $device (keys %$import_param) { - $conf = $reload_conf->($vmid); - my $drive = $import_param->{$device}; - my $storeid = PVE::Storage::parse_volume_id($drive->{file}); - my $imported = $import_disk_image->( - $storecfg, - $vmid, - $conf, - $diskimages->{$device}, - { - storeid => $storeid, - format => $drive->{format}, - options => $drive, - device => $device, - }, - ); - } - }; - my $err = $@; - if ($err) { - eval { PVE::QemuServer::destroy_vm($storecfg, $vmid, 1) }; - warn "Could not destroy VM $vmid: $@" if "$@"; - - die "Import failed: $err"; - } - - $conf = $reload_conf->($vmid); - if (!$boot) { - my $bootdevs = PVE::QemuServer::get_default_bootdevices($conf); - $boot = PVE::QemuServer::print_bootorder($bootdevs); - } - $update_vm_api->( - { - node => $node, - vmid => $vmid, - boot => $boot, - skiplock => 1, - digest => $conf->{digest}, - }, - 1, - ); - - eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; - warn $@ if $@; - - PVE::QemuServer::vm_start($storecfg, $vmid) if $start; - }; - - return $rpcenv->fork_worker('importvm', $vmid, $authuser, $worker); - }}); - - 1; ----->8----- On March 26, 2021 1:32 pm, Dominic Jäger wrote: > Extend qm importdisk/importovf functionality to the API. > > Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com> > > --- > v6->v7: Feedback by Fabian G > - Introduce a regex for the import syntax <storeid>:0 > - Use parameter list instead of hash for import helper > - More parsing, less string magic > - More VM config digest checking > - Create a schema format for diskimage source mapping > - Preliminarily remove some boot parameter handling > - Dare to really edit schema format subs for a cleaner solution > - Whitespace, variable names, ... > > PVE/API2/Qemu.pm | 383 ++++++++++++++++++++++++++++++++++++++++- > PVE/API2/Qemu/Makefile | 2 +- > PVE/API2/Qemu/OVF.pm | 68 ++++++++ > PVE/QemuServer.pm | 52 +++++- > PVE/QemuServer/OVF.pm | 10 +- > 5 files changed, 502 insertions(+), 13 deletions(-) > create mode 100644 PVE/API2/Qemu/OVF.pm > > diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm > index e95ab13..2f50f38 100644 > --- a/PVE/API2/Qemu.pm > +++ b/PVE/API2/Qemu.pm > @@ -45,7 +45,6 @@ BEGIN { > } > } > > -use Data::Dumper; # fixme: remove > > use base qw(PVE::RESTHandler); > > @@ -62,6 +61,7 @@ my $resolve_cdrom_alias = sub { > }; > > my $NEW_DISK_RE = qr!^(([^/:\s]+):)?(\d+(\.\d+)?)$!; > +my $IMPORT_DISK_RE = qr!^([^/:\s]+):0$!; > my $check_storage_access = sub { > my ($rpcenv, $authuser, $storecfg, $vmid, $settings, $default_storage) = @_; > > @@ -4377,4 +4377,385 @@ __PACKAGE__->register_method({ > return PVE::QemuServer::Cloudinit::dump_cloudinit_config($conf, $param->{vmid}, $param->{type}); > }}); > > +# Raise exception if $format is not supported by $storeid > +my $check_format_is_supported = sub { > + my ($format, $storeid, $storecfg) = @_; > + die "storage ID parameter must be passed to the sub" if !$storeid; > + die "storage configuration must be passed to the sub" if !$storecfg; > + > + return if !$format; > + > + my (undef, $valid_formats) = PVE::Storage::storage_default_format($storecfg, $storeid); > + my $supported = grep { $_ eq $format } @$valid_formats; > + > + die "format '$format' is not supported on storage $storeid" if !$supported; > +}; > + > +# storecfg ... PVE::Storage::config() > +# vmid ... target VM ID > +# vmconf ... target VM configuration > +# source ... source image (volid or absolute path) > +# target ... hash with > +# storeid => storage ID > +# format => disk format (optional) > +# options => hash with device options (may or may not contain <storeid>:0) > +# device => device where the disk is attached (for example, scsi3) (optional) > +# > +# returns ... volid of the allocated disk image (e.g. local-lvm:vm-100-disk-2) > +my $import_disk_image = sub { > + my ($storecfg, $vmid, $vmconf, $source, $target) = @_; > + my $requested_format = $target->{format}; > + my $storeid = $target->{storeid}; > + > + die "Source parameter is undefined!" if !defined $source; > + $source = PVE::Storage::abs_filesystem_path($storecfg, $source, 1); > + > + eval { PVE::Storage::storage_config($storecfg, $storeid) }; > + die "Error while importing disk image $source: $@\n" if $@; > + > + my $src_size = PVE::Storage::file_size_info($source); > + die "Could not get file size of $source" if !defined($src_size); > + > + $check_format_is_supported->($requested_format, $storeid, $storecfg); > + > + my $dst_format = PVE::QemuServer::resolve_dst_disk_format( > + $storecfg, > + $storeid, > + undef, > + $requested_format, > + ); > + my $dst_volid = PVE::Storage::vdisk_alloc( > + $storecfg, > + $storeid, > + $vmid, > + $dst_format, > + undef, > + $src_size / 102, > + ); > + > + print "Importing disk image '$source' as '$dst_volid'...\n"; > + eval { > + local $SIG{INT} = > + local $SIG{TERM} = > + local $SIG{QUIT} = > + local $SIG{HUP} = > + local $SIG{PIPE} = sub { die "Interrupted by signal $!\n"; }; > + > + my $zeroinit = PVE::Storage::volume_has_feature( > + $storecfg, > + 'sparseinit', > + $dst_volid, > + ); > + PVE::Storage::activate_volumes($storecfg, [$dst_volid]); > + PVE::QemuServer::qemu_img_convert( > + $source, > + $dst_volid, > + $src_size, > + undef, > + $zeroinit, > + ); > + PVE::Storage::deactivate_volumes($storecfg, [$dst_volid]); > + > + }; > + if (my $err = $@) { > + eval { PVE::Storage::vdisk_free($storecfg, $dst_volid) }; > + warn "Cleanup of $dst_volid failed: $@ \n" if $@; > + > + die "Importing disk '$source' failed: $err\n" if $err; > + } > + > + $target->{options}->{file} = $dst_volid; > + my $options_string = PVE::QemuServer::print_drive($target->{options}); > + $target->{device} = PVE::QemuConfig->add_unused_volume($vmconf, $dst_volid) > + if !$target->{device}; > + > + $update_vm_api->( > + { > + vmid => $vmid, > + $target->{device} => $options_string, > + skiplock => 1, > + digest => $vmconf->{digest}, > + }, > + 1, > + ); > + > + return $dst_volid; > +}; > + > +__PACKAGE__->register_method ({ > + name => 'importdisk', > + path => '{vmid}/importdisk', > + method => 'POST', > + proxyto => 'node', > + protected => 1, > + description => "Import an external disk image into a VM. The image format ". > + "has to be supported by qemu-img.", > + parameters => { > + additionalProperties => 0, > + properties => { > + node => get_standard_option('pve-node'), > + vmid => get_standard_option('pve-vmid', > + {completion => \&PVE::QemuServer::complete_vmid}), > + source => { > + description => "Disk image to import. Can be a volid ". > + "(local:99/imageToImport.raw) or an absolute path on the server.", > + type => 'string', > + }, > + device => { > + type => 'string', > + description => "Bus/Device type of the new disk (e.g. 'ide0', ". > + "'scsi2'). Will add the image as unused disk if omitted.", > + enum => [PVE::QemuServer::Drive::valid_drive_names()], > + optional => 1, > + }, > + device_options => { > + type => 'string', > + description => "Options to set for the new disk (e.g. 'discard=on,backup=0')", > + optional => 1, > + requires => 'device', > + }, > + storage => get_standard_option('pve-storage-id', { > + description => "The storage to which the image will be imported to.", > + completion => \&PVE::QemuServer::complete_storage, > + }), > + format => { > + type => 'string', > + description => 'Target format.', > + enum => [ 'raw', 'qcow2', 'vmdk' ], > + optional => 1, > + }, > + digest => get_standard_option('pve-config-digest'), > + }, > + }, > + returns => { type => 'string'}, > + code => sub { > + my ($param) = @_; > + my $vmid = extract_param($param, 'vmid'); > + my $node = extract_param($param, 'node'); > + my $source = extract_param($param, 'source'); > + my $digest = extract_param($param, 'digest'); > + my $device_options = extract_param($param, 'device_options'); > + my $device = extract_param($param, 'device'); > + my $storeid = extract_param($param, 'storage'); > + > + my $rpcenv = PVE::RPCEnvironment::get(); > + my $authuser = $rpcenv->get_user(); > + my $storecfg = PVE::Storage::config(); > + PVE::Storage::storage_config($storecfg, $storeid); > + > + > + if ($device_options) { > + # $device_options may or may not contain <storeid>:0 > + my $parsed = PVE::QemuServer::Drive::parse_drive($device, $device_options); > + if ($parsed) { > + raise_param_exc({$device_options => "Invalid import syntax"}) > + if !($parsed->{file} =~ $IMPORT_DISK_RE); > + } else { > + my $fake = "$storeid:0,$device_options"; > + $parsed = PVE::QemuServer::Drive::parse_drive($device, $fake); > + } > + delete $parsed->{file}; > + delete $parsed->{interface}; > + delete $parsed->{index}; > + $device_options = $parsed; > + } > + > + # Format can be set explicitly "--format vmdk" > + # or as part of device options "--device_options discard=on,format=vmdk" > + my $format = extract_param($param, 'format'); > + if ($device_options) { > + raise_param_exc({format => "Format already specified in device_options!"}) > + if $format && $device_options->{format}; > + $format = $format || $device_options->{format}; # may be undefined > + } > + $check_format_is_supported->($format, $storeid, $storecfg); > + > + # quick checks before fork + lock > + my $conf = PVE::QemuConfig->load_config($vmid); > + PVE::QemuConfig->check_lock($conf); > + PVE::Tools::assert_if_modified($conf->{digest}, $digest); > + if ($device && $conf->{$device}) { > + die "Could not import because device $device is already in ". > + "use in VM $vmid. Choose a different device!"; > + } > + > + my $worker = sub { > + PVE::QemuConfig->lock_config($vmid, sub { > + $conf = PVE::QemuConfig->load_config($vmid); > + PVE::QemuConfig->check_lock($conf); > + > + PVE::Tools::assert_if_modified($conf->{digest}, $digest); > + PVE::QemuConfig->set_lock($vmid, 'import'); > + $conf = PVE::QemuConfig->load_config($vmid); > + }); > + > + my $target = { > + node => $node, > + storeid => $storeid, > + }; > + $target->{format} = $format; > + $target->{device} = $device; > + $target->{options} = $device_options; > + eval { $import_disk_image->($storecfg, $vmid, $conf, $source, $target) }; > + my $err = $@; > + eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; > + warn $@ if $@; > + die $err if $err; > + }; > + return $rpcenv->fork_worker('importdisk', $vmid, $authuser, $worker); > + }}); > + > +__PACKAGE__->register_method({ > + name => 'importvm', > + path => '{vmid}/importvm', > + method => 'POST', > + description => "Import a VM from existing disk images.", > + protected => 1, > + proxyto => 'node', > + parameters => { > + additionalProperties => 0, > + properties => PVE::QemuServer::json_config_properties( > + { > + node => get_standard_option('pve-node'), > + vmid => get_standard_option('pve-vmid', { completion => > + \&PVE::Cluster::complete_next_vmid }), > + diskimage => { > + description => "\\0 delimited mapping of devices to disk images. For " . > + "example, scsi0=/mnt/nfs/image1.vmdk", > + type => 'string', > + format => 'device-image-pair-alist', > + }, > + start => { > + optional => 1, > + type => 'boolean', > + default => 0, > + description => "Start VM after it was imported successfully.", > + }, > + }), > + }, > + returns => { > + type => 'string', > + }, > + code => sub { > + my ($param) = @_; > + my $node = extract_param($param, 'node'); > + my $vmid = extract_param($param, 'vmid'); > + my $diskimages_string = extract_param($param, 'diskimage'); > + my $boot = extract_param($param, 'boot'); > + my $start = extract_param($param, 'start'); > + > + my $rpcenv = PVE::RPCEnvironment::get(); > + my $authuser = $rpcenv->get_user(); > + my $storecfg = PVE::Storage::config(); > + > + PVE::Cluster::check_cfs_quorum(); > + > + my $import_param = {}; > + foreach my $opt (keys %$param) { > + next if $opt eq 'efidisk0'; > + raise_param_exc({bootdisk => "Deprecated: Use --boot order= instead"}) > + if $opt eq 'bootdisk'; > + > + if (PVE::QemuServer::Drive::is_valid_drivename($opt)) { > + my $drive = PVE::QemuServer::parse_drive($opt, $param->{$opt}); > + if ($drive->{file} =~ $IMPORT_DISK_RE) { > + $import_param->{$opt} = $drive; > + delete $param->{$opt}; > + } > + } > + } > + > + my $diskimages = {}; > + foreach my $pair (PVE::Tools::split_list($diskimages_string)) { > + my ($device, $diskimage) = split('=', $pair); > + $diskimages->{$device} = $diskimage; > + raise_param_exc({ > + $device => "Device '$device' not marked for import, " . > + "but import source '$diskimage' specified", > + }) if !defined($import_param->{$device}); > + PVE::Storage::abs_filesystem_path($storecfg, $diskimage, 1); > + } > + > + foreach my $device (keys %$import_param) { > + raise_param_exc({ > + $device => "Device '$device' marked for import, but no source given\n", > + }) if !defined($diskimages->{$device}); > + } > + > + eval { PVE::QemuConfig->create_and_lock_config($vmid, 0, 'import') }; > + die "Unable to create config for VM import: $@" if $@; > + > + my $worker = sub { > + my $reload_conf = sub { > + my ($vmid) = @_; > + my $conf = PVE::QemuConfig->load_config($vmid); > + return $conf if PVE::QemuConfig->has_lock($conf, 'import'); > + die "import lock in VM $vmid config file missing!"; > + }; > + > + my $conf = $reload_conf->($vmid); > + $update_vm_api->( > + { > + %$param, > + node => $node, > + vmid => $vmid, > + skiplock => 1, > + digest => $conf->{digest}, > + }, > + 1 > + ); > + > + eval { > + foreach my $device (keys %$import_param) { > + $conf = $reload_conf->($vmid); > + my $drive = $import_param->{$device}; > + my $storeid = PVE::Storage::parse_volume_id($drive->{file}); > + my $imported = $import_disk_image->( > + $storecfg, > + $vmid, > + $conf, > + $diskimages->{$device}, > + { > + storeid => $storeid, > + format => $drive->{format}, > + options => $drive, > + device => $device, > + }, > + ); > + } > + }; > + my $err = $@; > + if ($err) { > + eval { PVE::QemuServer::destroy_vm($storecfg, $vmid, 1) }; > + warn "Could not destroy VM $vmid: $@" if "$@"; > + > + die "Import failed: $err"; > + } > + > + $conf = $reload_conf->($vmid); > + if (!$boot) { > + my $bootdevs = PVE::QemuServer::get_default_bootdevices($conf); > + $boot = PVE::QemuServer::print_bootorder($bootdevs); > + } > + $update_vm_api->( > + { > + node => $node, > + vmid => $vmid, > + boot => $boot, > + skiplock => 1, > + digest => $conf->{digest}, > + }, > + 1, > + ); > + > + eval { PVE::QemuConfig->remove_lock($vmid, 'import') }; > + warn $@ if $@; > + > + PVE::QemuServer::vm_start($storecfg, $vmid) if $start; > + }; > + > + return $rpcenv->fork_worker('importvm', $vmid, $authuser, $worker); > + }}); > + > + > 1; > diff --git a/PVE/API2/Qemu/Makefile b/PVE/API2/Qemu/Makefile > index 5d4abda..bdd4762 100644 > --- a/PVE/API2/Qemu/Makefile > +++ b/PVE/API2/Qemu/Makefile > @@ -1,4 +1,4 @@ > -SOURCES=Agent.pm CPU.pm Machine.pm > +SOURCES=Agent.pm CPU.pm Machine.pm OVF.pm > > .PHONY: install > install: > diff --git a/PVE/API2/Qemu/OVF.pm b/PVE/API2/Qemu/OVF.pm > new file mode 100644 > index 0000000..bd6e90b > --- /dev/null > +++ b/PVE/API2/Qemu/OVF.pm > @@ -0,0 +1,68 @@ > +package PVE::API2::Qemu::OVF; > + > +use strict; > +use warnings; > + > +use PVE::RESTHandler; > +use PVE::JSONSchema qw(get_standard_option); > +use PVE::QemuServer::OVF; > + > +use base qw(PVE::RESTHandler); > + > +__PACKAGE__->register_method ({ > + name => 'index', > + path => '', > + method => 'GET', > + proxyto => 'node', > + description => "Read an .ovf manifest.", > + parameters => { > + additionalProperties => 0, > + properties => { > + node => get_standard_option('pve-node'), > + manifest => { > + description => ".ovf manifest", > + type => 'string', > + }, > + }, > + }, > + returns => { > + description => "VM config according to .ovf manifest and digest of manifest", > + type => "object", > + }, > + returns => { > + type => 'object', > + additionalProperties => 1, > + properties => PVE::QemuServer::json_ovf_properties({ > + name => { > + type => 'string', > + optional => 1, > + }, > + cores => { > + type => 'integer', > + optional => 1, > + }, > + memory => { > + type => 'integer', > + optional => 1, > + }, > + }), > + }, > + code => sub { > + my ($param) = @_; > + > + my $manifest = $param->{manifest}; > + die "$manifest: non-existent or non-regular file\n" if (! -f $manifest); > + > + my $parsed = PVE::QemuServer::OVF::parse_ovf($manifest, 0, 1); > + my $result; > + $result->{cores} = $parsed->{qm}->{cores}; > + $result->{name} = $parsed->{qm}->{name}; > + $result->{memory} = $parsed->{qm}->{memory}; > + my $disks = $parsed->{disks}; > + foreach my $disk (@$disks) { > + $result->{$disk->{disk_address}} = $disk->{backing_file}; > + } > + return $result; > + }}); > + > +1; > \ No newline at end of file > diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm > index 1c0b5c2..131c0b6 100644 > --- a/PVE/QemuServer.pm > +++ b/PVE/QemuServer.pm > @@ -300,7 +300,7 @@ my $confdesc = { > optional => 1, > type => 'string', > description => "Lock/unlock the VM.", > - enum => [qw(backup clone create migrate rollback snapshot snapshot-delete suspending suspended)], > + enum => [qw(backup clone create migrate rollback snapshot snapshot-delete suspending suspended import)], > }, > cpulimit => { > optional => 1, > @@ -985,19 +985,41 @@ PVE::JSONSchema::register_format('pve-volume-id-or-qm-path', \&verify_volume_id_ > sub verify_volume_id_or_qm_path { > my ($volid, $noerr) = @_; > > - if ($volid eq 'none' || $volid eq 'cdrom' || $volid =~ m|^/|) { > - return $volid; > - } > + return $volid eq 'none' || $volid eq 'cdrom' ? > + $volid : > + verify_volume_id_or_absolute_path($volid, $noerr); > +} > + > +PVE::JSONSchema::register_format('pve-volume-id-or-absolute-path', \&verify_volume_id_or_absolute_path); > +sub verify_volume_id_or_absolute_path { > + my ($volid, $noerr) = @_; > + > + return $volid if $volid =~ m|^/|; > > - # if its neither 'none' nor 'cdrom' nor a path, check if its a volume-id > $volid = eval { PVE::JSONSchema::check_format('pve-volume-id', $volid, '') }; > if ($@) { > - return if $noerr; > + return undef if $noerr; > die $@; > } > return $volid; > } > > +PVE::JSONSchema::register_format('device-image-pair', \&verify_device_image_pair); > +sub verify_device_image_pair { > + my ($pair, $noerr) = @_; > + > + my $error = sub { > + return undef if $noerr; > + die $@; > + }; > + > + my ($device, $image) = split('=', $pair); > + $error->("Invalid device '$device'") if !PVE::QemuServer::Drive::is_valid_drivename($device); > + $error->("Invalid image '$image'") if !verify_volume_id_or_absolute_path($image); > + > + return $pair; > +} > + > my $usb_fmt = { > host => { > default_key => 1, > @@ -2030,6 +2052,22 @@ sub json_config_properties { > return $prop; > } > > +# Properties that we can read from an OVF file > +sub json_ovf_properties { > + my $prop = shift; > + > + foreach my $device ( PVE::QemuServer::Drive::valid_drive_names()) { > + $prop->{$device} = { > + type => 'string', > + format => 'pve-volume-id-or-absolute-path', > + description => "Disk image that gets imported to $device", > + optional => 1, > + }; > + } > + > + return $prop; > +} > + > # return copy of $confdesc_cloudinit to generate documentation > sub cloudinit_config_properties { > > @@ -6748,7 +6786,7 @@ sub qemu_img_convert { > $src_path = PVE::Storage::path($storecfg, $src_volid, $snapname); > $src_is_iscsi = ($src_path =~ m|^iscsi://|); > $cachemode = 'none' if $src_scfg->{type} eq 'zfspool'; > - } elsif (-f $src_volid) { > + } elsif (-f $src_volid || -b $src_volid) { > $src_path = $src_volid; > if ($src_path =~ m/\.($PVE::QemuServer::Drive::QEMU_FORMAT_RE)$/) { > $src_format = $1; > diff --git a/PVE/QemuServer/OVF.pm b/PVE/QemuServer/OVF.pm > index c76c199..48146e9 100644 > --- a/PVE/QemuServer/OVF.pm > +++ b/PVE/QemuServer/OVF.pm > @@ -87,7 +87,7 @@ sub id_to_pve { > > # returns two references, $qm which holds qm.conf style key/values, and \@disks > sub parse_ovf { > - my ($ovf, $debug) = @_; > + my ($ovf, $debug, $manifest_only) = @_; > > my $dom = XML::LibXML->load_xml(location => $ovf, no_blanks => 1); > > @@ -220,9 +220,11 @@ ovf:Item[rasd:InstanceID='%s']/rasd:ResourceType", $controller_id); > die "error parsing $filepath, file seems not to exist at $backing_file_path\n"; > } > > - my $virtual_size; > - if ( !($virtual_size = PVE::Storage::file_size_info($backing_file_path)) ) { > - die "error parsing $backing_file_path, size seems to be $virtual_size\n"; > + my $virtual_size = undef; > + if (!$manifest_only) { # Not possible if manifest is uploaded in web gui > + if ( !($virtual_size = PVE::Storage::file_size_info($backing_file_path)) ) { > + die "error parsing $backing_file_path: Could not get file size info: $@\n"; > + } > } > > $pve_disk = { > -- > 2.20.1 > > > _______________________________________________ > pve-devel mailing list > pve-devel@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel > ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pve-devel] [PATCH v7 qemu-server] Add API for import wizards 2021-03-31 15:12 ` Fabian Grünbichler @ 2021-04-01 10:19 ` Dominic Jäger 2021-04-01 11:30 ` Fabian Grünbichler 0 siblings, 1 reply; 7+ messages in thread From: Dominic Jäger @ 2021-04-01 10:19 UTC (permalink / raw) To: Proxmox VE development discussion On Wed, Mar 31, 2021 at 05:12:28PM +0200, Fabian Grünbichler wrote: > this is starting to shape up nicely. as promised, I now took a stab at > (roughly!) integrating this into our regular flow (see diff below): > .... > > - we could likely drop the separate import_disk API call, and let the > `importdisk` CLI command prepare parameters for a regular VM config > update I think dropping importdisk is a good idea, too. > I'm sure I've missed some corner cases, as I've only tested create_vm > with importing, and not the other newly exposed APIs. I didn't get the importdisk API to work yet. But I'd look closer at stuff like that if (!$device_options) { $device_options = "$storeid:0"; <=== should be -1 } only if we keep it? One thing is important: That import parameter was at the place of the default storage PVE/API2/Qemu.pm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm index 41e1ab7..73af497 100644 --- a/PVE/API2/Qemu.pm +++ b/PVE/API2/Qemu.pm @@ -1469,7 +1469,7 @@ my $update_vm_api = sub { PVE::QemuServer::vmconfig_register_unused_drive($storecfg, $vmid, $conf, PVE::QemuServer::parse_drive($opt, $conf->{pending}->{$opt})) if defined($conf->{pending}->{$opt}); - &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, {$opt => $import_devices->{$opt}}); + &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, undef, {$opt => $import_devices->{$opt}}); } elsif ($opt =~ m/^serial\d+/) { if ((!defined($conf->{$opt}) || $conf->{$opt} eq 'socket') && $param->{$opt} eq 'socket') { $rpcenv->check_vm_perm($authuser, $vmid, undef, ['VM.Config.HWType']); > stat shows how much boilerplace/duplication this removes, although there > is probably even more potential here since `create_vm` partly duplicates > the `update_vm_api` sub that ends up calling `create_disks`: Is that urgent? ^ permalink raw reply [flat|nested] 7+ messages in thread
* Re: [pve-devel] [PATCH v7 qemu-server] Add API for import wizards 2021-04-01 10:19 ` Dominic Jäger @ 2021-04-01 11:30 ` Fabian Grünbichler 0 siblings, 0 replies; 7+ messages in thread From: Fabian Grünbichler @ 2021-04-01 11:30 UTC (permalink / raw) To: Dominic Jäger, Proxmox VE development discussion On April 1, 2021 12:19 pm, Dominic Jäger wrote: > On Wed, Mar 31, 2021 at 05:12:28PM +0200, Fabian Grünbichler wrote: >> this is starting to shape up nicely. as promised, I now took a stab at >> (roughly!) integrating this into our regular flow (see diff below): >> .... >> >> - we could likely drop the separate import_disk API call, and let the >> `importdisk` CLI command prepare parameters for a regular VM config >> update > I think dropping importdisk is a good idea, too. > >> I'm sure I've missed some corner cases, as I've only tested create_vm >> with importing, and not the other newly exposed APIs. > > I didn't get the importdisk API to work yet. > But I'd look closer at stuff like that > if (!$device_options) { > $device_options = "$storeid:0"; <=== should be -1 > } > only if we keep it? > > One thing is important: That import parameter was at the place of the default storage yes, I just roughly moved the code to see whether the approach works out and then tested a single entry point. if there are no objections into merging it like that, it would definitely need a cleanup and more testing :) > > PVE/API2/Qemu.pm | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm > index 41e1ab7..73af497 100644 > --- a/PVE/API2/Qemu.pm > +++ b/PVE/API2/Qemu.pm > @@ -1469,7 +1469,7 @@ my $update_vm_api = sub { > PVE::QemuServer::vmconfig_register_unused_drive($storecfg, $vmid, $conf, PVE::QemuServer::parse_drive($opt, $conf->{pending}->{$opt})) > if defined($conf->{pending}->{$opt}); > > - &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, {$opt => $import_devices->{$opt}}); > + &$create_disks($rpcenv, $authuser, $conf->{pending}, $arch, $storecfg, $vmid, undef, {$opt => $param->{$opt}}, undef, {$opt => $import_devices->{$opt}}); > } elsif ($opt =~ m/^serial\d+/) { > if ((!defined($conf->{$opt}) || $conf->{$opt} eq 'socket') && $param->{$opt} eq 'socket') { > $rpcenv->check_vm_perm($authuser, $vmid, undef, ['VM.Config.HWType']); > > >> stat shows how much boilerplace/duplication this removes, although there >> is probably even more potential here since `create_vm` partly duplicates >> the `update_vm_api` sub that ends up calling `create_disks`: > Is that urgent? no, it's just something I noticed since I added the same code in the same context in two places ;) ^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] [PATCH v7 manager] gui: Add import for disk & VM 2021-03-26 12:32 [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Dominic Jäger 2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger @ 2021-03-26 12:32 ` Dominic Jäger 2021-04-01 13:40 ` [pve-devel] applied: [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Thomas Lamprecht 2 siblings, 0 replies; 7+ messages in thread From: Dominic Jäger @ 2021-03-26 12:32 UTC (permalink / raw) To: pve-devel Add GUI wizard to import whole VMs and a window to import single disks in Hardware View. Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com> --- v6->v7: - Update to API changes - Add helpers to Utils - Whitespace & line breaks according to style guide - Making conditional branches in HDEdit easier to read PVE/API2/Nodes.pm | 7 + www/manager6/Makefile | 2 + www/manager6/Utils.js | 12 + www/manager6/Workspace.js | 15 ++ www/manager6/form/ControllerSelector.js | 15 ++ www/manager6/node/CmdMenu.js | 13 + www/manager6/qemu/HDEdit.js | 169 +++++++++++- www/manager6/qemu/HardwareView.js | 25 ++ www/manager6/qemu/ImportWizard.js | 332 ++++++++++++++++++++++++ www/manager6/qemu/MultiHDEdit.js | 277 ++++++++++++++++++++ www/manager6/window/Wizard.js | 2 + 11 files changed, 856 insertions(+), 13 deletions(-) create mode 100644 www/manager6/qemu/ImportWizard.js create mode 100644 www/manager6/qemu/MultiHDEdit.js diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm index ba6621c6..1cee6cb5 100644 --- a/PVE/API2/Nodes.pm +++ b/PVE/API2/Nodes.pm @@ -48,6 +48,7 @@ use PVE::API2::LXC; use PVE::API2::Network; use PVE::API2::NodeConfig; use PVE::API2::Qemu::CPU; +use PVE::API2::Qemu::OVF; use PVE::API2::Qemu; use PVE::API2::Replication; use PVE::API2::Services; @@ -76,6 +77,11 @@ __PACKAGE__->register_method ({ path => 'cpu', }); +__PACKAGE__->register_method ({ + subclass => "PVE::API2::Qemu::OVF", + path => 'readovf', +}); + __PACKAGE__->register_method ({ subclass => "PVE::API2::LXC", path => 'lxc', @@ -2183,6 +2189,7 @@ __PACKAGE__->register_method ({ return undef; }}); + # bash completion helper sub complete_templet_repo { diff --git a/www/manager6/Makefile b/www/manager6/Makefile index a2f7be6d..753cd1c0 100644 --- a/www/manager6/Makefile +++ b/www/manager6/Makefile @@ -196,8 +196,10 @@ JSSRC= \ qemu/CmdMenu.js \ qemu/Config.js \ qemu/CreateWizard.js \ + qemu/ImportWizard.js \ qemu/DisplayEdit.js \ qemu/HDEdit.js \ + qemu/MultiHDEdit.js \ qemu/HDEfi.js \ qemu/HDMove.js \ qemu/HDResize.js \ diff --git a/www/manager6/Utils.js b/www/manager6/Utils.js index f502950f..dbfd65ce 100644 --- a/www/manager6/Utils.js +++ b/www/manager6/Utils.js @@ -1708,6 +1708,16 @@ Ext.define('PVE.Utils', { }); }, + // collection ... collection of strings of a subset of the descendants of container + // visible ... true to show and enable, false to hide and disable + setDescendantsVisible: function(container, collection, visible = 1) { + const hide = (element, value) => { + element.setHidden(value); + element.setDisabled(value); + }; + collection.map(e => container.down(e)).forEach(e => hide(e, !visible)); + }, + cpu_vendor_map: { 'default': 'QEMU', 'AuthenticAMD': 'AMD', @@ -1787,6 +1797,8 @@ Ext.define('PVE.Utils', { hastop: ['HA', gettext('Stop')], imgcopy: ['', gettext('Copy data')], imgdel: ['', gettext('Erase data')], + importdisk: ['VM', gettext('Import disk')], + importvm: ['VM', gettext('Import VM')], lvmcreate: [gettext('LVM Storage'), gettext('Create')], lvmthincreate: [gettext('LVM-Thin Storage'), gettext('Create')], migrateall: ['', gettext('Migrate all VMs and Containers')], diff --git a/www/manager6/Workspace.js b/www/manager6/Workspace.js index 0c1b9e0c..631739a0 100644 --- a/www/manager6/Workspace.js +++ b/www/manager6/Workspace.js @@ -280,11 +280,25 @@ Ext.define('PVE.StdWorkspace', { }, }); + var importVM = Ext.createWidget('button', { + pack: 'end', + margin: '3 5 0 0', + baseCls: 'x-btn', + iconCls: 'fa fa-desktop', + text: gettext("Import VM"), + hidden: Proxmox.UserName !== 'root@pam', + handler: function() { + var wiz = Ext.create('PVE.qemu.ImportWizard', {}); + wiz.show(); + }, + }); + sprovider.on('statechange', function(sp, key, value) { if (key === 'GuiCap' && value) { caps = value; createVM.setDisabled(!caps.vms['VM.Allocate']); createCT.setDisabled(!caps.vms['VM.Allocate']); + importVM.setDisabled(!caps.vms['VM.Allocate']); } }); @@ -332,6 +346,7 @@ Ext.define('PVE.StdWorkspace', { }, createVM, createCT, + importVM, { pack: 'end', margin: '0 5 0 0', diff --git a/www/manager6/form/ControllerSelector.js b/www/manager6/form/ControllerSelector.js index 23c61159..8e9aee98 100644 --- a/www/manager6/form/ControllerSelector.js +++ b/www/manager6/form/ControllerSelector.js @@ -68,6 +68,21 @@ clist_loop: deviceid.validate(); }, + getValues: function() { + return this.query('field').map(x => x.getValue()); + }, + + getValuesAsString: function() { + return this.getValues().join(''); + }, + + setValue: function(value) { + let regex = /([a-z]+)(\d+)/; + let [_, controller, deviceid] = regex.exec(value); + this.down('field[name=controller]').setValue(controller); + this.down('field[name=deviceid]').setValue(deviceid); + }, + initComponent: function() { var me = this; diff --git a/www/manager6/node/CmdMenu.js b/www/manager6/node/CmdMenu.js index b650bfa0..407cf2d0 100644 --- a/www/manager6/node/CmdMenu.js +++ b/www/manager6/node/CmdMenu.js @@ -29,6 +29,19 @@ Ext.define('PVE.node.CmdMenu', { wiz.show(); }, }, + { + text: gettext("Import VM"), + hidden: Proxmox.UserName !== 'root@pam', + itemId: 'importvm', + iconCls: 'fa fa-cube', + handler: function() { + var me = this.up('menu'); + var wiz = Ext.create('PVE.qemu.ImportWizard', { + nodename: me.nodename, + }); + wiz.show(); + }, + }, { xtype: 'menuseparator' }, { text: gettext('Bulk Start'), diff --git a/www/manager6/qemu/HDEdit.js b/www/manager6/qemu/HDEdit.js index e22111bf..a2f6c95a 100644 --- a/www/manager6/qemu/HDEdit.js +++ b/www/manager6/qemu/HDEdit.js @@ -58,6 +58,21 @@ Ext.define('PVE.qemu.HDInputPanel', { }, }, + isImport: function() { + return this.isImportVM || this.isImportDisk; + }, + + /* + All radiofields (esp. sourceRadioPath and sourceRadioStorage) have the + same scope for name. But we need a different scope for each HDInputPanel in + a MultiHDInputPanel to get the selection for each HDInputPanel => Make + names so that those within one HDInputPanel are equal, but different from other + HDInputPanels + */ + getSourceTypeID() { + return 'sourceType_' + this.id; + }, + onGetValues: function(values) { var me = this; @@ -70,6 +85,8 @@ Ext.define('PVE.qemu.HDInputPanel', { } else if (me.isCreate) { if (values.hdimage) { me.drive.file = values.hdimage; + } else if (me.isImport()) { + me.drive.file = `${values.hdstorage}:0`; } else { me.drive.file = values.hdstorage + ":" + values.disksize; } @@ -83,15 +100,33 @@ Ext.define('PVE.qemu.HDInputPanel', { PVE.Utils.propertyStringSet(me.drive, values.iothread, 'iothread', 'on'); PVE.Utils.propertyStringSet(me.drive, values.cache, 'cache'); - var names = ['mbps_rd', 'mbps_wr', 'iops_rd', 'iops_wr']; - Ext.Array.each(names, function(name) { - var burst_name = name + '_max'; + var names = ['mbps_rd', 'mbps_wr', 'iops_rd', 'iops_wr']; + Ext.Array.each(names, function(name) { + var burst_name = name + '_max'; PVE.Utils.propertyStringSet(me.drive, values[name], name); PVE.Utils.propertyStringSet(me.drive, values[burst_name], burst_name); - }); + }); + + const getSourceImageLocation = function() { + const type = values[me.getSourceTypeID()]; + return type === 'storage' ? values.sourceVolid : values.sourcePath; + }; + if (me.isImportVM) { + params.diskimage = `${confid}=${getSourceImageLocation()}`; + } + + const options = PVE.Parser.printQemuDrive(me.drive); - params[confid] = PVE.Parser.printQemuDrive(me.drive); + if (me.isImportDisk) { + params.device = confid; + params.device_options = options; + params.source = getSourceImageLocation(); + params.device = values.controller + values.deviceid; + params.storage = values.hdstorage; + } else { + params[confid] = options; + } return params; }, @@ -149,6 +184,10 @@ Ext.define('PVE.qemu.HDInputPanel', { me.setValues(values); }, + getDevice: function() { + return this.bussel.getValuesAsString(); + }, + setNodename: function(nodename) { var me = this; me.down('#hdstorage').setNodename(nodename); @@ -169,10 +208,15 @@ Ext.define('PVE.qemu.HDInputPanel', { me.advancedColumn2 = []; if (!me.confid || me.unused) { + let controllerColumn = me.isImport() ? me.column2 : me.column1; me.bussel = Ext.create('PVE.form.ControllerSelector', { + itemId: 'bussel', vmconfig: me.insideWizard ? { ide2: 'cdrom' } : {}, }); - me.column1.push(me.bussel); + if (me.isImport()) { + me.bussel.fieldLabel = 'Target Device'; + } + controllerColumn.push(me.bussel); me.scsiController = Ext.create('Ext.form.field.Display', { fieldLabel: gettext('SCSI Controller'), @@ -184,7 +228,7 @@ Ext.define('PVE.qemu.HDInputPanel', { submitValue: false, hidden: true, }); - me.column1.push(me.scsiController); + controllerColumn.push(me.scsiController); } if (me.unused) { @@ -199,14 +243,21 @@ Ext.define('PVE.qemu.HDInputPanel', { allowBlank: false, }); me.column1.push(me.unusedDisks); - } else if (me.isCreate) { - me.column1.push({ + } else if (me.isCreate || me.isImport()) { + let selector = { xtype: 'pveDiskStorageSelector', storageContent: 'images', name: 'disk', nodename: me.nodename, - autoSelect: me.insideWizard, - }); + hideSize: me.isImport(), + autoSelect: me.insideWizard || me.isImport(), + }; + if (me.isImport()) { + selector.storageLabel = gettext('Target storage'); + me.column2.push(selector); + } else { + me.column1.push(selector); + } } else { me.column1.push({ xtype: 'textfield', @@ -217,6 +268,12 @@ Ext.define('PVE.qemu.HDInputPanel', { }); } + if (me.isImport()) { + me.column2.push({ + xtype: 'box', + autoEl: { tag: 'hr' }, + }); + } me.column2.push( { xtype: 'CacheTypeSelector', @@ -231,6 +288,84 @@ Ext.define('PVE.qemu.HDInputPanel', { name: 'discard', }, ); + if (me.isImport()) { + me.column1.unshift( + { + xtype: 'radiofield', + itemId: 'sourceRadioStorage', + name: me.getSourceTypeID(), + inputValue: 'storage', + boxLabel: gettext('Use a storage as source'), + hidden: Proxmox.UserName !== 'root@pam', + checked: true, + listeners: { + change: (_, newValue) => { + const selectors = [ + '#sourceStorageSelector', + '#sourceFileSelector', + ]; + PVE.Utils.setDescendantsVisible(me, selectors, newValue); + }, + }, + }, { + xtype: 'pveStorageSelector', + itemId: 'sourceStorageSelector', + name: 'inputImageStorage', + nodename: me.nodename, + fieldLabel: gettext('Source Storage'), + storageContent: 'images', + autoSelect: me.insideWizard, + hidden: true, + disabled: true, + listeners: { + change: function(_, selectedStorage) { + me.down('#sourceFileSelector').setStorage(selectedStorage); + }, + }, + }, { + xtype: 'pveFileSelector', + itemId: 'sourceFileSelector', + name: 'sourceVolid', + nodename: me.nodename, + storageContent: 'images', + hidden: true, + disabled: true, + fieldLabel: gettext('Source Image'), + autoEl: { + tag: 'div', + 'data-qtip': gettext("Place your source images into a new folder <storageRoot>/images/<newVMID>, for example /var/lib/vz/images/999"), + }, + }, { + xtype: 'radiofield', + itemId: 'sourceRadioPath', + name: me.getSourceTypeID(), + inputValue: 'path', + boxLabel: gettext('Use an absolute path as source'), + hidden: Proxmox.UserName !== 'root@pam', + listeners: { + change: (_, newValue) => { + PVE.Utils.setDescendantsVisible(me, ['#sourcePathTextfield'], newValue); + }, + }, + }, { + xtype: 'textfield', + itemId: 'sourcePathTextfield', + fieldLabel: gettext('Source Path'), + name: 'sourcePath', + autoEl: { + tag: 'div', + 'data-qtip': gettext('Absolute path to the source disk image, for example: /home/user/somedisk.qcow2'), + }, + hidden: true, + disabled: true, + validator: function(insertedText) { + return insertedText.startsWith('/') || + insertedText.startsWith('http') || + gettext('Must be an absolute path or URL'); + }, + }, + ); + } me.advancedColumn1.push( { @@ -373,13 +508,18 @@ Ext.define('PVE.qemu.HDEdit', { nodename: nodename, unused: unused, isCreate: me.isCreate, + isImportVM: me.isImportVM, + isImportDisk: me.isImportDisk, }); - var subject; if (unused) { me.subject = gettext('Unused Disk'); + } else if (me.isImportDisk) { + me.subject = gettext('Import Disk'); + me.submitText = 'Import'; + me.backgroundDelay = undefined; } else if (me.isCreate) { - me.subject = gettext('Hard Disk'); + me.subject = gettext('Hard Disk'); } else { me.subject = gettext('Hard Disk') + ' (' + me.confid + ')'; } @@ -404,6 +544,9 @@ Ext.define('PVE.qemu.HDEdit', { ipanel.setDrive(drive); me.isValid(); // trigger validation } + if (me.isImportDisk) { + me.url = me.url.replace(/\/config$/, "/importdisk"); + } }, }); }, diff --git a/www/manager6/qemu/HardwareView.js b/www/manager6/qemu/HardwareView.js index 98352e3f..4fbf0e5e 100644 --- a/www/manager6/qemu/HardwareView.js +++ b/www/manager6/qemu/HardwareView.js @@ -431,6 +431,30 @@ Ext.define('PVE.qemu.HardwareView', { handler: run_move, }); + var import_btn = new Proxmox.button.Button({ + text: gettext('Import disk'), + hidden: Proxmox.UserName !== 'root@pam', + handler: function() { + var win = Ext.create('PVE.qemu.HDEdit', { + method: 'POST', + url: `/api2/extjs/${baseurl}`, + pveSelNode: me.pveSelNode, + isImportDisk: true, + listeners: { + add: function(_, component) { + const selectors = [ + '#sourceStorageSelector', + '#sourceFileSelector', + ]; + PVE.Utils.setDescendantsVisible(component, selectors); + }, + }, + }); + win.on('destroy', me.reload, me); + win.show(); + }, + }); + var remove_btn = new Proxmox.button.Button({ text: gettext('Remove'), defaultText: gettext('Remove'), @@ -759,6 +783,7 @@ Ext.define('PVE.qemu.HardwareView', { edit_btn, resize_btn, move_btn, + import_btn, revert_btn, ], rows: rows, diff --git a/www/manager6/qemu/ImportWizard.js b/www/manager6/qemu/ImportWizard.js new file mode 100644 index 00000000..0066adc4 --- /dev/null +++ b/www/manager6/qemu/ImportWizard.js @@ -0,0 +1,332 @@ +/*jslint confusion: true*/ +Ext.define('PVE.qemu.ImportWizard', { + extend: 'PVE.window.Wizard', + alias: 'widget.pveQemuImportWizard', + mixins: ['Proxmox.Mixin.CBind'], + + viewModel: { + data: { + nodename: '', + current: { + scsihw: '', + }, + }, + }, + + cbindData: { + nodename: undefined, + }, + + subject: gettext('Import Virtual Machine'), + + isImportVM: true, + + addDiskFunction: function() { + let me = this; + let wizard; + if (me.xtype === 'button') { + wizard = me.up('window'); + } else if (me.xtype === 'pveQemuImportWizard') { + wizard = me; + } + let multihd = wizard.down('pveQemuMultiHDInputPanel'); + multihd.addDiskFunction(); + }, + + items: [ + { + xtype: 'inputpanel', + title: gettext('Import'), + itemId: 'importInputpanel', + column1: [ + { + xtype: 'pveNodeSelector', + name: 'nodename', + cbind: { + selectCurNode: '{!nodename}', + preferredValue: '{nodename}', + }, + bind: { + value: '{nodename}', + }, + fieldLabel: gettext('Node'), + allowBlank: false, + onlineValidator: true, + }, + { + xtype: 'pveGuestIDSelector', + name: 'vmid', + guestType: 'qemu', + value: '', + loadNextFreeID: true, + validateExists: false, + }, + ], + column2: [ + { + xtype: 'label', + itemId: 'successTextfield', + hidden: true, + html: gettext('Manifest successfully uploaded'), + margin: '0 0 0 10', + }, + { + xtype: 'textfield', + itemId: 'server_ovf_manifest', + name: 'ovf_textfield', + emptyText: '/mnt/nfs/exported.ovf', + fieldLabel: 'Absolute path to .ovf manifest on your PVE host', + listeners: { + validitychange: function(_, isValid) { + let button = Ext.ComponentQuery.query('#load_remote_manifest_button').pop(); + button.setDisabled(!isValid); + }, + }, + validator: function(value) { + return (value && value.startsWith('/')) || gettext("Must start with /"); + }, + }, + { + xtype: 'proxmoxButton', + itemId: 'load_remote_manifest_button', + text: gettext('Load remote manifest'), + disabled: true, + handler: function() { + let inputpanel = this.up('#importInputpanel'); + let nodename = inputpanel.down('pveNodeSelector').getValue(); + // independent of onGetValues(), so that value of + // ovf_textfield can be removed for submit + let ovf_textfield_value = inputpanel.down('textfield[name=ovf_textfield]').getValue(); + let wizard = this.up('window'); + Proxmox.Utils.API2Request({ + url: '/nodes/' + nodename + '/readovf', + method: 'GET', + params: { + manifest: ovf_textfield_value, + }, + success: function(response) { + let ovfdata = response.result.data; + wizard.down('#vmNameTextfield').setValue(ovfdata.name); + wizard.down('#cpupanel').getViewModel().set('coreCount', ovfdata.cores); + wizard.down('#memorypanel').down('pveMemoryField').setValue(ovfdata.memory); + delete ovfdata.name; + delete ovfdata.cores; + delete ovfdata.memory; + delete ovfdata.digest; + let devices = Object.keys(ovfdata); // e.g. ide0, sata2 + let multihd = wizard.down('pveQemuMultiHDInputPanel'); + if (devices.length > 0) { + multihd.removeAllDisks(); + } + for (var device of devices) { + multihd.addDiskFunction(device, ovfdata[device]); + } + }, + failure: function(response, opts) { + console.warn("Failure of load manifest button"); + console.warn(response); + }, + }); + }, + }, + ], + onGetValues: function(values) { + delete values.server_ovf_manifest; + delete values.ovf_textfield; + return values; + }, + }, + { + xtype: 'inputpanel', + title: gettext('General'), + onlineHelp: 'qm_general_settings', + column1: [ + { + xtype: 'textfield', + name: 'name', + itemId: 'vmNameTextfield', + vtype: 'DnsName', + value: '', + fieldLabel: gettext('Name'), + allowBlank: true, + }, + ], + column2: [ + { + xtype: 'pvePoolSelector', + fieldLabel: gettext('Resource Pool'), + name: 'pool', + value: '', + allowBlank: true, + }, + ], + advancedColumn1: [ + { + xtype: 'proxmoxcheckbox', + name: 'onboot', + uncheckedValue: 0, + defaultValue: 0, + deleteDefaultValue: true, + fieldLabel: gettext('Start at boot'), + }, + ], + advancedColumn2: [ + { + xtype: 'textfield', + name: 'order', + defaultValue: '', + emptyText: 'any', + labelWidth: 120, + fieldLabel: gettext('Start/Shutdown order'), + }, + { + xtype: 'textfield', + name: 'up', + defaultValue: '', + emptyText: 'default', + labelWidth: 120, + fieldLabel: gettext('Startup delay'), + }, + { + xtype: 'textfield', + name: 'down', + defaultValue: '', + emptyText: 'default', + labelWidth: 120, + fieldLabel: gettext('Shutdown timeout'), + }, + ], + onGetValues: function(values) { + ['name', 'pool', 'onboot', 'agent'].forEach(function(field) { + if (!values[field]) { + delete values[field]; + } + }); + + var res = PVE.Parser.printStartup({ + order: values.order, + up: values.up, + down: values.down, + }); + + if (res) { + values.startup = res; + } + + delete values.order; + delete values.up; + delete values.down; + + return values; + }, + }, + { + xtype: 'pveQemuSystemPanel', + title: gettext('System'), + isCreate: true, + insideWizard: true, + }, + { + xtype: 'pveQemuMultiHDInputPanel', + title: gettext('Hard Disk'), + bind: { + nodename: '{nodename}', + }, + isCreate: true, + insideWizard: true, + }, + { + itemId: 'cpupanel', + xtype: 'pveQemuProcessorPanel', + insideWizard: true, + title: gettext('CPU'), + }, + { + itemId: 'memorypanel', + xtype: 'pveQemuMemoryPanel', + insideWizard: true, + title: gettext('Memory'), + }, + { + xtype: 'pveQemuNetworkInputPanel', + bind: { + nodename: '{nodename}', + }, + title: gettext('Network'), + insideWizard: true, + }, + { + title: gettext('Confirm'), + layout: 'fit', + items: [ + { + xtype: 'grid', + store: { + model: 'KeyValue', + sorters: [{ + property: 'key', + direction: 'ASC', + }], + }, + columns: [ + { header: 'Key', width: 150, dataIndex: 'key' }, + { header: 'Value', flex: 1, dataIndex: 'value' }, + ], + }, + ], + dockedItems: [ + { + xtype: 'proxmoxcheckbox', + name: 'start', + dock: 'bottom', + margin: '5 0 0 0', + boxLabel: gettext('Start after created'), + }, + ], + listeners: { + show: function(panel) { + var kv = this.up('window').getValues(); + var data = []; + Ext.Object.each(kv, function(key, value) { + if (key === 'delete') { // ignore + return; + } + data.push({ key: key, value: value }); + }); + + var summarystore = panel.down('grid').getStore(); + summarystore.suspendEvents(); + summarystore.removeAll(); + summarystore.add(data); + summarystore.sort(); + summarystore.resumeEvents(); + summarystore.fireEvent('refresh'); + }, + }, + onSubmit: function() { + var wizard = this.up('window'); + var params = wizard.getValues(); + + var nodename = params.nodename; + delete params.nodename; + delete params.delete; + if (Array.isArray(params.diskimages)) { + params.diskimages = params.diskimages.join(','); + } + + Proxmox.Utils.API2Request({ + url: `/nodes/${nodename}/qemu/${params.vmid}/importvm`, + waitMsgTarget: wizard, + method: 'POST', + params: params, + success: function() { + wizard.close(); + }, + failure: function(response) { + Ext.Msg.alert(gettext('Error'), response.htmlStatus); + }, + }); + }, + }, + ], +}); diff --git a/www/manager6/qemu/MultiHDEdit.js b/www/manager6/qemu/MultiHDEdit.js new file mode 100644 index 00000000..641a802f --- /dev/null +++ b/www/manager6/qemu/MultiHDEdit.js @@ -0,0 +1,277 @@ +Ext.define('PVE.qemu.MultiHDInputPanel', { + extend: 'Proxmox.panel.InputPanel', + alias: 'widget.pveQemuMultiHDInputPanel', + + insideWizard: false, + + hiddenDisks: [], + + leftColumnRatio: 0.25, + + column1: [ + { + // Adding to the HDInputPanelContainer below automatically adds + // items to this store + xtype: 'gridpanel', + scrollable: true, + store: { + xtype: 'store', + storeId: 'importwizard_diskstorage', + // Use the panel as id + // Panels have are objects and therefore unique + // E.g. while adding new panels 'device' is ambiguous + fields: ['device', 'panel'], + removeByPanel: function(panel) { + let recordIndex = this.findBy(record => record.data.panel === panel); + this.removeAt(recordIndex); + return recordIndex; + }, + }, + columns: [ + { + text: gettext('Target device'), + dataIndex: 'device', + flex: 1, + resizable: false, + }, + ], + listeners: { + select: function(_, record) { + this.up('pveQemuMultiHDInputPanel') + .down('#HDInputPanelContainer') + .setActiveItem(record.data.panel); + }, + }, + anchor: '100% 90%', + }, { + xtype: 'container', + layout: 'hbox', + center: true, + defaults: { + margin: '5', + xtype: 'button', + }, + items: [ + { + iconCls: 'fa fa-plus-circle', + itemId: 'addDisk', + handler: function(button) { + button.up('pveQemuMultiHDInputPanel').addDiskFunction(); + }, + }, { + iconCls: 'fa fa-trash-o', + itemId: 'removeDisk', + handler: function(button) { + button.up('pveQemuMultiHDInputPanel').removeCurrentDisk(); + }, + }, + ], + }, + ], + column2: [ + { + itemId: 'HDInputPanelContainer', + xtype: 'container', + layout: 'card', + items: [], + listeners: { + beforeRender: function() { + // Initial disk if none have been added by manifest yet + if (this.items.items.length === 0) { + this.addDiskFunction(); + } + }, + add: function(container, newPanel, index) { + let store = Ext.getStore('importwizard_diskstorage'); + store.add({ device: newPanel.getDevice(), panel: newPanel }); + container.setActiveItem(newPanel); + }, + remove: function(HDInputPanelContainer, HDInputPanel, eOpts) { + let store = Ext.getStore('importwizard_diskstorage'); + let indexOfRemoved = store.removeByPanel(HDInputPanel); + if (HDInputPanelContainer.items.getCount() > 0) { + HDInputPanelContainer.setActiveItem(indexOfRemoved - 1); + } + }, + }, + defaultItem: { + xtype: 'pveQemuHDInputPanel', + bind: { + nodename: '{nodename}', + }, + isCreate: true, + isImportVM: true, + returnSingleKey: true, + insideWizard: true, + setNodename: function(nodename) { + this.down('#hdstorage').setNodename(nodename); + this.down('#hdimage').setStorage(undefined, nodename); + this.down('#sourceStorageSelector').setNodename(nodename); + this.down('#sourceFileSelector').setNodename(nodename); + }, + listeners: { + // newHDInputPanel ... the defaultItem that has just been + // cloned and added into HDInputPnaleContainer parameter + // HDInputPanelContainer ... the container from column2 + // where all the new panels go into + added: function(newHDInputPanel, HDInputPanelContainer, pos) { + // The listeners cannot be added earlier, because its fields don't exist earlier + Ext.Array.each(this.down('pveControllerSelector') + .query('field'), function(field) { + field.on('change', function() { + // Note that one setValues in a controller + // selector makes one setValue in each of + // the two fields, so this listener fires + // two times in a row so to say e.g. + // changing controller selector from ide0 to + // sata1 makes ide0->sata0 and then + // sata0->sata1 + let store = Ext.getStore('importwizard_diskstorage'); + let controllerSelector = field.up('pveQemuHDInputPanel') + .down('pveControllerSelector'); + /* + * controller+device (ide0) might be + * ambiguous during creation => find by + * panel object instead + * + * There is no function that takes a + * function and returns the model directly + * => index & getAt + */ + let recordIndex = store.findBy(record => + record.data.panel === field.up('pveQemuHDInputPanel'), + ); + let newControllerAndId = controllerSelector.getValuesAsString(); + store.getAt(recordIndex).set('device', newControllerAndId); + }); + }, + ); + let wizard = this.up('pveQemuImportWizard'); + Ext.Array.each(this.query('field'), function(field) { + field.on('change', wizard.validcheck); + field.on('validitychange', wizard.validcheck); + }); + }, + }, + validator: function() { + var valid = true; + var fields = this.query('field, fieldcontainer'); + Ext.Array.each(fields, function(field) { + // Note: not all fielcontainer have isValid() + if (Ext.isFunction(field.isValid) && !field.isValid()) { + valid = false; + } + }); + return valid; + }, + }, + + // device ... device that the new disk should be assigned to, e.g. + // ide0, sata2 + // path ... if this is set to x then the disk will + // backed/imported from the path x, that is, the textfield will + // contain the value x + addDiskFunction(device, path) { + // creating directly removes binding => no storage found? + let item = Ext.clone(this.defaultItem); + let added = this.add(item); + // At this point the 'added' listener has fired and the fields + // in the variable added have the change listeners that update + // the store Therefore we can now set values only on the field + // and they will be updated in the store + if (path) { + added.down('#sourceRadioPath').setValue(true); + added.down('#sourcePathTextfield').setValue(path); + } else { + added.down('#sourceRadioStorage').setValue(true); + added.down('#sourceStorageSelector').setHidden(false); + added.down('#sourceFileSelector').setHidden(false); + added.down('#sourceFileSelector').enable(); + added.down('#sourceStorageSelector').enable(); + } + + let sp = Ext.state.Manager.getProvider(); + let advanced_checkbox = sp.get('proxmox-advanced-cb'); + added.setAdvancedVisible(advanced_checkbox); + + if (device) { + // This happens after the 'add' and 'added' listeners of the + // item/defaultItem clone/pveQemuHDInputPanel/added have fired + added.down('pveControllerSelector').setValue(device); + } + }, + removeCurrentDisk: function() { + let activePanel = this.getLayout().activeItem; // panel = disk + if (activePanel) { + this.remove(activePanel); + } + }, + }, + ], + + addDiskFunction: function(device, path) { + this.down('#HDInputPanelContainer').addDiskFunction(device, path); + }, + removeCurrentDisk: function() { + this.down('#HDInputPanelContainer').removeCurrentDisk(); + }, + removeAllDisks: function() { + let container = this.down('#HDInputPanelContainer'); + while (container.items.items.length > 0) { + container.removeCurrentDisk(); + } + }, + + beforeRender: function() { + let leftColumnPanel = this.items.get(0).items.get(0); + leftColumnPanel.setFlex(this.leftColumnRatio); + // any other panel because this has no height yet + let panelHeight = this.up('tabpanel').items.items[0].getHeight(); + leftColumnPanel.setHeight(panelHeight); + }, + + setNodename: function(nodename) { + this.nodename = nodename; + }, + + // Call with defined parameter or without + hasDuplicateDevices: function(values) { + if (!values) { + values = this.up('form').getValues(); + } + if (!Array.isArray(values.controller)) { + return false; + } + for (let i = 0; i < values.controller.length - 1; i++) { + for (let j = i+1; j < values.controller.length; j++) { + if (values.controller[i] === values.controller[j]) { + if (values.deviceid[i] === values.deviceid[j]) { + return true; + } + } + } + } + return false; + }, + + onGetValues: function(values) { + // Returning anything here would give wrong data in the form at the end + // of the wizrad Each HDInputPanel in this MultiHD panel already has a + // sufficient onGetValues() function for the form at the end of the + // wizard + if (this.hasDuplicateDevices(values)) { + Ext.Msg.alert(gettext('Error'), 'Equal target devices are forbidden. Make all unique!'); + } + }, + + validator: function() { + let inputpanels = this.down('#HDInputPanelContainer').items.getRange(); + if (inputpanels.some(panel => !panel.validator())) { + return false; + } + if (this.hasDuplicateDevices()) { + return false; + } + return true; + }, +}); diff --git a/www/manager6/window/Wizard.js b/www/manager6/window/Wizard.js index 8b930bbd..a3e3b690 100644 --- a/www/manager6/window/Wizard.js +++ b/www/manager6/window/Wizard.js @@ -261,6 +261,8 @@ Ext.define('PVE.window.Wizard', { }; field.on('change', validcheck); field.on('validitychange', validcheck); + // Make available for fields that get added later + me.validcheck = validcheck; }); }, }); -- 2.20.1 ^ permalink raw reply [flat|nested] 7+ messages in thread
* [pve-devel] applied: [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path 2021-03-26 12:32 [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Dominic Jäger 2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger 2021-03-26 12:32 ` [pve-devel] [PATCH v7 manager] gui: Add import for disk & VM Dominic Jäger @ 2021-04-01 13:40 ` Thomas Lamprecht 2 siblings, 0 replies; 7+ messages in thread From: Thomas Lamprecht @ 2021-04-01 13:40 UTC (permalink / raw) To: Proxmox VE development discussion, Dominic Jäger On 26.03.21 13:32, Dominic Jäger wrote: > This is required to import from LVM storages, for example > > Signed-off-by: Dominic Jäger <d.jaeger@proxmox.com> > --- > v6->v7: Feedback Fabian G > - Variables with _ instead of camelCase > - single if instead of if/else > > PVE/Storage.pm | 8 ++++---- > 1 file changed, 4 insertions(+), 4 deletions(-) > > applied, thanks! ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2021-04-01 13:40 UTC | newest] Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-03-26 12:32 [pve-devel] [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Dominic Jäger 2021-03-26 12:32 ` [pve-devel] [PATCH v7 qemu-server] Add API for import wizards Dominic Jäger 2021-03-31 15:12 ` Fabian Grünbichler 2021-04-01 10:19 ` Dominic Jäger 2021-04-01 11:30 ` Fabian Grünbichler 2021-03-26 12:32 ` [pve-devel] [PATCH v7 manager] gui: Add import for disk & VM Dominic Jäger 2021-04-01 13:40 ` [pve-devel] applied: [PATCH v7 storage] Optionally allow blockdev in abs_filesystem_path Thomas Lamprecht
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.Service provided by Proxmox Server Solutions GmbH | Privacy | Legal