Re: [pve-devel] [PATCH storage/qemu-server/manager v7] implement ova/ovf import for file based storages

[Filip Schauer <f.schauer@proxmox.com>]

I tried importing the GNS3 OVA files found here:
https://www.gns3.com/software/download-vm

I got the same XPath error with the VirtualBox file.

The ESXi and Workstation & Fusion OVA files were imported successfully
and they boot aswell.


On 18/11/2024 18:14, Filip Schauer wrote:
> Hello,
>
> I tried to quickly test the OVA import with a CentOS 7 OVA file that I
> downloaded from:
> https://dlconusc1.linuxvmimages.com/046389e06777452db2ccf9a32efa3760:dldatac/VirtualBox/C/7/CentOS_7.9.2009_VBM.zip 
>
>
> 6785cb04dc5e7ab5a28ec139f631dfc8aeb8a6278c5fc66e1e7030d3f5a83b18 
> /var/lib/vz/import/CentOS_7.9.2009_VBM_LinuxVMImages.COM.ova
>
> After enabling the "import" content type on my local directory storage,
> the ova file shows up in the import section of the storage.
>
> Upon clicking "Import" in the web UI the "Import Guest" dialog opens
> along with an error message stating:
> XPath error : Undefined namespace prefix at 
> /usr/share/perl5/PVE/GuestImport/OVF.pm line 239. (500)
>
> Closing the error message also closes the import dialog, preventing me
> from importing.
>
> The same thing happens with this ArcoLinux OVA file:
> https://master.dl.sourceforge.net/project/linuxovafiles/GUI/ArcoLinux-vm.ova?viasf=1 
> to /var/lib/vz/import/ArcoLinux-vm.ova
>
> 2b353ac59598bac24d4cf0721e367935d00178a31d84e09c94425bdee3dab6a1 
> /var/lib/vz/import/ArcoLinux-vm.ova
>
> Tested-by: Filip Schauer <f.schauer@proxmox.com>
>
> On 18/11/2024 16:29, Dominik Csapak wrote:
>> This series enables importing ova/ovf from directory based storages,
>> inclusive upload/download via the webui (ova only).
>>
>> It also improves the ovf importer by parsing the ostype, nics, bootorder
>> (and firmware from vmware exported files).
>>
>> We now extract the images into either a given target storage or in the
>> import storage in the 'images' dir so accidentally left over images
>> are discoverable by the ui/cli.
>>
>> This version includes a modified qemu-server patch of fabians 
>> hardening series:
>> https://lore.proxmox.com/pve-devel/20241104104221.228730-1-f.gruenbichler@proxmox.com/ 
>>
>>
>> I sent the qemu-server patch from fabian again but omitted some
>> problematic checks. I add them later with a check
>> against the import vtype again (last patch in qemu-server)
>>
>> changes from v6:
>> * rebase on master (omit already applied patches)
>> * style/indentation fixes
>> * omit explicit check for .ova in upload path
>> * use assert_file_validity check for download_file_from_url
>> * only warn on whitespace in parent image (file_size_info) to 
>> preserve backward compatibility
>>    for possible existing cases
>> * fix cleanup on live-import  (added wrong variable to cleanup list)
>> * renamed 'import-extraction-storage' to 'import-working-storage'
>> * mad priv check for extraction storage simpler (combine branches)
>> * reworded some gettexts
>>
>> changes from v5:
>> * removed leftover hunks in makefile
>> * moved ova checks to correct patch
>> * split up error messages for unexpected format
>> * remove unnecessary untaint
>> * reword error message
>> * reintroduce symlink check in ova/ovf check
>> * added sanity check for ovas after uploading/downloading
>> * added new patch for checking import vtypes
>> * fixed issue with files with absolute path
>>
>> changes from v4:
>> * rebased on master/fabians series
>> * added the file_size_info check for untrusted images after extracting
>>
>> changes from v3:
>> * fixed dependencies in control file
>> * removed unnecessary use statements
>> * removed unnecessary remove helper
>> * moved 'needs_extract' helper to qemu-server
>> * removed import storage param from PUT call
>> * check down/uploaded ova filename more strictly (same as listing)
>> * improved filepath checking in ovf
>> * forbid importing when extracted image references a base/backing file
>> * instead of trying to manually create a proper filename, use 'alloc' to
>>    create a small (1M) file with the same format and overwrite it with
>>    renaming. this also solves the cluster locking issue
>> * prefer using PVE::Storage functions instead of plugin methods in
>>    ova extraction code
>> * use $vollist for cleaning up extracted images in qemu-server and
>>    add manual cleanup for the success case
>>
>> changes from v2:
>> * use better 'format' values for embedded images (e.g. ova+vmdk)
>> * use this format to decide if images should be extracted
>> * consistent use of the 'safe character' classes when listing
>>    and parsing
>> * also list vmdk/qcow2/raw images in content listing
>>    (this will be useful when we have a gui for the 'import-from'
>>    in the wizard/disk edit for vms)
>> * a few gui adaptions
>>
>>
>> changes from v1:
>> * move ovf code to GuestImport
>> * move extract/checking code to GuestImport
>> * don't return 'image' types from import volumes
>> * use allow 'safe' characters for filenames of ova/ovfs and inside
>> * check for non-regular files (e.g. symlinks) after extraction
>> * add new 'import-extraction-storage' for import
>> * rename panel in gui for directory storages
>> * typo fixes
>> * and probably more, see the individual patches for details
>>
>> pve-storage:
>>
>> Dominik Csapak (11):
>>    plugin: dir: implement import content type
>>    plugin: dir: handle ova files for import
>>    ovf: improve and simplify path checking code
>>    ovf: implement parsing the ostype
>>    ovf: implement parsing out firmware type
>>    ovf: implement rudimentary boot order
>>    ovf: implement parsing nics
>>    api: allow ova upload/download
>>    plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs
>>    add 'import' content type to 'check_volume_access'
>>    plugin: file_size_info: warn on parent images with unusual path
>>
>>   src/PVE/API2/Storage/Status.pm                |  56 +++++-
>>   src/PVE/GuestImport.pm                        |  79 ++++++++
>>   src/PVE/GuestImport/OVF.pm                    | 187 ++++++++++++++++--
>>   src/PVE/Makefile                              |   1 +
>>   src/PVE/Storage.pm                            |  23 ++-
>>   src/PVE/Storage/BTRFSPlugin.pm                |   5 +
>>   src/PVE/Storage/CIFSPlugin.pm                 |   6 +-
>>   src/PVE/Storage/CephFSPlugin.pm               |   6 +-
>>   src/PVE/Storage/DirPlugin.pm                  |  52 ++++-
>>   src/PVE/Storage/GlusterfsPlugin.pm            |   6 +-
>>   src/PVE/Storage/NFSPlugin.pm                  |   6 +-
>>   src/PVE/Storage/Plugin.pm                     |  16 +-
>>   .../ovf_manifests/Win10-Liz_no_default_ns.ovf |   1 +
>>   src/test/parse_volname_test.pm                |  33 ++++
>>   src/test/path_to_volume_id_test.pm            |  21 ++
>>   src/test/run_ovf_tests.pl                     |  14 ++
>>   16 files changed, 480 insertions(+), 32 deletions(-)
>>   create mode 100644 src/PVE/GuestImport.pm
>>
>> qemu-server:
>>
>> Dominik Csapak (4):
>>    use OVF from Storage
>>    api: create: implement extracting disks when needed for import-from
>>    api: create: add 'import-working-storage' parameter
>>    api: check untrusted image files for import content type
>>
>> Fabian Grünbichler (1):
>>    disk import: add additional safeguards for imported image files
>>
>>   PVE/API2/Qemu.pm                              | 108 ++++++--
>>   PVE/CLI/qm.pm                                 |   4 +-
>>   PVE/QemuServer.pm                             |  12 +
>>   PVE/QemuServer/Helpers.pm                     |   5 +
>>   PVE/QemuServer/Makefile                       |   1 -
>>   PVE/QemuServer/OVF.pm                         | 242 ------------------
>>   debian/control                                |   2 -
>>   test/Makefile                                 |   5 +-
>>   test/ovf_manifests/Win10-Liz-disk1.vmdk       | Bin 65536 -> 0 bytes
>>   test/ovf_manifests/Win10-Liz.ovf              | 142 ----------
>>   .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 142 ----------
>>   test/ovf_manifests/Win_2008_R2_two-disks.ovf  | 145 -----------
>>   test/ovf_manifests/disk1.vmdk                 | Bin 65536 -> 0 bytes
>>   test/ovf_manifests/disk2.vmdk                 | Bin 65536 -> 0 bytes
>>   test/run_ovf_tests.pl                         |  71 -----
>>   15 files changed, 112 insertions(+), 767 deletions(-)
>>   delete mode 100644 PVE/QemuServer/OVF.pm
>>   delete mode 100644 test/ovf_manifests/Win10-Liz-disk1.vmdk
>>   delete mode 100755 test/ovf_manifests/Win10-Liz.ovf
>>   delete mode 100755 test/ovf_manifests/Win10-Liz_no_default_ns.ovf
>>   delete mode 100755 test/ovf_manifests/Win_2008_R2_two-disks.ovf
>>   delete mode 100644 test/ovf_manifests/disk1.vmdk
>>   delete mode 100644 test/ovf_manifests/disk2.vmdk
>>   delete mode 100755 test/run_ovf_tests.pl
>>
>> pve-manager:
>>
>> Dominik Csapak (9):
>>    ui: guest import: add ova-needs-extracting warning text
>>    ui: enable import content type for relevant storages
>>    ui: enable upload/download/remove buttons for 'import' type storages
>>    ui: disable 'import' button for non importable formats
>>    ui: import: improve rendering of volume names
>>    ui: guest import: add storage selector for ova extraction storage
>>    ui: guest import: change icon/text for non-esxi import storage
>>    ui: import: show size for dir-based storages
>>    ui: import: adapt live import help text to ova
>>
>>   www/manager6/Utils.js                    |  9 +++++++-
>>   www/manager6/form/ContentTypeSelector.js |  2 +-
>>   www/manager6/storage/Browser.js          | 25 ++++++++++++++++-----
>>   www/manager6/storage/CephFSEdit.js       |  2 +-
>>   www/manager6/storage/GlusterFsEdit.js    |  2 +-
>>   www/manager6/window/GuestImport.js       | 28 +++++++++++++++++++++++-
>>   www/manager6/window/UploadToStorage.js   |  1 +
>>   7 files changed, 58 insertions(+), 11 deletions(-)
>>
>
>
> _______________________________________________
> pve-devel mailing list
> pve-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel