public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pve-devel] [PATCH storage/qemu-server/manager v3] implement ova/ovf import for file based storages
@ 2024-04-29 11:21 Dominik Csapak
  2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 01/10] copy OVF.pm from qemu-server Dominik Csapak
                   ` (23 more replies)
  0 siblings, 24 replies; 38+ messages in thread
From: Dominik Csapak @ 2024-04-29 11:21 UTC (permalink / raw)
  To: pve-devel

This series enables importing ova/ovf from directory based storages,
inclusive upload/download via the webui (ova only).

It also improves the ovf importer by parsing the ostype, nics, bootorder
(and firmware from vmware exported files).

I opted to move the OVF.pm to pve-storage, since there is no
real other place where we could put it. I put it in a new module
'GuestImport'

We now extract the images into either a given target storage or in the
import storage in the 'images' dir so accidentally left over images
are discoverable by the ui/cli.

changes from v2:
* use better 'format' values for embedded images (e.g. ova+vmdk)
* use this format to decide if images should be extracted
* consistent use of the 'safe character' classes when listing
  and parsing
* also list vmdk/qcow2/raw images in content listing
  (this will be useful when we have a gui for the 'import-from'
  in the wizard/disk edit for vms)
* a few gui adaptions


changes from v1:
* move ovf code to GuestImport
* move extract/checking code to GuestImport
* don't return 'image' types from import volumes
* use allow 'safe' characters for filenames of ova/ovfs and inside
* check for non-regular files (e.g. symlinks) after extraction
* add new 'import-extraction-storage' for import
* rename panel in gui for directory storages
* typo fixes
* and probably more, see the individual patches for details

pve-storage:

Dominik Csapak (10):
  copy OVF.pm from qemu-server
  plugin: dir: implement import content type
  plugin: dir: handle ova files for import
  ovf: implement parsing the ostype
  ovf: implement parsing out firmware type
  ovf: implement rudimentary boot order
  ovf: implement parsing nics
  api: allow ova upload/download
  plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs
  add 'import' content type to 'check_volume_access'

 src/PVE/API2/Storage/Status.pm                |  19 +-
 src/PVE/GuestImport.pm                        | 100 +++++
 src/PVE/GuestImport/Makefile                  |   3 +
 src/PVE/GuestImport/OVF.pm                    | 383 ++++++++++++++++++
 src/PVE/Makefile                              |   2 +
 src/PVE/Storage.pm                            |  21 +-
 src/PVE/Storage/BTRFSPlugin.pm                |   5 +
 src/PVE/Storage/CIFSPlugin.pm                 |   6 +-
 src/PVE/Storage/CephFSPlugin.pm               |   6 +-
 src/PVE/Storage/DirPlugin.pm                  |  52 ++-
 src/PVE/Storage/GlusterfsPlugin.pm            |   6 +-
 src/PVE/Storage/Makefile                      |   1 +
 src/PVE/Storage/NFSPlugin.pm                  |   6 +-
 src/PVE/Storage/Plugin.pm                     |  16 +-
 src/test/Makefile                             |   5 +-
 src/test/ovf_manifests/Win10-Liz-disk1.vmdk   | Bin 0 -> 65536 bytes
 src/test/ovf_manifests/Win10-Liz.ovf          | 142 +++++++
 .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 143 +++++++
 .../ovf_manifests/Win_2008_R2_two-disks.ovf   | 145 +++++++
 src/test/ovf_manifests/disk1.vmdk             | Bin 0 -> 65536 bytes
 src/test/ovf_manifests/disk2.vmdk             | Bin 0 -> 65536 bytes
 src/test/parse_volname_test.pm                |  33 ++
 src/test/path_to_volume_id_test.pm            |  21 +
 src/test/run_ovf_tests.pl                     |  85 ++++
 24 files changed, 1188 insertions(+), 12 deletions(-)
 create mode 100644 src/PVE/GuestImport.pm
 create mode 100644 src/PVE/GuestImport/Makefile
 create mode 100644 src/PVE/GuestImport/OVF.pm
 create mode 100644 src/test/ovf_manifests/Win10-Liz-disk1.vmdk
 create mode 100755 src/test/ovf_manifests/Win10-Liz.ovf
 create mode 100755 src/test/ovf_manifests/Win10-Liz_no_default_ns.ovf
 create mode 100755 src/test/ovf_manifests/Win_2008_R2_two-disks.ovf
 create mode 100644 src/test/ovf_manifests/disk1.vmdk
 create mode 100644 src/test/ovf_manifests/disk2.vmdk
 create mode 100755 src/test/run_ovf_tests.pl

qemu-server:

Dominik Csapak (4):
  api: delete unused OVF.pm
  use OVF from Storage
  api: create: implement extracting disks when needed for import-from
  api: create: add 'import-extraction-storage' parameter

 PVE/API2/Qemu.pm                              |  92 ++++++-
 PVE/API2/Qemu/Makefile                        |   2 +-
 PVE/API2/Qemu/OVF.pm                          |  53 ----
 PVE/CLI/qm.pm                                 |   4 +-
 PVE/QemuServer.pm                             |   5 +-
 PVE/QemuServer/Helpers.pm                     |  10 +
 PVE/QemuServer/Makefile                       |   1 -
 PVE/QemuServer/OVF.pm                         | 242 ------------------
 test/Makefile                                 |   5 +-
 test/ovf_manifests/Win10-Liz-disk1.vmdk       | Bin 65536 -> 0 bytes
 test/ovf_manifests/Win10-Liz.ovf              | 142 ----------
 .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 142 ----------
 test/ovf_manifests/Win_2008_R2_two-disks.ovf  | 145 -----------
 test/ovf_manifests/disk1.vmdk                 | Bin 65536 -> 0 bytes
 test/ovf_manifests/disk2.vmdk                 | Bin 65536 -> 0 bytes
 test/run_ovf_tests.pl                         |  71 -----
 16 files changed, 96 insertions(+), 818 deletions(-)
 delete mode 100644 PVE/API2/Qemu/OVF.pm
 delete mode 100644 PVE/QemuServer/OVF.pm
 delete mode 100644 test/ovf_manifests/Win10-Liz-disk1.vmdk
 delete mode 100755 test/ovf_manifests/Win10-Liz.ovf
 delete mode 100755 test/ovf_manifests/Win10-Liz_no_default_ns.ovf
 delete mode 100755 test/ovf_manifests/Win_2008_R2_two-disks.ovf
 delete mode 100644 test/ovf_manifests/disk1.vmdk
 delete mode 100644 test/ovf_manifests/disk2.vmdk
 delete mode 100755 test/run_ovf_tests.pl

pve-manager:

Dominik Csapak (9):
  ui: fix special 'import' icon for non-esxi storages
  ui: guest import: add ova-needs-extracting warning text
  ui: enable import content type for relevant storages
  ui: enable upload/download/remove buttons for 'import' type storages
  ui: disable 'import' button for non importable formats
  ui: import: improve rendering of volume names
  ui: guest import: add storage selector for ova extraction storage
  ui: guest import: change icon/text for non-esxi import storage
  ui: import: show size for dir-based storages

 www/manager6/Utils.js                    | 11 +++++++++--
 www/manager6/form/ContentTypeSelector.js |  2 +-
 www/manager6/storage/Browser.js          | 25 ++++++++++++++++++------
 www/manager6/storage/CephFSEdit.js       |  2 +-
 www/manager6/storage/GlusterFsEdit.js    |  2 +-
 www/manager6/window/GuestImport.js       | 24 +++++++++++++++++++++++
 www/manager6/window/UploadToStorage.js   |  1 +
 7 files changed, 56 insertions(+), 11 deletions(-)

-- 
2.39.2



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel


^ permalink raw reply	[flat|nested] 38+ messages in thread

end of thread, other threads:[~2024-05-24 13:38 UTC | newest]

Thread overview: 38+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-29 11:21 [pve-devel] [PATCH storage/qemu-server/manager v3] implement ova/ovf import for file based storages Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 01/10] copy OVF.pm from qemu-server Dominik Csapak
2024-05-22  8:56   ` Fabian Grünbichler
2024-05-22  9:35   ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 02/10] plugin: dir: implement import content type Dominik Csapak
2024-05-22  9:24   ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 03/10] plugin: dir: handle ova files for import Dominik Csapak
2024-05-22 10:08   ` Fabian Grünbichler
2024-05-23 10:40     ` Dominik Csapak
2024-05-23 12:25       ` Fabian Grünbichler
2024-05-23 12:32         ` Dominik Csapak
2024-05-22 13:13   ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 04/10] ovf: implement parsing the ostype Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 05/10] ovf: implement parsing out firmware type Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 06/10] ovf: implement rudimentary boot order Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 07/10] ovf: implement parsing nics Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 08/10] api: allow ova upload/download Dominik Csapak
2024-05-22 10:20   ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 09/10] plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 10/10] add 'import' content type to 'check_volume_access' Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH qemu-server v3 1/4] api: delete unused OVF.pm Dominik Csapak
2024-05-22 10:25   ` Fabian Grünbichler
2024-05-22 10:26     ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH qemu-server v3 2/4] use OVF from Storage Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH qemu-server v3 3/4] api: create: implement extracting disks when needed for import-from Dominik Csapak
2024-05-22 12:55   ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH qemu-server v3 4/4] api: create: add 'import-extraction-storage' parameter Dominik Csapak
2024-05-22 12:16   ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 1/9] ui: fix special 'import' icon for non-esxi storages Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 2/9] ui: guest import: add ova-needs-extracting warning text Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 3/9] ui: enable import content type for relevant storages Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 4/9] ui: enable upload/download/remove buttons for 'import' type storages Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 5/9] ui: disable 'import' button for non importable formats Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 6/9] ui: import: improve rendering of volume names Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 7/9] ui: guest import: add storage selector for ova extraction storage Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 8/9] ui: guest import: change icon/text for non-esxi import storage Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 9/9] ui: import: show size for dir-based storages Dominik Csapak
2024-05-24 13:38 ` [pve-devel] [PATCH storage/qemu-server/manager v3] implement ova/ovf import for file based storages Dominik Csapak

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal