From: Dominik Csapak <d.csapak@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH storage/qemu-server/manager v3] implement ova/ovf import for file based storages
Date: Mon, 29 Apr 2024 13:21:01 +0200 [thread overview]
Message-ID: <20240429112124.3819357-1-d.csapak@proxmox.com> (raw)
This series enables importing ova/ovf from directory based storages,
inclusive upload/download via the webui (ova only).
It also improves the ovf importer by parsing the ostype, nics, bootorder
(and firmware from vmware exported files).
I opted to move the OVF.pm to pve-storage, since there is no
real other place where we could put it. I put it in a new module
'GuestImport'
We now extract the images into either a given target storage or in the
import storage in the 'images' dir so accidentally left over images
are discoverable by the ui/cli.
changes from v2:
* use better 'format' values for embedded images (e.g. ova+vmdk)
* use this format to decide if images should be extracted
* consistent use of the 'safe character' classes when listing
and parsing
* also list vmdk/qcow2/raw images in content listing
(this will be useful when we have a gui for the 'import-from'
in the wizard/disk edit for vms)
* a few gui adaptions
changes from v1:
* move ovf code to GuestImport
* move extract/checking code to GuestImport
* don't return 'image' types from import volumes
* use allow 'safe' characters for filenames of ova/ovfs and inside
* check for non-regular files (e.g. symlinks) after extraction
* add new 'import-extraction-storage' for import
* rename panel in gui for directory storages
* typo fixes
* and probably more, see the individual patches for details
pve-storage:
Dominik Csapak (10):
copy OVF.pm from qemu-server
plugin: dir: implement import content type
plugin: dir: handle ova files for import
ovf: implement parsing the ostype
ovf: implement parsing out firmware type
ovf: implement rudimentary boot order
ovf: implement parsing nics
api: allow ova upload/download
plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs
add 'import' content type to 'check_volume_access'
src/PVE/API2/Storage/Status.pm | 19 +-
src/PVE/GuestImport.pm | 100 +++++
src/PVE/GuestImport/Makefile | 3 +
src/PVE/GuestImport/OVF.pm | 383 ++++++++++++++++++
src/PVE/Makefile | 2 +
src/PVE/Storage.pm | 21 +-
src/PVE/Storage/BTRFSPlugin.pm | 5 +
src/PVE/Storage/CIFSPlugin.pm | 6 +-
src/PVE/Storage/CephFSPlugin.pm | 6 +-
src/PVE/Storage/DirPlugin.pm | 52 ++-
src/PVE/Storage/GlusterfsPlugin.pm | 6 +-
src/PVE/Storage/Makefile | 1 +
src/PVE/Storage/NFSPlugin.pm | 6 +-
src/PVE/Storage/Plugin.pm | 16 +-
src/test/Makefile | 5 +-
src/test/ovf_manifests/Win10-Liz-disk1.vmdk | Bin 0 -> 65536 bytes
src/test/ovf_manifests/Win10-Liz.ovf | 142 +++++++
.../ovf_manifests/Win10-Liz_no_default_ns.ovf | 143 +++++++
.../ovf_manifests/Win_2008_R2_two-disks.ovf | 145 +++++++
src/test/ovf_manifests/disk1.vmdk | Bin 0 -> 65536 bytes
src/test/ovf_manifests/disk2.vmdk | Bin 0 -> 65536 bytes
src/test/parse_volname_test.pm | 33 ++
src/test/path_to_volume_id_test.pm | 21 +
src/test/run_ovf_tests.pl | 85 ++++
24 files changed, 1188 insertions(+), 12 deletions(-)
create mode 100644 src/PVE/GuestImport.pm
create mode 100644 src/PVE/GuestImport/Makefile
create mode 100644 src/PVE/GuestImport/OVF.pm
create mode 100644 src/test/ovf_manifests/Win10-Liz-disk1.vmdk
create mode 100755 src/test/ovf_manifests/Win10-Liz.ovf
create mode 100755 src/test/ovf_manifests/Win10-Liz_no_default_ns.ovf
create mode 100755 src/test/ovf_manifests/Win_2008_R2_two-disks.ovf
create mode 100644 src/test/ovf_manifests/disk1.vmdk
create mode 100644 src/test/ovf_manifests/disk2.vmdk
create mode 100755 src/test/run_ovf_tests.pl
qemu-server:
Dominik Csapak (4):
api: delete unused OVF.pm
use OVF from Storage
api: create: implement extracting disks when needed for import-from
api: create: add 'import-extraction-storage' parameter
PVE/API2/Qemu.pm | 92 ++++++-
PVE/API2/Qemu/Makefile | 2 +-
PVE/API2/Qemu/OVF.pm | 53 ----
PVE/CLI/qm.pm | 4 +-
PVE/QemuServer.pm | 5 +-
PVE/QemuServer/Helpers.pm | 10 +
PVE/QemuServer/Makefile | 1 -
PVE/QemuServer/OVF.pm | 242 ------------------
test/Makefile | 5 +-
test/ovf_manifests/Win10-Liz-disk1.vmdk | Bin 65536 -> 0 bytes
test/ovf_manifests/Win10-Liz.ovf | 142 ----------
.../ovf_manifests/Win10-Liz_no_default_ns.ovf | 142 ----------
test/ovf_manifests/Win_2008_R2_two-disks.ovf | 145 -----------
test/ovf_manifests/disk1.vmdk | Bin 65536 -> 0 bytes
test/ovf_manifests/disk2.vmdk | Bin 65536 -> 0 bytes
test/run_ovf_tests.pl | 71 -----
16 files changed, 96 insertions(+), 818 deletions(-)
delete mode 100644 PVE/API2/Qemu/OVF.pm
delete mode 100644 PVE/QemuServer/OVF.pm
delete mode 100644 test/ovf_manifests/Win10-Liz-disk1.vmdk
delete mode 100755 test/ovf_manifests/Win10-Liz.ovf
delete mode 100755 test/ovf_manifests/Win10-Liz_no_default_ns.ovf
delete mode 100755 test/ovf_manifests/Win_2008_R2_two-disks.ovf
delete mode 100644 test/ovf_manifests/disk1.vmdk
delete mode 100644 test/ovf_manifests/disk2.vmdk
delete mode 100755 test/run_ovf_tests.pl
pve-manager:
Dominik Csapak (9):
ui: fix special 'import' icon for non-esxi storages
ui: guest import: add ova-needs-extracting warning text
ui: enable import content type for relevant storages
ui: enable upload/download/remove buttons for 'import' type storages
ui: disable 'import' button for non importable formats
ui: import: improve rendering of volume names
ui: guest import: add storage selector for ova extraction storage
ui: guest import: change icon/text for non-esxi import storage
ui: import: show size for dir-based storages
www/manager6/Utils.js | 11 +++++++++--
www/manager6/form/ContentTypeSelector.js | 2 +-
www/manager6/storage/Browser.js | 25 ++++++++++++++++++------
www/manager6/storage/CephFSEdit.js | 2 +-
www/manager6/storage/GlusterFsEdit.js | 2 +-
www/manager6/window/GuestImport.js | 24 +++++++++++++++++++++++
www/manager6/window/UploadToStorage.js | 1 +
7 files changed, 56 insertions(+), 11 deletions(-)
--
2.39.2
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
next reply other threads:[~2024-04-29 11:22 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-04-29 11:21 Dominik Csapak [this message]
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 01/10] copy OVF.pm from qemu-server Dominik Csapak
2024-05-22 8:56 ` Fabian Grünbichler
2024-05-22 9:35 ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 02/10] plugin: dir: implement import content type Dominik Csapak
2024-05-22 9:24 ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 03/10] plugin: dir: handle ova files for import Dominik Csapak
2024-05-22 10:08 ` Fabian Grünbichler
2024-05-23 10:40 ` Dominik Csapak
2024-05-23 12:25 ` Fabian Grünbichler
2024-05-23 12:32 ` Dominik Csapak
2024-05-22 13:13 ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 04/10] ovf: implement parsing the ostype Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 05/10] ovf: implement parsing out firmware type Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 06/10] ovf: implement rudimentary boot order Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 07/10] ovf: implement parsing nics Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 08/10] api: allow ova upload/download Dominik Csapak
2024-05-22 10:20 ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 09/10] plugin: enable import for nfs/btrfs/cifs/cephfs/glusterfs Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH storage v3 10/10] add 'import' content type to 'check_volume_access' Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH qemu-server v3 1/4] api: delete unused OVF.pm Dominik Csapak
2024-05-22 10:25 ` Fabian Grünbichler
2024-05-22 10:26 ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH qemu-server v3 2/4] use OVF from Storage Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH qemu-server v3 3/4] api: create: implement extracting disks when needed for import-from Dominik Csapak
2024-05-22 12:55 ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH qemu-server v3 4/4] api: create: add 'import-extraction-storage' parameter Dominik Csapak
2024-05-22 12:16 ` Fabian Grünbichler
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 1/9] ui: fix special 'import' icon for non-esxi storages Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 2/9] ui: guest import: add ova-needs-extracting warning text Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 3/9] ui: enable import content type for relevant storages Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 4/9] ui: enable upload/download/remove buttons for 'import' type storages Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 5/9] ui: disable 'import' button for non importable formats Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 6/9] ui: import: improve rendering of volume names Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 7/9] ui: guest import: add storage selector for ova extraction storage Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 8/9] ui: guest import: change icon/text for non-esxi import storage Dominik Csapak
2024-04-29 11:21 ` [pve-devel] [PATCH manager v3 9/9] ui: import: show size for dir-based storages Dominik Csapak
2024-05-24 13:38 ` [pve-devel] [PATCH storage/qemu-server/manager v3] implement ova/ovf import for file based storages Dominik Csapak
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20240429112124.3819357-1-d.csapak@proxmox.com \
--to=d.csapak@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox