From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 2E5C696E09 for ; Tue, 16 Apr 2024 15:19:49 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 08CB01B9EA for ; Tue, 16 Apr 2024 15:19:19 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Tue, 16 Apr 2024 15:19:13 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 2BE18451CF for ; Tue, 16 Apr 2024 15:19:13 +0200 (CEST) From: Dominik Csapak To: pve-devel@lists.proxmox.com Date: Tue, 16 Apr 2024 15:18:53 +0200 Message-Id: <20240416131909.2867605-1-d.csapak@proxmox.com> X-Mailer: git-send-email 2.39.2 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.236 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URI_NOVOWEL 0.5 URI hostname has long non-vowel sequence Subject: [pve-devel] [PATCH storage/qemu-server/pve-manager] implement ova/ovf import for directory type storages X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Apr 2024 13:19:49 -0000 This series enables importing ova/ovf from directory based storages, inclusive upload/download via the webui (ova only). It also improves the ovf importer by parsing the ostype, nics, bootorder (and firmware from vmware exported files). I currently opted to move the OVF.pm to pve-storage, since there is no real other place where we could put it. Building a seperate package from qemu-servers git repo would also not be ideal, since we still have a cyclic dev dependency then (If someone has a better idea how to handle that, please do tell, and i can do that in a v2) There are surely some wrinkles left i did not think of, but all in all, it should be pretty usable. E.g. i downloaded some ovas, uploaded them on my cephfs in my virtual cluster, and successfully imported that with live-import. The biggest caveat when importing from ovas is that we have to temporarily extract the disk images. I opted for doing that into the import storage, but if we have a better idea where to put that, i can implement it in a v2 (or as a follow up). For example, we could add a new 'tmpdir' parameter to the create call and use that for extractig. pve-storage: Dominik Csapak (9): copy OVF.pm from qemu-server plugin: dir: implement import content type plugin: dir: handle ova files for import ovf: implement parsing the ostype ovf: implement parsing out firmware type ovf: implement rudimentary boot order ovf: implement parsing nics api: allow ova upload/download plugin: enable import for nfs/btfs/cifs/cephfs src/PVE/API2/Storage/Status.pm | 15 +- src/PVE/Storage.pm | 78 +++- src/PVE/Storage/BTRFSPlugin.pm | 5 + src/PVE/Storage/CIFSPlugin.pm | 6 +- src/PVE/Storage/CephFSPlugin.pm | 6 +- src/PVE/Storage/DirPlugin.pm | 53 ++- src/PVE/Storage/Makefile | 1 + src/PVE/Storage/NFSPlugin.pm | 6 +- src/PVE/Storage/OVF.pm | 381 ++++++++++++++++++ src/PVE/Storage/Plugin.pm | 23 +- src/test/Makefile | 5 +- src/test/ovf_manifests/Win10-Liz-disk1.vmdk | Bin 0 -> 65536 bytes src/test/ovf_manifests/Win10-Liz.ovf | 142 +++++++ .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 143 +++++++ .../ovf_manifests/Win_2008_R2_two-disks.ovf | 145 +++++++ src/test/ovf_manifests/disk1.vmdk | Bin 0 -> 65536 bytes src/test/ovf_manifests/disk2.vmdk | Bin 0 -> 65536 bytes src/test/parse_volname_test.pm | 13 + src/test/path_to_volume_id_test.pm | 16 + src/test/run_ovf_tests.pl | 83 ++++ 20 files changed, 1112 insertions(+), 9 deletions(-) create mode 100644 src/PVE/Storage/OVF.pm create mode 100644 src/test/ovf_manifests/Win10-Liz-disk1.vmdk create mode 100755 src/test/ovf_manifests/Win10-Liz.ovf create mode 100755 src/test/ovf_manifests/Win10-Liz_no_default_ns.ovf create mode 100755 src/test/ovf_manifests/Win_2008_R2_two-disks.ovf create mode 100644 src/test/ovf_manifests/disk1.vmdk create mode 100644 src/test/ovf_manifests/disk2.vmdk create mode 100755 src/test/run_ovf_tests.pl qemu-server: Dominik Csapak (3): api: delete unused OVF.pm use OVF from Storage api: create: implement extracting disks when needed for import-from PVE/API2/Qemu.pm | 26 +- PVE/API2/Qemu/Makefile | 2 +- PVE/API2/Qemu/OVF.pm | 53 ---- PVE/CLI/qm.pm | 4 +- PVE/QemuServer.pm | 5 +- PVE/QemuServer/Helpers.pm | 9 + PVE/QemuServer/Makefile | 1 - PVE/QemuServer/OVF.pm | 242 ------------------ test/Makefile | 5 +- test/ovf_manifests/Win10-Liz-disk1.vmdk | Bin 65536 -> 0 bytes test/ovf_manifests/Win10-Liz.ovf | 142 ---------- .../ovf_manifests/Win10-Liz_no_default_ns.ovf | 142 ---------- test/ovf_manifests/Win_2008_R2_two-disks.ovf | 145 ----------- test/ovf_manifests/disk1.vmdk | Bin 65536 -> 0 bytes test/ovf_manifests/disk2.vmdk | Bin 65536 -> 0 bytes test/run_ovf_tests.pl | 71 ----- 16 files changed, 37 insertions(+), 810 deletions(-) delete mode 100644 PVE/API2/Qemu/OVF.pm delete mode 100644 PVE/QemuServer/OVF.pm delete mode 100644 test/ovf_manifests/Win10-Liz-disk1.vmdk delete mode 100755 test/ovf_manifests/Win10-Liz.ovf delete mode 100755 test/ovf_manifests/Win10-Liz_no_default_ns.ovf delete mode 100755 test/ovf_manifests/Win_2008_R2_two-disks.ovf delete mode 100644 test/ovf_manifests/disk1.vmdk delete mode 100644 test/ovf_manifests/disk2.vmdk delete mode 100755 test/run_ovf_tests.pl pve-manager: Dominik Csapak (4): ui: fix special 'import' icon for non-esxi storages ui: guest import: add ova-needs-extracting warning text ui: enable import content type for relevant storages ui: enable upload/download buttons for 'import' type storages www/manager6/Utils.js | 3 ++- www/manager6/form/ContentTypeSelector.js | 2 +- www/manager6/storage/Browser.js | 7 ++++++- www/manager6/storage/CephFSEdit.js | 2 +- www/manager6/window/GuestImport.js | 1 + www/manager6/window/UploadToStorage.js | 1 + 6 files changed, 12 insertions(+), 4 deletions(-) -- 2.39.2