public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: "Fabian Grünbichler" <f.gruenbichler@proxmox.com>
To: Dominik Csapak <d.csapak@proxmox.com>,
	Proxmox VE development discussion <pve-devel@lists.proxmox.com>
Subject: Re: [pve-devel] [PATCH storage 3/9] plugin: dir: handle ova files for import
Date: Thu, 18 Apr 2024 08:46:57 +0200 (CEST)	[thread overview]
Message-ID: <20492589.8060.1713422817586@webmail.proxmox.com> (raw)
In-Reply-To: <781d464c-3229-47c3-9004-12fd2632810b@proxmox.com>

> Dominik Csapak <d.csapak@proxmox.com> hat am 17.04.2024 16:07 CEST geschrieben:
> On 4/17/24 15:52, Fabian Grünbichler wrote:
> > On April 17, 2024 3:10 pm, Dominik Csapak wrote:
> >> On 4/17/24 14:45, Fabian Grünbichler wrote:
> >>> On April 16, 2024 3:18 pm, Dominik Csapak wrote:
> >>>> +sub cleanup_extracted_image {
> >>>
> >>> same for this?
> >>>
> >>>> +    my ($source) = @_;
> >>>> +
> >>>> +    if ($source =~ m|^(/.+/\.tmp_[0-9]+_[0-9]+)/[^/]+$|) {
> >>>> +	my $tmpdir = $1;
> >>>> +
> >>>> +	unlink $source or $! == ENOENT or die "removing image $source failed: $!\n";
> >>>> +	rmdir $tmpdir or $! == ENOENT or die "removing tmpdir $tmpdir failed: $!\n";
> >>>> +    } else {
> >>>> +	die "invalid extraced image path '$source'\n";
> >>>
> >>> nit: typo
> >>>
> >>> these are also not discoverable if the error handling in qemu-server
> >>> failed for some reason.. might be a source of unwanted space
> >>> consumption..
> >>
> >> any suggestions for better handling that cleanup?
> >> we could put it at the beginning of each cleanup step, that should
> >> at least make sure we cleaned up the temporary images
> > 
> > we could extract them into images/XXX/vm-XXX-disk-.. directly (or
> > rename/move them there after extraction), that way at least they could
> > be cleaned up via the storage API or rescan + delete (and via a regular
> > vdisk_free in qemu-server, instead of requiring a special helper).
> > 
> > other than that, I don't think we have an easy way of
> > - exposing them in list & free_image
> > - while ensuring nobody deletes them while the import is still going on
> >    (the target VM ownership checks ensure that at least via the UI if we
> >    make it an owned volume)
> > 
> > it would also allow skipping the conversion if the storage+format
> > already match the target spec as well..
> 
> mhmm that could work, but what if the storage does not have
> the 'images' content type enabled? should we simply fail then?

right, that would make it a bit limiting. we could clear the tmpdir on reboots? ;)

it might also be nice (as a follow-up?) to make the tmpdir configurable and/or see what limitations direct streaming actually has (other than live-import not working) - because if the OVA is on NFS/.. right now we incur a lot of back and forth copying..

I wonder if live-import even makes much sense here - if I have to copy/extract the disks anyway before starting the live-import (which then does another copy), I can just as well do a regular import and start the VM afterwards, especially if that saves me one copy action?


_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

  reply	other threads:[~2024-04-18  6:47 UTC|newest]

Thread overview: 67+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-04-16 13:18 [pve-devel] [PATCH storage/qemu-server/pve-manager] implement ova/ovf import for directory type storages Dominik Csapak
2024-04-16 13:18 ` [pve-devel] [PATCH storage 1/9] copy OVF.pm from qemu-server Dominik Csapak
2024-04-16 15:02   ` Thomas Lamprecht
2024-04-17  9:19     ` Fiona Ebner
2024-04-17  9:26       ` Thomas Lamprecht
2024-04-16 13:18 ` [pve-devel] [PATCH storage 2/9] plugin: dir: implement import content type Dominik Csapak
2024-04-17 10:07   ` Fiona Ebner
2024-04-17 10:07     ` Fiona Ebner
2024-04-17 13:13     ` Dominik Csapak
2024-04-17 12:46   ` Fabian Grünbichler
2024-04-16 13:18 ` [pve-devel] [PATCH storage 3/9] plugin: dir: handle ova files for import Dominik Csapak
2024-04-17 10:52   ` Fiona Ebner
2024-04-17 13:07     ` Dominik Csapak
2024-04-17 13:39       ` Fabian Grünbichler
2024-04-18  7:22       ` Fiona Ebner
2024-04-18  7:25         ` Fiona Ebner
2024-04-18  8:55         ` Fabian Grünbichler
2024-04-17 12:45   ` Fabian Grünbichler
2024-04-17 13:10     ` Dominik Csapak
2024-04-17 13:52       ` Fabian Grünbichler
2024-04-17 14:07         ` Dominik Csapak
2024-04-18  6:46           ` Fabian Grünbichler [this message]
2024-04-16 13:18 ` [pve-devel] [PATCH storage 4/9] ovf: implement parsing the ostype Dominik Csapak
2024-04-17 11:32   ` Fiona Ebner
2024-04-17 13:14     ` Dominik Csapak
2024-04-18  7:31       ` Fiona Ebner
2024-04-16 13:18 ` [pve-devel] [PATCH storage 5/9] ovf: implement parsing out firmware type Dominik Csapak
2024-04-17 11:43   ` Fiona Ebner
2024-04-16 13:18 ` [pve-devel] [PATCH storage 6/9] ovf: implement rudimentary boot order Dominik Csapak
2024-04-17 11:54   ` Fiona Ebner
2024-04-17 13:15     ` Dominik Csapak
2024-04-16 13:19 ` [pve-devel] [PATCH storage 7/9] ovf: implement parsing nics Dominik Csapak
2024-04-17 12:09   ` Fiona Ebner
2024-04-17 13:16     ` Dominik Csapak
2024-04-18  8:22   ` Fiona Ebner
2024-04-16 13:19 ` [pve-devel] [PATCH storage 8/9] api: allow ova upload/download Dominik Csapak
2024-04-18  8:05   ` Fiona Ebner
2024-04-16 13:19 ` [pve-devel] [PATCH storage 9/9] plugin: enable import for nfs/btfs/cifs/cephfs Dominik Csapak
2024-04-18  8:43   ` Fiona Ebner
2024-04-16 13:19 ` [pve-devel] [PATCH qemu-server 1/3] api: delete unused OVF.pm Dominik Csapak
2024-04-18  8:52   ` Fiona Ebner
2024-04-18  8:57     ` Dominik Csapak
2024-04-18  9:03       ` Fiona Ebner
2024-04-16 13:19 ` [pve-devel] [PATCH qemu-server 2/3] use OVF from Storage Dominik Csapak
2024-04-18  9:07   ` Fiona Ebner
2024-04-16 13:19 ` [pve-devel] [PATCH qemu-server 3/3] api: create: implement extracting disks when needed for import-from Dominik Csapak
2024-04-18  9:41   ` Fiona Ebner
2024-04-18  9:48     ` Dominik Csapak
2024-04-18  9:55       ` Fiona Ebner
2024-04-18  9:58         ` Dominik Csapak
2024-04-18 10:01           ` Fiona Ebner
2024-04-16 13:19 ` [pve-devel] [PATCH manager 1/4] ui: fix special 'import' icon for non-esxi storages Dominik Csapak
2024-04-16 13:19 ` [pve-devel] [PATCH manager 2/4] ui: guest import: add ova-needs-extracting warning text Dominik Csapak
2024-04-16 13:19 ` [pve-devel] [PATCH manager 3/4] ui: enable import content type for relevant storages Dominik Csapak
2024-04-16 13:19 ` [pve-devel] [PATCH manager 4/4] ui: enable upload/download buttons for 'import' type storages Dominik Csapak
2024-04-17 12:37   ` Fabian Grünbichler
2024-04-18 11:20   ` Fiona Ebner
2024-04-18 11:23     ` Dominik Csapak
2024-04-18 11:26       ` Fiona Ebner
2024-04-17 13:11 ` [pve-devel] [PATCH storage/qemu-server/pve-manager] implement ova/ovf import for directory " Fabian Grünbichler
2024-04-17 13:19   ` Dominik Csapak
2024-04-18  6:40     ` Fabian Grünbichler
2024-04-18  9:27 ` Dominik Csapak
2024-04-18 10:35   ` Fiona Ebner
2024-04-18 11:10     ` Dominik Csapak
2024-04-18 11:13       ` Fiona Ebner
2024-04-18 11:17     ` Fabian Grünbichler

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20492589.8060.1713422817586@webmail.proxmox.com \
    --to=f.gruenbichler@proxmox.com \
    --cc=d.csapak@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal