From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 28042617EB for ; Mon, 7 Sep 2020 09:00:00 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 087452F622 for ; Mon, 7 Sep 2020 08:59:30 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 27F6E2F614 for ; Mon, 7 Sep 2020 08:59:28 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id E350244A7C for ; Mon, 7 Sep 2020 08:59:27 +0200 (CEST) To: pve-devel@lists.proxmox.com References: <20200824092131.24617-1-s.reiter@proxmox.com> From: Dominik Csapak Message-ID: Date: Mon, 7 Sep 2020 08:59:26 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:81.0) Gecko/20100101 Thunderbird/81.0 MIME-Version: 1.0 In-Reply-To: <20200824092131.24617-1-s.reiter@proxmox.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 1.934 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -2.69 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [PATCH qemu-server] vzdump: use minimal VM config for offline backup X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Sep 2020 07:00:00 -0000 does that not break the feature that we can start a vm that started a backup while stopped? atm we can start a backup on a stopped vm, and then simply start it, without aborting the backup. if i read the patch correctly, the vm now has just a minimal config and not what the user configured On 8/24/20 11:21 AM, Stefan Reiter wrote: > In case we backup a stopped VM, we start an instance of QEMU to run the > backup job. This instance will be killed afterwards without ever running > the actual VM, so there's no need to potentially allocate or use host > system resources for features never used. > > The minimal_trim_opts array contains elements that will be cleaned from > the config before starting. > > We only write back the config in case of resume, which is never set together > with the new "minimal" option. > > Reported in the forum: > https://forum.proxmox.com/threads/pbs-tries-to-start-vms-during-backup-not-enough-ram.74773/ > > Signed-off-by: Stefan Reiter > --- > > It is weird to me that this even happens, AFAIU QEMU only allocates the guest > RAM as virtual as long as the guest doesn't write to it - but the forum post > proves that it does, and it's probably also a good idea to not assign hostpci > and usb devices for offline backups. > > PVE/QemuServer.pm | 34 ++++++++++++++++++++++++++++++++++ > PVE/VZDump/QemuServer.pm | 1 + > 2 files changed, 35 insertions(+) > > diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm > index a5ee8e2..73eb561 100644 > --- a/PVE/QemuServer.pm > +++ b/PVE/QemuServer.pm > @@ -4778,6 +4778,37 @@ sub vm_migrate_alloc_nbd_disks { > return $nbd; > } > > +my @minimal_trim_opts = ( > + qr/^hostpci\d+/, > + qr/^serial\d+/, > + qr/^usb\d+/, > + qr/^net\d+/, > + qr/^rng\d+/, > + qr/^ivshmem/, > + qr/^hugepages/, > + qr/^smp/, > + qr/^cores/, > + qr/^sockets/, > + qr/^vcpus/, > + qr/^cpu/, > + qr/^agent/, > + qr/^numa(?:\d+)?/, > +); > + > +my sub vm_trim_conf_minimal { > + my ($conf) = @_; > + > + $conf->{memory} = 1; > + $conf->{balloon} = 0; > + > + # remove anything that does not affect backup but can claim host resources > + foreach my $r (@minimal_trim_opts) { > + foreach my $key (keys %{$conf}) { > + delete $conf->{$key} if $key =~ $r; > + } > + } > +} > + > # see vm_start_nolock for parameters, additionally: > # migrate_opts: > # storagemap = parsed storage map for allocating NBD disks > @@ -4823,6 +4854,7 @@ sub vm_start { > # timeout => in seconds > # paused => start VM in paused state (backup) > # resume => resume from hibernation > +# minimal => only use necessary resources (backup) > # migrate_opts: > # nbd => volumes for NBD exports (vm_migrate_alloc_nbd_disks) > # migratedfrom => source node > @@ -4851,6 +4883,8 @@ sub vm_start_nolock { > $conf = PVE::QemuConfig->load_config($vmid); # update/reload > } > > + vm_trim_conf_minimal($conf) if $params->{minimal}; > + > PVE::QemuServer::Cloudinit::generate_cloudinitconfig($conf, $vmid); > > my $defaults = load_defaults(); > diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm > index 7297795..ea8faa3 100644 > --- a/PVE/VZDump/QemuServer.pm > +++ b/PVE/VZDump/QemuServer.pm > @@ -827,6 +827,7 @@ sub enforce_vm_running_for_backup { > skiplock => 1, > skiptemplate => 1, > paused => 1, > + minimal => !$self->{vm_was_running}, > }; > PVE::QemuServer::vm_start($self->{storecfg}, $vmid, $params); > }; >