From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <s.reiter@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id 873A863183
for <pve-devel@lists.proxmox.com>; Mon, 24 Aug 2020 11:21:45 +0200 (CEST)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 6BAEC1B923
for <pve-devel@lists.proxmox.com>; Mon, 24 Aug 2020 11:21:45 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[212.186.127.180])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256)
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS id 7DB6F1B91B
for <pve-devel@lists.proxmox.com>; Mon, 24 Aug 2020 11:21:44 +0200 (CEST)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 49DBE4480A
for <pve-devel@lists.proxmox.com>; Mon, 24 Aug 2020 11:21:44 +0200 (CEST)
From: Stefan Reiter <s.reiter@proxmox.com>
To: pve-devel@lists.proxmox.com
Date: Mon, 24 Aug 2020 11:21:31 +0200
Message-Id: <20200824092131.24617-1-s.reiter@proxmox.com>
X-Mailer: git-send-email 2.20.1
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.054 Adjusted score from AWL reputation of From: address
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/,
medium trust
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See
http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more
information. [proxmox.com, qemuserver.pm]
Subject: [pve-devel] [PATCH qemu-server] vzdump: use minimal VM config for
offline backup
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Mon, 24 Aug 2020 09:21:45 -0000
In case we backup a stopped VM, we start an instance of QEMU to run the
backup job. This instance will be killed afterwards without ever running
the actual VM, so there's no need to potentially allocate or use host
system resources for features never used.
The minimal_trim_opts array contains elements that will be cleaned from
the config before starting.
We only write back the config in case of resume, which is never set together
with the new "minimal" option.
Reported in the forum:
https://forum.proxmox.com/threads/pbs-tries-to-start-vms-during-backup-not-enough-ram.74773/
Signed-off-by: Stefan Reiter <s.reiter@proxmox.com>
---
It is weird to me that this even happens, AFAIU QEMU only allocates the guest
RAM as virtual as long as the guest doesn't write to it - but the forum post
proves that it does, and it's probably also a good idea to not assign hostpci
and usb devices for offline backups.
PVE/QemuServer.pm | 34 ++++++++++++++++++++++++++++++++++
PVE/VZDump/QemuServer.pm | 1 +
2 files changed, 35 insertions(+)
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index a5ee8e2..73eb561 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -4778,6 +4778,37 @@ sub vm_migrate_alloc_nbd_disks {
return $nbd;
}
+my @minimal_trim_opts = (
+ qr/^hostpci\d+/,
+ qr/^serial\d+/,
+ qr/^usb\d+/,
+ qr/^net\d+/,
+ qr/^rng\d+/,
+ qr/^ivshmem/,
+ qr/^hugepages/,
+ qr/^smp/,
+ qr/^cores/,
+ qr/^sockets/,
+ qr/^vcpus/,
+ qr/^cpu/,
+ qr/^agent/,
+ qr/^numa(?:\d+)?/,
+);
+
+my sub vm_trim_conf_minimal {
+ my ($conf) = @_;
+
+ $conf->{memory} = 1;
+ $conf->{balloon} = 0;
+
+ # remove anything that does not affect backup but can claim host resources
+ foreach my $r (@minimal_trim_opts) {
+ foreach my $key (keys %{$conf}) {
+ delete $conf->{$key} if $key =~ $r;
+ }
+ }
+}
+
# see vm_start_nolock for parameters, additionally:
# migrate_opts:
# storagemap = parsed storage map for allocating NBD disks
@@ -4823,6 +4854,7 @@ sub vm_start {
# timeout => in seconds
# paused => start VM in paused state (backup)
# resume => resume from hibernation
+# minimal => only use necessary resources (backup)
# migrate_opts:
# nbd => volumes for NBD exports (vm_migrate_alloc_nbd_disks)
# migratedfrom => source node
@@ -4851,6 +4883,8 @@ sub vm_start_nolock {
$conf = PVE::QemuConfig->load_config($vmid); # update/reload
}
+ vm_trim_conf_minimal($conf) if $params->{minimal};
+
PVE::QemuServer::Cloudinit::generate_cloudinitconfig($conf, $vmid);
my $defaults = load_defaults();
diff --git a/PVE/VZDump/QemuServer.pm b/PVE/VZDump/QemuServer.pm
index 7297795..ea8faa3 100644
--- a/PVE/VZDump/QemuServer.pm
+++ b/PVE/VZDump/QemuServer.pm
@@ -827,6 +827,7 @@ sub enforce_vm_running_for_backup {
skiplock => 1,
skiptemplate => 1,
paused => 1,
+ minimal => !$self->{vm_was_running},
};
PVE::QemuServer::vm_start($self->{storecfg}, $vmid, $params);
};
--
2.20.1