From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 2FE4461D41 for ; Thu, 22 Oct 2020 17:35:01 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 1355521512 for ; Thu, 22 Oct 2020 17:34:31 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id C80C0214F8 for ; Thu, 22 Oct 2020 17:34:29 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 8890E45EE3 for ; Thu, 22 Oct 2020 17:34:29 +0200 (CEST) From: Stefan Reiter To: pve-devel@lists.proxmox.com Date: Thu, 22 Oct 2020 17:34:16 +0200 Message-Id: <20201022153420.16971-1-s.reiter@proxmox.com> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.030 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [lib.rs, commands.rs, qemuserver.pm] Subject: [pve-devel] [PATCH 0/4] Keep dirty-bitmaps for PBS during migration X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Oct 2020 15:35:01 -0000 Allow dirty bitmaps to persist over a live migration, allowing incremental backups even after a VM has been moved to another node. Migrating the dirty-bitmaps themselves is supported natively by QEMU, only requiring a fix for a bug leading to hangs when migrating bitmaps with a granularity as low as we are using (see first patch). The second and third patches are the interesting bits, implementing "savevm"-style migration for all PBS state that is being kept in memory (checksums). The last patch brings it all together by enabling it in qemu-server when support is detected (which is important, since we never want to enable "dirty-bitmaps" capability on unpatched versions of QEMU, as that will lead to unrecoverable VM hangs). Compatility is maintained between all pve-qemu-kvm and qemu-server versions, with the usual exception of not being able to migrate VMs started with a newer QEMU version to a machine with an older one. qemu: Stefan Reiter (2): migration/block-dirty-bitmap: fix larger granularity bitmaps PVE: Migrate dirty bitmap state via savevm include/migration/misc.h | 3 ++ migration/Makefile.objs | 1 + migration/block-dirty-bitmap.c | 3 +- migration/pbs-state.c | 92 ++++++++++++++++++++++++++++++++++ pve-backup.c | 1 + qapi/block-core.json | 9 +++- softmmu/vl.c | 1 + 7 files changed, 108 insertions(+), 2 deletions(-) create mode 100644 migration/pbs-state.c proxmox-backup-qemu: Stefan Reiter (1): add state serializing and loading functions Cargo.toml | 1 + current-api.h | 20 ++++++++++++++++++++ src/commands.rs | 19 +++++++++++++++++++ src/lib.rs | 34 ++++++++++++++++++++++++++++++++++ 4 files changed, 74 insertions(+) qemu-server: Stefan Reiter (1): migrate: enable dirty-bitmap migration PVE/QemuServer.pm | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) -- 2.20.1