From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id C577174605 for ; Mon, 19 Apr 2021 08:49:44 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id B6A7E10FBE for ; Mon, 19 Apr 2021 08:49:44 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 2FC1D10FAC for ; Mon, 19 Apr 2021 08:49:41 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id F10FD45AE6 for ; Mon, 19 Apr 2021 08:49:40 +0200 (CEST) To: pve-devel@lists.proxmox.com References: <20210129151143.10014-1-f.ebner@proxmox.com> From: Fabian Ebner Message-ID: <8b189efb-377a-4c98-786a-5cce1525f48f@proxmox.com> Date: Mon, 19 Apr 2021 08:49:32 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.9.0 MIME-Version: 1.0 In-Reply-To: <20210129151143.10014-1-f.ebner@proxmox.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.008 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment NICE_REPLY_A -0.001 Looks like a legit reply (A) RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [PATCH-SERIES v2 qemu-server] Cleanup migration code and improve migration disk cleanup X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Apr 2021 06:49:44 -0000 Ping for this series. Patch #13 conflicts with Mira's proposed change for copying the conntrack information[0]. Patch #1 and Patch #5 are the only ones *not* conflicting with Fabian's remote migration RFC[1]. [0]: https://lists.proxmox.com/pipermail/pve-devel/2021-February/046873.html [1]: https://lists.proxmox.com/pipermail/pve-devel/2021-April/047600.html Am 29.01.21 um 16:11 schrieb Fabian Ebner: > This series intends to make the migration code more readable by > simplyfing/unifying how we keep track of local volumes and splitting up > sync_disks into multiple subroutines. > > This is done by keeping more information within the hash of local_volumes we > obtain in the very beginning and re-using it later. Also a method to filter by > migration/replication type is introduced, making it possible to get rid of some > special-case handling when iterating over local volumes. > > Still more should be possible, but the series is getting big, so I stopped here. > Namely, $self->{local_volumes}, $self->{volume_map} and $self->{target_drive} > still have some overlap and it might be possible to merge them somehow. > > Also it might make sense to put abstract more things to the common code, one > example is to make target storage mapping available for containers too. > > > Changes from v1: > * dropped already applied patches > * rebased > * prefix commit messages with 'migration' (and 'test') > * increased test coverage a little, see patch #1 and patch #13 > * reordered patches #8+#9 to avoid temporary breakage, see > patches #8 and patch #13 (added test-case) for the details > * add patch to fix bandwidth limit calculation > * add patch to split out run_replication from scan_local_volumes > * add patch to improve error handling when block job finishing fails > > > Fabian Ebner (13): > test: migration: add parse_volume_id calls > migration: split sync_disks into two functions > migration: avoid re-scanning all volumes > migration: split out config_update_local_disksizes from > scan_local_volumes > migration: fix calculation of bandwith limit for non-disk migration > migration: save targetstorage and bwlimit in local_volumes hash and > re-use information > migration: add nbd migrated volumes to volume_map earlier > migration: simplify removal of local volumes and get rid of > self->{volumes} > migration: cleanup_remotedisks: simplify and include more disks > migration: use storage_migration for checks instead of > online_local_volumes > migration: keep track of replicated volumes via local_volumes > migration: split out replication from scan_local_volumes > migration: move finishing block jobs to phase2 for better/uniform > error handling > > PVE/QemuMigrate.pm | 401 ++++++++++++++------------ > PVE/QemuServer.pm | 2 + > test/MigrationTest/QemuMigrateMock.pm | 9 + > test/MigrationTest/QmMock.pm | 2 + > test/MigrationTest/Shared.pm | 4 + > test/run_qemu_migrate_tests.pl | 40 ++- > 6 files changed, 267 insertions(+), 191 deletions(-) >