From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 1128674924 for ; Mon, 19 Apr 2021 13:50:38 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 071DA1576E for ; Mon, 19 Apr 2021 13:50:38 +0200 (CEST) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [212.186.127.180]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS id 6032515761 for ; Mon, 19 Apr 2021 13:50:34 +0200 (CEST) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 2750545A93 for ; Mon, 19 Apr 2021 13:50:34 +0200 (CEST) Message-ID: <33f70f3b-ee0d-5ded-6061-b9301d2ab3bd@proxmox.com> Date: Mon, 19 Apr 2021 13:50:31 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:88.0) Gecko/20100101 Thunderbird/88.0 Content-Language: en-US To: Proxmox VE development discussion , Fabian Ebner References: <20210129151143.10014-1-f.ebner@proxmox.com> From: Thomas Lamprecht In-Reply-To: <20210129151143.10014-1-f.ebner@proxmox.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.040 Adjusted score from AWL reputation of From: address KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: [pve-devel] applied-series: [PATCH-SERIES v2 qemu-server] Cleanup migration code and improve migration disk cleanup X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Apr 2021 11:50:38 -0000 On 29.01.21 16:11, Fabian Ebner wrote: > This series intends to make the migration code more readable by > simplyfing/unifying how we keep track of local volumes and splitting up > sync_disks into multiple subroutines. > > This is done by keeping more information within the hash of local_volumes we > obtain in the very beginning and re-using it later. Also a method to filter by > migration/replication type is introduced, making it possible to get rid of some > special-case handling when iterating over local volumes. > > Still more should be possible, but the series is getting big, so I stopped here. > Namely, $self->{local_volumes}, $self->{volume_map} and $self->{target_drive} > still have some overlap and it might be possible to merge them somehow. > > Also it might make sense to put abstract more things to the common code, one > example is to make target storage mapping available for containers too. > > > Changes from v1: > * dropped already applied patches > * rebased > * prefix commit messages with 'migration' (and 'test') > * increased test coverage a little, see patch #1 and patch #13 > * reordered patches #8+#9 to avoid temporary breakage, see > patches #8 and patch #13 (added test-case) for the details > * add patch to fix bandwidth limit calculation > * add patch to split out run_replication from scan_local_volumes > * add patch to improve error handling when block job finishing fails > > > Fabian Ebner (13): > test: migration: add parse_volume_id calls > migration: split sync_disks into two functions > migration: avoid re-scanning all volumes > migration: split out config_update_local_disksizes from > scan_local_volumes > migration: fix calculation of bandwith limit for non-disk migration > migration: save targetstorage and bwlimit in local_volumes hash and > re-use information > migration: add nbd migrated volumes to volume_map earlier > migration: simplify removal of local volumes and get rid of > self->{volumes} > migration: cleanup_remotedisks: simplify and include more disks > migration: use storage_migration for checks instead of > online_local_volumes > migration: keep track of replicated volumes via local_volumes > migration: split out replication from scan_local_volumes > migration: move finishing block jobs to phase2 for better/uniform > error handling > > PVE/QemuMigrate.pm | 401 ++++++++++++++------------ > PVE/QemuServer.pm | 2 + > test/MigrationTest/QemuMigrateMock.pm | 9 + > test/MigrationTest/QmMock.pm | 2 + > test/MigrationTest/Shared.pm | 4 + > test/run_qemu_migrate_tests.pl | 40 ++- > 6 files changed, 267 insertions(+), 191 deletions(-) > applied series, thanks!