From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id B908FBA420 for ; Wed, 13 Dec 2023 18:31:15 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 97A85EA80 for ; Wed, 13 Dec 2023 18:31:15 +0100 (CET) Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com [94.136.29.106]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 13 Dec 2023 18:31:14 +0100 (CET) Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1]) by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 58FE8471B6 for ; Wed, 13 Dec 2023 18:31:14 +0100 (CET) Message-ID: <7d2ebfc9-7172-410a-be4c-1fc681d07e43@proxmox.com> Date: Wed, 13 Dec 2023 18:31:13 +0100 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Content-Language: en-US To: Fiona Ebner , Proxmox VE development discussion References: <20231211141256.27565-1-f.schauer@proxmox.com> <474c3df6-a8ed-4294-8e46-37c3f2008689@proxmox.com> <544700e6-14ca-4db8-88b4-117202a972d9@proxmox.com> <93b6aa58-fea8-48ba-b007-a08fd02c734a@proxmox.com> From: Filip Schauer In-Reply-To: <93b6aa58-fea8-48ba-b007-a08fd02c734a@proxmox.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL -0.166 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record T_SCC_BODY_TEXT_LINE -0.01 - Subject: Re: [pve-devel] [PATCH qemu-server] Properly identify the CPU architecture of 32-bit VMs X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 13 Dec 2023 17:31:15 -0000 On 12/12/2023 12:48, Fiona Ebner wrote: > Am 12.12.23 um 11:39 schrieb Filip Schauer: >> It's actually not a different binary. qemu-system-i386 is a symlink that >> points to qemu-system-x86_64. But still this does indeed break migration >> between a node that has this patch applied and another node without the >> patch. >> > Oh, okay. But then that's a bit surprising. From a quick glance, we do > have some logic matching arch 'x86_64' specifically in CPUConfig.pm, so > that might be it. E.g.: > >> my $pve_forced_flags = {}; >> $pve_forced_flags->{'enforce'} = { >> reason => "error if requested CPU settings not available", >> } if $cputype ne 'host' && $kvm && $arch eq 'x86_64'; This check does not make any difference in my case since $kvm is not set when using a qemu32 CPU. Not sure what causes this to break migration, not that it matters but here are my results anyway. Journal of an unpatched node when migrating a running VM with CPU type qemu32 from a node with this patch v1 applied: Dec 13 18:09:28 pve2 QEMU[124616]: KVM: entry failed, hardware error 0x80000021 Dec 13 18:09:28 pve2 QEMU[124616]: If you're running a guest on an Intel machine without unrestricted mode Dec 13 18:09:28 pve2 QEMU[124616]: support, the failure can be most likely due to the guest entering an invalid Dec 13 18:09:28 pve2 QEMU[124616]: state for Intel VT. For example, the guest maybe running in big real mode Dec 13 18:09:28 pve2 QEMU[124616]: which is not supported on less recent Intel processors. Dec 13 18:09:28 pve2 QEMU[124616]: EAX=00003433 EBX=0006880c ECX=00002fa5 EDX=89817860 Dec 13 18:09:28 pve2 QEMU[124616]: ESI=898098c0 EDI=8980f758 EBP=00183aec ESP=00183ad0 Dec 13 18:09:28 pve2 QEMU[124616]: EIP=00292afa EFL=00200006 [-----P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 Dec 13 18:09:28 pve2 QEMU[124616]: ES =0030 00000000 ffffffff 00c09300 DPL=0 DS   [-WA] Dec 13 18:09:28 pve2 QEMU[124616]: CS =0020 00000000 ffffffff 00c09b00 DPL=0 CS32 [-RA] Dec 13 18:09:28 pve2 QEMU[124616]: SS =0030 00000000 ffffffff 00c09300 DPL=0 DS   [-WA] Dec 13 18:09:28 pve2 QEMU[124616]: DS =0030 00000000 ffffffff 00c09300 DPL=0 DS   [-WA] Dec 13 18:09:28 pve2 QEMU[124616]: FS =0060 00023de0 0000ffff 00009300 DPL=0 DS16 [-WA] Dec 13 18:09:28 pve2 QEMU[124616]: GS =0060 00023de0 0000ffff 00009300 DPL=0 DS16 [-WA] Dec 13 18:09:28 pve2 QEMU[124616]: LDT=0000 00000000 00000000 00008200 DPL=0 LDT Dec 13 18:09:28 pve2 QEMU[124616]: TR =0040 00025260 00000077 00008900 DPL=0 TSS32-avl Dec 13 18:09:28 pve2 QEMU[124616]: GDT=     00184000 0000007f Dec 13 18:09:28 pve2 QEMU[124616]: IDT=     00184080 000007ff Dec 13 18:09:28 pve2 QEMU[124616]: CR0=80000011 CR2=00000000 CR3=00185000 CR4=00000000 Dec 13 18:09:28 pve2 kernel: set kvm_intel.dump_invalid_vmcs=1 to dump internal KVM state. Dec 13 18:09:28 pve2 QEMU[124616]: DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 Dec 13 18:09:28 pve2 QEMU[124616]: DR6=00000000ffff0ff0 DR7=0000000000000400 Dec 13 18:09:28 pve2 QEMU[124616]: EFER=0000000000000000 Dec 13 18:09:28 pve2 QEMU[124616]: Code=0c 89 08 89 7a 0c 8b 45 0c 83 45 10 02 8b c8 2b cf 23 4a 08 <8b> 3a 8a 1c 0f 88 1c 07 40 41 ff 4d 10 83 7d 10 00 7f ed 3b 45 f4 8b 7d fc 89 45 0c 0f 8c Journal of the patched node when migrating the same VM from an unpatched node: Dec 13 18:12:17 pve1 QEMU[125150]: qemu-system-i386: Unknown savevm section or instance 'kvmclock' 0. Make sure that your current VM setup matches your saved VM setup, including any hotplugged devices Dec 13 18:12:17 pve1 QEMU[125150]: qemu-system-i386: load of migration failed: Invalid argument