From mboxrd@z Thu Jan 1 00:00:00 1970
Return-Path: <f.schauer@proxmox.com>
Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by lists.proxmox.com (Postfix) with ESMTPS id B908FBA420
for <pve-devel@lists.proxmox.com>; Wed, 13 Dec 2023 18:31:15 +0100 (CET)
Received: from firstgate.proxmox.com (localhost [127.0.0.1])
by firstgate.proxmox.com (Proxmox) with ESMTP id 97A85EA80
for <pve-devel@lists.proxmox.com>; Wed, 13 Dec 2023 18:31:15 +0100 (CET)
Received: from proxmox-new.maurer-it.com (proxmox-new.maurer-it.com
[94.136.29.106])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits))
(No client certificate requested)
by firstgate.proxmox.com (Proxmox) with ESMTPS
for <pve-devel@lists.proxmox.com>; Wed, 13 Dec 2023 18:31:14 +0100 (CET)
Received: from proxmox-new.maurer-it.com (localhost.localdomain [127.0.0.1])
by proxmox-new.maurer-it.com (Proxmox) with ESMTP id 58FE8471B6
for <pve-devel@lists.proxmox.com>; Wed, 13 Dec 2023 18:31:14 +0100 (CET)
Message-ID: <7d2ebfc9-7172-410a-be4c-1fc681d07e43@proxmox.com>
Date: Wed, 13 Dec 2023 18:31:13 +0100
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Content-Language: en-US
To: Fiona Ebner <f.ebner@proxmox.com>,
Proxmox VE development discussion <pve-devel@lists.proxmox.com>
References: <20231211141256.27565-1-f.schauer@proxmox.com>
<474c3df6-a8ed-4294-8e46-37c3f2008689@proxmox.com>
<544700e6-14ca-4db8-88b4-117202a972d9@proxmox.com>
<93b6aa58-fea8-48ba-b007-a08fd02c734a@proxmox.com>
From: Filip Schauer <f.schauer@proxmox.com>
In-Reply-To: <93b6aa58-fea8-48ba-b007-a08fd02c734a@proxmox.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-SPAM-LEVEL: Spam detection results: 0
AWL -0.166 Adjusted score from AWL reputation of From: address
BAYES_00 -1.9 Bayes spam probability is 0 to 1%
DMARC_MISSING 0.1 Missing DMARC policy
KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment
SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record
SPF_PASS -0.001 SPF: sender matches SPF record
T_SCC_BODY_TEXT_LINE -0.01 -
Subject: Re: [pve-devel] [PATCH qemu-server] Properly identify the CPU
architecture of 32-bit VMs
X-BeenThere: pve-devel@lists.proxmox.com
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Proxmox VE development discussion <pve-devel.lists.proxmox.com>
List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=unsubscribe>
List-Archive: <http://lists.proxmox.com/pipermail/pve-devel/>
List-Post: <mailto:pve-devel@lists.proxmox.com>
List-Help: <mailto:pve-devel-request@lists.proxmox.com?subject=help>
List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel>,
<mailto:pve-devel-request@lists.proxmox.com?subject=subscribe>
X-List-Received-Date: Wed, 13 Dec 2023 17:31:15 -0000
On 12/12/2023 12:48, Fiona Ebner wrote:
> Am 12.12.23 um 11:39 schrieb Filip Schauer:
>> It's actually not a different binary. qemu-system-i386 is a symlink that
>> points to qemu-system-x86_64. But still this does indeed break migration
>> between a node that has this patch applied and another node without the
>> patch.
>>
> Oh, okay. But then that's a bit surprising. From a quick glance, we do
> have some logic matching arch 'x86_64' specifically in CPUConfig.pm, so
> that might be it. E.g.:
>
>> my $pve_forced_flags = {};
>> $pve_forced_flags->{'enforce'} = {
>> reason => "error if requested CPU settings not available",
>> } if $cputype ne 'host' && $kvm && $arch eq 'x86_64';
This check does not make any difference in my case since $kvm is not set
when using a qemu32 CPU.
Not sure what causes this to break migration, not that it matters but
here are my results anyway.
Journal of an unpatched node when migrating a running VM with CPU type
qemu32 from a node with this patch v1 applied:
Dec 13 18:09:28 pve2 QEMU[124616]: KVM: entry failed, hardware error
0x80000021
Dec 13 18:09:28 pve2 QEMU[124616]: If you're running a guest on an Intel
machine without unrestricted mode
Dec 13 18:09:28 pve2 QEMU[124616]: support, the failure can be most
likely due to the guest entering an invalid
Dec 13 18:09:28 pve2 QEMU[124616]: state for Intel VT. For example, the
guest maybe running in big real mode
Dec 13 18:09:28 pve2 QEMU[124616]: which is not supported on less recent
Intel processors.
Dec 13 18:09:28 pve2 QEMU[124616]: EAX=00003433 EBX=0006880c
ECX=00002fa5 EDX=89817860
Dec 13 18:09:28 pve2 QEMU[124616]: ESI=898098c0 EDI=8980f758
EBP=00183aec ESP=00183ad0
Dec 13 18:09:28 pve2 QEMU[124616]: EIP=00292afa EFL=00200006 [-----P-]
CPL=0 II=0 A20=1 SMM=0 HLT=0
Dec 13 18:09:28 pve2 QEMU[124616]: ES =0030 00000000 ffffffff 00c09300
DPL=0 DS [-WA]
Dec 13 18:09:28 pve2 QEMU[124616]: CS =0020 00000000 ffffffff 00c09b00
DPL=0 CS32 [-RA]
Dec 13 18:09:28 pve2 QEMU[124616]: SS =0030 00000000 ffffffff 00c09300
DPL=0 DS [-WA]
Dec 13 18:09:28 pve2 QEMU[124616]: DS =0030 00000000 ffffffff 00c09300
DPL=0 DS [-WA]
Dec 13 18:09:28 pve2 QEMU[124616]: FS =0060 00023de0 0000ffff 00009300
DPL=0 DS16 [-WA]
Dec 13 18:09:28 pve2 QEMU[124616]: GS =0060 00023de0 0000ffff 00009300
DPL=0 DS16 [-WA]
Dec 13 18:09:28 pve2 QEMU[124616]: LDT=0000 00000000 00000000 00008200
DPL=0 LDT
Dec 13 18:09:28 pve2 QEMU[124616]: TR =0040 00025260 00000077 00008900
DPL=0 TSS32-avl
Dec 13 18:09:28 pve2 QEMU[124616]: GDT= 00184000 0000007f
Dec 13 18:09:28 pve2 QEMU[124616]: IDT= 00184080 000007ff
Dec 13 18:09:28 pve2 QEMU[124616]: CR0=80000011 CR2=00000000
CR3=00185000 CR4=00000000
Dec 13 18:09:28 pve2 kernel: set kvm_intel.dump_invalid_vmcs=1 to dump
internal KVM state.
Dec 13 18:09:28 pve2 QEMU[124616]: DR0=0000000000000000
DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000
Dec 13 18:09:28 pve2 QEMU[124616]: DR6=00000000ffff0ff0 DR7=0000000000000400
Dec 13 18:09:28 pve2 QEMU[124616]: EFER=0000000000000000
Dec 13 18:09:28 pve2 QEMU[124616]: Code=0c 89 08 89 7a 0c 8b 45 0c 83 45
10 02 8b c8 2b cf 23 4a 08 <8b> 3a 8a 1c 0f 88 1c 07 40 41 ff 4d 10 83
7d 10 00 7f ed 3b 45 f4 8b 7d fc 89 45 0c 0f 8c
Journal of the patched node when migrating the same VM from an unpatched
node:
Dec 13 18:12:17 pve1 QEMU[125150]: qemu-system-i386: Unknown savevm
section or instance 'kvmclock' 0. Make sure that your current VM setup
matches your saved VM setup, including any hotplugged devices
Dec 13 18:12:17 pve1 QEMU[125150]: qemu-system-i386: load of migration
failed: Invalid argument