From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by lists.proxmox.com (Postfix) with UTF8SMTPS id 81264708AD for ; Thu, 24 Jun 2021 16:59:36 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with UTF8SMTP id 78002EE30 for ; Thu, 24 Jun 2021 16:59:06 +0200 (CEST) Received: from vizir.gilouweb.com (vizir.gilouweb.com [IPv6:2001:41d0:d:2147::100]) by firstgate.proxmox.com (Proxmox) with UTF8SMTP id 770D9EE25 for ; Thu, 24 Jun 2021 16:59:04 +0200 (CEST) Received: from localhost (localhost.localdomain [127.0.0.1]) by vizir.gilouweb.com (Postfix) with UTF8SMTP id 61ACB82130B for ; Thu, 24 Jun 2021 16:58:57 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gilouweb.com; s=mail; t=1624546737; bh=7vYjivwEsxM/BMe8SqtdiPAHovpZbXaqfFeb6TqA86A=; h=Date:Subject:To:References:From:In-Reply-To:From; b=jqYyC4pWSHTyAZ2ir9a/Qk84JV9ZiDrvaTEcZJvMeYaqBAYAi5i6bJZVV0RkeqFOw CS44wcPui+Xv0oBQC/wF4xCtxpUSXdx72c41sSLGbGEhco9/C11h+OY7nAcGbAkkiX XTsQql6c3hOgXOOnJCGbSGV9WYul0XBw407+4EupTuXAXrxRxL3smBpVuVPK8XzvNM sgJIkRaQTsuXoeGMH1JXU7Jo+PGvpQpC5/Dge6mL//NyenJ8PNn6frrRW5EZAMJe5B 3hYXvh3o1yoRMu6GmLryZ5mgO0XsmptzSVOxazk5LvtWulPU0QAruVyT8JTTNCXfsW lPpG45sFKu5MA== X-Virus-Scanned: Debian amavisd-new at vizir.gilouweb.com Received: from vizir.gilouweb.com ([127.0.0.1]) by localhost (vizir.gilouweb.com [127.0.0.1]) (amavisd-new, port 10024) with UTF8SMTP id hPYYGvBRTqu4 for ; Thu, 24 Jun 2021 16:58:47 +0200 (CEST) Received: from [IPV6:2a00:5881:1090:3201:782f:60aa:2ed6:54e0] (unknown [IPv6:2a00:5881:1090:3201:782f:60aa:2ed6:54e0]) by vizir.gilouweb.com (Postfix) with UTF8SMTPSA id 697A7821309 for ; Thu, 24 Jun 2021 16:58:47 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=gilouweb.com; s=mail; t=1624546727; bh=7vYjivwEsxM/BMe8SqtdiPAHovpZbXaqfFeb6TqA86A=; h=Date:Subject:To:References:From:In-Reply-To:From; b=X5o7EEJIiSMDVhGduhkWRuPJRD1ynDxjczmiDVamep0U4TrvNf6ZuN2oQz+li27+t L2lu57uskYgCho9G6s4Q/pAw1g6MFJIRPK2TeG+n98qC9nPsXIXCKjoFYpqCBaM6D0 4wLO/2Fs1RLij1fFPv9BWffjlbVKgt35kFSNcEOLYe2LdY0wlWeBHZrQIreQ/vs9DQ wF7M0ymhtr9zYlkSbAD1luUGBo/FMOCquFWqpRFzYwqoEHme1bgKCkY6GrQTc5M+ZB OdyaIe7Ro+GmtWorb96sDxn+4AStZPAqTh8gnCRKLzKycn4PQbRZ1yBfUru0tR4iHO BpYK/78jXyQRg== Message-ID: <08400e55-6f85-ae8b-6363-8279ae2be279@gilouweb.com> Date: Thu, 24 Jun 2021 16:58:46 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:90.0) Gecko/20100101 Thunderbird/90.0 Content-Language: fr To: pve-user@lists.proxmox.com References: <38cc69ca-b0e9-6ee1-afb6-86a19c0db098@gilouweb.com> From: Gilles Pietri In-Reply-To: <38cc69ca-b0e9-6ee1-afb6-86a19c0db098@gilouweb.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-SPAM-LEVEL: Spam detection results: 0 AWL 0.000 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain NICE_REPLY_A -0.001 Looks like a legit reply (A) SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [PVE-User] Nested KVM on AMD EPYC processor, oops X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jun 2021 14:59:36 -0000 Le 15/06/2021 à 21:34, Gilles Pietri a écrit : > Hi, > > I'm running qemu (through openstack) on a proxmox instance running: > # pveversion -v > proxmox-ve: 6.4-1 (running kernel: 5.4.119-1-pve) > pve-manager: 6.4-8 (running version: 6.4-8/185e14db) > pve-kernel-5.4: 6.4-3 > pve-kernel-helper: 6.4-3 > [...] > pve-qemu-kvm: 5.2.0-6 > qemu-server: 6.4-2 Hi, I switched to pve-kernel 5.11 (7.0-2~bpo10), as suggested on #qemu, and nested virt works nicely! Regards, Gilles > > > The qemu version in Openstack (Wallaby) is > $ qemu-system-x86_64 -version > QEMU emulator version 4.2.1 (Debian 1:4.2-3ubuntu6.16) > Copyright (c) 2003-2019 Fabrice Bellard and the QEMU Project developers > > The VM is running using the host CPU, which is AMD EPYC 7451 24-Core > with relevant parameters here: > cores: 8 > cpu: host > memory: 32768 > numa: 0 > rng0: source=/dev/urandom > sockets: 1 > > > And I get spammed a lot with that kind of traces: > > [Tue Jun 15 19:13:29 2021] ------------[ cut here ]------------ > [Tue Jun 15 19:13:29 2021] WARNING: CPU: 6 PID: 47530 at > arch/x86/kvm/mmu.c:2250 nonpaging_update_pte+0x9/0x10 [kvm] > [Tue Jun 15 19:13:30 2021] Modules linked in: xt_nat > nf_conntrack_netlink tcp_diag inet_diag xt_MASQUERADE xfrm_user > iptable_nat nf_nat overlay binfmt_misc rpcsec_gss_krb5 auth_rpcgss nfsv4 > nfs lockd grace fscache sctp veth ebt_arp ebtable_filter ebtables > ip6table_raw ip6t_REJECT nf_reject_ipv6 ip6table_filter ip6_tables > iptable_raw xt_mac ipt_REJECT nf_reject_ipv4 xt_mark xt_set xt_physdev > xt_addrtype xt_comment xt_multiport xt_conntrack nf_conntrack > nf_defrag_ipv6 nf_defrag_ipv4 xt_tcpudp ip_set_hash_net ip_set > iptable_filter bpfilter softdog nfnetlink_log nfnetlink ipmi_ssif > amd64_edac_mod edac_mce_amd kvm_amd kvm drm_vram_helper irqbypass ttm > crct10dif_pclmul crc32_pclmul drm_kms_helper ghash_clmulni_intel drm > aesni_intel i2c_algo_bit crypto_simd fb_sys_fops cryptd syscopyarea > sysfillrect glue_helper k10temp sysimgblt ccp ipmi_si ipmi_devintf > ipmi_msghandler mac_hid zfs(PO) zunicode(PO) zzstd(O) zlua(O) zavl(PO) > icp(PO) zcommon(PO) znvpair(PO) spl(O) vhost_net vhost tap ib_iser > rdma_cm iw_cm ib_cm > [Tue Jun 15 19:13:30 2021] ib_core iscsi_tcp libiscsi_tcp libiscsi > scsi_transport_iscsi sunrpc ip_tables x_tables autofs4 dm_thin_pool > dm_persistent_data dm_bio_prison dm_bufio raid10 raid456 > async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq > libcrc32c raid0 multipath linear ixgbe ahci xhci_pci xfrm_algo raid1 > i2c_piix4 dca libahci xhci_hcd mdio > [Tue Jun 15 19:13:30 2021] CPU: 6 PID: 47530 Comm: kvm Tainted: P > W O 5.4.119-1-pve #1 > [Tue Jun 15 19:13:30 2021] Hardware name: empty empty/S8026GM2NRE-HOV-B, > BIOS V8.711 07/09/2020 > [Tue Jun 15 19:13:30 2021] RIP: 0010:nonpaging_update_pte+0x9/0x10 [kvm] > [Tue Jun 15 19:13:30 2021] Code: 00 0f 1f 44 00 00 55 31 c0 48 89 e5 5d > c3 0f 1f 00 0f 1f 44 00 00 55 48 89 e5 5d c3 0f 1f 44 00 00 0f 1f 44 00 > 00 55 48 89 e5 <0f> 0b 5d c3 0f 1f 00 0f 1f 44 00 00 31 f6 48 8b 04 77 > 48 63 54 37 > [Tue Jun 15 19:13:30 2021] RSP: 0018:ffffb904e4c7ba78 EFLAGS: 00010202 > [Tue Jun 15 19:13:30 2021] RAX: ffffffffc0dc0500 RBX: 0000000000000701 > RCX: ffffb904e4c7bac0 > [Tue Jun 15 19:13:30 2021] RDX: ffff909fa2bd0000 RSI: ffff908f2fe61e30 > RDI: ffff90a63593bca0 > [Tue Jun 15 19:13:30 2021] RBP: ffffb904e4c7ba78 R08: 000000000054a7ae > R09: ffff909fa2bd0000 > [Tue Jun 15 19:13:30 2021] R10: 0000000000000000 R11: 0000000000001970 > R12: ffff90a63593bca0 > [Tue Jun 15 19:13:30 2021] R13: 0000000000000000 R14: ffff909fa2bd0000 > R15: ffffb904e4c7bac8 > [Tue Jun 15 19:13:30 2021] FS: 00007f85da5fc700(0000) > GS:ffff9098ef800000(0000) knlGS:0000000000000000 > [Tue Jun 15 19:13:30 2021] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 > [Tue Jun 15 19:13:30 2021] CR2: 000000c420563010 CR3: 0000001fe1698000 > CR4: 00000000003406e0 > [Tue Jun 15 19:13:30 2021] Call Trace: > [Tue Jun 15 19:13:30 2021] kvm_mmu_pte_write+0x421/0x430 [kvm] > [Tue Jun 15 19:13:30 2021] kvm_page_track_write+0x82/0xc0 [kvm] > [Tue Jun 15 19:13:30 2021] emulator_write_phys+0x3b/0x50 [kvm] > [Tue Jun 15 19:13:30 2021] write_emulate+0xe/0x10 [kvm] > [Tue Jun 15 19:13:30 2021] emulator_read_write_onepage+0xfc/0x320 [kvm] > [Tue Jun 15 19:13:30 2021] emulator_read_write+0xd6/0x190 [kvm] > [Tue Jun 15 19:13:30 2021] emulator_write_emulated+0x15/0x20 [kvm] > [Tue Jun 15 19:13:30 2021] segmented_write+0x5d/0x80 [kvm] > [Tue Jun 15 19:13:30 2021] writeback+0x203/0x2e0 [kvm] > [Tue Jun 15 19:13:30 2021] x86_emulate_insn+0x990/0x1050 [kvm] > [Tue Jun 15 19:13:30 2021] x86_emulate_instruction+0x350/0x710 [kvm] > [Tue Jun 15 19:13:30 2021] complete_emulated_pio+0x3f/0x70 [kvm] > [Tue Jun 15 19:13:30 2021] kvm_arch_vcpu_ioctl_run+0x4cb/0x570 [kvm] > [Tue Jun 15 19:13:30 2021] kvm_vcpu_ioctl+0x24b/0x610 [kvm] > [Tue Jun 15 19:13:30 2021] do_vfs_ioctl+0xa9/0x640 > [Tue Jun 15 19:13:30 2021] ? task_numa_work+0x228/0x300 > [Tue Jun 15 19:13:30 2021] ksys_ioctl+0x67/0x90 > [Tue Jun 15 19:13:30 2021] __x64_sys_ioctl+0x1a/0x20 > [Tue Jun 15 19:13:30 2021] do_syscall_64+0x57/0x190 > [Tue Jun 15 19:13:30 2021] entry_SYSCALL_64_after_hwframe+0x44/0xa9 > [Tue Jun 15 19:13:30 2021] RIP: 0033:0x7f8df6dec427 > [Tue Jun 15 19:13:30 2021] Code: 00 00 90 48 8b 05 69 aa 0c 00 64 c7 00 > 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 > 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 aa 0c 00 f7 d8 > 64 89 01 48 > [Tue Jun 15 19:13:30 2021] RSP: 002b:00007f85da5f8c08 EFLAGS: 00000246 > ORIG_RAX: 0000000000000010 > [Tue Jun 15 19:13:30 2021] RAX: ffffffffffffffda RBX: 000000000000ae80 > RCX: 00007f8df6dec427 > [Tue Jun 15 19:13:30 2021] RDX: 0000000000000000 RSI: 000000000000ae80 > RDI: 0000000000000022 > [Tue Jun 15 19:13:30 2021] RBP: 0000000000000000 R08: 000055be0e527f58 > R09: 0000000000000000 > [Tue Jun 15 19:13:30 2021] R10: 0000000000000001 R11: 0000000000000246 > R12: 000055be0f426eb0 > [Tue Jun 15 19:13:30 2021] R13: 00007f8dea1f5000 R14: 0000000000000000 > R15: 000055be0f426eb0 > [Tue Jun 15 19:13:30 2021] ---[ end trace 4e3f65d27e26463c ]--- > > I'm guessing this is more of a qemu bug / issue, but it does have a lot > of impact on the performance of nested VMs, though it does not crash. I > was wondering if any of proxmox users noticed that with these CPUs / > versions before going upstream… > > Regards, > > Gilles > > _______________________________________________ > pve-user mailing list > pve-user@lists.proxmox.com > https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user > -- Gilles Pietri 29 avenue de la Libération, F-44400 Rezé 0634745199