From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 148E4952B7 for ; Wed, 18 Jan 2023 01:32:25 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id EDCD919269 for ; Wed, 18 Jan 2023 01:32:24 +0100 (CET) Received: from morty.keekles.org (Morty.keekles.org [199.47.174.151]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for ; Wed, 18 Jan 2023 01:32:22 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by morty.keekles.org (Postfix) with ESMTP id 3A6FC19E0C66 for ; Wed, 18 Jan 2023 00:32:20 +0000 (UTC) Received: from morty.keekles.org ([127.0.0.1]) by localhost (morty.keekles.org [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id IeHXY1ZxXglG for ; Wed, 18 Jan 2023 00:32:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by morty.keekles.org (Postfix) with ESMTP id CE88C19E1BAB for ; Wed, 18 Jan 2023 00:32:15 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 morty.keekles.org CE88C19E1BAB DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=bryanfields.net; s=909DCF92-EFE7-11EB-9235-648EB8AF1B81; t=1674001935; bh=xA8gzddQjSfJTnajBISfWvPx8LiNgmD7+pJzThXfxWI=; h=Message-ID:Date:MIME-Version:To:From; b=HAfXqqhv6c6IoIcUpmP4d2fEHFk48mqIVBucpFr/OdRCmv4jXFOPHOxUrMktY/NE0 wSqmPAycLXvQuZTolppaurMdbB3YD7VXZlG/ldE6NfqRhdL0fj/FwLOX2eXcnACKNo G74Z3WMRGVMET8LOxbyAgoIeRDslUNWbE5ynTCnKl1eIYFR0U1pjs9yi0Kq8aVdCk9 C7Xila6cXQe9teCAJjPfmZa0wZ6VhRem21u1PuIuIhhIljYGvOS3oS/F1NMbZ9QEko ir5IhEtQEWBBis5bfd0JqmqTmou+UXAHgvA/4FV0ZQVDw4wjB34csO5iOm/KEFjOxB tijX8h2nzqI8Q== X-Virus-Scanned: amavisd-new at morty.keekles.org Received: from morty.keekles.org ([127.0.0.1]) by localhost (morty.keekles.org [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id NmtKLyjYjLm3 for ; Wed, 18 Jan 2023 00:32:15 +0000 (UTC) Received: from [192.168.128.105] (static-47-206-239-202.tamp.fl.frontiernet.net [47.206.239.202]) by morty.keekles.org (Postfix) with ESMTPSA id A885419E0C66 for ; Wed, 18 Jan 2023 00:32:15 +0000 (UTC) Message-ID: <525f32af-d46a-1299-9b55-fdd9c6d7f429@bryanfields.net> Date: Tue, 17 Jan 2023 19:32:15 -0500 MIME-Version: 1.0 User-Agent: Mutt/1.12.0 (2019-05-25) Content-Language: en-US To: pve-user@lists.proxmox.com References: <2635f65d-33fb-5447-a3c1-d5cbab9e04e1@bryanfields.net> From: Bryan Fields In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: quoted-printable X-SPAM-LEVEL: Spam detection results: 0 BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [PVE-User] Debian 11 hard lock issues as VM X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 18 Jan 2023 00:32:25 -0000 On 1/17/23 3:22 AM, Eneko Lacunza via pve-user wrote: > Hi Bryan, >=20 > We started to upgrade our cluster from PVE 7.2 to 7.3 yesterday. >=20 > I have enabled the agent in our only VM with Debian 11 running on a > 7.3-4 node at the moment, and performed 5 full backups in a row, VM > continues working (no hang). This is replication, but I believe it's the same. > You haven't provided details about your setup: >=20 > - Server (especially CPU model). Debian could be suffering from weird > BIOS clock issues. The Hosts are HP DL360's Generation 7. ZFS Raid2 local storage using 1.6= TB=20 SAS SSD's. The life used indicator is now 6% or 7% on most disks. There is 192 GB of ram in each server 16384=C2=A0MB=C2=A01600=C2=A0MHz EC= C ram. There are dual 3.07 GHz 6 core (12 thread) CPU's. /proc/cpuinfo is below= . processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 44 model name : Intel(R) Xeon(R) CPU X5675 @ 3.07GHz stepping : 2 microcode : 0x1a cpu MHz : 1910.971 cache size : 12288 KB physical id : 0 siblings : 12 core id : 0 cpu cores : 6 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 11 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pa= t=20 pse36 clflush dts acpi mmx fxsr sse sse2 ht tm pbe syscall nx pdpe1gb rdt= scp=20 lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc= =20 cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse= 3=20 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 popcnt aes lahf_lm epb pti tpr_shad= ow=20 vnmi flexpriority ept vpid dtherm ida arat vmx flags : vnmi preemption_timer invvpid ept_x_only ept_1gb flexpriority= =20 tsc_offset vtpr mtf vapic ept vpid unrestricted_guest ple bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swa= pgs=20 itlb_multihit mmio_unknown bogomips : 6134.18 clflush size : 64 cache_alignment : 64 address sizes : 40 bits physical, 48 bits virtual power management: the proxmox config for the VM is here: agent: 1,fstrim_cloned_disks=3D1 bootdisk: scsi0 cores: 2 cpuunits: 2048 ide2: none,media=3Dcdrom memory: 8192 name: eyes.tampacoop.net net0: virtio=3D86:49:26:AA:86:E7,bridge=3Dvmbr199,firewall=3D1 net1: virtio=3DA2:C5:47:85:3E:3B,bridge=3Dvmbr8 numa: 0 onboot: 1 ostype: l26 parent: before_extend scsi0: local-zfs:vm-102-disk-0,discard=3Don,format=3Draw,iothread=3D1,siz= e=3D48G,ssd=3D1 scsihw: virtio-scsi-single smbios1: uuid=3D11ed5a86-3395-49f2-ac80-16804b237a0d sockets: 1 startup: order=3D1 vmgenid: 6238f0f2-ac90-43e0-b56c-05e1ed1c2431 > - Running kernel on PVE 7.3-4 . Kernel 5.15.x has been quite bad for us= , > have you tried kernel 5.13 or 5.19? I reverted to 4.9.0-19-amd64 #1 SMP Debian 4.9.320-2 (2022-06-30) x86_64=20 GNU/Linux Kernel on the guest OS and it's not locked up once now. This i= s=20 running either the 5.2.0 or 7.2.0 agent. I've moved the VM's across hosts and they have the same problem. FingerlessGloves mentioned there was the possibility of this being a mari= adb=20 issue and I can confirm we have the official Maria DB packages installed = on=20 this server. 10.10.2-MariaDB-1:10.10.2+maria~deb11 is what we're running= on=20 the server. Could this be some interaction of new kernel and new maria db? --=20 Bryan Fields 727-409-1194 - Voice http://bryanfields.net