From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 699BA1FF09B for ; Mon, 31 Aug 2026 23:10:52 +0200 (CEST) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 74704215A3; Mon, 31 Aug 2026 23:10:25 +0200 (CEST) From: Marco Gaiarin Subject: Two nodes asymmetrical cluster, backup, stall of one node. Date: Mon, 31 Aug 2026 10:48:30 +0200 Organization: Il gaio usa sempre TIN per le liste, fallo anche tu!!! Message-ID: X-Trace: eraldo.lilliput.linux.it 1788208988 4000609 192.168.1.45 (31 Aug 2026 20:43:08 GMT) X-Mailer: tin/2.6.4-20240224 ("Banff") (Linux/7.0.0-30-generic (x86_64)) X-Gateway-System: SmartGate 1.4.5 To: pve-user@lists.proxmox.com X-SPAM-LEVEL: Spam detection results: 1 AWL 0.075 Adjusted score from AWL reputation of From: address DATE_IN_PAST_12_24 0.804 Date: is 12 to 24 hours before Received: date DMARC_PASS -0.1 DMARC pass policy JMQ_SPF_NEUTRAL 0.5 SPF set to ?all KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: FVVGRW73XWV2SEQ2U27BVYBYUSL67WV6 X-Message-ID-Hash: FVVGRW73XWV2SEQ2U27BVYBYUSL67WV6 X-MailFrom: gaio@lilliput.linux.it X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE user list List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: I've hit a trouble i cannot understand very well; we have some asymmetrical clusters, composed of one main server and a 'backup', a little underpowered, server that act as backup, exporting their fs via NFS. At every backup run (weekly), the backup node got some pressure (this is expected and common to other clusters), kernel start to complain as: 2026-08-30T00:18:43.125643+02:00 pdpve2 pvescheduler[1474088]: jobs: cfs-lock 'file-jobs_cfg' error: got lock request timeout 2026-08-30T00:18:43.723075+02:00 pdpve2 pvescheduler[1474087]: replication: cfs-lock 'file-replication_cfg' error: got lock request timeout [...] 2026-08-30T00:49:59.269199+02:00 pdpve2 kernel: [481202.580590] INFO: task pvescheduler:1478618 blocked for more than 122 seconds. after that, backup node get 'offline' (from main node), pveproxy respond in backup node but i canot login (local PAM account or domain ones). If i SSH on the backup node, all seems normal and working, no service stopped, a little test VM running on it working as expected. Only a note, there what seems a stalled backup job: root@pdpve2:~# ps aux | grep pves[r] root 1497581 0.0 0.6 308788 110928 ? Ds Aug30 0:00 /usr/bin/perl -T /usr/bin/pvesr prepare-local-job 100-0 --scan local-zfs local-zfs:vm-100-disk-0 --last_sync 1788035400 i've trid to kill it, even with '-9', but whitout success. If i reboot the node, all come back as normal. Someone have some clue? If i have a set ov VM on node A and a set of VM on node B, and i run a backup of all VMs, there's some way to 'serialize' it, eg, do backup of VMs of node A and after that backup of VM on node B? I suppose that part of the trouble came fron the fact that backup start in parrallels on both nodes, but node B is also the destination of backup, so it get 3 time the IO (write from node A, read from node B, write to node B). I can surely setup TWO backup job, but... thanks. --