From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: <premjith_r@amritatech.com> Received: from firstgate.proxmox.com (firstgate.proxmox.com [212.224.123.68]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by lists.proxmox.com (Postfix) with ESMTPS id 83B5F95217 for <pve-user@lists.proxmox.com>; Tue, 17 Jan 2023 16:09:53 +0100 (CET) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id 67250D9BE for <pve-user@lists.proxmox.com>; Tue, 17 Jan 2023 16:09:23 +0100 (CET) Received: from atmail.amritatech.com (atmail.amritatech.com [123.63.2.10]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by firstgate.proxmox.com (Proxmox) with ESMTPS for <pve-user@lists.proxmox.com>; Tue, 17 Jan 2023 16:09:22 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by atmail.amritatech.com (Postfix) with ESMTP id 7F7A4586975 for <pve-user@lists.proxmox.com>; Tue, 17 Jan 2023 20:30:41 +0530 (IST) Received: from atmail.amritatech.com ([127.0.0.1]) by localhost (atmail.amritatech.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id 64yVbGHmCZlb for <pve-user@lists.proxmox.com>; Tue, 17 Jan 2023 20:30:41 +0530 (IST) Received: from localhost (localhost [127.0.0.1]) by atmail.amritatech.com (Postfix) with ESMTP id 0021E7672BF for <pve-user@lists.proxmox.com>; Tue, 17 Jan 2023 20:30:36 +0530 (IST) DKIM-Filter: OpenDKIM Filter v2.10.3 atmail.amritatech.com 0021E7672BF DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amritatech.com; s=5989CEE6-26D0-11EC-8007-383290FA839D; t=1673967637; bh=mfNtv3xvrzlZQk1Jv3RdHjhR4lymMmTiH8fkevqJ+yI=; h=Date:From:To:Message-ID:MIME-Version; b=XCeSWRpdG+6Jk5ct3Nnt6+UnhL0clxDCSc87fF2W1UDAJJ++3PI7UTkIFuZuJI+RR ipIyh3f30xgceJmPp4+RdEQsnQ2dTHFTkorPCemzYkgfmJlmTEq0O4Gky1HLoMhOAr KE8gabbzLcDMN9I8iKU1LPa7V5Z6HCJruKvRgbsciuR9WKXfdTmH18k8sif+m9Mk+q Iogp8qCIfZaGd3+9ou0nP3i0OiKcZ8tfRgOzisbv4UUgkLiqfGiRGgBBbFRsY8komk 6I3PVaUmedT8PgmpnWSaB1x0D/1OFLMVngsP8aJ9hMkcJKieIHF2N8Ct2VL7U5kvNI FGA3UyZzuDOGg== X-Virus-Scanned: amavisd-new at amritatech.com Received: from atmail.amritatech.com ([127.0.0.1]) by localhost (atmail.amritatech.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id M7m9YCZOzCsK for <pve-user@lists.proxmox.com>; Tue, 17 Jan 2023 20:30:36 +0530 (IST) Received: from atmail.amritatech.com (atmail.amritatech.com [10.4.4.50]) by atmail.amritatech.com (Postfix) with ESMTP id 6D6D3586975 for <pve-user@lists.proxmox.com>; Tue, 17 Jan 2023 20:30:33 +0530 (IST) Date: Tue, 17 Jan 2023 20:30:33 +0530 (IST) From: Premjith R <premjith_r@amritatech.com> To: pve-user@lists.proxmox.com Message-ID: <1040227390.2707882.1673967633283.JavaMail.zimbra@amritatech.com> MIME-Version: 1.0 X-Originating-IP: [10.4.4.50] X-Mailer: Zimbra 8.8.15_GA_3888 (ZimbraWebClient - FF99 (Linux)/8.8.15_GA_3869) Thread-Index: xPmHgmLdmYhRiwDwy3N3K5HfMUznqQ== Thread-Topic: One cluster node get hang X-SPAM-LEVEL: Spam detection results: 0 AWL 1.475 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DKIM_SIGNED 0.1 Message has a DKIM or DK signature, not necessarily valid DKIM_VALID -0.1 Message has at least one valid DKIM or DK signature DKIM_VALID_AU -0.1 Message has a valid DKIM or DK signature from author's domain DKIM_VALID_EF -0.1 Message has a valid DKIM or DK signature from envelope-from domain HTML_MESSAGE 0.001 HTML included in message SPF_HELO_PASS -0.001 SPF: HELO matches SPF record SPF_PASS -0.001 SPF: sender matches SPF record URIBL_BLOCKED 0.001 ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [amritatech.com] Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.29 Subject: [PVE-User] One cluster node get hang X-BeenThere: pve-user@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE user list <pve-user.lists.proxmox.com> List-Unsubscribe: <https://lists.proxmox.com/cgi-bin/mailman/options/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=unsubscribe> List-Archive: <http://lists.proxmox.com/pipermail/pve-user/> List-Post: <mailto:pve-user@lists.proxmox.com> List-Help: <mailto:pve-user-request@lists.proxmox.com?subject=help> List-Subscribe: <https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-user>, <mailto:pve-user-request@lists.proxmox.com?subject=subscribe> X-List-Received-Date: Tue, 17 Jan 2023 15:09:53 -0000 Dear Team , Myself using Proxmox VE community edition for my testing environment. Using three node cluster for this setup, but today noticed one node is not accessible and getting following error. Jan 17 02:49:13 pve21atrs001 pmxcfs[1534595]: [status] notice: received log Jan 17 02:49:14 pve21atrs001 kernel: [8871754.354335] server[1534598]: segfault at 7f3aee6c79c5 ip 0000555e7ed1ec3a sp 00007f3a7bffe7f0 error 4 in pmxcfs[555e7ed05000+1b000] Jan 17 02:49:14 pve21atrs001 kernel: [8871754.354348] Code: 48 83 c4 20 45 85 e4 0f 84 93 00 00 00 41 8b 47 04 44 39 e0 73 0e 05 00 10 00 00 44 39 e0 0f 83 7c 00 00 00 44 89 e3 4c 01 fb <44> 8b 23 44 39 6c 24 0c 76 6c 48 85 ed 74 06 48 39 6b 18 75 c5 0f Jan 17 02:49:14 pve21atrs001 systemd[1]: pve-cluster.service: Main process exited, code=killed, status=11/SEGV Jan 17 02:49:14 pve21atrs001 systemd[1]: pve-cluster.service: Failed with result 'signal'. Jan 17 02:49:14 pve21atrs001 systemd[1]: pve-cluster.service: Consumed 5h 7min 49.498s CPU time. Jan 17 02:49:14 pve21atrs001 systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 19. Jan 17 02:49:14 pve21atrs001 systemd[1]: Stopped The Proxmox VE cluster filesystem. Jan 17 02:49:14 pve21atrs001 systemd[1]: pve-cluster.service: Consumed 5h 7min 49.498s CPU time. Jan 17 02:49:14 pve21atrs001 systemd[1]: Starting The Proxmox VE cluster filesystem... Jan 17 02:49:14 pve21atrs001 pmxcfs[3872894]: fuse: failed to access mountpoint /etc/pve: Transport endpoint is not connected Jan 17 02:49:14 pve21atrs001 pmxcfs[3872894]: [main] crit: fuse_mount error: Transport endpoint is not connected Jan 17 02:49:14 pve21atrs001 pmxcfs[3872894]: [main] crit: fuse_mount error: Transport endpoint is not connected Jan 17 02:49:14 pve21atrs001 pmxcfs[3872894]: [main] notice: exit proxmox configuration filesystem (-1) Jan 17 02:49:14 pve21atrs001 pmxcfs[3872894]: [main] notice: exit proxmox configuration filesystem (-1) Jan 17 02:49:14 pve21atrs001 systemd[1]: pve-cluster.service: Control process exited, code=exited, status=255/EXCEPTION Jan 17 02:49:14 pve21atrs001 systemd[1]: pve-cluster.service: Failed with result 'exit-code'. Jan 17 02:49:14 pve21atrs001 systemd[1]: Failed to start The Proxmox VE cluster filesystem. Jan 17 02:49:15 pve21atrs001 pveproxy[3870252]: ipcc_send_rec[1] failed: Connection refused Jan 17 02:49:15 pve21atrs001 pveproxy[3870252]: ipcc_send_rec[2] failed: Connection refused Jan 17 02:49:15 pve21atrs001 pveproxy[3870252]: ipcc_send_rec[3] failed: Connection refused Jan 17 02:49:15 pve21atrs001 systemd[1]: pve-cluster.service: Scheduled restart job, restart counter is at 20. Jan 17 02:49:15 pve21atrs001 systemd[1]: Stopped The Proxmox VE cluster filesystem. Jan 17 02:49:15 pve21atrs001 systemd[1]: Starting The Proxmox VE cluster filesystem... Jan 17 02:49:15 pve21atrs001 pmxcfs[3872895]: fuse: failed to access mountpoint /etc/pve: Transport endpoint is not connected Thanks for reply. Regards, Premjith R