From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from gate001.proxmox.com (gate001.proxmox.com [IPv6:2a0f:8001:1:32::40]) by lore.proxmox.com (Postfix) with ESMTPS id 5D3C31FF0BC for ; Thu, 29 Oct 2026 06:54:21 +0100 (CET) Received: from gate001.proxmox.com (localhost.localdomain [127.0.0.1]) by gate001.proxmox.com (Proxmox) with ESMTP id 957BF2140D; Thu, 29 Oct 2026 06:54:17 +0100 (CET) From: Kefu Chai To: pve-devel@lists.proxmox.com Subject: [PATCH librados2-perl v2] do not shut down the handle twice if a signal arrives at exit Date: Thu, 29 Oct 2026 13:54:09 +0800 Message-ID: <20261029055409.3552673-1-k.chai@proxmox.com> X-Mailer: git-send-email 2.47.3 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Bm-Milter-Handled: 55990f41-d878-4baa-be0a-ee34c49e34d2 X-Bm-Transport-Timestamp: 1793253252841 X-SPAM-LEVEL: Spam detection results: 0 AWL 0.208 Adjusted score from AWL reputation of From: address DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment (newer systems) POISEN_SPAM_PILL_3 0.1 random spam to be learned in bayes RCVD_IN_DNSWL_MED -2.3 Sender listed at https://www.dnswl.org/, medium trust SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Message-ID-Hash: JFH34ZUMH3MKUOOS6CDYX4ZGN5VTOZOQ X-Message-ID-Hash: JFH34ZUMH3MKUOOS6CDYX4ZGN5VTOZOQ X-MailFrom: k.chai@proxmox.com X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; loop; banned-address; emergency; member-moderation; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header X-Mailman-Version: 3.3.10 Precedence: list List-Id: Proxmox VE development discussion List-Help: List-Owner: List-Post: List-Subscribe: List-Unsubscribe: When a task runs in the foreground from the command line, fork_worker() sends SIGTERM to the worker's process group once the worker exits. The worker's pverados child is in that group, and it is often already shutting down the handle after seeing EOF. The handler then calls rados_shutdown() on the freed handle, and the child segfaults: pverados[...]: segfault at 3bc0 ... in libc.so.6[95ba0,...] This affects commands like 'pveceph pool create' or 'pvesh create /nodes//ceph/pool', whose worker opens a RADOS connection. It is a race, but on a test cluster with Ceph 21.1.2, 'pveceph pool create' crashed this way in all four runs. Tasks started from the GUI or the API run in the background. Their process group is not signalled at the end, and they did not crash. The crash is a regression from commit 8cab00f ("reset inherited signal handlers in child worker"), first released in librados2-perl 1.4.1 in August 2023. Before that, the child kept the worker's handler, which dies instead of shutting down the handle again. The new handlers also outlived the eval, since the 'local' only covered $SIG{HUP}. So ignore the signals before shutting down, and install the handlers first thing in the child. Tested-by: Maximiliano Sandoval Signed-off-by: Kefu Chai --- Changes since v1 [1]: - say in the commit message which setups are affected and since when, as Maximiliano asked. The code is unchanged. [1] https://lore.proxmox.com/pve-devel/20261020141223.2814127-1-k.chai@proxmox.com/ PVE/RADOS.pm | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/PVE/RADOS.pm b/PVE/RADOS.pm index 5a79235..deb4912 100644 --- a/PVE/RADOS.pm +++ b/PVE/RADOS.pm @@ -200,6 +200,17 @@ sub new { } else { # child $0 = 'pverados'; + my @signals = qw(HUP INT QUIT TERM); + + # ignore signals from here on, the handler must not shut down a freed handle + my $shutdown = sub { + @SIG{@signals} = ('IGNORE') x @signals; + pve_rados_shutdown($self->{conn}) if $self->{conn}; + }; + + # override signal handlers inherited from the parent + @SIG{@signals} = (sub { $shutdown->(); POSIX::_exit(1); }) x @signals; + eval { PVE::INotify::inotify_close(); @@ -207,12 +218,6 @@ sub new { &$atfork(); } - # override signal handlers inherited from the parent - local $SIG{HUP} = $SIG{INT} = $SIG{QUIT} = $SIG{TERM} = sub { - pve_rados_shutdown($self->{conn}) if $self->{conn}; - POSIX::_exit(1); - }; - # fixme: timeout? close $child; @@ -222,7 +227,7 @@ sub new { my $err = $@; warn $err if $err; - pve_rados_shutdown($self->{conn}) if $self->{conn}; + $shutdown->(); POSIX::_exit($err ? 1 : 0); } -- 2.47.3