From: "Michael Köppl" <m.koeppl@proxmox.com>
To: "Hannes Laimer" <h.laimer@proxmox.com>, <pve-devel@lists.proxmox.com>
Subject: Re: [PATCH access-control/common 0/2] address probblem with SIGCHLD handler being temporarily overwritten
Date: Wed, 11 Mar 2026 15:56:42 +0100 [thread overview]
Message-ID: <DH01H2NUK8Y7.224EQCC6TTC30@proxmox.com> (raw)
In-Reply-To: <20260304134649.82272-1-h.laimer@proxmox.com>
I also encountered hanging tasks while running e2e tests, often leading
to tests running into timeouts even if the task was already "OK". I
applied these patches to the test VMs and did not encounter problems
with hanging tasks anymore, significantly speeding up the test runs.
Consider this:
Tested-by: Michael Köppl <m.koeppl@proxmox.com>
On Wed Mar 4, 2026 at 2:46 PM CET, Hannes Laimer wrote:
> Thanks a lot @Fabian and @Fiona for helping me debug this!
>
> The problem is that some libaries do overwrite the SIGCHLD handler
> temporarily, if the library is called fast enough this can lead to lost
> CHLD signals which in turn prevents `worker_reaper` from being called in
> RESTEnvironment. So tasks won't get cleaned-up until a different SIGCHLD
> arrives at the same `pvedeamon` process triggering `worker_reaper`.
>
> As @Fabian mentioned in [1] a general re-work of the task handling,
> potentially with `pidfd`s, would make a lot of sense.
>
> These two patches address the problem in the task handling structure as
> it currently is. They
> - run the PAM lib call in a fork, so signal handler changes the library
> does are isloated from our process
> - run `worker_reaper` periodically (5s) do catch any other potential
> instances of this, since it would be possible that the same happens
> with other libs, not just PAM
>
> [1] https://lore.proxmox.com/pve-devel/1772617908.i4bmsyq0kp.astroid@yuna.none/T/#m7b0f3873be5755f330e288cfa50905744f225b2b
>
>
> pve-common:
>
> Hannes Laimer (1):
> RESTEnvironment: periodically reap workers as SIGCHLD fallback
>
> src/PVE/RESTEnvironment.pm | 9 +++++++++
> 1 file changed, 9 insertions(+)
>
>
> pve-access-control:
>
> Hannes Laimer (1):
> pam: fork for PAM authentication to isolate SIGCHLD handler
>
> src/PVE/Auth/PAM.pm | 74 +++++++++++++++++++++++++--------------------
> 1 file changed, 42 insertions(+), 32 deletions(-)
>
>
> Summary over all repositories:
> 2 files changed, 51 insertions(+), 32 deletions(-)
prev parent reply other threads:[~2026-03-11 14:57 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-03-04 13:46 Hannes Laimer
2026-03-04 13:46 ` [PATCH pve-common 1/1] RESTEnvironment: periodically reap workers as SIGCHLD fallback Hannes Laimer
2026-03-12 9:48 ` Fabian Grünbichler
2026-03-04 13:46 ` [PATCH pve-access-control 1/1] pam: fork for PAM authentication to isolate SIGCHLD handler Hannes Laimer
2026-03-12 9:44 ` Fabian Grünbichler
2026-03-06 17:16 ` [PATCH access-control/common 0/2] address probblem with SIGCHLD handler being temporarily overwritten Stefan Hanreich
2026-03-11 14:56 ` Michael Köppl [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=DH01H2NUK8Y7.224EQCC6TTC30@proxmox.com \
--to=m.koeppl@proxmox.com \
--cc=h.laimer@proxmox.com \
--cc=pve-devel@lists.proxmox.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox