public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: "Michael Köppl" <m.koeppl@proxmox.com>
To: "Hannes Laimer" <h.laimer@proxmox.com>, <pve-devel@lists.proxmox.com>
Subject: Re: [PATCH access-control/common 0/2] address probblem with SIGCHLD handler being temporarily overwritten
Date: Wed, 11 Mar 2026 15:56:42 +0100	[thread overview]
Message-ID: <DH01H2NUK8Y7.224EQCC6TTC30@proxmox.com> (raw)
In-Reply-To: <20260304134649.82272-1-h.laimer@proxmox.com>

I also encountered hanging tasks while running e2e tests, often leading
to tests running into timeouts even if the task was already "OK". I
applied these patches to the test VMs and did not encounter problems
with hanging tasks anymore, significantly speeding up the test runs.

Consider this:
Tested-by: Michael Köppl <m.koeppl@proxmox.com>

On Wed Mar 4, 2026 at 2:46 PM CET, Hannes Laimer wrote:
> Thanks a lot @Fabian and @Fiona for helping me debug this!
>
> The problem is that some libaries do overwrite the SIGCHLD handler
> temporarily, if the library is called fast enough this can lead to lost
> CHLD signals which in turn prevents `worker_reaper` from being called in
> RESTEnvironment. So tasks won't get cleaned-up until a different SIGCHLD
> arrives at the same `pvedeamon` process triggering `worker_reaper`.
>
> As @Fabian mentioned in [1] a general re-work of the task handling,
> potentially with `pidfd`s, would make a lot of sense.
>
> These two patches address the problem in the task handling structure as
> it currently is. They
>  - run the PAM lib call in a fork, so signal handler changes the library
>    does are isloated from our process
>  - run `worker_reaper` periodically (5s) do catch any other potential
>    instances of this, since it would be possible that the same happens
>    with other libs, not just PAM
>
> [1] https://lore.proxmox.com/pve-devel/1772617908.i4bmsyq0kp.astroid@yuna.none/T/#m7b0f3873be5755f330e288cfa50905744f225b2b
>
>
> pve-common:
>
> Hannes Laimer (1):
>   RESTEnvironment: periodically reap workers as SIGCHLD fallback
>
>  src/PVE/RESTEnvironment.pm | 9 +++++++++
>  1 file changed, 9 insertions(+)
>
>
> pve-access-control:
>
> Hannes Laimer (1):
>   pam: fork for PAM authentication to isolate SIGCHLD handler
>
>  src/PVE/Auth/PAM.pm | 74 +++++++++++++++++++++++++--------------------
>  1 file changed, 42 insertions(+), 32 deletions(-)
>
>
> Summary over all repositories:
>   2 files changed, 51 insertions(+), 32 deletions(-)





      parent reply	other threads:[~2026-03-11 14:57 UTC|newest]

Thread overview: 7+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2026-03-04 13:46 Hannes Laimer
2026-03-04 13:46 ` [PATCH pve-common 1/1] RESTEnvironment: periodically reap workers as SIGCHLD fallback Hannes Laimer
2026-03-12  9:48   ` Fabian Grünbichler
2026-03-04 13:46 ` [PATCH pve-access-control 1/1] pam: fork for PAM authentication to isolate SIGCHLD handler Hannes Laimer
2026-03-12  9:44   ` Fabian Grünbichler
2026-03-06 17:16 ` [PATCH access-control/common 0/2] address probblem with SIGCHLD handler being temporarily overwritten Stefan Hanreich
2026-03-11 14:56 ` Michael Köppl [this message]

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=DH01H2NUK8Y7.224EQCC6TTC30@proxmox.com \
    --to=m.koeppl@proxmox.com \
    --cc=h.laimer@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal