Re: [pve-devel] [PATCH ha-manager 2/3] watchdog: warn when about to expire

[Thomas Lamprecht <t.lamprecht@proxmox.com>]

Am 19.05.25 um 15:09 schrieb Maximiliano Sandoval:
> Signed-off-by: Maximiliano Sandoval <m.sandoval@proxmox.com>
> ---
>  src/watchdog-mux.c | 26 ++++++++++++++++++++++++++
>  1 file changed, 26 insertions(+)
> 
> diff --git a/src/watchdog-mux.c b/src/watchdog-mux.c
> index a9017b3..e14c768 100644
> --- a/src/watchdog-mux.c
> +++ b/src/watchdog-mux.c
> @@ -29,15 +29,24 @@
>  
>  #define JOURNALCTL_BIN "/bin/journalctl"
>  
> +#define CLIENT_WATCHDOG_TIMEOUT_WARNING 50
> +
>  int watchdog_fd = -1;
>  int watchdog_timeout = 10;
>  int client_watchdog_timeout = 60;
>  int update_watchdog = 1;
>  
> +enum warning_state_t {
> +   NONE,
> +   WARNING_ISSUED,
> +   CRISIS_AVERTED,
> +};
> +
>  typedef struct {
>      int fd;
>      time_t time;
>      int magic_close;
> +    enum warning_state_t warning_state;
>  } wd_client_t;
>  
>  #define MAX_CLIENTS 100
> @@ -54,6 +63,7 @@ alloc_client(int fd, time_t time)
>              client_list[i].fd = fd;
>              client_list[i].time = time;
>              client_list[i].magic_close = 0;
> +            client_list[i].warning_state = NONE;
>              return &client_list[i];
>          }
>      }
> @@ -244,6 +254,22 @@ main(void)
>                  time_t ctime = time(NULL);
>                  for (i = 0; i < MAX_CLIENTS; i++) {
>                      if (client_list[i].fd != 0 && client_list[i].time != 0) {
> +                        if (
> +                            client_list[i].warning_state == WARNING_ISSUED
> +                            && (ctime - client_list[i].time) <= CLIENT_WATCHDOG_TIMEOUT_WARNING
> +                        ) {
> +                            client_list[i].warning_state = CRISIS_AVERTED;
> +                            fprintf(stderr, "phew, client watchdog was updated before expiring\n");

please no "phew" in the logs, this is a serious topic were being
factual and bland is definitively preferred.

> +                        }
> +
> +                        if (
> +                            client_list[i].warning_state != WARNING_ISSUED
> +                            && (ctime - client_list[i].time) > CLIENT_WATCHDOG_TIMEOUT_WARNING
> +                        ) {
> +                            client_list[i].warning_state = WARNING_ISSUED;
> +                            fprintf(stderr, "client watchdog is about to expire\n");

would be good to include the time left here in the message, allows
admins/support a more informed picture, especially if the timeouts
ever change.

> +                        }
> +
>                          if ((ctime - client_list[i].time) > client_watchdog_timeout) {
>                              update_watchdog = 0;
>                              fprintf(stderr, "client watchdog expired - disable watchdog updates\n");



_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel