[pve-devel] [PATCH common 1/2] REST environment: warn helpers: use warn instead of printing to stderr

[pve-devel] [PATCH common 1/2] REST environment: warn helpers: use warn instead of printing to stderr

[Fiona Ebner]

Like this, __WARN__ handlers will still be called. In particular,
daemons like pvestatd will set a __WARN__ handler and also log
warnings to syslog. The intention behind introducing log_warn() was to
make warnings more visible, not less, so fix the semantics to make
sure switching from warn to log_warn() does not have this unintended
side-effect.

Reported-by: Friedrich Weber <f.weber@proxmox.com>
Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/RESTEnvironment.pm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/src/PVE/RESTEnvironment.pm b/src/PVE/RESTEnvironment.pm
index 191c6eb..41efb16 100644
--- a/src/PVE/RESTEnvironment.pm
+++ b/src/PVE/RESTEnvironment.pm
@@ -723,7 +723,7 @@ sub log_warn {
 	$rest_env->warn($message);
     } else {
 	chomp($message);
-	print STDERR "WARN: $message\n";
+	warn "WARN: $message\n";
     }
 }
 
@@ -732,7 +732,7 @@ sub warn {
 
     chomp($message);
 
-    print STDERR "WARN: $message\n";
+    warn "WARN: $message\n";
 
     $self->{warning_count}++;
 }
-- 
2.39.2

[pve-devel] [RFC common 2/2] REST environment: fork worker: install custom __WARN__ handler

[Fiona Ebner]

So all warnings will be treated consistently inside a worker task. In
particular, all warnings will count towards the task warning count and
be more visible in the UI. Avoids the need to switch existing warnings
to log_warn().

When pvedaemon forked a worker, the worker would inherit its __WARN__
handler and also log any warnings to syslog, so make sure those
warnings are not less visible than before, by also logging to syslog.

The same warning would not show up in syslog when the task was invoked
via the CLI instead, which was another inconsistency.

The __WARN__ handler needs to increment the warning_count for warnings
issued with Perl's warn. But then in RESTEnvironment.pm's warn method,
warning_count cannot be incremented anymore, because otherwise a call
to log_warn() would increment it once, call warn, and then the
__WARN__ handler would increment it again. The variable is only ever
read in fork_worker(), so it is safe to just move the code
incrementing it to the __WARN__ handler, because that will be called
by both, Perl's warn and log_warn().

This effectively makes log_warn() and RESTEnvironment.pm's warn method
not worth using anymore, so deprecate them.

Signed-off-by: Fiona Ebner <f.ebner@proxmox.com>
---
 src/PVE/RESTEnvironment.pm | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/src/PVE/RESTEnvironment.pm b/src/PVE/RESTEnvironment.pm
index 41efb16..8394e2e 100644
--- a/src/PVE/RESTEnvironment.pm
+++ b/src/PVE/RESTEnvironment.pm
@@ -537,6 +537,17 @@ sub fork_worker {
 	$SIG{CHLD} = $SIG{PIPE} = 'DEFAULT';
 	$SIG{TTOU} = 'IGNORE';
 
+	$SIG{'__WARN__'} = sub {
+	    my $err = $@;
+	    my $message = $_[0];
+	    chomp($message);
+	    $message =~ s/^WARN: //; # avoid duplicate prefix when triggered by log_warn()
+	    print STDERR "WARN: $message\n";
+	    syslog('warning', $message);
+	    $self->{warning_count}++;
+	    $@ = $err;
+	};
+
 	my $ppgid;
 	# set session/process group allows to kill the process group
 	if ($sync && -t STDIN) {
@@ -716,6 +727,9 @@ sub fork_worker {
     return wantarray ? ($upid, $res) : $upid;
 }
 
+# NOTE: Deprecated - remove once all callers are gone
+# This was introduced for counting task warnings, which is now done via a __WARN__ handler inside
+# fork_worker().
 sub log_warn {
     my ($message) = @_;
 
@@ -727,14 +741,14 @@ sub log_warn {
     }
 }
 
+# NOTE: Deprecated - remove once all callers are gone
+# Remove once all users are gone
 sub warn {
     my ($self, $message) = @_;
 
     chomp($message);
 
     warn "WARN: $message\n";
-
-    $self->{warning_count}++;
 }
 
 # Abstract function
-- 
2.39.2

Re: [pve-devel] [PATCH common 1/2] REST environment: warn helpers: use warn instead of printing to stderr

[Thomas Lamprecht]

Am 05/02/2024 um 13:28 schrieb Fiona Ebner:
> Like this, __WARN__ handlers will still be called. In particular,
> daemons like pvestatd will set a __WARN__ handler and also log
> warnings to syslog. The intention behind introducing log_warn() was to
> make warnings more visible, not less, so fix the semantics to make
> sure switching from warn to log_warn() does not have this unintended
> side-effect.

did you saw my reply w.r.t. this [0]?

https://lists.proxmox.com/pipermail/pve-devel/2024-February/061610.html

Re: [pve-devel] [RFC common 2/2] REST environment: fork worker: install custom __WARN__ handler

[Thomas Lamprecht]

Am 05/02/2024 um 13:28 schrieb Fiona Ebner:
> So all warnings will be treated consistently inside a worker task. In
> particular, all warnings will count towards the task warning count and
> be more visible in the UI. Avoids the need to switch existing warnings
> to log_warn().
> 
> When pvedaemon forked a worker, the worker would inherit its __WARN__
> handler and also log any warnings to syslog, so make sure those
> warnings are not less visible than before, by also logging to syslog.
> 
> The same warning would not show up in syslog when the task was invoked
> via the CLI instead, which was another inconsistency.
> 
> The __WARN__ handler needs to increment the warning_count for warnings
> issued with Perl's warn. But then in RESTEnvironment.pm's warn method,
> warning_count cannot be incremented anymore, because otherwise a call
> to log_warn() would increment it once, call warn, and then the
> __WARN__ handler would increment it again. The variable is only ever
> read in fork_worker(), so it is safe to just move the code
> incrementing it to the __WARN__ handler, because that will be called
> by both, Perl's warn and log_warn().
> 
> This effectively makes log_warn() and RESTEnvironment.pm's warn method
> not worth using anymore, so deprecate them.
> 

they where deliberately added though, warn and task-log warnings can
be two very different things, which IMO should stay that way..

Re: [pve-devel] [RFC common 2/2] REST environment: fork worker: install custom __WARN__ handler

[Fiona Ebner]

Am 05.02.24 um 13:39 schrieb Thomas Lamprecht:
> Am 05/02/2024 um 13:28 schrieb Fiona Ebner:
>> So all warnings will be treated consistently inside a worker task. In
>> particular, all warnings will count towards the task warning count and
>> be more visible in the UI. Avoids the need to switch existing warnings
>> to log_warn().
>>
>> When pvedaemon forked a worker, the worker would inherit its __WARN__
>> handler and also log any warnings to syslog, so make sure those
>> warnings are not less visible than before, by also logging to syslog.
>>
>> The same warning would not show up in syslog when the task was invoked
>> via the CLI instead, which was another inconsistency.
>>
>> The __WARN__ handler needs to increment the warning_count for warnings
>> issued with Perl's warn. But then in RESTEnvironment.pm's warn method,
>> warning_count cannot be incremented anymore, because otherwise a call
>> to log_warn() would increment it once, call warn, and then the
>> __WARN__ handler would increment it again. The variable is only ever
>> read in fork_worker(), so it is safe to just move the code
>> incrementing it to the __WARN__ handler, because that will be called
>> by both, Perl's warn and log_warn().
>>
>> This effectively makes log_warn() and RESTEnvironment.pm's warn method
>> not worth using anymore, so deprecate them.
>>
> 
> they where deliberately added though, warn and task-log warnings can
> be two very different things, which IMO should stay that way..

Then I misunderstood. I though the plan was to have all warnings show up
as task warnings in the long term. It feels a bit strange to me to have
two different kinds of warnings. We should document when log_warn()
should and shouldn't be used then. I honestly don't know.

Re: [pve-devel] [PATCH common 1/2] REST environment: warn helpers: use warn instead of printing to stderr

[Fiona Ebner]

Am 05.02.24 um 13:38 schrieb Thomas Lamprecht:
> Am 05/02/2024 um 13:28 schrieb Fiona Ebner:
>> Like this, __WARN__ handlers will still be called. In particular,
>> daemons like pvestatd will set a __WARN__ handler and also log
>> warnings to syslog. The intention behind introducing log_warn() was to
>> make warnings more visible, not less, so fix the semantics to make
>> sure switching from warn to log_warn() does not have this unintended
>> side-effect.
> 
> did you saw my reply w.r.t. this [0]?
> 
> https://lists.proxmox.com/pipermail/pve-devel/2024-February/061610.html

Yes, but I wanted to make log_warn and warn behave more consistently,
since I wasn't aware that they should be different. I can send a v2 with
using syslog() in log_warn() instead.