From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from firstgate.proxmox.com (firstgate.proxmox.com [IPv6:2a01:7e0:0:424::9]) by lore.proxmox.com (Postfix) with ESMTPS id 6AD701FF161 for ; Tue, 24 Sep 2024 14:25:37 +0200 (CEST) Received: from firstgate.proxmox.com (localhost [127.0.0.1]) by firstgate.proxmox.com (Proxmox) with ESMTP id BB589D0A3; Tue, 24 Sep 2024 14:25:50 +0200 (CEST) Message-ID: <69e71cf0-a8c4-42a6-a1a5-36024e903687@proxmox.com> Date: Tue, 24 Sep 2024 14:25:18 +0200 MIME-Version: 1.0 User-Agent: Mozilla Thunderbird From: Daniel Kral To: Proxmox VE development discussion References: <20240917055020.10507-1-sascha.westermann@hl-services.de> Content-Language: en-US In-Reply-To: X-SPAM-LEVEL: Spam detection results: 0 AWL 0.003 Adjusted score from AWL reputation of From: address BAYES_00 -1.9 Bayes spam probability is 0 to 1% DMARC_MISSING 0.1 Missing DMARC policy KAM_DMARC_STATUS 0.01 Test Rule for DKIM or SPF Failure with Strict Alignment SPF_HELO_NONE 0.001 SPF: HELO does not publish an SPF Record SPF_PASS -0.001 SPF: sender matches SPF record Subject: Re: [pve-devel] [PATCH pve-manager 2/3] Fix #5708: Add CPU raw counters X-BeenThere: pve-devel@lists.proxmox.com X-Mailman-Version: 2.1.29 Precedence: list List-Id: Proxmox VE development discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Proxmox VE development discussion Cc: Sascha Westermann Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Errors-To: pve-devel-bounces@lists.proxmox.com Sender: "pve-devel" On 9/17/24 07:50, Sascha Westermann via pve-devel wrote: > Add a map containing raw values from /proc/stat and "uptime_ticks" which > can be used in combination with cpuinfo.user_hz to calculate CPU usage > from two samples. "uptime_ticks" is only defined at the top level, as > /proc/stat is read once, so that core-specific raw values match this > value. > > Signed-off-by: Sascha Westermann > --- > PVE/API2/Nodes.pm | 32 ++++++++++++++++++++++++++++++++ > 1 file changed, 32 insertions(+) > > diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm > index 9920e977..1943ec56 100644 > --- a/PVE/API2/Nodes.pm > +++ b/PVE/API2/Nodes.pm > @@ -5,6 +5,7 @@ use warnings; > > use Digest::MD5; > use Digest::SHA; > +use IO::File; > use Filesys::Df; > use HTTP::Status qw(:constants); > use JSON; > @@ -466,6 +467,37 @@ __PACKAGE__->register_method({ note: the same route also gets called when using the WebGUI and a set of the values that get returned are displayed on the "Node > Status" page. What I have seen, the added data size is very negligible. > $res->{cpu} = $stat->{cpu}; > $res->{wait} = $stat->{wait}; > > + if (my $fh = IO::File->new ("/proc/stat", "r")) { nit: Minor note, but there shouldn't be a space between the function's name and its parameter list [0]. > + my ($uptime_ticks) = PVE::ProcFSTools::read_proc_uptime(1); > + while (defined (my $line = <$fh>)) { > + if ($line =~ m|^cpu\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)(?:\s+(\d+)\s+(\d+))?|) { > + $res->{cpustat}->{user} = int($1); > + $res->{cpustat}->{nice} = int($2); > + $res->{cpustat}->{system} = int($3); > + $res->{cpustat}->{idle} = int($4); > + $res->{cpustat}->{iowait} = int($5); > + $res->{cpustat}->{irq} = int($6); > + $res->{cpustat}->{softirq} = int($7); > + $res->{cpustat}->{steal} = int($8); > + $res->{cpustat}->{guest} = int($9); > + $res->{cpustat}->{guest_nice} = int($10); > + $res->{cpustat}->{uptime_ticks} = $uptime_ticks; nit: I think this could be placed rather nicely at `$res->{uptime_ticks}`, like `$res->{uptime}`, to make `cpustat` a little more consistent with `PVE::ProcFSTools::read_proc_stat()` and > + } elsif ($line =~ m|^cpu(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)\s+(\d+)(?:\s+(\d+)\s+(\d+))?|) { > + $res->{cpustat}->{"cpu" . $1}->{user} = int($2); > + $res->{cpustat}->{"cpu" . $1}->{nice} = int($3); > + $res->{cpustat}->{"cpu" . $1}->{system} = int($4); > + $res->{cpustat}->{"cpu" . $1}->{idle} = int($5); > + $res->{cpustat}->{"cpu" . $1}->{iowait} = int($6); > + $res->{cpustat}->{"cpu" . $1}->{irq} = int($7); > + $res->{cpustat}->{"cpu" . $1}->{softirq} = int($8); > + $res->{cpustat}->{"cpu" . $1}->{steal} = int($9); > + $res->{cpustat}->{"cpu" . $1}->{guest} = int($10); > + $res->{cpustat}->{"cpu" . $1}->{guest_nice} = int($11); > + } > + } > + $fh->close; > + } Is there something that is holding us back to move this directly into `PVE::ProcFSTools::read_proc_stat()`? As far as I can tell, the output of `PVE::ProcFSTools::read_proc_stat()` is used at these locations: - the PVE `/nodes/{node}/status` API endpoint of course, which only uses the values of `cpu` and `wait` at the moment - `PMG::API2::Nodes`: also only uses the values of `cpu` and `wait` - the PMG `/nodes/{node}/status` API endpoint, which also only uses the values of `cpu` and `wait` - `PVE::Service::pvestatd::update_node_status`: retrieve the current node status and then update them for rrd via `broadcast_rrd` (uses only the values of `cpu` and `wait` selectively) and external metric servers The first three and a half (speaking of `broadcast_rrd` in the latter) look fine to me, but we should take a closer look how external metric servers will handle the added data, especially for existing queries/dashboards. It could also be a name collision, as 'cpustat' is also used for the data that gets sent to the metric servers. In my opinion, I think it would be a worthwhile feature to add the properties for external metric servers (either as part of this or a future patch series). > + > my $meminfo = PVE::ProcFSTools::read_meminfo(); > $res->{memory} = { > free => $meminfo->{memfree}, > -- > 2.46.0 It would also be very beneficial if the added data properties that are returned here are documented in the 'returns' JSONSchema, so that they can be easily understood by other users as well (especially in which unit those raw values are so that it's easier to know how they would need to get converted). --- Otherwise, this works just as intended when querying the API endpoint `/nodes/{node}/status` via curl and pvesh. Reviewed-by: Daniel Kral Tested-by: Daniel Kral [0] https://pve.proxmox.com/wiki/Perl_Style_Guide#Spacing_and_syntax_usage _______________________________________________ pve-devel mailing list pve-devel@lists.proxmox.com https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel