public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Lukas Wagner <l.wagner@proxmox.com>
To: pbs-devel@lists.proxmox.com
Subject: [pbs-devel] [PATCH proxmox-backup v2 00/13] add metric endpoint
Date: Tue, 15 Oct 2024 10:46:23 +0200	[thread overview]
Message-ID: <20241015084636.57106-1-l.wagner@proxmox.com> (raw)

This patch series adds a new metric endpoint at /status/metrics with the same
metric format as PVE's /cluster/metrics/export.

Patches 1 to 9 are preparatory patches, including
  - moving code around to improve the overall structure
  - style/doc comment improvments

These changes should not result in any functional changes.

Patches 10 to 12 store the results from the stat collection loop in a cache (using proxmox-shared-cache)
in /run/proxmox-backup/metrics. Same as in PVE, we cache the last 180 stat results for a total
of 30 minutes of history.

Finally, patch 13 adds the actual endpoint for fetching the metric data.

Changes since v1 (thx Wolfgang):
  - move api2/status.rs to api2/status/mod.rs
  - move api2/metrics.rs to api2/status/metrics.rs

proxmox-backup:

Lukas Wagner (13):
  proxy: server: move rrd stat/metric server to separate module
  metric collection: add doc comments for public functions
  metric collection: move rrd_cache to new metric_collection module
  metric_collection: split out push metric part
  metric collection: rrd: move rrd update function to rrd module
  metric collection: rrd: restrict function visibility
  metric collection: rrd: remove rrd prefix from some function names
  metric collection: drop std::path prefix where not needed
  metric collection: move impl block for DiskStats to metric_server
    module
  pbs-api-types: add types for the new metrics endpoint
  metric collection: initialize metric cache on startup
  metric collection: put metrics in a cache
  api: add /status/metrics API

 Cargo.toml                                    |   2 +
 pbs-api-types/src/metrics.rs                  |  66 +++
 src/api2/node/rrd.rs                          |   2 +-
 src/api2/status/metrics.rs                    |  73 +++
 src/api2/{status.rs => status/mod.rs}         |  17 +-
 src/bin/proxmox-backup-proxy.rs               | 419 +-----------------
 src/lib.rs                                    |   2 -
 src/server/metric_collection/metric_server.rs | 156 +++++++
 src/server/metric_collection/mod.rs           | 235 ++++++++++
 src/server/metric_collection/pull_metrics.rs  | 185 ++++++++
 .../metric_collection/rrd.rs}                 |  86 +++-
 src/server/mod.rs                             |   2 +
 12 files changed, 814 insertions(+), 431 deletions(-)
 create mode 100644 src/api2/status/metrics.rs
 rename src/api2/{status.rs => status/mod.rs} (93%)
 create mode 100644 src/server/metric_collection/metric_server.rs
 create mode 100644 src/server/metric_collection/mod.rs
 create mode 100644 src/server/metric_collection/pull_metrics.rs
 rename src/{rrd_cache.rs => server/metric_collection/rrd.rs} (60%)


Summary over all repositories:
  12 files changed, 814 insertions(+), 431 deletions(-)

-- 
Generated by git-murpp 0.7.3


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


             reply	other threads:[~2024-10-15  8:46 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-10-15  8:46 Lukas Wagner [this message]
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 01/13] proxy: server: move rrd stat/metric server to separate module Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 02/13] metric collection: add doc comments for public functions Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 03/13] metric collection: move rrd_cache to new metric_collection module Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 04/13] metric_collection: split out push metric part Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 05/13] metric collection: rrd: move rrd update function to rrd module Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 06/13] metric collection: rrd: restrict function visibility Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 07/13] metric collection: rrd: remove rrd prefix from some function names Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 08/13] metric collection: drop std::path prefix where not needed Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 09/13] metric collection: move impl block for DiskStats to metric_server module Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 10/13] pbs-api-types: add types for the new metrics endpoint Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 11/13] metric collection: initialize metric cache on startup Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 12/13] metric collection: put metrics in a cache Lukas Wagner
2024-10-15  8:46 ` [pbs-devel] [PATCH proxmox-backup v2 13/13] api: add /status/metrics API Lukas Wagner
2024-10-15 13:02 ` [pbs-devel] applied-series: [PATCH proxmox-backup v2 00/13] add metric endpoint Wolfgang Bumiller

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20241015084636.57106-1-l.wagner@proxmox.com \
    --to=l.wagner@proxmox.com \
    --cc=pbs-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal