public inbox for pdm-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: Lukas Wagner <l.wagner@proxmox.com>
To: pdm-devel@lists.proxmox.com
Subject: [pdm-devel] [PATCH proxmox-datacenter-manager v2 14/28] metric collection: add test for rrd task
Date: Fri, 14 Feb 2025 14:06:39 +0100	[thread overview]
Message-ID: <20250214130653.283012-15-l.wagner@proxmox.com> (raw)
In-Reply-To: <20250214130653.283012-1-l.wagner@proxmox.com>

This commit adds a quick smoke-test for the `store_in_rrd` task
function. Unfortunately, due to some race-condition in proxmox_rrd,
some extra measures had to be taken in order to make the test not flaky.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 server/src/metric_collection/rrd_task.rs | 92 ++++++++++++++++++++++++
 1 file changed, 92 insertions(+)

diff --git a/server/src/metric_collection/rrd_task.rs b/server/src/metric_collection/rrd_task.rs
index 3704b0e7..27327a29 100644
--- a/server/src/metric_collection/rrd_task.rs
+++ b/server/src/metric_collection/rrd_task.rs
@@ -138,3 +138,95 @@ fn store_metric_pbs(cache: &Cache, remote_name: &str, data_point: &MetricDataPoi
         data_source_type,
     );
 }
+
+#[cfg(test)]
+mod tests {
+    use proxmox_rrd_api_types::{RrdMode, RrdTimeframe};
+    use pve_api_types::{ClusterMetrics, ClusterMetricsData};
+
+    use crate::{
+        metric_collection::collection_task::tests::get_create_options,
+        test_support::temp::NamedTempDir,
+    };
+
+    use super::*;
+
+    #[tokio::test]
+    async fn test_rrd_task_persists_data() -> Result<(), Error> {
+        // Arrange
+        let dir = NamedTempDir::new()?;
+        let options = get_create_options().perm(nix::sys::stat::Mode::from_bits_truncate(0o700));
+        let cache = rrd_cache::init(&dir.path(), options.clone(), options.clone())?;
+
+        let (tx, rx) = tokio::sync::mpsc::channel(10);
+        let task = store_in_rrd_task(Arc::clone(&cache), rx);
+        let handle = tokio::task::spawn(task);
+
+        let now = proxmox_time::epoch_i64();
+
+        let metrics = ClusterMetrics {
+            data: vec![
+                ClusterMetricsData {
+                    id: "node/some-node".into(),
+                    metric: "cpu_current".into(),
+                    timestamp: now - 30,
+                    ty: ClusterMetricsDataType::Gauge,
+                    value: 0.1,
+                },
+                ClusterMetricsData {
+                    id: "node/some-node".into(),
+                    metric: "cpu_current".into(),
+                    timestamp: now - 20,
+                    ty: ClusterMetricsDataType::Gauge,
+                    value: 0.2,
+                },
+                ClusterMetricsData {
+                    id: "node/some-node".into(),
+                    metric: "cpu_current".into(),
+                    timestamp: now - 10,
+                    ty: ClusterMetricsDataType::Gauge,
+                    value: 0.1,
+                },
+                ClusterMetricsData {
+                    id: "node/some-node".into(),
+                    metric: "cpu_current".into(),
+                    timestamp: now,
+                    ty: ClusterMetricsDataType::Gauge,
+                    value: 0.2,
+                },
+            ],
+        };
+        let (tx_back, rx_back) = tokio::sync::oneshot::channel();
+        let request = RrdStoreRequest::Pve {
+            remote: "some-remote".into(),
+            metrics,
+            channel: tx_back,
+        };
+
+        // Act
+        tx.send(request).await?;
+        let result = rx_back.await?;
+
+        // Assert
+        assert_eq!(result.most_recent_timestamp, now);
+
+        drop(tx);
+        handle.await??;
+
+        // There is some race condition in proxmox_rrd, in some rare cases
+        // extract_data does not return any data directly after writing.
+        if let Some(data) = rrd_cache::extract_data(
+            &cache,
+            "pve/some-remote/node/some-node",
+            "cpu_current",
+            RrdTimeframe::Hour,
+            RrdMode::Max,
+        )? {
+            // Only assert that there are some data points, the exact position in the vec
+            // might vary due to changed boundaries.
+            assert!(data.data.iter().any(Option::is_some));
+        }
+
+        Ok(())
+    }
+}
-- 
2.39.5



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel


  parent reply	other threads:[~2025-02-14 13:07 UTC|newest]

Thread overview: 34+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-02-14 13:06 [pdm-devel] [PATCH proxmox-datacenter-manager v2 00/28] metric collection improvements (concurrency, config, API, CLI) Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 01/28] test support: add NamedTempFile helper Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 02/28] test support: add NamedTempDir helper Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 03/28] pdm-api-types: add CollectionSettings type Lukas Wagner
2025-02-18 15:26   ` Wolfgang Bumiller
2025-02-18 15:31     ` Stefan Hanreich
2025-02-21  8:27     ` Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 04/28] pdm-config: add functions for reading/writing metric collection settings Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 05/28] metric collection: split top_entities split into separate module Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 06/28] metric collection: save metric data to RRD in separate task Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 07/28] metric collection: rework metric poll task Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 08/28] metric collection: persist state after metric collection Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 09/28] metric collection: skip if last_collection < MIN_COLLECTION_INTERVAL Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 10/28] metric collection: collect overdue metrics on startup/timer change Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 11/28] metric collection: add tests for the fetch_remotes function Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 12/28] metric collection: add test for fetch_overdue Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 13/28] metric collection: pass rrd cache instance as function parameter Lukas Wagner
2025-02-14 13:06 ` Lukas Wagner [this message]
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 15/28] metric collection: wrap rrd_cache::Cache in a struct Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 16/28] metric collection: record remote response time in metric database Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 17/28] metric collection: save time needed for collection run to RRD Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 18/28] metric collection: periodically clean removed remotes from statefile Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 19/28] api: add endpoint for updating metric collection settings Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 20/28] api: add endpoint to trigger metric collection Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 21/28] api: remotes: trigger immediate metric collection for newly added nodes Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 22/28] api: add api for querying metric collection RRD data Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 23/28] api: metric-collection: add status endpoint Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 24/28] pdm-client: add metric collection API methods Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 25/28] cli: add commands for metric-collection settings, trigger, status Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 26/28] metric collection: factor out handle_tick and handle_control_message fns Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 27/28] metric collection: skip missed timer ticks Lukas Wagner
2025-02-14 13:06 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 28/28] metric collection: use JoinSet instead of joining from handles in a Vec Lukas Wagner
2025-02-21 13:19 ` [pdm-devel] [PATCH proxmox-datacenter-manager v2 00/28] metric collection improvements (concurrency, config, API, CLI) Maximiliano Sandoval
2025-03-14 14:10 ` Lukas Wagner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250214130653.283012-15-l.wagner@proxmox.com \
    --to=l.wagner@proxmox.com \
    --cc=pdm-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal