all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: Lukas Wagner <l.wagner@proxmox.com>
To: pdm-devel@lists.proxmox.com
Subject: [pdm-devel] [PATCH proxmox-datacenter-manager 14/25] metric collection: add test for rrd task
Date: Tue, 11 Feb 2025 13:05:30 +0100	[thread overview]
Message-ID: <20250211120541.163621-15-l.wagner@proxmox.com> (raw)
In-Reply-To: <20250211120541.163621-1-l.wagner@proxmox.com>

This commit adds a quick smoke-test for the `store_in_rrd` task
function. Unfortunately, due to some race-condition in proxmox_rrd,
some extra measures had to be taken in order to make the test not flaky.

Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
 server/src/metric_collection/rrd_task.rs | 93 ++++++++++++++++++++++++
 1 file changed, 93 insertions(+)

diff --git a/server/src/metric_collection/rrd_task.rs b/server/src/metric_collection/rrd_task.rs
index a678bb5..e5abb2d 100644
--- a/server/src/metric_collection/rrd_task.rs
+++ b/server/src/metric_collection/rrd_task.rs
@@ -131,3 +131,96 @@ fn store_metric_pbs(cache: &Cache, remote_name: &str, data_point: &MetricDataPoi
         data_source_type,
     );
 }
+
+#[cfg(test)]
+mod tests {
+    use proxmox_rrd_api_types::{RrdMode, RrdTimeframe};
+    use pve_api_types::{ClusterMetrics, ClusterMetricsData};
+
+    use crate::{
+        metric_collection::collection_task::tests::get_create_options,
+        test_support::temp::NamedTempDir,
+    };
+
+    use super::*;
+
+    #[tokio::test]
+    async fn test_rrd_task_persists_data() -> Result<(), Error> {
+        // Arrange
+        let dir = NamedTempDir::new()?;
+        let options = get_create_options().perm(nix::sys::stat::Mode::from_bits_truncate(0o700));
+        let cache = rrd_cache::init(&dir.path(), options.clone(), options.clone())?;
+
+        let (tx, rx) = tokio::sync::mpsc::channel(10);
+        let task = store_in_rrd_task(Arc::clone(&cache), rx);
+        let handle = tokio::task::spawn(task);
+
+        let now = proxmox_time::epoch_i64();
+
+        let metrics = ClusterMetrics {
+            data: vec![
+                ClusterMetricsData {
+                    id: "node/some-node".into(),
+                    metric: "cpu_current".into(),
+                    timestamp: now - 30,
+                    ty: ClusterMetricsDataType::Gauge,
+                    value: 0.1,
+                },
+                ClusterMetricsData {
+                    id: "node/some-node".into(),
+                    metric: "cpu_current".into(),
+                    timestamp: now - 20,
+                    ty: ClusterMetricsDataType::Gauge,
+                    value: 0.2,
+                },
+                ClusterMetricsData {
+                    id: "node/some-node".into(),
+                    metric: "cpu_current".into(),
+                    timestamp: now - 10,
+                    ty: ClusterMetricsDataType::Gauge,
+                    value: 0.1,
+                },
+                ClusterMetricsData {
+                    id: "node/some-node".into(),
+                    metric: "cpu_current".into(),
+                    timestamp: now,
+                    ty: ClusterMetricsDataType::Gauge,
+                    value: 0.2,
+                },
+            ],
+        };
+        let (tx_back, rx_back) = tokio::sync::oneshot::channel();
+        let request = RrdStoreRequest::Pve {
+            remote: "some-remote".into(),
+            metrics,
+            channel: tx_back,
+        };
+
+        // Act
+        tx.send(request).await?;
+        let result = rx_back.await?;
+
+        // Assert
+        assert_eq!(result.most_recent_timestamp, now);
+
+        drop(tx);
+        handle.await??;
+
+        // There is some race condition in proxmox_rrd, in some rare cases
+        // extract_data does not return any data directly after writing.
+        if let Some(data) = rrd_cache::extract_data(
+            &cache,
+            "pve/some-remote/node/some-node",
+            "cpu_current",
+            RrdTimeframe::Hour,
+            RrdMode::Max,
+        )? {
+            // Only assert that there are some data points, the
+            // exact position in the vec might vary due to
+            // changed boundaries
+            assert!(data.data.iter().any(Option::is_some));
+        }
+
+        Ok(())
+    }
+}
-- 
2.39.5



_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel


  parent reply	other threads:[~2025-02-11 12:06 UTC|newest]

Thread overview: 49+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2025-02-11 12:05 [pdm-devel] [PATCH proxmox-datacenter-manager 00/25] metric collection improvements (concurrency, config, API, CLI) Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 01/25] test support: add NamedTempFile helper Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 02/25] test support: add NamedTempDir helper Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 03/25] pdm-api-types: add CollectionSettings type Lukas Wagner
2025-02-11 14:18   ` Maximiliano Sandoval
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 04/25] pdm-config: add functions for reading/writing metric collection settings Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 05/25] metric collection: split top_entities split into separate module Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 06/25] metric collection: save metric data to RRD in separate task Lukas Wagner
2025-02-12 13:59   ` Wolfgang Bumiller
2025-02-12 14:32     ` Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 07/25] metric collection: rework metric poll task Lukas Wagner
2025-02-11 12:58   ` Lukas Wagner
2025-02-12 15:57   ` Wolfgang Bumiller
2025-02-13 12:31     ` Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 08/25] metric collection: persist state after metric collection Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 09/25] metric collection: skip if last_collection < MIN_COLLECTION_INTERVAL Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 10/25] metric collection: collect overdue metrics on startup/timer change Lukas Wagner
2025-02-13  8:55   ` Wolfgang Bumiller
2025-02-13 13:50     ` Lukas Wagner
2025-02-13 14:19       ` Wolfgang Bumiller
2025-02-13 15:21         ` Lukas Wagner
2025-02-13 15:34           ` Wolfgang Bumiller
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 11/25] metric collection: add tests for the fetch_remotes function Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 12/25] metric collection: add test for fetch_overdue Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 13/25] metric collection: pass rrd cache instance as function parameter Lukas Wagner
2025-02-11 12:05 ` Lukas Wagner [this message]
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 15/25] metric collection: wrap rrd_cache::Cache in a struct Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 16/25] metric collection: record remote response time in metric database Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 17/25] metric collection: save time needed for collection run to RRD Lukas Wagner
2025-02-13 11:53   ` Wolfgang Bumiller
2025-02-13 12:12     ` Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 18/25] metric collection: periodically clean removed remotes from statefile Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 19/25] api: add endpoint for updating metric collection settings Lukas Wagner
2025-02-13 12:09   ` Wolfgang Bumiller
2025-02-13 12:15     ` Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 20/25] api: add endpoint to trigger metric collection Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 21/25] api: remotes: trigger immediate metric collection for newly added nodes Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 22/25] api: add api for querying metric collection RRD data Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 23/25] api: metric-collection: add status endpoint Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 24/25] pdm-client: add metric collection API methods Lukas Wagner
2025-02-13 12:10   ` Wolfgang Bumiller
2025-02-13 13:52     ` Lukas Wagner
2025-02-11 12:05 ` [pdm-devel] [PATCH proxmox-datacenter-manager 25/25] cli: add commands for metric-collection settings, trigger, status Lukas Wagner
2025-02-13 12:14   ` Wolfgang Bumiller
2025-02-13 14:17     ` Lukas Wagner
2025-02-13 14:56       ` Wolfgang Bumiller
2025-02-13 14:58         ` Lukas Wagner
2025-02-13 15:11           ` Lukas Wagner
2025-02-14 13:08 ` [pdm-devel] [PATCH proxmox-datacenter-manager 00/25] metric collection improvements (concurrency, config, API, CLI) Lukas Wagner

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20250211120541.163621-15-l.wagner@proxmox.com \
    --to=l.wagner@proxmox.com \
    --cc=pdm-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal