* [pve-devel] [PATCH cluster-pve8 v2 1/2] cfs status.c: drop old pve2-vm rrd schema support
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH cluster-pve8 v2 2/2] status: handle new metrics update data Aaron Lauterer
` (22 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
the newer pve2.3-vm schema has been introduced with commit ba9dcfc1 back
in 2013. By now there should be no cluster where an older node might
still send the old pve2-vm schema.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/pmxcfs/status.c | 12 +++---------
1 file changed, 3 insertions(+), 9 deletions(-)
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index 0895e53..eba4e52 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -1234,16 +1234,10 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
create_rrd_file(filename, argcount, rrd_def_node);
}
- } else if ((strncmp(key, "pve2-vm/", 8) == 0) || (strncmp(key, "pve2.3-vm/", 10) == 0)) {
- const char *vmid;
+ } else if (strncmp(key, "pve2.3-vm/", 10) == 0) {
+ const char *vmid = key + 10;
- if (strncmp(key, "pve2-vm/", 8) == 0) {
- vmid = key + 8;
- skip = 2;
- } else {
- vmid = key + 10;
- skip = 4;
- }
+ skip = 4;
if (strchr(vmid, '/') != NULL) {
goto keyerror;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH cluster-pve8 v2 2/2] status: handle new metrics update data
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH cluster-pve8 v2 1/2] cfs status.c: drop old pve2-vm rrd schema support Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH manager-pve8 v2 1/2] api2tools: drop old VM rrd schema Aaron Lauterer
` (21 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
For PVE9 we plan to add additional fields in the metrics that are
collected and distributed in the cluster. The new fields/columns are
added at the end of the current ones. This makes it possible for PVE8
installations to still use them by cutting the new additional data.
To make it more future proof, the format of the keys for each metrics
are changed:
Old: pve{version}-{type}/{id}
New: pve-{type}-{version}/{id}
This way we have an easier time to handle new versions in the future as
we initially only need to check for `pve-{type}-`. If we know the
version, we can handle it accordingly; e.g. pad if older format with
missing data. If we don't know the version, it must be a newer one and
we cut the data stream at the length we need for the current version.
This means of course that to avoid a breaking change, we can only add
new columns if needed, but not remove any! But waiting for a breaking
change until the next major release is a worthy trade-off if it allows
us to expand the format in between if needed.
Since the full keys were used for the final location within the RRD
directory, we need to change that as well and set it manually to
'pve2-{type}' as the key we receive could be for a newer data format.
The 'rrd_skip_data' function got a new parameter defining the sepataring
character. This then makes it possible to use it to determine which part
of the key string is the version/type and which one is the actual
resource identifier.
We drop the pve2-vm schema as the newer pve2.3-vm has been introduced
with commit ba9dcfc1 back in 2013. By now there should be no cluster
where an older node might still send the old pve2-vm schema.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/pmxcfs/status.c | 79 ++++++++++++++++++++++++++++++---------------
1 file changed, 53 insertions(+), 26 deletions(-)
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index eba4e52..640540f 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -1185,16 +1185,33 @@ static void create_rrd_file(const char *filename, int argcount, const char *rrdd
}
}
-static inline const char *rrd_skip_data(const char *data, int count) {
+static inline const char *rrd_skip_data(const char *data, int count, char separator) {
int found = 0;
while (*data && found < count) {
- if (*data++ == ':') {
+ if (*data++ == separator) {
found++;
}
}
return data;
}
+// The key and subdirectory format used up until PVE8 is 'pve{version}-{type}/{id}' with version
+// being 2 or 2.3 for VMs. Starting with PVE9 'pve-{type}-{version}/{id}'. Newer versions are only
+// allowed to append new columns to the data! Otherwise this would be a breaking change.
+//
+// Type can be: node, vm, storage
+//
+// Version is the version of PVE with which it was introduced, e.g.: 9.0, 9.2, 10.0.
+//
+// ID is the actual identifier of the item in question. E.g. node name, VMID or for storage it is
+// '{node}/{storage name}'
+//
+// This way, we can handle unknown new formats gracefully and cut the data at the expected
+// column for the currently understood format. Receiving older formats will still need special
+// checks to determine how much padding is needed.
+//
+// Should we ever plan to change existing columns, we need to introduce this as a breaking
+// change!
static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
g_return_if_fail(key != NULL);
g_return_if_fail(data != NULL);
@@ -1210,12 +1227,13 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
char *filename = NULL;
- int skip = 0;
+ int skip = 0; // columns to skip at beginning. They contain non-archivable data, like uptime,
+ // status, is guest a template and such.
+ int keep_columns = 0; // how many columns do we want to keep (after initial skip) in case we get
+ // more columns than needed from a newer format
- if (strncmp(key, "pve2-node/", 10) == 0) {
- const char *node = key + 10;
-
- skip = 2;
+ if (strncmp(key, "pve2-node/", 10) == 0 || strncmp(key, "pve-node-", 9) == 0) {
+ const char *node = rrd_skip_data(key, 1, '/');
if (strchr(node, '/') != NULL) {
goto keyerror;
@@ -1225,19 +1243,23 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
- filename = g_strdup_printf(RRDDIR "/%s", key);
+ skip = 2; // first two columns are live data that isn't archived
- if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
+ if (strncmp(key, "pve-node-", 9) == 0) {
+ keep_columns = 12; // pve2-node format uses 12 columns
+ }
+ filename = g_strdup_printf(RRDDIR "/pve2-node/%s", node);
+
+ if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
mkdir(RRDDIR "/pve2-node", 0755);
int argcount = sizeof(rrd_def_node) / sizeof(void *) - 1;
create_rrd_file(filename, argcount, rrd_def_node);
}
- } else if (strncmp(key, "pve2.3-vm/", 10) == 0) {
- const char *vmid = key + 10;
+ } else if (strncmp(key, "pve2.3-vm/", 10) == 0 || strncmp(key, "pve-vm-", 7) == 0) {
- skip = 4;
+ const char *vmid = rrd_skip_data(key, 1, '/');
if (strchr(vmid, '/') != NULL) {
goto keyerror;
@@ -1247,29 +1269,29 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
+ skip = 4; // first 4 columns are live data that isn't archived
+
+ if (strncmp(key, "pve-vm-", 7) == 0) {
+ keep_columns = 10; // pve2.3-vm format uses 10 data columns
+ }
+
filename = g_strdup_printf(RRDDIR "/%s/%s", "pve2-vm", vmid);
if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
-
mkdir(RRDDIR "/pve2-vm", 0755);
int argcount = sizeof(rrd_def_vm) / sizeof(void *) - 1;
create_rrd_file(filename, argcount, rrd_def_vm);
}
- } else if (strncmp(key, "pve2-storage/", 13) == 0) {
- const char *node = key + 13;
+ } else if (strncmp(key, "pve2-storage/", 13) == 0 || strncmp(key, "pve-storage-", 12) == 0) {
+ const char *node = rrd_skip_data(key, 1, '/'); // will contain {node}/{storage}
- const char *storage = node;
- while (*storage && *storage != '/') {
- storage++;
- }
+ const char *storage = rrd_skip_data(node, 1, '/');
- if (*storage != '/' || ((storage - node) < 1)) {
+ if ((storage - node) < 1) {
goto keyerror;
}
- storage++;
-
if (strchr(storage, '/') != NULL) {
goto keyerror;
}
@@ -1278,12 +1300,10 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
- filename = g_strdup_printf(RRDDIR "/%s", key);
+ filename = g_strdup_printf(RRDDIR "/pve2-storage/%s", node);
if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
-
mkdir(RRDDIR "/pve2-storage", 0755);
-
char *dir = g_path_get_dirname(filename);
mkdir(dir, 0755);
g_free(dir);
@@ -1296,7 +1316,14 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
- const char *dp = skip ? rrd_skip_data(data, skip) : data;
+ const char *dp = skip ? rrd_skip_data(data, skip, ':') : data;
+
+ if (keep_columns) {
+ keep_columns++; // We specify the number of columns we want earlier, but we also have the
+ // always present timestamp column, so we need to skip one more column
+ char *cut = (char *)rrd_skip_data(dp, keep_columns, ':');
+ *(cut - 1) = 0; // terminate string by replacing colon from field separator with zero.
+ }
const char *update_args[] = {dp, NULL};
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH manager-pve8 v2 1/2] api2tools: drop old VM rrd schema
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH cluster-pve8 v2 1/2] cfs status.c: drop old pve2-vm rrd schema support Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH cluster-pve8 v2 2/2] status: handle new metrics update data Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH manager-pve8 v2 2/2] api2tools: extract stats: handle existence of new pve-{type}-9.0 data Aaron Lauterer
` (20 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
pve2.3-vm has been introduced with commit 3b6ad3ac back in 2013. By now
there should not be any combination of clustered nodes that still send
the old pve2-vm variant.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
PVE/API2Tools.pm | 18 +-----------------
1 file changed, 1 insertion(+), 17 deletions(-)
diff --git a/PVE/API2Tools.pm b/PVE/API2Tools.pm
index d6154925..1e235c47 100644
--- a/PVE/API2Tools.pm
+++ b/PVE/API2Tools.pm
@@ -97,23 +97,7 @@ sub extract_vm_stats {
my $d;
- if ($d = $rrd->{"pve2-vm/$vmid"}) {
-
- $entry->{uptime} = ($d->[0] || 0) + 0;
- $entry->{name} = $d->[1];
- $entry->{status} = $entry->{uptime} ? 'running' : 'stopped';
- $entry->{maxcpu} = ($d->[3] || 0) + 0;
- $entry->{cpu} = ($d->[4] || 0) + 0;
- $entry->{maxmem} = ($d->[5] || 0) + 0;
- $entry->{mem} = ($d->[6] || 0) + 0;
- $entry->{maxdisk} = ($d->[7] || 0) + 0;
- $entry->{disk} = ($d->[8] || 0) + 0;
- $entry->{netin} = ($d->[9] || 0) + 0;
- $entry->{netout} = ($d->[10] || 0) + 0;
- $entry->{diskread} = ($d->[11] || 0) + 0;
- $entry->{diskwrite} = ($d->[12] || 0) + 0;
-
- } elsif ($d = $rrd->{"pve2.3-vm/$vmid"}) {
+ if ($d = $rrd->{"pve2.3-vm/$vmid"}) {
$entry->{uptime} = ($d->[0] || 0) + 0;
$entry->{name} = $d->[1];
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH manager-pve8 v2 2/2] api2tools: extract stats: handle existence of new pve-{type}-9.0 data
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (2 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH manager-pve8 v2 1/2] api2tools: drop old VM rrd schema Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH pve9-rrd-migration-tool v2 1/1] introduce rrd migration tool for pve8 -> pve9 Aaron Lauterer
` (19 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
We add a new function to handle different key names, as it would
otherwise become quite unreadable.
It checks which key format exists for the type and resource:
* the old pve2-{type} / pve2.3-vm
* the new pve-{type}-{version}
and will return the one that was found. Since we will only have one key
per resource, we can return on the first hit.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
changes since:
RFC:
* switch from pve9- to pve-{type}-9.0 schema
PVE/API2Tools.pm | 28 ++++++++++++++++++++++++----
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/PVE/API2Tools.pm b/PVE/API2Tools.pm
index 1e235c47..08548524 100644
--- a/PVE/API2Tools.pm
+++ b/PVE/API2Tools.pm
@@ -41,6 +41,24 @@ sub get_hwaddress {
return $hwaddress;
}
+# each rrd key for a resource will only exist once. The key format might be different though. Therefore return on first hit
+sub get_rrd_key {
+ my ($rrd, $type, $id) = @_;
+
+ # check for old formats: pve2-{type}/{id}. For VMs and CTs the version number is different than for nodes and storages
+ if ($type ne "vm" && exists $rrd->{"pve2-${type}/${id}"}) {
+ return "pve2-${type}/${id}";
+ } elsif ($type eq "vm" && exists $rrd->{"pve2.3-${type}/${id}"}) {
+ return "pve2.3-${type}/${id}";
+ }
+
+ # if no old key has been found, we expect on in the newer format: pve-{type}-{version}/{id}
+ # We accept all new versions, as the expectation is that they are only allowed to add new colums as non-breaking change
+ for my $k (keys %$rrd) {
+ return $k if $k =~ m/^pve-\Q${type}\E-\d\d?.\d\/\Q${id}\E$/;
+ }
+}
+
sub extract_node_stats {
my ($node, $members, $rrd, $exclude_stats) = @_;
@@ -51,8 +69,8 @@ sub extract_node_stats {
status => 'unknown',
};
- if (my $d = $rrd->{"pve2-node/$node"}) {
-
+ my $key = get_rrd_key($rrd, "node", $node);
+ if (my $d = $rrd->{$key}) {
if (
!$members || # no cluster
($members->{$node} && $members->{$node}->{online})
@@ -96,8 +114,9 @@ sub extract_vm_stats {
};
my $d;
+ my $key = get_rrd_key($rrd, "vm", $vmid);
- if ($d = $rrd->{"pve2.3-vm/$vmid"}) {
+ if (my $d = $rrd->{$key}) {
$entry->{uptime} = ($d->[0] || 0) + 0;
$entry->{name} = $d->[1];
@@ -135,7 +154,8 @@ sub extract_storage_stats {
content => $content,
};
- if (my $d = $rrd->{"pve2-storage/$node/$storeid"}) {
+ my $key = get_rrd_key($rrd, "storage", "${node}/${storeid}");
+ if (my $d = $rrd->{$key}) {
$entry->{maxdisk} = ($d->[1] || 0) + 0;
$entry->{disk} = ($d->[2] || 0) + 0;
$entry->{status} = 'available';
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH pve9-rrd-migration-tool v2 1/1] introduce rrd migration tool for pve8 -> pve9
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (3 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH manager-pve8 v2 2/2] api2tools: extract stats: handle existence of new pve-{type}-9.0 data Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH cluster v2 1/4] cfs status.c: drop old pve2-vm rrd schema support Aaron Lauterer
` (18 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
.cargo/config.toml | 5 +
.gitignore | 5 +
Cargo.toml | 20 ++
build.rs | 29 +++
src/lib.rs | 5 +
src/main.rs | 502 ++++++++++++++++++++++++++++++++++++++++
src/parallel_handler.rs | 162 +++++++++++++
wrapper.h | 1 +
8 files changed, 729 insertions(+)
create mode 100644 .cargo/config.toml
create mode 100644 .gitignore
create mode 100644 Cargo.toml
create mode 100644 build.rs
create mode 100644 src/lib.rs
create mode 100644 src/main.rs
create mode 100644 src/parallel_handler.rs
create mode 100644 wrapper.h
diff --git a/.cargo/config.toml b/.cargo/config.toml
new file mode 100644
index 0000000..3b5b6e4
--- /dev/null
+++ b/.cargo/config.toml
@@ -0,0 +1,5 @@
+[source]
+[source.debian-packages]
+directory = "/usr/share/cargo/registry"
+[source.crates-io]
+replace-with = "debian-packages"
diff --git a/.gitignore b/.gitignore
new file mode 100644
index 0000000..7741e63
--- /dev/null
+++ b/.gitignore
@@ -0,0 +1,5 @@
+./target
+./build
+
+Cargo.lock
+
diff --git a/Cargo.toml b/Cargo.toml
new file mode 100644
index 0000000..d3523f3
--- /dev/null
+++ b/Cargo.toml
@@ -0,0 +1,20 @@
+[package]
+name = "proxmox_rrd_migration_8-9"
+version = "0.1.0"
+edition = "2021"
+authors = [
+ "Aaron Lauterer <a.lauterer@proxmox.com>",
+ "Proxmox Support Team <support@proxmox.com>",
+]
+license = "AGPL-3"
+homepage = "https://www.proxmox.com"
+
+[dependencies]
+anyhow = "1.0.86"
+pico-args = "0.5.0"
+proxmox-async = "0.4"
+crossbeam-channel = "0.5"
+
+[build-dependencies]
+bindgen = "0.66.1"
+pkg-config = "0.3"
diff --git a/build.rs b/build.rs
new file mode 100644
index 0000000..56d07cc
--- /dev/null
+++ b/build.rs
@@ -0,0 +1,29 @@
+use std::env;
+use std::path::PathBuf;
+
+fn main() {
+ println!("cargo:rustc-link-lib=rrd");
+
+ println!("cargo:rerun-if-changed=wrapper.h");
+ // The bindgen::Builder is the main entry point
+ // to bindgen, and lets you build up options for
+ // the resulting bindings.
+
+ let bindings = bindgen::Builder::default()
+ // The input header we would like to generate
+ // bindings for.
+ .header("wrapper.h")
+ // Tell cargo to invalidate the built crate whenever any of the
+ // included header files changed.
+ .parse_callbacks(Box::new(bindgen::CargoCallbacks))
+ // Finish the builder and generate the bindings.
+ .generate()
+ // Unwrap the Result and panic on failure.
+ .expect("Unable to generate bindings");
+
+ // Write the bindings to the $OUT_DIR/bindings.rs file.
+ let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
+ bindings
+ .write_to_file(out_path.join("bindings.rs"))
+ .expect("Couldn't write bindings!");
+}
diff --git a/src/lib.rs b/src/lib.rs
new file mode 100644
index 0000000..a38a13a
--- /dev/null
+++ b/src/lib.rs
@@ -0,0 +1,5 @@
+#![allow(non_upper_case_globals)]
+#![allow(non_camel_case_types)]
+#![allow(non_snake_case)]
+
+include!(concat!(env!("OUT_DIR"), "/bindings.rs"));
diff --git a/src/main.rs b/src/main.rs
new file mode 100644
index 0000000..6e6f91c
--- /dev/null
+++ b/src/main.rs
@@ -0,0 +1,502 @@
+use anyhow::{bail, Error, Result};
+use proxmox_rrd_migration_8_9::{rrd_clear_error, rrd_create_r2, rrd_get_context, rrd_get_error};
+use std::ffi::{CStr, CString, OsString};
+use std::fs;
+use std::os::unix::ffi::OsStrExt;
+use std::os::unix::fs::PermissionsExt;
+use std::path::PathBuf;
+use std::sync::Arc;
+
+use crate::parallel_handler::ParallelHandler;
+
+pub mod parallel_handler;
+
+const BASE_DIR: &str = "/var/lib/rrdcached/db";
+const SOURCE_SUBDIR_NODE: &str = "pve2-node";
+const SOURCE_SUBDIR_GUEST: &str = "pve2-vm";
+const SOURCE_SUBDIR_STORAGE: &str = "pve2-storage";
+const TARGET_SUBDIR_NODE: &str = "pve-node-9.0";
+const TARGET_SUBDIR_GUEST: &str = "pve-vm-9.0";
+const TARGET_SUBDIR_STORAGE: &str = "pve-storage-9.0";
+const MAX_THREADS: usize = 4;
+const RRD_STEP_SIZE: usize = 60;
+
+// RRAs are defined in the following way:
+//
+// RRA:CF:xff:step:rows
+// CF: AVERAGE or MAX
+// xff: 0.5
+// steps: stepsize is defined on rrd file creation! example: with 60 seconds step size:
+// e.g. 1 => 60 sec, 30 => 1800 seconds or 30 min
+// rows: how many aggregated rows are kept, as in how far back in time we store data
+//
+// how many seconds are aggregated per RRA: steps * stepsize * rows
+// how many hours are aggregated per RRA: steps * stepsize * rows / 3600
+// how many days are aggregated per RRA: steps * stepsize * rows / 3600 / 24
+// https://oss.oetiker.ch/rrdtool/tut/rrd-beginners.en.html#Understanding_by_an_example
+
+const RRD_VM_DEF: [&CStr; 25] = [
+ c"DS:maxcpu:GAUGE:120:0:U",
+ c"DS:cpu:GAUGE:120:0:U",
+ c"DS:maxmem:GAUGE:120:0:U",
+ c"DS:mem:GAUGE:120:0:U",
+ c"DS:maxdisk:GAUGE:120:0:U",
+ c"DS:disk:GAUGE:120:0:U",
+ c"DS:netin:DERIVE:120:0:U",
+ c"DS:netout:DERIVE:120:0:U",
+ c"DS:diskread:DERIVE:120:0:U",
+ c"DS:diskwrite:DERIVE:120:0:U",
+ c"DS:memhost:GAUGE:120:0:U",
+ c"DS:pressurecpusome:GAUGE:120:0:U",
+ c"DS:pressurecpufull:GAUGE:120:0:U",
+ c"DS:pressureiosome:GAUGE:120:0:U",
+ c"DS:pressureiofull:GAUGE:120:0:U",
+ c"DS:pressurememorysome:GAUGE:120:0:U",
+ c"DS:pressurememoryfull:GAUGE:120:0:U",
+ c"RRA:AVERAGE:0.5:1:1440", // 1 min * 1440 => 1 day
+ c"RRA:AVERAGE:0.5:30:1440", // 30 min * 1440 => 30 day
+ c"RRA:AVERAGE:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ c"RRA:AVERAGE:0.5:10080:570", // 1 week * 570 => ~10 years
+ c"RRA:MAX:0.5:1:1440", // 1 min * 1440 => 1 day
+ c"RRA:MAX:0.5:30:1440", // 30 min * 1440 => 30 day
+ c"RRA:MAX:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ c"RRA:MAX:0.5:10080:570", // 1 week * 570 => ~10 years
+];
+
+const RRD_NODE_DEF: [&CStr; 27] = [
+ c"DS:loadavg:GAUGE:120:0:U",
+ c"DS:maxcpu:GAUGE:120:0:U",
+ c"DS:cpu:GAUGE:120:0:U",
+ c"DS:iowait:GAUGE:120:0:U",
+ c"DS:memtotal:GAUGE:120:0:U",
+ c"DS:memused:GAUGE:120:0:U",
+ c"DS:swaptotal:GAUGE:120:0:U",
+ c"DS:swapused:GAUGE:120:0:U",
+ c"DS:roottotal:GAUGE:120:0:U",
+ c"DS:rootused:GAUGE:120:0:U",
+ c"DS:netin:DERIVE:120:0:U",
+ c"DS:netout:DERIVE:120:0:U",
+ c"DS:memfree:GAUGE:120:0:U",
+ c"DS:arcsize:GAUGE:120:0:U",
+ c"DS:pressurecpusome:GAUGE:120:0:U",
+ c"DS:pressureiosome:GAUGE:120:0:U",
+ c"DS:pressureiofull:GAUGE:120:0:U",
+ c"DS:pressurememorysome:GAUGE:120:0:U",
+ c"DS:pressurememoryfull:GAUGE:120:0:U",
+ c"RRA:AVERAGE:0.5:1:1440", // 1 min * 1440 => 1 day
+ c"RRA:AVERAGE:0.5:30:1440", // 30 min * 1440 => 30 day
+ c"RRA:AVERAGE:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ c"RRA:AVERAGE:0.5:10080:570", // 1 week * 570 => ~10 years
+ c"RRA:MAX:0.5:1:1440", // 1 min * 1440 => 1 day
+ c"RRA:MAX:0.5:30:1440", // 30 min * 1440 => 30 day
+ c"RRA:MAX:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ c"RRA:MAX:0.5:10080:570", // 1 week * 570 => ~10 years
+];
+
+const RRD_STORAGE_DEF: [&CStr; 10] = [
+ c"DS:total:GAUGE:120:0:U",
+ c"DS:used:GAUGE:120:0:U",
+ c"RRA:AVERAGE:0.5:1:1440", // 1 min * 1440 => 1 day
+ c"RRA:AVERAGE:0.5:30:1440", // 30 min * 1440 => 30 day
+ c"RRA:AVERAGE:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ c"RRA:AVERAGE:0.5:10080:570", // 1 week * 570 => ~10 years
+ c"RRA:MAX:0.5:1:1440", // 1 min * 1440 => 1 day
+ c"RRA:MAX:0.5:30:1440", // 30 min * 1440 => 30 day
+ c"RRA:MAX:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ c"RRA:MAX:0.5:10080:570", // 1 week * 570 => ~10 years
+];
+
+const HELP: &str = "\
+proxmox-rrd-migration tool
+
+Migrates existing RRD graph data to the new format.
+
+Use this only in the process of upgrading from Proxmox VE 8 to 9 according to the upgrade guide!
+
+USAGE:
+ proxmox-rrd-migration [OPTIONS]
+
+ FLAGS:
+ -h, --help Prints this help information
+
+ OPTIONS:
+ --force Migrate, even if the target already exists.
+ This will overwrite any migrated RRD files!
+
+ --threads THREADS Number of paralell threads. Default from 1 to 4.
+
+ --test For internal use only.
+ Tests parallel guest migration only!
+ --source For internal use only. Source directory.
+ --target For internal use only. Target directory.
+ ";
+
+#[derive(Debug)]
+struct Args {
+ force: bool,
+ threads: Option<usize>,
+ test: bool,
+ source: Option<PathBuf>,
+ target: Option<PathBuf>,
+}
+
+fn parse_args() -> Result<Args, Error> {
+ let mut pargs = pico_args::Arguments::from_env();
+
+ // Help has a higher priority and should be handled separately.
+ if pargs.contains(["-h", "--help"]) {
+ print!("{}", HELP);
+ std::process::exit(0);
+ }
+
+ let mut args = Args {
+ threads: pargs.opt_value_from_str("--threads").unwrap(),
+ force: false,
+ test: false,
+ source: pargs.opt_value_from_str("--source").unwrap(),
+ target: pargs.opt_value_from_str("--target").unwrap(),
+ };
+
+ if pargs.contains("--test") {
+ args.test = true;
+ }
+ if pargs.contains("--force") {
+ args.force = true;
+ }
+
+ // It's up to the caller what to do with the remaining arguments.
+ let remaining = pargs.finish();
+ if !remaining.is_empty() {
+ bail!(format!("Warning: unused arguments left: {:?}", remaining));
+ }
+
+ Ok(args)
+}
+
+fn main() {
+ let args = match parse_args() {
+ Ok(v) => v,
+ Err(e) => {
+ eprintln!("Error: {}.", e);
+ std::process::exit(1);
+ }
+ };
+
+ let mut source_dir_guests: PathBuf = [BASE_DIR, SOURCE_SUBDIR_GUEST].iter().collect();
+ let mut target_dir_guests: PathBuf = [BASE_DIR, TARGET_SUBDIR_GUEST].iter().collect();
+ let source_dir_nodes: PathBuf = [BASE_DIR, SOURCE_SUBDIR_NODE].iter().collect();
+ let target_dir_nodes: PathBuf = [BASE_DIR, TARGET_SUBDIR_NODE].iter().collect();
+ let source_dir_storage: PathBuf = [BASE_DIR, SOURCE_SUBDIR_STORAGE].iter().collect();
+ let target_dir_storage: PathBuf = [BASE_DIR, TARGET_SUBDIR_STORAGE].iter().collect();
+
+ if args.test {
+ source_dir_guests = args.source.clone().unwrap();
+ target_dir_guests = args.target.clone().unwrap();
+ }
+
+ if !args.force && target_dir_guests.exists() {
+ eprintln!(
+ "Aborting! Target path for guests already exists. Use '--force' to still migrate. It will overwrite existing files!"
+ );
+ std::process::exit(1);
+ }
+ if !args.force && target_dir_nodes.exists() {
+ eprintln!(
+ "Aborting! Target path for nodes already exists. Use '--force' to still migrate. It will overwrite existing files!"
+ );
+ std::process::exit(1);
+ }
+ if !args.force && target_dir_storage.exists() {
+ eprintln!(
+ "Aborting! Target path for storages already exists. Use '--force' to still migrate. It will overwrite existing files!"
+ );
+ std::process::exit(1);
+ }
+
+ if !args.test {
+ if let Err(e) = migrate_nodes(source_dir_nodes, target_dir_nodes) {
+ eprintln!("Error migrating nodes: {}", e);
+ std::process::exit(1);
+ }
+ if let Err(e) = migrate_storage(source_dir_storage, target_dir_storage) {
+ eprintln!("Error migrating storage: {}", e);
+ std::process::exit(1);
+ }
+ }
+ if let Err(e) = migrate_guests(source_dir_guests, target_dir_guests, set_threads(&args)) {
+ eprintln!("Error migrating guests: {}", e);
+ std::process::exit(1);
+ }
+}
+
+/// Set number of threads
+///
+/// Either a fixed parameter or determining a range between 1 to 4 threads
+/// based on the number of CPU cores available in the system.
+fn set_threads(args: &Args) -> usize {
+ if args.threads.is_some() {
+ return args.threads.unwrap();
+ }
+ // check for a way to get physical cores and not threads?
+ let cpus: usize = String::from_utf8_lossy(
+ std::process::Command::new("nproc")
+ .output()
+ .expect("Error running nproc")
+ .stdout
+ .as_slice()
+ .trim_ascii(),
+ )
+ .parse::<usize>()
+ .expect("Could not parse nproc output");
+
+ if cpus < 32 {
+ let threads = cpus / 8;
+ if threads == 0 {
+ return 1;
+ }
+ return threads;
+ }
+ return MAX_THREADS;
+}
+
+/// Migrate guest RRD files
+///
+/// In parallel to speed up the process as most time is spent on converting the
+/// data to the new format.
+fn migrate_guests(
+ source_dir_guests: PathBuf,
+ target_dir_guests: PathBuf,
+ threads: usize,
+) -> Result<(), Error> {
+ println!("Migrating RRD data for guests…");
+ println!("Using {} thread(s)", threads);
+
+ let mut guest_source_files: Vec<(CString, OsString)> = Vec::new();
+
+ fs::read_dir(&source_dir_guests)?
+ .filter(|f| f.is_ok())
+ .map(|f| f.unwrap().path())
+ .filter(|f| f.is_file())
+ .for_each(|file| {
+ let path = CString::new(file.as_path().as_os_str().as_bytes())
+ .expect("Could not convert path to CString.");
+ let fname = file
+ .file_name()
+ .map(|v| v.to_os_string())
+ .expect("Could not convert fname to OsString.");
+ guest_source_files.push((path, fname))
+ });
+ if !target_dir_guests.exists() {
+ println!("Creating new directory: '{}'", target_dir_guests.display());
+ std::fs::create_dir(&target_dir_guests)?;
+ }
+
+ let total_guests = guest_source_files.len();
+ let guests = Arc::new(std::sync::atomic::AtomicUsize::new(0));
+ let guests2 = guests.clone();
+ let start_time = std::time::SystemTime::now();
+
+ let migration_pool = ParallelHandler::new(
+ "guest rrd migration",
+ threads,
+ move |(path, fname): (CString, OsString)| {
+ let mut source: [*const i8; 2] = [std::ptr::null(); 2];
+ source[0] = path.as_ptr();
+
+ let node_name = fname;
+ let mut target_path = target_dir_guests.clone();
+ target_path.push(node_name);
+
+ let target_path = CString::new(target_path.to_str().unwrap()).unwrap();
+
+ unsafe {
+ rrd_get_context();
+ rrd_clear_error();
+ let res = rrd_create_r2(
+ target_path.as_ptr(),
+ RRD_STEP_SIZE as u64,
+ 0,
+ 0,
+ source.as_mut_ptr(),
+ std::ptr::null(),
+ RRD_VM_DEF.len() as i32,
+ RRD_VM_DEF.map(|v| v.as_ptr()).as_mut_ptr(),
+ );
+ if res != 0 {
+ bail!(
+ "RRD create Error: {}",
+ CStr::from_ptr(rrd_get_error()).to_string_lossy()
+ );
+ }
+ }
+ let current_guests = guests2.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
+ if current_guests > 0 && current_guests % 200 == 0 {
+ println!("Migrated {} of {} guests", current_guests, total_guests);
+ }
+ Ok(())
+ },
+ );
+ let migration_channel = migration_pool.channel();
+
+ for file in guest_source_files {
+ let migration_channel = migration_channel.clone();
+ migration_channel.send(file)?;
+ }
+
+ drop(migration_channel);
+ migration_pool.complete()?;
+
+ let elapsed = start_time.elapsed()?.as_secs_f64();
+ let guests = guests.load(std::sync::atomic::Ordering::SeqCst);
+ println!("Migrated {} guests in {:.2}s", guests, elapsed,);
+
+ Ok(())
+}
+
+/// Migrate node RRD files
+///
+/// In serial as the number of nodes will not be high.
+fn migrate_nodes(source_dir_nodes: PathBuf, target_dir_nodes: PathBuf) -> Result<(), Error> {
+ println!("Migrating RRD data for nodes…");
+
+ if !target_dir_nodes.exists() {
+ println!("Creating new directory: '{}'", target_dir_nodes.display());
+ std::fs::create_dir(&target_dir_nodes)?;
+ }
+
+ let mut node_source_files: Vec<(CString, OsString)> = Vec::new();
+ fs::read_dir(&source_dir_nodes)?
+ .filter(|f| f.is_ok())
+ .map(|f| f.unwrap().path())
+ .filter(|f| f.is_file())
+ .for_each(|file| {
+ let path = CString::new(file.as_path().as_os_str().as_bytes())
+ .expect("Could not convert path to CString.");
+ let fname = file
+ .file_name()
+ .map(|v| v.to_os_string())
+ .expect("Could not convert fname to OsString.");
+ node_source_files.push((path, fname))
+ });
+
+ for file in node_source_files {
+ println!("Node: '{}'", PathBuf::from(file.1.clone()).display());
+ let mut source: [*const i8; 2] = [std::ptr::null(); 2];
+
+ source[0] = file.0.as_ptr();
+
+ let node_name = file.1;
+ let mut target_path = target_dir_nodes.clone();
+ target_path.push(node_name);
+
+ let target_path = CString::new(target_path.to_str().unwrap()).unwrap();
+
+ unsafe {
+ rrd_get_context();
+ rrd_clear_error();
+ let res = rrd_create_r2(
+ target_path.as_ptr(),
+ RRD_STEP_SIZE as u64,
+ 0,
+ 0,
+ source.as_mut_ptr(),
+ std::ptr::null(),
+ RRD_NODE_DEF.len() as i32,
+ RRD_NODE_DEF.map(|v| v.as_ptr()).as_mut_ptr(),
+ );
+ if res != 0 {
+ bail!(
+ "RRD create Error: {}",
+ CStr::from_ptr(rrd_get_error()).to_string_lossy()
+ );
+ }
+ }
+ }
+ println!("Migrated all nodes");
+
+ Ok(())
+}
+
+/// Migrate storage RRD files
+///
+/// In serial as the number of storage will not be that high.
+fn migrate_storage(source_dir_storage: PathBuf, target_dir_storage: PathBuf) -> Result<(), Error> {
+ println!("Migrating RRD data for storages…");
+
+ if !target_dir_storage.exists() {
+ println!("Creating new directory: '{}'", target_dir_storage.display());
+ std::fs::create_dir(&target_dir_storage)?;
+ }
+
+ // storage has another layer of directories per node over which we need to iterate
+ fs::read_dir(&source_dir_storage)?
+ .filter(|f| f.is_ok())
+ .map(|f| f.unwrap().path())
+ .filter(|f| f.is_dir())
+ .try_for_each(|node| {
+ let mut storage_source_files: Vec<(CString, OsString)> = Vec::new();
+
+ let mut source_node_subdir = source_dir_storage.clone();
+ source_node_subdir.push(&node.file_name().unwrap());
+
+ let mut target_node_subdir = target_dir_storage.clone();
+ target_node_subdir.push(&node.file_name().unwrap());
+
+ fs::create_dir(target_node_subdir.as_path())?;
+ let metadata = target_node_subdir.metadata()?;
+ let mut permissions = metadata.permissions();
+ permissions.set_mode(0o755);
+
+ fs::read_dir(&source_node_subdir)?
+ .filter(|f| f.is_ok())
+ .map(|f| f.unwrap().path())
+ .filter(|f| f.is_file())
+ .for_each(|file| {
+ let path = CString::new(file.as_path().as_os_str().as_bytes())
+ .expect("Could not convert path to CString.");
+ let fname = file
+ .file_name()
+ .map(|v| v.to_os_string())
+ .expect("Could not convert fname to OsString.");
+ storage_source_files.push((path, fname))
+ });
+
+ for file in storage_source_files {
+ println!("Storage: '{}'", PathBuf::from(file.1.clone()).display());
+ let mut source: [*const i8; 2] = [std::ptr::null(); 2];
+
+ source[0] = file.0.as_ptr();
+
+ let node_name = file.1;
+ let mut target_path = target_node_subdir.clone();
+ target_path.push(node_name);
+
+ let target_path = CString::new(target_path.to_str().unwrap()).unwrap();
+
+ unsafe {
+ rrd_get_context();
+ rrd_clear_error();
+ let res = rrd_create_r2(
+ target_path.as_ptr(),
+ RRD_STEP_SIZE as u64,
+ 0,
+ 0,
+ source.as_mut_ptr(),
+ std::ptr::null(),
+ RRD_STORAGE_DEF.len() as i32,
+ RRD_STORAGE_DEF.map(|v| v.as_ptr()).as_mut_ptr(),
+ );
+ if res != 0 {
+ bail!(
+ "RRD create Error: {}",
+ CStr::from_ptr(rrd_get_error()).to_string_lossy()
+ );
+ }
+ }
+ }
+ Ok(())
+ })?;
+ println!("Migrated all nodes");
+
+ Ok(())
+}
diff --git a/src/parallel_handler.rs b/src/parallel_handler.rs
new file mode 100644
index 0000000..787742a
--- /dev/null
+++ b/src/parallel_handler.rs
@@ -0,0 +1,162 @@
+//! A thread pool which run a closure in parallel.
+
+use std::sync::{Arc, Mutex};
+use std::thread::JoinHandle;
+
+use anyhow::{Error, bail, format_err};
+use crossbeam_channel::{Sender, bounded};
+
+/// A handle to send data to the worker thread (implements clone)
+pub struct SendHandle<I> {
+ input: Sender<I>,
+ abort: Arc<Mutex<Option<String>>>,
+}
+
+/// Returns the first error happened, if any
+pub fn check_abort(abort: &Mutex<Option<String>>) -> Result<(), Error> {
+ let guard = abort.lock().unwrap();
+ if let Some(err_msg) = &*guard {
+ return Err(format_err!("{}", err_msg));
+ }
+ Ok(())
+}
+
+impl<I: Send> SendHandle<I> {
+ /// Send data to the worker threads
+ pub fn send(&self, input: I) -> Result<(), Error> {
+ check_abort(&self.abort)?;
+ match self.input.send(input) {
+ Ok(()) => Ok(()),
+ Err(_) => bail!("send failed - channel closed"),
+ }
+ }
+}
+
+/// A thread pool which run the supplied closure
+///
+/// The send command sends data to the worker threads. If one handler
+/// returns an error, we mark the channel as failed and it is no
+/// longer possible to send data.
+///
+/// When done, the 'complete()' method needs to be called to check for
+/// outstanding errors.
+pub struct ParallelHandler<I> {
+ handles: Vec<JoinHandle<()>>,
+ name: String,
+ input: Option<SendHandle<I>>,
+}
+
+impl<I> Clone for SendHandle<I> {
+ fn clone(&self) -> Self {
+ Self {
+ input: self.input.clone(),
+ abort: Arc::clone(&self.abort),
+ }
+ }
+}
+
+impl<I: Send + 'static> ParallelHandler<I> {
+ /// Create a new thread pool, each thread processing incoming data
+ /// with 'handler_fn'.
+ pub fn new<F>(name: &str, threads: usize, handler_fn: F) -> Self
+ where
+ F: Fn(I) -> Result<(), Error> + Send + Clone + 'static,
+ {
+ let mut handles = Vec::new();
+ let (input_tx, input_rx) = bounded::<I>(threads);
+
+ let abort = Arc::new(Mutex::new(None));
+
+ for i in 0..threads {
+ let input_rx = input_rx.clone();
+ let abort = Arc::clone(&abort);
+ let handler_fn = handler_fn.clone();
+
+ handles.push(
+ std::thread::Builder::new()
+ .name(format!("{} ({})", name, i))
+ .spawn(move || {
+ loop {
+ let data = match input_rx.recv() {
+ Ok(data) => data,
+ Err(_) => return,
+ };
+ if let Err(err) = (handler_fn)(data) {
+ let mut guard = abort.lock().unwrap();
+ if guard.is_none() {
+ *guard = Some(err.to_string());
+ }
+ }
+ }
+ })
+ .unwrap(),
+ );
+ }
+ Self {
+ handles,
+ name: name.to_string(),
+ input: Some(SendHandle {
+ input: input_tx,
+ abort,
+ }),
+ }
+ }
+
+ /// Returns a cloneable channel to send data to the worker threads
+ pub fn channel(&self) -> SendHandle<I> {
+ self.input.as_ref().unwrap().clone()
+ }
+
+ /// Send data to the worker threads
+ pub fn send(&self, input: I) -> Result<(), Error> {
+ self.input.as_ref().unwrap().send(input)?;
+ Ok(())
+ }
+
+ /// Wait for worker threads to complete and check for errors
+ pub fn complete(mut self) -> Result<(), Error> {
+ let input = self.input.take().unwrap();
+ let abort = Arc::clone(&input.abort);
+ check_abort(&abort)?;
+ drop(input);
+
+ let msg_list = self.join_threads();
+
+ // an error might be encountered while waiting for the join
+ check_abort(&abort)?;
+
+ if msg_list.is_empty() {
+ return Ok(());
+ }
+ Err(format_err!("{}", msg_list.join("\n")))
+ }
+
+ fn join_threads(&mut self) -> Vec<String> {
+ let mut msg_list = Vec::new();
+
+ let mut i = 0;
+ while let Some(handle) = self.handles.pop() {
+ if let Err(panic) = handle.join() {
+ if let Some(panic_msg) = panic.downcast_ref::<&str>() {
+ msg_list.push(format!("thread {} ({i}) panicked: {panic_msg}", self.name));
+ } else if let Some(panic_msg) = panic.downcast_ref::<String>() {
+ msg_list.push(format!("thread {} ({i}) panicked: {panic_msg}", self.name));
+ } else {
+ msg_list.push(format!("thread {} ({i}) panicked", self.name));
+ }
+ }
+ i += 1;
+ }
+ msg_list
+ }
+}
+
+// Note: We make sure that all threads will be joined
+impl<I> Drop for ParallelHandler<I> {
+ fn drop(&mut self) {
+ drop(self.input.take());
+ while let Some(handle) = self.handles.pop() {
+ let _ = handle.join();
+ }
+ }
+}
diff --git a/wrapper.h b/wrapper.h
new file mode 100644
index 0000000..64d0aa6
--- /dev/null
+++ b/wrapper.h
@@ -0,0 +1 @@
+#include <rrd.h>
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH cluster v2 1/4] cfs status.c: drop old pve2-vm rrd schema support
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (4 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH pve9-rrd-migration-tool v2 1/1] introduce rrd migration tool for pve8 -> pve9 Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH cluster v2 2/4] status: handle new metrics update data Aaron Lauterer
` (17 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
the newer pve2.3-vm schema has been introduced with commit ba9dcfc1 back
in 2013. By now there should be no cluster where an older node might
still send the old pve2-vm schema.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/pmxcfs/status.c | 12 +++---------
1 file changed, 3 insertions(+), 9 deletions(-)
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index 0895e53..eba4e52 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -1234,16 +1234,10 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
create_rrd_file(filename, argcount, rrd_def_node);
}
- } else if ((strncmp(key, "pve2-vm/", 8) == 0) || (strncmp(key, "pve2.3-vm/", 10) == 0)) {
- const char *vmid;
+ } else if (strncmp(key, "pve2.3-vm/", 10) == 0) {
+ const char *vmid = key + 10;
- if (strncmp(key, "pve2-vm/", 8) == 0) {
- vmid = key + 8;
- skip = 2;
- } else {
- vmid = key + 10;
- skip = 4;
- }
+ skip = 4;
if (strchr(vmid, '/') != NULL) {
goto keyerror;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH cluster v2 2/4] status: handle new metrics update data
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (5 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH cluster v2 1/4] cfs status.c: drop old pve2-vm rrd schema support Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH cluster v2 3/4] status: introduce new pve-{type}- rrd and metric format Aaron Lauterer
` (16 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
For PVE9 we plan to add additional fields in the metrics that are
collected and distributed in the cluster. The new fields/columns are
added at the end of the current ones. This makes it possible for PVE8
installations to still use them by cutting the new additional data.
To make it more future proof, the format of the keys for each metrics
are changed:
Old: pve{version}-{type}/{id}
New: pve-{type}-{version}/{id}
This way we have an easier time to handle new versions in the future as
we initially only need to check for `pve-{type}-`. If we know the
version, we can handle it accordingly; e.g. pad if older format with
missing data. If we don't know the version, it must be a newer one and
we cut the data stream at the length we need for the current version.
This means of course that to avoid a breaking change, we can only add
new columns if needed, but not remove any! But waiting for a breaking
change until the next major release is a worthy trade-off if it allows
us to expand the format in between if needed.
Since the full keys were used for the final location within the RRD
directory, we need to change that as well and set it manually to
'pve2-{type}' as the key we receive could be for a newer data format.
The 'rrd_skip_data' function got a new parameter defining the sepataring
character. This then makes it possible to use it to determine which part
of the key string is the version/type and which one is the actual
resource identifier.
We drop the pve2-vm schema as the newer pve2.3-vm has been introduced
with commit ba9dcfc1 back in 2013. By now there should be no cluster
where an older node might still send the old pve2-vm schema.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/pmxcfs/status.c | 79 ++++++++++++++++++++++++++++++---------------
1 file changed, 53 insertions(+), 26 deletions(-)
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index eba4e52..640540f 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -1185,16 +1185,33 @@ static void create_rrd_file(const char *filename, int argcount, const char *rrdd
}
}
-static inline const char *rrd_skip_data(const char *data, int count) {
+static inline const char *rrd_skip_data(const char *data, int count, char separator) {
int found = 0;
while (*data && found < count) {
- if (*data++ == ':') {
+ if (*data++ == separator) {
found++;
}
}
return data;
}
+// The key and subdirectory format used up until PVE8 is 'pve{version}-{type}/{id}' with version
+// being 2 or 2.3 for VMs. Starting with PVE9 'pve-{type}-{version}/{id}'. Newer versions are only
+// allowed to append new columns to the data! Otherwise this would be a breaking change.
+//
+// Type can be: node, vm, storage
+//
+// Version is the version of PVE with which it was introduced, e.g.: 9.0, 9.2, 10.0.
+//
+// ID is the actual identifier of the item in question. E.g. node name, VMID or for storage it is
+// '{node}/{storage name}'
+//
+// This way, we can handle unknown new formats gracefully and cut the data at the expected
+// column for the currently understood format. Receiving older formats will still need special
+// checks to determine how much padding is needed.
+//
+// Should we ever plan to change existing columns, we need to introduce this as a breaking
+// change!
static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
g_return_if_fail(key != NULL);
g_return_if_fail(data != NULL);
@@ -1210,12 +1227,13 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
char *filename = NULL;
- int skip = 0;
+ int skip = 0; // columns to skip at beginning. They contain non-archivable data, like uptime,
+ // status, is guest a template and such.
+ int keep_columns = 0; // how many columns do we want to keep (after initial skip) in case we get
+ // more columns than needed from a newer format
- if (strncmp(key, "pve2-node/", 10) == 0) {
- const char *node = key + 10;
-
- skip = 2;
+ if (strncmp(key, "pve2-node/", 10) == 0 || strncmp(key, "pve-node-", 9) == 0) {
+ const char *node = rrd_skip_data(key, 1, '/');
if (strchr(node, '/') != NULL) {
goto keyerror;
@@ -1225,19 +1243,23 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
- filename = g_strdup_printf(RRDDIR "/%s", key);
+ skip = 2; // first two columns are live data that isn't archived
- if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
+ if (strncmp(key, "pve-node-", 9) == 0) {
+ keep_columns = 12; // pve2-node format uses 12 columns
+ }
+ filename = g_strdup_printf(RRDDIR "/pve2-node/%s", node);
+
+ if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
mkdir(RRDDIR "/pve2-node", 0755);
int argcount = sizeof(rrd_def_node) / sizeof(void *) - 1;
create_rrd_file(filename, argcount, rrd_def_node);
}
- } else if (strncmp(key, "pve2.3-vm/", 10) == 0) {
- const char *vmid = key + 10;
+ } else if (strncmp(key, "pve2.3-vm/", 10) == 0 || strncmp(key, "pve-vm-", 7) == 0) {
- skip = 4;
+ const char *vmid = rrd_skip_data(key, 1, '/');
if (strchr(vmid, '/') != NULL) {
goto keyerror;
@@ -1247,29 +1269,29 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
+ skip = 4; // first 4 columns are live data that isn't archived
+
+ if (strncmp(key, "pve-vm-", 7) == 0) {
+ keep_columns = 10; // pve2.3-vm format uses 10 data columns
+ }
+
filename = g_strdup_printf(RRDDIR "/%s/%s", "pve2-vm", vmid);
if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
-
mkdir(RRDDIR "/pve2-vm", 0755);
int argcount = sizeof(rrd_def_vm) / sizeof(void *) - 1;
create_rrd_file(filename, argcount, rrd_def_vm);
}
- } else if (strncmp(key, "pve2-storage/", 13) == 0) {
- const char *node = key + 13;
+ } else if (strncmp(key, "pve2-storage/", 13) == 0 || strncmp(key, "pve-storage-", 12) == 0) {
+ const char *node = rrd_skip_data(key, 1, '/'); // will contain {node}/{storage}
- const char *storage = node;
- while (*storage && *storage != '/') {
- storage++;
- }
+ const char *storage = rrd_skip_data(node, 1, '/');
- if (*storage != '/' || ((storage - node) < 1)) {
+ if ((storage - node) < 1) {
goto keyerror;
}
- storage++;
-
if (strchr(storage, '/') != NULL) {
goto keyerror;
}
@@ -1278,12 +1300,10 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
- filename = g_strdup_printf(RRDDIR "/%s", key);
+ filename = g_strdup_printf(RRDDIR "/pve2-storage/%s", node);
if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
-
mkdir(RRDDIR "/pve2-storage", 0755);
-
char *dir = g_path_get_dirname(filename);
mkdir(dir, 0755);
g_free(dir);
@@ -1296,7 +1316,14 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
- const char *dp = skip ? rrd_skip_data(data, skip) : data;
+ const char *dp = skip ? rrd_skip_data(data, skip, ':') : data;
+
+ if (keep_columns) {
+ keep_columns++; // We specify the number of columns we want earlier, but we also have the
+ // always present timestamp column, so we need to skip one more column
+ char *cut = (char *)rrd_skip_data(dp, keep_columns, ':');
+ *(cut - 1) = 0; // terminate string by replacing colon from field separator with zero.
+ }
const char *update_args[] = {dp, NULL};
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH cluster v2 3/4] status: introduce new pve-{type}- rrd and metric format
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (6 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH cluster v2 2/4] status: handle new metrics update data Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH cluster v2 4/4] rrd: adapt to new RRD format with different aggregation windows Aaron Lauterer
` (15 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
With PVE9 now we have additional fields in the metrics that are
collected and distributed in the cluster. The new fields/columns are
added at the end of the existing ones. This makes it possible for PVE8
installations to still use them by cutting the new additional data.
To make it more future proof, the format of the keys for each metrics
are now changed:
Old pre PVE9: pve{version}-{type}/{id}
Now with PVE9: pve-{type}-{version}/{id}
This way we have an easier time to handle new versions in the future as
we initially only need to check for `pve-{type}-`. If we know the
version, we can handle it accordingly; e.g. pad if older format with
missing data. If we don't know the version, it must be a newer one and
we cut the data stream at the length we need for the current version.
This means of course that to avoid a breaking change, we can only add
new columns if needed, but not remove any! But waiting for a breaking
change until the next major release is a worthy trade-off if it allows
us to expand the format in between if needed.
The 'rrd_skip_data' function got a new parameter defining the sepataring
character. This then makes it possible to use it also to determine which
part of the key string is the version/type and which one is the actual
resource identifier.
We add several new columns to nodes and VMs (guest) RRDs. See futher
down for details. Additionally we change the RRA definitions on how we
aggregate the data to match how we do it for the Proxmox Backup Server
[0].
The migration of an existing installation is handled by a dedicated
tool. Only once that has happened, will we store data in the new
format.
This leaves us with a few cases to handle:
data recv → old new
↓ rrd files
-------------|---------------------------|-------------------------------------
none | check if directories exists:
| neither old or new -> new
| new -> new
| old only -> old
--------------|---------------------------|-------------------------------------
only old | use old file as is | cut new columns and use old file
--------------|---------------------------|-------------------------------------
new present | pad data to match new fmt | use new file as is and pass data
To handle the padding we use a buffer. Cutting can be handled as we
already do it in the stable/bookworm (PVE8) branch by introducing a null
terminator in the original string at the end of the expected columns.
We add the following new columns:
Nodes:
* memfree
* arcsize
* pressures:
* cpu some
* io some
* io full
* mem some
* mem full
VMs:
* memhost (memory consumption of all processes in the guests cgroup, host view)
* pressures:
* cpu some
* cpu full
* io some
* io full
* mem some
* mem full
[0] https://git.proxmox.com/?p=proxmox-backup.git;a=blob;f=src/server/metric_collection/rrd.rs;h=ed39cc94ee056924b7adbc21b84c0209478bcf42;hb=dc324716a688a67d700fa133725740ac5d3795ce#l76
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/pmxcfs/status.c | 261 +++++++++++++++++++++++++++++++++++++++-----
1 file changed, 236 insertions(+), 25 deletions(-)
diff --git a/src/pmxcfs/status.c b/src/pmxcfs/status.c
index 640540f..5ecda12 100644
--- a/src/pmxcfs/status.c
+++ b/src/pmxcfs/status.c
@@ -1096,6 +1096,9 @@ kventry_hash_set(GHashTable *kvhash, const char *key, gconstpointer data, size_t
return TRUE;
}
+// We create the RRD files with a 60 second stepsize, therefore, RRA timesteps
+// are alwys per 60 seconds. These 60 seconds are usually showing up in other
+// code paths where we interact with RRD data!
static const char *rrd_def_node[] = {
"DS:loadavg:GAUGE:120:0:U",
"DS:maxcpu:GAUGE:120:0:U",
@@ -1124,6 +1127,39 @@ static const char *rrd_def_node[] = {
NULL,
};
+static const char *rrd_def_node_pve9_0[] = {
+ "DS:loadavg:GAUGE:120:0:U",
+ "DS:maxcpu:GAUGE:120:0:U",
+ "DS:cpu:GAUGE:120:0:U",
+ "DS:iowait:GAUGE:120:0:U",
+ "DS:memtotal:GAUGE:120:0:U",
+ "DS:memused:GAUGE:120:0:U",
+ "DS:swaptotal:GAUGE:120:0:U",
+ "DS:swapused:GAUGE:120:0:U",
+ "DS:roottotal:GAUGE:120:0:U",
+ "DS:rootused:GAUGE:120:0:U",
+ "DS:netin:DERIVE:120:0:U",
+ "DS:netout:DERIVE:120:0:U",
+ "DS:memfree:GAUGE:120:0:U",
+ "DS:arcsize:GAUGE:120:0:U",
+ "DS:pressurecpusome:GAUGE:120:0:U",
+ "DS:pressureiosome:GAUGE:120:0:U",
+ "DS:pressureiofull:GAUGE:120:0:U",
+ "DS:pressurememorysome:GAUGE:120:0:U",
+ "DS:pressurememoryfull:GAUGE:120:0:U",
+
+ "RRA:AVERAGE:0.5:1:1440", // 1 min * 1440 => 1 day
+ "RRA:AVERAGE:0.5:30:1440", // 30 min * 1440 => 30 day
+ "RRA:AVERAGE:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ "RRA:AVERAGE:0.5:10080:570", // 1 week * 570 => ~10 years
+
+ "RRA:MAX:0.5:1:1440", // 1 min * 1440 => 1 day
+ "RRA:MAX:0.5:30:1440", // 30 min * 1440 => 30 day
+ "RRA:MAX:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ "RRA:MAX:0.5:10080:570", // 1 week * 570 => ~10 years
+ NULL,
+};
+
static const char *rrd_def_vm[] = {
"DS:maxcpu:GAUGE:120:0:U",
"DS:cpu:GAUGE:120:0:U",
@@ -1149,6 +1185,36 @@ static const char *rrd_def_vm[] = {
"RRA:MAX:0.5:10080:70", // 7 day max - ony year
NULL,
};
+static const char *rrd_def_vm_pve9_0[] = {
+ "DS:maxcpu:GAUGE:120:0:U",
+ "DS:cpu:GAUGE:120:0:U",
+ "DS:maxmem:GAUGE:120:0:U",
+ "DS:mem:GAUGE:120:0:U",
+ "DS:maxdisk:GAUGE:120:0:U",
+ "DS:disk:GAUGE:120:0:U",
+ "DS:netin:DERIVE:120:0:U",
+ "DS:netout:DERIVE:120:0:U",
+ "DS:diskread:DERIVE:120:0:U",
+ "DS:diskwrite:DERIVE:120:0:U",
+ "DS:memhost:GAUGE:120:0:U",
+ "DS:pressurecpusome:GAUGE:120:0:U",
+ "DS:pressurecpufull:GAUGE:120:0:U",
+ "DS:pressureiosome:GAUGE:120:0:U",
+ "DS:pressureiofull:GAUGE:120:0:U",
+ "DS:pressurememorysome:GAUGE:120:0:U",
+ "DS:pressurememoryfull:GAUGE:120:0:U",
+
+ "RRA:AVERAGE:0.5:1:1440", // 1 min * 1440 => 1 day
+ "RRA:AVERAGE:0.5:30:1440", // 30 min * 1440 => 30 day
+ "RRA:AVERAGE:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ "RRA:AVERAGE:0.5:10080:570", // 1 week * 570 => ~10 years
+
+ "RRA:MAX:0.5:1:1440", // 1 min * 1440 => 1 day
+ "RRA:MAX:0.5:30:1440", // 30 min * 1440 => 30 day
+ "RRA:MAX:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ "RRA:MAX:0.5:10080:570", // 1 week * 570 => ~10 years
+ NULL,
+};
static const char *rrd_def_storage[] = {
"DS:total:GAUGE:120:0:U",
@@ -1168,8 +1234,30 @@ static const char *rrd_def_storage[] = {
NULL,
};
+static const char *rrd_def_storage_pve9_0[] = {
+ "DS:total:GAUGE:120:0:U",
+ "DS:used:GAUGE:120:0:U",
+
+ "RRA:AVERAGE:0.5:1:1440", // 1 min * 1440 => 1 day
+ "RRA:AVERAGE:0.5:30:1440", // 30 min * 1440 => 30 day
+ "RRA:AVERAGE:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ "RRA:AVERAGE:0.5:10080:570", // 1 week * 570 => ~10 years
+
+ "RRA:MAX:0.5:1:1440", // 1 min * 1440 => 1 day
+ "RRA:MAX:0.5:30:1440", // 30 min * 1440 => 30 day
+ "RRA:MAX:0.5:360:1440", // 6 hours * 1440 => 360 day ~1 year
+ "RRA:MAX:0.5:10080:570", // 1 week * 570 => ~10 years
+ NULL,
+};
+
#define RRDDIR "/var/lib/rrdcached/db"
+// A 4k buffer should be plenty to temporarily store RRD data. 64 bit integers are 20 chars long,
+// plus the separator char: (4096-1)/21~195 columns This buffer is only used in the
+// `update_rrd_data` function. It is safe to use as the calling sites get the global mutex:
+// rrd_update_data -> rrdentry_hash_set -> cfs_status_set / and cfs_kvstore_node_set
+static char rrd_format_update_buffer[4096];
+
static void create_rrd_file(const char *filename, int argcount, const char *rrddef[]) {
/* start at day boundary */
time_t ctime;
@@ -1229,6 +1317,8 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
int skip = 0; // columns to skip at beginning. They contain non-archivable data, like uptime,
// status, is guest a template and such.
+ int padding = 0; // how many columns need to be added with "U" if we get an old format that is
+ // missing columns at the end.
int keep_columns = 0; // how many columns do we want to keep (after initial skip) in case we get
// more columns than needed from a newer format
@@ -1243,20 +1333,60 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
- skip = 2; // first two columns are live data that isn't archived
+ filename = g_strdup_printf(RRDDIR "/pve-node-9.0/%s", node);
+ char *filename_pve2 = g_strdup_printf(RRDDIR "/pve2-node/%s", node);
- if (strncmp(key, "pve-node-", 9) == 0) {
- keep_columns = 12; // pve2-node format uses 12 columns
+ int use_pve2_file = 0;
+
+ // check existing rrd files and directories
+ if (g_file_test(filename, G_FILE_TEST_EXISTS)) {
+ // pve-node-9.0 file exists, we use that
+ // TODO: get conditions so, that we do not have this empty branch
+ } else if (g_file_test(filename_pve2, G_FILE_TEST_EXISTS)) {
+ // old file exists, use it
+ use_pve2_file = 1;
+ filename = g_strdup_printf("%s", filename_pve2);
+ } else {
+ // neither file exists, check for directories to decide and create file
+ char *dir_pve2 = g_strdup_printf(RRDDIR "/pve2-node");
+ char *dir_pve90 = g_strdup_printf(RRDDIR "/pve-node-9.0");
+
+ if (g_file_test(dir_pve90, G_FILE_TEST_IS_DIR)) {
+
+ int argcount = sizeof(rrd_def_node_pve9_0) / sizeof(void *) - 1;
+ create_rrd_file(filename, argcount, rrd_def_node_pve9_0);
+ } else if (g_file_test(dir_pve2, G_FILE_TEST_IS_DIR)) {
+ use_pve2_file = 1;
+
+ filename = g_strdup_printf("%s", filename_pve2);
+
+ int argcount = sizeof(rrd_def_node) / sizeof(void *) - 1;
+ create_rrd_file(filename, argcount, rrd_def_node);
+ } else {
+ // no dir exists yet, use new pve-node-9.0
+ mkdir(RRDDIR "/pve-node-9.0", 0755);
+
+ int argcount = sizeof(rrd_def_node_pve9_0) / sizeof(void *) - 1;
+ create_rrd_file(filename, argcount, rrd_def_node_pve9_0);
+ }
+ g_free(dir_pve2);
+ g_free(dir_pve90);
}
- filename = g_strdup_printf(RRDDIR "/pve2-node/%s", node);
+ skip = 2; // first two columns are live data that isn't archived
- if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
- mkdir(RRDDIR "/pve2-node", 0755);
- int argcount = sizeof(rrd_def_node) / sizeof(void *) - 1;
- create_rrd_file(filename, argcount, rrd_def_node);
+ if (strncmp(key, "pve2-node/", 10) == 0 && !use_pve2_file) {
+ padding = 7; // pve-node-9.0 has 7 more columns than pve2-node
+ } else if (strncmp(key, "pve-node-", 9) == 0 && use_pve2_file) {
+ keep_columns = 12; // pve2-node format uses 12 columns
+ } else if (strncmp(key, "pve-node-9.0/", 13) != 0) {
+ // we received an unknown format, expectation is it is newer and has more columns
+ // than we can currently handle
+ keep_columns = 19; // pve-node-9.0 format uses 19 columns
}
+ g_free(filename_pve2);
+
} else if (strncmp(key, "pve2.3-vm/", 10) == 0 || strncmp(key, "pve-vm-", 7) == 0) {
const char *vmid = rrd_skip_data(key, 1, '/');
@@ -1269,20 +1399,60 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
- skip = 4; // first 4 columns are live data that isn't archived
+ filename = g_strdup_printf(RRDDIR "/pve-vm-9.0/%s", vmid);
+ char *filename_pve2 = g_strdup_printf(RRDDIR "/%s/%s", "pve2-vm", vmid);
+
+ int use_pve2_file = 0;
+
+ // check existing rrd files and directories
+ if (g_file_test(filename, G_FILE_TEST_EXISTS)) {
+ // pve-vm-9.0 file exists, we use that
+ // TODO: get conditions so, that we do not have this empty branch
+ } else if (g_file_test(filename_pve2, G_FILE_TEST_EXISTS)) {
+ // old file exists, use it
+ use_pve2_file = 1;
+ filename = g_strdup_printf("%s", filename_pve2);
+ } else {
+ // neither file exists, check for directories to decide and create file
+ char *dir_pve2 = g_strdup_printf(RRDDIR "/pve2-vm");
+ char *dir_pve90 = g_strdup_printf(RRDDIR "/pve-vm-9.0");
+
+ if (g_file_test(dir_pve90, G_FILE_TEST_IS_DIR)) {
- if (strncmp(key, "pve-vm-", 7) == 0) {
- keep_columns = 10; // pve2.3-vm format uses 10 data columns
+ int argcount = sizeof(rrd_def_vm_pve9_0) / sizeof(void *) - 1;
+ create_rrd_file(filename, argcount, rrd_def_vm_pve9_0);
+ } else if (g_file_test(dir_pve2, G_FILE_TEST_IS_DIR)) {
+ use_pve2_file = 1;
+
+ filename = g_strdup_printf("%s", filename_pve2);
+
+ int argcount = sizeof(rrd_def_vm) / sizeof(void *) - 1;
+ create_rrd_file(filename, argcount, rrd_def_vm);
+ } else {
+ // no dir exists yet, use new pve-vm-9.0
+ mkdir(RRDDIR "/pve-vm-9.0", 0755);
+
+ int argcount = sizeof(rrd_def_vm_pve9_0) / sizeof(void *) - 1;
+ create_rrd_file(filename, argcount, rrd_def_vm_pve9_0);
+ }
+ g_free(dir_pve2);
+ g_free(dir_pve90);
}
- filename = g_strdup_printf(RRDDIR "/%s/%s", "pve2-vm", vmid);
+ skip = 4; // first 4 columns are live data that isn't archived
- if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
- mkdir(RRDDIR "/pve2-vm", 0755);
- int argcount = sizeof(rrd_def_vm) / sizeof(void *) - 1;
- create_rrd_file(filename, argcount, rrd_def_vm);
+ if (strncmp(key, "pve2.3-vm/", 10) == 0 && !use_pve2_file) {
+ padding = 7; // pve-vm-9.0 has 7 more columns than pve2.3-vm
+ } else if (strncmp(key, "pve-vm-", 7) == 0 && use_pve2_file) {
+ keep_columns = 10; // pve2.3-vm format uses 10 columns
+ } else if (strncmp(key, "pve-vm-9.0/", 11) != 0) {
+ // we received an unknown format, expectation is it is newer and has more columns
+ // than we can currently handle
+ keep_columns = 17; // pve-vm-9.0 format uses 19 columns
}
+ g_free(filename_pve2);
+
} else if (strncmp(key, "pve2-storage/", 13) == 0 || strncmp(key, "pve-storage-", 12) == 0) {
const char *node = rrd_skip_data(key, 1, '/'); // will contain {node}/{storage}
@@ -1300,18 +1470,50 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
goto keyerror;
}
- filename = g_strdup_printf(RRDDIR "/pve2-storage/%s", node);
+ filename = g_strdup_printf(RRDDIR "/pve-storage-9.0/%s", node);
+ char *filename_pve2 = g_strdup_printf(RRDDIR "/%s/%s", "pve2-storage", node);
+
+ // check existing rrd files and directories
+ if (g_file_test(filename, G_FILE_TEST_EXISTS)) {
+ // pve-storage-9.0 file exists, we use that
+ // TODO: get conditions so, that we do not have this empty branch
+ } else if (g_file_test(filename_pve2, G_FILE_TEST_EXISTS)) {
+ // old file exists, use it
+ filename = g_strdup_printf("%s", filename_pve2);
+ } else {
+ // neither file exists, check for directories to decide and create file
+ char *dir_pve2 = g_strdup_printf(RRDDIR "/pve2-storage");
+ char *dir_pve90 = g_strdup_printf(RRDDIR "/pve-storage-9.0");
+
+ if (g_file_test(dir_pve90, G_FILE_TEST_IS_DIR)) {
+
+ int argcount = sizeof(rrd_def_storage_pve9_0) / sizeof(void *) - 1;
+ create_rrd_file(filename, argcount, rrd_def_storage_pve9_0);
+ } else if (g_file_test(dir_pve2, G_FILE_TEST_IS_DIR)) {
+ filename = g_strdup_printf("%s", filename_pve2);
- if (!g_file_test(filename, G_FILE_TEST_EXISTS)) {
- mkdir(RRDDIR "/pve2-storage", 0755);
- char *dir = g_path_get_dirname(filename);
- mkdir(dir, 0755);
- g_free(dir);
+ int argcount = sizeof(rrd_def_storage) / sizeof(void *) - 1;
+ create_rrd_file(filename, argcount, rrd_def_storage);
+ } else {
+ // no dir exists yet, use new pve-storage-9.0
+ mkdir(RRDDIR "/pve-storage-9.0", 0755);
- int argcount = sizeof(rrd_def_storage) / sizeof(void *) - 1;
- create_rrd_file(filename, argcount, rrd_def_storage);
+ int argcount = sizeof(rrd_def_storage_pve9_0) / sizeof(void *) - 1;
+ create_rrd_file(filename, argcount, rrd_def_storage_pve9_0);
+ }
+ g_free(dir_pve2);
+ g_free(dir_pve90);
}
+ // actual data columns didn't change between pve2-storage and pve-storage-9.0
+ if (strncmp(key, "pve-storage-", 12) == 0 && strncmp(key, "pve-storage-9.0/", 16) != 0) {
+ // we received an unknown format, expectation is it is newer and has more columns
+ // than we can currently handle
+ keep_columns = 2; // pve-storage-9.0 format uses 2 columns
+ }
+
+ g_free(filename_pve2);
+
} else {
goto keyerror;
}
@@ -1325,7 +1527,16 @@ static void update_rrd_data(const char *key, gconstpointer data, size_t len) {
*(cut - 1) = 0; // terminate string by replacing colon from field separator with zero.
}
- const char *update_args[] = {dp, NULL};
+ const char *update_args[] = {NULL, NULL};
+ if (padding) {
+ // add padding "U" columns to data string
+ char *padsrc =
+ ":U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U:U"; // can pad up to 25 columns
+ g_snprintf(rrd_format_update_buffer, 1024 * 4, "%s%.*s", dp, padding * 2, padsrc);
+ update_args[0] = rrd_format_update_buffer;
+ } else {
+ update_args[0] = dp;
+ }
if (use_daemon) {
int status;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH cluster v2 4/4] rrd: adapt to new RRD format with different aggregation windows
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (7 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH cluster v2 3/4] status: introduce new pve-{type}- rrd and metric format Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH common v2 1/2] fix error in pressure parsing Aaron Lauterer
` (14 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
With PVE9 we introduced a new RRD format that has different aggregation
steps, similar to what we use in the Backup Server.
We therefore need to adapt the functions that get data from RRD
accordingly.
The result is usually a finer resolution for time windows larger than
hourly.
We also introduce decade as a time window. In case existing RRD files
have not yet been converted to the new RRD format, we need keep using
the old time windows. Additionally, since they only store data up to a
year, we catch the situation where a full decade might be requested and
pin it to a year.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/PVE/RRD.pm | 52 ++++++++++++++++++++++++++++++++++++++++----------
1 file changed, 42 insertions(+), 10 deletions(-)
diff --git a/src/PVE/RRD.pm b/src/PVE/RRD.pm
index 93df608..c95f495 100644
--- a/src/PVE/RRD.pm
+++ b/src/PVE/RRD.pm
@@ -14,14 +14,30 @@ sub create_rrd_data {
my $rrd = "$rrddir/$rrdname";
+ # Format: [ resolution, number of data points/count]
+ # Old ranges, pre PVE9
+ my $setup_pve2 = {
+ hour => [60, 60], # 1 min resolution, one hour
+ day => [60 * 30, 70], # 30 min resolution, one day
+ week => [60 * 180, 70], # 3 hour resolution, one week
+ month => [60 * 720, 70], # 12 hour resolution, 1 month
+ year => [60 * 10080, 70], # 7 day resolution, 1 year
+ };
+
my $setup = {
- hour => [60, 70],
- day => [60 * 30, 70],
- week => [60 * 180, 70],
- month => [60 * 720, 70],
- year => [60 * 10080, 70],
+ hour => [60, 60], # 1 min resolution
+ day => [60, 1440], # 1 min resolution, full day
+ week => [60 * 30, 336], # 30 min resolution, 7 days
+ month => [3600 * 6, 121], # 6 hour resolution, 30 days, need one more count. Otherwise RRD gets wrong $step
+ year => [3600 * 6, 1140], # 6 hour resolution, 360 days
+ decade => [86400 * 7, 570], # 1 week resolution, 10 years
};
+ if ($rrdname =~ /^pve2/) {
+ $setup = $setup_pve2;
+ $timeframe = "year" if $timeframe eq "decade"; # we only store up to one year in the old format
+ }
+
my ($reso, $count) = @{ $setup->{$timeframe} };
my $ctime = $reso * int(time() / $reso);
my $req_start = $ctime - $reso * $count;
@@ -82,14 +98,30 @@ sub create_rrd_graph {
my $filename = "${rrd}_${ds_txt}.png";
+ # Format: [ resolution, number of data points/count]
+ # Old ranges, pre PVE9
+ my $setup_pve2 = {
+ hour => [60, 60], # 1 min resolution, one hour
+ day => [60 * 30, 70], # 30 min resolution, one day
+ week => [60 * 180, 70], # 3 hour resolution, one week
+ month => [60 * 720, 70], # 12 hour resolution, 1 month
+ year => [60 * 10080, 70], # 7 day resolution, 1 year
+ };
+
my $setup = {
- hour => [60, 60],
- day => [60 * 30, 70],
- week => [60 * 180, 70],
- month => [60 * 720, 70],
- year => [60 * 10080, 70],
+ hour => [60, 60], # 1 min resolution
+ day => [60, 1440], # 1 min resolution, full day
+ week => [60 * 30, 336], # 30 min resolution, 7 days
+ month => [3600 * 6, 121], # 6 hour resolution, 30 days, need one more count. Otherwise RRD gets wrong $step
+ year => [3600 * 6, 1140], # 6 hour resolution, 360 days
+ decade => [86400 * 7, 570], # 1 week resolution, 10 years
};
+ if ($rrdname =~ /^pve2/) {
+ $setup = $setup_pve2;
+ $timeframe = "year" if $timeframe eq "decade"; # we only store up to one year in the old format
+ }
+
my ($reso, $count) = @{ $setup->{$timeframe} };
my @args = (
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH common v2 1/2] fix error in pressure parsing
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (8 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH cluster v2 4/4] rrd: adapt to new RRD format with different aggregation windows Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH common v2 2/2] add functions to retrieve pressures for vm/ct Aaron Lauterer
` (13 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
From: Folke Gleumes <f.gleumes@proxmox.com>
Originally-by: Folke Gleumes <f.gleumes@proxmox.com>
[AL: rebased]
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/PVE/ProcFSTools.pm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/src/PVE/ProcFSTools.pm b/src/PVE/ProcFSTools.pm
index b67211e..f342890 100644
--- a/src/PVE/ProcFSTools.pm
+++ b/src/PVE/ProcFSTools.pm
@@ -144,7 +144,7 @@ sub parse_pressure {
$res->{$1}->{avg10} = $2;
$res->{$1}->{avg60} = $3;
$res->{$1}->{avg300} = $4;
- $res->{$1}->{total} = $4;
+ $res->{$1}->{total} = $5;
}
}
$fh->close;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH common v2 2/2] add functions to retrieve pressures for vm/ct
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (9 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH common v2 1/2] fix error in pressure parsing Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 1/5] api2tools: drop old VM rrd schema Aaron Lauterer
` (12 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
From: Folke Gleumes <f.gleumes@proxmox.com>
Originally-by: Folke Gleumes <f.gleumes@proxmox.com>
[AL:
rebased on current master
merged into single function for generic cgroups
]
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
while the read_cgroup_pressure function would fit better into
SysFSTools.pm I have kept it in ProcFSTools.pm for now, mainly because
we currently don't use ProcFSTools in SysFSTools, and the actual
function to parse the contents of the pressure files in in ProcFSTools.
If that would not be an issue, we could also move the
read_cgroup_pressure function to SysFSTools
changes since
RFC:
* instead of dedicated functions for CTs and VMs we use a more generic
for cgroups in general
src/PVE/ProcFSTools.pm | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/src/PVE/ProcFSTools.pm b/src/PVE/ProcFSTools.pm
index f342890..f28372b 100644
--- a/src/PVE/ProcFSTools.pm
+++ b/src/PVE/ProcFSTools.pm
@@ -151,6 +151,17 @@ sub parse_pressure {
return $res;
}
+sub read_cgroup_pressure {
+ my ($cgroup_path) = @_;
+
+ my $res = {};
+ for my $type (qw(cpu memory io)) {
+ my $stats = parse_pressure("sys/fs/cgroup/${cgroup_path}/${type}.pressure");
+ $res->{$type} = $stats if $stats;
+ }
+ return $res;
+}
+
sub read_pressure {
my $res = {};
foreach my $type (qw(cpu memory io)) {
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH manager v2 1/5] api2tools: drop old VM rrd schema
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (10 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH common v2 2/2] add functions to retrieve pressures for vm/ct Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 2/5] api2tools: extract stats: handle existence of new pve-{type}-9.0 data Aaron Lauterer
` (11 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
pve2.3-vm has been introduced with commit 3b6ad3ac back in 2013. By now
there should not be any combination of clustered nodes that still send
the old pve2-vm variant.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
PVE/API2Tools.pm | 18 +-----------------
1 file changed, 1 insertion(+), 17 deletions(-)
diff --git a/PVE/API2Tools.pm b/PVE/API2Tools.pm
index d6154925..1e235c47 100644
--- a/PVE/API2Tools.pm
+++ b/PVE/API2Tools.pm
@@ -97,23 +97,7 @@ sub extract_vm_stats {
my $d;
- if ($d = $rrd->{"pve2-vm/$vmid"}) {
-
- $entry->{uptime} = ($d->[0] || 0) + 0;
- $entry->{name} = $d->[1];
- $entry->{status} = $entry->{uptime} ? 'running' : 'stopped';
- $entry->{maxcpu} = ($d->[3] || 0) + 0;
- $entry->{cpu} = ($d->[4] || 0) + 0;
- $entry->{maxmem} = ($d->[5] || 0) + 0;
- $entry->{mem} = ($d->[6] || 0) + 0;
- $entry->{maxdisk} = ($d->[7] || 0) + 0;
- $entry->{disk} = ($d->[8] || 0) + 0;
- $entry->{netin} = ($d->[9] || 0) + 0;
- $entry->{netout} = ($d->[10] || 0) + 0;
- $entry->{diskread} = ($d->[11] || 0) + 0;
- $entry->{diskwrite} = ($d->[12] || 0) + 0;
-
- } elsif ($d = $rrd->{"pve2.3-vm/$vmid"}) {
+ if ($d = $rrd->{"pve2.3-vm/$vmid"}) {
$entry->{uptime} = ($d->[0] || 0) + 0;
$entry->{name} = $d->[1];
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH manager v2 2/5] api2tools: extract stats: handle existence of new pve-{type}-9.0 data
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (11 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 1/5] api2tools: drop old VM rrd schema Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 3/5] pvestatd: collect and distribute new pve-{type}-9.0 metrics Aaron Lauterer
` (10 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
We add a new function to handle different key names, as it would
otherwise become quite unreadable.
It checks which key format exists for the type and resource:
* the old pve2-{type} / pve2.3-vm
* the new pve-{type}-{version}
and will return the one that was found. Since we will only have one key
per resource, we can return on the first hit.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
This patch also needs to be in PVE8 as otherwise those hosts won't be able
to extract stats from newer nodes -> question marks and missing info in
the GUI
changes since:
RFC:
* switch from pve9- to pve-{type}-9.0 schema
PVE/API2Tools.pm | 28 ++++++++++++++++++++++++----
1 file changed, 24 insertions(+), 4 deletions(-)
diff --git a/PVE/API2Tools.pm b/PVE/API2Tools.pm
index 1e235c47..08548524 100644
--- a/PVE/API2Tools.pm
+++ b/PVE/API2Tools.pm
@@ -41,6 +41,24 @@ sub get_hwaddress {
return $hwaddress;
}
+# each rrd key for a resource will only exist once. The key format might be different though. Therefore return on first hit
+sub get_rrd_key {
+ my ($rrd, $type, $id) = @_;
+
+ # check for old formats: pve2-{type}/{id}. For VMs and CTs the version number is different than for nodes and storages
+ if ($type ne "vm" && exists $rrd->{"pve2-${type}/${id}"}) {
+ return "pve2-${type}/${id}";
+ } elsif ($type eq "vm" && exists $rrd->{"pve2.3-${type}/${id}"}) {
+ return "pve2.3-${type}/${id}";
+ }
+
+ # if no old key has been found, we expect on in the newer format: pve-{type}-{version}/{id}
+ # We accept all new versions, as the expectation is that they are only allowed to add new colums as non-breaking change
+ for my $k (keys %$rrd) {
+ return $k if $k =~ m/^pve-\Q${type}\E-\d\d?.\d\/\Q${id}\E$/;
+ }
+}
+
sub extract_node_stats {
my ($node, $members, $rrd, $exclude_stats) = @_;
@@ -51,8 +69,8 @@ sub extract_node_stats {
status => 'unknown',
};
- if (my $d = $rrd->{"pve2-node/$node"}) {
-
+ my $key = get_rrd_key($rrd, "node", $node);
+ if (my $d = $rrd->{$key}) {
if (
!$members || # no cluster
($members->{$node} && $members->{$node}->{online})
@@ -96,8 +114,9 @@ sub extract_vm_stats {
};
my $d;
+ my $key = get_rrd_key($rrd, "vm", $vmid);
- if ($d = $rrd->{"pve2.3-vm/$vmid"}) {
+ if (my $d = $rrd->{$key}) {
$entry->{uptime} = ($d->[0] || 0) + 0;
$entry->{name} = $d->[1];
@@ -135,7 +154,8 @@ sub extract_storage_stats {
content => $content,
};
- if (my $d = $rrd->{"pve2-storage/$node/$storeid"}) {
+ my $key = get_rrd_key($rrd, "storage", "${node}/${storeid}");
+ if (my $d = $rrd->{$key}) {
$entry->{maxdisk} = ($d->[1] || 0) + 0;
$entry->{disk} = ($d->[2] || 0) + 0;
$entry->{status} = 'available';
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH manager v2 3/5] pvestatd: collect and distribute new pve-{type}-9.0 metrics
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (12 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 2/5] api2tools: extract stats: handle existence of new pve-{type}-9.0 data Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 4/5] api: nodes: rrd and rrddata add decade option and use new pve-node-9.0 rrd files Aaron Lauterer
` (9 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
If we see that the migration to the new pve-{type}-9.0 rrd format has been done
or is ongoing (new dir exists), we collect and send out the new format with additional
columns for nodes and VMs (guests).
Those are:
Nodes:
* memfree
* arcsize
* pressures:
* cpu some
* io some
* io full
* mem some
* mem full
VMs:
* memhost (memory consumption of all processes in the guests cgroup -> host view)
* pressures:
* cpu some
* cpu full
* io some
* io full
* mem some
* mem full
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
changes since:
RFC:
* switch from pve9- to pve-{type}-9.0 schema
PVE/Service/pvestatd.pm | 342 +++++++++++++++++++++++++++++-----------
1 file changed, 250 insertions(+), 92 deletions(-)
diff --git a/PVE/Service/pvestatd.pm b/PVE/Service/pvestatd.pm
index e3ea06bb..a22ae7bb 100755
--- a/PVE/Service/pvestatd.pm
+++ b/PVE/Service/pvestatd.pm
@@ -82,6 +82,16 @@ my $cached_kvm_version = '';
my $next_flag_update_time;
my $failed_flag_update_delay_sec = 120;
+# Checks if RRD files exist in the specified location.
+my $rrd_dir_exists = sub {
+ my ($location) = @_;
+ if (-d "/var/lib/rrdcached/db/${location}") {
+ return 1;
+ } else {
+ return 0;
+ }
+};
+
sub update_supported_cpuflags {
my $kvm_version = PVE::QemuServer::kvm_user_version();
@@ -180,32 +190,66 @@ sub update_node_status {
my $meminfo = PVE::ProcFSTools::read_meminfo();
+ my $pressures = PVE::ProcFSTools::read_pressure();
+
my $dinfo = df('/', 1); # output is bytes
# everything not free is considered to be used
my $dused = $dinfo->{blocks} - $dinfo->{bfree};
my $ctime = time();
- my $data = $generate_rrd_string->(
- [
- $uptime,
- $sublevel,
- $ctime,
- $avg1,
- $maxcpu,
- $stat->{cpu},
- $stat->{wait},
- $meminfo->{memtotal},
- $meminfo->{memused},
- $meminfo->{swaptotal},
- $meminfo->{swapused},
- $dinfo->{blocks},
- $dused,
- $netin,
- $netout,
- ],
- );
- PVE::Cluster::broadcast_rrd("pve2-node/$nodename", $data);
+ my $data;
+ # TODO: drop old pve2- schema with PVE 10
+ if ($rrd_dir_exists->("pve-node-9.0")) {
+ $data = $generate_rrd_string->(
+ [
+ $uptime,
+ $sublevel,
+ $ctime,
+ $avg1,
+ $maxcpu,
+ $stat->{cpu},
+ $stat->{wait},
+ $meminfo->{memtotal},
+ $meminfo->{memused},
+ $meminfo->{swaptotal},
+ $meminfo->{swapused},
+ $dinfo->{blocks},
+ $dused,
+ $netin,
+ $netout,
+ $meminfo->{memavailable},
+ $meminfo->{arcsize},
+ $pressures->{cpu}->{some}->{avg10},
+ $pressures->{io}->{some}->{avg10},
+ $pressures->{io}->{full}->{avg10},
+ $pressures->{memory}->{some}->{avg10},
+ $pressures->{memory}->{full}->{avg10},
+ ],
+ );
+ PVE::Cluster::broadcast_rrd("pve-node-9.0/$nodename", $data);
+ } else {
+ $data = $generate_rrd_string->(
+ [
+ $uptime,
+ $sublevel,
+ $ctime,
+ $avg1,
+ $maxcpu,
+ $stat->{cpu},
+ $stat->{wait},
+ $meminfo->{memtotal},
+ $meminfo->{memused},
+ $meminfo->{swaptotal},
+ $meminfo->{swapused},
+ $dinfo->{blocks},
+ $dused,
+ $netin,
+ $netout,
+ ],
+ );
+ PVE::Cluster::broadcast_rrd("pve2-node/$nodename", $data);
+ }
my $node_metric = {
uptime => $uptime,
@@ -273,44 +317,101 @@ sub update_qemu_status {
my $data;
my $status = $d->{qmpstatus} || $d->{status} || 'stopped';
my $template = $d->{template} ? $d->{template} : "0";
- if ($d->{pid}) { # running
- $data = $generate_rrd_string->([
- $d->{uptime},
- $d->{name},
- $status,
- $template,
- $ctime,
- $d->{cpus},
- $d->{cpu},
- $d->{maxmem},
- $d->{mem},
- $d->{maxdisk},
- $d->{disk},
- $d->{netin},
- $d->{netout},
- $d->{diskread},
- $d->{diskwrite},
- ]);
+
+ # TODO: drop old pve2.3- schema with PVE 10
+ if ($rrd_dir_exists->("pve-vm-9.0")) {
+ if ($d->{pid}) { # running
+ $data = $generate_rrd_string->([
+ $d->{uptime},
+ $d->{name},
+ $status,
+ $template,
+ $ctime,
+ $d->{cpus},
+ $d->{cpu},
+ $d->{maxmem},
+ $d->{mem},
+ $d->{maxdisk},
+ $d->{disk},
+ $d->{netin},
+ $d->{netout},
+ $d->{diskread},
+ $d->{diskwrite},
+ $d->{memhost},
+ $d->{pressurecpusome},
+ $d->{pressurecpufull},
+ $d->{pressureiosome},
+ $d->{pressureiofull},
+ $d->{pressurememorysome},
+ $d->{pressurememoryfull},
+ ]);
+ } else {
+ $data = $generate_rrd_string->([
+ 0,
+ $d->{name},
+ $status,
+ $template,
+ $ctime,
+ $d->{cpus},
+ undef,
+ $d->{maxmem},
+ undef,
+ $d->{maxdisk},
+ $d->{disk},
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ ]);
+ }
+ PVE::Cluster::broadcast_rrd("pve-vm-9.0/$vmid", $data);
} else {
- $data = $generate_rrd_string->([
- 0,
- $d->{name},
- $status,
- $template,
- $ctime,
- $d->{cpus},
- undef,
- $d->{maxmem},
- undef,
- $d->{maxdisk},
- $d->{disk},
- undef,
- undef,
- undef,
- undef,
- ]);
+ if ($d->{pid}) { # running
+ $data = $generate_rrd_string->([
+ $d->{uptime},
+ $d->{name},
+ $status,
+ $template,
+ $ctime,
+ $d->{cpus},
+ $d->{cpu},
+ $d->{maxmem},
+ $d->{mem},
+ $d->{maxdisk},
+ $d->{disk},
+ $d->{netin},
+ $d->{netout},
+ $d->{diskread},
+ $d->{diskwrite},
+ ]);
+ } else {
+ $data = $generate_rrd_string->([
+ 0,
+ $d->{name},
+ $status,
+ $template,
+ $ctime,
+ $d->{cpus},
+ undef,
+ $d->{maxmem},
+ undef,
+ $d->{maxdisk},
+ $d->{disk},
+ undef,
+ undef,
+ undef,
+ undef,
+ ]);
+ }
+ PVE::Cluster::broadcast_rrd("pve2.3-vm/$vmid", $data);
}
- PVE::Cluster::broadcast_rrd("pve2.3-vm/$vmid", $data);
PVE::ExtMetric::update_all($transactions, 'qemu', $vmid, $d, $ctime, $nodename);
}
@@ -506,44 +607,100 @@ sub update_lxc_status {
my $d = $vmstatus->{$vmid};
my $template = $d->{template} ? $d->{template} : "0";
my $data;
- if ($d->{status} eq 'running') { # running
- $data = $generate_rrd_string->([
- $d->{uptime},
- $d->{name},
- $d->{status},
- $template,
- $ctime,
- $d->{cpus},
- $d->{cpu},
- $d->{maxmem},
- $d->{mem},
- $d->{maxdisk},
- $d->{disk},
- $d->{netin},
- $d->{netout},
- $d->{diskread},
- $d->{diskwrite},
- ]);
+ # TODO: drop old pve2.3-vm schema with PVE 10
+ if ($rrd_dir_exists->("pve-vm-9.0")) {
+ if ($d->{pid}) { # running
+ $data = $generate_rrd_string->([
+ $d->{uptime},
+ $d->{name},
+ $d->{status},
+ $template,
+ $ctime,
+ $d->{cpus},
+ $d->{cpu},
+ $d->{maxmem},
+ $d->{mem},
+ $d->{maxdisk},
+ $d->{disk},
+ $d->{netin},
+ $d->{netout},
+ $d->{diskread},
+ $d->{diskwrite},
+ undef,
+ $d->{pressurecpusome},
+ $d->{pressurecpufull},
+ $d->{pressureiosome},
+ $d->{pressureiofull},
+ $d->{pressurememorysome},
+ $d->{pressurememoryfull},
+ ]);
+ } else {
+ $data = $generate_rrd_string->([
+ 0,
+ $d->{name},
+ $d->{status},
+ $template,
+ $ctime,
+ $d->{cpus},
+ undef,
+ $d->{maxmem},
+ undef,
+ $d->{maxdisk},
+ $d->{disk},
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ undef,
+ ]);
+ }
+ PVE::Cluster::broadcast_rrd("pve-vm-9.0/$vmid", $data);
} else {
- $data = $generate_rrd_string->([
- 0,
- $d->{name},
- $d->{status},
- $template,
- $ctime,
- $d->{cpus},
- undef,
- $d->{maxmem},
- undef,
- $d->{maxdisk},
- $d->{disk},
- undef,
- undef,
- undef,
- undef,
- ]);
+ if ($d->{status} eq 'running') { # running
+ $data = $generate_rrd_string->([
+ $d->{uptime},
+ $d->{name},
+ $d->{status},
+ $template,
+ $ctime,
+ $d->{cpus},
+ $d->{cpu},
+ $d->{maxmem},
+ $d->{mem},
+ $d->{maxdisk},
+ $d->{disk},
+ $d->{netin},
+ $d->{netout},
+ $d->{diskread},
+ $d->{diskwrite},
+ ]);
+ } else {
+ $data = $generate_rrd_string->([
+ 0,
+ $d->{name},
+ $d->{status},
+ $template,
+ $ctime,
+ $d->{cpus},
+ undef,
+ $d->{maxmem},
+ undef,
+ $d->{maxdisk},
+ $d->{disk},
+ undef,
+ undef,
+ undef,
+ undef,
+ ]);
+ }
+ PVE::Cluster::broadcast_rrd("pve2.3-vm/$vmid", $data);
}
- PVE::Cluster::broadcast_rrd("pve2.3-vm/$vmid", $data);
PVE::ExtMetric::update_all($transactions, 'lxc', $vmid, $d, $ctime, $nodename);
}
@@ -568,6 +725,7 @@ sub update_storage_status {
my $data = $generate_rrd_string->([$ctime, $d->{total}, $d->{used}]);
my $key = "pve2-storage/${nodename}/$storeid";
+ $key = "pve-storage-9.0/${nodename}/$storeid" if $rrd_dir_exists->("pve-storage-9.0");
PVE::Cluster::broadcast_rrd($key, $data);
PVE::ExtMetric::update_all($transactions, 'storage', $nodename, $storeid, $d, $ctime);
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH manager v2 4/5] api: nodes: rrd and rrddata add decade option and use new pve-node-9.0 rrd files
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (13 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 3/5] pvestatd: collect and distribute new pve-{type}-9.0 metrics Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 5/5] api2tools: extract_vm_status add new vm memhost column Aaron Lauterer
` (8 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
if the new rrd pve-node-9.0 files are present, they contain the current
data and should be used.
'decade' is now possible as timeframe with the new RRD format.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
changes since:
RFC:
* switch from pve9- to pve-{type}-9.0 schema
PVE/API2/Nodes.pm | 16 +++++++++-------
1 file changed, 9 insertions(+), 7 deletions(-)
diff --git a/PVE/API2/Nodes.pm b/PVE/API2/Nodes.pm
index 1eb04d9a..69b3d873 100644
--- a/PVE/API2/Nodes.pm
+++ b/PVE/API2/Nodes.pm
@@ -836,7 +836,7 @@ __PACKAGE__->register_method({
timeframe => {
description => "Specify the time frame you are interested in.",
type => 'string',
- enum => ['hour', 'day', 'week', 'month', 'year'],
+ enum => ['hour', 'day', 'week', 'month', 'year', 'decade'],
},
ds => {
description => "The list of datasources you want to display.",
@@ -860,9 +860,10 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- return PVE::RRD::create_rrd_graph(
- "pve2-node/$param->{node}", $param->{timeframe}, $param->{ds}, $param->{cf},
- );
+ my $path = "pve-node-9.0/$param->{node}";
+ $path = "pve2-node/$param->{node}" if !-e "/var/lib/rrdcached/db/${path}";
+ return PVE::RRD::create_rrd_graph($path, $param->{timeframe},
+ $param->{ds}, $param->{cf});
},
});
@@ -883,7 +884,7 @@ __PACKAGE__->register_method({
timeframe => {
description => "Specify the time frame you are interested in.",
type => 'string',
- enum => ['hour', 'day', 'week', 'month', 'year'],
+ enum => ['hour', 'day', 'week', 'month', 'year', 'decade'],
},
cf => {
description => "The RRD consolidation function",
@@ -903,8 +904,9 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- return PVE::RRD::create_rrd_data("pve2-node/$param->{node}", $param->{timeframe},
- $param->{cf});
+ my $path = "pve-node-9.0/$param->{node}";
+ $path = "pve2-node/$param->{node}" if !-e "/var/lib/rrdcached/db/${path}";
+ return PVE::RRD::create_rrd_data($path, $param->{timeframe}, $param->{cf});
},
});
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH manager v2 5/5] api2tools: extract_vm_status add new vm memhost column
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (14 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 4/5] api: nodes: rrd and rrddata add decade option and use new pve-node-9.0 rrd files Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH storage v2 1/1] status: rrddata: use new pve-storage-9.0 rrd location if file is present Aaron Lauterer
` (7 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
as this will also be displayed in the status of VMs
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
this is a dedicated patch that should be applied only for PVE9 as it
adds new data in the result
PVE/API2/Cluster.pm | 7 +++++++
PVE/API2Tools.pm | 3 +++
2 files changed, 10 insertions(+)
diff --git a/PVE/API2/Cluster.pm b/PVE/API2/Cluster.pm
index a025d264..81cdf217 100644
--- a/PVE/API2/Cluster.pm
+++ b/PVE/API2/Cluster.pm
@@ -301,6 +301,13 @@ __PACKAGE__->register_method({
renderer => 'bytes',
minimum => 0,
},
+ memhost => {
+ description => "Used memory in bytes from the point of view of the host (for types 'qemu').",
+ type => 'integer',
+ optional => 1,
+ renderer => 'bytes',
+ minimum => 0,
+ },
maxmem => {
description => "Number of available memory in bytes"
. " (for types 'node', 'qemu' and 'lxc').",
diff --git a/PVE/API2Tools.pm b/PVE/API2Tools.pm
index 08548524..ed0bddbf 100644
--- a/PVE/API2Tools.pm
+++ b/PVE/API2Tools.pm
@@ -133,6 +133,9 @@ sub extract_vm_stats {
$entry->{netout} = ($d->[12] || 0) + 0;
$entry->{diskread} = ($d->[13] || 0) + 0;
$entry->{diskwrite} = ($d->[14] || 0) + 0;
+ if ($key =~ /^pve-vm-/) {
+ $entry->{memhost} = ($d->[15] || 0) +0;
+ }
}
return $entry;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH storage v2 1/1] status: rrddata: use new pve-storage-9.0 rrd location if file is present
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (15 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH manager v2 5/5] api2tools: extract_vm_status add new vm memhost column Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH qemu-server v2 1/4] metrics: add pressure to metrics Aaron Lauterer
` (6 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
changes since:
RFC:
* switch from pve9-storage to pve-storage-90 schema
src/PVE/API2/Storage/Status.pm | 9 ++++-----
1 file changed, 4 insertions(+), 5 deletions(-)
diff --git a/src/PVE/API2/Storage/Status.pm b/src/PVE/API2/Storage/Status.pm
index f7c3519..a74b085 100644
--- a/src/PVE/API2/Storage/Status.pm
+++ b/src/PVE/API2/Storage/Status.pm
@@ -413,11 +413,10 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- return PVE::RRD::create_rrd_data(
- "pve2-storage/$param->{node}/$param->{storage}",
- $param->{timeframe},
- $param->{cf},
- );
+ my $path = "pve-storage-9.0/$param->{node}/$param->{storage}";
+ $path = "pve2-storage/$param->{node}/$param->{storage}"
+ if !-e "/var/lib/rrdcached/db/${path}";
+ return PVE::RRD::create_rrd_data($path, $param->{timeframe}, $param->{cf});
},
});
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH qemu-server v2 1/4] metrics: add pressure to metrics
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (16 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH storage v2 1/1] status: rrddata: use new pve-storage-9.0 rrd location if file is present Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:36 ` [pve-devel] [PATCH qemu-server v2 2/4] vmstatus: add memhost for host view of vm mem consumption Aaron Lauterer
` (5 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
From: Folke Gleumes <f.gleumes@proxmox.com>
Originally-by: Folke Gleumes <f.gleumes@proxmox.com>
[AL:
* rebased on current master
* switch to new, more generic read_cgroup_pressure function
]
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/PVE/QemuServer.pm | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 92c8fad..01f7a4f 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -2662,6 +2662,14 @@ sub vmstatus {
} else {
$d->{cpu} = $old->{cpu};
}
+
+ my $pressures = PVE::ProcFSTools::read_cgroup_pressure("qemu.slice/${vmid}.scope");
+ $d->{pressurecpusome} = $pressures->{cpu}->{some}->{avg10};
+ $d->{pressurecpufull} = $pressures->{cpu}->{full}->{avg10};
+ $d->{pressureiosome} = $pressures->{io}->{some}->{avg10};
+ $d->{pressureiofull} = $pressures->{io}->{full}->{avg10};
+ $d->{pressurememorysome} = $pressures->{memory}->{some}->{avg10};
+ $d->{pressurememoryfull} = $pressures->{memory}->{full}->{avg10};
}
return $res if !$full;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH qemu-server v2 2/4] vmstatus: add memhost for host view of vm mem consumption
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (17 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH qemu-server v2 1/4] metrics: add pressure to metrics Aaron Lauterer
@ 2025-07-09 16:36 ` Aaron Lauterer
2025-07-09 16:37 ` [pve-devel] [PATCH qemu-server v2 3/4] vmstatus: switch mem stat to PSS of VM cgroup Aaron Lauterer
` (4 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:36 UTC (permalink / raw)
To: pve-devel
The mem field itself will switch from the outside view to the "inside"
view if the VM is reporting detailed memory usage informatio via the
ballooning device.
Since sometime other processes belong to a VM too, vor example swtpm,
we collect all PIDs belonging to the VM cgroup and fetch their PSS data
to account for shared libraries used.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
changes since:
RFC:
* collect memory info for all processes in cgroup directly without too
generic helper function
src/PVE/QemuServer.pm | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 01f7a4f..4dd30c4 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -2586,6 +2586,7 @@ sub vmstatus {
$d->{uptime} = 0;
$d->{cpu} = 0;
$d->{mem} = 0;
+ $d->{memhost} = 0;
$d->{netout} = 0;
$d->{netin} = 0;
@@ -2670,6 +2671,24 @@ sub vmstatus {
$d->{pressureiofull} = $pressures->{io}->{full}->{avg10};
$d->{pressurememorysome} = $pressures->{memory}->{some}->{avg10};
$d->{pressurememoryfull} = $pressures->{memory}->{full}->{avg10};
+
+ my $fh = IO::File->new("/sys/fs/cgroup/qemu.slice/${vmid}.scope/cgroup.procs", "r");
+ if ($fh) {
+ while (my $childPid = <$fh>) {
+ chomp($childPid);
+ open(my $SMAPS_FH, '<', "/proc/$childPid/smaps_rollup")
+ or die "failed to open PSS memory-stat from process - $!\n";
+
+ while (my $line = <$SMAPS_FH>) {
+ if ($line =~ m/^Pss:\s+([0-9]+) kB$/) {
+ $d->{memhost} = $d->{memhost} + int($1) * 1024;
+ last;
+ }
+ }
+ close $SMAPS_FH;
+ }
+ }
+ close($fh);
}
return $res if !$full;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH qemu-server v2 3/4] vmstatus: switch mem stat to PSS of VM cgroup
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (18 preceding siblings ...)
2025-07-09 16:36 ` [pve-devel] [PATCH qemu-server v2 2/4] vmstatus: add memhost for host view of vm mem consumption Aaron Lauterer
@ 2025-07-09 16:37 ` Aaron Lauterer
2025-07-09 16:37 ` [pve-devel] [PATCH qemu-server v2 4/4] rrddata: use new pve-vm-9.0 rrd location if file is present Aaron Lauterer
` (3 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:37 UTC (permalink / raw)
To: pve-devel
Instead of RSS, let's use the same PSS values as for the specific host
view as default, in case this value is not overwritten by the balloon
info.
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/PVE/QemuServer.pm | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/src/PVE/QemuServer.pm b/src/PVE/QemuServer.pm
index 4dd30c4..22172ea 100644
--- a/src/PVE/QemuServer.pm
+++ b/src/PVE/QemuServer.pm
@@ -2635,10 +2635,6 @@ sub vmstatus {
$d->{uptime} = int(($uptime - $pstat->{starttime}) / $cpuinfo->{user_hz});
- if ($pstat->{vsize}) {
- $d->{mem} = int(($pstat->{rss} / $pstat->{vsize}) * $d->{maxmem});
- }
-
my $old = $last_proc_pid_stat->{$pid};
if (!$old) {
$last_proc_pid_stat->{$pid} = {
@@ -2689,6 +2685,8 @@ sub vmstatus {
}
}
close($fh);
+
+ $d->{mem} = $d->{memhost};
}
return $res if !$full;
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH qemu-server v2 4/4] rrddata: use new pve-vm-9.0 rrd location if file is present
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (19 preceding siblings ...)
2025-07-09 16:37 ` [pve-devel] [PATCH qemu-server v2 3/4] vmstatus: switch mem stat to PSS of VM cgroup Aaron Lauterer
@ 2025-07-09 16:37 ` Aaron Lauterer
2025-07-09 16:37 ` [pve-devel] [PATCH container v2 1/2] metrics: add pressures to metrics Aaron Lauterer
` (2 subsequent siblings)
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:37 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
changes since:
RFC:
* switch from pve9-vm to pve-vm-90 schema
src/PVE/API2/Qemu.pm | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/src/PVE/API2/Qemu.pm b/src/PVE/API2/Qemu.pm
index 2e6358e..2867a53 100644
--- a/src/PVE/API2/Qemu.pm
+++ b/src/PVE/API2/Qemu.pm
@@ -1628,9 +1628,9 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- return PVE::RRD::create_rrd_graph(
- "pve2-vm/$param->{vmid}", $param->{timeframe}, $param->{ds}, $param->{cf},
- );
+ my $path = "pve-vm-9.0/$param->{vmid}";
+ $path = "pve2-vm/$param->{vmid}" if !-e "/var/lib/rrdcached/db/${path}";
+ return PVE::RRD::create_rrd_graph($path, $param->{timeframe}, $param->{cf});
},
});
@@ -1672,8 +1672,9 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- return PVE::RRD::create_rrd_data("pve2-vm/$param->{vmid}", $param->{timeframe},
- $param->{cf});
+ my $path = "pve-vm-9.0/$param->{vmid}";
+ $path = "pve2-vm/$param->{vmid}" if !-e "/var/lib/rrdcached/db/${path}";
+ return PVE::RRD::create_rrd_data($path, $param->{timeframe}, $param->{cf});
},
});
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH container v2 1/2] metrics: add pressures to metrics
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (20 preceding siblings ...)
2025-07-09 16:37 ` [pve-devel] [PATCH qemu-server v2 4/4] rrddata: use new pve-vm-9.0 rrd location if file is present Aaron Lauterer
@ 2025-07-09 16:37 ` Aaron Lauterer
2025-07-09 16:37 ` [pve-devel] [PATCH container v2 2/2] rrddata: use new pve-vm-9.0 rrd location if file is present Aaron Lauterer
2025-07-15 14:33 ` [pve-devel] SUPERSEEDED Re: [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:37 UTC (permalink / raw)
To: pve-devel
From: Folke Gleumes <f.gleumes@proxmox.com>
Originally-by: Folke Gleumes <f.gleumes@proxmox.com>
[AL:
* rebased on current master
* switch to new, more generic read_cgroup_pressure function
]
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
src/PVE/LXC.pm | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/src/PVE/LXC.pm b/src/PVE/LXC.pm
index ffedcb9..242e176 100644
--- a/src/PVE/LXC.pm
+++ b/src/PVE/LXC.pm
@@ -358,6 +358,14 @@ sub vmstatus {
} else {
$d->{cpu} = 0;
}
+
+ my $pressures = PVE::ProcFSTools::read_cgroup_pressure("lxc/${vmid}");
+ $d->{pressurecpusome} = $pressures->{cpu}{some}{avg10};
+ $d->{pressurecpufull} = $pressures->{cpu}{full}{avg10};
+ $d->{pressureiosome} = $pressures->{io}{some}{avg10};
+ $d->{pressureiofull} = $pressures->{io}{full}{avg10};
+ $d->{pressurememorysome} = $pressures->{memory}{some}{avg10};
+ $d->{pressurememoryfull} = $pressures->{memory}{full}{avg10};
}
my $netdev = PVE::ProcFSTools::read_proc_net_dev();
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] [PATCH container v2 2/2] rrddata: use new pve-vm-9.0 rrd location if file is present
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (21 preceding siblings ...)
2025-07-09 16:37 ` [pve-devel] [PATCH container v2 1/2] metrics: add pressures to metrics Aaron Lauterer
@ 2025-07-09 16:37 ` Aaron Lauterer
2025-07-15 14:33 ` [pve-devel] SUPERSEEDED Re: [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-09 16:37 UTC (permalink / raw)
To: pve-devel
Signed-off-by: Aaron Lauterer <a.lauterer@proxmox.com>
---
Notes:
changes since RFC:
* switch from pve9-vm to pve-vm-90 schema
src/PVE/API2/LXC.pm | 11 ++++++-----
1 file changed, 6 insertions(+), 5 deletions(-)
diff --git a/src/PVE/API2/LXC.pm b/src/PVE/API2/LXC.pm
index 28f7fdd..fc59ec9 100644
--- a/src/PVE/API2/LXC.pm
+++ b/src/PVE/API2/LXC.pm
@@ -712,9 +712,9 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- return PVE::RRD::create_rrd_graph(
- "pve2-vm/$param->{vmid}", $param->{timeframe}, $param->{ds}, $param->{cf},
- );
+ my $path = "pve-vm-9.0/$param->{vmid}";
+ $path = "pve2-vm/$param->{vmid}" if !-e "/var/lib/rrdcached/db/${path}";
+ return PVE::RRD::create_rrd_graph($path, $param->{timeframe}, $param->{cf});
},
});
@@ -756,8 +756,9 @@ __PACKAGE__->register_method({
code => sub {
my ($param) = @_;
- return PVE::RRD::create_rrd_data("pve2-vm/$param->{vmid}", $param->{timeframe},
- $param->{cf});
+ my $path = "pve-vm-9.0/$param->{vmid}";
+ $path = "pve2-vm/$param->{vmid}" if !-e "/var/lib/rrdcached/db/${path}";
+ return PVE::RRD::create_rrd_data($path, $param->{timeframe}, $param->{cf});
},
});
--
2.39.5
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread* [pve-devel] SUPERSEEDED Re: [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI)
2025-07-09 16:36 [pve-devel] [PATCH many v2 00/23] Expand and migrate RRD data (excluding GUI) Aaron Lauterer
` (22 preceding siblings ...)
2025-07-09 16:37 ` [pve-devel] [PATCH container v2 2/2] rrddata: use new pve-vm-9.0 rrd location if file is present Aaron Lauterer
@ 2025-07-15 14:33 ` Aaron Lauterer
23 siblings, 0 replies; 25+ messages in thread
From: Aaron Lauterer @ 2025-07-15 14:33 UTC (permalink / raw)
To: pve-devel
I sent a new v3 that includes fixes to bugs I found in v2 and also adds
the GUI part
https://lore.proxmox.com/pve-devel/20250715143218.1548306-1-a.lauterer@proxmox.com/T/#t
_______________________________________________
pve-devel mailing list
pve-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
^ permalink raw reply [flat|nested] 25+ messages in thread