public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH backup v5 1/3] fix #4077: Estimated Full metric on ext4 file systems
@ 2022-11-09 14:25 Daniel Tschlatscher
  2022-11-09 14:25 ` [pbs-devel] [PATCH backup v5 2/3] 'available' field in rrd data in the API and change usage of 'total' Daniel Tschlatscher
                   ` (2 more replies)
  0 siblings, 3 replies; 4+ messages in thread
From: Daniel Tschlatscher @ 2022-11-09 14:25 UTC (permalink / raw)
  To: pbs-devel

The rrd data now includes tracking the available field in disk usage.
The calculation for the estimated_time_full was adapted to use the
total for the unpriviliged user, which is the sum of used + available.

The total for unprivileged users is preferable, because datastores are
always written to by the backup user. Which means that any storage
space reserved for root is unusable for our purposes.

To avoid resetting the estimate when switching to this new version,
the backend will try to use the available value to calculate the
unprivileged total. When that is not an option, it will fall back to
using the absolute total.

Signed-off-by: Daniel Tschlatscher <d.tschlatscher@proxmox.com>
---
Changes from v4:
* Revised the implementation of calculating the estimated_full_date
  field. The new implementation now tries to calculate the total with
  the new rrd field 'available' + 'used', and if that fails, falls
  back to using the 'total' that was used up until now.

 src/api2/status.rs              | 49 ++++++++++++++++-----------------
 src/bin/proxmox-backup-proxy.rs |  2 ++
 2 files changed, 25 insertions(+), 26 deletions(-)

diff --git a/src/api2/status.rs b/src/api2/status.rs
index 50d401c2..87450d63 100644
--- a/src/api2/status.rs
+++ b/src/api2/status.rs
@@ -86,42 +86,39 @@ pub async fn datastore_status(
 
         let total_res = get_rrd("total")?;
         let used_res = get_rrd("used")?;
+        let avail_res = get_rrd("available")?;
 
-        if let (
-            Some(proxmox_rrd::Entry {
-                start,
-                resolution,
-                data: total_list,
-            }),
-            Some(proxmox_rrd::Entry {
-                data: used_list, ..
-            }),
-        ) = (total_res, used_res)
-        {
+        if let Some(((total_entry, used), avail)) = total_res.zip(used_res).zip(avail_res) {
             let mut usage_list: Vec<f64> = Vec::new();
             let mut time_list: Vec<u64> = Vec::new();
             let mut history = Vec::new();
 
-            for (idx, used) in used_list.iter().enumerate() {
-                let total = if idx < total_list.len() {
-                    total_list[idx]
+            for (idx, used) in used.data.iter().enumerate() {
+                let used = match used {
+                    Some(used) => used,
+                    _ => {
+                        history.push(None);
+                        continue;
+                    }
+                };
+
+                let total = if idx < avail.data.len() && avail.data[idx].is_some() {
+                    avail.data[idx].unwrap() + used
+                } else if idx < total_entry.data.len() && total_entry.data[idx].is_some() {
+                    total_entry.data[idx].unwrap()
                 } else {
-                    None
+                    history.push(None);
+                    continue;
                 };
 
-                match (total, used) {
-                    (Some(total), Some(used)) if total != 0.0 => {
-                        time_list.push(start + (idx as u64) * resolution);
-                        let usage = used / total;
-                        usage_list.push(usage);
-                        history.push(Some(usage));
-                    }
-                    _ => history.push(None),
-                }
+                let usage = used / total;
+                time_list.push(total_entry.start + (idx as u64) * total_entry.resolution);
+                usage_list.push(usage);
+                history.push(Some(usage));
             }
 
-            entry.history_start = Some(start);
-            entry.history_delta = Some(resolution);
+            entry.history_start = Some(total_entry.start);
+            entry.history_delta = Some(total_entry.resolution);
             entry.history = Some(history);
 
             // we skip the calculation for datastores with not enough data
diff --git a/src/bin/proxmox-backup-proxy.rs b/src/bin/proxmox-backup-proxy.rs
index 69863481..ec2a8744 100644
--- a/src/bin/proxmox-backup-proxy.rs
+++ b/src/bin/proxmox-backup-proxy.rs
@@ -1255,6 +1255,8 @@ fn rrd_update_disk_stat(disk: &DiskStat, rrd_prefix: &str) {
         rrd_update_gauge(&rrd_key, status.total as f64);
         let rrd_key = format!("{}/used", rrd_prefix);
         rrd_update_gauge(&rrd_key, status.used as f64);
+        let rrd_key = format!("{}/available", rrd_prefix);
+        rrd_update_gauge(&rrd_key, status.available as f64);
     }
 
     if let Some(stat) = &disk.dev {
-- 
2.30.2





^ permalink raw reply	[flat|nested] 4+ messages in thread
end of thread, other threads:[~2022-11-24 13:30 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-11-09 14:25 [pbs-devel] [PATCH backup v5 1/3] fix #4077: Estimated Full metric on ext4 file systems Daniel Tschlatscher
2022-11-09 14:25 ` [pbs-devel] [PATCH backup v5 2/3] 'available' field in rrd data in the API and change usage of 'total' Daniel Tschlatscher
2022-11-09 14:25 ` [pbs-devel] [PATCH backup v5 3/3] gui: change reporting of the estimated_time_full to "Full" if no space Daniel Tschlatscher
2022-11-24 13:29 ` [pbs-devel] applied-series: [PATCH backup v5 1/3] fix #4077: Estimated Full metric on ext4 file systems Wolfgang Bumiller

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal