all lists on lists.proxmox.com
 help / color / mirror / Atom feed
From: "Fabian Grünbichler" <f.gruenbichler@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH proxmox-offline-mirror 2/2] fix #4261: allow GC for all configured mirrors
Date: Tue,  4 Apr 2023 13:21:15 +0200	[thread overview]
Message-ID: <20230404112115.184009-2-f.gruenbichler@proxmox.com> (raw)
In-Reply-To: <20230404112115.184009-1-f.gruenbichler@proxmox.com>

by making the --id parameter optional, and structuring the output accordingly.
since pools are per base-dir, GC only needs to run once per base-dir instead of
for each mirror entry.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 src/bin/proxmox_offline_mirror_cmds/mirror.rs | 52 +++++++++++++++++--
 1 file changed, 47 insertions(+), 5 deletions(-)

diff --git a/src/bin/proxmox_offline_mirror_cmds/mirror.rs b/src/bin/proxmox_offline_mirror_cmds/mirror.rs
index a3fb258..07db92f 100644
--- a/src/bin/proxmox_offline_mirror_cmds/mirror.rs
+++ b/src/bin/proxmox_offline_mirror_cmds/mirror.rs
@@ -4,7 +4,7 @@ use proxmox_section_config::SectionConfigData;
 use proxmox_subscription::SubscriptionStatus;
 use serde_json::Value;
 use std::{
-    collections::{BTreeMap, HashMap},
+    collections::{BTreeMap, HashMap, HashSet},
     path::PathBuf,
 };
 
@@ -285,6 +285,7 @@ async fn remove_snapshot(
             },
             id: {
                 schema: MIRROR_ID_SCHEMA,
+                optional: true,
             },
             "output-format": {
                 schema: OUTPUT_FORMAT,
@@ -293,14 +294,55 @@ async fn remove_snapshot(
         }
     },
  )]
-/// Run Garbage Collection on pool
-async fn garbage_collect(config: Option<String>, id: String, _param: Value) -> Result<(), Error> {
+/// Run Garbage Collection on pool(s). If no `id` is specified, the pools of all configured mirrors
+/// will be GCed.
+async fn garbage_collect(
+    config: Option<String>,
+    id: Option<String>,
+    _param: Value,
+) -> Result<(), Error> {
     let config = config.unwrap_or_else(get_config_path);
 
     let (config, _digest) = proxmox_offline_mirror::config::config(&config)?;
-    let config: MirrorConfig = config.lookup("mirror", &id)?;
 
-    let (count, size) = mirror::gc(&config)?;
+    let (count, size) = if let Some(id) = id {
+        let config: MirrorConfig = config.lookup("mirror", &id)?;
+        mirror::gc(&config)?
+    } else {
+        let mut total_count = 0;
+        let mut total_size = 0;
+        let mut error_count = 0;
+        let mut base_dirs = HashSet::new();
+
+        for mirror_config in config.convert_to_typed_array::<MirrorConfig>("mirror")? {
+            if base_dirs.insert(mirror_config.base_dir.clone()) {
+                match mirror::gc(&mirror_config) {
+                    Ok((count, size)) => {
+                        println!(
+                            "{}: removed {count} files totalling {size}b",
+                            mirror_config.id
+                        );
+                        total_count += count;
+                        total_size += size;
+                    }
+                    Err(err) => {
+                        error_count += 1;
+                        eprintln!("{}: failed to run GC - {err}", mirror_config.id);
+                    }
+                }
+            } else {
+                println!(
+                    "{}: base dir '{}' already GCed",
+                    mirror_config.id, mirror_config.base_dir
+                );
+            }
+            println!();
+        }
+        if error_count > 0 {
+            eprintln!("Encountered {error_count} errors, please check log.");
+        }
+        (total_count, total_size)
+    };
 
     println!("Removed {} files totalling {}b", count, size);
 
-- 
2.30.2





  reply	other threads:[~2023-04-04 11:21 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-04-04 11:21 [pve-devel] [PATCH proxmox-offline-mirror 1/2] pool: drop redundant quote for paths Fabian Grünbichler
2023-04-04 11:21 ` Fabian Grünbichler [this message]
2023-04-06 11:25   ` [pve-devel] applied: [PATCH proxmox-offline-mirror 2/2] fix #4261: allow GC for all configured mirrors Thomas Lamprecht
2023-04-06 11:25 ` [pve-devel] applied: [PATCH proxmox-offline-mirror 1/2] pool: drop redundant quote for paths Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230404112115.184009-2-f.gruenbichler@proxmox.com \
    --to=f.gruenbichler@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal