public inbox for pve-devel@lists.proxmox.com
 help / color / mirror / Atom feed
From: "Fabian Grünbichler" <f.gruenbichler@proxmox.com>
To: pve-devel@lists.proxmox.com
Subject: [pve-devel] [PATCH proxmox-offline-mirror 2/2] fix #4261: allow GC for all configured mirrors
Date: Tue,  4 Apr 2023 13:21:15 +0200	[thread overview]
Message-ID: <20230404112115.184009-2-f.gruenbichler@proxmox.com> (raw)
In-Reply-To: <20230404112115.184009-1-f.gruenbichler@proxmox.com>

by making the --id parameter optional, and structuring the output accordingly.
since pools are per base-dir, GC only needs to run once per base-dir instead of
for each mirror entry.

Signed-off-by: Fabian Grünbichler <f.gruenbichler@proxmox.com>
---
 src/bin/proxmox_offline_mirror_cmds/mirror.rs | 52 +++++++++++++++++--
 1 file changed, 47 insertions(+), 5 deletions(-)

diff --git a/src/bin/proxmox_offline_mirror_cmds/mirror.rs b/src/bin/proxmox_offline_mirror_cmds/mirror.rs
index a3fb258..07db92f 100644
--- a/src/bin/proxmox_offline_mirror_cmds/mirror.rs
+++ b/src/bin/proxmox_offline_mirror_cmds/mirror.rs
@@ -4,7 +4,7 @@ use proxmox_section_config::SectionConfigData;
 use proxmox_subscription::SubscriptionStatus;
 use serde_json::Value;
 use std::{
-    collections::{BTreeMap, HashMap},
+    collections::{BTreeMap, HashMap, HashSet},
     path::PathBuf,
 };
 
@@ -285,6 +285,7 @@ async fn remove_snapshot(
             },
             id: {
                 schema: MIRROR_ID_SCHEMA,
+                optional: true,
             },
             "output-format": {
                 schema: OUTPUT_FORMAT,
@@ -293,14 +294,55 @@ async fn remove_snapshot(
         }
     },
  )]
-/// Run Garbage Collection on pool
-async fn garbage_collect(config: Option<String>, id: String, _param: Value) -> Result<(), Error> {
+/// Run Garbage Collection on pool(s). If no `id` is specified, the pools of all configured mirrors
+/// will be GCed.
+async fn garbage_collect(
+    config: Option<String>,
+    id: Option<String>,
+    _param: Value,
+) -> Result<(), Error> {
     let config = config.unwrap_or_else(get_config_path);
 
     let (config, _digest) = proxmox_offline_mirror::config::config(&config)?;
-    let config: MirrorConfig = config.lookup("mirror", &id)?;
 
-    let (count, size) = mirror::gc(&config)?;
+    let (count, size) = if let Some(id) = id {
+        let config: MirrorConfig = config.lookup("mirror", &id)?;
+        mirror::gc(&config)?
+    } else {
+        let mut total_count = 0;
+        let mut total_size = 0;
+        let mut error_count = 0;
+        let mut base_dirs = HashSet::new();
+
+        for mirror_config in config.convert_to_typed_array::<MirrorConfig>("mirror")? {
+            if base_dirs.insert(mirror_config.base_dir.clone()) {
+                match mirror::gc(&mirror_config) {
+                    Ok((count, size)) => {
+                        println!(
+                            "{}: removed {count} files totalling {size}b",
+                            mirror_config.id
+                        );
+                        total_count += count;
+                        total_size += size;
+                    }
+                    Err(err) => {
+                        error_count += 1;
+                        eprintln!("{}: failed to run GC - {err}", mirror_config.id);
+                    }
+                }
+            } else {
+                println!(
+                    "{}: base dir '{}' already GCed",
+                    mirror_config.id, mirror_config.base_dir
+                );
+            }
+            println!();
+        }
+        if error_count > 0 {
+            eprintln!("Encountered {error_count} errors, please check log.");
+        }
+        (total_count, total_size)
+    };
 
     println!("Removed {} files totalling {}b", count, size);
 
-- 
2.30.2





  reply	other threads:[~2023-04-04 11:21 UTC|newest]

Thread overview: 4+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-04-04 11:21 [pve-devel] [PATCH proxmox-offline-mirror 1/2] pool: drop redundant quote for paths Fabian Grünbichler
2023-04-04 11:21 ` Fabian Grünbichler [this message]
2023-04-06 11:25   ` [pve-devel] applied: [PATCH proxmox-offline-mirror 2/2] fix #4261: allow GC for all configured mirrors Thomas Lamprecht
2023-04-06 11:25 ` [pve-devel] applied: [PATCH proxmox-offline-mirror 1/2] pool: drop redundant quote for paths Thomas Lamprecht

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230404112115.184009-2-f.gruenbichler@proxmox.com \
    --to=f.gruenbichler@proxmox.com \
    --cc=pve-devel@lists.proxmox.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal