[pbs-devel] [PATCH v3 proxmox-backup 0/8] *** SUBJECT HERE ***

[pbs-devel] [PATCH v3 proxmox-backup 0/8] *** SUBJECT HERE ***

[Stefan Lendl]

*** BLURB HERE ***

Stefan Lendl (8):
  api: garbage collect job status
  gc: global prune and gc job view
  gc: move datastore/PruneAndGC to config/PruneAndGC
  gc: hide datastore column in local gc view
  ui: order Prune&GC before SyncJobs
  cli: list gc jobs with proxmox-backup-manager
  gc: show removed and pending chunks of last run in ui
  gc: configure width and flex on GC Jobs columns

 pbs-api-types/src/datastore.rs    |  40 +++++
 src/api2/admin/datastore.rs       | 129 +++++++++++++++-
 src/api2/admin/gc.rs              |  57 +++++++
 src/api2/admin/mod.rs             |   2 +
 src/bin/proxmox-backup-manager.rs |  33 ++++
 www/Makefile                      |   4 +-
 www/Utils.js                      |   6 +-
 www/config/GCView.js              | 243 ++++++++++++++++++++++++++++++
 www/config/PruneAndGC.js          |  52 +++++++
 www/datastore/DataStoreList.js    |  11 +-
 www/datastore/Panel.js            |   3 +-
 www/datastore/PruneAndGC.js       | 133 ----------------
 www/window/GCJobEdit.js           |  28 ++++
 13 files changed, 593 insertions(+), 148 deletions(-)
 create mode 100644 src/api2/admin/gc.rs
 create mode 100644 www/config/GCView.js
 create mode 100644 www/config/PruneAndGC.js
 delete mode 100644 www/datastore/PruneAndGC.js
 create mode 100644 www/window/GCJobEdit.js

-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 1/8] api: garbage collect job status

[Stefan Lendl]

Adds an api endpoint on the datastore that reports the gc job status
such as:
 - Schedule
 - State (of last run)
 - Duration (of last run)
 - Last Run
 - Next Run (if scheduled)
 - Pending Chunks (of last run)
 - Removed Chunks (of last run)

Adds a dedicated endpoint admin/gc that reports gc job status for all
datastores including the onces without a gc-schedule.

Originally-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
---
 pbs-api-types/src/datastore.rs |  40 ++++++++++
 src/api2/admin/datastore.rs    | 129 ++++++++++++++++++++++++++++++++-
 src/api2/admin/gc.rs           |  57 +++++++++++++++
 src/api2/admin/mod.rs          |   2 +
 4 files changed, 225 insertions(+), 3 deletions(-)
 create mode 100644 src/api2/admin/gc.rs

diff --git a/pbs-api-types/src/datastore.rs b/pbs-api-types/src/datastore.rs
index cce9888b..ba3879c9 100644
--- a/pbs-api-types/src/datastore.rs
+++ b/pbs-api-types/src/datastore.rs
@@ -1270,6 +1270,46 @@ pub struct GarbageCollectionStatus {
     pub still_bad: usize,
 }
 
+#[api(
+    properties: {
+        "last-run-upid": {
+            optional: true,
+            type: UPID,
+        },
+    },
+)]
+#[derive(Clone, Debug, Default, Serialize, Deserialize, PartialEq)]
+#[serde(rename_all = "kebab-case")]
+/// Garbage Collection general info
+pub struct GarbageCollectionJobStatus {
+    /// Datastore
+    pub store: String,
+    /// upid of the last run gc job
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub last_run_upid: Option<String>,
+    /// Number of removed chunks
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub removed_chunks: Option<usize>,
+    /// Number of pending chunks
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub pending_chunks: Option<usize>,
+    /// Schedule of the gc job
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub schedule: Option<String>,
+    /// Time of the next gc run
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub next_run: Option<i64>,
+    /// Endtime of the last gc run
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub last_run_endtime: Option<i64>,
+    /// State of the last gc run
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub last_run_state: Option<String>,
+    /// Duration of last gc run
+    #[serde(skip_serializing_if = "Option::is_none")]
+    pub duration: Option<i64>,
+}
+
 #[api(
     properties: {
         "gc-status": {
diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
index a95031e7..357cae0a 100644
--- a/src/api2/admin/datastore.rs
+++ b/src/api2/admin/datastore.rs
@@ -27,18 +27,20 @@ use proxmox_sys::fs::{
     file_read_firstline, file_read_optional_string, replace_file, CreateOptions,
 };
 use proxmox_sys::{task_log, task_warn};
+use proxmox_time::CalendarEvent;
 
 use pxar::accessor::aio::Accessor;
 use pxar::EntryKind;
 
 use pbs_api_types::{
     print_ns_and_snapshot, print_store_and_ns, Authid, BackupContent, BackupNamespace, BackupType,
-    Counts, CryptMode, DataStoreListItem, DataStoreStatus, GarbageCollectionStatus, GroupListItem,
+    Counts, CryptMode, DataStoreConfig, DataStoreListItem, DataStoreStatus,
+    GarbageCollectionJobStatus, GarbageCollectionStatus, GroupListItem, JobScheduleStatus,
     KeepOptions, Operation, PruneJobOptions, RRDMode, RRDTimeFrame, SnapshotListItem,
     SnapshotVerifyState, BACKUP_ARCHIVE_NAME_SCHEMA, BACKUP_ID_SCHEMA, BACKUP_NAMESPACE_SCHEMA,
     BACKUP_TIME_SCHEMA, BACKUP_TYPE_SCHEMA, DATASTORE_SCHEMA, IGNORE_VERIFIED_BACKUPS_SCHEMA,
     MAX_NAMESPACE_DEPTH, NS_MAX_DEPTH_SCHEMA, PRIV_DATASTORE_AUDIT, PRIV_DATASTORE_BACKUP,
-    PRIV_DATASTORE_MODIFY, PRIV_DATASTORE_PRUNE, PRIV_DATASTORE_READ, PRIV_DATASTORE_VERIFY,
+    PRIV_DATASTORE_MODIFY, PRIV_DATASTORE_PRUNE, PRIV_DATASTORE_READ, PRIV_DATASTORE_VERIFY, UPID,
     UPID_SCHEMA, VERIFICATION_OUTDATED_AFTER_SCHEMA,
 };
 use pbs_client::pxar::{create_tar, create_zip};
@@ -67,7 +69,7 @@ use crate::backup::{
     ListAccessibleBackupGroups, NS_PRIVS_OK,
 };
 
-use crate::server::jobstate::Job;
+use crate::server::jobstate::{compute_schedule_status, Job, JobState};
 
 const GROUP_NOTES_FILE_NAME: &str = "notes";
 
@@ -1199,6 +1201,123 @@ pub fn garbage_collection_status(
     Ok(status)
 }
 
+#[api(
+    input: {
+        properties: {
+            store: {
+                schema: DATASTORE_SCHEMA,
+            },
+        },
+    },
+    returns: {
+        type: GarbageCollectionJobStatus,
+    },
+    access: {
+        permission: &Permission::Privilege(&["datastore", "{store}"], PRIV_DATASTORE_AUDIT, false),
+    },
+)]
+/// Garbage collection status.
+pub fn garbage_collection_job_status(
+    store: String,
+    _info: &ApiMethod,
+    _rpcenv: &mut dyn RpcEnvironment,
+) -> Result<GarbageCollectionJobStatus, Error> {
+    let (config, _) = pbs_config::datastore::config()?;
+    let store_config: DataStoreConfig = config.lookup("datastore", &store)?;
+
+    let mut info = GarbageCollectionJobStatus {
+        store: store.clone(),
+        schedule: store_config.gc_schedule,
+        ..Default::default()
+    };
+
+    let datastore = DataStore::lookup_datastore(&store, Some(Operation::Read))?;
+    let status_in_memory = datastore.last_gc_status();
+    let state_file = JobState::load("garbage_collection", &store)
+        .map_err(|err| {
+            log::error!(
+                "could not open statefile for {:?}: {}",
+                info.last_run_upid,
+                err
+            )
+        })
+        .ok();
+
+    let mut selected_upid = None;
+    if status_in_memory.upid.is_some() {
+        selected_upid = status_in_memory.upid;
+    } else if let Some(JobState::Finished { upid, .. }) = &state_file {
+        selected_upid = Some(upid.to_owned());
+    }
+
+    info.last_run_upid = selected_upid.clone();
+
+    match selected_upid {
+        Some(upid) => {
+            info.removed_chunks = Some(status_in_memory.removed_chunks);
+            info.pending_chunks = Some(status_in_memory.pending_chunks);
+
+            let mut computed_schedule: JobScheduleStatus = JobScheduleStatus::default();
+            let mut duration = None;
+            if let Some(state) = state_file {
+                if let Ok(cs) = compute_schedule_status(&state, info.last_run_upid.as_deref()) {
+                    computed_schedule = cs;
+                }
+            }
+
+            if let Some(endtime) = computed_schedule.last_run_endtime {
+                computed_schedule.next_run = info
+                    .schedule
+                    .as_ref()
+                    .and_then(|s| {
+                        s.parse::<CalendarEvent>()
+                            .map_err(|err| log::error!("{err}"))
+                            .ok()
+                    })
+                    .and_then(|e| {
+                        e.compute_next_event(endtime)
+                            .map_err(|err| log::error!("{err}"))
+                            .ok()
+                    })
+                    .and_then(|ne| ne);
+
+                if let Ok(parsed_upid) = upid.parse::<UPID>() {
+                    duration = Some(endtime - parsed_upid.starttime);
+                }
+            }
+
+            info.next_run = computed_schedule.next_run;
+            info.last_run_endtime = computed_schedule.last_run_endtime;
+            info.last_run_state = computed_schedule.last_run_state;
+            info.duration = duration;
+        }
+        None => {
+            if let Some(schedule) = &info.schedule {
+                info.next_run = schedule
+                    .parse::<CalendarEvent>()
+                    .map_err(|err| log::error!("{err}"))
+                    .ok()
+                    .and_then(|e| {
+                        e.compute_next_event(proxmox_time::epoch_i64())
+                            .map_err(|err| log::error!("{err}"))
+                            .ok()
+                    })
+                    .and_then(|ne| ne);
+
+                if let Ok(event) = schedule.parse::<CalendarEvent>() {
+                    if let Ok(next_event) = event.compute_next_event(proxmox_time::epoch_i64()) {
+                        info.next_run = next_event;
+                    }
+                }
+            } else {
+                return Ok(info);
+            }
+        }
+    }
+
+    Ok(info)
+}
+
 #[api(
     returns: {
         description: "List the accessible datastores.",
@@ -2265,6 +2384,10 @@ const DATASTORE_INFO_SUBDIRS: SubdirMap = &[
             .get(&API_METHOD_GARBAGE_COLLECTION_STATUS)
             .post(&API_METHOD_START_GARBAGE_COLLECTION),
     ),
+    (
+        "gc-job-status",
+        &Router::new().get(&API_METHOD_GARBAGE_COLLECTION_JOB_STATUS),
+    ),
     (
         "group-notes",
         &Router::new()
diff --git a/src/api2/admin/gc.rs b/src/api2/admin/gc.rs
new file mode 100644
index 00000000..7535f369
--- /dev/null
+++ b/src/api2/admin/gc.rs
@@ -0,0 +1,57 @@
+use anyhow::Error;
+use pbs_api_types::GarbageCollectionJobStatus;
+
+use proxmox_router::{ApiMethod, Permission, Router, RpcEnvironment};
+use proxmox_schema::api;
+
+use pbs_api_types::DATASTORE_SCHEMA;
+
+use serde_json::Value;
+
+use crate::api2::admin::datastore::{garbage_collection_job_status, get_datastore_list};
+
+#[api(
+    input: {
+        properties: {
+            store: {
+                schema: DATASTORE_SCHEMA,
+                optional: true,
+            },
+        },
+    },
+    returns: {
+        description: "List configured gc jobs and their status",
+        type: Array,
+        items: { type: GarbageCollectionJobStatus },
+    },
+    access: {
+        permission: &Permission::Anybody,
+        description: "Requires Datastore.Audit or Datastore.Modify on datastore.",
+    },
+)]
+/// List all GC jobs (max one per datastore)
+pub fn list_all_gc_jobs(
+    store: Option<String>,
+    _param: Value,
+    _info: &ApiMethod,
+    rpcenv: &mut dyn RpcEnvironment,
+) -> Result<Vec<GarbageCollectionJobStatus>, Error> {
+    let gc_info = match store {
+        Some(store) => {
+            garbage_collection_job_status(store, _info, rpcenv).map(|info| vec![info])?
+        }
+        None => get_datastore_list(Value::Null, _info, rpcenv)?
+            .into_iter()
+            .map(|store_list_item| store_list_item.store)
+            .filter_map(|store| garbage_collection_job_status(store, _info, rpcenv).ok())
+            .collect::<Vec<_>>(),
+    };
+
+    Ok(gc_info)
+}
+
+const GC_ROUTER: Router = Router::new().get(&API_METHOD_LIST_ALL_GC_JOBS);
+
+pub const ROUTER: Router = Router::new()
+    .get(&API_METHOD_LIST_ALL_GC_JOBS)
+    .match_all("store", &GC_ROUTER);
diff --git a/src/api2/admin/mod.rs b/src/api2/admin/mod.rs
index 168dc038..a1c49f8e 100644
--- a/src/api2/admin/mod.rs
+++ b/src/api2/admin/mod.rs
@@ -5,6 +5,7 @@ use proxmox_router::{Router, SubdirMap};
 use proxmox_sortable_macro::sortable;
 
 pub mod datastore;
+pub mod gc;
 pub mod metrics;
 pub mod namespace;
 pub mod prune;
@@ -17,6 +18,7 @@ const SUBDIRS: SubdirMap = &sorted!([
     ("datastore", &datastore::ROUTER),
     ("metrics", &metrics::ROUTER),
     ("prune", &prune::ROUTER),
+    ("gc", &gc::ROUTER),
     ("sync", &sync::ROUTER),
     ("traffic-control", &traffic_control::ROUTER),
     ("verify", &verify::ROUTER),
-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 2/8] gc: global prune and gc job view

[Stefan Lendl]

In the global datastore view, extend the prune view to display gc job
status as a table.  Use the same widget in the local view and dispaly gc
job status as a single row.

The local PruneAndGC view is parameterized (cbind) with the datastore.
At initialization the only row is selected.  This allows the rest of the
grid to act on selected rows and it requires far less special casing if
the datastore is set on the view or not.

Having a single row always selected and therefore highlighted, is
visually not appealing.  Therefore, highlighting of selected rows is
disabled in the local view.

Moved GCView to different file and enhanced it with last, next run,
status and duration. Added button to show task log.

Changed `render_task_status()` to also take in account upids stored in
other 'columns'.

Originally-by: Gabriel Goller <g.goller@proxmox.com>
Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
---
 www/Makefile                   |   2 +
 www/Utils.js                   |   6 +-
 www/config/GCView.js           | 207 +++++++++++++++++++++++++++++++++
 www/datastore/DataStoreList.js |   4 +-
 www/datastore/Panel.js         |   1 -
 www/datastore/PruneAndGC.js    | 101 ++--------------
 www/window/GCJobEdit.js        |  28 +++++
 7 files changed, 252 insertions(+), 97 deletions(-)
 create mode 100644 www/config/GCView.js
 create mode 100644 www/window/GCJobEdit.js

diff --git a/www/Makefile b/www/Makefile
index c2755ac8..03263588 100644
--- a/www/Makefile
+++ b/www/Makefile
@@ -63,6 +63,7 @@ JSSRC=							\
 	config/SyncView.js				\
 	config/VerifyView.js				\
 	config/PruneView.js				\
+	config/GCView.js				\
 	config/WebauthnView.js				\
 	config/CertificateView.js			\
 	config/NodeOptionView.js			\
@@ -79,6 +80,7 @@ JSSRC=							\
 	window/NotifyOptions.js				\
 	window/SyncJobEdit.js				\
 	window/PruneJobEdit.js				\
+	window/GCJobEdit.js				\
 	window/UserEdit.js				\
 	window/UserPassword.js				\
 	window/Settings.js				\
diff --git a/www/Utils.js b/www/Utils.js
index 5357949b..acd6e0d8 100644
--- a/www/Utils.js
+++ b/www/Utils.js
@@ -199,12 +199,12 @@ Ext.define('PBS.Utils', {
 	return fingerprint.substring(0, 23);
     },
 
-    render_task_status: function(value, metadata, record) {
-	if (!record.data['last-run-upid']) {
+    render_task_status: function(value, metadata, record, rowIndex, colIndex, store) {
+	if (!record.data['last-run-upid'] && !store.getById('last-run-upid')?.data.value) {
 	    return '-';
 	}
 
-	if (!record.data['last-run-endtime']) {
+	if (!record.data['last-run-endtime'] && !store.getById('last-run-endtime')?.data.value) {
 	    metadata.tdCls = 'x-grid-row-loading';
 	    return '';
 	}
diff --git a/www/config/GCView.js b/www/config/GCView.js
new file mode 100644
index 00000000..6c9e1e23
--- /dev/null
+++ b/www/config/GCView.js
@@ -0,0 +1,207 @@
+Ext.define('pbs-gc-jobs-status', {
+    extend: 'Ext.data.Model',
+    fields: [
+	'store', 'last-run-upid', 'removed-chunks', 'pending-chunks', 'schedule',
+	'next-run', 'last-run-endtime', 'last-run-state',
+	{
+	    name: 'duration',
+	    calculate: function(data) {
+		let endtime = data['last-run-endtime'];
+		if (!endtime) return undefined;
+		let task = Proxmox.Utils.parse_task_upid(data['last-run-upid']);
+		return endtime - task.starttime;
+	    },
+	},
+    ],
+    idProperty: 'store',
+    proxy: {
+	type: 'proxmox',
+	url: '/api2/json/admin/gc',
+    },
+});
+
+Ext.define('PBS.config.GCJobView', {
+    extend: 'Ext.grid.GridPanel',
+    alias: 'widget.pbsGCJobView',
+
+    stateful: true,
+    stateId: 'grid-gc-jobs-v1',
+    allowDeselect: false,
+
+    title: gettext('Garbage Collect Jobs'),
+
+    controller: {
+	xclass: 'Ext.app.ViewController',
+
+	init: function(view) {
+	    let params = {};
+	    let store = view.getStore();
+	    let proxy = store.rstore.getProxy();
+	    if (view.datastore) {
+		params.store = view.datastore;
+
+		// after the store is loaded, select the row to enable the Edit,.. buttons
+		store.rstore.proxy.on({
+		    'afterload': {
+			fn: () => view.getSelectionModel().select(0),
+			single: true,
+		    },
+		});
+
+		// do not highlight the selected row
+		view.items.items[0].selectedItemCls = '';
+		view.items.items[0].overItemCls = '';
+	    }
+	    proxy.setExtraParams(params);
+	    Proxmox.Utils.monStoreErrors(view, store.rstore);
+	},
+
+	getDatastoreName: function() {
+	    return this.getView().getSelection()[0]?.data.store;
+	},
+
+	getData: function() {
+	    let view = this.getView();
+	    let datastore = this.getDatastoreName();
+	    return view.getStore().getById(datastore).data;
+	},
+
+	editGCJob: function() {
+	    let data = this.getData();
+	    Ext.create('PBS.window.GCJobEdit', {
+		datastore: data.store,
+		id: data.store,
+		schedule: data.schedule,
+		listeners: {
+		    destroy: () => this.reload(),
+		},
+	    }).show();
+	},
+
+	garbageCollect: function() {
+	    let datastore = this.getDatastoreName();
+	    Proxmox.Utils.API2Request({
+		url: `/admin/datastore/${datastore}/gc`,
+		method: 'POST',
+		failure: function(response) {
+		    Ext.Msg.alert(gettext('Error'), response.htmlStatus);
+		},
+		success: function(response, options) {
+		    Ext.create('Proxmox.window.TaskViewer', {
+			upid: response.result.data,
+		    }).show();
+		},
+	    });
+	},
+
+	showTaskLog: function() {
+	    let me = this;
+
+	    let upid = this.getData()['last-run-upid'];
+	    if (!upid) return;
+
+	    Ext.create('Proxmox.window.TaskViewer', { upid }).show();
+	},
+
+	startStore: function() { this.getView().getStore().rstore.startUpdate(); },
+	stopStore: function() { this.getView().getStore().rstore.stopUpdate(); },
+	reload: function() { this.getView().getStore().rstore.load(); },
+
+    },
+
+    listeners: {
+	activate: 'startStore',
+	destroy: 'stopStore',
+	deactivate: 'stopStore',
+	itemdblclick: 'editGCJob',
+    },
+
+    store: {
+	type: 'diff',
+	autoDestroy: true,
+	autoDestroyRstore: true,
+	sorters: 'store',
+	rstore: {
+	    type: 'update',
+	    storeid: 'pbs-gc-jobs-status',
+	    model: 'pbs-gc-jobs-status',
+	    interval: 5000,
+	},
+    },
+
+    tbar: [
+	{
+	    xtype: 'proxmoxButton',
+	    text: gettext('Edit'),
+	    handler: 'editGCJob',
+	    enableFn: (rec) => !!rec,
+	    disabled: true,
+	},
+	'-',
+	{
+	    xtype: 'proxmoxButton',
+	    text: gettext('Show Log'),
+	    handler: 'showTaskLog',
+	    enableFn: (rec) => !!rec.data["last-run-upid"],
+	    disabled: true,
+	},
+	{
+	    xtype: 'proxmoxButton',
+	    text: gettext('Run now'),
+	    handler: 'garbageCollect',
+	    enableFn: (rec) => !!rec,
+	    disabled: true,
+	},
+    ],
+
+    columns: [
+	{
+	    header: gettext('Datastore'),
+	    dataIndex: 'store',
+	    renderer: Ext.String.htmlEncode,
+	    width: 120,
+	    sortable: true,
+	    hideable: false,
+	},
+	{
+	    header: gettext('Schedule'),
+	    dataIndex: 'schedule',
+	    maxWidth: 220,
+	    minWidth: 80,
+	    flex: 1,
+	    sortable: false,
+	    hideable: false,
+	    renderer: (value) => value ? value : Proxmox.Utils.NoneText,
+	},
+	{
+	    header: gettext('Last GC'),
+	    dataIndex: 'last-run-endtime',
+	    renderer: PBS.Utils.render_optional_timestamp,
+	    minWidth: 150,
+	    sortable: true,
+	},
+	{
+	    text: gettext('Duration'),
+	    dataIndex: 'duration',
+	    renderer: Proxmox.Utils.render_duration,
+	    sortable: false,
+	    width: 80,
+	},
+	{
+	    header: gettext('Last Status'),
+	    dataIndex: 'last-run-state',
+	    renderer: PBS.Utils.render_task_status,
+	    sortable: true,
+	    flex: 3,
+	    maxWidth: 100,
+	    minWidth: 80,
+	},
+	{
+	    header: gettext('Next Run'),
+	    dataIndex: 'next-run',
+	    renderer: PBS.Utils.render_next_task_run,
+	    width: 150,
+	    sortable: true,
+	},
+    ],
+});
diff --git a/www/datastore/DataStoreList.js b/www/datastore/DataStoreList.js
index b496bcbc..a4b77dbf 100644
--- a/www/datastore/DataStoreList.js
+++ b/www/datastore/DataStoreList.js
@@ -239,8 +239,8 @@ Ext.define('PBS.datastore.DataStores', {
 	},
 	{
 	    iconCls: 'fa fa-trash-o',
-	    itemId: 'prunejobs',
-	    xtype: 'pbsPruneJobView',
+	    itemId: 'prunegc',
+	    xtype: 'pbsDatastorePruneAndGC',
 	},
 	{
 	    iconCls: 'fa fa-check-circle',
diff --git a/www/datastore/Panel.js b/www/datastore/Panel.js
index fd1b4611..0fc97d14 100644
--- a/www/datastore/Panel.js
+++ b/www/datastore/Panel.js
@@ -58,7 +58,6 @@ Ext.define('PBS.DataStorePanel', {
 	    },
 	},
 	{
-	    title: gettext('Prune & GC'),
 	    xtype: 'pbsDatastorePruneAndGC',
 	    itemId: 'prunegc',
 	    iconCls: 'fa fa-trash-o',
diff --git a/www/datastore/PruneAndGC.js b/www/datastore/PruneAndGC.js
index aab98dad..33ca0108 100644
--- a/www/datastore/PruneAndGC.js
+++ b/www/datastore/PruneAndGC.js
@@ -1,91 +1,8 @@
-Ext.define('PBS.Datastore.GCOptions', {
-    extend: 'Proxmox.grid.ObjectGrid',
-    alias: 'widget.pbsDatastoreGCOpts',
-    mixins: ['Proxmox.Mixin.CBind'],
-
-    onlineHelp: 'maintenance_pruning',
-
-    cbindData: function(initial) {
-	let me = this;
-
-	me.datastore = encodeURIComponent(me.datastore);
-	me.url = `/api2/json/config/datastore/${me.datastore}`;
-	me.editorConfig = {
-	    url: `/api2/extjs/config/datastore/${me.datastore}`,
-	};
-	return {};
-    },
-
-    controller: {
-	xclass: 'Ext.app.ViewController',
-
-	edit: function() { this.getView().run_editor(); },
-
-	garbageCollect: function() {
-	    let me = this;
-	    let view = me.getView();
-	    Proxmox.Utils.API2Request({
-		url: `/admin/datastore/${view.datastore}/gc`,
-		method: 'POST',
-		failure: function(response) {
-		    Ext.Msg.alert(gettext('Error'), response.htmlStatus);
-		},
-		success: function(response, options) {
-		    Ext.create('Proxmox.window.TaskViewer', {
-			upid: response.result.data,
-		    }).show();
-		},
-	    });
-	},
-    },
-
-    tbar: [
-	{
-	    xtype: 'proxmoxButton',
-	    text: gettext('Edit'),
-	    disabled: true,
-	    handler: 'edit',
-	},
-	'-',
-	{
-	    xtype: 'proxmoxButton',
-	    text: gettext('Start Garbage Collection'),
-	    selModel: null,
-	    handler: 'garbageCollect',
-	},
-    ],
-
-    listeners: {
-	activate: function() { this.rstore.startUpdate(); },
-	destroy: function() { this.rstore.stopUpdate(); },
-	deactivate: function() { this.rstore.stopUpdate(); },
-	itemdblclick: 'edit',
-    },
-
-    rows: {
-	"gc-schedule": {
-	    required: true,
-	    defaultValue: Proxmox.Utils.NoneText,
-	    header: gettext('Garbage Collection Schedule'),
-	    editor: {
-		xtype: 'proxmoxWindowEdit',
-		title: gettext('GC Schedule'),
-		onlineHelp: 'maintenance_gc',
-		items: {
-		    xtype: 'pbsCalendarEvent',
-		    name: 'gc-schedule',
-		    fieldLabel: gettext("GC Schedule"),
-		    emptyText: Proxmox.Utils.noneText,
-		    deleteEmpty: true,
-		},
-	    },
-	},
-    },
-});
-
 Ext.define('PBS.Datastore.PruneAndGC', {
     extend: 'Ext.panel.Panel',
     alias: 'widget.pbsDatastorePruneAndGC',
+    title: gettext('Prune & GC Jobs'),
+
     mixins: ['Proxmox.Mixin.CBind'],
 
     layout: {
@@ -99,9 +16,8 @@ Ext.define('PBS.Datastore.PruneAndGC', {
     },
     items: [
 	{
-	    xtype: 'pbsDatastoreGCOpts',
-	    title: gettext('Garbage Collection'),
-	    itemId: 'datastore-gc',
+	    xtype: 'pbsGCJobView',
+	    itemId: 'gcjobs',
 	    nodename: 'localhost',
 	    cbind: {
 		datastore: '{datastore}',
@@ -110,9 +26,7 @@ Ext.define('PBS.Datastore.PruneAndGC', {
 	{
 	    xtype: 'pbsPruneJobView',
 	    nodename: 'localhost',
-	    itemId: 'datastore-prune-jobs',
-	    flex: 1,
-	    minHeight: 200,
+	    itemId: 'prunejobs',
 	    cbind: {
 		datastore: '{datastore}',
 	    },
@@ -130,4 +44,9 @@ Ext.define('PBS.Datastore.PruneAndGC', {
 	    component.relayEvents(me, ['activate', 'deactivate', 'destroy']);
 	}
     },
+
+    cbindData: function(initalConfig) {
+        let me = this;
+        me.datastore = initalConfig.datastore ? initalConfig.datastore : undefined;
+    },
 });
diff --git a/www/window/GCJobEdit.js b/www/window/GCJobEdit.js
new file mode 100644
index 00000000..de74bf5c
--- /dev/null
+++ b/www/window/GCJobEdit.js
@@ -0,0 +1,28 @@
+Ext.define('PBS.window.GCJobEdit', {
+    extend: 'Proxmox.window.Edit',
+    alias: 'widget.pbsGCJobEdit',
+    mixins: ['Proxmox.Mixin.CBind'],
+
+    userid: undefined,
+    onlineHelp: 'maintenance_gc',
+    isAdd: false,
+
+    subject: gettext('Garbage Collect Schedule'),
+
+    cbindData: function(initial) {
+        let me = this;
+
+        me.datastore = encodeURIComponent(me.datastore);
+	me.url = `/api2/extjs/config/datastore/${me.datastore}`;
+        me.method = 'PUT';
+        me.autoLoad = true;
+	return {};
+    },
+
+    items: {
+	xtype: 'pbsCalendarEvent',
+	name: 'gc-schedule',
+	fieldLabel: gettext("GC Schedule"),
+	emptyText: gettext(Proxmox.Utils.NoneText + " (disabled)"),
+    },
+});
-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 3/8] gc: move datastore/PruneAndGC to config/PruneAndGC

[Stefan Lendl]

renaming the widgets to PBS.config.PruneAndGC

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
---
 www/Makefile                            | 2 +-
 www/{datastore => config}/PruneAndGC.js | 4 ++--
 www/datastore/DataStoreList.js          | 2 +-
 www/datastore/Panel.js                  | 2 +-
 4 files changed, 5 insertions(+), 5 deletions(-)
 rename www/{datastore => config}/PruneAndGC.js (92%)

diff --git a/www/Makefile b/www/Makefile
index 03263588..cbfe7213 100644
--- a/www/Makefile
+++ b/www/Makefile
@@ -68,6 +68,7 @@ JSSRC=							\
 	config/CertificateView.js			\
 	config/NodeOptionView.js			\
 	config/MetricServerView.js			\
+	config/PruneAndGC.js				\
 	window/ACLEdit.js				\
 	window/BackupGroupChangeOwner.js		\
 	window/CreateDirectory.js			\
@@ -107,7 +108,6 @@ JSSRC=							\
 	Subscription.js					\
 	datastore/Summary.js				\
 	datastore/Notes.js				\
-	datastore/PruneAndGC.js				\
 	datastore/Prune.js				\
 	datastore/Content.js				\
 	datastore/OptionView.js				\
diff --git a/www/datastore/PruneAndGC.js b/www/config/PruneAndGC.js
similarity index 92%
rename from www/datastore/PruneAndGC.js
rename to www/config/PruneAndGC.js
index 33ca0108..a1163402 100644
--- a/www/datastore/PruneAndGC.js
+++ b/www/config/PruneAndGC.js
@@ -1,6 +1,6 @@
-Ext.define('PBS.Datastore.PruneAndGC', {
+Ext.define('PBS.config.PruneAndGC', {
     extend: 'Ext.panel.Panel',
-    alias: 'widget.pbsDatastorePruneAndGC',
+    alias: 'widget.pbsPruneAndGC',
     title: gettext('Prune & GC Jobs'),
 
     mixins: ['Proxmox.Mixin.CBind'],
diff --git a/www/datastore/DataStoreList.js b/www/datastore/DataStoreList.js
index a4b77dbf..a31a9b4b 100644
--- a/www/datastore/DataStoreList.js
+++ b/www/datastore/DataStoreList.js
@@ -240,7 +240,7 @@ Ext.define('PBS.datastore.DataStores', {
 	{
 	    iconCls: 'fa fa-trash-o',
 	    itemId: 'prunegc',
-	    xtype: 'pbsDatastorePruneAndGC',
+	    xtype: 'pbsPruneAndGC',
 	},
 	{
 	    iconCls: 'fa fa-check-circle',
diff --git a/www/datastore/Panel.js b/www/datastore/Panel.js
index 0fc97d14..0ee38a1b 100644
--- a/www/datastore/Panel.js
+++ b/www/datastore/Panel.js
@@ -58,7 +58,7 @@ Ext.define('PBS.DataStorePanel', {
 	    },
 	},
 	{
-	    xtype: 'pbsDatastorePruneAndGC',
+	    xtype: 'pbsPruneAndGC',
 	    itemId: 'prunegc',
 	    iconCls: 'fa fa-trash-o',
 	    cbind: {
-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 4/8] gc: hide datastore column in local gc view

[Stefan Lendl]

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
---
 www/config/GCView.js | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/www/config/GCView.js b/www/config/GCView.js
index 6c9e1e23..63e84111 100644
--- a/www/config/GCView.js
+++ b/www/config/GCView.js
@@ -204,4 +204,18 @@ Ext.define('PBS.config.GCJobView', {
 	    sortable: true,
 	},
     ],
+
+    initComponent: function() {
+	let me = this;
+	let hideLocalDatastore = !!me.datastore;
+
+	for (let column of me.columns) {
+	    if (column.dataIndex === 'store') {
+		column.hidden = hideLocalDatastore;
+		break;
+	    }
+	}
+
+	me.callParent();
+    },
 });
-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 5/8] ui: order Prune&GC before SyncJobs

[Stefan Lendl]

make the order identical to local datastore view

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
---
 www/datastore/DataStoreList.js | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/www/datastore/DataStoreList.js b/www/datastore/DataStoreList.js
index a31a9b4b..fc68cfc1 100644
--- a/www/datastore/DataStoreList.js
+++ b/www/datastore/DataStoreList.js
@@ -231,17 +231,16 @@ Ext.define('PBS.datastore.DataStores', {
 	    xtype: 'pbsDataStoreList',
 	    iconCls: 'fa fa-book',
 	},
-
-	{
-	    iconCls: 'fa fa-refresh',
-	    itemId: 'syncjobs',
-	    xtype: 'pbsSyncJobView',
-	},
 	{
 	    iconCls: 'fa fa-trash-o',
 	    itemId: 'prunegc',
 	    xtype: 'pbsPruneAndGC',
 	},
+	{
+	    iconCls: 'fa fa-refresh',
+	    itemId: 'syncjobs',
+	    xtype: 'pbsSyncJobView',
+	},
 	{
 	    iconCls: 'fa fa-check-circle',
 	    itemId: 'verifyjobs',
-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 6/8] cli: list gc jobs with proxmox-backup-manager

[Stefan Lendl]

proxmox-backup-manager garbage-collection list
  to list the garbage collection job status for all datastores,
  including datastores without gc jobs.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
---
 src/bin/proxmox-backup-manager.rs | 33 +++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/src/bin/proxmox-backup-manager.rs b/src/bin/proxmox-backup-manager.rs
index 115207f3..94fc6b2f 100644
--- a/src/bin/proxmox-backup-manager.rs
+++ b/src/bin/proxmox-backup-manager.rs
@@ -93,6 +93,35 @@ async fn garbage_collection_status(param: Value) -> Result<Value, Error> {
     Ok(Value::Null)
 }
 
+#[api(
+   input: {
+        properties: {
+            "output-format": {
+                schema: OUTPUT_FORMAT,
+                optional: true,
+            },
+        }
+   }
+)]
+/// List garbage collection job status for all datastores, including datastores without gc jobs.
+async fn garbage_collection_list_jobs(param: Value) -> Result<Value, Error> {
+    let output_format = get_output_format(&param);
+
+    let client = connect_to_localhost()?;
+
+    let path = "api2/json/admin/gc";
+
+    let mut result = client.get(&path, None).await?;
+    let mut data = result["data"].take();
+    let return_type = &api2::admin::gc::API_METHOD_LIST_ALL_GC_JOBS.returns;
+
+    let options = default_table_format_options();
+
+    format_and_print_result_full(&mut data, return_type, &output_format, &options);
+
+    Ok(Value::Null)
+}
+
 fn garbage_collection_commands() -> CommandLineInterface {
     let cmd_def = CliCommandMap::new()
         .insert(
@@ -106,6 +135,10 @@ fn garbage_collection_commands() -> CommandLineInterface {
             CliCommand::new(&API_METHOD_START_GARBAGE_COLLECTION)
                 .arg_param(&["store"])
                 .completion_cb("store", pbs_config::datastore::complete_datastore_name),
+        )
+        .insert(
+            "list",
+            CliCommand::new(&API_METHOD_GARBAGE_COLLECTION_LIST_JOBS),
         );
 
     cmd_def.into()
-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 7/8] gc: show removed and pending chunks of last run in ui

[Stefan Lendl]

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
---
 www/config/GCView.js | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/www/config/GCView.js b/www/config/GCView.js
index 63e84111..d1f1bf71 100644
--- a/www/config/GCView.js
+++ b/www/config/GCView.js
@@ -203,6 +203,20 @@ Ext.define('PBS.config.GCJobView', {
 	    width: 150,
 	    sortable: true,
 	},
+	{
+	    header: gettext('Removed Chunks'),
+	    dataIndex: 'removed-chunks',
+	    renderer: (value) => value !== undefined ? value : "-",
+	    maxWidth: 100,
+	    sortable: false,
+	},
+	{
+	    header: gettext('Pending Chunks'),
+	    dataIndex: 'pending-chunks',
+	    renderer: (value) => value !== undefined ? value : "-",
+	    maxWidth: 100,
+	    sortable: false,
+	},
     ],
 
     initComponent: function() {
-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 8/8] gc: configure width and flex on GC Jobs columns

[Stefan Lendl]

table expands to the full width and relevant data is still visible on a
narrow screen.

Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
---
 www/config/GCView.js | 30 +++++++++++++++++++-----------
 1 file changed, 19 insertions(+), 11 deletions(-)

diff --git a/www/config/GCView.js b/www/config/GCView.js
index d1f1bf71..0e49a0ac 100644
--- a/www/config/GCView.js
+++ b/www/config/GCView.js
@@ -159,63 +159,71 @@ Ext.define('PBS.config.GCJobView', {
 	    header: gettext('Datastore'),
 	    dataIndex: 'store',
 	    renderer: Ext.String.htmlEncode,
-	    width: 120,
 	    sortable: true,
 	    hideable: false,
+	    width: 150,
+	    minWidth: 120,
+	    maxWidth: 300,
+	    flex: 2,
 	},
 	{
 	    header: gettext('Schedule'),
 	    dataIndex: 'schedule',
-	    maxWidth: 220,
-	    minWidth: 80,
-	    flex: 1,
 	    sortable: false,
 	    hideable: false,
 	    renderer: (value) => value ? value : Proxmox.Utils.NoneText,
+	    width: 85,
+	    minWidth: 85,
+	    flex: 1,
 	},
 	{
 	    header: gettext('Last GC'),
 	    dataIndex: 'last-run-endtime',
 	    renderer: PBS.Utils.render_optional_timestamp,
-	    minWidth: 150,
 	    sortable: true,
+	    minWidth: 150,
+	    flex: 1,
 	},
 	{
 	    text: gettext('Duration'),
 	    dataIndex: 'duration',
 	    renderer: Proxmox.Utils.render_duration,
 	    sortable: false,
-	    width: 80,
+	    minWidth: 80,
+	    flex: 1,
 	},
 	{
 	    header: gettext('Last Status'),
 	    dataIndex: 'last-run-state',
 	    renderer: PBS.Utils.render_task_status,
 	    sortable: true,
-	    flex: 3,
-	    maxWidth: 100,
+	    width: 100,
 	    minWidth: 80,
+	    flex: 1,
 	},
 	{
 	    header: gettext('Next Run'),
 	    dataIndex: 'next-run',
 	    renderer: PBS.Utils.render_next_task_run,
-	    width: 150,
 	    sortable: true,
+	    minWidth: 150,
+	    flex: 1,
 	},
 	{
 	    header: gettext('Removed Chunks'),
 	    dataIndex: 'removed-chunks',
 	    renderer: (value) => value !== undefined ? value : "-",
-	    maxWidth: 100,
 	    sortable: false,
+	    minWidth: 85,
+	    flex: 1,
 	},
 	{
 	    header: gettext('Pending Chunks'),
 	    dataIndex: 'pending-chunks',
 	    renderer: (value) => value !== undefined ? value : "-",
-	    maxWidth: 100,
 	    sortable: false,
+	    minWidth: 80,
+	    flex: 3,
 	},
     ],
 
-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 0/8] Add GC job status to datastore and global prune job view.

[Stefan Lendl]

From: Stefan Lendl <s.lendl@proxmox.com>

Fix #3217: Addition of a new tab "Prune & GC" in "Datastore" and not in each Datastore created
Fix #4723: add last, next run, status, duration to gc

Extends the garbage collection view to display in addition to the schedule:
 - State (of last run)
 - Duration (of last run)
 - Last Run Date
 - Next Run Date (if scheduled)
 - Removed Chunks (in last run)
 - Pending Chunks (as of last run)

In the Datastore global overview, the prune view is extended to show the same
details for all availible datastores also the ones without a gc-schedule.

Allows editing the schedule, showing the log of the last run and manually
running the gc job.  In the global view, by selecting the row of the datastore.

Adds a proxmox-backup-manager cli command to list all gc jobs
`proxmox-backup-manager garbage-collection list`


Changes v2 -> v3:
 - fixed indentation (how hard can it be?..)
 - added git trailers

Changes v1 -> v2:
 - sort imports
 - fix eslint warnings
 - update columns in GC Job view to fill the entire width
 - not include path PruneJobEdit (sent separatly)

This is based on a series from g.goller
Changes to g.goller's series:
 - rename endpoint from gc-info to gc-job-status
 - add list-all-gc-jobs endpoint
 - UI uses Grid (table) view instead of model grid
 - implement GC job view in global view

Stefan Lendl (8):
  api: garbage collect job status
  gc: global prune and gc job view
  gc: move datastore/PruneAndGC to config/PruneAndGC
  gc: hide datastore column in local gc view
  ui: order Prune&GC before SyncJobs
  cli: list gc jobs with proxmox-backup-manager
  gc: show removed and pending chunks of last run in ui
  gc: configure width and flex on GC Jobs columns

 pbs-api-types/src/datastore.rs    |  40 +++++
 src/api2/admin/datastore.rs       | 129 +++++++++++++++-
 src/api2/admin/gc.rs              |  57 +++++++
 src/api2/admin/mod.rs             |   2 +
 src/bin/proxmox-backup-manager.rs |  33 ++++
 www/Makefile                      |   4 +-
 www/Utils.js                      |   6 +-
 www/config/GCView.js              | 243 ++++++++++++++++++++++++++++++
 www/config/PruneAndGC.js          |  52 +++++++
 www/datastore/DataStoreList.js    |  11 +-
 www/datastore/Panel.js            |   3 +-
 www/datastore/PruneAndGC.js       | 133 ----------------
 www/window/GCJobEdit.js           |  28 ++++
 13 files changed, 593 insertions(+), 148 deletions(-)
 create mode 100644 src/api2/admin/gc.rs
 create mode 100644 www/config/GCView.js
 create mode 100644 www/config/PruneAndGC.js
 delete mode 100644 www/datastore/PruneAndGC.js
 create mode 100644 www/window/GCJobEdit.js

-- 
2.43.0

[pbs-devel] [PATCH v3 proxmox-backup 0/8] Add GC job status to datastore and global prune job view.

[Stefan Lendl]

From: Stefan Lendl <s.lendl@proxmox.com>

*Adding the missing Cover Letter here*

Fix #3217: Addition of a new tab "Prune & GC" in "Datastore" and not in each Datastore created
Fix #4723: add last, next run, status, duration to gc

Extends the garbage collection view to display in addition to the schedule:
 - State (of last run)
 - Duration (of last run)
 - Last Run Date
 - Next Run Date (if scheduled)
 - Removed Chunks (in last run)
 - Pending Chunks (as of last run)

In the Datastore global overview, the prune view is extended to show the same
details for all availible datastores also the ones without a gc-schedule.

Allows editing the schedule, showing the log of the last run and manually
running the gc job.  In the global view, by selecting the row of the datastore.

Adds a proxmox-backup-manager cli command to list all gc jobs
`proxmox-backup-manager garbage-collection list`


Changes v2 -> v3:
 - fixed indentation (how hard can it be?..)
 - added git trailers

Changes v1 -> v2:
 - sort imports
 - fix eslint warnings
 - update columns in GC Job view to fill the entire width
 - not include path PruneJobEdit (sent separatly)

This is based on a series from g.goller
Changes to g.goller's series:
 - rename endpoint from gc-info to gc-job-status
 - add list-all-gc-jobs endpoint
 - UI uses Grid (table) view instead of model grid
 - implement GC job view in global view

Stefan Lendl (8):
  api: garbage collect job status
  gc: global prune and gc job view
  gc: move datastore/PruneAndGC to config/PruneAndGC
  gc: hide datastore column in local gc view
  ui: order Prune&GC before SyncJobs
  cli: list gc jobs with proxmox-backup-manager
  gc: show removed and pending chunks of last run in ui
  gc: configure width and flex on GC Jobs columns

 pbs-api-types/src/datastore.rs    |  40 +++++
 src/api2/admin/datastore.rs       | 129 +++++++++++++++-
 src/api2/admin/gc.rs              |  57 +++++++
 src/api2/admin/mod.rs             |   2 +
 src/bin/proxmox-backup-manager.rs |  33 ++++
 www/Makefile                      |   4 +-
 www/Utils.js                      |   6 +-
 www/config/GCView.js              | 243 ++++++++++++++++++++++++++++++
 www/config/PruneAndGC.js          |  52 +++++++
 www/datastore/DataStoreList.js    |  11 +-
 www/datastore/Panel.js            |   3 +-
 www/datastore/PruneAndGC.js       | 133 ----------------
 www/window/GCJobEdit.js           |  28 ++++
 13 files changed, 593 insertions(+), 148 deletions(-)
 create mode 100644 src/api2/admin/gc.rs
 create mode 100644 www/config/GCView.js
 create mode 100644 www/config/PruneAndGC.js
 delete mode 100644 www/datastore/PruneAndGC.js
 create mode 100644 www/window/GCJobEdit.js

-- 
2.43.0

Re: [pbs-devel] [PATCH v3 proxmox-backup 0/8] Add GC job status to datastore and global prune job view.

[Stefan Lendl]

ping, still applies.

> From: Stefan Lendl <s.lendl@proxmox.com>
>
> Fix #3217: Addition of a new tab "Prune & GC" in "Datastore" and not in each Datastore created
> Fix #4723: add last, next run, status, duration to gc
>
> Extends the garbage collection view to display in addition to the schedule:
>  - State (of last run)
>  - Duration (of last run)
>  - Last Run Date
>  - Next Run Date (if scheduled)
>  - Removed Chunks (in last run)
>  - Pending Chunks (as of last run)
>
> In the Datastore global overview, the prune view is extended to show the same
> details for all availible datastores also the ones without a gc-schedule.
>
> Allows editing the schedule, showing the log of the last run and manually
> running the gc job.  In the global view, by selecting the row of the datastore.
>
> Adds a proxmox-backup-manager cli command to list all gc jobs
> `proxmox-backup-manager garbage-collection list`
>
>
> Changes v2 -> v3:
>  - fixed indentation (how hard can it be?..)
>  - added git trailers
>
> Changes v1 -> v2:
>  - sort imports
>  - fix eslint warnings
>  - update columns in GC Job view to fill the entire width
>  - not include path PruneJobEdit (sent separatly)
>
> This is based on a series from g.goller
> Changes to g.goller's series:
>  - rename endpoint from gc-info to gc-job-status
>  - add list-all-gc-jobs endpoint
>  - UI uses Grid (table) view instead of model grid
>  - implement GC job view in global view
>
> Stefan Lendl (8):
>   api: garbage collect job status
>   gc: global prune and gc job view
>   gc: move datastore/PruneAndGC to config/PruneAndGC
>   gc: hide datastore column in local gc view
>   ui: order Prune&GC before SyncJobs
>   cli: list gc jobs with proxmox-backup-manager
>   gc: show removed and pending chunks of last run in ui
>   gc: configure width and flex on GC Jobs columns
>
>  pbs-api-types/src/datastore.rs    |  40 +++++
>  src/api2/admin/datastore.rs       | 129 +++++++++++++++-
>  src/api2/admin/gc.rs              |  57 +++++++
>  src/api2/admin/mod.rs             |   2 +
>  src/bin/proxmox-backup-manager.rs |  33 ++++
>  www/Makefile                      |   4 +-
>  www/Utils.js                      |   6 +-
>  www/config/GCView.js              | 243 ++++++++++++++++++++++++++++++
>  www/config/PruneAndGC.js          |  52 +++++++
>  www/datastore/DataStoreList.js    |  11 +-
>  www/datastore/Panel.js            |   3 +-
>  www/datastore/PruneAndGC.js       | 133 ----------------
>  www/window/GCJobEdit.js           |  28 ++++
>  13 files changed, 593 insertions(+), 148 deletions(-)
>  create mode 100644 src/api2/admin/gc.rs
>  create mode 100644 www/config/GCView.js
>  create mode 100644 www/config/PruneAndGC.js
>  delete mode 100644 www/datastore/PruneAndGC.js
>  create mode 100644 www/window/GCJobEdit.js

Re: [pbs-devel] [PATCH v3 proxmox-backup 1/8] api: garbage collect job status

[Lukas Wagner]

On  2024-02-08 14:59, Stefan Lendl wrote:
> Adds an api endpoint on the datastore that reports the gc job status
> such as:
>  - Schedule
>  - State (of last run)
>  - Duration (of last run)
>  - Last Run
>  - Next Run (if scheduled)
>  - Pending Chunks (of last run)
>  - Removed Chunks (of last run)

From a user's perspective I think it would make more sense to report
bytes, not chunks, especially since we deal with non-constant chunk
sizes.

> 
> Adds a dedicated endpoint admin/gc that reports gc job status for all
> datastores including the onces without a gc-schedule.
> 
> Originally-by: Gabriel Goller <g.goller@proxmox.com>
> Signed-off-by: Stefan Lendl <s.lendl@proxmox.com>
> Tested-by: Gabriel Goller <g.goller@proxmox.com>
> Reviewd-by: Gabriel Goller <g.goller@proxmox.com>
        ^ typo, also applies to the other patches

> ---
>  pbs-api-types/src/datastore.rs |  40 ++++++++++
>  src/api2/admin/datastore.rs    | 129 ++++++++++++++++++++++++++++++++-
>  src/api2/admin/gc.rs           |  57 +++++++++++++++
>  src/api2/admin/mod.rs          |   2 +
>  4 files changed, 225 insertions(+), 3 deletions(-)
>  create mode 100644 src/api2/admin/gc.rs
> 
> diff --git a/pbs-api-types/src/datastore.rs b/pbs-api-types/src/datastore.rs
> index cce9888b..ba3879c9 100644
> --- a/pbs-api-types/src/datastore.rs
> +++ b/pbs-api-types/src/datastore.rs
> @@ -1270,6 +1270,46 @@ pub struct GarbageCollectionStatus {
>      pub still_bad: usize,
>  }
>  
> +#[api(
> +    properties: {
> +        "last-run-upid": {
> +            optional: true,
> +            type: UPID,
> +        },
> +    },
> +)]
> +#[derive(Clone, Debug, Default, Serialize, Deserialize, PartialEq)]
> +#[serde(rename_all = "kebab-case")]
> +/// Garbage Collection general info
> +pub struct GarbageCollectionJobStatus {
> +    /// Datastore
> +    pub store: String,
> +    /// upid of the last run gc job
> +    #[serde(skip_serializing_if = "Option::is_none")]
> +    pub last_run_upid: Option<String>,
> +    /// Number of removed chunks
> +    #[serde(skip_serializing_if = "Option::is_none")]
> +    pub removed_chunks: Option<usize>,
> +    /// Number of pending chunks
> +    #[serde(skip_serializing_if = "Option::is_none")]
> +    pub pending_chunks: Option<usize>,
> +    /// Schedule of the gc job
> +    #[serde(skip_serializing_if = "Option::is_none")]
> +    pub schedule: Option<String>,
> +    /// Time of the next gc run
> +    #[serde(skip_serializing_if = "Option::is_none")]
> +    pub next_run: Option<i64>,
> +    /// Endtime of the last gc run
> +    #[serde(skip_serializing_if = "Option::is_none")]
> +    pub last_run_endtime: Option<i64>,
> +    /// State of the last gc run
> +    #[serde(skip_serializing_if = "Option::is_none")]
> +    pub last_run_state: Option<String>,
> +    /// Duration of last gc run
> +    #[serde(skip_serializing_if = "Option::is_none")]
> +    pub duration: Option<i64>,
> +}
> +
>  #[api(
>      properties: {
>          "gc-status": {
> diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
> index a95031e7..357cae0a 100644
> --- a/src/api2/admin/datastore.rs
> +++ b/src/api2/admin/datastore.rs
> @@ -27,18 +27,20 @@ use proxmox_sys::fs::{
>      file_read_firstline, file_read_optional_string, replace_file, CreateOptions,
>  };
>  use proxmox_sys::{task_log, task_warn};
> +use proxmox_time::CalendarEvent;
>  
>  use pxar::accessor::aio::Accessor;
>  use pxar::EntryKind;
>  
>  use pbs_api_types::{
>      print_ns_and_snapshot, print_store_and_ns, Authid, BackupContent, BackupNamespace, BackupType,
> -    Counts, CryptMode, DataStoreListItem, DataStoreStatus, GarbageCollectionStatus, GroupListItem,
> +    Counts, CryptMode, DataStoreConfig, DataStoreListItem, DataStoreStatus,
> +    GarbageCollectionJobStatus, GarbageCollectionStatus, GroupListItem, JobScheduleStatus,
>      KeepOptions, Operation, PruneJobOptions, RRDMode, RRDTimeFrame, SnapshotListItem,
>      SnapshotVerifyState, BACKUP_ARCHIVE_NAME_SCHEMA, BACKUP_ID_SCHEMA, BACKUP_NAMESPACE_SCHEMA,
>      BACKUP_TIME_SCHEMA, BACKUP_TYPE_SCHEMA, DATASTORE_SCHEMA, IGNORE_VERIFIED_BACKUPS_SCHEMA,
>      MAX_NAMESPACE_DEPTH, NS_MAX_DEPTH_SCHEMA, PRIV_DATASTORE_AUDIT, PRIV_DATASTORE_BACKUP,
> -    PRIV_DATASTORE_MODIFY, PRIV_DATASTORE_PRUNE, PRIV_DATASTORE_READ, PRIV_DATASTORE_VERIFY,
> +    PRIV_DATASTORE_MODIFY, PRIV_DATASTORE_PRUNE, PRIV_DATASTORE_READ, PRIV_DATASTORE_VERIFY, UPID,
>      UPID_SCHEMA, VERIFICATION_OUTDATED_AFTER_SCHEMA,
>  };
>  use pbs_client::pxar::{create_tar, create_zip};
> @@ -67,7 +69,7 @@ use crate::backup::{
>      ListAccessibleBackupGroups, NS_PRIVS_OK,
>  };
>  
> -use crate::server::jobstate::Job;
> +use crate::server::jobstate::{compute_schedule_status, Job, JobState};
>  
>  const GROUP_NOTES_FILE_NAME: &str = "notes";
>  
> @@ -1199,6 +1201,123 @@ pub fn garbage_collection_status(
>      Ok(status)
>  }
>  
> +#[api(
> +    input: {
> +        properties: {
> +            store: {
> +                schema: DATASTORE_SCHEMA,
> +            },
> +        },
> +    },
> +    returns: {
> +        type: GarbageCollectionJobStatus,
> +    },
> +    access: {
> +        permission: &Permission::Privilege(&["datastore", "{store}"], PRIV_DATASTORE_AUDIT, false),
> +    },
> +)]
> +/// Garbage collection status.
> +pub fn garbage_collection_job_status(
> +    store: String,
> +    _info: &ApiMethod,
> +    _rpcenv: &mut dyn RpcEnvironment,
> +) -> Result<GarbageCollectionJobStatus, Error> {
> +    let (config, _) = pbs_config::datastore::config()?;
> +    let store_config: DataStoreConfig = config.lookup("datastore", &store)?;
> +
> +    let mut info = GarbageCollectionJobStatus {
> +        store: store.clone(),
> +        schedule: store_config.gc_schedule,
> +        ..Default::default()
> +    };
> +
> +    let datastore = DataStore::lookup_datastore(&store, Some(Operation::Read))?;
> +    let status_in_memory = datastore.last_gc_status();
> +    let state_file = JobState::load("garbage_collection", &store)
> +        .map_err(|err| {
> +            log::error!(
> +                "could not open statefile for {:?}: {}",
> +                info.last_run_upid,
> +                err
> +            )
> +        })
> +        .ok();
> +
> +    let mut selected_upid = None;
> +    if status_in_memory.upid.is_some() {
> +        selected_upid = status_in_memory.upid;
> +    } else if let Some(JobState::Finished { upid, .. }) = &state_file {
> +        selected_upid = Some(upid.to_owned());
> +    }
> +
> +    info.last_run_upid = selected_upid.clone();
> +
> +    match selected_upid {
> +        Some(upid) => {
> +            info.removed_chunks = Some(status_in_memory.removed_chunks);
> +            info.pending_chunks = Some(status_in_memory.pending_chunks);
> +

... seems like the gc-stats also contain the removed/pending bytes, so this should be
an easy change. :)

-- 
- Lukas

Re: [pbs-devel] [PATCH v3 proxmox-backup 0/8] *** SUBJECT HERE ***

[Lukas Wagner]

On  2024-02-08 14:59, Stefan Lendl wrote:
> *** BLURB HERE ***
> 
> Stefan Lendl (8):
>   api: garbage collect job status
>   gc: global prune and gc job view
>   gc: move datastore/PruneAndGC to config/PruneAndGC
>   gc: hide datastore column in local gc view
>   ui: order Prune&GC before SyncJobs
>   cli: list gc jobs with proxmox-backup-manager
>   gc: show removed and pending chunks of last run in ui
>   gc: configure width and flex on GC Jobs columns
> 
>  pbs-api-types/src/datastore.rs    |  40 +++++
>  src/api2/admin/datastore.rs       | 129 +++++++++++++++-
>  src/api2/admin/gc.rs              |  57 +++++++
>  src/api2/admin/mod.rs             |   2 +
>  src/bin/proxmox-backup-manager.rs |  33 ++++
>  www/Makefile                      |   4 +-
>  www/Utils.js                      |   6 +-
>  www/config/GCView.js              | 243 ++++++++++++++++++++++++++++++
>  www/config/PruneAndGC.js          |  52 +++++++
>  www/datastore/DataStoreList.js    |  11 +-
>  www/datastore/Panel.js            |   3 +-
>  www/datastore/PruneAndGC.js       | 133 ----------------
>  www/window/GCJobEdit.js           |  28 ++++
>  13 files changed, 593 insertions(+), 148 deletions(-)
>  create mode 100644 src/api2/admin/gc.rs
>  create mode 100644 www/config/GCView.js
>  create mode 100644 www/config/PruneAndGC.js
>  delete mode 100644 www/datastore/PruneAndGC.js
>  create mode 100644 www/window/GCJobEdit.js
> 

Apart from the requested change from chunks to bytes, these 
changes look good to me :)
Please make sure to include the cover letter in v4 ;)

With the suggested UI/API changes, consider this:

Tested-by: Lukas Wagner <l.wagner@proxmox.com>
Reviewed-by: Lukas Wagner <l.wagner@proxmox.com>


-- 
- Lukas

Re: [pbs-devel] [PATCH v3 proxmox-backup 0/8] *** SUBJECT HERE ***

[Stefan Lendl]

Thanks for the review, sent v4.