* [pbs-devel] [Stefan Lendl] Re: [PATCH proxmox-backup v3] close #4723: updated gc view in the ui
[not found] ` <85e6743a-57ca-a662-847f-c102c66bccc7@proxmox.com>
@ 2023-09-22 9:13 ` Stefan Lendl
0 siblings, 0 replies; only message in thread
From: Stefan Lendl @ 2023-09-22 9:13 UTC (permalink / raw)
To: pbs-devel
Handling of GC tasks that never ran works great now. GUI alignment looks good.
> Updated the GC overview in the datastore page. This enables
> us to see:
> - schedule
> - State (of last run)
> - Duration (of last run)
> - Last Run
> - Next Run
> - Pending Chunks (of last run)
> - Removed Chunks (of last run)
>
> Added `ObjectGrid` for GCView, moved to different file. Added
> endpoint `gc_info` that returns all the necessary config and
> last run stats.
>
> Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
> ---
>
> update v3:
> - ui: removed `required` attribute on items to get the sorting right
> - made `pending_chunks` and `removed_chunks` options, so that they
> are not shown when no gc run exists
>
> update v2:
> - skip serializing if value is `None`
> - return just the schedule if `upid` doesn't exist (means no gc has been
> run)
> - ui: removed default values on timestamps
> - ui: removed flex and minHeight properties
>
> pbs-api-types/src/datastore.rs | 38 +++++++++++
> src/api2/admin/datastore.rs | 98 +++++++++++++++++++++++++---
> www/Makefile | 1 +
> www/Utils.js | 12 ++--
> www/config/GCView.js | 116 +++++++++++++++++++++++++++++++++
> www/datastore/PruneAndGC.js | 91 +-------------------------
> 6 files changed, 253 insertions(+), 103 deletions(-)
> mode change 100644 => 100755 src/api2/admin/datastore.rs
> create mode 100644 www/config/GCView.js
>
> diff --git a/pbs-api-types/src/datastore.rs b/pbs-api-types/src/datastore.rs
> index 73c4890e..55caa963 100644
> --- a/pbs-api-types/src/datastore.rs
> +++ b/pbs-api-types/src/datastore.rs
> @@ -1250,6 +1250,44 @@ pub struct GarbageCollectionStatus {
> pub still_bad: usize,
> }
>
> +#[api(
> + properties: {
> + "last-run-upid": {
> + optional: true,
> + type: UPID,
> + },
> + },
> +)]
> +#[derive(Clone, Default, Serialize, Deserialize, PartialEq)]
> +#[serde(rename_all = "kebab-case")]
> +/// Garbage Collection general info
> +pub struct GarbageCollectionInfo {
> + /// upid of the last run gc job
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub last_run_upid: Option<String>,
> + /// Number of removed chunks
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub removed_chunks: Option<usize>,
> + /// Number of pending chunks
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub pending_chunks: Option<usize>,
> + /// Schedule of the gc job
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub schedule: Option<String>,
> + /// Time of the next gc run
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub next_run: Option<i64>,
> + /// Endtime of the last gc run
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub last_run_endtime: Option<i64>,
> + /// State of the last gc run
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub last_run_state: Option<String>,
> + /// Duration of last gc run
> + #[serde(skip_serializing_if = "Option::is_none")]
> + pub duration: Option<i64>,
> +}
> +
> #[api(
> properties: {
> "gc-status": {
> diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
> old mode 100644
> new mode 100755
> index a95031e7..1127bd95
> --- a/src/api2/admin/datastore.rs
> +++ b/src/api2/admin/datastore.rs
> @@ -10,6 +10,7 @@ use anyhow::{bail, format_err, Error};
> use futures::*;
> use hyper::http::request::Parts;
> use hyper::{header, Body, Response, StatusCode};
> +use proxmox_time::CalendarEvent;
> use serde::Deserialize;
> use serde_json::{json, Value};
> use tokio_stream::wrappers::ReceiverStream;
> @@ -33,13 +34,14 @@ use pxar::EntryKind;
>
> use pbs_api_types::{
> print_ns_and_snapshot, print_store_and_ns, Authid, BackupContent, BackupNamespace, BackupType,
> - Counts, CryptMode, DataStoreListItem, DataStoreStatus, GarbageCollectionStatus, GroupListItem,
> - KeepOptions, Operation, PruneJobOptions, RRDMode, RRDTimeFrame, SnapshotListItem,
> - SnapshotVerifyState, BACKUP_ARCHIVE_NAME_SCHEMA, BACKUP_ID_SCHEMA, BACKUP_NAMESPACE_SCHEMA,
> - BACKUP_TIME_SCHEMA, BACKUP_TYPE_SCHEMA, DATASTORE_SCHEMA, IGNORE_VERIFIED_BACKUPS_SCHEMA,
> - MAX_NAMESPACE_DEPTH, NS_MAX_DEPTH_SCHEMA, PRIV_DATASTORE_AUDIT, PRIV_DATASTORE_BACKUP,
> - PRIV_DATASTORE_MODIFY, PRIV_DATASTORE_PRUNE, PRIV_DATASTORE_READ, PRIV_DATASTORE_VERIFY,
> - UPID_SCHEMA, VERIFICATION_OUTDATED_AFTER_SCHEMA,
> + Counts, CryptMode, DataStoreConfig, DataStoreListItem, DataStoreStatus, GarbageCollectionInfo,
> + GarbageCollectionStatus, GroupListItem, JobScheduleStatus, KeepOptions, Operation,
> + PruneJobOptions, RRDMode, RRDTimeFrame, SnapshotListItem, SnapshotVerifyState,
> + BACKUP_ARCHIVE_NAME_SCHEMA, BACKUP_ID_SCHEMA, BACKUP_NAMESPACE_SCHEMA, BACKUP_TIME_SCHEMA,
> + BACKUP_TYPE_SCHEMA, DATASTORE_SCHEMA, IGNORE_VERIFIED_BACKUPS_SCHEMA, MAX_NAMESPACE_DEPTH,
> + NS_MAX_DEPTH_SCHEMA, PRIV_DATASTORE_AUDIT, PRIV_DATASTORE_BACKUP, PRIV_DATASTORE_MODIFY,
> + PRIV_DATASTORE_PRUNE, PRIV_DATASTORE_READ, PRIV_DATASTORE_VERIFY, UPID, UPID_SCHEMA,
> + VERIFICATION_OUTDATED_AFTER_SCHEMA,
> };
> use pbs_client::pxar::{create_tar, create_zip};
> use pbs_config::CachedUserInfo;
> @@ -67,7 +69,7 @@ use crate::backup::{
> ListAccessibleBackupGroups, NS_PRIVS_OK,
> };
>
> -use crate::server::jobstate::Job;
> +use crate::server::jobstate::{compute_schedule_status, Job, JobState};
>
> const GROUP_NOTES_FILE_NAME: &str = "notes";
>
> @@ -1199,6 +1201,82 @@ pub fn garbage_collection_status(
> Ok(status)
> }
>
> +#[api(
> + input: {
> + properties: {
> + store: {
> + schema: DATASTORE_SCHEMA,
> + },
> + },
> + },
> + returns: {
> + type: GarbageCollectionInfo,
> + },
> + access: {
> + permission: &Permission::Privilege(&["datastore", "{store}"], PRIV_DATASTORE_AUDIT, false),
> + },
> +)]
> +/// Garbage collection status.
> +pub fn garbage_collection_info(
> + store: String,
> + _info: &ApiMethod,
> + _rpcenv: &mut dyn RpcEnvironment,
> +) -> Result<GarbageCollectionInfo, Error> {
> + let (config, _) = pbs_config::datastore::config()?;
> + let store_config: DataStoreConfig = config.lookup("datastore", &store)?;
> +
> + let datastore = DataStore::lookup_datastore(&store, Some(Operation::Read))?;
> + let status = datastore.last_gc_status();
> +
> + let upid = match status.upid {
> + Some(upid) => upid,
> + None => {
> + return Ok(GarbageCollectionInfo {
> + schedule: store_config.gc_schedule,
> + ..Default::default()
> + })
> + }
> + };
> + let last_state = JobState::load("garbage_collection", &store)
> + .map_err(|err| format_err!("could not open statefile for {}: {}", upid, err))
> + .ok();
> +
> + let mut computed_schedule: JobScheduleStatus = JobScheduleStatus::default();
> + if let Some(state) = last_state {
> + computed_schedule = compute_schedule_status(&state, Some(&upid)).unwrap();
> + }
> +
> + // calculate next event
> + if let Some(schedule) = &store_config.gc_schedule {
> + if let (Ok(event), Some(last_run)) = (
> + schedule.parse::<CalendarEvent>(),
> + computed_schedule.last_run_endtime,
> + ) {
> + if let Ok(next_event) = event.compute_next_event(last_run) {
> + computed_schedule.next_run = next_event;
> + }
> + }
> + }
If GC never ran but is scheduled, it would also be nice to show the next
scheduled run.
This might be usefull if people had not GC configured, then create a job
to further indicate that the job is scheduled correctly.
> +
> + let mut duration = None;
> + if let (Ok(upid), Some(endtime)) = (upid.parse::<UPID>(), computed_schedule.last_run_endtime) {
> + duration = Some(endtime - upid.starttime);
> + }
> +
> + let info = GarbageCollectionInfo {
> + last_run_upid: Some(upid),
> + removed_chunks: Some(status.removed_chunks),
> + pending_chunks: Some(status.pending_chunks),
> + schedule: store_config.gc_schedule,
> + next_run: computed_schedule.next_run,
> + last_run_endtime: computed_schedule.last_run_endtime,
> + last_run_state: computed_schedule.last_run_state,
> + duration,
> + };
> +
> + Ok(info)
> +}
> +
> #[api(
> returns: {
> description: "List the accessible datastores.",
> @@ -2265,6 +2343,10 @@ const DATASTORE_INFO_SUBDIRS: SubdirMap = &[
> .get(&API_METHOD_GARBAGE_COLLECTION_STATUS)
> .post(&API_METHOD_START_GARBAGE_COLLECTION),
> ),
> + (
> + "gc_info",
> + &Router::new().get(&API_METHOD_GARBAGE_COLLECTION_INFO),
> + ),
> (
> "group-notes",
> &Router::new()
> diff --git a/www/Makefile b/www/Makefile
> index 04c12b31..b67fd73f 100644
> --- a/www/Makefile
> +++ b/www/Makefile
> @@ -62,6 +62,7 @@ JSSRC= \
> config/SyncView.js \
> config/VerifyView.js \
> config/PruneView.js \
> + config/GCView.js \
> config/WebauthnView.js \
> config/CertificateView.js \
> config/NodeOptionView.js \
> diff --git a/www/Utils.js b/www/Utils.js
> index 2eca600e..66293b17 100644
> --- a/www/Utils.js
> +++ b/www/Utils.js
> @@ -199,14 +199,14 @@ Ext.define('PBS.Utils', {
> return fingerprint.substring(0, 23);
> },
>
> - render_task_status: function(value, metadata, record) {
> - if (!record.data['last-run-upid']) {
> - return '-';
> + render_task_status: function(value, metadata, record, rowIndex, colIndex, store) {
> + if (!record.data['last-run-upid'] && !store.getById('last-run-upid')?.data.value) {
> + return '-';
> }
>
> - if (!record.data['last-run-endtime']) {
> - metadata.tdCls = 'x-grid-row-loading';
> - return '';
> + if (!record.data['last-run-endtime'] && !store.getById('last-run-endtime')?.data.value) {
> + metadata.tdCls = 'x-grid-row-loading';
> + return '';
> }
>
> let parsed = Proxmox.Utils.parse_task_status(value);
> diff --git a/www/config/GCView.js b/www/config/GCView.js
> new file mode 100644
> index 00000000..1a79bb01
> --- /dev/null
> +++ b/www/config/GCView.js
> @@ -0,0 +1,116 @@
> +Ext.define('PBS.Datastore.GCOptions', {
> + extend: 'Proxmox.grid.ObjectGrid',
> + alias: 'widget.pbsGCJobView',
> + mixins: ['Proxmox.Mixin.CBind'],
> +
> + onlineHelp: 'maintenance_pruning',
> +
> + cbindData: function(initial) {
> + let me = this;
> +
> + me.datastore = encodeURIComponent(me.datastore);
> + me.url = `/api2/json/admin/datastore/${me.datastore}/gc_info`;
> + me.editorConfig = {
> + url: `/api2/extjs/config/datastore/${me.datastore}`,
> + };
> + return {};
> + },
> +
> + controller: {
> + xclass: 'Ext.app.ViewController',
> +
> + edit: function() {
> + this.getView().run_editor();
> + },
> +
> + garbageCollect: function() {
> + let me = this;
> + let view = me.getView();
> + Proxmox.Utils.API2Request({
> + url: `/admin/datastore/${view.datastore}/gc`,
> + method: 'POST',
> + failure: function(response) {
> + Ext.Msg.alert(gettext('Error'), response.htmlStatus);
> + },
> + success: function(response, options) {
> + Ext.create('Proxmox.window.TaskViewer', {
> + upid: response.result.data,
> + }).show();
> + },
> + });
> + },
> + },
> +
> + tbar: [
> + {
> + xtype: 'proxmoxButton',
> + text: gettext('Edit'),
> + enableFn: (rec) => rec.id === 'schedule',
> + disabled: true,
> + handler: 'edit',
> + },
> + '-',
> + {
> + xtype: 'proxmoxButton',
> + text: gettext('Start Garbage Collection'),
> + selModel: null,
> + handler: 'garbageCollect',
> + },
> + ],
> +
> + listeners: {
> + activate: function() { this.rstore.startUpdate(); },
> + destroy: function() { this.rstore.stopUpdate(); },
> + deactivate: function() { this.rstore.stopUpdate(); },
> + itemdblclick: 'edit',
> + },
> +
> + rows: {
> + "schedule": {
> + required: true,
> + defaultValue: Proxmox.Utils.NoneText,
> + header: gettext('Schedule'),
> + editor: {
> + xtype: 'proxmoxWindowEdit',
> + title: gettext('GC Schedule'),
> + onlineHelp: 'maintenance_gc',
> + items: {
> + xtype: 'pbsCalendarEvent',
> + name: 'gc-schedule',
> + fieldLabel: gettext("GC Schedule"),
> + emptyText: Proxmox.Utils.NoneText,
> + deleteEmpty: true,
> + },
> + },
> + },
> + "last-run-state": {
> + header: gettext('State'),
> + renderer: PBS.Utils.render_task_status,
> + disabled: true,
> + },
> + "duration": {
> + header: gettext('Duration'),
> + renderer: Proxmox.Utils.render_duration,
> + disabled: true,
> + },
> + "last-run-endtime": {
> + header: gettext('Last Run'),
> + renderer: PBS.Utils.render_optional_timestamp,
> + disabled: true,
> + },
> + "next-run": {
> + header: gettext('Next Run'),
> + renderer: PBS.Utils.render_next_task_run,
> + disabled: true,
> + },
> + "pending-chunks": {
> + header: gettext('Pending Chunks'),
> + disabled: true,
> + },
> + "removed-chunks": {
> + header: gettext('Removed Chunks'),
> + disabled: true,
> + },
> + "last-run-upid": { visible: false },
> + },
> +});
> diff --git a/www/datastore/PruneAndGC.js b/www/datastore/PruneAndGC.js
> index aab98dad..3b9878a3 100644
> --- a/www/datastore/PruneAndGC.js
> +++ b/www/datastore/PruneAndGC.js
> @@ -1,88 +1,3 @@
> -Ext.define('PBS.Datastore.GCOptions', {
> - extend: 'Proxmox.grid.ObjectGrid',
> - alias: 'widget.pbsDatastoreGCOpts',
> - mixins: ['Proxmox.Mixin.CBind'],
> -
> - onlineHelp: 'maintenance_pruning',
> -
> - cbindData: function(initial) {
> - let me = this;
> -
> - me.datastore = encodeURIComponent(me.datastore);
> - me.url = `/api2/json/config/datastore/${me.datastore}`;
> - me.editorConfig = {
> - url: `/api2/extjs/config/datastore/${me.datastore}`,
> - };
> - return {};
> - },
> -
> - controller: {
> - xclass: 'Ext.app.ViewController',
> -
> - edit: function() { this.getView().run_editor(); },
> -
> - garbageCollect: function() {
> - let me = this;
> - let view = me.getView();
> - Proxmox.Utils.API2Request({
> - url: `/admin/datastore/${view.datastore}/gc`,
> - method: 'POST',
> - failure: function(response) {
> - Ext.Msg.alert(gettext('Error'), response.htmlStatus);
> - },
> - success: function(response, options) {
> - Ext.create('Proxmox.window.TaskViewer', {
> - upid: response.result.data,
> - }).show();
> - },
> - });
> - },
> - },
> -
> - tbar: [
> - {
> - xtype: 'proxmoxButton',
> - text: gettext('Edit'),
> - disabled: true,
> - handler: 'edit',
> - },
> - '-',
> - {
> - xtype: 'proxmoxButton',
> - text: gettext('Start Garbage Collection'),
> - selModel: null,
> - handler: 'garbageCollect',
> - },
> - ],
> -
> - listeners: {
> - activate: function() { this.rstore.startUpdate(); },
> - destroy: function() { this.rstore.stopUpdate(); },
> - deactivate: function() { this.rstore.stopUpdate(); },
> - itemdblclick: 'edit',
> - },
> -
> - rows: {
> - "gc-schedule": {
> - required: true,
> - defaultValue: Proxmox.Utils.NoneText,
> - header: gettext('Garbage Collection Schedule'),
> - editor: {
> - xtype: 'proxmoxWindowEdit',
> - title: gettext('GC Schedule'),
> - onlineHelp: 'maintenance_gc',
> - items: {
> - xtype: 'pbsCalendarEvent',
> - name: 'gc-schedule',
> - fieldLabel: gettext("GC Schedule"),
> - emptyText: Proxmox.Utils.noneText,
> - deleteEmpty: true,
> - },
> - },
> - },
> - },
> -});
> -
> Ext.define('PBS.Datastore.PruneAndGC', {
> extend: 'Ext.panel.Panel',
> alias: 'widget.pbsDatastorePruneAndGC',
> @@ -99,9 +14,9 @@ Ext.define('PBS.Datastore.PruneAndGC', {
> },
> items: [
> {
> - xtype: 'pbsDatastoreGCOpts',
> + xtype: 'pbsGCJobView',
> title: gettext('Garbage Collection'),
> - itemId: 'datastore-gc',
> + itemId: 'datastore-gc-jobs',
> nodename: 'localhost',
> cbind: {
> datastore: '{datastore}',
> @@ -111,8 +26,6 @@ Ext.define('PBS.Datastore.PruneAndGC', {
> xtype: 'pbsPruneJobView',
> nodename: 'localhost',
> itemId: 'datastore-prune-jobs',
> - flex: 1,
> - minHeight: 200,
> cbind: {
> datastore: '{datastore}',
> },
> --
> 2.39.2
>
>
>
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
-------------------- Start of forwarded message --------------------
Date: Fri, 22 Sep 2023 10:51:35 +0200
Subject: Re: [pbs-devel] [PATCH proxmox-backup v3] close #4723: updated gc
view in the ui
To: Stefan Lendl <s.lendl@proxmox.com>
From: Gabriel Goller <g.goller@proxmox.com>
Thanks for having a look at it!
Could you post the message again on the mailing list (using "reply all"
instead of "reply"). Thanks!
On 9/22/23 10:49, Stefan Lendl wrote:
> Gabriel Goller <g.goller@proxmox.com> writes:
>
> Handling of GC tasks that never ran works great now. GUI alignment looks good.
>
>> Updated the GC overview in the datastore page. This enables
>> us to see:
>> - schedule
>> - State (of last run)
>> - Duration (of last run)
>> - Last Run
>> - Next Run
>> - Pending Chunks (of last run)
>> - Removed Chunks (of last run)
>>
>> Added `ObjectGrid` for GCView, moved to different file. Added
>> endpoint `gc_info` that returns all the necessary config and
>> last run stats.
>>
>> Signed-off-by: Gabriel Goller <g.goller@proxmox.com>
>> ---
>>
>> update v3:
>> - ui: removed `required` attribute on items to get the sorting right
>> - made `pending_chunks` and `removed_chunks` options, so that they
>> are not shown when no gc run exists
>>
>> update v2:
>> - skip serializing if value is `None`
>> - return just the schedule if `upid` doesn't exist (means no gc has been
>> run)
>> - ui: removed default values on timestamps
>> - ui: removed flex and minHeight properties
>>
>> pbs-api-types/src/datastore.rs | 38 +++++++++++
>> src/api2/admin/datastore.rs | 98 +++++++++++++++++++++++++---
>> www/Makefile | 1 +
>> www/Utils.js | 12 ++--
>> www/config/GCView.js | 116 +++++++++++++++++++++++++++++++++
>> www/datastore/PruneAndGC.js | 91 +-------------------------
>> 6 files changed, 253 insertions(+), 103 deletions(-)
>> mode change 100644 => 100755 src/api2/admin/datastore.rs
>> create mode 100644 www/config/GCView.js
>>
>> diff --git a/pbs-api-types/src/datastore.rs b/pbs-api-types/src/datastore.rs
>> index 73c4890e..55caa963 100644
>> --- a/pbs-api-types/src/datastore.rs
>> +++ b/pbs-api-types/src/datastore.rs
>> @@ -1250,6 +1250,44 @@ pub struct GarbageCollectionStatus {
>> pub still_bad: usize,
>> }
>>
>> +#[api(
>> + properties: {
>> + "last-run-upid": {
>> + optional: true,
>> + type: UPID,
>> + },
>> + },
>> +)]
>> +#[derive(Clone, Default, Serialize, Deserialize, PartialEq)]
>> +#[serde(rename_all = "kebab-case")]
>> +/// Garbage Collection general info
>> +pub struct GarbageCollectionInfo {
>> + /// upid of the last run gc job
>> + #[serde(skip_serializing_if = "Option::is_none")]
>> + pub last_run_upid: Option<String>,
>> + /// Number of removed chunks
>> + #[serde(skip_serializing_if = "Option::is_none")]
>> + pub removed_chunks: Option<usize>,
>> + /// Number of pending chunks
>> + #[serde(skip_serializing_if = "Option::is_none")]
>> + pub pending_chunks: Option<usize>,
>> + /// Schedule of the gc job
>> + #[serde(skip_serializing_if = "Option::is_none")]
>> + pub schedule: Option<String>,
>> + /// Time of the next gc run
>> + #[serde(skip_serializing_if = "Option::is_none")]
>> + pub next_run: Option<i64>,
>> + /// Endtime of the last gc run
>> + #[serde(skip_serializing_if = "Option::is_none")]
>> + pub last_run_endtime: Option<i64>,
>> + /// State of the last gc run
>> + #[serde(skip_serializing_if = "Option::is_none")]
>> + pub last_run_state: Option<String>,
>> + /// Duration of last gc run
>> + #[serde(skip_serializing_if = "Option::is_none")]
>> + pub duration: Option<i64>,
>> +}
>> +
>> #[api(
>> properties: {
>> "gc-status": {
>> diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
>> old mode 100644
>> new mode 100755
>> index a95031e7..1127bd95
>> --- a/src/api2/admin/datastore.rs
>> +++ b/src/api2/admin/datastore.rs
>> @@ -10,6 +10,7 @@ use anyhow::{bail, format_err, Error};
>> use futures::*;
>> use hyper::http::request::Parts;
>> use hyper::{header, Body, Response, StatusCode};
>> +use proxmox_time::CalendarEvent;
>> use serde::Deserialize;
>> use serde_json::{json, Value};
>> use tokio_stream::wrappers::ReceiverStream;
>> @@ -33,13 +34,14 @@ use pxar::EntryKind;
>>
>> use pbs_api_types::{
>> print_ns_and_snapshot, print_store_and_ns, Authid, BackupContent, BackupNamespace, BackupType,
>> - Counts, CryptMode, DataStoreListItem, DataStoreStatus, GarbageCollectionStatus, GroupListItem,
>> - KeepOptions, Operation, PruneJobOptions, RRDMode, RRDTimeFrame, SnapshotListItem,
>> - SnapshotVerifyState, BACKUP_ARCHIVE_NAME_SCHEMA, BACKUP_ID_SCHEMA, BACKUP_NAMESPACE_SCHEMA,
>> - BACKUP_TIME_SCHEMA, BACKUP_TYPE_SCHEMA, DATASTORE_SCHEMA, IGNORE_VERIFIED_BACKUPS_SCHEMA,
>> - MAX_NAMESPACE_DEPTH, NS_MAX_DEPTH_SCHEMA, PRIV_DATASTORE_AUDIT, PRIV_DATASTORE_BACKUP,
>> - PRIV_DATASTORE_MODIFY, PRIV_DATASTORE_PRUNE, PRIV_DATASTORE_READ, PRIV_DATASTORE_VERIFY,
>> - UPID_SCHEMA, VERIFICATION_OUTDATED_AFTER_SCHEMA,
>> + Counts, CryptMode, DataStoreConfig, DataStoreListItem, DataStoreStatus, GarbageCollectionInfo,
>> + GarbageCollectionStatus, GroupListItem, JobScheduleStatus, KeepOptions, Operation,
>> + PruneJobOptions, RRDMode, RRDTimeFrame, SnapshotListItem, SnapshotVerifyState,
>> + BACKUP_ARCHIVE_NAME_SCHEMA, BACKUP_ID_SCHEMA, BACKUP_NAMESPACE_SCHEMA, BACKUP_TIME_SCHEMA,
>> + BACKUP_TYPE_SCHEMA, DATASTORE_SCHEMA, IGNORE_VERIFIED_BACKUPS_SCHEMA, MAX_NAMESPACE_DEPTH,
>> + NS_MAX_DEPTH_SCHEMA, PRIV_DATASTORE_AUDIT, PRIV_DATASTORE_BACKUP, PRIV_DATASTORE_MODIFY,
>> + PRIV_DATASTORE_PRUNE, PRIV_DATASTORE_READ, PRIV_DATASTORE_VERIFY, UPID, UPID_SCHEMA,
>> + VERIFICATION_OUTDATED_AFTER_SCHEMA,
>> };
>> use pbs_client::pxar::{create_tar, create_zip};
>> use pbs_config::CachedUserInfo;
>> @@ -67,7 +69,7 @@ use crate::backup::{
>> ListAccessibleBackupGroups, NS_PRIVS_OK,
>> };
>>
>> -use crate::server::jobstate::Job;
>> +use crate::server::jobstate::{compute_schedule_status, Job, JobState};
>>
>> const GROUP_NOTES_FILE_NAME: &str = "notes";
>>
>> @@ -1199,6 +1201,82 @@ pub fn garbage_collection_status(
>> Ok(status)
>> }
>>
>> +#[api(
>> + input: {
>> + properties: {
>> + store: {
>> + schema: DATASTORE_SCHEMA,
>> + },
>> + },
>> + },
>> + returns: {
>> + type: GarbageCollectionInfo,
>> + },
>> + access: {
>> + permission: &Permission::Privilege(&["datastore", "{store}"], PRIV_DATASTORE_AUDIT, false),
>> + },
>> +)]
>> +/// Garbage collection status.
>> +pub fn garbage_collection_info(
>> + store: String,
>> + _info: &ApiMethod,
>> + _rpcenv: &mut dyn RpcEnvironment,
>> +) -> Result<GarbageCollectionInfo, Error> {
>> + let (config, _) = pbs_config::datastore::config()?;
>> + let store_config: DataStoreConfig = config.lookup("datastore", &store)?;
>> +
>> + let datastore = DataStore::lookup_datastore(&store, Some(Operation::Read))?;
>> + let status = datastore.last_gc_status();
>> +
>> + let upid = match status.upid {
>> + Some(upid) => upid,
>> + None => {
>> + return Ok(GarbageCollectionInfo {
>> + schedule: store_config.gc_schedule,
>> + ..Default::default()
>> + })
>> + }
>> + };
>> + let last_state = JobState::load("garbage_collection", &store)
>> + .map_err(|err| format_err!("could not open statefile for {}: {}", upid, err))
>> + .ok();
>> +
>> + let mut computed_schedule: JobScheduleStatus = JobScheduleStatus::default();
>> + if let Some(state) = last_state {
>> + computed_schedule = compute_schedule_status(&state, Some(&upid)).unwrap();
>> + }
>> +
>> + // calculate next event
>> + if let Some(schedule) = &store_config.gc_schedule {
>> + if let (Ok(event), Some(last_run)) = (
>> + schedule.parse::<CalendarEvent>(),
>> + computed_schedule.last_run_endtime,
>> + ) {
>> + if let Ok(next_event) = event.compute_next_event(last_run) {
>> + computed_schedule.next_run = next_event;
>> + }
>> + }
>> + }
>
> If GC never ran but is scheduled, it would also be nice to show the next
> scheduled run.
> This might be usefull if people had not GC configured, then create a job
> to further indicate that the job is scheduled correctly.
>
>
>> +
>> + let mut duration = None;
>> + if let (Ok(upid), Some(endtime)) = (upid.parse::<UPID>(), computed_schedule.last_run_endtime) {
>> + duration = Some(endtime - upid.starttime);
>> + }
>> +
>> + let info = GarbageCollectionInfo {
>> + last_run_upid: Some(upid),
>> + removed_chunks: Some(status.removed_chunks),
>> + pending_chunks: Some(status.pending_chunks),
>> + schedule: store_config.gc_schedule,
>> + next_run: computed_schedule.next_run,
>> + last_run_endtime: computed_schedule.last_run_endtime,
>> + last_run_state: computed_schedule.last_run_state,
>> + duration,
>> + };
>> +
>> + Ok(info)
>> +}
>> +
>> #[api(
>> returns: {
>> description: "List the accessible datastores.",
>> @@ -2265,6 +2343,10 @@ const DATASTORE_INFO_SUBDIRS: SubdirMap = &[
>> .get(&API_METHOD_GARBAGE_COLLECTION_STATUS)
>> .post(&API_METHOD_START_GARBAGE_COLLECTION),
>> ),
>> + (
>> + "gc_info",
>> + &Router::new().get(&API_METHOD_GARBAGE_COLLECTION_INFO),
>> + ),
>> (
>> "group-notes",
>> &Router::new()
>> diff --git a/www/Makefile b/www/Makefile
>> index 04c12b31..b67fd73f 100644
>> --- a/www/Makefile
>> +++ b/www/Makefile
>> @@ -62,6 +62,7 @@ JSSRC= \
>> config/SyncView.js \
>> config/VerifyView.js \
>> config/PruneView.js \
>> + config/GCView.js \
>> config/WebauthnView.js \
>> config/CertificateView.js \
>> config/NodeOptionView.js \
>> diff --git a/www/Utils.js b/www/Utils.js
>> index 2eca600e..66293b17 100644
>> --- a/www/Utils.js
>> +++ b/www/Utils.js
>> @@ -199,14 +199,14 @@ Ext.define('PBS.Utils', {
>> return fingerprint.substring(0, 23);
>> },
>>
>> - render_task_status: function(value, metadata, record) {
>> - if (!record.data['last-run-upid']) {
>> - return '-';
>> + render_task_status: function(value, metadata, record, rowIndex, colIndex, store) {
>> + if (!record.data['last-run-upid'] && !store.getById('last-run-upid')?.data.value) {
>> + return '-';
>> }
>>
>> - if (!record.data['last-run-endtime']) {
>> - metadata.tdCls = 'x-grid-row-loading';
>> - return '';
>> + if (!record.data['last-run-endtime'] && !store.getById('last-run-endtime')?.data.value) {
>> + metadata.tdCls = 'x-grid-row-loading';
>> + return '';
>> }
>>
>> let parsed = Proxmox.Utils.parse_task_status(value);
>> diff --git a/www/config/GCView.js b/www/config/GCView.js
>> new file mode 100644
>> index 00000000..1a79bb01
>> --- /dev/null
>> +++ b/www/config/GCView.js
>> @@ -0,0 +1,116 @@
>> +Ext.define('PBS.Datastore.GCOptions', {
>> + extend: 'Proxmox.grid.ObjectGrid',
>> + alias: 'widget.pbsGCJobView',
>> + mixins: ['Proxmox.Mixin.CBind'],
>> +
>> + onlineHelp: 'maintenance_pruning',
>> +
>> + cbindData: function(initial) {
>> + let me = this;
>> +
>> + me.datastore = encodeURIComponent(me.datastore);
>> + me.url = `/api2/json/admin/datastore/${me.datastore}/gc_info`;
>> + me.editorConfig = {
>> + url: `/api2/extjs/config/datastore/${me.datastore}`,
>> + };
>> + return {};
>> + },
>> +
>> + controller: {
>> + xclass: 'Ext.app.ViewController',
>> +
>> + edit: function() {
>> + this.getView().run_editor();
>> + },
>> +
>> + garbageCollect: function() {
>> + let me = this;
>> + let view = me.getView();
>> + Proxmox.Utils.API2Request({
>> + url: `/admin/datastore/${view.datastore}/gc`,
>> + method: 'POST',
>> + failure: function(response) {
>> + Ext.Msg.alert(gettext('Error'), response.htmlStatus);
>> + },
>> + success: function(response, options) {
>> + Ext.create('Proxmox.window.TaskViewer', {
>> + upid: response.result.data,
>> + }).show();
>> + },
>> + });
>> + },
>> + },
>> +
>> + tbar: [
>> + {
>> + xtype: 'proxmoxButton',
>> + text: gettext('Edit'),
>> + enableFn: (rec) => rec.id === 'schedule',
>> + disabled: true,
>> + handler: 'edit',
>> + },
>> + '-',
>> + {
>> + xtype: 'proxmoxButton',
>> + text: gettext('Start Garbage Collection'),
>> + selModel: null,
>> + handler: 'garbageCollect',
>> + },
>> + ],
>> +
>> + listeners: {
>> + activate: function() { this.rstore.startUpdate(); },
>> + destroy: function() { this.rstore.stopUpdate(); },
>> + deactivate: function() { this.rstore.stopUpdate(); },
>> + itemdblclick: 'edit',
>> + },
>> +
>> + rows: {
>> + "schedule": {
>> + required: true,
>> + defaultValue: Proxmox.Utils.NoneText,
>> + header: gettext('Schedule'),
>> + editor: {
>> + xtype: 'proxmoxWindowEdit',
>> + title: gettext('GC Schedule'),
>> + onlineHelp: 'maintenance_gc',
>> + items: {
>> + xtype: 'pbsCalendarEvent',
>> + name: 'gc-schedule',
>> + fieldLabel: gettext("GC Schedule"),
>> + emptyText: Proxmox.Utils.NoneText,
>> + deleteEmpty: true,
>> + },
>> + },
>> + },
>> + "last-run-state": {
>> + header: gettext('State'),
>> + renderer: PBS.Utils.render_task_status,
>> + disabled: true,
>> + },
>> + "duration": {
>> + header: gettext('Duration'),
>> + renderer: Proxmox.Utils.render_duration,
>> + disabled: true,
>> + },
>> + "last-run-endtime": {
>> + header: gettext('Last Run'),
>> + renderer: PBS.Utils.render_optional_timestamp,
>> + disabled: true,
>> + },
>> + "next-run": {
>> + header: gettext('Next Run'),
>> + renderer: PBS.Utils.render_next_task_run,
>> + disabled: true,
>> + },
>> + "pending-chunks": {
>> + header: gettext('Pending Chunks'),
>> + disabled: true,
>> + },
>> + "removed-chunks": {
>> + header: gettext('Removed Chunks'),
>> + disabled: true,
>> + },
>> + "last-run-upid": { visible: false },
>> + },
>> +});
>> diff --git a/www/datastore/PruneAndGC.js b/www/datastore/PruneAndGC.js
>> index aab98dad..3b9878a3 100644
>> --- a/www/datastore/PruneAndGC.js
>> +++ b/www/datastore/PruneAndGC.js
>> @@ -1,88 +1,3 @@
>> -Ext.define('PBS.Datastore.GCOptions', {
>> - extend: 'Proxmox.grid.ObjectGrid',
>> - alias: 'widget.pbsDatastoreGCOpts',
>> - mixins: ['Proxmox.Mixin.CBind'],
>> -
>> - onlineHelp: 'maintenance_pruning',
>> -
>> - cbindData: function(initial) {
>> - let me = this;
>> -
>> - me.datastore = encodeURIComponent(me.datastore);
>> - me.url = `/api2/json/config/datastore/${me.datastore}`;
>> - me.editorConfig = {
>> - url: `/api2/extjs/config/datastore/${me.datastore}`,
>> - };
>> - return {};
>> - },
>> -
>> - controller: {
>> - xclass: 'Ext.app.ViewController',
>> -
>> - edit: function() { this.getView().run_editor(); },
>> -
>> - garbageCollect: function() {
>> - let me = this;
>> - let view = me.getView();
>> - Proxmox.Utils.API2Request({
>> - url: `/admin/datastore/${view.datastore}/gc`,
>> - method: 'POST',
>> - failure: function(response) {
>> - Ext.Msg.alert(gettext('Error'), response.htmlStatus);
>> - },
>> - success: function(response, options) {
>> - Ext.create('Proxmox.window.TaskViewer', {
>> - upid: response.result.data,
>> - }).show();
>> - },
>> - });
>> - },
>> - },
>> -
>> - tbar: [
>> - {
>> - xtype: 'proxmoxButton',
>> - text: gettext('Edit'),
>> - disabled: true,
>> - handler: 'edit',
>> - },
>> - '-',
>> - {
>> - xtype: 'proxmoxButton',
>> - text: gettext('Start Garbage Collection'),
>> - selModel: null,
>> - handler: 'garbageCollect',
>> - },
>> - ],
>> -
>> - listeners: {
>> - activate: function() { this.rstore.startUpdate(); },
>> - destroy: function() { this.rstore.stopUpdate(); },
>> - deactivate: function() { this.rstore.stopUpdate(); },
>> - itemdblclick: 'edit',
>> - },
>> -
>> - rows: {
>> - "gc-schedule": {
>> - required: true,
>> - defaultValue: Proxmox.Utils.NoneText,
>> - header: gettext('Garbage Collection Schedule'),
>> - editor: {
>> - xtype: 'proxmoxWindowEdit',
>> - title: gettext('GC Schedule'),
>> - onlineHelp: 'maintenance_gc',
>> - items: {
>> - xtype: 'pbsCalendarEvent',
>> - name: 'gc-schedule',
>> - fieldLabel: gettext("GC Schedule"),
>> - emptyText: Proxmox.Utils.noneText,
>> - deleteEmpty: true,
>> - },
>> - },
>> - },
>> - },
>> -});
>> -
>> Ext.define('PBS.Datastore.PruneAndGC', {
>> extend: 'Ext.panel.Panel',
>> alias: 'widget.pbsDatastorePruneAndGC',
>> @@ -99,9 +14,9 @@ Ext.define('PBS.Datastore.PruneAndGC', {
>> },
>> items: [
>> {
>> - xtype: 'pbsDatastoreGCOpts',
>> + xtype: 'pbsGCJobView',
>> title: gettext('Garbage Collection'),
>> - itemId: 'datastore-gc',
>> + itemId: 'datastore-gc-jobs',
>> nodename: 'localhost',
>> cbind: {
>> datastore: '{datastore}',
>> @@ -111,8 +26,6 @@ Ext.define('PBS.Datastore.PruneAndGC', {
>> xtype: 'pbsPruneJobView',
>> nodename: 'localhost',
>> itemId: 'datastore-prune-jobs',
>> - flex: 1,
>> - minHeight: 200,
>> cbind: {
>> datastore: '{datastore}',
>> },
>> --
>> 2.39.2
>>
>>
>>
>> _______________________________________________
>> pbs-devel mailing list
>> pbs-devel@lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
-------------------- End of forwarded message --------------------
^ permalink raw reply [flat|nested] only message in thread
only message in thread, other threads:[~2023-09-22 9:13 UTC | newest]
Thread overview: (only message) (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
[not found] <87o7huwq21.fsf@gmail.com>
[not found] ` <85e6743a-57ca-a662-847f-c102c66bccc7@proxmox.com>
2023-09-22 9:13 ` [pbs-devel] [Stefan Lendl] Re: [PATCH proxmox-backup v3] close #4723: updated gc view in the ui Stefan Lendl
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal