[pbs-devel] [PATCH proxmox{, -backup} v2 0/5] unmount datastores after sync job

[pbs-devel] [PATCH proxmox{, -backup} v2 0/5] unmount datastores after sync job

[Hannes Laimer]

Adds the option to automatically unmount a datastore after a sync job
finishes.

The idea is that, in combination with run-on-mount, it is possible to
have datastores sync to external drives without the need to open the web
ui or terminal. This came up a handful of times in support and a recent
thread on the forum. Basically, also non-tech people could be tasked
with plugging and unplugging different drives regularly and mounting,
sync and unmounting would be done automatically.

Currently if any of the triggered jobs have the 'unmount-on-done' flag
set the datastore will be unmounted right after the last of the
triggered jobs finishes.

This seemed pretty straight forward and should be good in most use-cases
I came up with. Also, I did consider having 'unmount-on-done' also for
normally(schedule/manual) started jobs, I guess there could be some
situations where that might useful. But, as I mentioned on the commit
itself, we'd probably have to go through the command socket since sync jobs run
on the proxy. And I did not think it adds that much, also not sure if
we'd even want that.

Tested-by: Robert Obkircher <r.obkircher@proxmox.com>

v2, thanks @Robert and @Shannon
 - include short docs section
 - fix typo
 - fix test
 - use `|=` (instead of `= ... || ...`)

proxmox:

Hannes Laimer (1):
  pbs-api-types: add 'unmount-on-done' field to sync job config

 pbs-api-types/src/jobs.rs | 8 ++++++++
 1 file changed, 8 insertions(+)


proxmox-backup:

Hannes Laimer (4):
  api: syncjob: correctly update/delete 'unmount-on-done' field
  api: datastore: unmount datastore after sync if configured
  ui: add 'unmount-on-done' field to SyncJobEdit window
  docs: add section about `unmount-on-done`

 docs/managing-remotes.rst   |  4 ++++
 src/api2/admin/datastore.rs | 21 +++++++++++++++++++--
 src/api2/config/sync.rs     |  9 +++++++++
 www/window/SyncJobEdit.js   | 23 +++++++++++++++++++++++
 4 files changed, 55 insertions(+), 2 deletions(-)


Summary over all repositories:
  5 files changed, 63 insertions(+), 2 deletions(-)

-- 
Generated by git-murpp 0.8.1


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox v2 1/1] pbs-api-types: add 'unmount-on-done' field to sync job config

[Hannes Laimer]

Tested-by: Robert Obkircher <r.obkircher@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
 pbs-api-types/src/jobs.rs | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/pbs-api-types/src/jobs.rs b/pbs-api-types/src/jobs.rs
index 3eb61cde..3284a4c8 100644
--- a/pbs-api-types/src/jobs.rs
+++ b/pbs-api-types/src/jobs.rs
@@ -538,6 +538,8 @@ pub const SYNC_VERIFIED_ONLY_SCHEMA: Schema =
     BooleanSchema::new("Only synchronize verified backup snapshots, exclude others.").schema();
 pub const RUN_SYNC_ON_MOUNT_SCHEMA: Schema =
     BooleanSchema::new("Run this job when a relevant datastore is mounted.").schema();
+pub const UNMOUNT_ON_SYNC_DONE_SCHEMA: Schema =
+    BooleanSchema::new("Unmount involved removable datastore after sync job finishes.").schema();
 
 #[api(
     properties: {
@@ -609,6 +611,10 @@ pub const RUN_SYNC_ON_MOUNT_SCHEMA: Schema =
             schema: RUN_SYNC_ON_MOUNT_SCHEMA,
             optional: true,
         },
+        "unmount-on-done": {
+            schema: UNMOUNT_ON_SYNC_DONE_SCHEMA,
+            optional: true,
+        },
         "sync-direction": {
             type: SyncDirection,
             optional: true,
@@ -655,6 +661,8 @@ pub struct SyncJobConfig {
     #[serde(skip_serializing_if = "Option::is_none")]
     pub run_on_mount: Option<bool>,
     #[serde(skip_serializing_if = "Option::is_none")]
+    pub unmount_on_done: Option<bool>,
+    #[serde(skip_serializing_if = "Option::is_none")]
     pub sync_direction: Option<SyncDirection>,
 }
 
-- 
2.47.3



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox-backup v2 1/4] api: syncjob: correctly update/delete 'unmount-on-done' field

[Hannes Laimer]

Tested-by: Robert Obkircher <r.obkircher@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
 src/api2/config/sync.rs | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/src/api2/config/sync.rs b/src/api2/config/sync.rs
index 358409b5..3b9d4286 100644
--- a/src/api2/config/sync.rs
+++ b/src/api2/config/sync.rs
@@ -341,6 +341,8 @@ pub enum DeletableProperty {
     VerifiedOnly,
     /// Delete the run_on_mount property,
     RunOnMount,
+    /// Delete the unmount_on_done property,
+    UnmountOnDone,
     /// Delete the sync_direction property,
     SyncDirection,
 }
@@ -463,6 +465,9 @@ pub fn update_sync_job(
                 DeletableProperty::RunOnMount => {
                     data.run_on_mount = None;
                 }
+                DeletableProperty::UnmountOnDone => {
+                    data.unmount_on_done = None;
+                }
                 DeletableProperty::SyncDirection => {
                     data.sync_direction = None;
                 }
@@ -515,6 +520,9 @@ pub fn update_sync_job(
     if let Some(run_on_mount) = update.run_on_mount {
         data.run_on_mount = Some(run_on_mount);
     }
+    if let Some(unmount_on_done) = update.unmount_on_done {
+        data.unmount_on_done = Some(unmount_on_done);
+    }
     if let Some(sync_direction) = update.sync_direction {
         data.sync_direction = Some(sync_direction);
     }
@@ -692,6 +700,7 @@ acl:1:/remote/remote1/remotestore1:write@pbs:RemoteSyncOperator
         encrypted_only: None,
         verified_only: None,
         run_on_mount: None,
+        unmount_on_done: None,
         sync_direction: None, // use default
     };
 
-- 
2.47.3



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox-backup v2 2/4] api: datastore: unmount datastore after sync if configured

[Hannes Laimer]

When a sync job is triggered by the mounting of a datastore, we now check
whether it should also be unmounted automatically afterwards. This is only
done for jobs triggered by mounting.

We do not do this for manually started or scheduled sync jobs, as those
run in the proxy process and therefore cannot call the privileged API
endpoint for unmounting.

The task that starts sync jobs on mount runs in the API process (where the
mounting occurs), so in that privileged context, we can also perform the
unmounting.

Tested-by: Robert Obkircher <r.obkircher@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
 src/api2/admin/datastore.rs | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/src/api2/admin/datastore.rs b/src/api2/admin/datastore.rs
index 643d1694..75122260 100644
--- a/src/api2/admin/datastore.rs
+++ b/src/api2/admin/datastore.rs
@@ -2430,6 +2430,7 @@ pub fn do_mount_device(datastore: DataStoreConfig) -> Result<bool, Error> {
 
 async fn do_sync_jobs(
     jobs_to_run: Vec<SyncJobConfig>,
+    store: String,
     worker: Arc<WorkerTask>,
 ) -> Result<(), Error> {
     let count = jobs_to_run.len();
@@ -2442,6 +2443,8 @@ async fn do_sync_jobs(
             .join(", ")
     );
 
+    let mut unmount_on_done = false;
+
     let client = crate::client_helpers::connect_to_localhost()
         .context("Failed to connect to localhost for starting sync jobs")?;
     for (i, job_config) in jobs_to_run.into_iter().enumerate() {
@@ -2484,7 +2487,21 @@ async fn do_sync_jobs(
                 }
             }
         }
+        unmount_on_done |= job_config.unmount_on_done.unwrap_or_default();
+    }
+    if unmount_on_done {
+        match client
+            .post(
+                format!("api2/json/admin/datastore/{store}/unmount").as_str(),
+                None,
+            )
+            .await
+        {
+            Ok(_) => info!("triggered unmounting successfully"),
+            Err(err) => warn!("could not unmount: {err}"),
+        };
     }
+
     Ok(())
 }
 
@@ -2566,10 +2583,10 @@ pub fn mount(store: String, rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Er
                 info!("starting {} sync jobs", jobs_to_run.len());
                 let _ = WorkerTask::spawn(
                     "mount-sync-jobs",
-                    Some(store),
+                    Some(store.clone()),
                     auth_id.to_string(),
                     false,
-                    move |worker| async move { do_sync_jobs(jobs_to_run, worker).await },
+                    move |worker| async move { do_sync_jobs(jobs_to_run, store, worker).await },
                 );
             }
             Ok(())
-- 
2.47.3



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox-backup v2 3/4] ui: add 'unmount-on-done' field to SyncJobEdit window

[Hannes Laimer]

Tested-by: Robert Obkircher <r.obkircher@proxmox.com>
Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
 www/window/SyncJobEdit.js | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/www/window/SyncJobEdit.js b/www/window/SyncJobEdit.js
index 14ffddcd..426c271f 100644
--- a/www/window/SyncJobEdit.js
+++ b/www/window/SyncJobEdit.js
@@ -122,6 +122,7 @@ Ext.define('PBS.window.SyncJobEdit', {
                     }
                     if (!me.isCreate) {
                         PBS.Utils.delete_if_default(values, 'run-on-mount', false);
+                        PBS.Utils.delete_if_default(values, 'unmount-on-done', false);
                         PBS.Utils.delete_if_default(values, 'rate-in');
                         PBS.Utils.delete_if_default(values, 'rate-out');
                         PBS.Utils.delete_if_default(values, 'remote');
@@ -499,8 +500,30 @@ Ext.define('PBS.window.SyncJobEdit', {
                                 'Run this job when a relevant removable datastore gets mounted.',
                             ),
                         },
+                        listeners: {
+                            change: function (field, runOnMount) {
+                                let me = this;
+                                let view = me.up('pbsSyncJobEdit');
+                                let unmountOnDoneCb = view.down('field[name=unmount-on-done]');
+                                unmountOnDoneCb.setDisabled(!runOnMount);
+                            },
+                        },
+                        uncheckedValue: false,
+                        value: false,
+                    },
+                    {
+                        xtype: 'proxmoxcheckbox',
+                        name: 'unmount-on-done',
+                        fieldLabel: gettext('Unmount when done'),
+                        autoEl: {
+                            tag: 'div',
+                            'data-qtip': gettext(
+                                'Unmount relevant removable datastore once sync job finishes.',
+                            ),
+                        },
                         uncheckedValue: false,
                         value: false,
+                        disabled: true,
                     },
                 ],
             },
-- 
2.47.3



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH proxmox-backup v2 4/4] docs: add section about `unmount-on-done`

[Hannes Laimer]

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
 docs/managing-remotes.rst | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/docs/managing-remotes.rst b/docs/managing-remotes.rst
index b6ebda18..60ac3bc6 100644
--- a/docs/managing-remotes.rst
+++ b/docs/managing-remotes.rst
@@ -150,6 +150,10 @@ relevant removable datastore is mounted. If mounting a removable datastore would
 multiple sync jobs, these jobs will be run sequentially in alphabetical order based on
 their ID.
 
+If the ``unmount-on-done`` flag is set, the datastore will be automatically unmounted
+after the sync job finishes. This option is only available for sync jobs triggered by
+mounting (``run-on-mount``), enabling fully automated external drive workflows.
+
 Namespace Support
 ^^^^^^^^^^^^^^^^^
 
-- 
2.47.3



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel