[pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications

[pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications

[Dominik Csapak]

combines my last three series[0][1][2], since they touch the same code

changes from v1:
* improve display trait code for timespan
* add tests for template registration
* rebase on master

0: https://lists.proxmox.com/pipermail/pbs-devel/2021-March/002438.html
1: https://lists.proxmox.com/pipermail/pbs-devel/2021-March/002441.html
2: https://lists.proxmox.com/pipermail/pbs-devel/2021-March/002431.html

Dominik Csapak (5):
  tools/systemd/time: implement some Traits for TimeSpan
  server/email_notifications: do not panic on template registration
  server/email_notifications: do not double html escape
  api2/tape/backup: include a summary on notification e-mails
  api2/tape/backup: wait indefinitely for lock in scheduled backup jobs

 src/api2/tape/backup.rs           | 85 ++++++++++++++++++++++---------
 src/api2/types/tape/mod.rs        |  9 ++++
 src/server/email_notifications.rs | 69 ++++++++++++++++++++-----
 src/tools/systemd/time.rs         | 82 +++++++++++++++++++++++++++++
 4 files changed, 209 insertions(+), 36 deletions(-)

-- 
2.20.1

[pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan

[Dominik Csapak]

namely
* From<Duration> (to convert easily from duration to timespan)
* Display (for better formatting)

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* reorder msec, usec
* improve display trait code

 src/tools/systemd/time.rs | 82 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 82 insertions(+)

diff --git a/src/tools/systemd/time.rs b/src/tools/systemd/time.rs
index 7cc42415..cbcb23df 100644
--- a/src/tools/systemd/time.rs
+++ b/src/tools/systemd/time.rs
@@ -141,6 +141,88 @@ impl From<TimeSpan> for f64 {
     }
 }
 
+impl From<std::time::Duration> for TimeSpan {
+    fn from(duration: std::time::Duration) -> Self {
+        let mut duration = duration.as_nanos();
+        let nsec = (duration % 1000) as u64;
+        duration /= 1000;
+        let usec = (duration % 1000) as u64;
+        duration /= 1000;
+        let msec = (duration % 1000) as u64;
+        duration /= 1000;
+        let seconds = (duration % 60) as u64;
+        duration /= 60;
+        let minutes = (duration % 60) as u64;
+        duration /= 60;
+        let hours = (duration % 24) as u64;
+        duration /= 24;
+        let years = (duration as f64 / 365.25) as u64;
+        let ydays = (duration as f64 % 365.25) as u64;
+        let months = (ydays as f64 / 30.44) as u64;
+        let mdays = (ydays as f64 % 30.44) as u64;
+        let weeks = mdays / 7;
+        let days = mdays % 7;
+        Self {
+            nsec,
+            usec,
+            msec,
+            seconds,
+            minutes,
+            hours,
+            days,
+            weeks,
+            months,
+            years,
+        }
+    }
+}
+
+impl std::fmt::Display for TimeSpan {
+    fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> Result<(), std::fmt::Error> {
+        let mut first = true;
+        { // block scope for mutable borrows
+            let mut do_write = |v: u64, unit: &str| -> Result<(), std::fmt::Error> {
+                if !first {
+                    write!(f, " ")?;
+                }
+                first = false;
+                write!(f, "{}{}", v, unit)
+            };
+            if self.years > 0 {
+                do_write(self.years, "y")?;
+            }
+            if self.months > 0 {
+                do_write(self.months, "m")?;
+            }
+            if self.weeks > 0 {
+                do_write(self.weeks, "w")?;
+            }
+            if self.days > 0 {
+                do_write(self.days, "d")?;
+            }
+            if self.hours > 0 {
+                do_write(self.hours, "h")?;
+            }
+            if self.minutes > 0 {
+                do_write(self.minutes, "min")?;
+            }
+        }
+        if !first {
+            write!(f, " ")?;
+        }
+        let seconds = self.seconds as f64 + (self.msec as f64 / 1000.0);
+        if seconds >= 0.1 {
+            if seconds >= 1.0 || !first {
+                write!(f, "{:.0}s", seconds)?;
+            } else {
+                write!(f, "{:.1}s", seconds)?;
+            }
+        } else if first {
+            write!(f, "<0.1s")?;
+        }
+        Ok(())
+    }
+}
 
 pub fn verify_time_span(i: &str) -> Result<(), Error> {
     parse_time_span(i)?;
-- 
2.20.1

[pbs-devel] [PATCH proxmox-backup v2 2/5] server/email_notifications: do not panic on template registration

[Dominik Csapak]

instead print an error and continue, the rendering functions will error
out if one of the templates could not be registered

if we `.unwrap()` here, it can lead to problems if the templates are
not correct, i.e. we could panic while holding a lock, if something holds
a mutex while this is called for the first time

add a test to catch registration issues during package build

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* add test to actually test registering, this will fail
  on package build if the template registering fails

 src/server/email_notifications.rs | 55 +++++++++++++++++++++++--------
 1 file changed, 42 insertions(+), 13 deletions(-)

diff --git a/src/server/email_notifications.rs b/src/server/email_notifications.rs
index 229443f6..cd5a43fe 100644
--- a/src/server/email_notifications.rs
+++ b/src/server/email_notifications.rs
@@ -1,10 +1,11 @@
 use anyhow::Error;
 use serde_json::json;
 
-use handlebars::{Handlebars, Helper, Context, RenderError, RenderContext, Output, HelperResult};
+use handlebars::{Handlebars, Helper, Context, RenderError, RenderContext, Output, HelperResult, TemplateError};
 
 use proxmox::tools::email::sendmail;
 use proxmox::api::schema::parse_property_string;
+use proxmox::try_block;
 
 use crate::{
     config::datastore::DataStoreConfig,
@@ -181,25 +182,33 @@ lazy_static::lazy_static!{
 
     static ref HANDLEBARS: Handlebars<'static> = {
         let mut hb = Handlebars::new();
+        let result: Result<(), TemplateError> = try_block!({
 
-        hb.set_strict_mode(true);
+            hb.set_strict_mode(true);
 
-        hb.register_helper("human-bytes", Box::new(handlebars_humam_bytes_helper));
-        hb.register_helper("relative-percentage", Box::new(handlebars_relative_percentage_helper));
+            hb.register_helper("human-bytes", Box::new(handlebars_humam_bytes_helper));
+            hb.register_helper("relative-percentage", Box::new(handlebars_relative_percentage_helper));
 
-        hb.register_template_string("gc_ok_template", GC_OK_TEMPLATE).unwrap();
-        hb.register_template_string("gc_err_template", GC_ERR_TEMPLATE).unwrap();
+            hb.register_template_string("gc_ok_template", GC_OK_TEMPLATE)?;
+            hb.register_template_string("gc_err_template", GC_ERR_TEMPLATE)?;
 
-        hb.register_template_string("verify_ok_template", VERIFY_OK_TEMPLATE).unwrap();
-        hb.register_template_string("verify_err_template", VERIFY_ERR_TEMPLATE).unwrap();
+            hb.register_template_string("verify_ok_template", VERIFY_OK_TEMPLATE)?;
+            hb.register_template_string("verify_err_template", VERIFY_ERR_TEMPLATE)?;
 
-        hb.register_template_string("sync_ok_template", SYNC_OK_TEMPLATE).unwrap();
-        hb.register_template_string("sync_err_template", SYNC_ERR_TEMPLATE).unwrap();
+            hb.register_template_string("sync_ok_template", SYNC_OK_TEMPLATE)?;
+            hb.register_template_string("sync_err_template", SYNC_ERR_TEMPLATE)?;
 
-        hb.register_template_string("tape_backup_ok_template", TAPE_BACKUP_OK_TEMPLATE).unwrap();
-        hb.register_template_string("tape_backup_err_template", TAPE_BACKUP_ERR_TEMPLATE).unwrap();
+            hb.register_template_string("tape_backup_ok_template", TAPE_BACKUP_OK_TEMPLATE)?;
+            hb.register_template_string("tape_backup_err_template", TAPE_BACKUP_ERR_TEMPLATE)?;
 
-        hb.register_template_string("package_update_template", PACKAGE_UPDATES_TEMPLATE).unwrap();
+            hb.register_template_string("package_update_template", PACKAGE_UPDATES_TEMPLATE)?;
+
+            Ok(())
+        });
+
+        if let Err(err) = result {
+            eprintln!("error during template registration: {}", err);
+        }
 
         hb
     };
@@ -600,3 +609,23 @@ fn handlebars_relative_percentage_helper(
     }
     Ok(())
 }
+
+#[test]
+fn test_template_register() {
+    HANDLEBARS.get_helper("human-bytes").unwrap();
+    HANDLEBARS.get_helper("relative-percentage").unwrap();
+
+    assert!(HANDLEBARS.has_template("gc_ok_template"));
+    assert!(HANDLEBARS.has_template("gc_err_template"));
+
+    assert!(HANDLEBARS.has_template("verify_ok_template"));
+    assert!(HANDLEBARS.has_template("verify_err_template"));
+
+    assert!(HANDLEBARS.has_template("sync_ok_template"));
+    assert!(HANDLEBARS.has_template("sync_err_template"));
+
+    assert!(HANDLEBARS.has_template("tape_backup_ok_template"));
+    assert!(HANDLEBARS.has_template("tape_backup_err_template"));
+
+    assert!(HANDLEBARS.has_template("package_update_template"));
+}
-- 
2.20.1

[pbs-devel] [PATCH proxmox-backup v2 3/5] server/email_notifications: do not double html escape

[Dominik Csapak]

the default escape handler is handlebars::html_escape, but this are
plain text emails and we manually escape them for the html part, so
set the default escape handler to 'no_escape'

this avoids double html escape for the characters: '&"<>' in emails

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/server/email_notifications.rs | 1 +
 1 file changed, 1 insertion(+)

diff --git a/src/server/email_notifications.rs b/src/server/email_notifications.rs
index cd5a43fe..e92c8091 100644
--- a/src/server/email_notifications.rs
+++ b/src/server/email_notifications.rs
@@ -185,6 +185,7 @@ lazy_static::lazy_static!{
         let result: Result<(), TemplateError> = try_block!({
 
             hb.set_strict_mode(true);
+            hb.register_escape_fn(handlebars::no_escape);
 
             hb.register_helper("human-bytes", Box::new(handlebars_humam_bytes_helper));
             hb.register_helper("relative-percentage", Box::new(handlebars_relative_percentage_helper));
-- 
2.20.1

[pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails

[Dominik Csapak]

for now only contains the list of included snapshots (if any),
as well as the backup duration

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/api2/tape/backup.rs           | 32 +++++++++++++++++++++++++------
 src/api2/types/tape/mod.rs        |  9 +++++++++
 src/server/email_notifications.rs | 13 +++++++++++++
 3 files changed, 48 insertions(+), 6 deletions(-)

diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
index 690b6855..0d42c31a 100644
--- a/src/api2/tape/backup.rs
+++ b/src/api2/tape/backup.rs
@@ -51,6 +51,7 @@ use crate::{
         JOB_ID_SCHEMA,
         MediaPoolConfig,
         Userid,
+        TapeBackupJobSummary,
     },
     server::WorkerTask,
     task::TaskState,
@@ -198,13 +199,16 @@ pub fn do_tape_backup_job(
             let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
             let email = lookup_user_email(notify_user);
 
-            let job_result = backup_worker(
+            let (job_result, summary) = match backup_worker(
                 &worker,
                 datastore,
                 &pool_config,
                 &setup,
                 email.clone(),
-            );
+            ) {
+                Ok(summary) => (Ok(()), summary),
+                Err(err) => (Err(err), Default::default()),
+            };
 
             let status = worker.create_state(&job_result);
 
@@ -214,6 +218,7 @@ pub fn do_tape_backup_job(
                     Some(job.jobname()),
                     &setup,
                     &job_result,
+                    summary,
                 ) {
                     eprintln!("send tape backup notification failed: {}", err);
                 }
@@ -340,13 +345,17 @@ pub fn backup(
         move |worker| {
             let _drive_lock = drive_lock; // keep lock guard
             set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
-            let job_result = backup_worker(
+
+            let (job_result, summary) = match backup_worker(
                 &worker,
                 datastore,
                 &pool_config,
                 &setup,
                 email.clone(),
-            );
+            ) {
+                Ok(summary) => (Ok(()), summary),
+                Err(err) => (Err(err), Default::default()),
+            };
 
             if let Some(email) = email {
                 if let Err(err) = crate::server::send_tape_backup_status(
@@ -354,6 +363,7 @@ pub fn backup(
                     None,
                     &setup,
                     &job_result,
+                    summary,
                 ) {
                     eprintln!("send tape backup notification failed: {}", err);
                 }
@@ -374,9 +384,11 @@ fn backup_worker(
     pool_config: &MediaPoolConfig,
     setup: &TapeBackupJobSetup,
     email: Option<String>,
-) -> Result<(), Error> {
+) -> Result<TapeBackupJobSummary, Error> {
 
     let status_path = Path::new(TAPE_STATUS_DIR);
+    let start = std::time::Instant::now();
+    let mut summary: TapeBackupJobSummary = Default::default();
 
     let _lock = MediaPool::lock(status_path, &pool_config.name)?;
 
@@ -422,8 +434,11 @@ fn backup_worker(
                     task_log!(worker, "skip snapshot {}", info.backup_dir);
                     continue;
                 }
+                let snapshot_name = info.backup_dir.to_string();
                 if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
                     errors = true;
+                } else {
+                    summary.snapshot_list.push(snapshot_name);
                 }
                 progress.done_snapshots = 1;
                 task_log!(
@@ -439,8 +454,11 @@ fn backup_worker(
                     task_log!(worker, "skip snapshot {}", info.backup_dir);
                     continue;
                 }
+                let snapshot_name = info.backup_dir.to_string();
                 if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
                     errors = true;
+                } else {
+                    summary.snapshot_list.push(snapshot_name);
                 }
                 progress.done_snapshots = snapshot_number as u64 + 1;
                 task_log!(
@@ -464,7 +482,9 @@ fn backup_worker(
         bail!("Tape backup finished with some errors. Please check the task log.");
     }
 
-    Ok(())
+    summary.duration = start.elapsed();
+
+    Ok(summary)
 }
 
 // Try to update the the media online status
diff --git a/src/api2/types/tape/mod.rs b/src/api2/types/tape/mod.rs
index 68b2cf12..44c2236b 100644
--- a/src/api2/types/tape/mod.rs
+++ b/src/api2/types/tape/mod.rs
@@ -20,3 +20,12 @@ pub use media_location::*;
 
 mod media;
 pub use media::*;
+
+/// Summary of a successful Tape Job
+#[derive(Default)]
+pub struct TapeBackupJobSummary {
+    /// The list of snaphots backed up
+    pub snapshot_list: Vec<String>,
+    /// The total time of the backup job
+    pub duration: std::time::Duration,
+}
diff --git a/src/server/email_notifications.rs b/src/server/email_notifications.rs
index e92c8091..26002d39 100644
--- a/src/server/email_notifications.rs
+++ b/src/server/email_notifications.rs
@@ -18,6 +18,7 @@ use crate::{
         Userid,
         Notify,
         DatastoreNotify,
+        TapeBackupJobSummary,
     },
     tools::format::HumanByte,
 };
@@ -149,6 +150,14 @@ Datastore:  {{job.store}}
 Tape Pool:  {{job.pool}}
 Tape Drive: {{job.drive}}
 
+{{#if snapshot-list ~}}
+Snapshots included:
+
+{{#each snapshot-list~}}
+{{this}}
+{{/each~}}
+{{/if}}
+Duration: {{duration}}
 
 Tape Backup successful.
 
@@ -412,14 +421,18 @@ pub fn send_tape_backup_status(
     id: Option<&str>,
     job: &TapeBackupJobSetup,
     result: &Result<(), Error>,
+    summary: TapeBackupJobSummary,
 ) -> Result<(), Error> {
 
     let (fqdn, port) = get_server_url();
+    let duration: crate::tools::systemd::time::TimeSpan = summary.duration.into();
     let mut data = json!({
         "job": job,
         "fqdn": fqdn,
         "port": port,
         "id": id,
+        "snapshot-list": summary.snapshot_list,
+        "duration": duration.to_string(),
     });
 
     let text = match result {
-- 
2.20.1

Re: [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails

[Dietmar Maurer]

comments inline

> On 03/18/2021 1:01 PM Dominik Csapak <d.csapak@proxmox.com> wrote:
> 
>  
> for now only contains the list of included snapshots (if any),
> as well as the backup duration
> 
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
>  src/api2/tape/backup.rs           | 32 +++++++++++++++++++++++++------
>  src/api2/types/tape/mod.rs        |  9 +++++++++
>  src/server/email_notifications.rs | 13 +++++++++++++
>  3 files changed, 48 insertions(+), 6 deletions(-)
> 
> diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
> index 690b6855..0d42c31a 100644
> --- a/src/api2/tape/backup.rs
> +++ b/src/api2/tape/backup.rs
> @@ -51,6 +51,7 @@ use crate::{
>          JOB_ID_SCHEMA,
>          MediaPoolConfig,
>          Userid,
> +        TapeBackupJobSummary,
>      },
>      server::WorkerTask,
>      task::TaskState,
> @@ -198,13 +199,16 @@ pub fn do_tape_backup_job(
>              let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
>              let email = lookup_user_email(notify_user);
>  
> -            let job_result = backup_worker(
> +            let (job_result, summary) = match backup_worker(
>                  &worker,
>                  datastore,
>                  &pool_config,
>                  &setup,
>                  email.clone(),
> -            );
> +            ) {
> +                Ok(summary) => (Ok(()), summary),
> +                Err(err) => (Err(err), Default::default()),
> +            };
>  
>              let status = worker.create_state(&job_result);
>  
> @@ -214,6 +218,7 @@ pub fn do_tape_backup_job(
>                      Some(job.jobname()),
>                      &setup,
>                      &job_result,
> +                    summary,
>                  ) {
>                      eprintln!("send tape backup notification failed: {}", err);
>                  }
> @@ -340,13 +345,17 @@ pub fn backup(
>          move |worker| {
>              let _drive_lock = drive_lock; // keep lock guard
>              set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
> -            let job_result = backup_worker(
> +
> +            let (job_result, summary) = match backup_worker(
>                  &worker,
>                  datastore,
>                  &pool_config,
>                  &setup,
>                  email.clone(),
> -            );
> +            ) {
> +                Ok(summary) => (Ok(()), summary),
> +                Err(err) => (Err(err), Default::default()),
> +            };
>  
>              if let Some(email) = email {
>                  if let Err(err) = crate::server::send_tape_backup_status(
> @@ -354,6 +363,7 @@ pub fn backup(
>                      None,
>                      &setup,
>                      &job_result,
> +                    summary,
>                  ) {
>                      eprintln!("send tape backup notification failed: {}", err);
>                  }
> @@ -374,9 +384,11 @@ fn backup_worker(
>      pool_config: &MediaPoolConfig,
>      setup: &TapeBackupJobSetup,
>      email: Option<String>,
> -) -> Result<(), Error> {
> +) -> Result<TapeBackupJobSummary, Error> {
>  
>      let status_path = Path::new(TAPE_STATUS_DIR);
> +    let start = std::time::Instant::now();
> +    let mut summary: TapeBackupJobSummary = Default::default();
>  
>      let _lock = MediaPool::lock(status_path, &pool_config.name)?;
>  
> @@ -422,8 +434,11 @@ fn backup_worker(
>                      task_log!(worker, "skip snapshot {}", info.backup_dir);
>                      continue;
>                  }
> +                let snapshot_name = info.backup_dir.to_string();
>                  if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
>                      errors = true;
> +                } else {
> +                    summary.snapshot_list.push(snapshot_name);
>                  }
>                  progress.done_snapshots = 1;
>                  task_log!(
> @@ -439,8 +454,11 @@ fn backup_worker(
>                      task_log!(worker, "skip snapshot {}", info.backup_dir);
>                      continue;
>                  }
> +                let snapshot_name = info.backup_dir.to_string();
>                  if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
>                      errors = true;
> +                } else {
> +                    summary.snapshot_list.push(snapshot_name);
>                  }
>                  progress.done_snapshots = snapshot_number as u64 + 1;
>                  task_log!(
> @@ -464,7 +482,9 @@ fn backup_worker(
>          bail!("Tape backup finished with some errors. Please check the task log.");
>      }
>  
> -    Ok(())
> +    summary.duration = start.elapsed();
> +
> +    Ok(summary)
>  }
>  
>  // Try to update the the media online status
> diff --git a/src/api2/types/tape/mod.rs b/src/api2/types/tape/mod.rs
> index 68b2cf12..44c2236b 100644
> --- a/src/api2/types/tape/mod.rs
> +++ b/src/api2/types/tape/mod.rs
> @@ -20,3 +20,12 @@ pub use media_location::*;
>  
>  mod media;
>  pub use media::*;
> +
> +/// Summary of a successful Tape Job
> +#[derive(Default)]
> +pub struct TapeBackupJobSummary {
> +    /// The list of snaphots backed up
> +    pub snapshot_list: Vec<String>,
> +    /// The total time of the backup job
> +    pub duration: std::time::Duration,
> +}

This is not an API type, and is not use in the API at all!
Please can  you define this type somewhere else?

[pbs-devel] [PATCH proxmox-backup v2 5/5] api2/tape/backup: wait indefinitely for lock in scheduled backup jobs

[Dominik Csapak]

so that a user can schedule multiple backup jobs onto a single
media pool without having to consider timing them apart

this makes sense since we can backup multiple datastores onto
the same media-set but can only specify one datastore per backup job

Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
 src/api2/tape/backup.rs | 57 +++++++++++++++++++++++++++--------------
 1 file changed, 38 insertions(+), 19 deletions(-)

diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
index 0d42c31a..f29eb715 100644
--- a/src/api2/tape/backup.rs
+++ b/src/api2/tape/backup.rs
@@ -5,6 +5,7 @@ use anyhow::{bail, format_err, Error};
 use serde_json::Value;
 
 use proxmox::{
+    try_block,
     api::{
         api,
         RpcEnvironment,
@@ -177,8 +178,15 @@ pub fn do_tape_backup_job(
 
     let (drive_config, _digest) = config::drive::config()?;
 
-    // early check/lock before starting worker
-    let drive_lock = lock_tape_device(&drive_config, &setup.drive)?;
+    // for scheduled jobs we acquire the lock later in the worker
+    let drive_lock = if schedule.is_some() {
+        None
+    } else {
+        Some(lock_tape_device(&drive_config, &setup.drive)?)
+    };
+
+    let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
+    let email = lookup_user_email(notify_user);
 
     let upid_str = WorkerTask::new_thread(
         &worker_type,
@@ -186,26 +194,37 @@ pub fn do_tape_backup_job(
         auth_id.clone(),
         false,
         move |worker| {
-            let _drive_lock = drive_lock; // keep lock guard
-
-            set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
             job.start(&worker.upid().to_string())?;
+            let mut drive_lock = drive_lock;
+
+            let (job_result, summary) = match try_block!({
+                if schedule.is_some() {
+                    // for scheduled tape backup jobs, we wait indefinitely for the lock
+                    task_log!(worker, "waiting for drive lock...");
+                    loop {
+                        if let Ok(lock) = lock_tape_device(&drive_config, &setup.drive) {
+                            drive_lock = Some(lock);
+                            break;
+                        } // ignore errors
+
+                        worker.check_abort()?;
+                    }
+                }
+                set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
 
-            task_log!(worker,"Starting tape backup job '{}'", job_id);
-            if let Some(event_str) = schedule {
-                task_log!(worker,"task triggered by schedule '{}'", event_str);
-            }
-
-            let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
-            let email = lookup_user_email(notify_user);
+                task_log!(worker,"Starting tape backup job '{}'", job_id);
+                if let Some(event_str) = schedule {
+                    task_log!(worker,"task triggered by schedule '{}'", event_str);
+                }
 
-            let (job_result, summary) = match backup_worker(
-                &worker,
-                datastore,
-                &pool_config,
-                &setup,
-                email.clone(),
-            ) {
+                backup_worker(
+                    &worker,
+                    datastore,
+                    &pool_config,
+                    &setup,
+                    email.clone(),
+                )
+            }) {
                 Ok(summary) => (Ok(()), summary),
                 Err(err) => (Err(err), Default::default()),
             };
-- 
2.20.1

Re: [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications

[Dietmar Maurer]

applied patch 1, 2 an d 3

please rework 4, and rebase 5 (does not apply anymore)