* [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications
@ 2021-03-18 12:01 Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan Dominik Csapak
` (5 more replies)
0 siblings, 6 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
combines my last three series[0][1][2], since they touch the same code
changes from v1:
* improve display trait code for timespan
* add tests for template registration
* rebase on master
0: https://lists.proxmox.com/pipermail/pbs-devel/2021-March/002438.html
1: https://lists.proxmox.com/pipermail/pbs-devel/2021-March/002441.html
2: https://lists.proxmox.com/pipermail/pbs-devel/2021-March/002431.html
Dominik Csapak (5):
tools/systemd/time: implement some Traits for TimeSpan
server/email_notifications: do not panic on template registration
server/email_notifications: do not double html escape
api2/tape/backup: include a summary on notification e-mails
api2/tape/backup: wait indefinitely for lock in scheduled backup jobs
src/api2/tape/backup.rs | 85 ++++++++++++++++++++++---------
src/api2/types/tape/mod.rs | 9 ++++
src/server/email_notifications.rs | 69 ++++++++++++++++++++-----
src/tools/systemd/time.rs | 82 +++++++++++++++++++++++++++++
4 files changed, 209 insertions(+), 36 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 2/5] server/email_notifications: do not panic on template registration Dominik Csapak
` (4 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
namely
* From<Duration> (to convert easily from duration to timespan)
* Display (for better formatting)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* reorder msec, usec
* improve display trait code
src/tools/systemd/time.rs | 82 +++++++++++++++++++++++++++++++++++++++
1 file changed, 82 insertions(+)
diff --git a/src/tools/systemd/time.rs b/src/tools/systemd/time.rs
index 7cc42415..cbcb23df 100644
--- a/src/tools/systemd/time.rs
+++ b/src/tools/systemd/time.rs
@@ -141,6 +141,88 @@ impl From<TimeSpan> for f64 {
}
}
+impl From<std::time::Duration> for TimeSpan {
+ fn from(duration: std::time::Duration) -> Self {
+ let mut duration = duration.as_nanos();
+ let nsec = (duration % 1000) as u64;
+ duration /= 1000;
+ let usec = (duration % 1000) as u64;
+ duration /= 1000;
+ let msec = (duration % 1000) as u64;
+ duration /= 1000;
+ let seconds = (duration % 60) as u64;
+ duration /= 60;
+ let minutes = (duration % 60) as u64;
+ duration /= 60;
+ let hours = (duration % 24) as u64;
+ duration /= 24;
+ let years = (duration as f64 / 365.25) as u64;
+ let ydays = (duration as f64 % 365.25) as u64;
+ let months = (ydays as f64 / 30.44) as u64;
+ let mdays = (ydays as f64 % 30.44) as u64;
+ let weeks = mdays / 7;
+ let days = mdays % 7;
+ Self {
+ nsec,
+ usec,
+ msec,
+ seconds,
+ minutes,
+ hours,
+ days,
+ weeks,
+ months,
+ years,
+ }
+ }
+}
+
+impl std::fmt::Display for TimeSpan {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> Result<(), std::fmt::Error> {
+ let mut first = true;
+ { // block scope for mutable borrows
+ let mut do_write = |v: u64, unit: &str| -> Result<(), std::fmt::Error> {
+ if !first {
+ write!(f, " ")?;
+ }
+ first = false;
+ write!(f, "{}{}", v, unit)
+ };
+ if self.years > 0 {
+ do_write(self.years, "y")?;
+ }
+ if self.months > 0 {
+ do_write(self.months, "m")?;
+ }
+ if self.weeks > 0 {
+ do_write(self.weeks, "w")?;
+ }
+ if self.days > 0 {
+ do_write(self.days, "d")?;
+ }
+ if self.hours > 0 {
+ do_write(self.hours, "h")?;
+ }
+ if self.minutes > 0 {
+ do_write(self.minutes, "min")?;
+ }
+ }
+ if !first {
+ write!(f, " ")?;
+ }
+ let seconds = self.seconds as f64 + (self.msec as f64 / 1000.0);
+ if seconds >= 0.1 {
+ if seconds >= 1.0 || !first {
+ write!(f, "{:.0}s", seconds)?;
+ } else {
+ write!(f, "{:.1}s", seconds)?;
+ }
+ } else if first {
+ write!(f, "<0.1s")?;
+ }
+ Ok(())
+ }
+}
pub fn verify_time_span(i: &str) -> Result<(), Error> {
parse_time_span(i)?;
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 2/5] server/email_notifications: do not panic on template registration
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 3/5] server/email_notifications: do not double html escape Dominik Csapak
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
instead print an error and continue, the rendering functions will error
out if one of the templates could not be registered
if we `.unwrap()` here, it can lead to problems if the templates are
not correct, i.e. we could panic while holding a lock, if something holds
a mutex while this is called for the first time
add a test to catch registration issues during package build
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* add test to actually test registering, this will fail
on package build if the template registering fails
src/server/email_notifications.rs | 55 +++++++++++++++++++++++--------
1 file changed, 42 insertions(+), 13 deletions(-)
diff --git a/src/server/email_notifications.rs b/src/server/email_notifications.rs
index 229443f6..cd5a43fe 100644
--- a/src/server/email_notifications.rs
+++ b/src/server/email_notifications.rs
@@ -1,10 +1,11 @@
use anyhow::Error;
use serde_json::json;
-use handlebars::{Handlebars, Helper, Context, RenderError, RenderContext, Output, HelperResult};
+use handlebars::{Handlebars, Helper, Context, RenderError, RenderContext, Output, HelperResult, TemplateError};
use proxmox::tools::email::sendmail;
use proxmox::api::schema::parse_property_string;
+use proxmox::try_block;
use crate::{
config::datastore::DataStoreConfig,
@@ -181,25 +182,33 @@ lazy_static::lazy_static!{
static ref HANDLEBARS: Handlebars<'static> = {
let mut hb = Handlebars::new();
+ let result: Result<(), TemplateError> = try_block!({
- hb.set_strict_mode(true);
+ hb.set_strict_mode(true);
- hb.register_helper("human-bytes", Box::new(handlebars_humam_bytes_helper));
- hb.register_helper("relative-percentage", Box::new(handlebars_relative_percentage_helper));
+ hb.register_helper("human-bytes", Box::new(handlebars_humam_bytes_helper));
+ hb.register_helper("relative-percentage", Box::new(handlebars_relative_percentage_helper));
- hb.register_template_string("gc_ok_template", GC_OK_TEMPLATE).unwrap();
- hb.register_template_string("gc_err_template", GC_ERR_TEMPLATE).unwrap();
+ hb.register_template_string("gc_ok_template", GC_OK_TEMPLATE)?;
+ hb.register_template_string("gc_err_template", GC_ERR_TEMPLATE)?;
- hb.register_template_string("verify_ok_template", VERIFY_OK_TEMPLATE).unwrap();
- hb.register_template_string("verify_err_template", VERIFY_ERR_TEMPLATE).unwrap();
+ hb.register_template_string("verify_ok_template", VERIFY_OK_TEMPLATE)?;
+ hb.register_template_string("verify_err_template", VERIFY_ERR_TEMPLATE)?;
- hb.register_template_string("sync_ok_template", SYNC_OK_TEMPLATE).unwrap();
- hb.register_template_string("sync_err_template", SYNC_ERR_TEMPLATE).unwrap();
+ hb.register_template_string("sync_ok_template", SYNC_OK_TEMPLATE)?;
+ hb.register_template_string("sync_err_template", SYNC_ERR_TEMPLATE)?;
- hb.register_template_string("tape_backup_ok_template", TAPE_BACKUP_OK_TEMPLATE).unwrap();
- hb.register_template_string("tape_backup_err_template", TAPE_BACKUP_ERR_TEMPLATE).unwrap();
+ hb.register_template_string("tape_backup_ok_template", TAPE_BACKUP_OK_TEMPLATE)?;
+ hb.register_template_string("tape_backup_err_template", TAPE_BACKUP_ERR_TEMPLATE)?;
- hb.register_template_string("package_update_template", PACKAGE_UPDATES_TEMPLATE).unwrap();
+ hb.register_template_string("package_update_template", PACKAGE_UPDATES_TEMPLATE)?;
+
+ Ok(())
+ });
+
+ if let Err(err) = result {
+ eprintln!("error during template registration: {}", err);
+ }
hb
};
@@ -600,3 +609,23 @@ fn handlebars_relative_percentage_helper(
}
Ok(())
}
+
+#[test]
+fn test_template_register() {
+ HANDLEBARS.get_helper("human-bytes").unwrap();
+ HANDLEBARS.get_helper("relative-percentage").unwrap();
+
+ assert!(HANDLEBARS.has_template("gc_ok_template"));
+ assert!(HANDLEBARS.has_template("gc_err_template"));
+
+ assert!(HANDLEBARS.has_template("verify_ok_template"));
+ assert!(HANDLEBARS.has_template("verify_err_template"));
+
+ assert!(HANDLEBARS.has_template("sync_ok_template"));
+ assert!(HANDLEBARS.has_template("sync_err_template"));
+
+ assert!(HANDLEBARS.has_template("tape_backup_ok_template"));
+ assert!(HANDLEBARS.has_template("tape_backup_err_template"));
+
+ assert!(HANDLEBARS.has_template("package_update_template"));
+}
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 3/5] server/email_notifications: do not double html escape
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 2/5] server/email_notifications: do not panic on template registration Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails Dominik Csapak
` (2 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
the default escape handler is handlebars::html_escape, but this are
plain text emails and we manually escape them for the html part, so
set the default escape handler to 'no_escape'
this avoids double html escape for the characters: '&"<>' in emails
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/server/email_notifications.rs | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/server/email_notifications.rs b/src/server/email_notifications.rs
index cd5a43fe..e92c8091 100644
--- a/src/server/email_notifications.rs
+++ b/src/server/email_notifications.rs
@@ -185,6 +185,7 @@ lazy_static::lazy_static!{
let result: Result<(), TemplateError> = try_block!({
hb.set_strict_mode(true);
+ hb.register_escape_fn(handlebars::no_escape);
hb.register_helper("human-bytes", Box::new(handlebars_humam_bytes_helper));
hb.register_helper("relative-percentage", Box::new(handlebars_relative_percentage_helper));
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
` (2 preceding siblings ...)
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 3/5] server/email_notifications: do not double html escape Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-19 6:07 ` Dietmar Maurer
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 5/5] api2/tape/backup: wait indefinitely for lock in scheduled backup jobs Dominik Csapak
2021-03-19 6:09 ` [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dietmar Maurer
5 siblings, 1 reply; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
for now only contains the list of included snapshots (if any),
as well as the backup duration
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/api2/tape/backup.rs | 32 +++++++++++++++++++++++++------
src/api2/types/tape/mod.rs | 9 +++++++++
src/server/email_notifications.rs | 13 +++++++++++++
3 files changed, 48 insertions(+), 6 deletions(-)
diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
index 690b6855..0d42c31a 100644
--- a/src/api2/tape/backup.rs
+++ b/src/api2/tape/backup.rs
@@ -51,6 +51,7 @@ use crate::{
JOB_ID_SCHEMA,
MediaPoolConfig,
Userid,
+ TapeBackupJobSummary,
},
server::WorkerTask,
task::TaskState,
@@ -198,13 +199,16 @@ pub fn do_tape_backup_job(
let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
let email = lookup_user_email(notify_user);
- let job_result = backup_worker(
+ let (job_result, summary) = match backup_worker(
&worker,
datastore,
&pool_config,
&setup,
email.clone(),
- );
+ ) {
+ Ok(summary) => (Ok(()), summary),
+ Err(err) => (Err(err), Default::default()),
+ };
let status = worker.create_state(&job_result);
@@ -214,6 +218,7 @@ pub fn do_tape_backup_job(
Some(job.jobname()),
&setup,
&job_result,
+ summary,
) {
eprintln!("send tape backup notification failed: {}", err);
}
@@ -340,13 +345,17 @@ pub fn backup(
move |worker| {
let _drive_lock = drive_lock; // keep lock guard
set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
- let job_result = backup_worker(
+
+ let (job_result, summary) = match backup_worker(
&worker,
datastore,
&pool_config,
&setup,
email.clone(),
- );
+ ) {
+ Ok(summary) => (Ok(()), summary),
+ Err(err) => (Err(err), Default::default()),
+ };
if let Some(email) = email {
if let Err(err) = crate::server::send_tape_backup_status(
@@ -354,6 +363,7 @@ pub fn backup(
None,
&setup,
&job_result,
+ summary,
) {
eprintln!("send tape backup notification failed: {}", err);
}
@@ -374,9 +384,11 @@ fn backup_worker(
pool_config: &MediaPoolConfig,
setup: &TapeBackupJobSetup,
email: Option<String>,
-) -> Result<(), Error> {
+) -> Result<TapeBackupJobSummary, Error> {
let status_path = Path::new(TAPE_STATUS_DIR);
+ let start = std::time::Instant::now();
+ let mut summary: TapeBackupJobSummary = Default::default();
let _lock = MediaPool::lock(status_path, &pool_config.name)?;
@@ -422,8 +434,11 @@ fn backup_worker(
task_log!(worker, "skip snapshot {}", info.backup_dir);
continue;
}
+ let snapshot_name = info.backup_dir.to_string();
if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
errors = true;
+ } else {
+ summary.snapshot_list.push(snapshot_name);
}
progress.done_snapshots = 1;
task_log!(
@@ -439,8 +454,11 @@ fn backup_worker(
task_log!(worker, "skip snapshot {}", info.backup_dir);
continue;
}
+ let snapshot_name = info.backup_dir.to_string();
if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
errors = true;
+ } else {
+ summary.snapshot_list.push(snapshot_name);
}
progress.done_snapshots = snapshot_number as u64 + 1;
task_log!(
@@ -464,7 +482,9 @@ fn backup_worker(
bail!("Tape backup finished with some errors. Please check the task log.");
}
- Ok(())
+ summary.duration = start.elapsed();
+
+ Ok(summary)
}
// Try to update the the media online status
diff --git a/src/api2/types/tape/mod.rs b/src/api2/types/tape/mod.rs
index 68b2cf12..44c2236b 100644
--- a/src/api2/types/tape/mod.rs
+++ b/src/api2/types/tape/mod.rs
@@ -20,3 +20,12 @@ pub use media_location::*;
mod media;
pub use media::*;
+
+/// Summary of a successful Tape Job
+#[derive(Default)]
+pub struct TapeBackupJobSummary {
+ /// The list of snaphots backed up
+ pub snapshot_list: Vec<String>,
+ /// The total time of the backup job
+ pub duration: std::time::Duration,
+}
diff --git a/src/server/email_notifications.rs b/src/server/email_notifications.rs
index e92c8091..26002d39 100644
--- a/src/server/email_notifications.rs
+++ b/src/server/email_notifications.rs
@@ -18,6 +18,7 @@ use crate::{
Userid,
Notify,
DatastoreNotify,
+ TapeBackupJobSummary,
},
tools::format::HumanByte,
};
@@ -149,6 +150,14 @@ Datastore: {{job.store}}
Tape Pool: {{job.pool}}
Tape Drive: {{job.drive}}
+{{#if snapshot-list ~}}
+Snapshots included:
+
+{{#each snapshot-list~}}
+{{this}}
+{{/each~}}
+{{/if}}
+Duration: {{duration}}
Tape Backup successful.
@@ -412,14 +421,18 @@ pub fn send_tape_backup_status(
id: Option<&str>,
job: &TapeBackupJobSetup,
result: &Result<(), Error>,
+ summary: TapeBackupJobSummary,
) -> Result<(), Error> {
let (fqdn, port) = get_server_url();
+ let duration: crate::tools::systemd::time::TimeSpan = summary.duration.into();
let mut data = json!({
"job": job,
"fqdn": fqdn,
"port": port,
"id": id,
+ "snapshot-list": summary.snapshot_list,
+ "duration": duration.to_string(),
});
let text = match result {
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 5/5] api2/tape/backup: wait indefinitely for lock in scheduled backup jobs
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
` (3 preceding siblings ...)
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-19 6:09 ` [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dietmar Maurer
5 siblings, 0 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
so that a user can schedule multiple backup jobs onto a single
media pool without having to consider timing them apart
this makes sense since we can backup multiple datastores onto
the same media-set but can only specify one datastore per backup job
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/api2/tape/backup.rs | 57 +++++++++++++++++++++++++++--------------
1 file changed, 38 insertions(+), 19 deletions(-)
diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
index 0d42c31a..f29eb715 100644
--- a/src/api2/tape/backup.rs
+++ b/src/api2/tape/backup.rs
@@ -5,6 +5,7 @@ use anyhow::{bail, format_err, Error};
use serde_json::Value;
use proxmox::{
+ try_block,
api::{
api,
RpcEnvironment,
@@ -177,8 +178,15 @@ pub fn do_tape_backup_job(
let (drive_config, _digest) = config::drive::config()?;
- // early check/lock before starting worker
- let drive_lock = lock_tape_device(&drive_config, &setup.drive)?;
+ // for scheduled jobs we acquire the lock later in the worker
+ let drive_lock = if schedule.is_some() {
+ None
+ } else {
+ Some(lock_tape_device(&drive_config, &setup.drive)?)
+ };
+
+ let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
+ let email = lookup_user_email(notify_user);
let upid_str = WorkerTask::new_thread(
&worker_type,
@@ -186,26 +194,37 @@ pub fn do_tape_backup_job(
auth_id.clone(),
false,
move |worker| {
- let _drive_lock = drive_lock; // keep lock guard
-
- set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
job.start(&worker.upid().to_string())?;
+ let mut drive_lock = drive_lock;
+
+ let (job_result, summary) = match try_block!({
+ if schedule.is_some() {
+ // for scheduled tape backup jobs, we wait indefinitely for the lock
+ task_log!(worker, "waiting for drive lock...");
+ loop {
+ if let Ok(lock) = lock_tape_device(&drive_config, &setup.drive) {
+ drive_lock = Some(lock);
+ break;
+ } // ignore errors
+
+ worker.check_abort()?;
+ }
+ }
+ set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
- task_log!(worker,"Starting tape backup job '{}'", job_id);
- if let Some(event_str) = schedule {
- task_log!(worker,"task triggered by schedule '{}'", event_str);
- }
-
- let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
- let email = lookup_user_email(notify_user);
+ task_log!(worker,"Starting tape backup job '{}'", job_id);
+ if let Some(event_str) = schedule {
+ task_log!(worker,"task triggered by schedule '{}'", event_str);
+ }
- let (job_result, summary) = match backup_worker(
- &worker,
- datastore,
- &pool_config,
- &setup,
- email.clone(),
- ) {
+ backup_worker(
+ &worker,
+ datastore,
+ &pool_config,
+ &setup,
+ email.clone(),
+ )
+ }) {
Ok(summary) => (Ok(()), summary),
Err(err) => (Err(err), Default::default()),
};
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails Dominik Csapak
@ 2021-03-19 6:07 ` Dietmar Maurer
0 siblings, 0 replies; 8+ messages in thread
From: Dietmar Maurer @ 2021-03-19 6:07 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Dominik Csapak
comments inline
> On 03/18/2021 1:01 PM Dominik Csapak <d.csapak@proxmox.com> wrote:
>
>
> for now only contains the list of included snapshots (if any),
> as well as the backup duration
>
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> src/api2/tape/backup.rs | 32 +++++++++++++++++++++++++------
> src/api2/types/tape/mod.rs | 9 +++++++++
> src/server/email_notifications.rs | 13 +++++++++++++
> 3 files changed, 48 insertions(+), 6 deletions(-)
>
> diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
> index 690b6855..0d42c31a 100644
> --- a/src/api2/tape/backup.rs
> +++ b/src/api2/tape/backup.rs
> @@ -51,6 +51,7 @@ use crate::{
> JOB_ID_SCHEMA,
> MediaPoolConfig,
> Userid,
> + TapeBackupJobSummary,
> },
> server::WorkerTask,
> task::TaskState,
> @@ -198,13 +199,16 @@ pub fn do_tape_backup_job(
> let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
> let email = lookup_user_email(notify_user);
>
> - let job_result = backup_worker(
> + let (job_result, summary) = match backup_worker(
> &worker,
> datastore,
> &pool_config,
> &setup,
> email.clone(),
> - );
> + ) {
> + Ok(summary) => (Ok(()), summary),
> + Err(err) => (Err(err), Default::default()),
> + };
>
> let status = worker.create_state(&job_result);
>
> @@ -214,6 +218,7 @@ pub fn do_tape_backup_job(
> Some(job.jobname()),
> &setup,
> &job_result,
> + summary,
> ) {
> eprintln!("send tape backup notification failed: {}", err);
> }
> @@ -340,13 +345,17 @@ pub fn backup(
> move |worker| {
> let _drive_lock = drive_lock; // keep lock guard
> set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
> - let job_result = backup_worker(
> +
> + let (job_result, summary) = match backup_worker(
> &worker,
> datastore,
> &pool_config,
> &setup,
> email.clone(),
> - );
> + ) {
> + Ok(summary) => (Ok(()), summary),
> + Err(err) => (Err(err), Default::default()),
> + };
>
> if let Some(email) = email {
> if let Err(err) = crate::server::send_tape_backup_status(
> @@ -354,6 +363,7 @@ pub fn backup(
> None,
> &setup,
> &job_result,
> + summary,
> ) {
> eprintln!("send tape backup notification failed: {}", err);
> }
> @@ -374,9 +384,11 @@ fn backup_worker(
> pool_config: &MediaPoolConfig,
> setup: &TapeBackupJobSetup,
> email: Option<String>,
> -) -> Result<(), Error> {
> +) -> Result<TapeBackupJobSummary, Error> {
>
> let status_path = Path::new(TAPE_STATUS_DIR);
> + let start = std::time::Instant::now();
> + let mut summary: TapeBackupJobSummary = Default::default();
>
> let _lock = MediaPool::lock(status_path, &pool_config.name)?;
>
> @@ -422,8 +434,11 @@ fn backup_worker(
> task_log!(worker, "skip snapshot {}", info.backup_dir);
> continue;
> }
> + let snapshot_name = info.backup_dir.to_string();
> if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
> errors = true;
> + } else {
> + summary.snapshot_list.push(snapshot_name);
> }
> progress.done_snapshots = 1;
> task_log!(
> @@ -439,8 +454,11 @@ fn backup_worker(
> task_log!(worker, "skip snapshot {}", info.backup_dir);
> continue;
> }
> + let snapshot_name = info.backup_dir.to_string();
> if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
> errors = true;
> + } else {
> + summary.snapshot_list.push(snapshot_name);
> }
> progress.done_snapshots = snapshot_number as u64 + 1;
> task_log!(
> @@ -464,7 +482,9 @@ fn backup_worker(
> bail!("Tape backup finished with some errors. Please check the task log.");
> }
>
> - Ok(())
> + summary.duration = start.elapsed();
> +
> + Ok(summary)
> }
>
> // Try to update the the media online status
> diff --git a/src/api2/types/tape/mod.rs b/src/api2/types/tape/mod.rs
> index 68b2cf12..44c2236b 100644
> --- a/src/api2/types/tape/mod.rs
> +++ b/src/api2/types/tape/mod.rs
> @@ -20,3 +20,12 @@ pub use media_location::*;
>
> mod media;
> pub use media::*;
> +
> +/// Summary of a successful Tape Job
> +#[derive(Default)]
> +pub struct TapeBackupJobSummary {
> + /// The list of snaphots backed up
> + pub snapshot_list: Vec<String>,
> + /// The total time of the backup job
> + pub duration: std::time::Duration,
> +}
This is not an API type, and is not use in the API at all!
Please can you define this type somewhere else?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
` (4 preceding siblings ...)
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 5/5] api2/tape/backup: wait indefinitely for lock in scheduled backup jobs Dominik Csapak
@ 2021-03-19 6:09 ` Dietmar Maurer
5 siblings, 0 replies; 8+ messages in thread
From: Dietmar Maurer @ 2021-03-19 6:09 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Dominik Csapak
applied patch 1, 2 an d 3
please rework 4, and rebase 5 (does not apply anymore)
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-03-19 6:09 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 2/5] server/email_notifications: do not panic on template registration Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 3/5] server/email_notifications: do not double html escape Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails Dominik Csapak
2021-03-19 6:07 ` Dietmar Maurer
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 5/5] api2/tape/backup: wait indefinitely for lock in scheduled backup jobs Dominik Csapak
2021-03-19 6:09 ` [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dietmar Maurer
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox