* [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications
@ 2021-03-18 12:01 Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan Dominik Csapak
` (5 more replies)
0 siblings, 6 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
combines my last three series[0][1][2], since they touch the same code
changes from v1:
* improve display trait code for timespan
* add tests for template registration
* rebase on master
0: https://lists.proxmox.com/pipermail/pbs-devel/2021-March/002438.html
1: https://lists.proxmox.com/pipermail/pbs-devel/2021-March/002441.html
2: https://lists.proxmox.com/pipermail/pbs-devel/2021-March/002431.html
Dominik Csapak (5):
tools/systemd/time: implement some Traits for TimeSpan
server/email_notifications: do not panic on template registration
server/email_notifications: do not double html escape
api2/tape/backup: include a summary on notification e-mails
api2/tape/backup: wait indefinitely for lock in scheduled backup jobs
src/api2/tape/backup.rs | 85 ++++++++++++++++++++++---------
src/api2/types/tape/mod.rs | 9 ++++
src/server/email_notifications.rs | 69 ++++++++++++++++++++-----
src/tools/systemd/time.rs | 82 +++++++++++++++++++++++++++++
4 files changed, 209 insertions(+), 36 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 2/5] server/email_notifications: do not panic on template registration Dominik Csapak
` (4 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
namely
* From<Duration> (to convert easily from duration to timespan)
* Display (for better formatting)
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* reorder msec, usec
* improve display trait code
src/tools/systemd/time.rs | 82 +++++++++++++++++++++++++++++++++++++++
1 file changed, 82 insertions(+)
diff --git a/src/tools/systemd/time.rs b/src/tools/systemd/time.rs
index 7cc42415..cbcb23df 100644
--- a/src/tools/systemd/time.rs
+++ b/src/tools/systemd/time.rs
@@ -141,6 +141,88 @@ impl From<TimeSpan> for f64 {
}
}
+impl From<std::time::Duration> for TimeSpan {
+ fn from(duration: std::time::Duration) -> Self {
+ let mut duration = duration.as_nanos();
+ let nsec = (duration % 1000) as u64;
+ duration /= 1000;
+ let usec = (duration % 1000) as u64;
+ duration /= 1000;
+ let msec = (duration % 1000) as u64;
+ duration /= 1000;
+ let seconds = (duration % 60) as u64;
+ duration /= 60;
+ let minutes = (duration % 60) as u64;
+ duration /= 60;
+ let hours = (duration % 24) as u64;
+ duration /= 24;
+ let years = (duration as f64 / 365.25) as u64;
+ let ydays = (duration as f64 % 365.25) as u64;
+ let months = (ydays as f64 / 30.44) as u64;
+ let mdays = (ydays as f64 % 30.44) as u64;
+ let weeks = mdays / 7;
+ let days = mdays % 7;
+ Self {
+ nsec,
+ usec,
+ msec,
+ seconds,
+ minutes,
+ hours,
+ days,
+ weeks,
+ months,
+ years,
+ }
+ }
+}
+
+impl std::fmt::Display for TimeSpan {
+ fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> Result<(), std::fmt::Error> {
+ let mut first = true;
+ { // block scope for mutable borrows
+ let mut do_write = |v: u64, unit: &str| -> Result<(), std::fmt::Error> {
+ if !first {
+ write!(f, " ")?;
+ }
+ first = false;
+ write!(f, "{}{}", v, unit)
+ };
+ if self.years > 0 {
+ do_write(self.years, "y")?;
+ }
+ if self.months > 0 {
+ do_write(self.months, "m")?;
+ }
+ if self.weeks > 0 {
+ do_write(self.weeks, "w")?;
+ }
+ if self.days > 0 {
+ do_write(self.days, "d")?;
+ }
+ if self.hours > 0 {
+ do_write(self.hours, "h")?;
+ }
+ if self.minutes > 0 {
+ do_write(self.minutes, "min")?;
+ }
+ }
+ if !first {
+ write!(f, " ")?;
+ }
+ let seconds = self.seconds as f64 + (self.msec as f64 / 1000.0);
+ if seconds >= 0.1 {
+ if seconds >= 1.0 || !first {
+ write!(f, "{:.0}s", seconds)?;
+ } else {
+ write!(f, "{:.1}s", seconds)?;
+ }
+ } else if first {
+ write!(f, "<0.1s")?;
+ }
+ Ok(())
+ }
+}
pub fn verify_time_span(i: &str) -> Result<(), Error> {
parse_time_span(i)?;
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 2/5] server/email_notifications: do not panic on template registration
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 3/5] server/email_notifications: do not double html escape Dominik Csapak
` (3 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
instead print an error and continue, the rendering functions will error
out if one of the templates could not be registered
if we `.unwrap()` here, it can lead to problems if the templates are
not correct, i.e. we could panic while holding a lock, if something holds
a mutex while this is called for the first time
add a test to catch registration issues during package build
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
changes from v1:
* add test to actually test registering, this will fail
on package build if the template registering fails
src/server/email_notifications.rs | 55 +++++++++++++++++++++++--------
1 file changed, 42 insertions(+), 13 deletions(-)
diff --git a/src/server/email_notifications.rs b/src/server/email_notifications.rs
index 229443f6..cd5a43fe 100644
--- a/src/server/email_notifications.rs
+++ b/src/server/email_notifications.rs
@@ -1,10 +1,11 @@
use anyhow::Error;
use serde_json::json;
-use handlebars::{Handlebars, Helper, Context, RenderError, RenderContext, Output, HelperResult};
+use handlebars::{Handlebars, Helper, Context, RenderError, RenderContext, Output, HelperResult, TemplateError};
use proxmox::tools::email::sendmail;
use proxmox::api::schema::parse_property_string;
+use proxmox::try_block;
use crate::{
config::datastore::DataStoreConfig,
@@ -181,25 +182,33 @@ lazy_static::lazy_static!{
static ref HANDLEBARS: Handlebars<'static> = {
let mut hb = Handlebars::new();
+ let result: Result<(), TemplateError> = try_block!({
- hb.set_strict_mode(true);
+ hb.set_strict_mode(true);
- hb.register_helper("human-bytes", Box::new(handlebars_humam_bytes_helper));
- hb.register_helper("relative-percentage", Box::new(handlebars_relative_percentage_helper));
+ hb.register_helper("human-bytes", Box::new(handlebars_humam_bytes_helper));
+ hb.register_helper("relative-percentage", Box::new(handlebars_relative_percentage_helper));
- hb.register_template_string("gc_ok_template", GC_OK_TEMPLATE).unwrap();
- hb.register_template_string("gc_err_template", GC_ERR_TEMPLATE).unwrap();
+ hb.register_template_string("gc_ok_template", GC_OK_TEMPLATE)?;
+ hb.register_template_string("gc_err_template", GC_ERR_TEMPLATE)?;
- hb.register_template_string("verify_ok_template", VERIFY_OK_TEMPLATE).unwrap();
- hb.register_template_string("verify_err_template", VERIFY_ERR_TEMPLATE).unwrap();
+ hb.register_template_string("verify_ok_template", VERIFY_OK_TEMPLATE)?;
+ hb.register_template_string("verify_err_template", VERIFY_ERR_TEMPLATE)?;
- hb.register_template_string("sync_ok_template", SYNC_OK_TEMPLATE).unwrap();
- hb.register_template_string("sync_err_template", SYNC_ERR_TEMPLATE).unwrap();
+ hb.register_template_string("sync_ok_template", SYNC_OK_TEMPLATE)?;
+ hb.register_template_string("sync_err_template", SYNC_ERR_TEMPLATE)?;
- hb.register_template_string("tape_backup_ok_template", TAPE_BACKUP_OK_TEMPLATE).unwrap();
- hb.register_template_string("tape_backup_err_template", TAPE_BACKUP_ERR_TEMPLATE).unwrap();
+ hb.register_template_string("tape_backup_ok_template", TAPE_BACKUP_OK_TEMPLATE)?;
+ hb.register_template_string("tape_backup_err_template", TAPE_BACKUP_ERR_TEMPLATE)?;
- hb.register_template_string("package_update_template", PACKAGE_UPDATES_TEMPLATE).unwrap();
+ hb.register_template_string("package_update_template", PACKAGE_UPDATES_TEMPLATE)?;
+
+ Ok(())
+ });
+
+ if let Err(err) = result {
+ eprintln!("error during template registration: {}", err);
+ }
hb
};
@@ -600,3 +609,23 @@ fn handlebars_relative_percentage_helper(
}
Ok(())
}
+
+#[test]
+fn test_template_register() {
+ HANDLEBARS.get_helper("human-bytes").unwrap();
+ HANDLEBARS.get_helper("relative-percentage").unwrap();
+
+ assert!(HANDLEBARS.has_template("gc_ok_template"));
+ assert!(HANDLEBARS.has_template("gc_err_template"));
+
+ assert!(HANDLEBARS.has_template("verify_ok_template"));
+ assert!(HANDLEBARS.has_template("verify_err_template"));
+
+ assert!(HANDLEBARS.has_template("sync_ok_template"));
+ assert!(HANDLEBARS.has_template("sync_err_template"));
+
+ assert!(HANDLEBARS.has_template("tape_backup_ok_template"));
+ assert!(HANDLEBARS.has_template("tape_backup_err_template"));
+
+ assert!(HANDLEBARS.has_template("package_update_template"));
+}
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 3/5] server/email_notifications: do not double html escape
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 2/5] server/email_notifications: do not panic on template registration Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails Dominik Csapak
` (2 subsequent siblings)
5 siblings, 0 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
the default escape handler is handlebars::html_escape, but this are
plain text emails and we manually escape them for the html part, so
set the default escape handler to 'no_escape'
this avoids double html escape for the characters: '&"<>' in emails
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/server/email_notifications.rs | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/server/email_notifications.rs b/src/server/email_notifications.rs
index cd5a43fe..e92c8091 100644
--- a/src/server/email_notifications.rs
+++ b/src/server/email_notifications.rs
@@ -185,6 +185,7 @@ lazy_static::lazy_static!{
let result: Result<(), TemplateError> = try_block!({
hb.set_strict_mode(true);
+ hb.register_escape_fn(handlebars::no_escape);
hb.register_helper("human-bytes", Box::new(handlebars_humam_bytes_helper));
hb.register_helper("relative-percentage", Box::new(handlebars_relative_percentage_helper));
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
` (2 preceding siblings ...)
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 3/5] server/email_notifications: do not double html escape Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-19 6:07 ` Dietmar Maurer
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 5/5] api2/tape/backup: wait indefinitely for lock in scheduled backup jobs Dominik Csapak
2021-03-19 6:09 ` [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dietmar Maurer
5 siblings, 1 reply; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
for now only contains the list of included snapshots (if any),
as well as the backup duration
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/api2/tape/backup.rs | 32 +++++++++++++++++++++++++------
src/api2/types/tape/mod.rs | 9 +++++++++
src/server/email_notifications.rs | 13 +++++++++++++
3 files changed, 48 insertions(+), 6 deletions(-)
diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
index 690b6855..0d42c31a 100644
--- a/src/api2/tape/backup.rs
+++ b/src/api2/tape/backup.rs
@@ -51,6 +51,7 @@ use crate::{
JOB_ID_SCHEMA,
MediaPoolConfig,
Userid,
+ TapeBackupJobSummary,
},
server::WorkerTask,
task::TaskState,
@@ -198,13 +199,16 @@ pub fn do_tape_backup_job(
let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
let email = lookup_user_email(notify_user);
- let job_result = backup_worker(
+ let (job_result, summary) = match backup_worker(
&worker,
datastore,
&pool_config,
&setup,
email.clone(),
- );
+ ) {
+ Ok(summary) => (Ok(()), summary),
+ Err(err) => (Err(err), Default::default()),
+ };
let status = worker.create_state(&job_result);
@@ -214,6 +218,7 @@ pub fn do_tape_backup_job(
Some(job.jobname()),
&setup,
&job_result,
+ summary,
) {
eprintln!("send tape backup notification failed: {}", err);
}
@@ -340,13 +345,17 @@ pub fn backup(
move |worker| {
let _drive_lock = drive_lock; // keep lock guard
set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
- let job_result = backup_worker(
+
+ let (job_result, summary) = match backup_worker(
&worker,
datastore,
&pool_config,
&setup,
email.clone(),
- );
+ ) {
+ Ok(summary) => (Ok(()), summary),
+ Err(err) => (Err(err), Default::default()),
+ };
if let Some(email) = email {
if let Err(err) = crate::server::send_tape_backup_status(
@@ -354,6 +363,7 @@ pub fn backup(
None,
&setup,
&job_result,
+ summary,
) {
eprintln!("send tape backup notification failed: {}", err);
}
@@ -374,9 +384,11 @@ fn backup_worker(
pool_config: &MediaPoolConfig,
setup: &TapeBackupJobSetup,
email: Option<String>,
-) -> Result<(), Error> {
+) -> Result<TapeBackupJobSummary, Error> {
let status_path = Path::new(TAPE_STATUS_DIR);
+ let start = std::time::Instant::now();
+ let mut summary: TapeBackupJobSummary = Default::default();
let _lock = MediaPool::lock(status_path, &pool_config.name)?;
@@ -422,8 +434,11 @@ fn backup_worker(
task_log!(worker, "skip snapshot {}", info.backup_dir);
continue;
}
+ let snapshot_name = info.backup_dir.to_string();
if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
errors = true;
+ } else {
+ summary.snapshot_list.push(snapshot_name);
}
progress.done_snapshots = 1;
task_log!(
@@ -439,8 +454,11 @@ fn backup_worker(
task_log!(worker, "skip snapshot {}", info.backup_dir);
continue;
}
+ let snapshot_name = info.backup_dir.to_string();
if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
errors = true;
+ } else {
+ summary.snapshot_list.push(snapshot_name);
}
progress.done_snapshots = snapshot_number as u64 + 1;
task_log!(
@@ -464,7 +482,9 @@ fn backup_worker(
bail!("Tape backup finished with some errors. Please check the task log.");
}
- Ok(())
+ summary.duration = start.elapsed();
+
+ Ok(summary)
}
// Try to update the the media online status
diff --git a/src/api2/types/tape/mod.rs b/src/api2/types/tape/mod.rs
index 68b2cf12..44c2236b 100644
--- a/src/api2/types/tape/mod.rs
+++ b/src/api2/types/tape/mod.rs
@@ -20,3 +20,12 @@ pub use media_location::*;
mod media;
pub use media::*;
+
+/// Summary of a successful Tape Job
+#[derive(Default)]
+pub struct TapeBackupJobSummary {
+ /// The list of snaphots backed up
+ pub snapshot_list: Vec<String>,
+ /// The total time of the backup job
+ pub duration: std::time::Duration,
+}
diff --git a/src/server/email_notifications.rs b/src/server/email_notifications.rs
index e92c8091..26002d39 100644
--- a/src/server/email_notifications.rs
+++ b/src/server/email_notifications.rs
@@ -18,6 +18,7 @@ use crate::{
Userid,
Notify,
DatastoreNotify,
+ TapeBackupJobSummary,
},
tools::format::HumanByte,
};
@@ -149,6 +150,14 @@ Datastore: {{job.store}}
Tape Pool: {{job.pool}}
Tape Drive: {{job.drive}}
+{{#if snapshot-list ~}}
+Snapshots included:
+
+{{#each snapshot-list~}}
+{{this}}
+{{/each~}}
+{{/if}}
+Duration: {{duration}}
Tape Backup successful.
@@ -412,14 +421,18 @@ pub fn send_tape_backup_status(
id: Option<&str>,
job: &TapeBackupJobSetup,
result: &Result<(), Error>,
+ summary: TapeBackupJobSummary,
) -> Result<(), Error> {
let (fqdn, port) = get_server_url();
+ let duration: crate::tools::systemd::time::TimeSpan = summary.duration.into();
let mut data = json!({
"job": job,
"fqdn": fqdn,
"port": port,
"id": id,
+ "snapshot-list": summary.snapshot_list,
+ "duration": duration.to_string(),
});
let text = match result {
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* [pbs-devel] [PATCH proxmox-backup v2 5/5] api2/tape/backup: wait indefinitely for lock in scheduled backup jobs
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
` (3 preceding siblings ...)
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails Dominik Csapak
@ 2021-03-18 12:01 ` Dominik Csapak
2021-03-19 6:09 ` [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dietmar Maurer
5 siblings, 0 replies; 8+ messages in thread
From: Dominik Csapak @ 2021-03-18 12:01 UTC (permalink / raw)
To: pbs-devel
so that a user can schedule multiple backup jobs onto a single
media pool without having to consider timing them apart
this makes sense since we can backup multiple datastores onto
the same media-set but can only specify one datastore per backup job
Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
---
src/api2/tape/backup.rs | 57 +++++++++++++++++++++++++++--------------
1 file changed, 38 insertions(+), 19 deletions(-)
diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
index 0d42c31a..f29eb715 100644
--- a/src/api2/tape/backup.rs
+++ b/src/api2/tape/backup.rs
@@ -5,6 +5,7 @@ use anyhow::{bail, format_err, Error};
use serde_json::Value;
use proxmox::{
+ try_block,
api::{
api,
RpcEnvironment,
@@ -177,8 +178,15 @@ pub fn do_tape_backup_job(
let (drive_config, _digest) = config::drive::config()?;
- // early check/lock before starting worker
- let drive_lock = lock_tape_device(&drive_config, &setup.drive)?;
+ // for scheduled jobs we acquire the lock later in the worker
+ let drive_lock = if schedule.is_some() {
+ None
+ } else {
+ Some(lock_tape_device(&drive_config, &setup.drive)?)
+ };
+
+ let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
+ let email = lookup_user_email(notify_user);
let upid_str = WorkerTask::new_thread(
&worker_type,
@@ -186,26 +194,37 @@ pub fn do_tape_backup_job(
auth_id.clone(),
false,
move |worker| {
- let _drive_lock = drive_lock; // keep lock guard
-
- set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
job.start(&worker.upid().to_string())?;
+ let mut drive_lock = drive_lock;
+
+ let (job_result, summary) = match try_block!({
+ if schedule.is_some() {
+ // for scheduled tape backup jobs, we wait indefinitely for the lock
+ task_log!(worker, "waiting for drive lock...");
+ loop {
+ if let Ok(lock) = lock_tape_device(&drive_config, &setup.drive) {
+ drive_lock = Some(lock);
+ break;
+ } // ignore errors
+
+ worker.check_abort()?;
+ }
+ }
+ set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
- task_log!(worker,"Starting tape backup job '{}'", job_id);
- if let Some(event_str) = schedule {
- task_log!(worker,"task triggered by schedule '{}'", event_str);
- }
-
- let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
- let email = lookup_user_email(notify_user);
+ task_log!(worker,"Starting tape backup job '{}'", job_id);
+ if let Some(event_str) = schedule {
+ task_log!(worker,"task triggered by schedule '{}'", event_str);
+ }
- let (job_result, summary) = match backup_worker(
- &worker,
- datastore,
- &pool_config,
- &setup,
- email.clone(),
- ) {
+ backup_worker(
+ &worker,
+ datastore,
+ &pool_config,
+ &setup,
+ email.clone(),
+ )
+ }) {
Ok(summary) => (Ok(()), summary),
Err(err) => (Err(err), Default::default()),
};
--
2.20.1
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails Dominik Csapak
@ 2021-03-19 6:07 ` Dietmar Maurer
0 siblings, 0 replies; 8+ messages in thread
From: Dietmar Maurer @ 2021-03-19 6:07 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Dominik Csapak
comments inline
> On 03/18/2021 1:01 PM Dominik Csapak <d.csapak@proxmox.com> wrote:
>
>
> for now only contains the list of included snapshots (if any),
> as well as the backup duration
>
> Signed-off-by: Dominik Csapak <d.csapak@proxmox.com>
> ---
> src/api2/tape/backup.rs | 32 +++++++++++++++++++++++++------
> src/api2/types/tape/mod.rs | 9 +++++++++
> src/server/email_notifications.rs | 13 +++++++++++++
> 3 files changed, 48 insertions(+), 6 deletions(-)
>
> diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
> index 690b6855..0d42c31a 100644
> --- a/src/api2/tape/backup.rs
> +++ b/src/api2/tape/backup.rs
> @@ -51,6 +51,7 @@ use crate::{
> JOB_ID_SCHEMA,
> MediaPoolConfig,
> Userid,
> + TapeBackupJobSummary,
> },
> server::WorkerTask,
> task::TaskState,
> @@ -198,13 +199,16 @@ pub fn do_tape_backup_job(
> let notify_user = setup.notify_user.as_ref().unwrap_or_else(|| &Userid::root_userid());
> let email = lookup_user_email(notify_user);
>
> - let job_result = backup_worker(
> + let (job_result, summary) = match backup_worker(
> &worker,
> datastore,
> &pool_config,
> &setup,
> email.clone(),
> - );
> + ) {
> + Ok(summary) => (Ok(()), summary),
> + Err(err) => (Err(err), Default::default()),
> + };
>
> let status = worker.create_state(&job_result);
>
> @@ -214,6 +218,7 @@ pub fn do_tape_backup_job(
> Some(job.jobname()),
> &setup,
> &job_result,
> + summary,
> ) {
> eprintln!("send tape backup notification failed: {}", err);
> }
> @@ -340,13 +345,17 @@ pub fn backup(
> move |worker| {
> let _drive_lock = drive_lock; // keep lock guard
> set_tape_device_state(&setup.drive, &worker.upid().to_string())?;
> - let job_result = backup_worker(
> +
> + let (job_result, summary) = match backup_worker(
> &worker,
> datastore,
> &pool_config,
> &setup,
> email.clone(),
> - );
> + ) {
> + Ok(summary) => (Ok(()), summary),
> + Err(err) => (Err(err), Default::default()),
> + };
>
> if let Some(email) = email {
> if let Err(err) = crate::server::send_tape_backup_status(
> @@ -354,6 +363,7 @@ pub fn backup(
> None,
> &setup,
> &job_result,
> + summary,
> ) {
> eprintln!("send tape backup notification failed: {}", err);
> }
> @@ -374,9 +384,11 @@ fn backup_worker(
> pool_config: &MediaPoolConfig,
> setup: &TapeBackupJobSetup,
> email: Option<String>,
> -) -> Result<(), Error> {
> +) -> Result<TapeBackupJobSummary, Error> {
>
> let status_path = Path::new(TAPE_STATUS_DIR);
> + let start = std::time::Instant::now();
> + let mut summary: TapeBackupJobSummary = Default::default();
>
> let _lock = MediaPool::lock(status_path, &pool_config.name)?;
>
> @@ -422,8 +434,11 @@ fn backup_worker(
> task_log!(worker, "skip snapshot {}", info.backup_dir);
> continue;
> }
> + let snapshot_name = info.backup_dir.to_string();
> if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
> errors = true;
> + } else {
> + summary.snapshot_list.push(snapshot_name);
> }
> progress.done_snapshots = 1;
> task_log!(
> @@ -439,8 +454,11 @@ fn backup_worker(
> task_log!(worker, "skip snapshot {}", info.backup_dir);
> continue;
> }
> + let snapshot_name = info.backup_dir.to_string();
> if !backup_snapshot(worker, &mut pool_writer, datastore.clone(), info.backup_dir)? {
> errors = true;
> + } else {
> + summary.snapshot_list.push(snapshot_name);
> }
> progress.done_snapshots = snapshot_number as u64 + 1;
> task_log!(
> @@ -464,7 +482,9 @@ fn backup_worker(
> bail!("Tape backup finished with some errors. Please check the task log.");
> }
>
> - Ok(())
> + summary.duration = start.elapsed();
> +
> + Ok(summary)
> }
>
> // Try to update the the media online status
> diff --git a/src/api2/types/tape/mod.rs b/src/api2/types/tape/mod.rs
> index 68b2cf12..44c2236b 100644
> --- a/src/api2/types/tape/mod.rs
> +++ b/src/api2/types/tape/mod.rs
> @@ -20,3 +20,12 @@ pub use media_location::*;
>
> mod media;
> pub use media::*;
> +
> +/// Summary of a successful Tape Job
> +#[derive(Default)]
> +pub struct TapeBackupJobSummary {
> + /// The list of snaphots backed up
> + pub snapshot_list: Vec<String>,
> + /// The total time of the backup job
> + pub duration: std::time::Duration,
> +}
This is not an API type, and is not use in the API at all!
Please can you define this type somewhere else?
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
` (4 preceding siblings ...)
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 5/5] api2/tape/backup: wait indefinitely for lock in scheduled backup jobs Dominik Csapak
@ 2021-03-19 6:09 ` Dietmar Maurer
5 siblings, 0 replies; 8+ messages in thread
From: Dietmar Maurer @ 2021-03-19 6:09 UTC (permalink / raw)
To: Proxmox Backup Server development discussion, Dominik Csapak
applied patch 1, 2 an d 3
please rework 4, and rebase 5 (does not apply anymore)
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-03-19 6:09 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-03-18 12:01 [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 1/5] tools/systemd/time: implement some Traits for TimeSpan Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 2/5] server/email_notifications: do not panic on template registration Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 3/5] server/email_notifications: do not double html escape Dominik Csapak
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 4/5] api2/tape/backup: include a summary on notification e-mails Dominik Csapak
2021-03-19 6:07 ` Dietmar Maurer
2021-03-18 12:01 ` [pbs-devel] [PATCH proxmox-backup v2 5/5] api2/tape/backup: wait indefinitely for lock in scheduled backup jobs Dominik Csapak
2021-03-19 6:09 ` [pbs-devel] [PATCH proxmox-backup v2 0/5] improve tape backup and notifications Dietmar Maurer
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal