[pbs-devel] [PATCH backup-qemu/vma-to-pbs 0/2] add support for notes and logs

[pbs-devel] [PATCH backup-qemu/vma-to-pbs 0/2] add support for notes and logs

[Filip Schauer]

Allow the user of vma-to-pbs to specify a notes file and a log file to
associate with the backup.

proxmox-backup-qemu:

Filip Schauer (1):
  add support for notes

 src/backup.rs | 6 +++++-
 src/lib.rs    | 9 +++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)


vma-to-pbs:

Filip Schauer (1):
  add support for notes and logs

 src/main.rs    | 16 +++++++++++++
 src/vma2pbs.rs | 64 ++++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 70 insertions(+), 10 deletions(-)


Summary over all repositories:
  4 files changed, 84 insertions(+), 11 deletions(-)

-- 
Generated by git-murpp 0.6.0


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH backup-qemu 1/2] add support for notes

[Filip Schauer]

Allow adding a comment/notes to a new backup.

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 src/backup.rs | 6 +++++-
 src/lib.rs    | 9 +++++++++
 2 files changed, 14 insertions(+), 1 deletion(-)

diff --git a/src/backup.rs b/src/backup.rs
index bbe4f00..b7d383a 100644
--- a/src/backup.rs
+++ b/src/backup.rs
@@ -73,7 +73,11 @@ impl BackupTask {
 
         let (abort, _) = tokio::sync::broadcast::channel(16);
 
-        let manifest = Arc::new(Mutex::new(BackupManifest::new(setup.backup_dir.clone())));
+        let mut manifest = BackupManifest::new(setup.backup_dir.clone());
+        if let Some(notes) = &setup.notes {
+            manifest.unprotected["notes"] = serde_json::Value::String(notes.clone());
+        }
+        let manifest = Arc::new(Mutex::new(manifest));
 
         let registry = Arc::new(Mutex::new(Registry::<ImageUploadInfo>::new()));
         let known_chunks = Arc::new(Mutex::new(HashSet::new()));
diff --git a/src/lib.rs b/src/lib.rs
index 9067fc5..034b7ab 100644
--- a/src/lib.rs
+++ b/src/lib.rs
@@ -140,6 +140,7 @@ pub(crate) struct BackupSetup {
     pub key_password: Option<String>,
     pub master_keyfile: Option<std::path::PathBuf>,
     pub fingerprint: Option<String>,
+    pub notes: Option<String>,
 }
 
 // helper class to implement synchrounous interface
@@ -212,6 +213,7 @@ pub extern "C" fn proxmox_backup_new(
     compress: bool,
     encrypt: bool,
     fingerprint: *const c_char,
+    notes: *const c_char,
     error: *mut *mut c_char,
 ) -> *mut ProxmoxBackupHandle {
     proxmox_backup_new_ns(
@@ -227,6 +229,7 @@ pub extern "C" fn proxmox_backup_new(
         compress,
         encrypt,
         fingerprint,
+        notes,
         error,
     )
 }
@@ -251,6 +254,7 @@ pub extern "C" fn proxmox_backup_new_ns(
     compress: bool,
     encrypt: bool,
     fingerprint: *const c_char,
+    notes: *const c_char,
     error: *mut *mut c_char,
 ) -> *mut ProxmoxBackupHandle {
     let task: Result<_, Error> = try_block!({
@@ -277,6 +281,8 @@ pub extern "C" fn proxmox_backup_new_ns(
             return Err(format_err!("can't use master keyfile without keyfile"));
         }
 
+        let notes = tools::utf8_c_string(notes)?;
+
         let crypt_mode = if keyfile.is_some() {
             if encrypt {
                 CryptMode::Encrypt
@@ -304,6 +310,7 @@ pub extern "C" fn proxmox_backup_new_ns(
             key_password,
             master_keyfile,
             fingerprint,
+            notes,
         };
 
         BackupTask::new(setup, compress, crypt_mode)
@@ -816,6 +823,7 @@ pub extern "C" fn proxmox_restore_new(
             key_password,
             master_keyfile: None,
             fingerprint,
+            notes: None,
         };
 
         RestoreTask::new(setup)
@@ -876,6 +884,7 @@ pub extern "C" fn proxmox_restore_new_ns(
             key_password,
             master_keyfile: None,
             fingerprint,
+            notes: None,
         };
 
         RestoreTask::new(setup)
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

[pbs-devel] [PATCH vma-to-pbs 2/2] add support for notes and logs

[Filip Schauer]

Allow the user to specify a notes file and a log file to associate with
the backup.

Signed-off-by: Filip Schauer <f.schauer@proxmox.com>
---
 src/main.rs    | 16 +++++++++++++
 src/vma2pbs.rs | 64 ++++++++++++++++++++++++++++++++++++++++++--------
 2 files changed, 70 insertions(+), 10 deletions(-)

diff --git a/src/main.rs b/src/main.rs
index 2653d3e..de789c1 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -37,6 +37,10 @@ Options:
           Password file
       --key-password-file <KEY_PASSWORD_FILE>
           Key password file
+      [--notes-file <NOTES_FILE>]
+          File containing a comment/notes
+      [--log-file <LOG_FILE>]
+          Log file
   -h, --help
           Print help
   -V, --version
@@ -93,6 +97,8 @@ fn parse_args() -> Result<BackupVmaToPbsArgs, Error> {
     let encrypt = args.contains(["-e", "--encrypt"]);
     let password_file: Option<OsString> = args.opt_value_from_str("--password-file")?;
     let key_password_file: Option<OsString> = args.opt_value_from_str("--key-password-file")?;
+    let notes_file: Option<OsString> = args.opt_value_from_str("--notes-file")?;
+    let log_file_path: Option<OsString> = args.opt_value_from_str("--log-file")?;
 
     match (encrypt, keyfile.is_some()) {
         (true, false) => bail!("--encrypt requires a --keyfile!"),
@@ -170,6 +176,14 @@ fn parse_args() -> Result<BackupVmaToPbsArgs, Error> {
         None
     };
 
+    let notes = if let Some(notes_file) = notes_file {
+        let notes = std::fs::read_to_string(notes_file).context("Could not read notes file")?;
+
+        Some(notes)
+    } else {
+        None
+    };
+
     let options = BackupVmaToPbsArgs {
         vma_file_path: vma_file_path.cloned(),
         pbs_repository,
@@ -183,6 +197,8 @@ fn parse_args() -> Result<BackupVmaToPbsArgs, Error> {
         fingerprint,
         compress,
         encrypt,
+        notes,
+        log_file_path,
     };
 
     Ok(options)
diff --git a/src/vma2pbs.rs b/src/vma2pbs.rs
index 199cf50..5c357ef 100644
--- a/src/vma2pbs.rs
+++ b/src/vma2pbs.rs
@@ -34,6 +34,8 @@ pub struct BackupVmaToPbsArgs {
     pub fingerprint: String,
     pub compress: bool,
     pub encrypt: bool,
+    pub notes: Option<String>,
+    pub log_file_path: Option<OsString>,
 }
 
 #[derive(Copy, Clone)]
@@ -52,7 +54,7 @@ fn handle_pbs_error(pbs_err: *mut c_char, function_name: &str) -> Result<(), Err
     bail!("{function_name} failed: {pbs_err_str}");
 }
 
-fn create_pbs_backup_task(args: BackupVmaToPbsArgs) -> Result<*mut ProxmoxBackupHandle, Error> {
+fn create_pbs_backup_task(args: &BackupVmaToPbsArgs) -> Result<*mut ProxmoxBackupHandle, Error> {
     println!("PBS repository: {}", args.pbs_repository);
     if let Some(ns) = &args.namespace {
         println!("PBS namespace: {}", ns);
@@ -65,26 +67,43 @@ fn create_pbs_backup_task(args: BackupVmaToPbsArgs) -> Result<*mut ProxmoxBackup
 
     let mut pbs_err: *mut c_char = ptr::null_mut();
 
-    let pbs_repository_cstr = CString::new(args.pbs_repository)?;
-    let ns_cstr = CString::new(args.namespace.unwrap_or("".to_string()))?;
-    let backup_id_cstr = CString::new(args.backup_id)?;
-    let pbs_password_cstr = CString::new(args.pbs_password)?;
-    let fingerprint_cstr = CString::new(args.fingerprint)?;
-    let keyfile_cstr = args.keyfile.map(|v| CString::new(v).unwrap());
+    let pbs_repository_cstr = CString::new(args.pbs_repository.as_str())?;
+    let ns_cstr = CString::new(args.namespace.as_deref().unwrap_or(""))?;
+    let backup_id_cstr = CString::new(args.backup_id.as_str())?;
+    let pbs_password_cstr = CString::new(args.pbs_password.as_str())?;
+    let fingerprint_cstr = CString::new(args.fingerprint.as_str())?;
+    let keyfile_cstr = args
+        .keyfile
+        .as_ref()
+        .map(|v| CString::new(v.as_str()).unwrap());
     let keyfile_ptr = keyfile_cstr
         .as_ref()
         .map(|v| v.as_ptr())
         .unwrap_or(ptr::null());
-    let key_password_cstr = args.key_password.map(|v| CString::new(v).unwrap());
+    let key_password_cstr = args
+        .key_password
+        .as_ref()
+        .map(|v| CString::new(v.as_str()).unwrap());
     let key_password_ptr = key_password_cstr
         .as_ref()
         .map(|v| v.as_ptr())
         .unwrap_or(ptr::null());
-    let master_keyfile_cstr = args.master_keyfile.map(|v| CString::new(v).unwrap());
+    let master_keyfile_cstr = args
+        .master_keyfile
+        .as_ref()
+        .map(|v| CString::new(v.as_str()).unwrap());
     let master_keyfile_ptr = master_keyfile_cstr
         .as_ref()
         .map(|v| v.as_ptr())
         .unwrap_or(ptr::null());
+    let notes_cstr = args
+        .notes
+        .as_ref()
+        .map(|v| CString::new(v.as_str()).unwrap());
+    let notes_ptr = notes_cstr
+        .as_ref()
+        .map(|v| v.as_ptr())
+        .unwrap_or(ptr::null());
 
     let pbs = proxmox_backup_new_ns(
         pbs_repository_cstr.as_ptr(),
@@ -99,6 +118,7 @@ fn create_pbs_backup_task(args: BackupVmaToPbsArgs) -> Result<*mut ProxmoxBackup
         args.compress,
         args.encrypt,
         fingerprint_cstr.as_ptr(),
+        notes_ptr,
         &mut pbs_err,
     );
 
@@ -343,6 +363,26 @@ where
     Ok(())
 }
 
+fn upload_log(pbs: *mut ProxmoxBackupHandle, log_file_path: OsString) -> Result<(), Error> {
+    let name_cstr = CString::new("client.log")?;
+    let data = std::fs::read(log_file_path)?;
+    let size = data.len();
+    let mut pbs_err: *mut c_char = ptr::null_mut();
+
+    if proxmox_backup_add_config(
+        pbs,
+        name_cstr.as_ptr(),
+        data.as_ptr(),
+        size as u64,
+        &mut pbs_err,
+    ) < 0
+    {
+        handle_pbs_error(pbs_err, "proxmox_backup_add_config")?;
+    }
+
+    Ok(())
+}
+
 pub fn backup_vma_to_pbs(args: BackupVmaToPbsArgs) -> Result<(), Error> {
     let vma_file: Box<dyn BufRead> = match &args.vma_file_path {
         Some(vma_file_path) => match File::open(vma_file_path) {
@@ -353,7 +393,7 @@ pub fn backup_vma_to_pbs(args: BackupVmaToPbsArgs) -> Result<(), Error> {
     };
     let vma_reader = VmaReader::new(vma_file)?;
 
-    let pbs = create_pbs_backup_task(args)?;
+    let pbs = create_pbs_backup_task(&args)?;
 
     defer! {
         proxmox_backup_disconnect(pbs);
@@ -373,6 +413,10 @@ pub fn backup_vma_to_pbs(args: BackupVmaToPbsArgs) -> Result<(), Error> {
     upload_configs(&vma_reader, pbs)?;
     upload_block_devices(vma_reader, pbs)?;
 
+    if let Some(log_file_path) = args.log_file_path {
+        upload_log(pbs, log_file_path)?;
+    }
+
     if proxmox_backup_finish(pbs, &mut pbs_err) < 0 {
         handle_pbs_error(pbs_err, "proxmox_backup_finish")?;
     }
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH backup-qemu/vma-to-pbs 0/2] add support for notes and logs

[Fabian Grünbichler]

On May 27, 2024 3:07 pm, Filip Schauer wrote:
> Allow the user of vma-to-pbs to specify a notes file and a log file to
> associate with the backup.
> 
> proxmox-backup-qemu:
> 
> Filip Schauer (1):
>   add support for notes

I am not sure this is really warranted, especially since we don't want
to keep using the proxmox-backup-qemu lib here in the long run anyway,
but switch to a (to-be-extracted) pure Rust client directly..

as a stop-gap measure, couldn't we just use `proxmox-backup-client` to
set the notes?

> 
>  src/backup.rs | 6 +++++-
>  src/lib.rs    | 9 +++++++++
>  2 files changed, 14 insertions(+), 1 deletion(-)
> 
> 
> vma-to-pbs:
> 
> Filip Schauer (1):
>   add support for notes and logs

and the log as well, since that is not really handled correctly right
now anyway (the log is not a config file, there is a special command/API
endpoint for uploading the log).

> 
>  src/main.rs    | 16 +++++++++++++
>  src/vma2pbs.rs | 64 ++++++++++++++++++++++++++++++++++++++++++--------
>  2 files changed, 70 insertions(+), 10 deletions(-)
> 
> 
> Summary over all repositories:
>   4 files changed, 84 insertions(+), 11 deletions(-)
> 
> -- 
> Generated by git-murpp 0.6.0
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
> 


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH backup-qemu/vma-to-pbs 0/2] add support for notes and logs

[Filip Schauer]

On 03/07/2024 15:06, Fabian Grünbichler wrote:
>> proxmox-backup-qemu:
>>
>> Filip Schauer (1):
>>    add support for notes
> I am not sure this is really warranted, especially since we don't want
> to keep using the proxmox-backup-qemu lib here in the long run anyway,
> but switch to a (to-be-extracted) pure Rust client directly..
>
> as a stop-gap measure, couldn't we just use `proxmox-backup-client` to
> set the notes?


Yes, we could. The only downside being that we would have to open two
connections to the Proxmox Backup Server.



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH backup-qemu/vma-to-pbs 0/2] add support for notes and logs

[Fabian Grünbichler]

On July 3, 2024 3:28 pm, Filip Schauer wrote:
> On 03/07/2024 15:06, Fabian Grünbichler wrote:
>>> proxmox-backup-qemu:
>>>
>>> Filip Schauer (1):
>>>    add support for notes
>> I am not sure this is really warranted, especially since we don't want
>> to keep using the proxmox-backup-qemu lib here in the long run anyway,
>> but switch to a (to-be-extracted) pure Rust client directly..
>>
>> as a stop-gap measure, couldn't we just use `proxmox-backup-client` to
>> set the notes?
> 
> 
> Yes, we could. The only downside being that we would have to open two
> connections to the Proxmox Backup Server.

yes, that's also how it works with vzdump :)


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

Re: [pbs-devel] [PATCH backup-qemu/vma-to-pbs 0/2] add support for notes and logs

[Filip Schauer]

Superseded by:
https://lists.proxmox.com/pipermail/pbs-devel/2024-July/010136.html

On 03/07/2024 16:24, Fabian Grünbichler wrote:
> On July 3, 2024 3:28 pm, Filip Schauer wrote:
>> On 03/07/2024 15:06, Fabian Grünbichler wrote:
>>>> proxmox-backup-qemu:
>>>>
>>>> Filip Schauer (1):
>>>>     add support for notes
>>> I am not sure this is really warranted, especially since we don't want
>>> to keep using the proxmox-backup-qemu lib here in the long run anyway,
>>> but switch to a (to-be-extracted) pure Rust client directly..
>>>
>>> as a stop-gap measure, couldn't we just use `proxmox-backup-client` to
>>> set the notes?
>>
>> Yes, we could. The only downside being that we would have to open two
>> connections to the Proxmox Backup Server.
> yes, that's also how it works with vzdump :)
>
>
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel