public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox-backup] add inspection in proxmox-backup-manager for .blob, .didx and .fidx files
@ 2020-12-09  8:42 Hannes Laimer
  2020-12-09  9:03 ` Dietmar Maurer
  0 siblings, 1 reply; 3+ messages in thread
From: Hannes Laimer @ 2020-12-09  8:42 UTC (permalink / raw)
  To: pbs-devel

Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
---
Add print_info function for DynamicIndex and inspection of .blob, .didx and .fidx
files to proxmox-backup-manager

 src/api2/types/mod.rs                     |  2 +
 src/backup/dynamic_index.rs               | 14 ++++
 src/bin/proxmox-backup-manager.rs         |  1 +
 src/bin/proxmox_backup_manager/inspect.rs | 80 +++++++++++++++++++++++
 src/bin/proxmox_backup_manager/mod.rs     |  2 +
 5 files changed, 99 insertions(+)
 create mode 100644 src/bin/proxmox_backup_manager/inspect.rs

diff --git a/src/api2/types/mod.rs b/src/api2/types/mod.rs
index 59b2433f..fe89dc78 100644
--- a/src/api2/types/mod.rs
+++ b/src/api2/types/mod.rs
@@ -253,6 +253,8 @@ pub const TIME_ZONE_SCHEMA: Schema = StringSchema::new(
     .max_length(64)
     .schema();
 
+pub const FILE_SCHEMA: Schema = StringSchema::new("File").schema();
+
 pub const ACL_PATH_SCHEMA: Schema = StringSchema::new(
     "Access control path.")
     .format(&ACL_PATH_FORMAT)
diff --git a/src/backup/dynamic_index.rs b/src/backup/dynamic_index.rs
index df651906..35dde880 100644
--- a/src/backup/dynamic_index.rs
+++ b/src/backup/dynamic_index.rs
@@ -183,6 +183,20 @@ impl DynamicIndexReader {
             self.binary_search(middle_idx + 1, middle_end, end_idx, end, offset)
         }
     }
+
+    pub fn print_info(&self) {
+        println!("Size: {}", self.size);
+
+        let mut ctime_str = self.ctime.to_string();
+        if let Ok(s) = proxmox::tools::time::strftime_local("%c", self.ctime) {
+            ctime_str = s;
+        }
+
+        println!("Index entries: {}", self.index.len());
+
+        println!("CTime: {}", ctime_str);
+        println!("UUID: {:?}", self.uuid);
+    }
 }
 
 impl IndexFile for DynamicIndexReader {
diff --git a/src/bin/proxmox-backup-manager.rs b/src/bin/proxmox-backup-manager.rs
index a763d6d6..3e023965 100644
--- a/src/bin/proxmox-backup-manager.rs
+++ b/src/bin/proxmox-backup-manager.rs
@@ -409,6 +409,7 @@ fn main() {
         .insert("datastore", datastore_commands())
         .insert("disk", disk_commands())
         .insert("dns", dns_commands())
+        .insert("inspect", inspect_commands())
         .insert("network", network_commands())
         .insert("user", user_commands())
         .insert("remote", remote_commands())
diff --git a/src/bin/proxmox_backup_manager/inspect.rs b/src/bin/proxmox_backup_manager/inspect.rs
new file mode 100644
index 00000000..8dd834c3
--- /dev/null
+++ b/src/bin/proxmox_backup_manager/inspect.rs
@@ -0,0 +1,80 @@
+use std::fs::File;
+use std::io::Read;
+use std::path::Path;
+
+use anyhow::Error;
+use proxmox::api::{api, cli::*, RpcEnvironment};
+use serde_json::Value;
+
+use proxmox_backup::tools;
+use proxmox_backup::api2::types::*;
+use proxmox_backup::backup::{DataBlob, DynamicIndexReader, FixedIndexReader};
+
+#[api(
+    input: {
+        properties: {
+            file: {
+                schema: FILE_SCHEMA,
+            }
+        }
+    }
+)]
+/// Inspect a .blob file
+fn inspect_blob(param: Value, _rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
+    let path = tools::required_string_param(&param, "file")?;
+
+    let mut file = File::open(path)?;
+    let mut buffer = Vec::new();
+    file.read_to_end(&mut buffer)?;
+    let data_blob = DataBlob::load_from_reader(&mut &buffer[..])?;
+
+    println!("{}", String::from_utf8(data_blob.decode(None, None)?)?);
+
+    Ok(Value::Null)
+}
+
+#[api(
+    input: {
+        properties: {
+            file: {
+                schema: FILE_SCHEMA,
+            }
+        }
+    }
+)]
+/// Inspect a .fidx file
+fn inspect_fixed_index(param: Value, _rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
+    let path = tools::required_string_param(&param, "file")?;
+
+    let index = FixedIndexReader::open(Path::new(path))?;
+    index.print_info();
+    Ok(Value::Null)
+}
+
+#[api(
+    input: {
+        properties: {
+            file: {
+                schema: FILE_SCHEMA,
+            }
+        }
+    }
+)]
+/// Inspect a .didx file
+fn inspect_dynamic_index(param: Value, _rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
+    let path = tools::required_string_param(&param, "file")?;
+
+    let index = DynamicIndexReader::open(Path::new(path))?;
+    index.print_info();
+    Ok(Value::Null)
+}
+
+pub fn inspect_commands() -> CommandLineInterface {
+
+    let cmd_def = CliCommandMap::new()
+        .insert("blob",CliCommand::new(&API_METHOD_INSPECT_BLOB))
+        .insert("fidx",CliCommand::new(&API_METHOD_INSPECT_FIXED_INDEX))
+        .insert("didx",CliCommand::new(&API_METHOD_INSPECT_DYNAMIC_INDEX));
+
+    cmd_def.into()
+}
diff --git a/src/bin/proxmox_backup_manager/mod.rs b/src/bin/proxmox_backup_manager/mod.rs
index 1f3ff92e..7b2854d7 100644
--- a/src/bin/proxmox_backup_manager/mod.rs
+++ b/src/bin/proxmox_backup_manager/mod.rs
@@ -6,6 +6,8 @@ mod datastore;
 pub use datastore::*;
 mod dns;
 pub use dns::*;
+mod inspect;
+pub use inspect::*;
 mod network;
 pub use network::*;
 mod remote;
-- 
2.20.1





^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] add inspection in proxmox-backup-manager for .blob, .didx and .fidx files
  2020-12-09  8:42 [pbs-devel] [PATCH proxmox-backup] add inspection in proxmox-backup-manager for .blob, .didx and .fidx files Hannes Laimer
@ 2020-12-09  9:03 ` Dietmar Maurer
  2020-12-09  9:32   ` Thomas Lamprecht
  0 siblings, 1 reply; 3+ messages in thread
From: Dietmar Maurer @ 2020-12-09  9:03 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Hannes Laimer


> On 12/09/2020 9:42 AM Hannes Laimer <h.laimer@proxmox.com> wrote:
> 
>  
> Signed-off-by: Hannes Laimer <h.laimer@proxmox.com>
> ---
> Add print_info function for DynamicIndex and inspection of .blob, .didx and .fidx
> files to proxmox-backup-manager
> 
>  src/api2/types/mod.rs                     |  2 +
>  src/backup/dynamic_index.rs               | 14 ++++
>  src/bin/proxmox-backup-manager.rs         |  1 +
>  src/bin/proxmox_backup_manager/inspect.rs | 80 +++++++++++++++++++++++
>  src/bin/proxmox_backup_manager/mod.rs     |  2 +
>  5 files changed, 99 insertions(+)
>  create mode 100644 src/bin/proxmox_backup_manager/inspect.rs
> 
> diff --git a/src/api2/types/mod.rs b/src/api2/types/mod.rs
> index 59b2433f..fe89dc78 100644
> --- a/src/api2/types/mod.rs
> +++ b/src/api2/types/mod.rs
> @@ -253,6 +253,8 @@ pub const TIME_ZONE_SCHEMA: Schema = StringSchema::new(
>      .max_length(64)
>      .schema();
>  
> +pub const FILE_SCHEMA: Schema = StringSchema::new("File").schema();
> +
>  pub const ACL_PATH_SCHEMA: Schema = StringSchema::new(
>      "Access control path.")
>      .format(&ACL_PATH_FORMAT)
> diff --git a/src/backup/dynamic_index.rs b/src/backup/dynamic_index.rs
> index df651906..35dde880 100644
> --- a/src/backup/dynamic_index.rs
> +++ b/src/backup/dynamic_index.rs
> @@ -183,6 +183,20 @@ impl DynamicIndexReader {
>              self.binary_search(middle_idx + 1, middle_end, end_idx, end, offset)
>          }
>      }
> +
> +    pub fn print_info(&self) {
> +        println!("Size: {}", self.size);
> +
> +        let mut ctime_str = self.ctime.to_string();
> +        if let Ok(s) = proxmox::tools::time::strftime_local("%c", self.ctime) {
> +            ctime_str = s;
> +        }
> +
> +        println!("Index entries: {}", self.index.len());
> +
> +        println!("CTime: {}", ctime_str);
> +        println!("UUID: {:?}", self.uuid);
> +    }

What about referenced chunks?

>  }
>  
>  impl IndexFile for DynamicIndexReader {
> diff --git a/src/bin/proxmox-backup-manager.rs b/src/bin/proxmox-backup-manager.rs
> index a763d6d6..3e023965 100644
> --- a/src/bin/proxmox-backup-manager.rs
> +++ b/src/bin/proxmox-backup-manager.rs
> @@ -409,6 +409,7 @@ fn main() {
>          .insert("datastore", datastore_commands())
>          .insert("disk", disk_commands())
>          .insert("dns", dns_commands())
> +        .insert("inspect", inspect_commands())
>          .insert("network", network_commands())
>          .insert("user", user_commands())
>          .insert("remote", remote_commands())
> diff --git a/src/bin/proxmox_backup_manager/inspect.rs b/src/bin/proxmox_backup_manager/inspect.rs
> new file mode 100644
> index 00000000..8dd834c3
> --- /dev/null
> +++ b/src/bin/proxmox_backup_manager/inspect.rs
> @@ -0,0 +1,80 @@
> +use std::fs::File;
> +use std::io::Read;
> +use std::path::Path;
> +
> +use anyhow::Error;
> +use proxmox::api::{api, cli::*, RpcEnvironment};
> +use serde_json::Value;
> +
> +use proxmox_backup::tools;
> +use proxmox_backup::api2::types::*;
> +use proxmox_backup::backup::{DataBlob, DynamicIndexReader, FixedIndexReader};
> +
> +#[api(
> +    input: {
> +        properties: {
> +            file: {
> +                schema: FILE_SCHEMA,
> +            }
> +        }
> +    }
> +)]
> +/// Inspect a .blob file
> +fn inspect_blob(param: Value, _rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
> +    let path = tools::required_string_param(&param, "file")?;
> +
> +    let mut file = File::open(path)?;
> +    let mut buffer = Vec::new();
> +    file.read_to_end(&mut buffer)?;
> +    let data_blob = DataBlob::load_from_reader(&mut &buffer[..])?;
> +
> +    println!("{}", String::from_utf8(data_blob.decode(None, None)?)?);

blobs are binary data, not utf8!

> +
> +    Ok(Value::Null)
> +}
> +
> +#[api(
> +    input: {
> +        properties: {
> +            file: {
> +                schema: FILE_SCHEMA,
> +            }
> +        }
> +    }
> +)]
> +/// Inspect a .fidx file
> +fn inspect_fixed_index(param: Value, _rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
> +    let path = tools::required_string_param(&param, "file")?;
> +
> +    let index = FixedIndexReader::open(Path::new(path))?;
> +    index.print_info();
> +    Ok(Value::Null)
> +}
> +
> +#[api(
> +    input: {
> +        properties: {
> +            file: {
> +                schema: FILE_SCHEMA,
> +            }
> +        }
> +    }
> +)]
> +/// Inspect a .didx file
> +fn inspect_dynamic_index(param: Value, _rpcenv: &mut dyn RpcEnvironment) -> Result<Value, Error> {
> +    let path = tools::required_string_param(&param, "file")?;
> +
> +    let index = DynamicIndexReader::open(Path::new(path))?;
> +    index.print_info();
> +    Ok(Value::Null)
> +}
> +
> +pub fn inspect_commands() -> CommandLineInterface {
> +
> +    let cmd_def = CliCommandMap::new()
> +        .insert("blob",CliCommand::new(&API_METHOD_INSPECT_BLOB))
> +        .insert("fidx",CliCommand::new(&API_METHOD_INSPECT_FIXED_INDEX))
> +        .insert("didx",CliCommand::new(&API_METHOD_INSPECT_DYNAMIC_INDEX));

Can't we autodetect the file type?

> +
> +    cmd_def.into()
> +}
> diff --git a/src/bin/proxmox_backup_manager/mod.rs b/src/bin/proxmox_backup_manager/mod.rs
> index 1f3ff92e..7b2854d7 100644
> --- a/src/bin/proxmox_backup_manager/mod.rs
> +++ b/src/bin/proxmox_backup_manager/mod.rs
> @@ -6,6 +6,8 @@ mod datastore;
>  pub use datastore::*;
>  mod dns;
>  pub use dns::*;
> +mod inspect;
> +pub use inspect::*;
>  mod network;
>  pub use network::*;
>  mod remote;
> -- 
> 2.20.1
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel




^ permalink raw reply	[flat|nested] 3+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup] add inspection in proxmox-backup-manager for .blob, .didx and .fidx files
  2020-12-09  9:03 ` Dietmar Maurer
@ 2020-12-09  9:32   ` Thomas Lamprecht
  0 siblings, 0 replies; 3+ messages in thread
From: Thomas Lamprecht @ 2020-12-09  9:32 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Dietmar Maurer,
	Hannes Laimer

On 09.12.20 10:03, Dietmar Maurer wrote:
>> On 12/09/2020 9:42 AM Hannes Laimer <h.laimer@proxmox.com> wrote:
...
>> +pub fn inspect_commands() -> CommandLineInterface {
>> +
>> +    let cmd_def = CliCommandMap::new()
>> +        .insert("blob",CliCommand::new(&API_METHOD_INSPECT_BLOB))
>> +        .insert("fidx",CliCommand::new(&API_METHOD_INSPECT_FIXED_INDEX))
>> +        .insert("didx",CliCommand::new(&API_METHOD_INSPECT_DYNAMIC_INDEX));
> 
> Can't we autodetect the file type?
> 

proxmox-backup-client.rs has a 'parse_archive_type' function we maybe can reuse here,
it's used there when passing an "archive-name" on restore.





^ permalink raw reply	[flat|nested] 3+ messages in thread

end of thread, other threads:[~2020-12-09  9:32 UTC | newest]

Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-12-09  8:42 [pbs-devel] [PATCH proxmox-backup] add inspection in proxmox-backup-manager for .blob, .didx and .fidx files Hannes Laimer
2020-12-09  9:03 ` Dietmar Maurer
2020-12-09  9:32   ` Thomas Lamprecht

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal