public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH v2 proxmox-backup] fix #2996: client: allow optional match patterns for restore
@ 2024-04-30 15:58 Christian Ebner
  2024-05-24 10:21 ` Fabian Grünbichler
  2024-06-13 10:58 ` Christian Ebner
  0 siblings, 2 replies; 4+ messages in thread
From: Christian Ebner @ 2024-04-30 15:58 UTC (permalink / raw)
  To: pbs-devel

When the user is only interested in a subset of the entries stored in
a file-level backup, it is convenient to be able to provide a list of
match patterns for the entries intended to be restored.

The required restore logic is already in place. Therefore, expose it
for the `proxmox-backup-client restore` command by adding the optional
array of patterns as command line argument and parse these before
passing them via the pxar restore options to the archive extractor.

Link to bugtracker issue:
https://bugzilla.proxmox.com/show_bug.cgi?id=2996

Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
Tested-by: Gabriel Goller <g.goller@proxmox.com>
---
Changes since version 1:
 - bail when `matches` are used in combination with restore to stdout
 - fix typo in commit title

 proxmox-backup-client/src/main.rs | 31 +++++++++++++++++++++++++++++--
 1 file changed, 29 insertions(+), 2 deletions(-)

diff --git a/proxmox-backup-client/src/main.rs b/proxmox-backup-client/src/main.rs
index 32fe914c4..08582c86d 100644
--- a/proxmox-backup-client/src/main.rs
+++ b/proxmox-backup-client/src/main.rs
@@ -1186,6 +1186,15 @@ We do not extract '.pxar' archives when writing to standard output.
 
 "###
             },
+            "matches": {
+                type: Array,
+                description: "Only restore entries matching given paths or patterns.",
+                optional: true,
+                items: {
+                    type: String,
+                    description: "Path or match pattern.",
+                },
+            },
             rate: {
                 schema: TRAFFIC_CONTROL_RATE_SCHEMA,
                 optional: true,
@@ -1306,6 +1315,24 @@ async fn restore(
     let target = json::required_string_param(&param, "target")?;
     let target = if target == "-" { None } else { Some(target) };
 
+    let mut match_list = Vec::new();
+    if let Some(patterns) = param["matches"].as_array() {
+        if target.is_none() {
+            bail!("matches not allowed when restoring to stdout");
+        }
+
+        for pattern in patterns {
+            if let Some(pattern) = pattern.as_str() {
+                let match_entry = MatchEntry::parse_pattern(
+                    pattern,
+                    PatternFlag::PATH_NAME,
+                    MatchType::Include
+                )?;
+                match_list.push(match_entry);
+            }
+        }
+    };
+
     let crypto = crypto_parameters(&param)?;
 
     let crypt_config = match crypto.enc_key {
@@ -1429,8 +1456,8 @@ async fn restore(
         }
 
         let options = pbs_client::pxar::PxarExtractOptions {
-            match_list: &[],
-            extract_match_default: true,
+            match_list: match_list.as_slice(),
+            extract_match_default: match_list.is_empty(),
             allow_existing_dirs,
             overwrite_flags,
             on_error,
-- 
2.39.2



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [pbs-devel] [PATCH v2 proxmox-backup] fix #2996: client: allow optional match patterns for restore
  2024-04-30 15:58 [pbs-devel] [PATCH v2 proxmox-backup] fix #2996: client: allow optional match patterns for restore Christian Ebner
@ 2024-05-24 10:21 ` Fabian Grünbichler
  2024-05-24 10:45   ` Christian Ebner
  2024-06-13 10:58 ` Christian Ebner
  1 sibling, 1 reply; 4+ messages in thread
From: Fabian Grünbichler @ 2024-05-24 10:21 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion

On April 30, 2024 5:58 pm, Christian Ebner wrote:
> When the user is only interested in a subset of the entries stored in
> a file-level backup, it is convenient to be able to provide a list of
> match patterns for the entries intended to be restored.
> 
> The required restore logic is already in place. Therefore, expose it
> for the `proxmox-backup-client restore` command by adding the optional
> array of patterns as command line argument and parse these before
> passing them via the pxar restore options to the archive extractor.
> 
> Link to bugtracker issue:
> https://bugzilla.proxmox.com/show_bug.cgi?id=2996
> 
> Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
> Tested-by: Gabriel Goller <g.goller@proxmox.com>
> ---
> Changes since version 1:
>  - bail when `matches` are used in combination with restore to stdout
>  - fix typo in commit title
> 
>  proxmox-backup-client/src/main.rs | 31 +++++++++++++++++++++++++++++--
>  1 file changed, 29 insertions(+), 2 deletions(-)
> 
> diff --git a/proxmox-backup-client/src/main.rs b/proxmox-backup-client/src/main.rs
> index 32fe914c4..08582c86d 100644
> --- a/proxmox-backup-client/src/main.rs
> +++ b/proxmox-backup-client/src/main.rs
> @@ -1186,6 +1186,15 @@ We do not extract '.pxar' archives when writing to standard output.
>  
>  "###
>              },
> +            "matches": {
> +                type: Array,
> +                description: "Only restore entries matching given paths or patterns.",
> +                optional: true,
> +                items: {
> +                    type: String,
> +                    description: "Path or match pattern.",
> +                },
> +            },

in `pxar extract`, we call this "pattern" with the following:

            pattern: {
                description: "List of paths or pattern matching files to restore",
                type: Array,
                items: {
                    type: String,
                    description: "Path or pattern matching files to restore.",
                },
                optional: true,
            },

do we maybe want to copy that wording/name? do we have other places
where patterns are used for inclusion that we could unify?

would it make sense to create an API type for this, or at least add some
sort of schema-based validation up front (e.g., empty strings are not
valid, but also trailing back slashes, and I don't know what else ;))?

>              rate: {
>                  schema: TRAFFIC_CONTROL_RATE_SCHEMA,
>                  optional: true,
> @@ -1306,6 +1315,24 @@ async fn restore(
>      let target = json::required_string_param(&param, "target")?;
>      let target = if target == "-" { None } else { Some(target) };
>  
> +    let mut match_list = Vec::new();
> +    if let Some(patterns) = param["matches"].as_array() {
> +        if target.is_none() {
> +            bail!("matches not allowed when restoring to stdout");
> +        }
> +
> +        for pattern in patterns {
> +            if let Some(pattern) = pattern.as_str() {
> +                let match_entry = MatchEntry::parse_pattern(
> +                    pattern,
> +                    PatternFlag::PATH_NAME,
> +                    MatchType::Include
> +                )?;
> +                match_list.push(match_entry);
> +            }
> +        }
> +    };
> +
>      let crypto = crypto_parameters(&param)?;
>  
>      let crypt_config = match crypto.enc_key {
> @@ -1429,8 +1456,8 @@ async fn restore(
>          }
>  
>          let options = pbs_client::pxar::PxarExtractOptions {
> -            match_list: &[],
> -            extract_match_default: true,
> +            match_list: match_list.as_slice(),

nit: we don't really use as_slice to convert Vecs to slices when passing
them to another fn ;) a simple `&` does the job as well (because &Vec
derefs to &[]).

> +            extract_match_default: match_list.is_empty(),
>              allow_existing_dirs,
>              overwrite_flags,
>              on_error,
> -- 
> 2.39.2
> 
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 
> 


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [pbs-devel] [PATCH v2 proxmox-backup] fix #2996: client: allow optional match patterns for restore
  2024-05-24 10:21 ` Fabian Grünbichler
@ 2024-05-24 10:45   ` Christian Ebner
  0 siblings, 0 replies; 4+ messages in thread
From: Christian Ebner @ 2024-05-24 10:45 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Fabian Grünbichler

On 5/24/24 12:21, Fabian Grünbichler wrote:
> On April 30, 2024 5:58 pm, Christian Ebner wrote:
>> When the user is only interested in a subset of the entries stored in
>> a file-level backup, it is convenient to be able to provide a list of
>> match patterns for the entries intended to be restored.
>>
>> The required restore logic is already in place. Therefore, expose it
>> for the `proxmox-backup-client restore` command by adding the optional
>> array of patterns as command line argument and parse these before
>> passing them via the pxar restore options to the archive extractor.
>>
>> Link to bugtracker issue:
>> https://bugzilla.proxmox.com/show_bug.cgi?id=2996
>>
>> Signed-off-by: Christian Ebner <c.ebner@proxmox.com>
>> Tested-by: Gabriel Goller <g.goller@proxmox.com>
>> ---
>> Changes since version 1:
>>   - bail when `matches` are used in combination with restore to stdout
>>   - fix typo in commit title
>>
>>   proxmox-backup-client/src/main.rs | 31 +++++++++++++++++++++++++++++--
>>   1 file changed, 29 insertions(+), 2 deletions(-)
>>
>> diff --git a/proxmox-backup-client/src/main.rs b/proxmox-backup-client/src/main.rs
>> index 32fe914c4..08582c86d 100644
>> --- a/proxmox-backup-client/src/main.rs
>> +++ b/proxmox-backup-client/src/main.rs
>> @@ -1186,6 +1186,15 @@ We do not extract '.pxar' archives when writing to standard output.
>>   
>>   "###
>>               },
>> +            "matches": {
>> +                type: Array,
>> +                description: "Only restore entries matching given paths or patterns.",
>> +                optional: true,
>> +                items: {
>> +                    type: String,
>> +                    description: "Path or match pattern.",
>> +                },
>> +            },
> 
> in `pxar extract`, we call this "pattern" with the following:
> 
>              pattern: {
>                  description: "List of paths or pattern matching files to restore",
>                  type: Array,
>                  items: {
>                      type: String,
>                      description: "Path or pattern matching files to restore.",
>                  },
>                  optional: true,
>              },
> 
> do we maybe want to copy that wording/name? do we have other places
> where patterns are used for inclusion that we could unify?
> 
> would it make sense to create an API type for this, or at least add some
> sort of schema-based validation up front (e.g., empty strings are not
> valid, but also trailing back slashes, and I don't know what else ;))?


Yes, makes sense to me. Will incorporate this in a new version.

> 
>>               rate: {
>>                   schema: TRAFFIC_CONTROL_RATE_SCHEMA,
>>                   optional: true,
>> @@ -1306,6 +1315,24 @@ async fn restore(
>>       let target = json::required_string_param(&param, "target")?;
>>       let target = if target == "-" { None } else { Some(target) };
>>   
>> +    let mut match_list = Vec::new();
>> +    if let Some(patterns) = param["matches"].as_array() {
>> +        if target.is_none() {
>> +            bail!("matches not allowed when restoring to stdout");
>> +        }
>> +
>> +        for pattern in patterns {
>> +            if let Some(pattern) = pattern.as_str() {
>> +                let match_entry = MatchEntry::parse_pattern(
>> +                    pattern,
>> +                    PatternFlag::PATH_NAME,
>> +                    MatchType::Include
>> +                )?;
>> +                match_list.push(match_entry);
>> +            }
>> +        }
>> +    };
>> +
>>       let crypto = crypto_parameters(&param)?;
>>   
>>       let crypt_config = match crypto.enc_key {
>> @@ -1429,8 +1456,8 @@ async fn restore(
>>           }
>>   
>>           let options = pbs_client::pxar::PxarExtractOptions {
>> -            match_list: &[],
>> -            extract_match_default: true,
>> +            match_list: match_list.as_slice(),
> 
> nit: we don't really use as_slice to convert Vecs to slices when passing
> them to another fn ;) a simple `&` does the job as well (because &Vec
> derefs to &[]).

Noted!

> 
>> +            extract_match_default: match_list.is_empty(),
>>               allow_existing_dirs,
>>               overwrite_flags,
>>               on_error,
>> -- 
>> 2.39.2
>>
>>
>>
>> _______________________________________________
>> pbs-devel mailing list
>> pbs-devel@lists.proxmox.com
>> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
>>
>>
>>
> 
> 
> _______________________________________________
> pbs-devel mailing list
> pbs-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel
> 
> 



_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel

^ permalink raw reply	[flat|nested] 4+ messages in thread

* Re: [pbs-devel] [PATCH v2 proxmox-backup] fix #2996: client: allow optional match patterns for restore
  2024-04-30 15:58 [pbs-devel] [PATCH v2 proxmox-backup] fix #2996: client: allow optional match patterns for restore Christian Ebner
  2024-05-24 10:21 ` Fabian Grünbichler
@ 2024-06-13 10:58 ` Christian Ebner
  1 sibling, 0 replies; 4+ messages in thread
From: Christian Ebner @ 2024-06-13 10:58 UTC (permalink / raw)
  To: pbs-devel

Superseded by version 3:
https://lists.proxmox.com/pipermail/pbs-devel/2024-June/009851.html


_______________________________________________
pbs-devel mailing list
pbs-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pbs-devel


^ permalink raw reply	[flat|nested] 4+ messages in thread

end of thread, other threads:[~2024-06-13 10:58 UTC | newest]

Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2024-04-30 15:58 [pbs-devel] [PATCH v2 proxmox-backup] fix #2996: client: allow optional match patterns for restore Christian Ebner
2024-05-24 10:21 ` Fabian Grünbichler
2024-05-24 10:45   ` Christian Ebner
2024-06-13 10:58 ` Christian Ebner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal