public inbox for pbs-devel@lists.proxmox.com
 help / color / mirror / Atom feed
* [pbs-devel] [PATCH proxmox-backup v2 0/3] fix #4315: datastore: Exclude entries from sync
@ 2023-11-09 14:43 Philipp Hufnagl
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 1/3] fix #4315: jobs: modify GroupFilter so include/exclude is tracked Philipp Hufnagl
                   ` (2 more replies)
  0 siblings, 3 replies; 7+ messages in thread
From: Philipp Hufnagl @ 2023-11-09 14:43 UTC (permalink / raw)
  To: pbs-devel

This allows to use Group Filter for sync jobs so matches can not just be
included but also excluded. For this the "group-filter" configuration
syntax has been extended with an optional "behaviour" parameter.
this can be "include" or "exclude". First, all include filter will be
applied, then all exclude filter.

Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
---

Changes since v1:
 * rename behaviour in behavior
 * modify behavior, so that first all include filter will be executed,
   then all exlude one


Philipp Hufnagl (3):
  fix #4315: jobs: modify GroupFilter so include/exclude is tracked
  ui: Show if Filter includes or excludes
  docs: document new include/exclude paramenter

 docs/managing-remotes.rst      |   8 ++
 pbs-api-types/src/datastore.rs |  11 +-
 pbs-api-types/src/jobs.rs      |  64 +++++++--
 src/api2/pull.rs               |  11 +-
 src/api2/tape/backup.rs        |   8 +-
 src/server/pull.rs             |   8 +-
 www/form/GroupFilter.js        | 233 ++++++++++++++++++++++++---------
 7 files changed, 257 insertions(+), 86 deletions(-)

-- 
2.39.2





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [pbs-devel] [PATCH proxmox-backup v2 1/3] fix #4315: jobs: modify GroupFilter so include/exclude is tracked
  2023-11-09 14:43 [pbs-devel] [PATCH proxmox-backup v2 0/3] fix #4315: datastore: Exclude entries from sync Philipp Hufnagl
@ 2023-11-09 14:43 ` Philipp Hufnagl
  2023-11-28  9:11   ` Lukas Wagner
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 2/3] ui: Show if Filter includes or excludes Philipp Hufnagl
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 3/3] docs: document new include/exclude paramenter Philipp Hufnagl
  2 siblings, 1 reply; 7+ messages in thread
From: Philipp Hufnagl @ 2023-11-09 14:43 UTC (permalink / raw)
  To: pbs-devel

After some discussion I canged the include/exclude behavior to first run
all include filter and after that all exclude filter (rather then
allowing to alternate inbetween). This is simply done by sorting the
list (include first) before executing it.

Since a GroupFilter now also features an behavior, the Struct has been
renamed To GroupType (since simply type is a keyword). The new
GroupFilter now has a behaviour as a flag 'is_exclude'.

I considered calling it 'is_include' but a reader later then might not
know what the opposite of 'include' is (do not include?  deactivate?). I
also considered making a new enum 'behaviour' but since there are only 2
values I considered it over engeneered.

Matching a filter will now iterate with a forech loop in order to also
exclude matches.

Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
---
 pbs-api-types/src/datastore.rs | 11 +++---
 pbs-api-types/src/jobs.rs      | 64 +++++++++++++++++++++++++++-------
 src/api2/pull.rs               | 11 +++++-
 src/api2/tape/backup.rs        |  8 ++++-
 src/server/pull.rs             |  8 ++++-
 5 files changed, 80 insertions(+), 22 deletions(-)

diff --git a/pbs-api-types/src/datastore.rs b/pbs-api-types/src/datastore.rs
index 73c4890e..c3beedaf 100644
--- a/pbs-api-types/src/datastore.rs
+++ b/pbs-api-types/src/datastore.rs
@@ -841,17 +841,16 @@ impl BackupGroup {
     }
 
     pub fn matches(&self, filter: &crate::GroupFilter) -> bool {
-        use crate::GroupFilter;
-
-        match filter {
-            GroupFilter::Group(backup_group) => {
+        use crate::FilterType;
+        match &filter.filter_type {
+            FilterType::Group(backup_group) => {
                 match backup_group.parse::<BackupGroup>() {
                     Ok(group) => *self == group,
                     Err(_) => false, // shouldn't happen if value is schema-checked
                 }
             }
-            GroupFilter::BackupType(ty) => self.ty == *ty,
-            GroupFilter::Regex(regex) => regex.is_match(&self.to_string()),
+            FilterType::BackupType(ty) => self.ty == *ty,
+            FilterType::Regex(regex) => regex.is_match(&self.to_string()),
         }
     }
 }
diff --git a/pbs-api-types/src/jobs.rs b/pbs-api-types/src/jobs.rs
index 23e19b7b..ef182795 100644
--- a/pbs-api-types/src/jobs.rs
+++ b/pbs-api-types/src/jobs.rs
@@ -3,6 +3,7 @@ use std::str::FromStr;
 
 use regex::Regex;
 use serde::{Deserialize, Serialize};
+use std::cmp::Ordering;
 
 use proxmox_schema::*;
 
@@ -383,7 +384,7 @@ pub struct TapeBackupJobStatus {
 
 #[derive(Clone, Debug)]
 /// Filter for matching `BackupGroup`s, for use with `BackupGroup::filter`.
-pub enum GroupFilter {
+pub enum FilterType {
     /// BackupGroup type - either `vm`, `ct`, or `host`.
     BackupType(BackupType),
     /// Full identifier of BackupGroup, including type
@@ -392,7 +393,7 @@ pub enum GroupFilter {
     Regex(Regex),
 }
 
-impl PartialEq for GroupFilter {
+impl PartialEq for FilterType {
     fn eq(&self, other: &Self) -> bool {
         match (self, other) {
             (Self::BackupType(a), Self::BackupType(b)) => a == b,
@@ -403,27 +404,64 @@ impl PartialEq for GroupFilter {
     }
 }
 
+#[derive(Clone, Debug)]
+pub struct GroupFilter {
+    pub is_exclude: bool,
+    pub filter_type: FilterType,
+}
+
+impl PartialEq for GroupFilter {
+    fn eq(&self, other: &Self) -> bool {
+        self.filter_type == other.filter_type && self.is_exclude == other.is_exclude
+    }
+}
+
+impl Eq for GroupFilter {}
+
+impl PartialOrd for GroupFilter {
+    fn partial_cmp(&self, other: &Self) -> Option<Ordering> {
+        self.is_exclude.partial_cmp(&other.is_exclude)
+    }
+}
+
+impl Ord for GroupFilter {
+    fn cmp(&self, other: &Self) -> Ordering {
+        self.is_exclude.cmp(&other.is_exclude)
+    }
+}
+
 impl std::str::FromStr for GroupFilter {
     type Err = anyhow::Error;
 
     fn from_str(s: &str) -> Result<Self, Self::Err> {
-        match s.split_once(':') {
-            Some(("group", value)) => BACKUP_GROUP_SCHEMA.parse_simple_value(value).map(|_| GroupFilter::Group(value.to_string())),
-            Some(("type", value)) => Ok(GroupFilter::BackupType(value.parse()?)),
-            Some(("regex", value)) => Ok(GroupFilter::Regex(Regex::new(value)?)),
+        let (is_exclude, type_str) = match s.split_once(':') {
+            Some(("include", value)) => (false, value),
+            Some(("exclude", value)) => (true, value),
+            _ => (false, s),
+        };
+
+        let filter_type = match type_str.split_once(':') {
+            Some(("group", value)) => BACKUP_GROUP_SCHEMA.parse_simple_value(value).map(|_| FilterType::Group(value.to_string())),
+            Some(("type", value)) => Ok(FilterType::BackupType(value.parse()?)),
+            Some(("regex", value)) => Ok(FilterType::Regex(Regex::new(value)?)),
             Some((ty, _value)) => Err(format_err!("expected 'group', 'type' or 'regex' prefix, got '{}'", ty)),
             None => Err(format_err!("input doesn't match expected format '<group:GROUP||type:<vm|ct|host>|regex:REGEX>'")),
-        }.map_err(|err| format_err!("'{}' - {}", s, err))
+        }?;
+        Ok(GroupFilter {
+            is_exclude,
+            filter_type,
+        })
     }
 }
 
 // used for serializing below, caution!
 impl std::fmt::Display for GroupFilter {
     fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
-        match self {
-            GroupFilter::BackupType(backup_type) => write!(f, "type:{}", backup_type),
-            GroupFilter::Group(backup_group) => write!(f, "group:{}", backup_group),
-            GroupFilter::Regex(regex) => write!(f, "regex:{}", regex.as_str()),
+        let exclude = if self.is_exclude { "exclude:" } else { "" };
+        match &self.filter_type {
+            FilterType::BackupType(backup_type) => write!(f, "{}type:{}", exclude, backup_type),
+            FilterType::Group(backup_group) => write!(f, "{}group:{}", exclude, backup_group),
+            FilterType::Regex(regex) => write!(f, "{}regex:{}", exclude, regex.as_str()),
         }
     }
 }
@@ -436,9 +474,9 @@ fn verify_group_filter(input: &str) -> Result<(), anyhow::Error> {
 }
 
 pub const GROUP_FILTER_SCHEMA: Schema = StringSchema::new(
-    "Group filter based on group identifier ('group:GROUP'), group type ('type:<vm|ct|host>'), or regex ('regex:RE').")
+    "Group filter based on group identifier ('group:GROUP'), group type ('type:<vm|ct|host>'), or regex ('regex:RE'). Can be inverted by adding 'exclude:' before.")
     .format(&ApiStringFormat::VerifyFn(verify_group_filter))
-    .type_text("<type:<vm|ct|host>|group:GROUP|regex:RE>")
+    .type_text("[<exclude:|include:>]<type:<vm|ct|host>|group:GROUP|regex:RE>")
     .schema();
 
 pub const GROUP_FILTER_LIST_SCHEMA: Schema =
diff --git a/src/api2/pull.rs b/src/api2/pull.rs
index daeba7cf..20571345 100644
--- a/src/api2/pull.rs
+++ b/src/api2/pull.rs
@@ -62,6 +62,15 @@ impl TryFrom<&SyncJobConfig> for PullParameters {
     type Error = Error;
 
     fn try_from(sync_job: &SyncJobConfig) -> Result<Self, Self::Error> {
+        let filters = match &sync_job.group_filter {
+            Some(v) => {
+                let mut f = v.clone();
+                f.sort();
+                Some(f)
+            }
+            None => None,
+        };
+
         PullParameters::new(
             &sync_job.store,
             sync_job.ns.clone().unwrap_or_default(),
@@ -75,7 +84,7 @@ impl TryFrom<&SyncJobConfig> for PullParameters {
                 .clone(),
             sync_job.remove_vanished,
             sync_job.max_depth,
-            sync_job.group_filter.clone(),
+            filters,
             sync_job.limit.clone(),
             sync_job.transfer_last,
         )
diff --git a/src/api2/tape/backup.rs b/src/api2/tape/backup.rs
index 2f9385a7..04abdb71 100644
--- a/src/api2/tape/backup.rs
+++ b/src/api2/tape/backup.rs
@@ -413,7 +413,13 @@ fn backup_worker(
 
     let (group_list, group_count) = if let Some(group_filters) = &setup.group_filter {
         let filter_fn = |group: &BackupGroup, group_filters: &[GroupFilter]| {
-            group_filters.iter().any(|filter| group.matches(filter))
+            let mut is_match = false;
+            for filter in group_filters.iter() {
+                if group.matches(filter) {
+                    is_match = !filter.is_exclude;
+                }
+            }
+            is_match
         };
 
         let group_count_full = group_list.len();
diff --git a/src/server/pull.rs b/src/server/pull.rs
index a973a10e..0124bd3f 100644
--- a/src/server/pull.rs
+++ b/src/server/pull.rs
@@ -1084,7 +1084,13 @@ pub(crate) async fn pull_ns(
     });
 
     let apply_filters = |group: &pbs_api_types::BackupGroup, filters: &[GroupFilter]| -> bool {
-        filters.iter().any(|filter| group.matches(filter))
+        let mut is_match = false;
+        for filter in filters.iter() {
+            if group.matches(filter) {
+                is_match = !filter.is_exclude;
+            }
+        }
+        is_match
     };
 
     // Get groups with target NS set
-- 
2.39.2





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [pbs-devel] [PATCH proxmox-backup v2 2/3] ui: Show if Filter includes or excludes
  2023-11-09 14:43 [pbs-devel] [PATCH proxmox-backup v2 0/3] fix #4315: datastore: Exclude entries from sync Philipp Hufnagl
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 1/3] fix #4315: jobs: modify GroupFilter so include/exclude is tracked Philipp Hufnagl
@ 2023-11-09 14:43 ` Philipp Hufnagl
  2023-11-28  9:57   ` Lukas Wagner
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 3/3] docs: document new include/exclude paramenter Philipp Hufnagl
  2 siblings, 1 reply; 7+ messages in thread
From: Philipp Hufnagl @ 2023-11-09 14:43 UTC (permalink / raw)
  To: pbs-devel

To make the UI compatible, the Group Filter dialogue has been extended
by a second list, so it now features a list for all include filter and
one for all exclude filters.

Internally, all include as well as exclude filter are managed into one
list. The 2 list view is just for a cleaner representation in the UI.

Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
---
 www/form/GroupFilter.js | 233 +++++++++++++++++++++++++++++-----------
 1 file changed, 169 insertions(+), 64 deletions(-)

diff --git a/www/form/GroupFilter.js b/www/form/GroupFilter.js
index dee37b0b..fb4a843a 100644
--- a/www/form/GroupFilter.js
+++ b/www/form/GroupFilter.js
@@ -35,13 +35,36 @@ Ext.define('PBS.form.GroupFilter', {
 	    // break cyclic reference
 	    me.removeReferences(record);
 
-	    me.lookup('grid').getStore().remove(record);
+	    me.lookup('grid-include').getStore().remove(record);
+	    me.lookup('grid-exclude').getStore().remove(record);
 	    me.updateRealField();
 	},
 
-	addFilter: function() {
+	addIncludeFilter: function() {
 	    let me = this;
-	    me.lookup('grid').getStore().add({});
+	    me.lookup('grid-include').getStore().add({ behavior: 'include' });
+	    me.updateRealField();
+	},
+
+	addExcludeFilter: function() {
+	    let me = this;
+	    me.lookup('grid-exclude').getStore().add({ behavior: 'exclude' });
+	    me.updateRealField();
+	},
+
+
+	onBehaviorChange: function(field, value) {
+	    let me = this;
+	    let record = field.getWidgetRecord();
+	    if (record === undefined) {
+		return;
+	    }
+
+	    record.set('behavior', value);
+	    record.commit();
+	    if (record.widgets) {
+		me.setInputValue(record.widgets, record);
+	    }
 	    me.updateRealField();
 	},
 
@@ -77,8 +100,12 @@ Ext.define('PBS.form.GroupFilter', {
 	},
 
 	parseGroupFilter: function(filter) {
-	    let [, type, input] = filter.match(/^(type|group|regex):(.*)$/);
+	    let [, behavior, type, input] = filter.match(/^(?:(exclude|include):)?(type|group|regex):(.*)$/);
+	    if (behavior === undefined) {
+		behavior = "include";
+	    }
 	    return {
+		behavior,
 		type,
 		input,
 	    };
@@ -86,13 +113,16 @@ Ext.define('PBS.form.GroupFilter', {
 
 	onValueChange: function(field, values) {
 	    let me = this;
-	    let grid = me.lookup('grid');
+	    let grid_include = me.lookup('grid-include');
+	    let grid_exclude = me.lookup('grid-exclude');
 	    if (!values || values.length === 0) {
-		grid.getStore().removeAll();
+		grid_include.getStore().removeAll();
+		grid_exclude.getStore().removeAll();
 		return;
 	    }
 	    let records = values.map((filter) => me.parseGroupFilter(filter));
-	    grid.getStore().setData(records);
+	    grid_include.getStore().setData(records);
+	    grid_exclude.getStore().setData(records);
 	},
 
 	setInputValue: function(widgets, rec) {
@@ -162,9 +192,18 @@ Ext.define('PBS.form.GroupFilter', {
 	    let me = this;
 
 	    let filter = [];
-	    me.lookup('grid').getStore().each((rec) => {
-		if (rec.data.type && rec.data.input) {
-		    filter.push(`${rec.data.type}:${rec.data.input}`);
+	    me.lookup('grid-include').getStore().each((rec) => {
+	    if (rec.data.type && rec.data.input) {
+		filter.push(`${rec.data.type}:${rec.data.input}`);
+		}
+	    });
+	    me.lookup('grid-exclude').getStore().each((rec) => {
+		if (rec.data.type && rec.data.input && rec.data.behavior) {
+		    let behavior_string = '';
+		    if (rec.data.behavior === 'exclude') {
+			behavior_string = 'exclude:';
+		    }
+		    filter.push(`${behavior_string}${rec.data.type}:${rec.data.input}`);
 		}
 	    });
 
@@ -175,6 +214,9 @@ Ext.define('PBS.form.GroupFilter', {
 	},
 
 	control: {
+	    'grid pbsGroupBehaviorSelector': {
+		change: 'onBehaviorChange',
+	    },
 	    'grid pbsGroupFilterTypeSelector': {
 		change: 'onTypeChange',
 	    },
@@ -264,72 +306,59 @@ Ext.define('PBS.form.GroupFilter', {
 
     items: [
 	{
-	    xtype: 'grid',
-	    reference: 'grid',
+	    xtype: 'pbsGroupFilterGrid',
+	    title: 'Include filters',
 	    margin: '0 0 5 0',
-	    scrollable: true,
-	    height: 300,
+	    reference: 'grid-include',
 	    store: {
-		fields: ['type', 'input'],
-	    },
+		filters: [
+		    function(item) {
+                    return item.data.behavior === "include";
+		    },
+		],
+            },
 	    emptyText: gettext('Include all groups'),
 	    viewConfig: {
 		deferEmptyText: false,
 	    },
-	    columns: [
+	},
+	{
+	    xtype: 'container',
+	    layout: {
+		type: 'hbox',
+	    },
+	    items: [
 		{
-		    text: gettext('Filter Type'),
-		    xtype: 'widgetcolumn',
-		    dataIndex: 'type',
-		    flex: 1,
-		    widget: {
-			xtype: 'pbsGroupFilterTypeSelector',
-			isFormField: false,
-		    },
+		    xtype: 'button',
+		    text: gettext('Add include'),
+		    iconCls: 'fa fa-plus-circle',
+		    handler: 'addIncludeFilter',
 		},
 		{
-		    text: gettext('Filter Value'),
-		    xtype: 'widgetcolumn',
+		    xtype: 'box',
 		    flex: 1,
-		    onWidgetAttach: 'newInputColumn',
-		    widget: {
-			padding: 0,
-			bodyPadding: 0,
-			xtype: 'fieldcontainer',
-			layout: 'fit',
-			defaults: {
-			    margin: 0,
-			},
-			items: [
-			    {
-				hidden: true,
-				xtype: 'pbsGroupTypeSelector',
-				isFormField: false,
-			    },
-			    {
-				hidden: true,
-				xtype: 'textfield',
-				type: 'regex',
-				isFormField: false,
-			    },
-			    {
-				hidden: true,
-				xtype: 'pbsGroupSelector',
-				isFormField: false,
-			    },
-			],
-		    },
 		},
 		{
-		    xtype: 'widgetcolumn',
-		    width: 40,
-		    widget: {
-			xtype: 'button',
-			iconCls: 'fa fa-trash-o',
-		    },
+		    xtype: 'box',
+		    style: 'margin: 3px 0px;',
+		    html: `<span class="pmx-hint">${gettext('Note')}</span>: `
+			+ gettext('Filters are additive'),
 		},
 	    ],
 	},
+	{
+	    xtype: 'pbsGroupFilterGrid',
+	    title: 'Exclude filters',
+	    margin: '10 0 5 0',
+	    reference: 'grid-exclude',
+	    store: {
+		filters: [
+		    function(item) {
+                    return item.data.behavior === "exclude";
+		    },
+		],
+            },
+	},
 	{
 	    xtype: 'hiddenfield',
 	    reference: 'realfield',
@@ -356,9 +385,9 @@ Ext.define('PBS.form.GroupFilter', {
 	    items: [
 		{
 		    xtype: 'button',
-		    text: gettext('Add'),
+		    text: gettext('Add exclude'),
 		    iconCls: 'fa fa-plus-circle',
-		    handler: 'addFilter',
+		    handler: 'addExcludeFilter',
 		},
 		{
 		    xtype: 'box',
@@ -368,7 +397,7 @@ Ext.define('PBS.form.GroupFilter', {
 		    xtype: 'box',
 		    style: 'margin: 3px 0px;',
 		    html: `<span class="pmx-hint">${gettext('Note')}</span>: `
-			+ gettext('Filters are additive (OR-like)'),
+			+ gettext('Exclude filters will be applied after include filters'),
 		},
 	    ],
 	},
@@ -384,6 +413,82 @@ Ext.define('PBS.form.GroupFilter', {
     },
 });
 
+Ext.define('PBS.form.pbsGroupBehaviorSelector', {
+    extend: 'Proxmox.form.KVComboBox',
+    alias: 'widget.pbsGroupBehaviorSelector',
+
+    allowBlank: false,
+
+    comboItems: [
+	['include', gettext('Include')],
+	['exclude', gettext('Exclude')],
+    ],
+});
+Ext.define('PBS.form.GroupFilterGrid', {
+    extend: 'Ext.grid.Panel',
+    alias: 'widget.pbsGroupFilterGrid',
+
+    scrollable: true,
+    height: 300,
+    store: {
+	fields: ['type', 'input'],
+            },
+    columns: [
+	{
+	    text: gettext('Filter Type'),
+	    xtype: 'widgetcolumn',
+	    dataIndex: 'type',
+	    flex: 1,
+	    widget: {
+		xtype: 'pbsGroupFilterTypeSelector',
+		isFormField: false,
+	    },
+	},
+	{
+	    text: gettext('Filter Value'),
+	    xtype: 'widgetcolumn',
+	    flex: 1,
+	    onWidgetAttach: 'newInputColumn',
+	    widget: {
+		padding: 0,
+		bodyPadding: 0,
+		xtype: 'fieldcontainer',
+		layout: 'fit',
+		defaults:
+		{
+		    margin: 0,
+		},
+		items: [
+		    {
+			hidden: true,
+			xtype: 'pbsGroupTypeSelector',
+			isFormField: false,
+		    },
+		    {
+			hidden: true,
+			xtype: 'textfield',
+			type: 'regex',
+			isFormField: false,
+		    },
+		    {
+			hidden: true,
+			xtype: 'pbsGroupSelector',
+			isFormField: false,
+		    },
+		],
+	    },
+	},
+	{
+	    xtype: 'widgetcolumn',
+	    width: 30,
+	    widget: {
+		xtype: 'button',
+		iconCls: 'fa fa-trash-o',
+	    },
+	},
+    ],
+});
+
 Ext.define('PBS.form.GroupFilterTypeSelector', {
     extend: 'Proxmox.form.KVComboBox',
     alias: 'widget.pbsGroupFilterTypeSelector',
-- 
2.39.2





^ permalink raw reply	[flat|nested] 7+ messages in thread

* [pbs-devel] [PATCH proxmox-backup v2 3/3] docs: document new include/exclude paramenter
  2023-11-09 14:43 [pbs-devel] [PATCH proxmox-backup v2 0/3] fix #4315: datastore: Exclude entries from sync Philipp Hufnagl
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 1/3] fix #4315: jobs: modify GroupFilter so include/exclude is tracked Philipp Hufnagl
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 2/3] ui: Show if Filter includes or excludes Philipp Hufnagl
@ 2023-11-09 14:43 ` Philipp Hufnagl
  2023-11-28  9:30   ` Lukas Wagner
  2 siblings, 1 reply; 7+ messages in thread
From: Philipp Hufnagl @ 2023-11-09 14:43 UTC (permalink / raw)
  To: pbs-devel

Adding the newly introduced optional include/exclude parameter to the
PBS documentation.

Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
---
 docs/managing-remotes.rst | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/docs/managing-remotes.rst b/docs/managing-remotes.rst
index 1c52e120..2f08c3b1 100644
--- a/docs/managing-remotes.rst
+++ b/docs/managing-remotes.rst
@@ -116,6 +116,14 @@ of the specified criteria are synced. The available criteria are:
 The same filter is applied to local groups, for handling of the
 ``remove-vanished`` option.
 
+A ``group-filter`` can be inverted by adding ``exclude:`` to its beginning.
+
+* Regular expression example, excluding the match:
+    .. code-block:: console
+
+     # proxmox-backup-manager sync-job update ID --group-filter exclude:regex:'^vm/1\d{2,3}$'
+
+If include as well as exclude filters are used, all include filters will be run before the exclude filters.
 .. note:: The ``protected`` flag of remote backup snapshots will not be synced.
 
 Namespace Support
-- 
2.39.2





^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup v2 1/3] fix #4315: jobs: modify GroupFilter so include/exclude is tracked
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 1/3] fix #4315: jobs: modify GroupFilter so include/exclude is tracked Philipp Hufnagl
@ 2023-11-28  9:11   ` Lukas Wagner
  0 siblings, 0 replies; 7+ messages in thread
From: Lukas Wagner @ 2023-11-28  9:11 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Philipp Hufnagl



On 11/9/23 15:43, Philipp Hufnagl wrote:
> After some discussion I canged the include/exclude behavior to first run
> all include filter and after that all exclude filter (rather then
> allowing to alternate inbetween). This is simply done by sorting the
> list (include first) before executing it.
> 
> Since a GroupFilter now also features an behavior, the Struct has been
> renamed To GroupType (since simply type is a keyword). The new
> GroupFilter now has a behaviour as a flag 'is_exclude'.
> 
> I considered calling it 'is_include' but a reader later then might not
> know what the opposite of 'include' is (do not include?  deactivate?). I
> also considered making a new enum 'behaviour' but since there are only 2
> values I considered it over engeneered.
> 
> Matching a filter will now iterate with a forech loop in order to also
> exclude matches.

This patch does not apply on the latest master, needs to be rebased :)

-- 
- Lukas




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup v2 3/3] docs: document new include/exclude paramenter
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 3/3] docs: document new include/exclude paramenter Philipp Hufnagl
@ 2023-11-28  9:30   ` Lukas Wagner
  0 siblings, 0 replies; 7+ messages in thread
From: Lukas Wagner @ 2023-11-28  9:30 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Philipp Hufnagl



On 11/9/23 15:43, Philipp Hufnagl wrote:
> Adding the newly introduced optional include/exclude parameter to the
> PBS documentation.
> 
> Signed-off-by: Philipp Hufnagl <p.hufnagl@proxmox.com>
> ---
>   docs/managing-remotes.rst | 8 ++++++++
>   1 file changed, 8 insertions(+)
> 
> diff --git a/docs/managing-remotes.rst b/docs/managing-remotes.rst
> index 1c52e120..2f08c3b1 100644
> --- a/docs/managing-remotes.rst
> +++ b/docs/managing-remotes.rst
> @@ -116,6 +116,14 @@ of the specified criteria are synced. The available criteria are:
>   The same filter is applied to local groups, for handling of the
>   ``remove-vanished`` option.
>   
> +A ``group-filter`` can be inverted by adding ``exclude:`` to its beginning.
> +
> +* Regular expression example, excluding the match:
> +    .. code-block:: console
> +
> +     # proxmox-backup-manager sync-job update ID --group-filter exclude:regex:'^vm/1\d{2,3}$'
> +
> +If include as well as exclude filters are used, all include filters will be run before the exclude filters.

Maybe it would make sense here to enumerate the different cases and 
their exact behaviors, e.g. in a table?

  - no filters: all backup groups
  - include: only those matching the include filters
  - exclude: all but those matching the exclude filters
  - both: those matching the include filters, but without those matching 
the exclude filters

Also, I'm not sure if 'will be run before' is easy to understand for 
users. I think something alike how I described the 'both' case should be 
easier to understand


-- 
- Lukas




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: [pbs-devel] [PATCH proxmox-backup v2 2/3] ui: Show if Filter includes or excludes
  2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 2/3] ui: Show if Filter includes or excludes Philipp Hufnagl
@ 2023-11-28  9:57   ` Lukas Wagner
  0 siblings, 0 replies; 7+ messages in thread
From: Lukas Wagner @ 2023-11-28  9:57 UTC (permalink / raw)
  To: Proxmox Backup Server development discussion, Philipp Hufnagl



On 11/9/23 15:43, Philipp Hufnagl wrote:
> To make the UI compatible, the Group Filter dialogue has been extended
> by a second list, so it now features a list for all include filter and
> one for all exclude filters.
> 
> Internally, all include as well as exclude filter are managed into one
> list. The 2 list view is just for a cleaner representation in the UI.
> 

The edit window becomes pretty tall with the second grid for the 
excludes. Assuming a screen resolution of 1368x768 (which is still in 
use in some low-end laptops) and subtracting a few pixels for task/menu 
bars, the edit window is not really usable on a screen with that 
resolution. You can test/simulate that by pressing Ctrl-Shift-M in 
Firefox while being in web inspector view.

Maybe we could have the exludes in the same grid view, but sort it so 
that the 'includes' come before 'exludes'? Alternatively, we could 
reduce the height of the two grid views (not sure how many rules people 
have on average, it'd be bad if people have to scroll a lot)


-- 
- Lukas




^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2023-11-28  9:57 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-11-09 14:43 [pbs-devel] [PATCH proxmox-backup v2 0/3] fix #4315: datastore: Exclude entries from sync Philipp Hufnagl
2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 1/3] fix #4315: jobs: modify GroupFilter so include/exclude is tracked Philipp Hufnagl
2023-11-28  9:11   ` Lukas Wagner
2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 2/3] ui: Show if Filter includes or excludes Philipp Hufnagl
2023-11-28  9:57   ` Lukas Wagner
2023-11-09 14:43 ` [pbs-devel] [PATCH proxmox-backup v2 3/3] docs: document new include/exclude paramenter Philipp Hufnagl
2023-11-28  9:30   ` Lukas Wagner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox
Service provided by Proxmox Server Solutions GmbH | Privacy | Legal