* [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters
@ 2025-10-29 14:48 Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 01/13] fake remote: add missing parameter for cluster_metrics_export function Lukas Wagner
` (13 more replies)
0 siblings, 14 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
Key aspects:
- new config file at /etc/proxmox-datacenter-manager/views/filters.cfg
(subject to change, depending on where the view templates are stored)
- ViewFilterConfig as a filter defintion type,has
- {include,exclude}-{remote,resource-id,resource-type,resource-pool,tag}
- Filter implementation & big suite of unit tests
- excludes are processed after includes
- if no include rules are defined, all resources but those which were
excluded are matched
- if no rules are defined in a filter, everything is matched
- Added the 'view-filter' parameter to a couple of API endpoints
- /resources/list
- /resources/status
- /resources/subscription
- /resources/top-entities
- /remote-tasks/list
- /remote-tasks/statistis
- ACL checks are done on /view/{view-filter-id} for now, replace
any other permission check in the handler iff the view-filter paramter
is set
Left to do:
- CRUD for filter definition
- Decide on how exactly they will work together with the view templates
-> most likely a new config type
View {
filter: String,
template: String,
}
- UI for filter rules
proxmox-datacenter-manager:
Lukas Wagner (13):
fake remote: add missing parameter for cluster_metrics_export function
pdm-api-types: views: add ViewFilterConfig type
pdm-config: views: add support for view-filters
acl: add '/view' and '/view/{view-id}' as allowed ACL paths
views: add implementation for view filters
views: add tests for view filter implementation
api: resources: list: add support for view-filter parameter
api: resources: top entities: add support for view-filter parameter
api: resources: status: add support for view-filter parameter
api: subscription status: add support for view-filter parameter
api: remote-tasks: add support for view-filter parameter
pdm-client: resource list: add view-filter parameter
pdm-client: top entities: add view-filter parameter
cli/client/src/resources.rs | 2 +-
lib/pdm-api-types/src/lib.rs | 8 +
lib/pdm-api-types/src/views.rs | 147 ++++++
lib/pdm-client/src/lib.rs | 19 +-
lib/pdm-config/src/lib.rs | 2 +-
lib/pdm-config/src/views.rs | 62 +++
server/src/acl.rs | 6 +
server/src/api/remote_tasks.rs | 36 +-
server/src/api/resources.rs | 152 +++++-
server/src/lib.rs | 1 +
server/src/metric_collection/top_entities.rs | 5 +
server/src/remote_tasks/mod.rs | 39 +-
server/src/resource_cache.rs | 3 +-
server/src/test_support/fake_remote.rs | 1 +
server/src/views/mod.rs | 4 +
server/src/views/tests.rs | 486 +++++++++++++++++++
server/src/views/view_filter.rs | 225 +++++++++
ui/src/dashboard/mod.rs | 2 +-
ui/src/sdn/zone_tree.rs | 2 +-
19 files changed, 1157 insertions(+), 45 deletions(-)
create mode 100644 lib/pdm-api-types/src/views.rs
create mode 100644 lib/pdm-config/src/views.rs
create mode 100644 server/src/views/mod.rs
create mode 100644 server/src/views/tests.rs
create mode 100644 server/src/views/view_filter.rs
Summary over all repositories:
19 files changed, 1157 insertions(+), 45 deletions(-)
--
Generated by murpp 0.9.0
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 01/13] fake remote: add missing parameter for cluster_metrics_export function
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-30 10:26 ` [pdm-devel] applied: " Wolfgang Bumiller
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 02/13] pdm-api-types: views: add ViewFilterConfig type Lukas Wagner
` (12 subsequent siblings)
13 siblings, 1 reply; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/test_support/fake_remote.rs | 1 +
1 file changed, 1 insertion(+)
diff --git a/server/src/test_support/fake_remote.rs b/server/src/test_support/fake_remote.rs
index f4f6967e..cd2ccf60 100644
--- a/server/src/test_support/fake_remote.rs
+++ b/server/src/test_support/fake_remote.rs
@@ -276,6 +276,7 @@ impl pve_api_types::client::PveClient for FakePveClient {
&self,
_history: Option<bool>,
_local_only: Option<bool>,
+ _node_list: Option<String>,
start_time: Option<i64>,
) -> Result<ClusterMetrics, proxmox_client::Error> {
tokio::time::sleep(Duration::from_millis(self.api_delay_ms as u64)).await;
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 02/13] pdm-api-types: views: add ViewFilterConfig type
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 01/13] fake remote: add missing parameter for cluster_metrics_export function Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 03/13] pdm-config: views: add support for view-filters Lukas Wagner
` (11 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
This type will be used to define view filters.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
lib/pdm-api-types/src/lib.rs | 2 +
lib/pdm-api-types/src/views.rs | 147 +++++++++++++++++++++++++++++++++
2 files changed, 149 insertions(+)
create mode 100644 lib/pdm-api-types/src/views.rs
diff --git a/lib/pdm-api-types/src/lib.rs b/lib/pdm-api-types/src/lib.rs
index 2fb61ef7..a7ba6d89 100644
--- a/lib/pdm-api-types/src/lib.rs
+++ b/lib/pdm-api-types/src/lib.rs
@@ -106,6 +106,8 @@ pub mod subscription;
pub mod sdn;
+pub mod views;
+
const_regex! {
// just a rough check - dummy acceptor is used before persisting
pub OPENSSL_CIPHERS_REGEX = r"^[0-9A-Za-z_:, +!\-@=.]+$";
diff --git a/lib/pdm-api-types/src/views.rs b/lib/pdm-api-types/src/views.rs
new file mode 100644
index 00000000..8a00f4de
--- /dev/null
+++ b/lib/pdm-api-types/src/views.rs
@@ -0,0 +1,147 @@
+use serde::{Deserialize, Serialize};
+
+use proxmox_schema::{api, Updater};
+
+use crate::{remotes::REMOTE_ID_SCHEMA, resource::ResourceType};
+
+#[api(
+ properties: {
+ "include-resource-id": {
+ type: Array,
+ items: {
+ // TODO: Define some schema for this somewhere.
+ type: String,
+ description: "Global resource ID, e.g. 'remote/{remote}/node/{nodename}'",
+ },
+ optional: true,
+ },
+ "exclude-resource-id": {
+ type: Array,
+ items: {
+ // TODO: Define some schema for this somewhere.
+ type: String,
+ description: "Global resource ID, e.g. 'remote/{remote}/node/{nodename}'",
+ },
+ optional: true,
+ },
+ "include-remote": {
+ type: Array,
+ items: {
+ schema: REMOTE_ID_SCHEMA,
+ },
+ optional: true,
+ },
+ "exclude-remote": {
+ type: Array,
+ items: {
+ schema: REMOTE_ID_SCHEMA,
+ },
+ optional: true,
+ },
+ "include-resource-type": {
+ type: Array,
+ items: {
+ type: ResourceType,
+ },
+ optional: true,
+ },
+ "exclude-resource-type": {
+ type: Array,
+ items: {
+ type: ResourceType,
+ },
+ optional: true,
+ },
+ "include-tag": {
+ type: Array,
+ items: {
+ type: String,
+ description: "A tag of remote guest."
+ },
+ optional: true,
+ },
+ "exclude-tag": {
+ type: Array,
+ items: {
+ type: String,
+ description: "A tag of remote guest."
+ },
+ optional: true,
+ },
+ "include-resource-pool": {
+ type: Array,
+ items: {
+ // TODO: Define some schema for this somewhere.
+ type: String,
+ description: "Pool name."
+ },
+ optional: true,
+ },
+ "exclude-resource-pool": {
+ type: Array,
+ items: {
+ // TODO: Define some schema for this somewhere.
+ type: String,
+ description: "Pool name."
+ },
+ optional: true,
+ }
+ }
+)]
+#[derive(Clone, Default, Deserialize, Serialize, Updater)]
+#[serde(rename_all = "kebab-case")]
+/// View definition
+pub struct ViewFilterConfig {
+ /// View filter name
+ pub id: String,
+
+ /// List of resource IDs to include.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub include_resource_id: Vec<String>,
+
+ /// List of remotes to include.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub include_remote: Vec<String>,
+
+ /// List of included resource types.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub include_resource_type: Vec<ResourceType>,
+
+ /// List of included tags.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub include_tag: Vec<String>,
+
+ /// List of included resource pools.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub include_resource_pool: Vec<String>,
+
+ /// List of resource IDs to exclude.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub exclude_resource_id: Vec<String>,
+
+ /// List of remotes to exclude.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub exclude_remote: Vec<String>,
+
+ /// List of excluded resource types.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub exclude_resource_type: Vec<ResourceType>,
+
+ /// List of excluded tags.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub exclude_tag: Vec<String>,
+
+ /// List of excluded resource pools.
+ #[serde(default, skip_serializing_if = "Vec::is_empty")]
+ #[updater(serde(skip_serializing_if = "Option::is_none"))]
+ pub exclude_resource_pool: Vec<String>,
+}
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 03/13] pdm-config: views: add support for view-filters
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 01/13] fake remote: add missing parameter for cluster_metrics_export function Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 02/13] pdm-api-types: views: add ViewFilterConfig type Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 04/13] acl: add '/view' and '/view/{view-id}' as allowed ACL paths Lukas Wagner
` (10 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
This allows to read ViewFilterConfig entries from a new config file at
/etc/proxmox-datacenter-manager/views/filters.cfg.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
lib/pdm-api-types/src/lib.rs | 6 ++++
lib/pdm-config/src/lib.rs | 2 +-
lib/pdm-config/src/views.rs | 62 ++++++++++++++++++++++++++++++++++++
3 files changed, 69 insertions(+), 1 deletion(-)
create mode 100644 lib/pdm-config/src/views.rs
diff --git a/lib/pdm-api-types/src/lib.rs b/lib/pdm-api-types/src/lib.rs
index a7ba6d89..ed284ab2 100644
--- a/lib/pdm-api-types/src/lib.rs
+++ b/lib/pdm-api-types/src/lib.rs
@@ -159,6 +159,12 @@ pub const REALM_ID_SCHEMA: Schema = StringSchema::new("Realm name.")
.max_length(32)
.schema();
+pub const VIEW_FILTER_ID_SCHEMA: Schema = StringSchema::new("View filter name.")
+ .format(&PROXMOX_SAFE_ID_FORMAT)
+ .min_length(2)
+ .max_length(32)
+ .schema();
+
pub const VMID_SCHEMA: Schema = IntegerSchema::new("A guest ID").minimum(1).schema();
pub const SNAPSHOT_NAME_SCHEMA: Schema = StringSchema::new("The name of the snapshot")
.format(&PROXMOX_SAFE_ID_FORMAT)
diff --git a/lib/pdm-config/src/lib.rs b/lib/pdm-config/src/lib.rs
index ac398cab..4c490541 100644
--- a/lib/pdm-config/src/lib.rs
+++ b/lib/pdm-config/src/lib.rs
@@ -1,6 +1,5 @@
use anyhow::{format_err, Error};
use nix::unistd::{Gid, Group, Uid, User};
-
pub use pdm_buildcfg::{BACKUP_GROUP_NAME, BACKUP_USER_NAME};
pub mod certificate_config;
@@ -8,6 +7,7 @@ pub mod domains;
pub mod node;
pub mod remotes;
pub mod setup;
+pub mod views;
mod config_version_cache;
pub use config_version_cache::ConfigVersionCache;
diff --git a/lib/pdm-config/src/views.rs b/lib/pdm-config/src/views.rs
new file mode 100644
index 00000000..adeb67b1
--- /dev/null
+++ b/lib/pdm-config/src/views.rs
@@ -0,0 +1,62 @@
+use std::sync::LazyLock;
+
+use anyhow::Error;
+
+use proxmox_schema::ApiType;
+use proxmox_section_config::{SectionConfig, SectionConfigData, SectionConfigPlugin};
+
+use pdm_api_types::{views::ViewFilterConfig, ConfigDigest, VIEW_FILTER_ID_SCHEMA};
+
+use pdm_buildcfg::configdir;
+
+static VIEW_FILTER_SECTION_NAME: &str = "view-filter";
+
+static CONFIG: LazyLock<SectionConfig> = LazyLock::new(init);
+
+fn init() -> SectionConfig {
+ let mut config = SectionConfig::new(&VIEW_FILTER_ID_SCHEMA);
+
+ let view_plugin = SectionConfigPlugin::new(
+ VIEW_FILTER_SECTION_NAME.to_string(),
+ Some("id".to_string()),
+ ViewFilterConfig::API_SCHEMA.unwrap_object_schema(),
+ );
+ config.register_plugin(view_plugin);
+
+ config
+}
+
+const VIEW_FILTER_CFG_FILENAME: &str = configdir!("/views/filters.cfg");
+
+// TODO: Will be needed once the CRUD API for view filters is implemented.
+// const VIEW_FILTER_CFG_LOCKFILE: &str = configdir!("/views/.filters.lock");
+
+// TODO: Will be needed once the CRUD API for view filters is implemented.
+/// Get exclusive lock
+// fn lock_config() -> Result<ApiLockGuard, Error> {
+// open_api_lockfile(VIEW_FILTER_CFG_LOCKFILE, None, true)
+// }
+
+fn config() -> Result<(SectionConfigData, ConfigDigest), Error> {
+ let content =
+ proxmox_sys::fs::file_read_optional_string(VIEW_FILTER_CFG_FILENAME)?.unwrap_or_default();
+ let digest = ConfigDigest::from_slice(content.as_bytes());
+ let data = CONFIG.parse(VIEW_FILTER_CFG_FILENAME, &content)?;
+ Ok((data, digest))
+}
+
+// TODO: Will be needed once the CRUD API for view filters is implemented.
+// fn save_config(config: &SectionConfigData) -> Result<(), Error> {
+// let raw = CONFIG.write(VIEW_FILTER_CFG_FILENAME, config)?;
+// replace_privileged_config(VIEW_FILTER_CFG_FILENAME, raw.as_bytes())
+// }
+//
+
+/// Get the [`ViewFilterConfig`] entry for a view filter with a given ID.
+///
+/// This will fail if the config file does not exist or if the view filter does not exist.
+pub fn get_view_filter_config(filter_name: &str) -> Result<ViewFilterConfig, Error> {
+ let (cfg, _) = config()?;
+
+ cfg.lookup(VIEW_FILTER_SECTION_NAME, filter_name)
+}
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 04/13] acl: add '/view' and '/view/{view-id}' as allowed ACL paths
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (2 preceding siblings ...)
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 03/13] pdm-config: views: add support for view-filters Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 05/13] views: add implementation for view filters Lukas Wagner
` (9 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
These paths will be used for ACL objects for views. A view has filter
rules that specify which resources/remotes are included in the view. If
a user has permissions on the corresponding ACL object for the view,
then the privileges are transitively applied to the included resources
as well.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/acl.rs | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/server/src/acl.rs b/server/src/acl.rs
index 52a1f972..f5f57c03 100644
--- a/server/src/acl.rs
+++ b/server/src/acl.rs
@@ -150,6 +150,12 @@ impl proxmox_access_control::init::AccessControlConfig for AccessControlConfig {
_ => {}
}
}
+ "view" => {
+ // `/view` and `/view/{view-id}`
+ if components_len <= 2 {
+ return Ok(());
+ }
+ }
_ => {}
}
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 05/13] views: add implementation for view filters
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (3 preceding siblings ...)
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 04/13] acl: add '/view' and '/view/{view-id}' as allowed ACL paths Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-30 11:44 ` Shannon Sterz
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 06/13] views: add tests for view filter implementation Lukas Wagner
` (8 subsequent siblings)
13 siblings, 1 reply; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
This commit adds the filter implementation for the previously defined
ViewFilterConfig type.
There are include/exclude rules for the following properties:
- (global) resource-id
- resource pool
- resource type
- remote
- tags
The rules are interpreted as follows:
- no rules: everything matches
- only includes: included resources match
- only excluded: everything *but* the excluded resources match
- include and exclude: excludes are applied *after* includes, meaning if
one has a `include-remote foo` and `exclude-remote foo` at the same
time, the remote `foo` will never match
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/lib.rs | 1 +
server/src/views/mod.rs | 1 +
server/src/views/view_filter.rs | 225 ++++++++++++++++++++++++++++++++
3 files changed, 227 insertions(+)
create mode 100644 server/src/views/mod.rs
create mode 100644 server/src/views/view_filter.rs
diff --git a/server/src/lib.rs b/server/src/lib.rs
index 964807eb..0f25aa71 100644
--- a/server/src/lib.rs
+++ b/server/src/lib.rs
@@ -12,6 +12,7 @@ pub mod remote_tasks;
pub mod remote_updates;
pub mod resource_cache;
pub mod task_utils;
+pub mod views;
pub mod connection;
pub mod pbs_client;
diff --git a/server/src/views/mod.rs b/server/src/views/mod.rs
new file mode 100644
index 00000000..9a2856a4
--- /dev/null
+++ b/server/src/views/mod.rs
@@ -0,0 +1 @@
+pub mod view_filter;
diff --git a/server/src/views/view_filter.rs b/server/src/views/view_filter.rs
new file mode 100644
index 00000000..656b5523
--- /dev/null
+++ b/server/src/views/view_filter.rs
@@ -0,0 +1,225 @@
+use anyhow::Error;
+
+use pdm_api_types::{
+ resource::{Resource, ResourceType},
+ views::ViewFilterConfig,
+};
+
+/// Get view filter with a given ID.
+///
+/// Returns an error if the view filter configuration file could not be read, or
+/// if the view filter with the provided ID does not exist.
+pub fn get_view_filter(filter_id: &str) -> Result<ViewFilter, Error> {
+ pdm_config::views::get_view_filter_config(filter_id).map(ViewFilter::new)
+}
+
+/// View filter implementation.
+///
+/// Given a [`ViewFilterConfig`], this struct can be used to check if a resource/remote/node
+/// matches the filter rules.
+#[derive(Clone)]
+pub struct ViewFilter {
+ config: ViewFilterConfig,
+}
+
+impl ViewFilter {
+ /// Create a new [`ViewFiler`].
+ pub fn new(config: ViewFilterConfig) -> Self {
+ Self { config }
+ }
+
+ /// Check if a [`Resource`] matches the filter rules.
+ pub fn resource_matches(&self, remote: &str, resource: &Resource) -> bool {
+ // NOTE: Establishing a cache here is not worth the effort at the moment, evaluation of
+ // rules is *very* fast.
+ //
+ // Some experiments were performed with a cache that works roughly as following:
+ // - HashMap<ViewId, HashMap<ResourceId, bool>> in a mutex
+ // - Cache invalidated if view-filter config digest changed
+ // - Cache invalidated if certain resource fields such as tags or resource pools change
+ // from the last time (also with a digest-based implementation)
+ //
+ // Experimented with the `fake-remote` feature and and 15000 guests showed that
+ // caching was only faster than direct evaluation if the number of rules in the
+ // ViewFilterConfig is *huge* (e.g. >1000 `include-resource-id` entries). But even for those,
+ // direct evaluation was always plenty fast, with evaluation times ~20ms for *all* resources.
+ //
+ // -> for any *realistic* filter config, we should be good with direct evaluation, as long
+ // as we don't add any filter rules which are very expensive to evaluate.
+
+ let resource_data = resource.into();
+
+ self.check_if_included(remote, &resource_data)
+ && !self.check_if_excluded(remote, &resource_data)
+ }
+
+ /// Check if a remote can be safely skipped based on the filter rule definition.
+ ///
+ /// When there are `include-remote` or `exclude-remote` rules, we can use these to
+ /// check if a remote needs to be considered at all.
+ pub fn can_skip_remote(&self, remote: &str) -> bool {
+ let no_includes = self.config.include_remote.is_empty();
+ let any_include = self.config.include_remote.iter().any(|r| r == remote);
+ let any_exclude = self.config.exclude_remote.iter().any(|r| r == remote);
+
+ (!no_includes && !any_include) || any_exclude
+ }
+
+ /// Check if a node is matched by the filter rules.
+ ///
+ /// This is equivalent to checking an actual node resource.
+ pub fn is_node_included(&self, remote: &str, node: &str) -> bool {
+ let resource_data = ResourceData {
+ resource_type: ResourceType::Node,
+ tags: None,
+ resource_pool: None,
+ resource_id: &format!("remote/{remote}/node/{node}"),
+ };
+
+ self.check_if_included(remote, &resource_data)
+ && !self.check_if_excluded(remote, &resource_data)
+ }
+
+ /// Returns the name of the view filter.
+ pub fn name(&self) -> &str {
+ &self.config.id
+ }
+
+ fn check_if_included(&self, remote: &str, resource: &ResourceData) -> bool {
+ let rules = Rules {
+ ruleset_type: RulesetType::Include,
+ tags: &self.config.include_tag,
+ resource_ids: &self.config.include_resource_id,
+ resource_type: &self.config.include_resource_type,
+ resource_pools: &self.config.include_resource_pool,
+ remotes: &self.config.include_remote,
+ };
+
+ check_rules(rules, remote, resource)
+ }
+
+ fn check_if_excluded(&self, remote: &str, resource: &ResourceData) -> bool {
+ let rules = Rules {
+ ruleset_type: RulesetType::Exclude,
+ tags: &self.config.exclude_tag,
+ resource_ids: &self.config.exclude_resource_id,
+ resource_type: &self.config.exclude_resource_type,
+ resource_pools: &self.config.exclude_resource_pool,
+ remotes: &self.config.exclude_remote,
+ };
+
+ check_rules(rules, remote, resource)
+ }
+}
+
+enum RulesetType {
+ Include,
+ Exclude,
+}
+
+struct Rules<'a> {
+ ruleset_type: RulesetType,
+ tags: &'a [String],
+ resource_ids: &'a [String],
+ resource_pools: &'a [String],
+ resource_type: &'a [ResourceType],
+ remotes: &'a [String],
+}
+
+struct ResourceData<'a> {
+ resource_type: ResourceType,
+ tags: Option<&'a [String]>,
+ resource_pool: Option<&'a String>,
+ resource_id: &'a str,
+}
+
+impl<'a> From<&'a Resource> for ResourceData<'a> {
+ fn from(value: &'a Resource) -> Self {
+ match value {
+ Resource::PveStorage(_) => ResourceData {
+ resource_type: value.resource_type(),
+ tags: None,
+ resource_pool: None,
+ resource_id: value.global_id(),
+ },
+ Resource::PveQemu(resource) => ResourceData {
+ resource_type: value.resource_type(),
+ tags: Some(&resource.tags),
+ resource_pool: Some(&resource.pool),
+ resource_id: value.global_id(),
+ },
+ Resource::PveLxc(resource) => ResourceData {
+ resource_type: value.resource_type(),
+ tags: Some(&resource.tags),
+ resource_pool: Some(&resource.pool),
+ resource_id: value.global_id(),
+ },
+ Resource::PveNode(_) => ResourceData {
+ resource_type: value.resource_type(),
+ tags: None,
+ resource_pool: None,
+ resource_id: value.global_id(),
+ },
+ Resource::PveSdn(_) => ResourceData {
+ resource_type: value.resource_type(),
+ tags: None,
+ resource_pool: None,
+ resource_id: value.global_id(),
+ },
+ Resource::PbsNode(_) => ResourceData {
+ resource_type: value.resource_type(),
+ tags: None,
+ resource_pool: None,
+ resource_id: value.global_id(),
+ },
+ Resource::PbsDatastore(_) => ResourceData {
+ resource_type: value.resource_type(),
+ tags: None,
+ resource_pool: None,
+ resource_id: value.global_id(),
+ },
+ }
+ }
+}
+
+fn check_rules(rules: Rules, remote: &str, resource: &ResourceData) -> bool {
+ let has_any_rules = !rules.tags.is_empty()
+ || !rules.remotes.is_empty()
+ || !rules.resource_pools.is_empty()
+ || !rules.resource_type.is_empty()
+ || !rules.resource_ids.is_empty();
+
+ if !has_any_rules {
+ return matches!(rules.ruleset_type, RulesetType::Include);
+ }
+
+ if let Some(tags) = resource.tags {
+ if rules.tags.iter().any(|tag| tags.contains(tag)) {
+ return true;
+ }
+ }
+
+ if let Some(pool) = resource.resource_pool {
+ if rules.resource_pools.iter().any(|p| p == pool) {
+ return true;
+ }
+ }
+
+ if rules.remotes.iter().any(|r| r == remote) {
+ return true;
+ }
+
+ if rules.resource_ids.iter().any(|r| r == resource.resource_id) {
+ return true;
+ }
+
+ if rules
+ .resource_type
+ .iter()
+ .any(|ty| *ty == resource.resource_type)
+ {
+ return true;
+ }
+
+ false
+}
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 06/13] views: add tests for view filter implementation
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (4 preceding siblings ...)
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 05/13] views: add implementation for view filters Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 07/13] api: resources: list: add support for view-filter parameter Lukas Wagner
` (7 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/views/mod.rs | 3 +
server/src/views/tests.rs | 486 ++++++++++++++++++++++++++++++++++++++
2 files changed, 489 insertions(+)
create mode 100644 server/src/views/tests.rs
diff --git a/server/src/views/mod.rs b/server/src/views/mod.rs
index 9a2856a4..ea9e6de7 100644
--- a/server/src/views/mod.rs
+++ b/server/src/views/mod.rs
@@ -1 +1,4 @@
pub mod view_filter;
+
+#[cfg(test)]
+mod tests;
diff --git a/server/src/views/tests.rs b/server/src/views/tests.rs
new file mode 100644
index 00000000..e2608c27
--- /dev/null
+++ b/server/src/views/tests.rs
@@ -0,0 +1,486 @@
+use pdm_api_types::{
+ resource::{PveLxcResource, PveQemuResource, PveStorageResource, Resource, ResourceType},
+ views::ViewFilterConfig,
+};
+
+use super::view_filter::ViewFilter;
+
+fn make_storage_resource(remote: &str, node: &str, storage_name: &str) -> Resource {
+ Resource::PveStorage(PveStorageResource {
+ disk: 1000,
+ maxdisk: 2000,
+ id: format!("remote/{remote}/storage/{node}/{storage_name}"),
+ storage: storage_name.into(),
+ node: node.into(),
+ status: "available".into(),
+ })
+}
+
+fn make_qemu_resource(
+ remote: &str,
+ node: &str,
+ vmid: u32,
+ pool: Option<&str>,
+ tags: &[&str],
+) -> Resource {
+ Resource::PveQemu(PveQemuResource {
+ disk: 1000,
+ maxdisk: 2000,
+ id: format!("remote/{remote}/guest/{vmid}"),
+ node: node.into(),
+ status: "available".into(),
+ cpu: 0.0,
+ maxcpu: 0.0,
+ maxmem: 1024,
+ mem: 512,
+ name: format!("vm-{vmid}"),
+ // TODO: Check the API type - i guess it should be an option?
+ pool: pool.map_or_else(String::new, |a| a.into()),
+ tags: tags.iter().map(|tag| String::from(*tag)).collect(),
+ template: false,
+ uptime: 1337,
+ vmid,
+ })
+}
+
+fn make_lxc_resource(
+ remote: &str,
+ node: &str,
+ vmid: u32,
+ pool: Option<&str>,
+ tags: &[&str],
+) -> Resource {
+ Resource::PveLxc(PveLxcResource {
+ disk: 1000,
+ maxdisk: 2000,
+ id: format!("remote/{remote}/guest/{vmid}"),
+ node: node.into(),
+ status: "available".into(),
+ cpu: 0.0,
+ maxcpu: 0.0,
+ maxmem: 1024,
+ mem: 512,
+ name: format!("vm-{vmid}"),
+ // TODO: Check the API type - i guess it should be an option?
+ pool: pool.map_or_else(String::new, |a| a.into()),
+ tags: tags.iter().map(|tag| String::from(*tag)).collect(),
+ template: false,
+ uptime: 1337,
+ vmid,
+ })
+}
+
+fn run_test(config: ViewFilterConfig, tests: &[((&str, &Resource), bool)]) {
+ let filter = ViewFilter::new(config);
+
+ for ((remote_name, resource), expected) in tests {
+ eprintln!("remote: {remote_name}, resource: {resource:?}");
+ assert_eq!(filter.resource_matches(remote_name, resource), *expected);
+ }
+}
+
+const NODE: &str = "somenode";
+const STORAGE: &str = "somestorage";
+const REMOTE: &str = "someremote";
+
+#[test]
+fn include_remotes() {
+ let config = ViewFilterConfig {
+ id: "only-includes".into(),
+ include_remote: vec!["remote-a".into(), "remote-b".into()],
+ ..Default::default()
+ };
+ run_test(
+ config.clone(),
+ &[
+ (
+ (
+ "remote-a",
+ &make_storage_resource("remote-a", NODE, STORAGE),
+ ),
+ true,
+ ),
+ (
+ (
+ "remote-b",
+ &make_storage_resource("remote-b", NODE, STORAGE),
+ ),
+ true,
+ ),
+ (
+ (
+ "remote-c",
+ &make_storage_resource("remote-c", NODE, STORAGE),
+ ),
+ false,
+ ),
+ ],
+ );
+
+ let filter = ViewFilter::new(config);
+
+ assert!(!filter.can_skip_remote("remote-a"));
+ assert!(!filter.can_skip_remote("remote-b"));
+ assert!(filter.can_skip_remote("remote-c"));
+}
+
+#[test]
+fn exclude_remotes() {
+ let config = ViewFilterConfig {
+ id: "only-excludes".into(),
+ exclude_remote: vec!["remote-a".into(), "remote-b".into()],
+ ..Default::default()
+ };
+
+ run_test(
+ config.clone(),
+ &[
+ (
+ (
+ "remote-a",
+ &make_storage_resource("remote-a", NODE, STORAGE),
+ ),
+ false,
+ ),
+ (
+ (
+ "remote-b",
+ &make_storage_resource("remote-b", NODE, STORAGE),
+ ),
+ false,
+ ),
+ (
+ (
+ "remote-c",
+ &make_storage_resource("remote-c", NODE, STORAGE),
+ ),
+ true,
+ ),
+ ],
+ );
+
+ let filter = ViewFilter::new(config);
+
+ assert!(filter.can_skip_remote("remote-a"));
+ assert!(filter.can_skip_remote("remote-b"));
+ assert!(!filter.can_skip_remote("remote-c"));
+}
+
+#[test]
+fn include_exclude_remotes() {
+ let config = ViewFilterConfig {
+ id: "both".into(),
+ include_remote: vec!["remote-a".into(), "remote-b".into()],
+ exclude_remote: vec!["remote-b".into(), "remote-c".into()],
+ ..Default::default()
+ };
+ run_test(
+ config.clone(),
+ &[
+ (
+ (
+ "remote-a",
+ &make_storage_resource("remote-a", NODE, STORAGE),
+ ),
+ true,
+ ),
+ (
+ (
+ "remote-b",
+ &make_storage_resource("remote-b", NODE, STORAGE),
+ ),
+ false,
+ ),
+ (
+ (
+ "remote-c",
+ &make_storage_resource("remote-c", NODE, STORAGE),
+ ),
+ false,
+ ),
+ ],
+ );
+
+ let filter = ViewFilter::new(config);
+
+ assert!(!filter.can_skip_remote("remote-a"));
+ assert!(filter.can_skip_remote("remote-b"));
+ assert!(filter.can_skip_remote("remote-c"));
+ assert!(filter.can_skip_remote("remote-d"));
+}
+
+#[test]
+fn empty_config() {
+ let config = ViewFilterConfig {
+ id: "empty".into(),
+ ..Default::default()
+ };
+ run_test(
+ config.clone(),
+ &[
+ (
+ (
+ "remote-a",
+ &make_storage_resource("remote-a", NODE, STORAGE),
+ ),
+ true,
+ ),
+ (
+ (
+ "remote-b",
+ &make_storage_resource("remote-b", NODE, STORAGE),
+ ),
+ true,
+ ),
+ (
+ (
+ "remote-c",
+ &make_storage_resource("remote-c", NODE, STORAGE),
+ ),
+ true,
+ ),
+ (
+ (REMOTE, &make_qemu_resource(REMOTE, NODE, 100, None, &[])),
+ true,
+ ),
+ ],
+ );
+
+ let filter = ViewFilter::new(config);
+
+ assert!(!filter.can_skip_remote("remote-a"));
+ assert!(!filter.can_skip_remote("remote-b"));
+ assert!(!filter.can_skip_remote("remote-c"));
+}
+
+#[test]
+fn include_type() {
+ run_test(
+ ViewFilterConfig {
+ id: "include-resource-type".into(),
+ include_resource_type: vec![ResourceType::PveStorage, ResourceType::PveQemu],
+ ..Default::default()
+ },
+ &[
+ (
+ (REMOTE, &make_storage_resource(REMOTE, NODE, STORAGE)),
+ true,
+ ),
+ (
+ (REMOTE, &make_qemu_resource(REMOTE, NODE, 100, None, &[])),
+ true,
+ ),
+ (
+ (REMOTE, &make_lxc_resource(REMOTE, NODE, 101, None, &[])),
+ false,
+ ),
+ ],
+ );
+}
+
+#[test]
+fn exclude_type() {
+ run_test(
+ ViewFilterConfig {
+ id: "exclude-resource-type".into(),
+ exclude_resource_type: vec![ResourceType::PveStorage, ResourceType::PveQemu],
+ ..Default::default()
+ },
+ &[
+ (
+ (REMOTE, &make_storage_resource(REMOTE, NODE, STORAGE)),
+ false,
+ ),
+ (
+ (REMOTE, &make_qemu_resource(REMOTE, NODE, 100, None, &[])),
+ false,
+ ),
+ (
+ (REMOTE, &make_lxc_resource(REMOTE, NODE, 101, None, &[])),
+ true,
+ ),
+ ],
+ );
+}
+
+#[test]
+fn include_exclude_type() {
+ run_test(
+ ViewFilterConfig {
+ id: "exclude-resource-type".into(),
+ include_resource_type: vec![ResourceType::PveQemu],
+ exclude_resource_type: vec![ResourceType::PveStorage],
+ ..Default::default()
+ },
+ &[
+ (
+ (REMOTE, &make_storage_resource(REMOTE, NODE, STORAGE)),
+ false,
+ ),
+ (
+ (REMOTE, &make_qemu_resource(REMOTE, NODE, 100, None, &[])),
+ true,
+ ),
+ (
+ (REMOTE, &make_lxc_resource(REMOTE, NODE, 101, None, &[])),
+ false,
+ ),
+ ],
+ );
+}
+
+#[test]
+fn include_exclude_tags() {
+ run_test(
+ ViewFilterConfig {
+ id: "include-tags".into(),
+ include_tag: vec!["tag1".into(), "tag2".into()],
+ exclude_tag: vec!["tag3".into()],
+ ..Default::default()
+ },
+ &[
+ (
+ (REMOTE, &make_storage_resource(REMOTE, NODE, STORAGE)),
+ // only qemu/lxc can match tags for now
+ false,
+ ),
+ (
+ (
+ REMOTE,
+ &make_qemu_resource(REMOTE, NODE, 100, None, &["tag1", "tag3"]),
+ ),
+ // because tag3 is excluded
+ false,
+ ),
+ (
+ (
+ REMOTE,
+ &make_lxc_resource(REMOTE, NODE, 101, None, &["tag1"]),
+ ),
+ // matches since it's in the includes
+ true,
+ ),
+ (
+ (
+ REMOTE,
+ &make_lxc_resource(REMOTE, NODE, 102, None, &["tag4"]),
+ ),
+ // Not in includes, can never match
+ false,
+ ),
+ ],
+ );
+}
+
+#[test]
+fn include_exclude_resource_pool() {
+ run_test(
+ ViewFilterConfig {
+ id: "pools".into(),
+ include_resource_pool: vec!["pool1".into(), "pool2".into()],
+ exclude_resource_pool: vec!["pool2".into()],
+ ..Default::default()
+ },
+ &[
+ (
+ (REMOTE, &make_storage_resource(REMOTE, NODE, STORAGE)),
+ // only qemu/lxc can match pools for now
+ false,
+ ),
+ (
+ (
+ REMOTE,
+ &make_qemu_resource(REMOTE, NODE, 100, Some("pool2"), &[]),
+ ),
+ // because pool2 is excluded (takes precedence over includes)
+ false,
+ ),
+ (
+ (
+ REMOTE,
+ &make_lxc_resource(REMOTE, NODE, 101, Some("pool1"), &[]),
+ ),
+ // matches since it's in the includes
+ true,
+ ),
+ (
+ (
+ REMOTE,
+ &make_lxc_resource(REMOTE, NODE, 102, Some("pool4"), &[]),
+ ),
+ // Not in includes, can never match
+ false,
+ ),
+ ],
+ );
+}
+
+#[test]
+fn include_exclude_resource_id() {
+ run_test(
+ ViewFilterConfig {
+ id: "resource-id".into(),
+ include_resource_id: vec![
+ format!("remote/{REMOTE}/guest/100"),
+ format!("remote/{REMOTE}/storage/{NODE}/{STORAGE}"),
+ ],
+ exclude_resource_id: vec![
+ format!("remote/{REMOTE}/guest/101"),
+ format!("remote/otherremote/guest/101"),
+ format!("remote/{REMOTE}/storage/{NODE}/otherstorage"),
+ ],
+ ..Default::default()
+ },
+ &[
+ (
+ (REMOTE, &make_storage_resource(REMOTE, NODE, STORAGE)),
+ true,
+ ),
+ (
+ (REMOTE, &make_qemu_resource(REMOTE, NODE, 100, None, &[])),
+ true,
+ ),
+ (
+ (REMOTE, &make_lxc_resource(REMOTE, NODE, 101, None, &[])),
+ false,
+ ),
+ (
+ (REMOTE, &make_lxc_resource(REMOTE, NODE, 102, None, &[])),
+ false,
+ ),
+ (
+ (
+ "otherremote",
+ &make_lxc_resource("otherremote", NODE, 101, None, &[]),
+ ),
+ false,
+ ),
+ (
+ (
+ "yetanoterremote",
+ &make_lxc_resource("yetanotherremote", NODE, 104, None, &[]),
+ ),
+ false,
+ ),
+ ],
+ );
+}
+
+#[test]
+fn node_included() {
+ let filter = ViewFilter::new(ViewFilterConfig {
+ id: "both".into(),
+ include_remote: vec!["remote-a".into()],
+ exclude_remote: vec!["remote-b".into()],
+ include_resource_id: vec!["remote/someremote/node/test".into()],
+ ..Default::default()
+ });
+
+ assert!(filter.is_node_included("remote-a", "somenode"));
+ assert!(filter.is_node_included("remote-a", "somenode2"));
+ assert!(!filter.is_node_included("remote-b", "somenode"));
+ assert!(!filter.is_node_included("remote-b", "somenode2"));
+ assert!(filter.is_node_included("someremote", "test"));
+
+ assert_eq!(filter.name(), "both");
+}
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 07/13] api: resources: list: add support for view-filter parameter
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (5 preceding siblings ...)
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 06/13] views: add tests for view filter implementation Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-30 11:21 ` Wolfgang Bumiller
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 08/13] api: resources: top entities: " Lukas Wagner
` (6 subsequent siblings)
13 siblings, 1 reply; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
A view filter allows one to get filtered subset of all resources, based
on filter rules defined in a config file. View filters integrate with
the permission system - if a user has permissions on
/view/{view-filter-id}, then these privileges are transitively applied
to all resources which are matched by the rules. All other permission
checks are replaced if requesting data through a view filter.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/api/resources.rs | 48 ++++++++++++++++++++++++++++++------
server/src/resource_cache.rs | 3 ++-
2 files changed, 43 insertions(+), 8 deletions(-)
diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index dfadcbb1..c7f208aa 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -18,7 +18,7 @@ use pdm_api_types::resource::{
use pdm_api_types::subscription::{
NodeSubscriptionInfo, RemoteSubscriptionState, RemoteSubscriptions, SubscriptionLevel,
};
-use pdm_api_types::{Authid, PRIV_RESOURCE_AUDIT};
+use pdm_api_types::{Authid, PRIV_RESOURCE_AUDIT, VIEW_FILTER_ID_SCHEMA};
use pdm_search::{Search, SearchTerm};
use proxmox_access_control::CachedUserInfo;
use proxmox_router::{
@@ -30,8 +30,8 @@ use proxmox_sortable_macro::sortable;
use proxmox_subscription::SubscriptionStatus;
use pve_api_types::{ClusterResource, ClusterResourceType};
-use crate::connection;
use crate::metric_collection::top_entities;
+use crate::{connection, views};
pub const ROUTER: Router = Router::new()
.get(&list_subdirs_api_method!(SUBDIRS))
@@ -221,6 +221,10 @@ impl Into<RemoteResources> for RemoteWithResources {
type: ResourceType,
optional: true,
},
+ "view-filter": {
+ schema: VIEW_FILTER_ID_SCHEMA,
+ optional: true,
+ },
}
},
returns: {
@@ -236,10 +240,11 @@ pub async fn get_resources(
max_age: u64,
resource_type: Option<ResourceType>,
search: Option<String>,
+ view_filter: Option<String>,
rpcenv: &mut dyn RpcEnvironment,
) -> Result<Vec<RemoteResources>, Error> {
let remotes_with_resources =
- get_resources_impl(max_age, search, resource_type, Some(rpcenv)).await?;
+ get_resources_impl(max_age, search, resource_type, view_filter, Some(rpcenv)).await?;
let resources = remotes_with_resources.into_iter().map(Into::into).collect();
Ok(resources)
}
@@ -276,6 +281,7 @@ pub(crate) async fn get_resources_impl(
max_age: u64,
search: Option<String>,
resource_type: Option<ResourceType>,
+ view_filter: Option<String>,
rpcenv: Option<&mut dyn RpcEnvironment>,
) -> Result<Vec<RemoteWithResources>, Error> {
let user_info = CachedUserInfo::new()?;
@@ -285,9 +291,15 @@ pub(crate) async fn get_resources_impl(
.get_auth_id()
.ok_or_else(|| format_err!("no authid available"))?
.parse()?;
- if !user_info.any_privs_below(&auth_id, &["resource"], PRIV_RESOURCE_AUDIT)? {
+
+ // NOTE: Assumption is that the regular permission check is completely replaced by a check
+ // on the view ACL object *if* a view parameter is passed.
+ if let Some(view_filter) = &view_filter {
+ user_info.check_privs(&auth_id, &["view", view_filter], PRIV_RESOURCE_AUDIT, false)?;
+ } else if !user_info.any_privs_below(&auth_id, &["resource"], PRIV_RESOURCE_AUDIT)? {
http_bail!(UNAUTHORIZED, "user has no access to resources");
}
+
opt_auth_id = Some(auth_id);
}
@@ -296,12 +308,24 @@ pub(crate) async fn get_resources_impl(
let filters = search.map(Search::from).unwrap_or_default();
+ let view_filter = view_filter
+ .map(|filter_name| views::view_filter::get_view_filter(&filter_name))
+ .transpose()?;
+
let remotes_only = is_remotes_only(&filters);
for (remote_name, remote) in remotes_config {
if let Some(ref auth_id) = opt_auth_id {
- let remote_privs = user_info.lookup_privs(auth_id, &["resource", &remote_name]);
- if remote_privs & PRIV_RESOURCE_AUDIT == 0 {
+ if view_filter.is_none() {
+ let remote_privs = user_info.lookup_privs(auth_id, &["resource", &remote_name]);
+ if remote_privs & PRIV_RESOURCE_AUDIT == 0 {
+ continue;
+ }
+ }
+ }
+
+ if let Some(view_filter) = &view_filter {
+ if view_filter.can_skip_remote(&remote_name) {
continue;
}
}
@@ -374,6 +398,15 @@ pub(crate) async fn get_resources_impl(
}
}
+ if let Some(filter) = &view_filter {
+ remote_resources.retain_mut(|r| {
+ r.resources
+ .retain(|resource| filter.resource_matches(&r.remote_name, resource));
+
+ !r.resources.is_empty()
+ });
+ }
+
Ok(remote_resources)
}
@@ -405,7 +438,8 @@ pub async fn get_status(
max_age: u64,
rpcenv: &mut dyn RpcEnvironment,
) -> Result<ResourcesStatus, Error> {
- let remotes_with_resources = get_resources_impl(max_age, None, None, Some(rpcenv)).await?;
+ let remotes_with_resources =
+ get_resources_impl(max_age, None, None, None, Some(rpcenv)).await?;
let mut counts = ResourcesStatus::default();
for remote_with_resources in remotes_with_resources {
if let Some(err) = remote_with_resources.error {
diff --git a/server/src/resource_cache.rs b/server/src/resource_cache.rs
index aa20c54e..dc3cbeaf 100644
--- a/server/src/resource_cache.rs
+++ b/server/src/resource_cache.rs
@@ -21,7 +21,8 @@ pub fn start_task() {
async fn resource_caching_task() -> Result<(), Error> {
loop {
if let Err(err) =
- crate::api::resources::get_resources_impl(METRIC_POLL_INTERVALL, None, None, None).await
+ crate::api::resources::get_resources_impl(METRIC_POLL_INTERVALL, None, None, None, None)
+ .await
{
log::error!("could not update resource cache: {err}");
}
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 08/13] api: resources: top entities: add support for view-filter parameter
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (6 preceding siblings ...)
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 07/13] api: resources: list: add support for view-filter parameter Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 09/13] api: resources: status: " Lukas Wagner
` (5 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
A view filter allows one to get filtered subset of all resources, based
on filter rules defined in a config file. View filters integrate with
the permission system - if a user has permissions on
/view/{view-filter-id}, then these privileges are transitively applied
to all resources which are matched by the rules. All other permission
checks are replaced if requesting data through a view filter.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/api/resources.rs | 41 ++++++++++++++++++--
server/src/metric_collection/top_entities.rs | 5 +++
2 files changed, 42 insertions(+), 4 deletions(-)
diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index c7f208aa..8059596d 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -617,7 +617,11 @@ pub async fn get_subscription_status(
"timeframe": {
type: RrdTimeframe,
optional: true,
- }
+ },
+ "view-filter": {
+ schema: VIEW_FILTER_ID_SCHEMA,
+ optional: true,
+ },
}
},
access: {
@@ -630,6 +634,7 @@ pub async fn get_subscription_status(
/// Returns the top X entities regarding the chosen type
async fn get_top_entities(
timeframe: Option<RrdTimeframe>,
+ view_filter: Option<String>,
rpcenv: &mut dyn RpcEnvironment,
) -> Result<TopEntities, Error> {
let user_info = CachedUserInfo::new()?;
@@ -638,17 +643,45 @@ async fn get_top_entities(
.ok_or_else(|| format_err!("no authid available"))?
.parse()?;
- if !user_info.any_privs_below(&auth_id, &["resource"], PRIV_RESOURCE_AUDIT)? {
+ if let Some(view_filter) = &view_filter {
+ user_info.check_privs(&auth_id, &["view", view_filter], PRIV_RESOURCE_AUDIT, false)?;
+ } else if !user_info.any_privs_below(&auth_id, &["resource"], PRIV_RESOURCE_AUDIT)? {
http_bail!(FORBIDDEN, "user has no access to resources");
}
+ let view_filter = view_filter
+ .map(|a| views::view_filter::get_view_filter(&a))
+ .transpose()?;
+
let (remotes_config, _) = pdm_config::remotes::config()?;
+
let check_remote_privs = |remote_name: &str| {
- user_info.lookup_privs(&auth_id, &["resource", remote_name]) & PRIV_RESOURCE_AUDIT != 0
+ if let Some(view_filter) = &view_filter {
+ // if `include-remote` or `exclude-remote` are used we can limit the
+ // number of remotes to check.
+ !view_filter.can_skip_remote(remote_name)
+ } else {
+ user_info.lookup_privs(&auth_id, &["resource", remote_name]) & PRIV_RESOURCE_AUDIT != 0
+ }
+ };
+
+ let is_resource_included = |remote: &str, resource: &Resource| {
+ if let Some(view_filter) = &view_filter {
+ view_filter.resource_matches(remote, resource)
+ } else {
+ true
+ }
};
let timeframe = timeframe.unwrap_or(RrdTimeframe::Day);
- let res = top_entities::calculate_top(&remotes_config, timeframe, 10, check_remote_privs);
+ let res = top_entities::calculate_top(
+ &remotes_config,
+ timeframe,
+ 10,
+ check_remote_privs,
+ is_resource_included,
+ );
+
Ok(res)
}
diff --git a/server/src/metric_collection/top_entities.rs b/server/src/metric_collection/top_entities.rs
index 73a3e63f..a91d586c 100644
--- a/server/src/metric_collection/top_entities.rs
+++ b/server/src/metric_collection/top_entities.rs
@@ -37,6 +37,7 @@ pub fn calculate_top(
timeframe: proxmox_rrd_api_types::RrdTimeframe,
num: usize,
check_remote_privs: impl Fn(&str) -> bool,
+ is_resource_included: impl Fn(&str, &Resource) -> bool,
) -> TopEntities {
let mut guest_cpu = Vec::new();
let mut node_cpu = Vec::new();
@@ -51,6 +52,10 @@ pub fn calculate_top(
crate::api::resources::get_cached_resources(remote_name, i64::MAX as u64)
{
for res in data.resources {
+ if !is_resource_included(remote_name, &res) {
+ continue;
+ }
+
let id = res.id().to_string();
let name = format!("pve/{remote_name}/{id}");
match &res {
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 09/13] api: resources: status: add support for view-filter parameter
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (7 preceding siblings ...)
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 08/13] api: resources: top entities: " Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 10/13] api: subscription " Lukas Wagner
` (4 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
A view filter allows one to get filtered subset of all resources, based
on filter rules defined in a config file. View filters integrate with
the permission system - if a user has permissions on
/view/{view-filter-id}, then these privileges are transitively applied
to all resources which are matched by the rules. All other permission
checks are replaced if requesting data through a view filter.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/api/resources.rs | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index 8059596d..61960295 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -423,6 +423,10 @@ pub(crate) async fn get_resources_impl(
default: 30,
optional: true,
},
+ "view-filter": {
+ schema: VIEW_FILTER_ID_SCHEMA,
+ optional: true,
+ },
}
},
returns: {
@@ -436,10 +440,11 @@ pub(crate) async fn get_resources_impl(
/// Return the amount of configured/seen resources by type
pub async fn get_status(
max_age: u64,
+ view_filter: Option<String>,
rpcenv: &mut dyn RpcEnvironment,
) -> Result<ResourcesStatus, Error> {
let remotes_with_resources =
- get_resources_impl(max_age, None, None, None, Some(rpcenv)).await?;
+ get_resources_impl(max_age, None, None, view_filter, Some(rpcenv)).await?;
let mut counts = ResourcesStatus::default();
for remote_with_resources in remotes_with_resources {
if let Some(err) = remote_with_resources.error {
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 10/13] api: subscription status: add support for view-filter parameter
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (8 preceding siblings ...)
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 09/13] api: resources: status: " Lukas Wagner
@ 2025-10-29 14:48 ` Lukas Wagner
2025-10-30 11:31 ` Wolfgang Bumiller
2025-10-30 11:44 ` Shannon Sterz
2025-10-29 14:49 ` [pdm-devel] [PATCH datacenter-manager 11/13] api: remote-tasks: " Lukas Wagner
` (3 subsequent siblings)
13 siblings, 2 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:48 UTC (permalink / raw)
To: pdm-devel
A view filter allows one to get filtered subset of all resources, based
on filter rules defined in a config file. View filters integrate with
the permission system - if a user has permissions on
/view/{view-filter-id}, then these privileges are transitively applied
to all resources which are matched by the rules. All other permission
checks are replaced if requesting data through a view filter.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/api/resources.rs | 58 +++++++++++++++++++++++++++++++------
1 file changed, 49 insertions(+), 9 deletions(-)
diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
index 61960295..89b84c3c 100644
--- a/server/src/api/resources.rs
+++ b/server/src/api/resources.rs
@@ -546,6 +546,10 @@ pub async fn get_status(
default: false,
description: "If true, includes subscription information per node (with enough privileges)",
},
+ "view-filter": {
+ schema: VIEW_FILTER_ID_SCHEMA,
+ optional: true,
+ },
},
},
returns: {
@@ -560,6 +564,7 @@ pub async fn get_status(
pub async fn get_subscription_status(
max_age: u64,
verbose: bool,
+ view_filter: Option<String>,
rpcenv: &mut dyn RpcEnvironment,
) -> Result<Vec<RemoteSubscriptions>, Error> {
let (remotes_config, _) = pdm_config::remotes::config()?;
@@ -572,6 +577,14 @@ pub async fn get_subscription_status(
.check_privs(&auth_id, &["resources"], PRIV_RESOURCE_AUDIT, false)
.is_ok();
+ if let Some(view_filter) = &view_filter {
+ user_info.check_privs(&auth_id, &["view", view_filter], PRIV_RESOURCE_AUDIT, false)?;
+ }
+
+ let view_filter = view_filter
+ .map(|filter_name| views::view_filter::get_view_filter(&filter_name))
+ .transpose()?;
+
let check_priv = |remote_name: &str| -> bool {
user_info
.check_privs(
@@ -584,35 +597,62 @@ pub async fn get_subscription_status(
};
for (remote_name, remote) in remotes_config {
- if !allow_all && !check_priv(&remote_name) {
+ if let Some(filter) = &view_filter {
+ if filter.can_skip_remote(&remote_name) {
+ continue;
+ }
+ } else if !allow_all && !check_priv(&remote_name) {
continue;
}
+ let view_filter_clone = view_filter.clone();
+
let future = async move {
let (node_status, error) =
match get_subscription_info_for_remote(&remote, max_age).await {
- Ok(node_status) => (Some(node_status), None),
+ Ok(mut node_status) => {
+ node_status.retain(|node, _| {
+ if let Some(filter) = &view_filter_clone {
+ filter.is_node_included(&remote.id, node)
+ } else {
+ true
+ }
+ });
+ (Some(node_status), None)
+ }
Err(error) => (None, Some(error.to_string())),
};
- let mut state = RemoteSubscriptionState::Unknown;
+ let state = if let Some(node_status) = &node_status {
+ if error.is_some() {
+ // Don't leak the existence of failed remotes, since we cannot apply
+ // view-filters here.
+ return None;
+ }
- if let Some(node_status) = &node_status {
- state = map_node_subscription_list_to_state(node_status);
- }
+ if node_status.is_empty() {
+ return None;
+ }
- RemoteSubscriptions {
+ map_node_subscription_list_to_state(node_status)
+ } else {
+ RemoteSubscriptionState::Unknown
+ };
+
+ Some(RemoteSubscriptions {
remote: remote_name,
error,
state,
node_status: if verbose { node_status } else { None },
- }
+ })
};
futures.push(future);
}
- Ok(join_all(futures).await)
+ let status = join_all(futures).await.into_iter().flatten().collect();
+
+ Ok(status)
}
// FIXME: make timeframe and count parameters?
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 11/13] api: remote-tasks: add support for view-filter parameter
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (9 preceding siblings ...)
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 10/13] api: subscription " Lukas Wagner
@ 2025-10-29 14:49 ` Lukas Wagner
2025-10-29 14:49 ` [pdm-devel] [PATCH datacenter-manager 12/13] pdm-client: resource list: add " Lukas Wagner
` (2 subsequent siblings)
13 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:49 UTC (permalink / raw)
To: pdm-devel
A view filter allows one to get filtered subset of all resources, based
on filter rules defined in a config file. View filters integrate with
the permission system - if a user has permissions on
/view/{view-filter-id}, then these privileges are transitively applied
to all resources which are matched by the rules. All other permission
checks are replaced if requesting data through a view filter.
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
server/src/api/remote_tasks.rs | 36 +++++++++++++++++++++++++++----
server/src/remote_tasks/mod.rs | 39 +++++++++++++++++++++++-----------
2 files changed, 59 insertions(+), 16 deletions(-)
diff --git a/server/src/api/remote_tasks.rs b/server/src/api/remote_tasks.rs
index 7b97b9cd..4a68c5d9 100644
--- a/server/src/api/remote_tasks.rs
+++ b/server/src/api/remote_tasks.rs
@@ -4,9 +4,10 @@ use anyhow::Error;
use pdm_api_types::{
remotes::REMOTE_ID_SCHEMA, RemoteUpid, TaskCount, TaskFilters, TaskListItem, TaskStateType,
- TaskStatistics,
+ TaskStatistics, PRIV_RESOURCE_AUDIT, VIEW_FILTER_ID_SCHEMA,
};
-use proxmox_router::{list_subdirs_api_method, Permission, Router, SubdirMap};
+use proxmox_access_control::CachedUserInfo;
+use proxmox_router::{list_subdirs_api_method, Permission, Router, RpcEnvironment, SubdirMap};
use proxmox_schema::api;
use proxmox_sortable_macro::sortable;
@@ -41,6 +42,11 @@ const SUBDIRS: SubdirMap = &sorted!([
schema: REMOTE_ID_SCHEMA,
optional: true,
},
+ "view-filter": {
+ schema: VIEW_FILTER_ID_SCHEMA,
+ optional: true,
+ },
+
},
},
)]
@@ -48,8 +54,17 @@ const SUBDIRS: SubdirMap = &sorted!([
async fn list_tasks(
filters: TaskFilters,
remote: Option<String>,
+ view_filter: Option<String>,
+ rpcenv: &mut dyn RpcEnvironment,
) -> Result<Vec<TaskListItem>, Error> {
- let tasks = remote_tasks::get_tasks(filters, remote).await?;
+ let auth_id = rpcenv.get_auth_id().unwrap().parse()?;
+ let user_info = CachedUserInfo::new()?;
+
+ if let Some(view_filter) = &view_filter {
+ user_info.check_privs(&auth_id, &["view", view_filter], PRIV_RESOURCE_AUDIT, false)?;
+ }
+
+ let tasks = remote_tasks::get_tasks(filters, remote, view_filter).await?;
Ok(tasks)
}
@@ -70,6 +85,10 @@ async fn list_tasks(
schema: REMOTE_ID_SCHEMA,
optional: true,
},
+ "view-filter": {
+ schema: VIEW_FILTER_ID_SCHEMA,
+ optional: true,
+ },
},
},
)]
@@ -77,8 +96,17 @@ async fn list_tasks(
async fn task_statistics(
filters: TaskFilters,
remote: Option<String>,
+ view_filter: Option<String>,
+ rpcenv: &mut dyn RpcEnvironment,
) -> Result<TaskStatistics, Error> {
- let tasks = remote_tasks::get_tasks(filters, remote).await?;
+ let auth_id = rpcenv.get_auth_id().unwrap().parse()?;
+ let user_info = CachedUserInfo::new()?;
+
+ if let Some(view_filter) = &view_filter {
+ user_info.check_privs(&auth_id, &["view", view_filter], PRIV_RESOURCE_AUDIT, false)?;
+ }
+
+ let tasks = remote_tasks::get_tasks(filters, remote, view_filter).await?;
let mut by_type: HashMap<String, TaskCount> = HashMap::new();
let mut by_remote: HashMap<String, TaskCount> = HashMap::new();
diff --git a/server/src/remote_tasks/mod.rs b/server/src/remote_tasks/mod.rs
index c4939742..4d93d9b8 100644
--- a/server/src/remote_tasks/mod.rs
+++ b/server/src/remote_tasks/mod.rs
@@ -9,6 +9,8 @@ pub mod task_cache;
use task_cache::{GetTasks, TaskCache, TaskCacheItem};
+use crate::views;
+
/// Base directory for the remote task cache.
pub const REMOTE_TASKS_DIR: &str = concat!(pdm_buildcfg::PDM_CACHE_DIR_M!(), "/remote-tasks");
@@ -29,7 +31,12 @@ const NUMBER_OF_UNCOMPRESSED_FILES: u32 = 2;
pub async fn get_tasks(
filters: TaskFilters,
remote_filter: Option<String>,
+ view_filter: Option<String>,
) -> Result<Vec<TaskListItem>, Error> {
+ let view_filter = view_filter
+ .map(|filter_name| views::view_filter::get_view_filter(&filter_name))
+ .transpose()?;
+
tokio::task::spawn_blocking(move || {
let cache = get_cache()?.read()?;
@@ -52,21 +59,29 @@ pub async fn get_tasks(
return None;
}
}
+
// TODO: Handle PBS tasks
let pve_upid: Result<PveUpid, Error> = task.upid.upid.parse();
match pve_upid {
- Ok(pve_upid) => Some(TaskListItem {
- upid: task.upid.to_string(),
- node: pve_upid.node,
- pid: pve_upid.pid as i64,
- pstart: pve_upid.pstart,
- starttime: pve_upid.starttime,
- worker_type: pve_upid.worker_type,
- worker_id: None,
- user: pve_upid.auth_id,
- endtime: task.endtime,
- status: task.status,
- }),
+ Ok(pve_upid) => {
+ if let Some(view_filter) = &view_filter {
+ if !view_filter.is_node_included(task.upid.remote(), &pve_upid.node) {
+ return None;
+ }
+ }
+ Some(TaskListItem {
+ upid: task.upid.to_string(),
+ node: pve_upid.node,
+ pid: pve_upid.pid as i64,
+ pstart: pve_upid.pstart,
+ starttime: pve_upid.starttime,
+ worker_type: pve_upid.worker_type,
+ worker_id: None,
+ user: pve_upid.auth_id,
+ endtime: task.endtime,
+ status: task.status,
+ })
+ }
Err(err) => {
log::error!("could not parse UPID: {err:#}");
None
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 12/13] pdm-client: resource list: add view-filter parameter
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (10 preceding siblings ...)
2025-10-29 14:49 ` [pdm-devel] [PATCH datacenter-manager 11/13] api: remote-tasks: " Lukas Wagner
@ 2025-10-29 14:49 ` Lukas Wagner
2025-10-29 14:49 ` [pdm-devel] [PATCH datacenter-manager 13/13] pdm-client: top entities: " Lukas Wagner
2025-10-30 11:41 ` [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Wolfgang Bumiller
13 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:49 UTC (permalink / raw)
To: pdm-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
cli/client/src/resources.rs | 2 +-
lib/pdm-client/src/lib.rs | 9 ++++++++-
ui/src/sdn/zone_tree.rs | 2 +-
3 files changed, 10 insertions(+), 3 deletions(-)
diff --git a/cli/client/src/resources.rs b/cli/client/src/resources.rs
index dbf9f265..bc3a98ba 100644
--- a/cli/client/src/resources.rs
+++ b/cli/client/src/resources.rs
@@ -31,7 +31,7 @@ pub fn cli() -> CommandLineInterface {
)]
/// List all the remotes this instance is managing.
async fn get_resources(max_age: Option<u64>) -> Result<(), Error> {
- let mut resources = client()?.resources(max_age).await?;
+ let mut resources = client()?.resources(max_age, None).await?;
let output_format = env().format_args.output_format;
if output_format == OutputFormat::Text {
if resources.is_empty() {
diff --git a/lib/pdm-client/src/lib.rs b/lib/pdm-client/src/lib.rs
index 0cab7691..7102ee05 100644
--- a/lib/pdm-client/src/lib.rs
+++ b/lib/pdm-client/src/lib.rs
@@ -863,9 +863,14 @@ impl<T: HttpApiClient> PdmClient<T> {
Ok(self.0.get(&path).await?.expect_json()?.data)
}
- pub async fn resources(&self, max_age: Option<u64>) -> Result<Vec<RemoteResources>, Error> {
+ pub async fn resources(
+ &self,
+ max_age: Option<u64>,
+ view_filter: Option<&str>,
+ ) -> Result<Vec<RemoteResources>, Error> {
let path = ApiPathBuilder::new("/api2/extjs/resources/list")
.maybe_arg("max-age", &max_age)
+ .maybe_arg("view-filter", &view_filter)
.build();
Ok(self.0.get(&path).await?.expect_json()?.data)
}
@@ -874,10 +879,12 @@ impl<T: HttpApiClient> PdmClient<T> {
&self,
max_age: Option<u64>,
resource_type: ResourceType,
+ view_filter: Option<&str>,
) -> Result<Vec<RemoteResources>, Error> {
let path = ApiPathBuilder::new("/api2/extjs/resources/list")
.maybe_arg("max-age", &max_age)
.arg("resource-type", resource_type)
+ .maybe_arg("view-filter", &view_filter)
.build();
Ok(self.0.get(&path).await?.expect_json()?.data)
diff --git a/ui/src/sdn/zone_tree.rs b/ui/src/sdn/zone_tree.rs
index 2f907641..9b688769 100644
--- a/ui/src/sdn/zone_tree.rs
+++ b/ui/src/sdn/zone_tree.rs
@@ -281,7 +281,7 @@ impl LoadableComponent for ZoneTreeComponent {
Box::pin(async move {
let client = pdm_client();
let remote_resources = client
- .resources_by_type(None, ResourceType::PveSdnZone)
+ .resources_by_type(None, ResourceType::PveSdnZone, None)
.await?;
link.send_message(Self::Message::LoadFinished(remote_resources));
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] [PATCH datacenter-manager 13/13] pdm-client: top entities: add view-filter parameter
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (11 preceding siblings ...)
2025-10-29 14:49 ` [pdm-devel] [PATCH datacenter-manager 12/13] pdm-client: resource list: add " Lukas Wagner
@ 2025-10-29 14:49 ` Lukas Wagner
2025-10-30 11:41 ` [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Wolfgang Bumiller
13 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-29 14:49 UTC (permalink / raw)
To: pdm-devel
Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
---
lib/pdm-client/src/lib.rs | 10 +++++++---
ui/src/dashboard/mod.rs | 2 +-
2 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/lib/pdm-client/src/lib.rs b/lib/pdm-client/src/lib.rs
index 7102ee05..c9ed18cc 100644
--- a/lib/pdm-client/src/lib.rs
+++ b/lib/pdm-client/src/lib.rs
@@ -956,9 +956,13 @@ impl<T: HttpApiClient> PdmClient<T> {
Ok(self.0.get(&path).await?.expect_json()?.data)
}
- pub async fn get_top_entities(&self) -> Result<TopEntities, Error> {
- let path = "/api2/extjs/resources/top-entities";
- Ok(self.0.get(path).await?.expect_json()?.data)
+ pub async fn get_top_entities(&self, view_filter: Option<&str>) -> Result<TopEntities, Error> {
+ let builder = ApiPathBuilder::new("/api2/extjs/resources/top-entities".to_string())
+ .maybe_arg("view-filter", &view_filter);
+
+ let path = builder.build();
+
+ Ok(self.0.get(&path).await?.expect_json()?.data)
}
pub async fn pve_node_status(&self, remote: &str, node: &str) -> Result<NodeStatus, Error> {
diff --git a/ui/src/dashboard/mod.rs b/ui/src/dashboard/mod.rs
index 07d5cd99..ce64d847 100644
--- a/ui/src/dashboard/mod.rs
+++ b/ui/src/dashboard/mod.rs
@@ -319,7 +319,7 @@ impl PdmDashboard {
let top_entities_future = {
let link = link.clone();
async move {
- let res = client.get_top_entities().await;
+ let res = client.get_top_entities(None).await;
link.send_message(Msg::LoadingFinished(LoadingResult::TopEntities(res)));
}
};
--
2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* [pdm-devel] applied: [PATCH datacenter-manager 01/13] fake remote: add missing parameter for cluster_metrics_export function
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 01/13] fake remote: add missing parameter for cluster_metrics_export function Lukas Wagner
@ 2025-10-30 10:26 ` Wolfgang Bumiller
0 siblings, 0 replies; 21+ messages in thread
From: Wolfgang Bumiller @ 2025-10-30 10:26 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pdm-devel
applied this one early, as it is just a build fix, basically
On Wed, Oct 29, 2025 at 03:48:50PM +0100, Lukas Wagner wrote:
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> server/src/test_support/fake_remote.rs | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/server/src/test_support/fake_remote.rs b/server/src/test_support/fake_remote.rs
> index f4f6967e..cd2ccf60 100644
> --- a/server/src/test_support/fake_remote.rs
> +++ b/server/src/test_support/fake_remote.rs
> @@ -276,6 +276,7 @@ impl pve_api_types::client::PveClient for FakePveClient {
> &self,
> _history: Option<bool>,
> _local_only: Option<bool>,
> + _node_list: Option<String>,
> start_time: Option<i64>,
> ) -> Result<ClusterMetrics, proxmox_client::Error> {
> tokio::time::sleep(Duration::from_millis(self.api_delay_ms as u64)).await;
> --
> 2.47.3
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [pdm-devel] [PATCH datacenter-manager 07/13] api: resources: list: add support for view-filter parameter
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 07/13] api: resources: list: add support for view-filter parameter Lukas Wagner
@ 2025-10-30 11:21 ` Wolfgang Bumiller
0 siblings, 0 replies; 21+ messages in thread
From: Wolfgang Bumiller @ 2025-10-30 11:21 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pdm-devel
tiny nit inline
On Wed, Oct 29, 2025 at 03:48:56PM +0100, Lukas Wagner wrote:
> A view filter allows one to get filtered subset of all resources, based
> on filter rules defined in a config file. View filters integrate with
> the permission system - if a user has permissions on
> /view/{view-filter-id}, then these privileges are transitively applied
> to all resources which are matched by the rules. All other permission
> checks are replaced if requesting data through a view filter.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> server/src/api/resources.rs | 48 ++++++++++++++++++++++++++++++------
> server/src/resource_cache.rs | 3 ++-
> 2 files changed, 43 insertions(+), 8 deletions(-)
>
> diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
> index dfadcbb1..c7f208aa 100644
> --- a/server/src/api/resources.rs
> +++ b/server/src/api/resources.rs
> @@ -18,7 +18,7 @@ use pdm_api_types::resource::{
> use pdm_api_types::subscription::{
> NodeSubscriptionInfo, RemoteSubscriptionState, RemoteSubscriptions, SubscriptionLevel,
> };
> -use pdm_api_types::{Authid, PRIV_RESOURCE_AUDIT};
> +use pdm_api_types::{Authid, PRIV_RESOURCE_AUDIT, VIEW_FILTER_ID_SCHEMA};
> use pdm_search::{Search, SearchTerm};
> use proxmox_access_control::CachedUserInfo;
> use proxmox_router::{
> @@ -30,8 +30,8 @@ use proxmox_sortable_macro::sortable;
> use proxmox_subscription::SubscriptionStatus;
> use pve_api_types::{ClusterResource, ClusterResourceType};
>
> -use crate::connection;
> use crate::metric_collection::top_entities;
> +use crate::{connection, views};
>
> pub const ROUTER: Router = Router::new()
> .get(&list_subdirs_api_method!(SUBDIRS))
> @@ -221,6 +221,10 @@ impl Into<RemoteResources> for RemoteWithResources {
> type: ResourceType,
> optional: true,
> },
> + "view-filter": {
> + schema: VIEW_FILTER_ID_SCHEMA,
> + optional: true,
> + },
> }
> },
> returns: {
> @@ -236,10 +240,11 @@ pub async fn get_resources(
> max_age: u64,
> resource_type: Option<ResourceType>,
> search: Option<String>,
> + view_filter: Option<String>,
> rpcenv: &mut dyn RpcEnvironment,
> ) -> Result<Vec<RemoteResources>, Error> {
> let remotes_with_resources =
> - get_resources_impl(max_age, search, resource_type, Some(rpcenv)).await?;
> + get_resources_impl(max_age, search, resource_type, view_filter, Some(rpcenv)).await?;
> let resources = remotes_with_resources.into_iter().map(Into::into).collect();
> Ok(resources)
> }
> @@ -276,6 +281,7 @@ pub(crate) async fn get_resources_impl(
> max_age: u64,
> search: Option<String>,
> resource_type: Option<ResourceType>,
> + view_filter: Option<String>,
`search` is a `String` to avoid the clone when converting it to a
`Search`, but `view_filter` can just be `&str`, which IMO makes more
sense from an api perspective.
The callers would have to use `.as_deref()`.
> rpcenv: Option<&mut dyn RpcEnvironment>,
> ) -> Result<Vec<RemoteWithResources>, Error> {
> let user_info = CachedUserInfo::new()?;
> @@ -285,9 +291,15 @@ pub(crate) async fn get_resources_impl(
> .get_auth_id()
> .ok_or_else(|| format_err!("no authid available"))?
> .parse()?;
> - if !user_info.any_privs_below(&auth_id, &["resource"], PRIV_RESOURCE_AUDIT)? {
> +
> + // NOTE: Assumption is that the regular permission check is completely replaced by a check
> + // on the view ACL object *if* a view parameter is passed.
> + if let Some(view_filter) = &view_filter {
> + user_info.check_privs(&auth_id, &["view", view_filter], PRIV_RESOURCE_AUDIT, false)?;
> + } else if !user_info.any_privs_below(&auth_id, &["resource"], PRIV_RESOURCE_AUDIT)? {
> http_bail!(UNAUTHORIZED, "user has no access to resources");
> }
> +
> opt_auth_id = Some(auth_id);
> }
>
> @@ -296,12 +308,24 @@ pub(crate) async fn get_resources_impl(
>
> let filters = search.map(Search::from).unwrap_or_default();
>
> + let view_filter = view_filter
> + .map(|filter_name| views::view_filter::get_view_filter(&filter_name))
> + .transpose()?;
> +
> let remotes_only = is_remotes_only(&filters);
>
> for (remote_name, remote) in remotes_config {
> if let Some(ref auth_id) = opt_auth_id {
> - let remote_privs = user_info.lookup_privs(auth_id, &["resource", &remote_name]);
> - if remote_privs & PRIV_RESOURCE_AUDIT == 0 {
> + if view_filter.is_none() {
> + let remote_privs = user_info.lookup_privs(auth_id, &["resource", &remote_name]);
> + if remote_privs & PRIV_RESOURCE_AUDIT == 0 {
> + continue;
> + }
> + }
> + }
> +
> + if let Some(view_filter) = &view_filter {
> + if view_filter.can_skip_remote(&remote_name) {
> continue;
> }
> }
> @@ -374,6 +398,15 @@ pub(crate) async fn get_resources_impl(
> }
> }
>
> + if let Some(filter) = &view_filter {
> + remote_resources.retain_mut(|r| {
> + r.resources
> + .retain(|resource| filter.resource_matches(&r.remote_name, resource));
> +
> + !r.resources.is_empty()
> + });
> + }
> +
> Ok(remote_resources)
> }
>
> @@ -405,7 +438,8 @@ pub async fn get_status(
> max_age: u64,
> rpcenv: &mut dyn RpcEnvironment,
> ) -> Result<ResourcesStatus, Error> {
> - let remotes_with_resources = get_resources_impl(max_age, None, None, Some(rpcenv)).await?;
> + let remotes_with_resources =
> + get_resources_impl(max_age, None, None, None, Some(rpcenv)).await?;
> let mut counts = ResourcesStatus::default();
> for remote_with_resources in remotes_with_resources {
> if let Some(err) = remote_with_resources.error {
> diff --git a/server/src/resource_cache.rs b/server/src/resource_cache.rs
> index aa20c54e..dc3cbeaf 100644
> --- a/server/src/resource_cache.rs
> +++ b/server/src/resource_cache.rs
> @@ -21,7 +21,8 @@ pub fn start_task() {
> async fn resource_caching_task() -> Result<(), Error> {
> loop {
> if let Err(err) =
> - crate::api::resources::get_resources_impl(METRIC_POLL_INTERVALL, None, None, None).await
> + crate::api::resources::get_resources_impl(METRIC_POLL_INTERVALL, None, None, None, None)
> + .await
> {
> log::error!("could not update resource cache: {err}");
> }
> --
> 2.47.3
>
>
>
> _______________________________________________
> pdm-devel mailing list
> pdm-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
>
>
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [pdm-devel] [PATCH datacenter-manager 10/13] api: subscription status: add support for view-filter parameter
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 10/13] api: subscription " Lukas Wagner
@ 2025-10-30 11:31 ` Wolfgang Bumiller
2025-10-30 11:44 ` Shannon Sterz
1 sibling, 0 replies; 21+ messages in thread
From: Wolfgang Bumiller @ 2025-10-30 11:31 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pdm-devel
On Wed, Oct 29, 2025 at 03:48:59PM +0100, Lukas Wagner wrote:
> A view filter allows one to get filtered subset of all resources, based
> on filter rules defined in a config file. View filters integrate with
> the permission system - if a user has permissions on
> /view/{view-filter-id}, then these privileges are transitively applied
> to all resources which are matched by the rules. All other permission
> checks are replaced if requesting data through a view filter.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> server/src/api/resources.rs | 58 +++++++++++++++++++++++++++++++------
> 1 file changed, 49 insertions(+), 9 deletions(-)
>
> diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
> index 61960295..89b84c3c 100644
> --- a/server/src/api/resources.rs
> +++ b/server/src/api/resources.rs
> @@ -546,6 +546,10 @@ pub async fn get_status(
> default: false,
> description: "If true, includes subscription information per node (with enough privileges)",
> },
> + "view-filter": {
> + schema: VIEW_FILTER_ID_SCHEMA,
> + optional: true,
> + },
> },
> },
> returns: {
> @@ -560,6 +564,7 @@ pub async fn get_status(
> pub async fn get_subscription_status(
> max_age: u64,
> verbose: bool,
> + view_filter: Option<String>,
> rpcenv: &mut dyn RpcEnvironment,
> ) -> Result<Vec<RemoteSubscriptions>, Error> {
> let (remotes_config, _) = pdm_config::remotes::config()?;
> @@ -572,6 +577,14 @@ pub async fn get_subscription_status(
> .check_privs(&auth_id, &["resources"], PRIV_RESOURCE_AUDIT, false)
> .is_ok();
^ view_filter being Some could cause the above check_privs to be skipped
and just assume `allow_all=false`. Even if the code bellow re-checks
`view_filter.is_some()` before using `allow_all`, we can still skip over
the priv check.
>
> + if let Some(view_filter) = &view_filter {
> + user_info.check_privs(&auth_id, &["view", view_filter], PRIV_RESOURCE_AUDIT, false)?;
> + }
> +
> + let view_filter = view_filter
> + .map(|filter_name| views::view_filter::get_view_filter(&filter_name))
> + .transpose()?;
> +
> let check_priv = |remote_name: &str| -> bool {
> user_info
> .check_privs(
> @@ -584,35 +597,62 @@ pub async fn get_subscription_status(
> };
>
> for (remote_name, remote) in remotes_config {
> - if !allow_all && !check_priv(&remote_name) {
> + if let Some(filter) = &view_filter {
> + if filter.can_skip_remote(&remote_name) {
> + continue;
> + }
> + } else if !allow_all && !check_priv(&remote_name) {
> continue;
> }
>
> + let view_filter_clone = view_filter.clone();
> +
> let future = async move {
> let (node_status, error) =
> match get_subscription_info_for_remote(&remote, max_age).await {
> - Ok(node_status) => (Some(node_status), None),
> + Ok(mut node_status) => {
> + node_status.retain(|node, _| {
> + if let Some(filter) = &view_filter_clone {
> + filter.is_node_included(&remote.id, node)
> + } else {
> + true
> + }
> + });
> + (Some(node_status), None)
> + }
> Err(error) => (None, Some(error.to_string())),
> };
>
> - let mut state = RemoteSubscriptionState::Unknown;
> + let state = if let Some(node_status) = &node_status {
> + if error.is_some() {
> + // Don't leak the existence of failed remotes, since we cannot apply
> + // view-filters here.
> + return None;
> + }
>
> - if let Some(node_status) = &node_status {
> - state = map_node_subscription_list_to_state(node_status);
> - }
> + if node_status.is_empty() {
> + return None;
> + }
>
> - RemoteSubscriptions {
> + map_node_subscription_list_to_state(node_status)
> + } else {
> + RemoteSubscriptionState::Unknown
> + };
> +
> + Some(RemoteSubscriptions {
> remote: remote_name,
> error,
> state,
> node_status: if verbose { node_status } else { None },
> - }
> + })
> };
>
> futures.push(future);
> }
>
> - Ok(join_all(futures).await)
> + let status = join_all(futures).await.into_iter().flatten().collect();
> +
> + Ok(status)
> }
>
> // FIXME: make timeframe and count parameters?
> --
> 2.47.3
>
>
>
> _______________________________________________
> pdm-devel mailing list
> pdm-devel@lists.proxmox.com
> https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
>
>
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
` (12 preceding siblings ...)
2025-10-29 14:49 ` [pdm-devel] [PATCH datacenter-manager 13/13] pdm-client: top entities: " Lukas Wagner
@ 2025-10-30 11:41 ` Wolfgang Bumiller
13 siblings, 0 replies; 21+ messages in thread
From: Wolfgang Bumiller @ 2025-10-30 11:41 UTC (permalink / raw)
To: Lukas Wagner; +Cc: pdm-devel
Code-wise LGTM - nits could be addressed later.
Whether this is the way we want it needs to be decided I guess. Not sure
splitting by include/exclude this way is ideal, but I don't have better
ideas for the moment, although it may be interesting to add the ability
to use globs there at some point, then having both includes and excludes
makes *some* sense (but wouldn't allow for something like, "include all
'x*' except 'xy*' but do still include 'xyz*'" – that would need
orders/priorities.
On Wed, Oct 29, 2025 at 03:48:49PM +0100, Lukas Wagner wrote:
> Key aspects:
> - new config file at /etc/proxmox-datacenter-manager/views/filters.cfg
> (subject to change, depending on where the view templates are stored)
> - ViewFilterConfig as a filter defintion type,has
> - {include,exclude}-{remote,resource-id,resource-type,resource-pool,tag}
>
> - Filter implementation & big suite of unit tests
> - excludes are processed after includes
> - if no include rules are defined, all resources but those which were
> excluded are matched
> - if no rules are defined in a filter, everything is matched
>
> - Added the 'view-filter' parameter to a couple of API endpoints
> - /resources/list
> - /resources/status
> - /resources/subscription
> - /resources/top-entities
> - /remote-tasks/list
> - /remote-tasks/statistis
>
> - ACL checks are done on /view/{view-filter-id} for now, replace
> any other permission check in the handler iff the view-filter paramter
> is set
>
> Left to do:
> - CRUD for filter definition
> - Decide on how exactly they will work together with the view templates
> -> most likely a new config type
>
> View {
> filter: String,
> template: String,
> }
> - UI for filter rules
>
>
> proxmox-datacenter-manager:
>
> Lukas Wagner (13):
> fake remote: add missing parameter for cluster_metrics_export function
> pdm-api-types: views: add ViewFilterConfig type
> pdm-config: views: add support for view-filters
> acl: add '/view' and '/view/{view-id}' as allowed ACL paths
> views: add implementation for view filters
> views: add tests for view filter implementation
> api: resources: list: add support for view-filter parameter
> api: resources: top entities: add support for view-filter parameter
> api: resources: status: add support for view-filter parameter
> api: subscription status: add support for view-filter parameter
> api: remote-tasks: add support for view-filter parameter
> pdm-client: resource list: add view-filter parameter
> pdm-client: top entities: add view-filter parameter
>
> cli/client/src/resources.rs | 2 +-
> lib/pdm-api-types/src/lib.rs | 8 +
> lib/pdm-api-types/src/views.rs | 147 ++++++
> lib/pdm-client/src/lib.rs | 19 +-
> lib/pdm-config/src/lib.rs | 2 +-
> lib/pdm-config/src/views.rs | 62 +++
> server/src/acl.rs | 6 +
> server/src/api/remote_tasks.rs | 36 +-
> server/src/api/resources.rs | 152 +++++-
> server/src/lib.rs | 1 +
> server/src/metric_collection/top_entities.rs | 5 +
> server/src/remote_tasks/mod.rs | 39 +-
> server/src/resource_cache.rs | 3 +-
> server/src/test_support/fake_remote.rs | 1 +
> server/src/views/mod.rs | 4 +
> server/src/views/tests.rs | 486 +++++++++++++++++++
> server/src/views/view_filter.rs | 225 +++++++++
> ui/src/dashboard/mod.rs | 2 +-
> ui/src/sdn/zone_tree.rs | 2 +-
> 19 files changed, 1157 insertions(+), 45 deletions(-)
> create mode 100644 lib/pdm-api-types/src/views.rs
> create mode 100644 lib/pdm-config/src/views.rs
> create mode 100644 server/src/views/mod.rs
> create mode 100644 server/src/views/tests.rs
> create mode 100644 server/src/views/view_filter.rs
>
>
> Summary over all repositories:
> 19 files changed, 1157 insertions(+), 45 deletions(-)
>
> --
> Generated by murpp 0.9.0
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [pdm-devel] [PATCH datacenter-manager 10/13] api: subscription status: add support for view-filter parameter
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 10/13] api: subscription " Lukas Wagner
2025-10-30 11:31 ` Wolfgang Bumiller
@ 2025-10-30 11:44 ` Shannon Sterz
1 sibling, 0 replies; 21+ messages in thread
From: Shannon Sterz @ 2025-10-30 11:44 UTC (permalink / raw)
To: Lukas Wagner; +Cc: Proxmox Datacenter Manager development discussion
On Wed Oct 29, 2025 at 3:48 PM CET, Lukas Wagner wrote:
> A view filter allows one to get filtered subset of all resources, based
> on filter rules defined in a config file. View filters integrate with
> the permission system - if a user has permissions on
> /view/{view-filter-id}, then these privileges are transitively applied
> to all resources which are matched by the rules. All other permission
> checks are replaced if requesting data through a view filter.
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> server/src/api/resources.rs | 58 +++++++++++++++++++++++++++++++------
> 1 file changed, 49 insertions(+), 9 deletions(-)
>
> diff --git a/server/src/api/resources.rs b/server/src/api/resources.rs
> index 61960295..89b84c3c 100644
> --- a/server/src/api/resources.rs
> +++ b/server/src/api/resources.rs
> @@ -546,6 +546,10 @@ pub async fn get_status(
> default: false,
> description: "If true, includes subscription information per node (with enough privileges)",
> },
> + "view-filter": {
> + schema: VIEW_FILTER_ID_SCHEMA,
> + optional: true,
> + },
> },
> },
> returns: {
> @@ -560,6 +564,7 @@ pub async fn get_status(
> pub async fn get_subscription_status(
> max_age: u64,
> verbose: bool,
> + view_filter: Option<String>,
> rpcenv: &mut dyn RpcEnvironment,
> ) -> Result<Vec<RemoteSubscriptions>, Error> {
> let (remotes_config, _) = pdm_config::remotes::config()?;
> @@ -572,6 +577,14 @@ pub async fn get_subscription_status(
> .check_privs(&auth_id, &["resources"], PRIV_RESOURCE_AUDIT, false)
> .is_ok();
>
> + if let Some(view_filter) = &view_filter {
> + user_info.check_privs(&auth_id, &["view", view_filter], PRIV_RESOURCE_AUDIT, false)?;
> + }
this is minor, but maybe consider moving the view check before the
`allow_all` check and skipping the `allow_all` check (by setting it to
false) if a view was found. that should safe us an unecessary traversal
of the acl tree here.
i think wolfgang already noted that tho.
> +
> + let view_filter = view_filter
> + .map(|filter_name| views::view_filter::get_view_filter(&filter_name))
> + .transpose()?;
> +
> let check_priv = |remote_name: &str| -> bool {
> user_info
> .check_privs(
> @@ -584,35 +597,62 @@ pub async fn get_subscription_status(
> };
>
> for (remote_name, remote) in remotes_config {
> - if !allow_all && !check_priv(&remote_name) {
> + if let Some(filter) = &view_filter {
> + if filter.can_skip_remote(&remote_name) {
> + continue;
> + }
> + } else if !allow_all && !check_priv(&remote_name) {
> continue;
> }
>
> + let view_filter_clone = view_filter.clone();
> +
> let future = async move {
> let (node_status, error) =
> match get_subscription_info_for_remote(&remote, max_age).await {
> - Ok(node_status) => (Some(node_status), None),
> + Ok(mut node_status) => {
> + node_status.retain(|node, _| {
> + if let Some(filter) = &view_filter_clone {
> + filter.is_node_included(&remote.id, node)
> + } else {
> + true
> + }
> + });
> + (Some(node_status), None)
> + }
> Err(error) => (None, Some(error.to_string())),
> };
>
> - let mut state = RemoteSubscriptionState::Unknown;
> + let state = if let Some(node_status) = &node_status {
> + if error.is_some() {
> + // Don't leak the existence of failed remotes, since we cannot apply
> + // view-filters here.
> + return None;
shouldn't this be gated by checking if view_filters was defined?
otherwise we now return nothing every time we get an error?
also correct me if i'm wrong, but isn't this `return None;` unreachable?
error is assigned `Some` iff the `get_subscription_status` call above
returns an `Err`, but in that case `node_status` is set to `None`, so
the `if let Some(node_status) = &node_status` above should be false?
> + }
>
> - if let Some(node_status) = &node_status {
> - state = map_node_subscription_list_to_state(node_status);
> - }
> + if node_status.is_empty() {
> + return None;
> + }
>
> - RemoteSubscriptions {
> + map_node_subscription_list_to_state(node_status)
> + } else {
> + RemoteSubscriptionState::Unknown
> + };
> +
> + Some(RemoteSubscriptions {
> remote: remote_name,
> error,
> state,
> node_status: if verbose { node_status } else { None },
> - }
> + })
> };
>
> futures.push(future);
> }
>
> - Ok(join_all(futures).await)
> + let status = join_all(futures).await.into_iter().flatten().collect();
> +
> + Ok(status)
> }
>
> // FIXME: make timeframe and count parameters?
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [pdm-devel] [PATCH datacenter-manager 05/13] views: add implementation for view filters
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 05/13] views: add implementation for view filters Lukas Wagner
@ 2025-10-30 11:44 ` Shannon Sterz
2025-10-30 13:30 ` Lukas Wagner
0 siblings, 1 reply; 21+ messages in thread
From: Shannon Sterz @ 2025-10-30 11:44 UTC (permalink / raw)
To: Lukas Wagner; +Cc: Proxmox Datacenter Manager development discussion
On Wed Oct 29, 2025 at 3:48 PM CET, Lukas Wagner wrote:
> This commit adds the filter implementation for the previously defined
> ViewFilterConfig type.
>
> There are include/exclude rules for the following properties:
> - (global) resource-id
> - resource pool
> - resource type
> - remote
> - tags
>
> The rules are interpreted as follows:
> - no rules: everything matches
> - only includes: included resources match
> - only excluded: everything *but* the excluded resources match
> - include and exclude: excludes are applied *after* includes, meaning if
> one has a `include-remote foo` and `exclude-remote foo` at the same
> time, the remote `foo` will never match
>
> Signed-off-by: Lukas Wagner <l.wagner@proxmox.com>
> ---
> server/src/lib.rs | 1 +
> server/src/views/mod.rs | 1 +
> server/src/views/view_filter.rs | 225 ++++++++++++++++++++++++++++++++
> 3 files changed, 227 insertions(+)
> create mode 100644 server/src/views/mod.rs
> create mode 100644 server/src/views/view_filter.rs
>
> diff --git a/server/src/lib.rs b/server/src/lib.rs
> index 964807eb..0f25aa71 100644
> --- a/server/src/lib.rs
> +++ b/server/src/lib.rs
> @@ -12,6 +12,7 @@ pub mod remote_tasks;
> pub mod remote_updates;
> pub mod resource_cache;
> pub mod task_utils;
> +pub mod views;
>
> pub mod connection;
> pub mod pbs_client;
> diff --git a/server/src/views/mod.rs b/server/src/views/mod.rs
> new file mode 100644
> index 00000000..9a2856a4
> --- /dev/null
> +++ b/server/src/views/mod.rs
> @@ -0,0 +1 @@
> +pub mod view_filter;
> diff --git a/server/src/views/view_filter.rs b/server/src/views/view_filter.rs
> new file mode 100644
> index 00000000..656b5523
> --- /dev/null
> +++ b/server/src/views/view_filter.rs
> @@ -0,0 +1,225 @@
> +use anyhow::Error;
> +
> +use pdm_api_types::{
> + resource::{Resource, ResourceType},
> + views::ViewFilterConfig,
> +};
> +
> +/// Get view filter with a given ID.
> +///
> +/// Returns an error if the view filter configuration file could not be read, or
> +/// if the view filter with the provided ID does not exist.
> +pub fn get_view_filter(filter_id: &str) -> Result<ViewFilter, Error> {
> + pdm_config::views::get_view_filter_config(filter_id).map(ViewFilter::new)
> +}
> +
> +/// View filter implementation.
> +///
> +/// Given a [`ViewFilterConfig`], this struct can be used to check if a resource/remote/node
> +/// matches the filter rules.
> +#[derive(Clone)]
> +pub struct ViewFilter {
> + config: ViewFilterConfig,
> +}
> +
> +impl ViewFilter {
> + /// Create a new [`ViewFiler`].
> + pub fn new(config: ViewFilterConfig) -> Self {
> + Self { config }
> + }
> +
> + /// Check if a [`Resource`] matches the filter rules.
> + pub fn resource_matches(&self, remote: &str, resource: &Resource) -> bool {
> + // NOTE: Establishing a cache here is not worth the effort at the moment, evaluation of
> + // rules is *very* fast.
> + //
> + // Some experiments were performed with a cache that works roughly as following:
> + // - HashMap<ViewId, HashMap<ResourceId, bool>> in a mutex
> + // - Cache invalidated if view-filter config digest changed
> + // - Cache invalidated if certain resource fields such as tags or resource pools change
> + // from the last time (also with a digest-based implementation)
> + //
> + // Experimented with the `fake-remote` feature and and 15000 guests showed that
> + // caching was only faster than direct evaluation if the number of rules in the
> + // ViewFilterConfig is *huge* (e.g. >1000 `include-resource-id` entries). But even for those,
> + // direct evaluation was always plenty fast, with evaluation times ~20ms for *all* resources.
> + //
> + // -> for any *realistic* filter config, we should be good with direct evaluation, as long
> + // as we don't add any filter rules which are very expensive to evaluate.
> +
> + let resource_data = resource.into();
> +
> + self.check_if_included(remote, &resource_data)
> + && !self.check_if_excluded(remote, &resource_data)
> + }
> +
> + /// Check if a remote can be safely skipped based on the filter rule definition.
> + ///
> + /// When there are `include-remote` or `exclude-remote` rules, we can use these to
> + /// check if a remote needs to be considered at all.
> + pub fn can_skip_remote(&self, remote: &str) -> bool {
> + let no_includes = self.config.include_remote.is_empty();
> + let any_include = self.config.include_remote.iter().any(|r| r == remote);
> + let any_exclude = self.config.exclude_remote.iter().any(|r| r == remote);
> +
> + (!no_includes && !any_include) || any_exclude
> + }
> +
> + /// Check if a node is matched by the filter rules.
> + ///
> + /// This is equivalent to checking an actual node resource.
> + pub fn is_node_included(&self, remote: &str, node: &str) -> bool {
> + let resource_data = ResourceData {
> + resource_type: ResourceType::Node,
> + tags: None,
> + resource_pool: None,
> + resource_id: &format!("remote/{remote}/node/{node}"),
> + };
> +
> + self.check_if_included(remote, &resource_data)
> + && !self.check_if_excluded(remote, &resource_data)
> + }
> +
> + /// Returns the name of the view filter.
> + pub fn name(&self) -> &str {
> + &self.config.id
> + }
> +
> + fn check_if_included(&self, remote: &str, resource: &ResourceData) -> bool {
> + let rules = Rules {
> + ruleset_type: RulesetType::Include,
> + tags: &self.config.include_tag,
> + resource_ids: &self.config.include_resource_id,
> + resource_type: &self.config.include_resource_type,
> + resource_pools: &self.config.include_resource_pool,
> + remotes: &self.config.include_remote,
> + };
> +
> + check_rules(rules, remote, resource)
> + }
> +
> + fn check_if_excluded(&self, remote: &str, resource: &ResourceData) -> bool {
> + let rules = Rules {
> + ruleset_type: RulesetType::Exclude,
> + tags: &self.config.exclude_tag,
> + resource_ids: &self.config.exclude_resource_id,
> + resource_type: &self.config.exclude_resource_type,
> + resource_pools: &self.config.exclude_resource_pool,
> + remotes: &self.config.exclude_remote,
> + };
> +
> + check_rules(rules, remote, resource)
> + }
> +}
> +
> +enum RulesetType {
> + Include,
> + Exclude,
> +}
> +
> +struct Rules<'a> {
> + ruleset_type: RulesetType,
> + tags: &'a [String],
> + resource_ids: &'a [String],
> + resource_pools: &'a [String],
> + resource_type: &'a [ResourceType],
> + remotes: &'a [String],
> +}
> +
> +struct ResourceData<'a> {
> + resource_type: ResourceType,
> + tags: Option<&'a [String]>,
> + resource_pool: Option<&'a String>,
> + resource_id: &'a str,
> +}
> +
> +impl<'a> From<&'a Resource> for ResourceData<'a> {
> + fn from(value: &'a Resource) -> Self {
> + match value {
> + Resource::PveStorage(_) => ResourceData {
> + resource_type: value.resource_type(),
> + tags: None,
> + resource_pool: None,
> + resource_id: value.global_id(),
> + },
> + Resource::PveQemu(resource) => ResourceData {
> + resource_type: value.resource_type(),
> + tags: Some(&resource.tags),
> + resource_pool: Some(&resource.pool),
> + resource_id: value.global_id(),
> + },
> + Resource::PveLxc(resource) => ResourceData {
> + resource_type: value.resource_type(),
> + tags: Some(&resource.tags),
> + resource_pool: Some(&resource.pool),
> + resource_id: value.global_id(),
> + },
> + Resource::PveNode(_) => ResourceData {
> + resource_type: value.resource_type(),
> + tags: None,
> + resource_pool: None,
> + resource_id: value.global_id(),
> + },
> + Resource::PveSdn(_) => ResourceData {
> + resource_type: value.resource_type(),
> + tags: None,
> + resource_pool: None,
> + resource_id: value.global_id(),
> + },
> + Resource::PbsNode(_) => ResourceData {
> + resource_type: value.resource_type(),
> + tags: None,
> + resource_pool: None,
> + resource_id: value.global_id(),
> + },
> + Resource::PbsDatastore(_) => ResourceData {
> + resource_type: value.resource_type(),
> + tags: None,
> + resource_pool: None,
> + resource_id: value.global_id(),
> + },
> + }
> + }
> +}
nit: imo this would be fine to group this a match statement a bit. i.e.
PveStorage, PveNode, PveSdn, PbsNode and PbsDatastore have essentially
the same arm here. could be simply or-ed (|)
> +
> +fn check_rules(rules: Rules, remote: &str, resource: &ResourceData) -> bool {
> + let has_any_rules = !rules.tags.is_empty()
> + || !rules.remotes.is_empty()
> + || !rules.resource_pools.is_empty()
> + || !rules.resource_type.is_empty()
> + || !rules.resource_ids.is_empty();
> +
> + if !has_any_rules {
> + return matches!(rules.ruleset_type, RulesetType::Include);
> + }
> +
> + if let Some(tags) = resource.tags {
> + if rules.tags.iter().any(|tag| tags.contains(tag)) {
> + return true;
> + }
> + }
> +
> + if let Some(pool) = resource.resource_pool {
> + if rules.resource_pools.iter().any(|p| p == pool) {
> + return true;
> + }
> + }
> +
> + if rules.remotes.iter().any(|r| r == remote) {
> + return true;
> + }
> +
> + if rules.resource_ids.iter().any(|r| r == resource.resource_id) {
> + return true;
> + }
> +
> + if rules
> + .resource_type
> + .iter()
> + .any(|ty| *ty == resource.resource_type)
> + {
> + return true;
> + }
> +
> + false
> +}
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
* Re: [pdm-devel] [PATCH datacenter-manager 05/13] views: add implementation for view filters
2025-10-30 11:44 ` Shannon Sterz
@ 2025-10-30 13:30 ` Lukas Wagner
0 siblings, 0 replies; 21+ messages in thread
From: Lukas Wagner @ 2025-10-30 13:30 UTC (permalink / raw)
To: Shannon Sterz, Lukas Wagner
Cc: Proxmox Datacenter Manager development discussion
On Thu Oct 30, 2025 at 12:44 PM CET, Shannon Sterz wrote:
>> +impl<'a> From<&'a Resource> for ResourceData<'a> {
>> + fn from(value: &'a Resource) -> Self {
>> + match value {
>> + Resource::PveStorage(_) => ResourceData {
>> + resource_type: value.resource_type(),
>> + tags: None,
>> + resource_pool: None,
>> + resource_id: value.global_id(),
>> + },
>> + Resource::PveQemu(resource) => ResourceData {
>> + resource_type: value.resource_type(),
>> + tags: Some(&resource.tags),
>> + resource_pool: Some(&resource.pool),
>> + resource_id: value.global_id(),
>> + },
>> + Resource::PveLxc(resource) => ResourceData {
>> + resource_type: value.resource_type(),
>> + tags: Some(&resource.tags),
>> + resource_pool: Some(&resource.pool),
>> + resource_id: value.global_id(),
>> + },
>> + Resource::PveNode(_) => ResourceData {
>> + resource_type: value.resource_type(),
>> + tags: None,
>> + resource_pool: None,
>> + resource_id: value.global_id(),
>> + },
>> + Resource::PveSdn(_) => ResourceData {
>> + resource_type: value.resource_type(),
>> + tags: None,
>> + resource_pool: None,
>> + resource_id: value.global_id(),
>> + },
>> + Resource::PbsNode(_) => ResourceData {
>> + resource_type: value.resource_type(),
>> + tags: None,
>> + resource_pool: None,
>> + resource_id: value.global_id(),
>> + },
>> + Resource::PbsDatastore(_) => ResourceData {
>> + resource_type: value.resource_type(),
>> + tags: None,
>> + resource_pool: None,
>> + resource_id: value.global_id(),
>> + },
>> + }
>> + }
>> +}
>
> nit: imo this would be fine to group this a match statement a bit. i.e.
> PveStorage, PveNode, PveSdn, PbsNode and PbsDatastore have essentially
> the same arm here. could be simply or-ed (|)
>
Ah, yes, of course! Will be fixed in the next version.
Thanks!
_______________________________________________
pdm-devel mailing list
pdm-devel@lists.proxmox.com
https://lists.proxmox.com/cgi-bin/mailman/listinfo/pdm-devel
^ permalink raw reply [flat|nested] 21+ messages in thread
end of thread, other threads:[~2025-10-30 13:30 UTC | newest]
Thread overview: 21+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2025-10-29 14:48 [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 01/13] fake remote: add missing parameter for cluster_metrics_export function Lukas Wagner
2025-10-30 10:26 ` [pdm-devel] applied: " Wolfgang Bumiller
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 02/13] pdm-api-types: views: add ViewFilterConfig type Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 03/13] pdm-config: views: add support for view-filters Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 04/13] acl: add '/view' and '/view/{view-id}' as allowed ACL paths Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 05/13] views: add implementation for view filters Lukas Wagner
2025-10-30 11:44 ` Shannon Sterz
2025-10-30 13:30 ` Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 06/13] views: add tests for view filter implementation Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 07/13] api: resources: list: add support for view-filter parameter Lukas Wagner
2025-10-30 11:21 ` Wolfgang Bumiller
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 08/13] api: resources: top entities: " Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 09/13] api: resources: status: " Lukas Wagner
2025-10-29 14:48 ` [pdm-devel] [PATCH datacenter-manager 10/13] api: subscription " Lukas Wagner
2025-10-30 11:31 ` Wolfgang Bumiller
2025-10-30 11:44 ` Shannon Sterz
2025-10-29 14:49 ` [pdm-devel] [PATCH datacenter-manager 11/13] api: remote-tasks: " Lukas Wagner
2025-10-29 14:49 ` [pdm-devel] [PATCH datacenter-manager 12/13] pdm-client: resource list: add " Lukas Wagner
2025-10-29 14:49 ` [pdm-devel] [PATCH datacenter-manager 13/13] pdm-client: top entities: " Lukas Wagner
2025-10-30 11:41 ` [pdm-devel] [RFC datacenter-manager 00/13] backend implementation for view filters Wolfgang Bumiller
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.